2024-03-22T06:55:05.445 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-22T06:55:05.446 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi --verbose --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616023 --description upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616023/orig.config.yaml 2024-03-22T06:55:05.499 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-22T06:55:05.589 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616023 branch: squid description: upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7616023' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 768f6cd6e95d0c7a626042719c230df59bdba9d3 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 3446 sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 sleep_before_teardown: 0 subset: 21/32 suite: upgrade suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_768f6cd6e95d0c7a626042719c230df59bdba9d3/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 768f6cd6e95d0c7a626042719c230df59bdba9d3 targets: smithi063.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJKvUlsItojDO8g0NjfNHRQp3hNkPS3/FsIWFM+n2IsC/x1EMD0HGNq5cX3rLLjuXZTxGT44MCHQYhp0MmxH8C0= smithi069.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPT39+ckwNB4xQceDnGnllN7zYrxcb899ohP4de9TrFdfnzEWoZgXMGBFoQiQzPFK+gnYNsR9dWJnDlaNmYXotM= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-22_02:08:13 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-03-22T06:55:05.589 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_768f6cd6e95d0c7a626042719c230df59bdba9d3/qa; will attempt to use it 2024-03-22T06:55:05.590 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_768f6cd6e95d0c7a626042719c230df59bdba9d3/qa/tasks 2024-03-22T06:55:05.590 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-22T06:55:05.592 INFO:teuthology.task.internal:Checking packages... 2024-03-22T06:55:05.616 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'db0330b1e4e2470d52b750e251e55a522b4f7d69' 2024-03-22T06:55:05.616 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T06:55:05.616 INFO:teuthology.packaging:ref: None 2024-03-22T06:55:05.616 INFO:teuthology.packaging:tag: None 2024-03-22T06:55:05.616 INFO:teuthology.packaging:branch: squid 2024-03-22T06:55:05.616 INFO:teuthology.packaging:sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T06:55:05.616 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-03-22T06:55:05.773 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1667.gdb0330b1 2024-03-22T06:55:05.775 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-22T06:55:05.781 INFO:teuthology.task.internal:no buildpackages task found 2024-03-22T06:55:05.781 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-22T06:55:05.814 INFO:teuthology.task.internal:Saving configuration 2024-03-22T06:55:05.833 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-22T06:55:05.859 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-22T06:55:05.883 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi063.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616023', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 06:50:14.029751', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJKvUlsItojDO8g0NjfNHRQp3hNkPS3/FsIWFM+n2IsC/x1EMD0HGNq5cX3rLLjuXZTxGT44MCHQYhp0MmxH8C0='} 2024-03-22T06:55:05.905 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616023', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 06:50:14.031536', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPT39+ckwNB4xQceDnGnllN7zYrxcb899ohP4de9TrFdfnzEWoZgXMGBFoQiQzPFK+gnYNsR9dWJnDlaNmYXotM='} 2024-03-22T06:55:05.905 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-22T06:55:05.911 INFO:teuthology.task.internal:roles: ubuntu@smithi063.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-03-22T06:55:05.911 INFO:teuthology.task.internal:roles: ubuntu@smithi069.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-03-22T06:55:05.911 INFO:teuthology.run_tasks:Running task console_log... 2024-03-22T06:55:05.979 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4ca2c130d0>, signals=[15]) 2024-03-22T06:55:05.979 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-22T06:55:05.985 INFO:teuthology.task.internal:Opening connections... 2024-03-22T06:55:05.985 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi063.front.sepia.ceph.com 2024-03-22T06:55:05.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:55:06.094 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi069.front.sepia.ceph.com 2024-03-22T06:55:06.095 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:55:06.183 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-22T06:55:06.189 DEBUG:teuthology.orchestra.run.smithi063:> uname -m 2024-03-22T06:55:06.207 INFO:teuthology.orchestra.run.smithi063.stdout:x86_64 2024-03-22T06:55:06.207 DEBUG:teuthology.orchestra.run.smithi063:> cat /etc/os-release 2024-03-22T06:55:06.264 INFO:teuthology.orchestra.run.smithi063.stdout:NAME="CentOS Stream" 2024-03-22T06:55:06.264 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION="9" 2024-03-22T06:55:06.264 INFO:teuthology.orchestra.run.smithi063.stdout:ID="centos" 2024-03-22T06:55:06.264 INFO:teuthology.orchestra.run.smithi063.stdout:ID_LIKE="rhel fedora" 2024-03-22T06:55:06.265 INFO:teuthology.orchestra.run.smithi063.stdout:VERSION_ID="9" 2024-03-22T06:55:06.265 INFO:teuthology.orchestra.run.smithi063.stdout:PLATFORM_ID="platform:el9" 2024-03-22T06:55:06.265 INFO:teuthology.orchestra.run.smithi063.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-22T06:55:06.265 INFO:teuthology.orchestra.run.smithi063.stdout:ANSI_COLOR="0;31" 2024-03-22T06:55:06.265 INFO:teuthology.orchestra.run.smithi063.stdout:LOGO="fedora-logo-icon" 2024-03-22T06:55:06.265 INFO:teuthology.orchestra.run.smithi063.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-22T06:55:06.265 INFO:teuthology.orchestra.run.smithi063.stdout:HOME_URL="https://centos.org/" 2024-03-22T06:55:06.265 INFO:teuthology.orchestra.run.smithi063.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-22T06:55:06.265 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-22T06:55:06.265 INFO:teuthology.orchestra.run.smithi063.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-22T06:55:06.266 INFO:teuthology.lock.ops:Updating smithi063.front.sepia.ceph.com on lock server 2024-03-22T06:55:06.289 DEBUG:teuthology.orchestra.run.smithi069:> uname -m 2024-03-22T06:55:06.306 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64 2024-03-22T06:55:06.307 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2024-03-22T06:55:06.362 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Stream" 2024-03-22T06:55:06.362 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="9" 2024-03-22T06:55:06.362 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos" 2024-03-22T06:55:06.362 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora" 2024-03-22T06:55:06.362 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="9" 2024-03-22T06:55:06.362 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el9" 2024-03-22T06:55:06.362 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-22T06:55:06.362 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2024-03-22T06:55:06.362 INFO:teuthology.orchestra.run.smithi069.stdout:LOGO="fedora-logo-icon" 2024-03-22T06:55:06.362 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-22T06:55:06.362 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://centos.org/" 2024-03-22T06:55:06.362 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-22T06:55:06.363 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-22T06:55:06.363 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-22T06:55:06.363 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2024-03-22T06:55:06.388 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-22T06:55:06.396 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-22T06:55:06.401 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-22T06:55:06.402 DEBUG:teuthology.orchestra.run.smithi063:> test '!' -e /home/ubuntu/cephtest 2024-03-22T06:55:06.404 DEBUG:teuthology.orchestra.run.smithi069:> test '!' -e /home/ubuntu/cephtest 2024-03-22T06:55:06.420 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-22T06:55:06.425 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-22T06:55:06.425 DEBUG:teuthology.orchestra.run.smithi063:> test -z $(ls -A /var/lib/ceph) 2024-03-22T06:55:06.460 DEBUG:teuthology.orchestra.run.smithi069:> test -z $(ls -A /var/lib/ceph) 2024-03-22T06:55:06.490 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-22T06:55:06.545 INFO:teuthology.run_tasks:Running task kernel... 2024-03-22T06:55:06.558 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-22T06:55:06.558 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-22T06:55:06.559 DEBUG:teuthology.orchestra.run.smithi063:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-22T06:55:06.559 DEBUG:teuthology.orchestra.run.smithi069:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-22T06:55:06.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T06:55:06.576 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-03-22T06:55:06.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T06:55:06.577 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2024-03-22T06:55:06.632 INFO:teuthology.orchestra.run.smithi069.stdout:5.14.0-361.el9.x86_64 2024-03-22T06:55:06.632 INFO:teuthology.task.kernel:Running kernel on smithi069: 5.14.0-361.el9.x86_64 2024-03-22T06:55:06.632 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum install -y kernel 2024-03-22T06:55:06.634 INFO:teuthology.orchestra.run.smithi063.stdout:5.14.0-361.el9.x86_64 2024-03-22T06:55:06.635 INFO:teuthology.task.kernel:Running kernel on smithi063: 5.14.0-361.el9.x86_64 2024-03-22T06:55:06.635 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum install -y kernel 2024-03-22T06:55:08.712 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-22T06:55:08.739 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-22T06:55:10.147 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS-9 - BaseOS 7.8 MB/s | 2.0 MB 00:00 2024-03-22T06:55:10.354 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-03-22T06:55:11.196 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-03-22T06:55:11.378 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-22T06:55:14.122 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-03-22T06:55:22.131 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - CRB 1.4 MB/s | 6.1 MB 00:04 2024-03-22T06:55:24.848 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - Extras packages 10 kB/s | 16 kB 00:01 2024-03-22T06:55:25.675 INFO:teuthology.orchestra.run.smithi063.stdout:Extra Packages for Enterprise Linux 29 MB/s | 21 MB 00:00 2024-03-22T06:55:30.825 INFO:teuthology.orchestra.run.smithi063.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-03-22T06:55:32.965 INFO:teuthology.orchestra.run.smithi063.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-22T06:55:32.965 INFO:teuthology.orchestra.run.smithi063.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-22T06:55:33.062 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T06:55:33.071 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T06:55:33.071 INFO:teuthology.orchestra.run.smithi063.stdout: Package Architecture Version Repository Size 2024-03-22T06:55:33.071 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T06:55:33.072 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-03-22T06:55:33.072 INFO:teuthology.orchestra.run.smithi063.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-22T06:55:33.072 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-03-22T06:55:33.072 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-22T06:55:33.072 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-22T06:55:33.072 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-22T06:55:33.072 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T06:55:33.072 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-03-22T06:55:33.072 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T06:55:33.072 INFO:teuthology.orchestra.run.smithi063.stdout:Install 4 Packages 2024-03-22T06:55:33.072 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T06:55:33.073 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 76 M 2024-03-22T06:55:33.073 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 123 M 2024-03-22T06:55:33.073 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-03-22T06:55:33.751 INFO:teuthology.orchestra.run.smithi063.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 70 kB/s | 44 kB 00:00 2024-03-22T06:55:34.143 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 887 kB/s | 19 MB 00:21 2024-03-22T06:55:38.679 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 7.4 MB/s | 6.1 MB 00:00 2024-03-22T06:55:40.269 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-03-22T06:55:41.723 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-22T06:55:46.679 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-22T06:55:48.777 INFO:teuthology.orchestra.run.smithi069.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-22T06:55:48.777 INFO:teuthology.orchestra.run.smithi069.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-22T06:55:48.873 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T06:55:48.884 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T06:55:48.884 INFO:teuthology.orchestra.run.smithi069.stdout: Package Architecture Version Repository Size 2024-03-22T06:55:48.884 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T06:55:48.884 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-03-22T06:55:48.884 INFO:teuthology.orchestra.run.smithi069.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-22T06:55:48.884 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-03-22T06:55:48.884 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-22T06:55:48.884 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-22T06:55:48.884 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-22T06:55:48.884 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T06:55:48.884 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-03-22T06:55:48.884 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T06:55:48.884 INFO:teuthology.orchestra.run.smithi069.stdout:Install 4 Packages 2024-03-22T06:55:48.885 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T06:55:48.885 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 76 M 2024-03-22T06:55:48.885 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 123 M 2024-03-22T06:55:48.885 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-03-22T06:55:49.030 INFO:teuthology.orchestra.run.smithi069.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 571 kB/s | 44 kB 00:00 2024-03-22T06:55:49.589 INFO:teuthology.orchestra.run.smithi069.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 23 MB/s | 15 MB 00:00 2024-03-22T06:55:50.156 INFO:teuthology.orchestra.run.smithi069.stdout:(3/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 28 MB/s | 33 MB 00:01 2024-03-22T06:55:50.428 INFO:teuthology.orchestra.run.smithi069.stdout:(4/4): kernel-modules-core-5.14.0-430.el9.x86_6 20 MB/s | 27 MB 00:01 2024-03-22T06:55:50.428 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:55:50.428 INFO:teuthology.orchestra.run.smithi069.stdout:Total 49 MB/s | 76 MB 00:01 2024-03-22T06:55:51.095 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-03-22T06:55:51.454 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-03-22T06:55:51.454 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-03-22T06:55:52.441 INFO:teuthology.orchestra.run.smithi063.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 790 kB/s | 15 MB 00:19 2024-03-22T06:55:52.445 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-03-22T06:55:52.446 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-03-22T06:55:54.910 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-03-22T06:55:55.513 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-22T06:55:55.597 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-22T06:55:58.338 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-22T06:55:58.473 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-22T06:56:03.202 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-22T06:56:03.323 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:56:08.194 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:56:09.415 INFO:teuthology.orchestra.run.smithi063.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 789 kB/s | 27 MB 00:35 2024-03-22T06:56:15.456 INFO:teuthology.orchestra.run.smithi063.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 809 kB/s | 33 MB 00:42 2024-03-22T06:56:15.456 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-03-22T06:56:15.456 INFO:teuthology.orchestra.run.smithi063.stdout:Total 1.8 MB/s | 76 MB 00:42 2024-03-22T06:56:16.144 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-03-22T06:56:16.564 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-03-22T06:56:16.564 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-03-22T06:56:17.610 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-03-22T06:56:17.611 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-03-22T06:56:20.243 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-03-22T06:56:20.869 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-22T06:56:20.964 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-22T06:56:23.832 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-22T06:56:23.965 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-22T06:56:29.039 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-22T06:56:29.091 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:56:34.186 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:56:37.618 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:56:37.618 INFO:teuthology.orchestra.run.smithi069.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-22T06:56:37.618 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T06:56:37.663 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:56:38.052 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:56:38.053 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-22T06:56:38.053 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-22T06:56:38.053 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-22T06:56:38.468 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:56:38.468 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T06:56:38.468 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-03-22T06:56:38.468 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-22T06:56:38.468 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-22T06:56:38.468 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-22T06:56:38.469 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-22T06:56:38.469 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T06:56:38.469 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T06:56:38.796 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q kernel | sort -rV | head -n 1 2024-03-22T06:56:38.900 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-22T06:56:38.900 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-22T06:56:38.900 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-22T06:56:38.900 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-22T06:56:38.900 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-22T06:56:38.901 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-03-22T06:56:38.901 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi069.front.sepia.ceph.com, path=None, version=distro) 2024-03-22T06:56:38.901 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q kernel | sort -rV | head -n 1 2024-03-22T06:56:38.981 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-22T06:56:38.981 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-22T06:56:38.981 DEBUG:teuthology.orchestra.run.smithi069:> sudo rpm -qi grub2-tools 2024-03-22T06:56:39.026 INFO:teuthology.orchestra.run.smithi069.stdout:Name : grub2-tools 2024-03-22T06:56:39.026 INFO:teuthology.orchestra.run.smithi069.stdout:Epoch : 1 2024-03-22T06:56:39.026 INFO:teuthology.orchestra.run.smithi069.stdout:Version : 2.06 2024-03-22T06:56:39.026 INFO:teuthology.orchestra.run.smithi069.stdout:Release : 68.el9 2024-03-22T06:56:39.026 INFO:teuthology.orchestra.run.smithi069.stdout:Architecture: x86_64 2024-03-22T06:56:39.027 INFO:teuthology.orchestra.run.smithi069.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-22T06:56:39.027 INFO:teuthology.orchestra.run.smithi069.stdout:Group : Unspecified 2024-03-22T06:56:39.027 INFO:teuthology.orchestra.run.smithi069.stdout:Size : 8264254 2024-03-22T06:56:39.027 INFO:teuthology.orchestra.run.smithi069.stdout:License : GPLv3+ 2024-03-22T06:56:39.027 INFO:teuthology.orchestra.run.smithi069.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-22T06:56:39.027 INFO:teuthology.orchestra.run.smithi069.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-22T06:56:39.027 INFO:teuthology.orchestra.run.smithi069.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-22T06:56:39.027 INFO:teuthology.orchestra.run.smithi069.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-22T06:56:39.027 INFO:teuthology.orchestra.run.smithi069.stdout:Packager : builder@centos.org 2024-03-22T06:56:39.027 INFO:teuthology.orchestra.run.smithi069.stdout:Vendor : CentOS 2024-03-22T06:56:39.027 INFO:teuthology.orchestra.run.smithi069.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-22T06:56:39.027 INFO:teuthology.orchestra.run.smithi069.stdout:Summary : Support tools for GRUB. 2024-03-22T06:56:39.027 INFO:teuthology.orchestra.run.smithi069.stdout:Description : 2024-03-22T06:56:39.027 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T06:56:39.027 INFO:teuthology.orchestra.run.smithi069.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-22T06:56:39.028 INFO:teuthology.orchestra.run.smithi069.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-22T06:56:39.028 INFO:teuthology.orchestra.run.smithi069.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-22T06:56:39.028 INFO:teuthology.orchestra.run.smithi069.stdout:hardware devices. 2024-03-22T06:56:39.028 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T06:56:39.028 INFO:teuthology.orchestra.run.smithi069.stdout:This subpackage provides tools for support of all platforms. 2024-03-22T06:56:39.029 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-22T06:56:39.029 INFO:teuthology.task.kernel:Updating grub on smithi069 to boot 5.14.0-430.el9.x86_64 2024-03-22T06:56:39.029 DEBUG:teuthology.orchestra.run.smithi069:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-22T06:56:39.613 INFO:teuthology.orchestra.run.smithi069.stderr:Generating grub configuration file ... 2024-03-22T06:56:41.013 INFO:teuthology.orchestra.run.smithi069.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-22T06:56:41.051 INFO:teuthology.orchestra.run.smithi069.stderr:done 2024-03-22T06:56:41.054 DEBUG:teuthology.orchestra.run.smithi069:> mktemp 2024-03-22T06:56:41.068 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.Ee5IkkXFg2 2024-03-22T06:56:41.069 DEBUG:teuthology.orchestra.run.smithi069:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Ee5IkkXFg2 2024-03-22T06:56:41.138 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.Ee5IkkXFg2 2024-03-22T06:56:41.282 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.Ee5IkkXFg2 is 6KB 2024-03-22T06:56:41.301 DEBUG:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.Ee5IkkXFg2 2024-03-22T06:56:41.315 DEBUG:teuthology.orchestra.run.smithi069:> sudo /bin/ls /boot/loader/entries || true 2024-03-22T06:56:41.379 INFO:teuthology.orchestra.run.smithi069.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-22T06:56:41.379 INFO:teuthology.orchestra.run.smithi069.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-22T06:56:41.379 INFO:teuthology.orchestra.run.smithi069.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-22T06:56:41.379 INFO:teuthology.orchestra.run.smithi069.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-22T06:56:41.380 DEBUG:teuthology.orchestra.run.smithi069:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-22T06:56:41.530 DEBUG:teuthology.orchestra.run.smithi069:> sudo shutdown -r now 2024-03-22T06:57:05.075 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:57:05.075 INFO:teuthology.orchestra.run.smithi063.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-22T06:57:05.075 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T06:57:05.122 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:57:05.519 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:57:05.519 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-22T06:57:05.520 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-22T06:57:05.520 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-22T06:57:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-22T06:57:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T06:57:05.934 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-03-22T06:57:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-22T06:57:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-22T06:57:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-22T06:57:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-22T06:57:05.934 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T06:57:05.934 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T06:57:06.299 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q kernel | sort -rV | head -n 1 2024-03-22T06:57:06.415 INFO:teuthology.orchestra.run.smithi063.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-22T06:57:06.450 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-22T06:57:06.450 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-22T06:57:06.450 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-22T06:57:06.450 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-22T06:57:06.450 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-03-22T06:57:06.450 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi063.front.sepia.ceph.com, path=None, version=distro) 2024-03-22T06:57:06.450 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q kernel | sort -rV | head -n 1 2024-03-22T06:57:06.532 INFO:teuthology.orchestra.run.smithi063.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-22T06:57:06.532 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-22T06:57:06.532 DEBUG:teuthology.orchestra.run.smithi063:> sudo rpm -qi grub2-tools 2024-03-22T06:57:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:Name : grub2-tools 2024-03-22T06:57:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:Epoch : 1 2024-03-22T06:57:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:Version : 2.06 2024-03-22T06:57:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:Release : 68.el9 2024-03-22T06:57:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:Architecture: x86_64 2024-03-22T06:57:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-22T06:57:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:Group : Unspecified 2024-03-22T06:57:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:Size : 8264254 2024-03-22T06:57:06.570 INFO:teuthology.orchestra.run.smithi063.stdout:License : GPLv3+ 2024-03-22T06:57:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-22T06:57:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-22T06:57:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-22T06:57:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-22T06:57:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:Packager : builder@centos.org 2024-03-22T06:57:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:Vendor : CentOS 2024-03-22T06:57:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-22T06:57:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:Summary : Support tools for GRUB. 2024-03-22T06:57:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:Description : 2024-03-22T06:57:06.571 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T06:57:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-22T06:57:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-22T06:57:06.571 INFO:teuthology.orchestra.run.smithi063.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-22T06:57:06.572 INFO:teuthology.orchestra.run.smithi063.stdout:hardware devices. 2024-03-22T06:57:06.572 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T06:57:06.572 INFO:teuthology.orchestra.run.smithi063.stdout:This subpackage provides tools for support of all platforms. 2024-03-22T06:57:06.572 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-22T06:57:06.572 INFO:teuthology.task.kernel:Updating grub on smithi063 to boot 5.14.0-430.el9.x86_64 2024-03-22T06:57:06.573 DEBUG:teuthology.orchestra.run.smithi063:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-22T06:57:07.194 INFO:teuthology.orchestra.run.smithi063.stderr:Generating grub configuration file ... 2024-03-22T06:57:08.706 INFO:teuthology.orchestra.run.smithi063.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-22T06:57:08.736 INFO:teuthology.orchestra.run.smithi063.stderr:done 2024-03-22T06:57:08.747 DEBUG:teuthology.orchestra.run.smithi063:> mktemp 2024-03-22T06:57:08.803 INFO:teuthology.orchestra.run.smithi063.stdout:/tmp/tmp.XjDunKeZ6R 2024-03-22T06:57:08.803 DEBUG:teuthology.orchestra.run.smithi063:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XjDunKeZ6R 2024-03-22T06:57:08.875 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0666 /tmp/tmp.XjDunKeZ6R 2024-03-22T06:57:09.022 DEBUG:teuthology.orchestra.remote:smithi063:/tmp/tmp.XjDunKeZ6R is 6KB 2024-03-22T06:57:09.072 DEBUG:teuthology.orchestra.run.smithi063:> rm -fr /tmp/tmp.XjDunKeZ6R 2024-03-22T06:57:09.086 DEBUG:teuthology.orchestra.run.smithi063:> sudo /bin/ls /boot/loader/entries || true 2024-03-22T06:57:09.154 INFO:teuthology.orchestra.run.smithi063.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-22T06:57:09.155 INFO:teuthology.orchestra.run.smithi063.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-22T06:57:09.155 INFO:teuthology.orchestra.run.smithi063.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-22T06:57:09.155 INFO:teuthology.orchestra.run.smithi063.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-22T06:57:09.156 DEBUG:teuthology.orchestra.run.smithi063:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-22T06:57:09.304 DEBUG:teuthology.orchestra.run.smithi063:> sudo shutdown -r now 2024-03-22T06:57:11.534 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-03-22T06:57:11.535 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-03-22T06:57:11.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:57:39.329 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-03-22T06:57:39.330 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-03-22T06:57:39.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:58:11.569 DEBUG:teuthology.orchestra.remote:timed out 2024-03-22T06:58:13.922 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-03-22T06:58:20.574 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-03-22T06:58:20.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:58:20.913 DEBUG:teuthology.orchestra.run.smithi069:> true 2024-03-22T06:58:21.381 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-03-22T06:58:21.381 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-430.el9.x86_64"... 2024-03-22T06:58:21.381 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2024-03-22T06:58:21.396 INFO:teuthology.orchestra.run.smithi069.stdout:5.14.0-430.el9.x86_64 2024-03-22T06:58:21.396 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-22T06:58:21.396 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-22T06:58:21.396 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-22T06:58:22.396 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-03-22T06:58:22.397 DEBUG:teuthology.orchestra.run.smithi069:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-22T06:58:22.486 INFO:teuthology.orchestra.run.smithi069.stdout:ttyS1 2024-03-22T06:58:22.532 DEBUG:teuthology.parallel:result is None 2024-03-22T06:58:22.923 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-03-22T06:58:22.924 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:58:41.314 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.63 2024-03-22T06:58:53.315 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-03-22T06:58:53.316 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T06:58:53.754 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-03-22T06:58:54.165 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-03-22T06:58:54.165 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-430.el9.x86_64"... 2024-03-22T06:58:54.165 DEBUG:teuthology.orchestra.run.smithi063:> uname -r 2024-03-22T06:58:54.178 INFO:teuthology.orchestra.run.smithi063.stdout:5.14.0-430.el9.x86_64 2024-03-22T06:58:54.179 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-22T06:58:54.179 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-22T06:58:54.179 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-22T06:58:55.180 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-03-22T06:58:55.180 DEBUG:teuthology.orchestra.run.smithi063:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-22T06:58:55.240 INFO:teuthology.orchestra.run.smithi063.stdout:ttyS1 2024-03-22T06:58:55.286 DEBUG:teuthology.parallel:result is None 2024-03-22T06:58:55.287 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-22T06:58:55.300 INFO:teuthology.task.internal:Creating test directory... 2024-03-22T06:58:55.300 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-22T06:58:55.302 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-22T06:58:55.321 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-22T06:58:55.327 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-22T06:58:55.340 INFO:teuthology.task.internal:Creating archive directory... 2024-03-22T06:58:55.340 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-22T06:58:55.359 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-22T06:58:55.424 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-22T06:58:55.431 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-22T06:58:55.431 DEBUG:teuthology.orchestra.run.smithi063:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-22T06:58:55.433 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-22T06:58:55.471 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T06:58:55.471 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T06:58:55.487 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T06:58:55.488 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-22T06:58:55.488 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-22T06:58:55.495 INFO:teuthology.task.internal:Configuring sudo... 2024-03-22T06:58:55.495 DEBUG:teuthology.orchestra.run.smithi063:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-22T06:58:55.533 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-22T06:58:55.563 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-22T06:58:55.572 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-22T06:58:55.573 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-22T06:58:55.603 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-22T06:58:55.619 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T06:58:55.701 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T06:58:55.758 DEBUG:teuthology.orchestra.run.smithi063:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T06:58:55.844 DEBUG:teuthology.orchestra.run.smithi063:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T06:58:55.886 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T06:58:55.886 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-22T06:58:55.956 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T06:58:56.001 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-22T06:58:56.055 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T06:58:56.139 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-22T06:58:56.184 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-22T06:58:56.184 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-22T06:58:56.253 DEBUG:teuthology.orchestra.run.smithi063:> sudo service rsyslog restart 2024-03-22T06:58:56.255 DEBUG:teuthology.orchestra.run.smithi069:> sudo service rsyslog restart 2024-03-22T06:58:56.302 INFO:teuthology.orchestra.run.smithi063.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T06:58:56.352 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T06:58:56.814 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-22T06:58:56.821 INFO:teuthology.task.internal:Starting timer... 2024-03-22T06:58:56.821 INFO:teuthology.run_tasks:Running task pcp... 2024-03-22T06:58:56.841 INFO:teuthology.run_tasks:Running task selinux... 2024-03-22T06:58:56.850 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-22T06:58:56.850 DEBUG:teuthology.orchestra.run.smithi063:> sudo service auditd rotate 2024-03-22T06:58:56.911 INFO:teuthology.orchestra.run.smithi063.stdout:Rotating logs: 2024-03-22T06:58:56.913 DEBUG:teuthology.orchestra.run.smithi069:> sudo service auditd rotate 2024-03-22T06:58:56.983 INFO:teuthology.orchestra.run.smithi069.stdout:Rotating logs: 2024-03-22T06:58:56.985 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-22T06:58:56.985 DEBUG:teuthology.orchestra.run.smithi063:> /usr/sbin/getenforce 2024-03-22T06:58:57.009 INFO:teuthology.orchestra.run.smithi063.stdout:Permissive 2024-03-22T06:58:57.009 DEBUG:teuthology.orchestra.run.smithi069:> /usr/sbin/getenforce 2024-03-22T06:58:57.029 INFO:teuthology.orchestra.run.smithi069.stdout:Permissive 2024-03-22T06:58:57.030 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi063.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi069.front.sepia.ceph.com': 'permissive'} 2024-03-22T06:58:57.030 DEBUG:teuthology.orchestra.run.smithi063:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-22T06:58:57.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T06:58:57.078 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-22T06:58:57.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T06:58:57.106 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-22T06:58:57.106 DEBUG:teuthology.orchestra.run.smithi063:> sudo /usr/sbin/setenforce permissive 2024-03-22T06:58:57.162 DEBUG:teuthology.orchestra.run.smithi069:> sudo /usr/sbin/setenforce permissive 2024-03-22T06:58:57.198 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-22T06:58:57.208 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-22T06:58:57.224 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-22T06:58:57.226 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi063.front.sepia.ceph.com,smithi069.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-22T07:03:13.237 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi063.front.sepia.ceph.com'), Remote(name='ubuntu@smithi069.front.sepia.ceph.com')] 2024-03-22T07:03:13.239 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-03-22T07:03:13.240 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi063.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T07:03:13.330 DEBUG:teuthology.orchestra.run.smithi063:> true 2024-03-22T07:03:13.417 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi063.front.sepia.ceph.com' 2024-03-22T07:03:13.417 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-03-22T07:03:13.417 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-22T07:03:13.507 DEBUG:teuthology.orchestra.run.smithi069:> true 2024-03-22T07:03:13.590 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-03-22T07:03:13.590 INFO:teuthology.run_tasks:Running task clock... 2024-03-22T07:03:13.602 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-22T07:03:13.603 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-22T07:03:13.603 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T07:03:13.605 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-22T07:03:13.605 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T07:03:13.634 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-22T07:03:13.650 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-22T07:03:13.662 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-22T07:03:13.679 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-22T07:03:13.679 INFO:teuthology.orchestra.run.smithi063.stderr:sudo: ntpd: command not found 2024-03-22T07:03:13.693 INFO:teuthology.orchestra.run.smithi063.stdout:506 Cannot talk to daemon 2024-03-22T07:03:13.708 INFO:teuthology.orchestra.run.smithi069.stderr:sudo: ntpd: command not found 2024-03-22T07:03:13.709 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-22T07:03:13.722 INFO:teuthology.orchestra.run.smithi069.stdout:506 Cannot talk to daemon 2024-03-22T07:03:13.725 INFO:teuthology.orchestra.run.smithi063.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-22T07:03:13.738 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-22T07:03:13.754 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-22T07:03:13.783 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: ntpq: command not found 2024-03-22T07:03:13.786 INFO:teuthology.orchestra.run.smithi063.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T07:03:13.787 INFO:teuthology.orchestra.run.smithi063.stdout:=============================================================================== 2024-03-22T07:03:13.787 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T07:03:13.787 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T07:03:13.787 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T07:03:13.787 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T07:03:13.806 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-03-22T07:03:13.810 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T07:03:13.810 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-03-22T07:03:13.810 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T07:03:13.810 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T07:03:13.810 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T07:03:13.810 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T07:03:13.810 INFO:teuthology.run_tasks:Running task install... 2024-03-22T07:03:13.818 DEBUG:teuthology.task.install:project ceph 2024-03-22T07:03:13.819 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'db0330b1e4e2470d52b750e251e55a522b4f7d69'}} 2024-03-22T07:03:13.819 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'db0330b1e4e2470d52b750e251e55a522b4f7d69'} 2024-03-22T07:03:13.819 INFO:teuthology.task.install:Using flavor: default 2024-03-22T07:03:13.824 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-22T07:03:13.824 INFO:teuthology.task.install:extra packages: [] 2024-03-22T07:03:13.825 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'db0330b1e4e2470d52b750e251e55a522b4f7d69', 'tag': None, 'wait_for_package': False} 2024-03-22T07:03:13.825 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T07:03:13.825 INFO:teuthology.packaging:ref: None 2024-03-22T07:03:13.825 INFO:teuthology.packaging:tag: None 2024-03-22T07:03:13.825 INFO:teuthology.packaging:branch: quincy 2024-03-22T07:03:13.825 INFO:teuthology.packaging:sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:03:13.825 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-22T07:03:13.827 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'db0330b1e4e2470d52b750e251e55a522b4f7d69', 'tag': None, 'wait_for_package': False} 2024-03-22T07:03:13.828 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T07:03:13.828 INFO:teuthology.packaging:ref: None 2024-03-22T07:03:13.828 INFO:teuthology.packaging:tag: None 2024-03-22T07:03:13.828 INFO:teuthology.packaging:branch: quincy 2024-03-22T07:03:13.828 INFO:teuthology.packaging:sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:03:13.828 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-22T07:03:14.161 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/ 2024-03-22T07:03:14.162 INFO:teuthology.task.install.rpm:Package version is 17.2.7-726.g9c4e0c34 2024-03-22T07:03:14.176 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/ 2024-03-22T07:03:14.176 INFO:teuthology.task.install.rpm:Package version is 17.2.7-726.g9c4e0c34 2024-03-22T07:03:14.279 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-22T07:03:14.279 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-22T07:03:14.279 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-22T07:03:14.309 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-22T07:03:14.310 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T07:03:14.310 INFO:teuthology.packaging:ref: None 2024-03-22T07:03:14.310 INFO:teuthology.packaging:tag: None 2024-03-22T07:03:14.310 INFO:teuthology.packaging:branch: quincy 2024-03-22T07:03:14.310 INFO:teuthology.packaging:sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:03:14.310 DEBUG:teuthology.orchestra.run.smithi069:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-22T07:03:14.379 DEBUG:teuthology.orchestra.run.smithi069:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-22T07:03:14.460 DEBUG:teuthology.orchestra.run.smithi069:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-22T07:03:14.468 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-22T07:03:14.468 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T07:03:14.468 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-22T07:03:14.497 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-22T07:03:14.497 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T07:03:14.497 INFO:teuthology.packaging:ref: None 2024-03-22T07:03:14.497 INFO:teuthology.packaging:tag: None 2024-03-22T07:03:14.497 INFO:teuthology.packaging:branch: quincy 2024-03-22T07:03:14.497 INFO:teuthology.packaging:sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:03:14.497 DEBUG:teuthology.orchestra.run.smithi063:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-22T07:03:14.527 INFO:teuthology.orchestra.run.smithi069.stdout:check_obsoletes = 1 2024-03-22T07:03:14.529 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum clean all 2024-03-22T07:03:14.567 DEBUG:teuthology.orchestra.run.smithi063:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-22T07:03:14.648 DEBUG:teuthology.orchestra.run.smithi063:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-22T07:03:14.716 INFO:teuthology.orchestra.run.smithi063.stdout:check_obsoletes = 1 2024-03-22T07:03:14.718 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum clean all 2024-03-22T07:03:14.782 INFO:teuthology.orchestra.run.smithi069.stdout:74 files removed 2024-03-22T07:03:14.809 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-radosgw 2024-03-22T07:03:14.967 INFO:teuthology.orchestra.run.smithi063.stdout:74 files removed 2024-03-22T07:03:14.994 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-radosgw 2024-03-22T07:03:15.502 INFO:teuthology.orchestra.run.smithi069.stdout:ceph packages for x86_64 193 kB/s | 77 kB 00:00 2024-03-22T07:03:15.701 INFO:teuthology.orchestra.run.smithi063.stdout:ceph packages for x86_64 191 kB/s | 77 kB 00:00 2024-03-22T07:03:15.853 INFO:teuthology.orchestra.run.smithi069.stdout:ceph noarch packages 30 kB/s | 9.2 kB 00:00 2024-03-22T07:03:16.144 INFO:teuthology.orchestra.run.smithi063.stdout:ceph noarch packages 23 kB/s | 9.2 kB 00:00 2024-03-22T07:03:16.221 INFO:teuthology.orchestra.run.smithi069.stdout:ceph source packages 5.4 kB/s | 1.8 kB 00:00 2024-03-22T07:03:16.594 INFO:teuthology.orchestra.run.smithi063.stdout:ceph source packages 4.3 kB/s | 1.8 kB 00:00 2024-03-22T07:03:16.640 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS-9 - AppStream 18 MB/s | 6.7 MB 00:00 2024-03-22T07:03:17.022 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-03-22T07:03:18.315 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS-9 - BaseOS 3.6 MB/s | 2.0 MB 00:00 2024-03-22T07:03:18.597 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS-9 - BaseOS 4.6 MB/s | 2.0 MB 00:00 2024-03-22T07:03:19.396 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-03-22T07:03:21.005 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 3.4 MB/s | 8.0 MB 00:02 2024-03-22T07:03:21.339 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-03-22T07:03:22.940 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-03-22T07:03:25.524 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.1 MB 00:00 2024-03-22T07:03:26.757 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.1 MB 00:00 2024-03-22T07:03:27.176 INFO:teuthology.orchestra.run.smithi063.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-03-22T07:03:28.126 INFO:teuthology.orchestra.run.smithi063.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-03-22T07:03:28.974 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-03-22T07:03:33.310 INFO:teuthology.orchestra.run.smithi063.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-03-22T07:03:35.849 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:03:35.861 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:03:35.861 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-03-22T07:03:35.861 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:03:35.861 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-03-22T07:03:35.861 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-radosgw x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 14 M 2024-03-22T07:03:35.861 INFO:teuthology.orchestra.run.smithi063.stdout:Upgrading: 2024-03-22T07:03:35.861 INFO:teuthology.orchestra.run.smithi063.stdout: librados2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 3.1 M 2024-03-22T07:03:35.861 INFO:teuthology.orchestra.run.smithi063.stdout: librbd1 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.9 M 2024-03-22T07:03:35.861 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-03-22T07:03:35.861 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 5.0 M 2024-03-22T07:03:35.862 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 23 M 2024-03-22T07:03:35.862 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-selinux x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 25 k 2024-03-22T07:03:35.862 INFO:teuthology.orchestra.run.smithi063.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-22T07:03:35.862 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 636 k 2024-03-22T07:03:35.862 INFO:teuthology.orchestra.run.smithi063.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-22T07:03:35.862 INFO:teuthology.orchestra.run.smithi063.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-22T07:03:35.862 INFO:teuthology.orchestra.run.smithi063.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-22T07:03:35.862 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 457 k 2024-03-22T07:03:35.862 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-22T07:03:35.862 INFO:teuthology.orchestra.run.smithi063.stdout: librgw2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 4.9 M 2024-03-22T07:03:35.862 INFO:teuthology.orchestra.run.smithi063.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-22T07:03:35.862 INFO:teuthology.orchestra.run.smithi063.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-22T07:03:35.862 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 45 k 2024-03-22T07:03:35.863 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 117 k 2024-03-22T07:03:35.863 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cephfs x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 156 k 2024-03-22T07:03:35.863 INFO:teuthology.orchestra.run.smithi063.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-22T07:03:35.863 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 320 k 2024-03-22T07:03:35.863 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rbd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 293 k 2024-03-22T07:03:35.863 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 99 k 2024-03-22T07:03:35.863 INFO:teuthology.orchestra.run.smithi063.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-22T07:03:35.863 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:03:35.863 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-03-22T07:03:35.863 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:03:35.863 INFO:teuthology.orchestra.run.smithi063.stdout:Install 22 Packages 2024-03-22T07:03:35.863 INFO:teuthology.orchestra.run.smithi063.stdout:Upgrade 2 Packages 2024-03-22T07:03:35.863 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:03:35.864 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 58 M 2024-03-22T07:03:35.864 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-03-22T07:03:36.795 INFO:teuthology.orchestra.run.smithi063.stdout:(1/24): ceph-base-17.2.7-726.g9c4e0c34.el9.x86_ 6.7 MB/s | 5.0 MB 00:00 2024-03-22T07:03:36.870 INFO:teuthology.orchestra.run.smithi063.stdout:(2/24): ceph-selinux-17.2.7-726.g9c4e0c34.el9.x 336 kB/s | 25 kB 00:00 2024-03-22T07:03:36.970 INFO:teuthology.orchestra.run.smithi063.stdout:(3/24): libcephfs2-17.2.7-726.g9c4e0c34.el9.x86 6.2 MB/s | 636 kB 00:00 2024-03-22T07:03:37.070 INFO:teuthology.orchestra.run.smithi063.stdout:(4/24): libradosstriper1-17.2.7-726.g9c4e0c34.e 4.5 MB/s | 457 kB 00:00 2024-03-22T07:03:37.588 INFO:teuthology.orchestra.run.smithi063.stdout:(5/24): librgw2-17.2.7-726.g9c4e0c34.el9.x86_64 9.5 MB/s | 4.9 MB 00:00 2024-03-22T07:03:37.688 INFO:teuthology.orchestra.run.smithi063.stdout:(6/24): python3-ceph-argparse-17.2.7-726.g9c4e0 452 kB/s | 45 kB 00:00 2024-03-22T07:03:37.771 INFO:teuthology.orchestra.run.smithi063.stdout:(7/24): python3-ceph-common-17.2.7-726.g9c4e0c3 1.4 MB/s | 117 kB 00:00 2024-03-22T07:03:37.854 INFO:teuthology.orchestra.run.smithi063.stdout:(8/24): python3-cephfs-17.2.7-726.g9c4e0c34.el9 1.8 MB/s | 156 kB 00:00 2024-03-22T07:03:37.971 INFO:teuthology.orchestra.run.smithi063.stdout:(9/24): python3-rados-17.2.7-726.g9c4e0c34.el9. 2.7 MB/s | 320 kB 00:00 2024-03-22T07:03:38.072 INFO:teuthology.orchestra.run.smithi063.stdout:(10/24): python3-rbd-17.2.7-726.g9c4e0c34.el9.x 2.9 MB/s | 293 kB 00:00 2024-03-22T07:03:38.147 INFO:teuthology.orchestra.run.smithi063.stdout:(11/24): python3-rgw-17.2.7-726.g9c4e0c34.el9.x 1.3 MB/s | 99 kB 00:00 2024-03-22T07:03:38.365 INFO:teuthology.orchestra.run.smithi063.stdout:(12/24): ceph-common-17.2.7-726.g9c4e0c34.el9.x 9.7 MB/s | 23 MB 00:02 2024-03-22T07:03:38.573 INFO:teuthology.orchestra.run.smithi063.stdout:(13/24): ceph-radosgw-17.2.7-726.g9c4e0c34.el9. 5.6 MB/s | 14 MB 00:02 2024-03-22T07:03:38.623 INFO:teuthology.orchestra.run.smithi063.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 190 kB/s | 44 kB 00:00 2024-03-22T07:03:38.648 INFO:teuthology.orchestra.run.smithi063.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 318 kB/s | 159 kB 00:00 2024-03-22T07:03:38.715 INFO:teuthology.orchestra.run.smithi063.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 661 kB 00:00 2024-03-22T07:03:38.741 INFO:teuthology.orchestra.run.smithi063.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 282 kB 00:00 2024-03-22T07:03:38.766 INFO:teuthology.orchestra.run.smithi063.stdout:(18/24): python3-prettytable-0.7.2-27.el9.noarc 352 kB/s | 41 kB 00:00 2024-03-22T07:03:39.049 INFO:teuthology.orchestra.run.smithi063.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 171 kB/s | 49 kB 00:00 2024-03-22T07:03:39.075 INFO:teuthology.orchestra.run.smithi063.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 218 kB/s | 67 kB 00:00 2024-03-22T07:03:39.200 INFO:teuthology.orchestra.run.smithi063.stdout:(21/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 638 kB/s | 309 kB 00:00 2024-03-22T07:03:39.350 INFO:teuthology.orchestra.run.smithi063.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 5.3 MB/s | 1.6 MB 00:00 2024-03-22T07:03:39.401 INFO:teuthology.orchestra.run.smithi063.stdout:(23/24): librados2-17.2.7-726.g9c4e0c34.el9.x86 9.5 MB/s | 3.1 MB 00:00 2024-03-22T07:03:39.568 INFO:teuthology.orchestra.run.smithi063.stdout:(24/24): librbd1-17.2.7-726.g9c4e0c34.el9.x86_6 8.0 MB/s | 2.9 MB 00:00 2024-03-22T07:03:39.575 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:03:39.575 INFO:teuthology.orchestra.run.smithi063.stdout:Total 16 MB/s | 58 MB 00:03 2024-03-22T07:03:39.576 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-03-22T07:03:39.613 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-03-22T07:03:39.614 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-03-22T07:03:40.091 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-03-22T07:03:40.091 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-03-22T07:03:41.357 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-03-22T07:03:41.697 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-03-22T07:03:41.736 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-22T07:03:41.813 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-22T07:03:41.893 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/26 2024-03-22T07:03:41.921 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-22T07:03:42.010 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-22T07:03:42.062 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-03-22T07:03:42.092 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-03-22T07:03:42.551 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-03-22T07:03:42.582 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 8/26 2024-03-22T07:03:42.645 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 8/26 2024-03-22T07:03:42.674 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-03-22T07:03:42.717 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x 10/26 2024-03-22T07:03:42.754 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 11/26 2024-03-22T07:03:42.818 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 12/26 2024-03-22T07:03:42.843 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-22T07:03:42.892 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-22T07:03:42.932 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-03-22T07:03:42.977 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-03-22T07:03:43.009 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-03-22T07:03:43.319 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-03-22T07:03:43.347 INFO:teuthology.orchestra.run.smithi063.stdout: Upgrading : librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 18/26 2024-03-22T07:03:43.438 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 18/26 2024-03-22T07:03:43.513 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 19/26 2024-03-22T07:03:43.541 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86 20/26 2024-03-22T07:03:46.038 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-22T07:03:46.071 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-22T07:03:46.653 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-22T07:03:46.691 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 22/26 2024-03-22T07:03:46.784 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 22/26 2024-03-22T07:03:46.784 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-22T07:03:46.785 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-22T07:03:46.785 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:03:46.817 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-22T07:03:50.244 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 21 MB 00:21 2024-03-22T07:03:55.274 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-03-22T07:03:56.822 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-22T07:03:56.823 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /sys 2024-03-22T07:03:56.823 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /proc 2024-03-22T07:03:56.823 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /mnt 2024-03-22T07:03:56.823 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /var/tmp 2024-03-22T07:03:56.823 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /home 2024-03-22T07:03:56.823 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /root 2024-03-22T07:03:56.823 INFO:teuthology.orchestra.run.smithi063.stdout:skipping the directory /tmp 2024-03-22T07:03:56.823 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:03:57.709 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:03:57.715 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:03:57.716 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-03-22T07:03:57.716 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:03:57.716 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-03-22T07:03:57.716 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 14 M 2024-03-22T07:03:57.716 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrading: 2024-03-22T07:03:57.716 INFO:teuthology.orchestra.run.smithi069.stdout: librados2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 3.1 M 2024-03-22T07:03:57.716 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.9 M 2024-03-22T07:03:57.716 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-03-22T07:03:57.716 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 5.0 M 2024-03-22T07:03:57.716 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 23 M 2024-03-22T07:03:57.716 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 25 k 2024-03-22T07:03:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-22T07:03:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 636 k 2024-03-22T07:03:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-22T07:03:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-22T07:03:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-22T07:03:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 457 k 2024-03-22T07:03:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-22T07:03:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2 x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 4.9 M 2024-03-22T07:03:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-22T07:03:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-22T07:03:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 45 k 2024-03-22T07:03:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 117 k 2024-03-22T07:03:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 156 k 2024-03-22T07:03:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-22T07:03:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 320 k 2024-03-22T07:03:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 293 k 2024-03-22T07:03:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 99 k 2024-03-22T07:03:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-22T07:03:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:03:57.718 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-03-22T07:03:57.718 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:03:57.718 INFO:teuthology.orchestra.run.smithi069.stdout:Install 22 Packages 2024-03-22T07:03:57.718 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrade 2 Packages 2024-03-22T07:03:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:03:57.719 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 58 M 2024-03-22T07:03:57.719 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-03-22T07:03:58.164 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 24/26 2024-03-22T07:03:58.250 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 24/26 2024-03-22T07:03:58.250 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T07:03:58.250 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-22T07:03:58.250 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-22T07:03:58.250 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-22T07:03:58.250 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:03:58.250 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-22T07:03:58.331 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-22T07:03:58.331 INFO:teuthology.orchestra.run.smithi063.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-22T07:03:59.150 INFO:teuthology.orchestra.run.smithi069.stdout:(1/24): ceph-base-17.2.7-726.g9c4e0c34.el9.x86_ 4.7 MB/s | 5.0 MB 00:01 2024-03-22T07:03:59.208 INFO:teuthology.orchestra.run.smithi069.stdout:(2/24): ceph-selinux-17.2.7-726.g9c4e0c34.el9.x 432 kB/s | 25 kB 00:00 2024-03-22T07:03:59.284 INFO:teuthology.orchestra.run.smithi069.stdout:(3/24): libcephfs2-17.2.7-726.g9c4e0c34.el9.x86 8.3 MB/s | 636 kB 00:00 2024-03-22T07:03:59.384 INFO:teuthology.orchestra.run.smithi069.stdout:(4/24): libradosstriper1-17.2.7-726.g9c4e0c34.e 4.5 MB/s | 457 kB 00:00 2024-03-22T07:03:59.554 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-22T07:03:59.554 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/26 2024-03-22T07:03:59.554 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-22T07:03:59.554 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/26 2024-03-22T07:03:59.554 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-22T07:03:59.554 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/26 2024-03-22T07:03:59.554 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/26 2024-03-22T07:03:59.554 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 7/26 2024-03-22T07:03:59.555 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x 8/26 2024-03-22T07:03:59.555 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86 9/26 2024-03-22T07:03:59.555 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 10/26 2024-03-22T07:03:59.555 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 11/26 2024-03-22T07:03:59.555 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 12/26 2024-03-22T07:03:59.555 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-22T07:03:59.555 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-03-22T07:03:59.555 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-03-22T07:03:59.555 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-03-22T07:03:59.555 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-03-22T07:03:59.555 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-03-22T07:03:59.555 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-03-22T07:03:59.555 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-03-22T07:03:59.555 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-03-22T07:03:59.556 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-03-22T07:03:59.556 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-22T07:03:59.556 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-03-22T07:03:59.556 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 25/26 2024-03-22T07:03:59.793 INFO:teuthology.orchestra.run.smithi069.stdout:(5/24): librgw2-17.2.7-726.g9c4e0c34.el9.x86_64 12 MB/s | 4.9 MB 00:00 2024-03-22T07:03:59.859 INFO:teuthology.orchestra.run.smithi069.stdout:(6/24): python3-ceph-argparse-17.2.7-726.g9c4e0 681 kB/s | 45 kB 00:00 2024-03-22T07:03:59.935 INFO:teuthology.orchestra.run.smithi069.stdout:(7/24): python3-ceph-common-17.2.7-726.g9c4e0c3 1.5 MB/s | 117 kB 00:00 2024-03-22T07:04:00.051 INFO:teuthology.orchestra.run.smithi069.stdout:(8/24): python3-cephfs-17.2.7-726.g9c4e0c34.el9 1.3 MB/s | 156 kB 00:00 2024-03-22T07:04:00.143 INFO:teuthology.orchestra.run.smithi069.stdout:(9/24): python3-rados-17.2.7-726.g9c4e0c34.el9. 3.4 MB/s | 320 kB 00:00 2024-03-22T07:04:00.559 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-03-22T07:04:00.559 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:00.559 INFO:teuthology.orchestra.run.smithi063.stdout:Upgraded: 2024-03-22T07:04:00.559 INFO:teuthology.orchestra.run.smithi063.stdout: librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:00.559 INFO:teuthology.orchestra.run.smithi063.stdout: librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:00.559 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-03-22T07:04:00.559 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:00.559 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:00.559 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:00.559 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:00.559 INFO:teuthology.orchestra.run.smithi063.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-22T07:04:00.559 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:00.559 INFO:teuthology.orchestra.run.smithi063.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-22T07:04:00.560 INFO:teuthology.orchestra.run.smithi063.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-22T07:04:00.560 INFO:teuthology.orchestra.run.smithi063.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-22T07:04:00.560 INFO:teuthology.orchestra.run.smithi063.stdout: libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:00.560 INFO:teuthology.orchestra.run.smithi063.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-22T07:04:00.560 INFO:teuthology.orchestra.run.smithi063.stdout: librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:00.560 INFO:teuthology.orchestra.run.smithi063.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-22T07:04:00.560 INFO:teuthology.orchestra.run.smithi063.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-22T07:04:00.560 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:00.560 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:00.560 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:00.560 INFO:teuthology.orchestra.run.smithi063.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-22T07:04:00.560 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:00.560 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:00.560 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:00.561 INFO:teuthology.orchestra.run.smithi063.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-22T07:04:00.561 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:00.561 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:04:00.906 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-test 2024-03-22T07:04:00.969 INFO:teuthology.orchestra.run.smithi069.stdout:(10/24): python3-rbd-17.2.7-726.g9c4e0c34.el9.x 355 kB/s | 293 kB 00:00 2024-03-22T07:04:01.044 INFO:teuthology.orchestra.run.smithi069.stdout:(11/24): python3-rgw-17.2.7-726.g9c4e0c34.el9.x 1.3 MB/s | 99 kB 00:00 2024-03-22T07:04:01.153 INFO:teuthology.orchestra.run.smithi069.stdout:(12/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 159 kB 00:00 2024-03-22T07:04:01.178 INFO:teuthology.orchestra.run.smithi069.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.7 MB/s | 44 kB 00:00 2024-03-22T07:04:01.237 INFO:teuthology.orchestra.run.smithi069.stdout:(14/24): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-03-22T07:04:01.278 INFO:teuthology.orchestra.run.smithi069.stdout:(15/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-03-22T07:04:01.304 INFO:teuthology.orchestra.run.smithi069.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 1.6 MB/s | 41 kB 00:00 2024-03-22T07:04:01.477 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:00:28 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:04:01.632 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:04:01.633 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:04:01.633 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-03-22T07:04:01.633 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:04:01.633 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-03-22T07:04:01.633 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-test x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 48 M 2024-03-22T07:04:01.633 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-03-22T07:04:01.633 INFO:teuthology.orchestra.run.smithi063.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-22T07:04:01.633 INFO:teuthology.orchestra.run.smithi063.stdout: libcephsqlite x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 163 k 2024-03-22T07:04:01.634 INFO:teuthology.orchestra.run.smithi063.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-22T07:04:01.634 INFO:teuthology.orchestra.run.smithi063.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-22T07:04:01.634 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-22T07:04:01.634 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:01.634 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-03-22T07:04:01.634 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:04:01.634 INFO:teuthology.orchestra.run.smithi063.stdout:Install 6 Packages 2024-03-22T07:04:01.634 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:01.635 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 49 M 2024-03-22T07:04:01.635 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 196 M 2024-03-22T07:04:01.635 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-03-22T07:04:01.914 INFO:teuthology.orchestra.run.smithi069.stdout:(17/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 506 kB/s | 309 kB 00:00 2024-03-22T07:04:02.031 INFO:teuthology.orchestra.run.smithi069.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 417 kB/s | 49 kB 00:00 2024-03-22T07:04:02.130 INFO:teuthology.orchestra.run.smithi063.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 946 kB/s | 217 kB 00:00 2024-03-22T07:04:02.140 INFO:teuthology.orchestra.run.smithi069.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 623 kB/s | 67 kB 00:00 2024-03-22T07:04:02.155 INFO:teuthology.orchestra.run.smithi063.stdout:(2/6): libcephsqlite-17.2.7-726.g9c4e0c34.el9.x 640 kB/s | 163 kB 00:00 2024-03-22T07:04:02.239 INFO:teuthology.orchestra.run.smithi063.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 300 kB 00:00 2024-03-22T07:04:02.264 INFO:teuthology.orchestra.run.smithi063.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 576 kB/s | 63 kB 00:00 2024-03-22T07:04:02.339 INFO:teuthology.orchestra.run.smithi063.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 1.8 MB/s | 187 kB 00:00 2024-03-22T07:04:02.457 INFO:teuthology.orchestra.run.smithi069.stdout:(20/24): thrift-0.15.0-2.el9.x86_64.rpm 5.0 MB/s | 1.6 MB 00:00 2024-03-22T07:04:02.974 INFO:teuthology.orchestra.run.smithi069.stdout:(21/24): ceph-common-17.2.7-726.g9c4e0c34.el9.x 4.6 MB/s | 23 MB 00:04 2024-03-22T07:04:03.241 INFO:teuthology.orchestra.run.smithi069.stdout:(22/24): librados2-17.2.7-726.g9c4e0c34.el9.x86 3.9 MB/s | 3.1 MB 00:00 2024-03-22T07:04:04.149 INFO:teuthology.orchestra.run.smithi069.stdout:(23/24): librbd1-17.2.7-726.g9c4e0c34.el9.x86_6 2.5 MB/s | 2.9 MB 00:01 2024-03-22T07:04:04.766 INFO:teuthology.orchestra.run.smithi069.stdout:(24/24): ceph-radosgw-17.2.7-726.g9c4e0c34.el9. 2.1 MB/s | 14 MB 00:06 2024-03-22T07:04:04.771 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:04:04.771 INFO:teuthology.orchestra.run.smithi069.stdout:Total 8.2 MB/s | 58 MB 00:07 2024-03-22T07:04:04.771 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-03-22T07:04:04.805 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-03-22T07:04:04.805 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-03-22T07:04:05.265 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-03-22T07:04:05.266 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-03-22T07:04:06.505 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-03-22T07:04:06.849 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-03-22T07:04:06.910 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-22T07:04:07.001 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-22T07:04:07.079 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/26 2024-03-22T07:04:07.127 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-22T07:04:07.198 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-22T07:04:07.256 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-03-22T07:04:07.291 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-03-22T07:04:07.734 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-03-22T07:04:07.772 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 8/26 2024-03-22T07:04:07.826 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 8/26 2024-03-22T07:04:07.857 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-03-22T07:04:07.897 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x 10/26 2024-03-22T07:04:07.935 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 11/26 2024-03-22T07:04:07.998 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 12/26 2024-03-22T07:04:08.024 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-22T07:04:08.094 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-22T07:04:08.130 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-03-22T07:04:08.166 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-03-22T07:04:08.207 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-03-22T07:04:08.504 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-03-22T07:04:08.528 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 18/26 2024-03-22T07:04:08.618 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 18/26 2024-03-22T07:04:08.691 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 19/26 2024-03-22T07:04:08.731 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86 20/26 2024-03-22T07:04:11.145 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-22T07:04:11.186 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-22T07:04:11.723 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 21/26 2024-03-22T07:04:11.764 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 22/26 2024-03-22T07:04:11.854 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 22/26 2024-03-22T07:04:11.854 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-22T07:04:11.854 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-22T07:04:11.854 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:11.890 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-22T07:04:14.989 INFO:teuthology.orchestra.run.smithi063.stdout:(6/6): ceph-test-17.2.7-726.g9c4e0c34.el9.x86_6 3.7 MB/s | 48 MB 00:13 2024-03-22T07:04:14.991 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:04:14.992 INFO:teuthology.orchestra.run.smithi063.stdout:Total 3.7 MB/s | 49 MB 00:13 2024-03-22T07:04:15.005 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-03-22T07:04:15.025 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-03-22T07:04:15.025 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-03-22T07:04:15.421 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-03-22T07:04:15.421 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-03-22T07:04:16.020 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-03-22T07:04:16.117 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-22T07:04:16.157 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-22T07:04:16.188 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-22T07:04:16.231 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-22T07:04:16.258 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/6 2024-03-22T07:04:20.962 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/6 2024-03-22T07:04:21.039 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/6 2024-03-22T07:04:21.257 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-22T07:04:21.258 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /sys 2024-03-22T07:04:21.258 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /proc 2024-03-22T07:04:21.258 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /mnt 2024-03-22T07:04:21.258 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /var/tmp 2024-03-22T07:04:21.258 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /home 2024-03-22T07:04:21.258 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /root 2024-03-22T07:04:21.258 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /tmp 2024-03-22T07:04:21.258 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:22.566 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 24/26 2024-03-22T07:04:22.644 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 24/26 2024-03-22T07:04:22.644 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T07:04:22.644 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-22T07:04:22.644 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-22T07:04:22.644 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-22T07:04:22.644 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:22.646 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-22T07:04:22.735 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-03-22T07:04:22.735 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-22T07:04:23.215 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/6 2024-03-22T07:04:23.215 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/6 2024-03-22T07:04:23.215 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/6 2024-03-22T07:04:23.216 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-22T07:04:23.216 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-22T07:04:23.216 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-22T07:04:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-22T07:04:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:23.630 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-03-22T07:04:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: jq-1.6-16.el9.x86_64 2024-03-22T07:04:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-22T07:04:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-22T07:04:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-22T07:04:23.630 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:23.630 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:04:24.032 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-03-22T07:04:24.032 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/26 2024-03-22T07:04:24.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/26 2024-03-22T07:04:24.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/26 2024-03-22T07:04:24.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/26 2024-03-22T07:04:24.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/26 2024-03-22T07:04:24.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/26 2024-03-22T07:04:24.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 7/26 2024-03-22T07:04:24.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x 8/26 2024-03-22T07:04:24.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86 9/26 2024-03-22T07:04:24.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 10/26 2024-03-22T07:04:24.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 11/26 2024-03-22T07:04:24.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 12/26 2024-03-22T07:04:24.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 13/26 2024-03-22T07:04:24.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-03-22T07:04:24.033 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-03-22T07:04:24.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-03-22T07:04:24.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-03-22T07:04:24.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-03-22T07:04:24.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-03-22T07:04:24.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-03-22T07:04:24.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-03-22T07:04:24.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-03-22T07:04:24.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 23/26 2024-03-22T07:04:24.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-03-22T07:04:24.034 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 25/26 2024-03-22T07:04:24.062 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph 2024-03-22T07:04:24.631 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:00:51 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:04:24.789 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:04:24.791 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================== 2024-03-22T07:04:24.791 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-03-22T07:04:24.791 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================== 2024-03-22T07:04:24.791 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-03-22T07:04:24.791 INFO:teuthology.orchestra.run.smithi063.stdout: ceph x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 6.5 k 2024-03-22T07:04:24.791 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-03-22T07:04:24.791 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mds x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.1 M 2024-03-22T07:04:24.792 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 1.4 M 2024-03-22T07:04:24.792 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 240 k 2024-03-22T07:04:24.792 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mon x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 3.7 M 2024-03-22T07:04:24.792 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-osd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 15 M 2024-03-22T07:04:24.792 INFO:teuthology.orchestra.run.smithi063.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-22T07:04:24.792 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-22T07:04:24.792 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-22T07:04:24.792 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-22T07:04:24.792 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-22T07:04:24.792 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-22T07:04:24.792 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-22T07:04:24.792 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-22T07:04:24.792 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-22T07:04:24.792 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-22T07:04:24.793 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-22T07:04:24.793 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-22T07:04:24.793 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-22T07:04:24.793 INFO:teuthology.orchestra.run.smithi063.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-22T07:04:24.793 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-22T07:04:24.793 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-22T07:04:24.793 INFO:teuthology.orchestra.run.smithi063.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-22T07:04:24.793 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-22T07:04:24.793 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-22T07:04:24.793 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-22T07:04:24.793 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-22T07:04:24.793 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-22T07:04:24.793 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-22T07:04:24.793 INFO:teuthology.orchestra.run.smithi063.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-22T07:04:24.794 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-22T07:04:24.794 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-22T07:04:24.794 INFO:teuthology.orchestra.run.smithi063.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-22T07:04:24.794 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:24.794 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-03-22T07:04:24.794 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================== 2024-03-22T07:04:24.794 INFO:teuthology.orchestra.run.smithi063.stdout:Install 33 Packages 2024-03-22T07:04:24.794 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:24.794 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 27 M 2024-03-22T07:04:24.795 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 100 M 2024-03-22T07:04:24.795 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-03-22T07:04:24.891 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-03-22T07:04:24.891 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:24.891 INFO:teuthology.orchestra.run.smithi069.stdout:Upgraded: 2024-03-22T07:04:24.891 INFO:teuthology.orchestra.run.smithi069.stdout: librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:24.891 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:24.891 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-03-22T07:04:24.891 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:24.891 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:24.891 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:24.892 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:24.892 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-22T07:04:24.892 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:24.892 INFO:teuthology.orchestra.run.smithi069.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-22T07:04:24.892 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-22T07:04:24.892 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-22T07:04:24.892 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:24.892 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-22T07:04:24.892 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:24.892 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-22T07:04:24.892 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-22T07:04:24.892 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:24.892 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:24.892 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:24.892 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-22T07:04:24.893 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:24.893 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:24.893 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:24.893 INFO:teuthology.orchestra.run.smithi069.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-22T07:04:24.893 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:24.893 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:04:25.237 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-test 2024-03-22T07:04:25.398 INFO:teuthology.orchestra.run.smithi063.stdout:(1/33): ceph-17.2.7-726.g9c4e0c34.el9.x86_64.rp 25 kB/s | 6.5 kB 00:00 2024-03-22T07:04:25.796 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:00:30 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:04:25.954 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:04:25.955 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:04:25.955 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-03-22T07:04:25.955 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:04:25.955 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-03-22T07:04:25.955 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 48 M 2024-03-22T07:04:25.955 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-03-22T07:04:25.955 INFO:teuthology.orchestra.run.smithi069.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-22T07:04:25.955 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 163 k 2024-03-22T07:04:25.955 INFO:teuthology.orchestra.run.smithi069.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-22T07:04:25.955 INFO:teuthology.orchestra.run.smithi069.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-22T07:04:25.955 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-22T07:04:25.955 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:25.956 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-03-22T07:04:25.956 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:04:25.956 INFO:teuthology.orchestra.run.smithi069.stdout:Install 6 Packages 2024-03-22T07:04:25.956 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:25.956 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 49 M 2024-03-22T07:04:25.956 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 196 M 2024-03-22T07:04:25.956 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-03-22T07:04:26.365 INFO:teuthology.orchestra.run.smithi063.stdout:(2/33): ceph-mgr-17.2.7-726.g9c4e0c34.el9.x86_6 1.1 MB/s | 1.4 MB 00:01 2024-03-22T07:04:26.504 INFO:teuthology.orchestra.run.smithi069.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 972 kB/s | 217 kB 00:00 2024-03-22T07:04:26.515 INFO:teuthology.orchestra.run.smithi063.stdout:(3/33): ceph-mds-17.2.7-726.g9c4e0c34.el9.x86_6 1.5 MB/s | 2.1 MB 00:01 2024-03-22T07:04:26.530 INFO:teuthology.orchestra.run.smithi069.stdout:(2/6): libcephsqlite-17.2.7-726.g9c4e0c34.el9.x 655 kB/s | 163 kB 00:00 2024-03-22T07:04:26.563 INFO:teuthology.orchestra.run.smithi069.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-03-22T07:04:26.580 INFO:teuthology.orchestra.run.smithi069.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-22T07:04:26.615 INFO:teuthology.orchestra.run.smithi063.stdout:(4/33): ceph-mgr-modules-core-17.2.7-726.g9c4e0 2.3 MB/s | 240 kB 00:00 2024-03-22T07:04:26.724 INFO:teuthology.orchestra.run.smithi063.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 144 kB 00:00 2024-03-22T07:04:26.738 INFO:teuthology.orchestra.run.smithi069.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-03-22T07:04:26.749 INFO:teuthology.orchestra.run.smithi063.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 1.2 MB/s | 32 kB 00:00 2024-03-22T07:04:26.775 INFO:teuthology.orchestra.run.smithi063.stdout:(7/33): python3-pytz-2021.1-5.el9.noarch.rpm 1.8 MB/s | 47 kB 00:00 2024-03-22T07:04:26.800 INFO:teuthology.orchestra.run.smithi063.stdout:(8/33): python3-toml-0.10.2-6.el9.noarch.rpm 1.5 MB/s | 39 kB 00:00 2024-03-22T07:04:26.850 INFO:teuthology.orchestra.run.smithi063.stdout:(9/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-22T07:04:26.926 INFO:teuthology.orchestra.run.smithi063.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 15 MB/s | 1.1 MB 00:00 2024-03-22T07:04:26.959 INFO:teuthology.orchestra.run.smithi063.stdout:(11/33): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2024-03-22T07:04:27.043 INFO:teuthology.orchestra.run.smithi063.stdout:(12/33): python3-pycparser-2.20-6.el9.noarch.rp 1.5 MB/s | 124 kB 00:00 2024-03-22T07:04:27.110 INFO:teuthology.orchestra.run.smithi063.stdout:(13/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.8 MB/s | 187 kB 00:00 2024-03-22T07:04:27.710 INFO:teuthology.orchestra.run.smithi063.stdout:(14/33): ceph-mon-17.2.7-726.g9c4e0c34.el9.x86_ 1.6 MB/s | 3.7 MB 00:02 2024-03-22T07:04:28.135 INFO:teuthology.orchestra.run.smithi063.stdout:(15/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 102 kB/s | 43 kB 00:00 2024-03-22T07:04:28.177 INFO:teuthology.orchestra.run.smithi063.stdout:(16/33): python3-requests-2.25.1-8.el9.noarch.r 117 kB/s | 125 kB 00:01 2024-03-22T07:04:28.244 INFO:teuthology.orchestra.run.smithi063.stdout:(17/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-03-22T07:04:28.320 INFO:teuthology.orchestra.run.smithi063.stdout:(18/33): python3-cherrypy-18.6.1-2.el9.noarch.r 2.5 MB/s | 358 kB 00:00 2024-03-22T07:04:28.345 INFO:teuthology.orchestra.run.smithi063.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 107 kB/s | 11 kB 00:00 2024-03-22T07:04:28.387 INFO:teuthology.orchestra.run.smithi063.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 265 kB/s | 18 kB 00:00 2024-03-22T07:04:28.420 INFO:teuthology.orchestra.run.smithi063.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 309 kB/s | 23 kB 00:00 2024-03-22T07:04:28.445 INFO:teuthology.orchestra.run.smithi063.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 333 kB/s | 19 kB 00:00 2024-03-22T07:04:28.479 INFO:teuthology.orchestra.run.smithi063.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 335 kB/s | 20 kB 00:00 2024-03-22T07:04:28.504 INFO:teuthology.orchestra.run.smithi063.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 789 kB/s | 46 kB 00:00 2024-03-22T07:04:28.546 INFO:teuthology.orchestra.run.smithi063.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.2 MB/s | 79 kB 00:00 2024-03-22T07:04:28.580 INFO:teuthology.orchestra.run.smithi063.stdout:(26/33): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-03-22T07:04:28.638 INFO:teuthology.orchestra.run.smithi063.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 179 kB/s | 16 kB 00:00 2024-03-22T07:04:28.672 INFO:teuthology.orchestra.run.smithi063.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 980 kB/s | 90 kB 00:00 2024-03-22T07:04:28.706 INFO:teuthology.orchestra.run.smithi063.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 534 kB/s | 36 kB 00:00 2024-03-22T07:04:28.739 INFO:teuthology.orchestra.run.smithi063.stdout:(30/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-03-22T07:04:28.782 INFO:teuthology.orchestra.run.smithi063.stdout:(31/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.5 MB/s | 427 kB 00:00 2024-03-22T07:04:28.806 INFO:teuthology.orchestra.run.smithi063.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 299 kB/s | 20 kB 00:00 2024-03-22T07:04:28.905 INFO:teuthology.orchestra.run.smithi069.stdout:(6/6): ceph-test-17.2.7-726.g9c4e0c34.el9.x86_6 18 MB/s | 48 MB 00:02 2024-03-22T07:04:28.909 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:04:28.909 INFO:teuthology.orchestra.run.smithi069.stdout:Total 17 MB/s | 49 MB 00:02 2024-03-22T07:04:28.923 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-03-22T07:04:28.943 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-03-22T07:04:28.943 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-03-22T07:04:29.336 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-03-22T07:04:29.336 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-03-22T07:04:29.903 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-03-22T07:04:29.981 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-22T07:04:30.015 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-22T07:04:30.046 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-22T07:04:30.089 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-22T07:04:30.117 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/6 2024-03-22T07:04:30.665 INFO:teuthology.orchestra.run.smithi063.stdout:(33/33): ceph-osd-17.2.7-726.g9c4e0c34.el9.x86_ 3.6 MB/s | 15 MB 00:04 2024-03-22T07:04:30.670 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:04:30.670 INFO:teuthology.orchestra.run.smithi063.stdout:Total 4.6 MB/s | 27 MB 00:05 2024-03-22T07:04:30.684 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-03-22T07:04:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-03-22T07:04:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-03-22T07:04:31.077 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-03-22T07:04:31.077 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-03-22T07:04:32.436 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-03-22T07:04:32.501 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-22T07:04:32.534 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-22T07:04:32.567 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-22T07:04:32.663 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-22T07:04:32.740 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-22T07:04:32.787 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-22T07:04:32.862 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-22T07:04:33.029 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-22T07:04:33.067 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-22T07:04:33.117 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-22T07:04:33.187 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-22T07:04:33.384 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-22T07:04:33.437 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-22T07:04:33.476 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-22T07:04:33.511 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-22T07:04:33.540 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-22T07:04:33.620 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-22T07:04:33.650 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-22T07:04:33.684 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-22T07:04:33.716 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-22T07:04:33.759 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-22T07:04:33.793 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-22T07:04:33.888 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-22T07:04:34.041 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-22T07:04:34.131 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-22T07:04:34.272 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-22T07:04:34.403 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-22T07:04:34.549 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/6 2024-03-22T07:04:34.562 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.n 28/33 2024-03-22T07:04:34.595 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 29/33 2024-03-22T07:04:34.619 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 29/33 2024-03-22T07:04:34.619 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T07:04:34.619 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-22T07:04:34.619 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-22T07:04:34.619 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-22T07:04:34.620 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:34.632 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/6 2024-03-22T07:04:36.029 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 30/33 2024-03-22T07:04:36.055 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 30/33 2024-03-22T07:04:36.055 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T07:04:36.055 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-22T07:04:36.055 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-22T07:04:36.055 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-22T07:04:36.055 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:36.455 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 31/33 2024-03-22T07:04:36.477 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 31/33 2024-03-22T07:04:36.477 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T07:04:36.477 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-22T07:04:36.477 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-22T07:04:36.478 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-22T07:04:36.478 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:36.680 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 32/33 2024-03-22T07:04:36.703 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 32/33 2024-03-22T07:04:36.703 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T07:04:36.703 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-22T07:04:36.703 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-22T07:04:36.703 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-22T07:04:36.703 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:36.763 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 33/33 2024-03-22T07:04:36.842 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 6/6 2024-03-22T07:04:36.842 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/6 2024-03-22T07:04:36.842 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/6 2024-03-22T07:04:36.842 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-22T07:04:36.842 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-22T07:04:36.842 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-22T07:04:37.190 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-22T07:04:37.190 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:37.190 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-03-22T07:04:37.190 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:37.191 INFO:teuthology.orchestra.run.smithi069.stdout: jq-1.6-16.el9.x86_64 2024-03-22T07:04:37.191 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:37.191 INFO:teuthology.orchestra.run.smithi069.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-22T07:04:37.191 INFO:teuthology.orchestra.run.smithi069.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-22T07:04:37.191 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-22T07:04:37.191 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:37.191 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:04:37.480 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph 2024-03-22T07:04:38.037 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:00:43 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:04:38.187 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:04:38.189 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================== 2024-03-22T07:04:38.189 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-03-22T07:04:38.190 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================== 2024-03-22T07:04:38.190 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-03-22T07:04:38.190 INFO:teuthology.orchestra.run.smithi069.stdout: ceph x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 6.5 k 2024-03-22T07:04:38.190 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-03-22T07:04:38.190 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.1 M 2024-03-22T07:04:38.190 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 1.4 M 2024-03-22T07:04:38.190 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 240 k 2024-03-22T07:04:38.190 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 3.7 M 2024-03-22T07:04:38.190 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 15 M 2024-03-22T07:04:38.190 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-22T07:04:38.190 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-22T07:04:38.190 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-22T07:04:38.191 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-22T07:04:38.191 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-22T07:04:38.191 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-22T07:04:38.191 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-22T07:04:38.191 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-22T07:04:38.191 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-22T07:04:38.191 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-22T07:04:38.191 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-22T07:04:38.191 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-22T07:04:38.191 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-22T07:04:38.191 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-22T07:04:38.191 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-22T07:04:38.191 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-22T07:04:38.191 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-22T07:04:38.191 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-22T07:04:38.192 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-22T07:04:38.192 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-22T07:04:38.192 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-22T07:04:38.192 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-22T07:04:38.192 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-22T07:04:38.192 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-22T07:04:38.192 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-22T07:04:38.192 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-22T07:04:38.192 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-22T07:04:38.192 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:38.192 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-03-22T07:04:38.192 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================== 2024-03-22T07:04:38.192 INFO:teuthology.orchestra.run.smithi069.stdout:Install 33 Packages 2024-03-22T07:04:38.192 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:38.193 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 27 M 2024-03-22T07:04:38.193 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 100 M 2024-03-22T07:04:38.193 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-03-22T07:04:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 33/33 2024-03-22T07:04:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/33 2024-03-22T07:04:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/33 2024-03-22T07:04:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/33 2024-03-22T07:04:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/33 2024-03-22T07:04:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/33 2024-03-22T07:04:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.n 6/33 2024-03-22T07:04:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-22T07:04:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-22T07:04:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-22T07:04:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-22T07:04:38.345 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-22T07:04:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-22T07:04:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-22T07:04:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-22T07:04:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-22T07:04:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-22T07:04:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-22T07:04:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-22T07:04:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-22T07:04:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-22T07:04:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-22T07:04:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-22T07:04:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-22T07:04:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-22T07:04:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-22T07:04:38.346 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-22T07:04:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-22T07:04:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-22T07:04:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-22T07:04:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-22T07:04:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-22T07:04:38.347 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-22T07:04:38.748 INFO:teuthology.orchestra.run.smithi069.stdout:(1/33): ceph-17.2.7-726.g9c4e0c34.el9.x86_64.rp 29 kB/s | 6.5 kB 00:00 2024-03-22T07:04:38.932 INFO:teuthology.orchestra.run.smithi069.stdout:(2/33): ceph-mgr-17.2.7-726.g9c4e0c34.el9.x86_6 3.4 MB/s | 1.4 MB 00:00 2024-03-22T07:04:38.991 INFO:teuthology.orchestra.run.smithi069.stdout:(3/33): ceph-mds-17.2.7-726.g9c4e0c34.el9.x86_6 4.4 MB/s | 2.1 MB 00:00 2024-03-22T07:04:39.041 INFO:teuthology.orchestra.run.smithi069.stdout:(4/33): ceph-mgr-modules-core-17.2.7-726.g9c4e0 4.7 MB/s | 240 kB 00:00 2024-03-22T07:04:39.150 INFO:teuthology.orchestra.run.smithi069.stdout:(5/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 144 kB 00:00 2024-03-22T07:04:39.183 INFO:teuthology.orchestra.run.smithi069.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 950 kB/s | 32 kB 00:00 2024-03-22T07:04:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-22T07:04:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:39.206 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-03-22T07:04:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T07:04:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-22T07:04:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-22T07:04:39.206 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-22T07:04:39.207 INFO:teuthology.orchestra.run.smithi063.stdout: python3-ply-3.11-14.el9.noarch 2024-03-22T07:04:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-22T07:04:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-22T07:04:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-22T07:04:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-22T07:04:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-22T07:04:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-22T07:04:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-22T07:04:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-22T07:04:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-22T07:04:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-22T07:04:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-22T07:04:39.208 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:39.208 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:04:39.217 INFO:teuthology.orchestra.run.smithi069.stdout:(7/33): python3-pytz-2021.1-5.el9.noarch.rpm 1.4 MB/s | 47 kB 00:00 2024-03-22T07:04:39.259 INFO:teuthology.orchestra.run.smithi069.stdout:(8/33): python3-toml-0.10.2-6.el9.noarch.rpm 928 kB/s | 39 kB 00:00 2024-03-22T07:04:39.309 INFO:teuthology.orchestra.run.smithi069.stdout:(9/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-22T07:04:39.376 INFO:teuthology.orchestra.run.smithi069.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 17 MB/s | 1.1 MB 00:00 2024-03-22T07:04:39.409 INFO:teuthology.orchestra.run.smithi069.stdout:(11/33): python3-ply-3.11-14.el9.noarch.rpm 3.0 MB/s | 103 kB 00:00 2024-03-22T07:04:39.452 INFO:teuthology.orchestra.run.smithi069.stdout:(12/33): python3-pycparser-2.20-6.el9.noarch.rp 2.9 MB/s | 124 kB 00:00 2024-03-22T07:04:39.485 INFO:teuthology.orchestra.run.smithi069.stdout:(13/33): python3-urllib3-1.26.5-5.el9.noarch.rp 5.5 MB/s | 187 kB 00:00 2024-03-22T07:04:39.569 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-base 2024-03-22T07:04:39.594 INFO:teuthology.orchestra.run.smithi069.stdout:(14/33): ceph-mon-17.2.7-726.g9c4e0c34.el9.x86_ 4.4 MB/s | 3.7 MB 00:00 2024-03-22T07:04:39.878 INFO:teuthology.orchestra.run.smithi069.stdout:(15/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 153 kB/s | 43 kB 00:00 2024-03-22T07:04:40.028 INFO:teuthology.orchestra.run.smithi069.stdout:(16/33): ceph-osd-17.2.7-726.g9c4e0c34.el9.x86_ 14 MB/s | 15 MB 00:01 2024-03-22T07:04:40.128 INFO:teuthology.orchestra.run.smithi069.stdout:(17/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 687 kB/s | 172 kB 00:00 2024-03-22T07:04:40.146 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:07 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:04:40.244 INFO:teuthology.orchestra.run.smithi063.stdout:Package ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T07:04:40.286 INFO:teuthology.orchestra.run.smithi069.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 68 kB/s | 11 kB 00:00 2024-03-22T07:04:40.298 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:04:40.299 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-03-22T07:04:40.299 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:04:40.370 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install cephadm 2024-03-22T07:04:40.470 INFO:teuthology.orchestra.run.smithi069.stdout:(19/33): python3-cherrypy-18.6.1-2.el9.noarch.r 811 kB/s | 358 kB 00:00 2024-03-22T07:04:40.487 INFO:teuthology.orchestra.run.smithi069.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 88 kB/s | 18 kB 00:00 2024-03-22T07:04:40.537 INFO:teuthology.orchestra.run.smithi069.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 346 kB/s | 23 kB 00:00 2024-03-22T07:04:40.554 INFO:teuthology.orchestra.run.smithi069.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 291 kB/s | 19 kB 00:00 2024-03-22T07:04:40.604 INFO:teuthology.orchestra.run.smithi069.stdout:(23/33): python3-requests-2.25.1-8.el9.noarch.r 112 kB/s | 125 kB 00:01 2024-03-22T07:04:40.621 INFO:teuthology.orchestra.run.smithi069.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 235 kB/s | 20 kB 00:00 2024-03-22T07:04:40.771 INFO:teuthology.orchestra.run.smithi069.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 213 kB/s | 46 kB 00:00 2024-03-22T07:04:40.797 INFO:teuthology.orchestra.run.smithi069.stdout:(26/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-03-22T07:04:40.856 INFO:teuthology.orchestra.run.smithi069.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 197 kB/s | 16 kB 00:00 2024-03-22T07:04:40.906 INFO:teuthology.orchestra.run.smithi069.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 828 kB/s | 90 kB 00:00 2024-03-22T07:04:40.939 INFO:teuthology.orchestra.run.smithi069.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 427 kB/s | 36 kB 00:00 2024-03-22T07:04:40.943 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:07 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:04:40.964 INFO:teuthology.orchestra.run.smithi069.stdout:(30/33): python3-more-itertools-8.12.0-2.el9.no 219 kB/s | 79 kB 00:00 2024-03-22T07:04:41.041 INFO:teuthology.orchestra.run.smithi069.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-03-22T07:04:41.073 INFO:teuthology.orchestra.run.smithi069.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 184 kB/s | 20 kB 00:00 2024-03-22T07:04:41.096 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:04:41.096 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:04:41.096 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-03-22T07:04:41.097 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:04:41.097 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-03-22T07:04:41.097 INFO:teuthology.orchestra.run.smithi063.stdout: cephadm noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 82 k 2024-03-22T07:04:41.097 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:41.097 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-03-22T07:04:41.097 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:04:41.097 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-03-22T07:04:41.097 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:41.098 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 82 k 2024-03-22T07:04:41.098 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 372 k 2024-03-22T07:04:41.098 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-03-22T07:04:41.157 INFO:teuthology.orchestra.run.smithi069.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.9 MB/s | 427 kB 00:00 2024-03-22T07:04:41.164 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:04:41.164 INFO:teuthology.orchestra.run.smithi069.stdout:Total 9.0 MB/s | 27 MB 00:02 2024-03-22T07:04:41.178 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-03-22T07:04:41.221 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-03-22T07:04:41.222 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-03-22T07:04:41.378 INFO:teuthology.orchestra.run.smithi063.stdout:cephadm-17.2.7-726.g9c4e0c34.el9.noarch.rpm 292 kB/s | 82 kB 00:00 2024-03-22T07:04:41.379 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:04:41.379 INFO:teuthology.orchestra.run.smithi063.stdout:Total 291 kB/s | 82 kB 00:00 2024-03-22T07:04:41.379 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-03-22T07:04:41.384 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-03-22T07:04:41.384 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-03-22T07:04:41.386 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-03-22T07:04:41.387 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-03-22T07:04:41.463 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-03-22T07:04:41.565 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-03-22T07:04:41.565 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-03-22T07:04:41.850 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T07:04:41.951 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T07:04:42.311 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T07:04:42.598 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T07:04:42.598 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:42.598 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-03-22T07:04:42.598 INFO:teuthology.orchestra.run.smithi063.stdout: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T07:04:42.598 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:42.598 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:04:42.792 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-03-22T07:04:42.814 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-immutable-object-cache 2024-03-22T07:04:42.843 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-22T07:04:42.875 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-22T07:04:42.908 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-22T07:04:43.000 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-22T07:04:43.090 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-22T07:04:43.127 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-22T07:04:43.198 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-22T07:04:43.253 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-22T07:04:43.291 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-22T07:04:43.339 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-22T07:04:43.388 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:10 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:04:43.410 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-22T07:04:43.543 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:04:43.544 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:04:43.544 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repo Size 2024-03-22T07:04:43.544 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:04:43.544 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-03-22T07:04:43.544 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 137 k 2024-03-22T07:04:43.544 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:43.544 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-03-22T07:04:43.545 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:04:43.545 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-03-22T07:04:43.545 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:43.545 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 137 k 2024-03-22T07:04:43.545 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 379 k 2024-03-22T07:04:43.545 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-03-22T07:04:43.596 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-22T07:04:43.647 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-22T07:04:43.678 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-22T07:04:43.713 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-22T07:04:43.743 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-22T07:04:43.819 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-22T07:04:43.861 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-22T07:04:43.894 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-22T07:04:43.902 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-immutable-object-cache-17.2.7-726.g9c4e0c3 385 kB/s | 137 kB 00:00 2024-03-22T07:04:43.903 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:04:43.903 INFO:teuthology.orchestra.run.smithi063.stdout:Total 383 kB/s | 137 kB 00:00 2024-03-22T07:04:43.903 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-03-22T07:04:43.911 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-03-22T07:04:43.911 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-03-22T07:04:43.952 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-22T07:04:43.965 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-03-22T07:04:43.965 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-03-22T07:04:43.995 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-22T07:04:44.029 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-22T07:04:44.106 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-03-22T07:04:44.118 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-22T07:04:44.152 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-22T07:04:44.165 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-22T07:04:44.189 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-22T07:04:44.189 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T07:04:44.189 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-22T07:04:44.190 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:44.224 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-22T07:04:44.360 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-22T07:04:44.476 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-22T07:04:44.626 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.n 28/33 2024-03-22T07:04:44.656 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 29/33 2024-03-22T07:04:44.680 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 29/33 2024-03-22T07:04:44.680 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T07:04:44.680 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-22T07:04:44.681 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-22T07:04:44.681 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-22T07:04:44.681 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:45.081 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-22T07:04:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:45.082 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-03-22T07:04:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:45.082 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:45.082 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:04:45.342 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr 2024-03-22T07:04:45.917 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:12 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:04:46.016 INFO:teuthology.orchestra.run.smithi063.stdout:Package ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T07:04:46.067 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 30/33 2024-03-22T07:04:46.070 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:04:46.071 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-03-22T07:04:46.071 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:04:46.095 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 30/33 2024-03-22T07:04:46.095 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T07:04:46.095 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-22T07:04:46.095 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-22T07:04:46.095 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-22T07:04:46.095 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:46.149 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr-dashboard 2024-03-22T07:04:46.484 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 31/33 2024-03-22T07:04:46.507 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 31/33 2024-03-22T07:04:46.507 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T07:04:46.508 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-22T07:04:46.508 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-22T07:04:46.508 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-22T07:04:46.508 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:46.710 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 32/33 2024-03-22T07:04:46.733 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 32/33 2024-03-22T07:04:46.733 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T07:04:46.733 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-22T07:04:46.733 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-22T07:04:46.733 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-22T07:04:46.734 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:46.740 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:13 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:04:46.800 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 33/33 2024-03-22T07:04:46.897 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:04:46.898 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:04:46.898 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-03-22T07:04:46.898 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:04:46.898 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-03-22T07:04:46.898 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 1.7 M 2024-03-22T07:04:46.899 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-03-22T07:04:46.899 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-grafana-dashboards noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 23 k 2024-03-22T07:04:46.899 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-prometheus-alerts noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 15 k 2024-03-22T07:04:46.899 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-22T07:04:46.899 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-22T07:04:46.899 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-22T07:04:46.899 INFO:teuthology.orchestra.run.smithi063.stdout:Installing weak dependencies: 2024-03-22T07:04:46.899 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-22T07:04:46.899 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:46.899 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-03-22T07:04:46.899 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:04:46.900 INFO:teuthology.orchestra.run.smithi063.stdout:Install 7 Packages 2024-03-22T07:04:46.900 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:46.900 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 2.0 M 2024-03-22T07:04:46.900 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 8.1 M 2024-03-22T07:04:46.900 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-03-22T07:04:47.186 INFO:teuthology.orchestra.run.smithi063.stdout:(1/7): ceph-grafana-dashboards-17.2.7-726.g9c4e 120 kB/s | 23 kB 00:00 2024-03-22T07:04:47.210 INFO:teuthology.orchestra.run.smithi063.stdout:(2/7): ceph-prometheus-alerts-17.2.7-726.g9c4e0 67 kB/s | 15 kB 00:00 2024-03-22T07:04:47.394 INFO:teuthology.orchestra.run.smithi063.stdout:(3/7): ceph-mgr-dashboard-17.2.7-726.g9c4e0c34. 4.2 MB/s | 1.7 MB 00:00 2024-03-22T07:04:47.419 INFO:teuthology.orchestra.run.smithi063.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 39 kB/s | 9.0 kB 00:00 2024-03-22T07:04:47.519 INFO:teuthology.orchestra.run.smithi063.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 132 kB/s | 41 kB 00:00 2024-03-22T07:04:47.545 INFO:teuthology.orchestra.run.smithi063.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 205 kB/s | 31 kB 00:00 2024-03-22T07:04:47.787 INFO:teuthology.orchestra.run.smithi063.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 513 kB/s | 188 kB 00:00 2024-03-22T07:04:47.790 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:04:47.791 INFO:teuthology.orchestra.run.smithi063.stdout:Total 2.2 MB/s | 2.0 MB 00:00 2024-03-22T07:04:47.791 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-03-22T07:04:47.799 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-03-22T07:04:47.799 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-03-22T07:04:47.861 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-03-22T07:04:47.862 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-03-22T07:04:48.167 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-03-22T07:04:48.238 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-22T07:04:48.272 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-22T07:04:48.319 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-22T07:04:48.350 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-22T07:04:48.379 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.no 5/7 2024-03-22T07:04:48.500 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 33/33 2024-03-22T07:04:48.500 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/33 2024-03-22T07:04:48.500 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/33 2024-03-22T07:04:48.500 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 3/33 2024-03-22T07:04:48.500 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 4/33 2024-03-22T07:04:48.500 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 5/33 2024-03-22T07:04:48.500 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.n 6/33 2024-03-22T07:04:48.500 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-22T07:04:48.500 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-22T07:04:48.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-22T07:04:48.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-22T07:04:48.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-22T07:04:48.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-22T07:04:48.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-22T07:04:48.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-22T07:04:48.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-22T07:04:48.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-22T07:04:48.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-22T07:04:48.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-22T07:04:48.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-22T07:04:48.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-22T07:04:48.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-22T07:04:48.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-22T07:04:48.502 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-22T07:04:48.502 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-22T07:04:48.502 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-22T07:04:48.502 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-22T07:04:48.502 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-22T07:04:48.502 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-22T07:04:48.502 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-22T07:04:48.502 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-22T07:04:48.502 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-22T07:04:48.502 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-22T07:04:48.618 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.n 6/7 2024-03-22T07:04:48.657 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 7/7 2024-03-22T07:04:49.198 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 7/7 2024-03-22T07:04:49.198 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.n 1/7 2024-03-22T07:04:49.198 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 2/7 2024-03-22T07:04:49.198 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.no 3/7 2024-03-22T07:04:49.198 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-22T07:04:49.198 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-22T07:04:49.198 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-22T07:04:49.343 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-22T07:04:49.343 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:49.343 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-03-22T07:04:49.343 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:49.343 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:49.343 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:49.343 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T07:04:49.344 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:49.344 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:49.344 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-22T07:04:49.344 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-22T07:04:49.344 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-22T07:04:49.344 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-22T07:04:49.344 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-22T07:04:49.344 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-22T07:04:49.344 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-22T07:04:49.344 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-22T07:04:49.344 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-22T07:04:49.344 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-22T07:04:49.344 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-22T07:04:49.345 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-22T07:04:49.345 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-22T07:04:49.345 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-22T07:04:49.345 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-22T07:04:49.345 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply-3.11-14.el9.noarch 2024-03-22T07:04:49.345 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-22T07:04:49.345 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-22T07:04:49.345 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-22T07:04:49.345 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-22T07:04:49.345 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-22T07:04:49.345 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-22T07:04:49.345 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-22T07:04:49.345 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-22T07:04:49.345 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-22T07:04:49.345 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-22T07:04:49.346 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-22T07:04:49.346 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:49.346 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:04:49.498 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-22T07:04:49.499 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:49.499 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-03-22T07:04:49.499 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T07:04:49.499 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T07:04:49.499 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T07:04:49.499 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-22T07:04:49.499 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-22T07:04:49.499 INFO:teuthology.orchestra.run.smithi063.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-22T07:04:49.499 INFO:teuthology.orchestra.run.smithi063.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-22T07:04:49.499 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:49.499 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:04:49.656 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-base 2024-03-22T07:04:49.705 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-22T07:04:50.223 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:00:55 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:04:50.277 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:17 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:04:50.318 INFO:teuthology.orchestra.run.smithi069.stdout:Package ceph-base-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T07:04:50.369 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:04:50.370 INFO:teuthology.orchestra.run.smithi069.stdout:Nothing to do. 2024-03-22T07:04:50.370 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:04:50.431 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install cephadm 2024-03-22T07:04:50.433 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:04:50.434 INFO:teuthology.orchestra.run.smithi063.stdout:=========================================================================================== 2024-03-22T07:04:50.434 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-03-22T07:04:50.434 INFO:teuthology.orchestra.run.smithi063.stdout:=========================================================================================== 2024-03-22T07:04:50.434 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-03-22T07:04:50.434 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 7.4 M 2024-03-22T07:04:50.435 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-03-22T07:04:50.435 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-22T07:04:50.435 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-22T07:04:50.435 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-22T07:04:50.435 INFO:teuthology.orchestra.run.smithi063.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-22T07:04:50.435 INFO:teuthology.orchestra.run.smithi063.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-22T07:04:50.435 INFO:teuthology.orchestra.run.smithi063.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-22T07:04:50.435 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-22T07:04:50.435 INFO:teuthology.orchestra.run.smithi063.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-22T07:04:50.435 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-22T07:04:50.435 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-22T07:04:50.435 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-22T07:04:50.435 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:50.435 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-03-22T07:04:50.435 INFO:teuthology.orchestra.run.smithi063.stdout:=========================================================================================== 2024-03-22T07:04:50.436 INFO:teuthology.orchestra.run.smithi063.stdout:Install 12 Packages 2024-03-22T07:04:50.436 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:04:50.436 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 37 M 2024-03-22T07:04:50.436 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 209 M 2024-03-22T07:04:50.436 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-03-22T07:04:50.776 INFO:teuthology.orchestra.run.smithi063.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 223 kB/s | 32 kB 00:00 2024-03-22T07:04:50.819 INFO:teuthology.orchestra.run.smithi063.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 410 kB/s | 17 kB 00:00 2024-03-22T07:04:50.869 INFO:teuthology.orchestra.run.smithi063.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 691 kB/s | 34 kB 00:00 2024-03-22T07:04:50.952 INFO:teuthology.orchestra.run.smithi063.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.4 MB/s | 3.0 MB 00:00 2024-03-22T07:04:50.990 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:00:55 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:04:51.137 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:04:51.138 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:04:51.138 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-03-22T07:04:51.138 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:04:51.138 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-03-22T07:04:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 82 k 2024-03-22T07:04:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:51.139 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-03-22T07:04:51.139 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:04:51.139 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package 2024-03-22T07:04:51.139 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:51.139 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 82 k 2024-03-22T07:04:51.139 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 372 k 2024-03-22T07:04:51.139 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-03-22T07:04:51.261 INFO:teuthology.orchestra.run.smithi063.stdout:(5/12): ceph-mgr-diskprediction-local-17.2.7-72 12 MB/s | 7.4 MB 00:00 2024-03-22T07:04:51.336 INFO:teuthology.orchestra.run.smithi063.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 10 MB/s | 4.7 MB 00:00 2024-03-22T07:04:51.395 INFO:teuthology.orchestra.run.smithi063.stdout:(7/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 4.8 MB 00:00 2024-03-22T07:04:51.420 INFO:teuthology.orchestra.run.smithi063.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.3 MB/s | 217 kB 00:00 2024-03-22T07:04:51.533 INFO:teuthology.orchestra.run.smithi069.stdout:cephadm-17.2.7-726.g9c4e0c34.el9.noarch.rpm 208 kB/s | 82 kB 00:00 2024-03-22T07:04:51.534 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:04:51.534 INFO:teuthology.orchestra.run.smithi069.stdout:Total 207 kB/s | 82 kB 00:00 2024-03-22T07:04:51.534 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-03-22T07:04:51.539 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-03-22T07:04:51.540 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-03-22T07:04:51.541 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-03-22T07:04:51.542 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-03-22T07:04:51.623 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-03-22T07:04:51.937 INFO:teuthology.orchestra.run.smithi063.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-03-22T07:04:52.004 INFO:teuthology.orchestra.run.smithi063.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 335 kB/s | 195 kB 00:00 2024-03-22T07:04:52.047 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T07:04:52.122 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T07:04:52.458 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T07:04:52.629 INFO:teuthology.orchestra.run.smithi063.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 355 kB/s | 245 kB 00:00 2024-03-22T07:04:52.654 INFO:teuthology.orchestra.run.smithi063.stdout:(12/12): libgfortran-11.4.1-3.el9.x86_64.rpm 638 kB/s | 803 kB 00:01 2024-03-22T07:04:52.658 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:04:52.658 INFO:teuthology.orchestra.run.smithi063.stdout:Total 17 MB/s | 37 MB 00:02 2024-03-22T07:04:52.699 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-03-22T07:04:52.725 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-03-22T07:04:52.725 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-03-22T07:04:52.760 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/1 2024-03-22T07:04:52.761 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:52.761 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-03-22T07:04:52.761 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T07:04:52.761 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:52.761 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:04:52.980 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-immutable-object-cache 2024-03-22T07:04:53.169 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-03-22T07:04:53.169 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-03-22T07:04:53.561 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:00:58 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:04:53.710 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:04:53.710 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:04:53.710 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repo Size 2024-03-22T07:04:53.710 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:04:53.711 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-03-22T07:04:53.711 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 137 k 2024-03-22T07:04:53.711 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:53.711 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-03-22T07:04:53.711 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:04:53.711 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package 2024-03-22T07:04:53.711 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:53.712 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 137 k 2024-03-22T07:04:53.712 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 379 k 2024-03-22T07:04:53.712 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-03-22T07:04:53.889 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-03-22T07:04:53.914 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-immutable-object-cache-17.2.7-726.g9c4e0c3 678 kB/s | 137 kB 00:00 2024-03-22T07:04:53.914 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:04:53.915 INFO:teuthology.orchestra.run.smithi069.stdout:Total 674 kB/s | 137 kB 00:00 2024-03-22T07:04:53.915 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-03-22T07:04:53.923 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-03-22T07:04:53.923 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-03-22T07:04:53.974 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-03-22T07:04:53.975 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-03-22T07:04:53.977 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-22T07:04:54.009 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-22T07:04:54.106 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-22T07:04:54.122 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-03-22T07:04:54.137 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-22T07:04:54.182 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-22T07:04:54.206 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-22T07:04:54.206 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T07:04:54.206 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-22T07:04:54.206 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:54.321 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-22T07:04:54.360 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-22T07:04:54.439 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-22T07:04:55.131 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-22T07:04:55.162 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.e 1/1 2024-03-22T07:04:55.163 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:55.163 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-03-22T07:04:55.163 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:04:55.163 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:55.163 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:04:55.235 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-22T07:04:55.418 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-mgr 2024-03-22T07:04:55.998 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:00 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:04:56.094 INFO:teuthology.orchestra.run.smithi069.stdout:Package ceph-mgr-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T07:04:56.146 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:04:56.147 INFO:teuthology.orchestra.run.smithi069.stdout:Nothing to do. 2024-03-22T07:04:56.147 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:04:56.222 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-mgr-dashboard 2024-03-22T07:04:56.568 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-22T07:04:56.783 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:01 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:04:56.933 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:04:56.934 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:04:56.934 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-03-22T07:04:56.934 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:04:56.934 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-03-22T07:04:56.934 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 1.7 M 2024-03-22T07:04:56.934 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-03-22T07:04:56.934 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 23 k 2024-03-22T07:04:56.934 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 15 k 2024-03-22T07:04:56.934 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-22T07:04:56.935 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-22T07:04:56.935 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-22T07:04:56.935 INFO:teuthology.orchestra.run.smithi069.stdout:Installing weak dependencies: 2024-03-22T07:04:56.935 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-22T07:04:56.935 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:56.935 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-03-22T07:04:56.935 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:04:56.935 INFO:teuthology.orchestra.run.smithi069.stdout:Install 7 Packages 2024-03-22T07:04:56.935 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:04:56.936 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 2.0 M 2024-03-22T07:04:56.936 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 8.1 M 2024-03-22T07:04:56.936 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-03-22T07:04:57.560 INFO:teuthology.orchestra.run.smithi069.stdout:(1/7): ceph-grafana-dashboards-17.2.7-726.g9c4e 86 kB/s | 23 kB 00:00 2024-03-22T07:04:57.585 INFO:teuthology.orchestra.run.smithi069.stdout:(2/7): ceph-prometheus-alerts-17.2.7-726.g9c4e0 49 kB/s | 15 kB 00:00 2024-03-22T07:04:57.661 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-22T07:04:57.692 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 12/12 2024-03-22T07:04:57.752 INFO:teuthology.orchestra.run.smithi069.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 47 kB/s | 9.0 kB 00:00 2024-03-22T07:04:57.794 INFO:teuthology.orchestra.run.smithi069.stdout:(4/7): ceph-mgr-dashboard-17.2.7-726.g9c4e0c34. 3.3 MB/s | 1.7 MB 00:00 2024-03-22T07:04:57.903 INFO:teuthology.orchestra.run.smithi069.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 129 kB/s | 41 kB 00:00 2024-03-22T07:04:57.929 INFO:teuthology.orchestra.run.smithi069.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 184 kB/s | 31 kB 00:00 2024-03-22T07:04:58.295 INFO:teuthology.orchestra.run.smithi069.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 377 kB/s | 188 kB 00:00 2024-03-22T07:04:58.297 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:04:58.297 INFO:teuthology.orchestra.run.smithi069.stdout:Total 1.5 MB/s | 2.0 MB 00:01 2024-03-22T07:04:58.298 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-03-22T07:04:58.306 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-03-22T07:04:58.306 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-03-22T07:04:58.375 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-03-22T07:04:58.375 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-03-22T07:04:58.668 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-03-22T07:04:58.721 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-22T07:04:58.755 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-22T07:04:58.801 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-22T07:04:58.842 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-22T07:04:58.870 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.no 5/7 2024-03-22T07:04:59.115 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.n 6/7 2024-03-22T07:04:59.148 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 7/7 2024-03-22T07:04:59.701 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 7/7 2024-03-22T07:04:59.701 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.n 1/7 2024-03-22T07:04:59.701 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 2/7 2024-03-22T07:04:59.702 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.no 3/7 2024-03-22T07:04:59.702 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-22T07:04:59.702 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-22T07:04:59.702 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-22T07:05:00.005 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-22T07:05:00.005 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:00.005 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-03-22T07:05:00.005 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T07:05:00.005 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T07:05:00.005 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T07:05:00.005 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-22T07:05:00.006 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-22T07:05:00.006 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-22T07:05:00.006 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-22T07:05:00.006 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:00.006 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:05:00.068 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 12/12 2024-03-22T07:05:00.068 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 1/12 2024-03-22T07:05:00.069 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-22T07:05:00.069 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-22T07:05:00.069 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-22T07:05:00.069 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-22T07:05:00.069 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-22T07:05:00.069 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-22T07:05:00.069 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-22T07:05:00.069 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-22T07:05:00.069 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-22T07:05:00.069 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-22T07:05:00.238 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-22T07:05:00.484 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-03-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi063.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi063.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi063.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi063.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi063.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi063.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi063.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:00.485 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:05:00.794 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:05 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:05:00.845 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr-rook 2024-03-22T07:05:00.943 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:05:00.944 INFO:teuthology.orchestra.run.smithi069.stdout:=========================================================================================== 2024-03-22T07:05:00.944 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-03-22T07:05:00.944 INFO:teuthology.orchestra.run.smithi069.stdout:=========================================================================================== 2024-03-22T07:05:00.945 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-03-22T07:05:00.945 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 7.4 M 2024-03-22T07:05:00.945 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-03-22T07:05:00.945 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-22T07:05:00.945 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-22T07:05:00.945 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-22T07:05:00.945 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-22T07:05:00.945 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-22T07:05:00.945 INFO:teuthology.orchestra.run.smithi069.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-22T07:05:00.945 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-22T07:05:00.945 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-22T07:05:00.945 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-22T07:05:00.945 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-22T07:05:00.946 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-22T07:05:00.946 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:00.946 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-03-22T07:05:00.946 INFO:teuthology.orchestra.run.smithi069.stdout:=========================================================================================== 2024-03-22T07:05:00.946 INFO:teuthology.orchestra.run.smithi069.stdout:Install 12 Packages 2024-03-22T07:05:00.946 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:00.946 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 37 M 2024-03-22T07:05:00.947 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 209 M 2024-03-22T07:05:00.947 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-03-22T07:05:01.424 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:28 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:05:01.452 INFO:teuthology.orchestra.run.smithi069.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 161 kB/s | 32 kB 00:00 2024-03-22T07:05:01.519 INFO:teuthology.orchestra.run.smithi069.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 258 kB/s | 17 kB 00:00 2024-03-22T07:05:01.581 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:05:01.582 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-03-22T07:05:01.582 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-03-22T07:05:01.582 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-03-22T07:05:01.582 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-03-22T07:05:01.582 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-rook noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 47 k 2024-03-22T07:05:01.582 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-03-22T07:05:01.582 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-22T07:05:01.582 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-22T07:05:01.582 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-22T07:05:01.582 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-22T07:05:01.582 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-22T07:05:01.583 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-22T07:05:01.583 INFO:teuthology.orchestra.run.smithi063.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-22T07:05:01.583 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-22T07:05:01.583 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-22T07:05:01.583 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-22T07:05:01.583 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-22T07:05:01.583 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-22T07:05:01.583 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:01.583 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-03-22T07:05:01.583 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-03-22T07:05:01.583 INFO:teuthology.orchestra.run.smithi063.stdout:Install 13 Packages 2024-03-22T07:05:01.583 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:01.584 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 2.0 M 2024-03-22T07:05:01.584 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 26 M 2024-03-22T07:05:01.584 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-03-22T07:05:01.594 INFO:teuthology.orchestra.run.smithi069.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 456 kB/s | 34 kB 00:00 2024-03-22T07:05:01.808 INFO:teuthology.orchestra.run.smithi063.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 142 kB/s | 24 kB 00:00 2024-03-22T07:05:01.825 INFO:teuthology.orchestra.run.smithi063.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 96 kB/s | 18 kB 00:00 2024-03-22T07:05:02.016 INFO:teuthology.orchestra.run.smithi063.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 842 kB/s | 174 kB 00:00 2024-03-22T07:05:02.042 INFO:teuthology.orchestra.run.smithi063.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 610 kB/s | 132 kB 00:00 2024-03-22T07:05:02.053 INFO:teuthology.orchestra.run.smithi069.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.7 MB/s | 3.0 MB 00:00 2024-03-22T07:05:02.092 INFO:teuthology.orchestra.run.smithi063.stdout:(5/13): ceph-mgr-rook-17.2.7-726.g9c4e0c34.el9. 104 kB/s | 47 kB 00:00 2024-03-22T07:05:02.117 INFO:teuthology.orchestra.run.smithi063.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.0 MB/s | 211 kB 00:00 2024-03-22T07:05:02.134 INFO:teuthology.orchestra.run.smithi063.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 463 kB/s | 43 kB 00:00 2024-03-22T07:05:02.170 INFO:teuthology.orchestra.run.smithi069.stdout:(5/12): ceph-mgr-diskprediction-local-17.2.7-72 8.1 MB/s | 7.4 MB 00:00 2024-03-22T07:05:02.236 INFO:teuthology.orchestra.run.smithi069.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 7.4 MB/s | 4.7 MB 00:00 2024-03-22T07:05:02.454 INFO:teuthology.orchestra.run.smithi069.stdout:(7/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 4.8 MB 00:00 2024-03-22T07:05:02.479 INFO:teuthology.orchestra.run.smithi069.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 709 kB/s | 217 kB 00:00 2024-03-22T07:05:02.651 INFO:teuthology.orchestra.run.smithi063.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 29 kB/s | 15 kB 00:00 2024-03-22T07:05:02.676 INFO:teuthology.orchestra.run.smithi063.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 55 kB/s | 32 kB 00:00 2024-03-22T07:05:02.944 INFO:teuthology.orchestra.run.smithi063.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 221 kB/s | 59 kB 00:00 2024-03-22T07:05:02.954 INFO:teuthology.orchestra.run.smithi069.stdout:(9/12): libquadmath-11.4.1-3.el9.x86_64.rpm 413 kB/s | 195 kB 00:00 2024-03-22T07:05:03.027 INFO:teuthology.orchestra.run.smithi063.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 227 kB/s | 203 kB 00:00 2024-03-22T07:05:03.115 INFO:teuthology.orchestra.run.smithi069.stdout:(10/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-03-22T07:05:03.137 INFO:teuthology.orchestra.run.smithi063.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 468 kB/s | 90 kB 00:00 2024-03-22T07:05:03.157 INFO:teuthology.orchestra.run.smithi069.stdout:(11/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-03-22T07:05:03.223 INFO:teuthology.orchestra.run.smithi069.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 920 kB/s | 245 kB 00:00 2024-03-22T07:05:03.226 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:05:03.226 INFO:teuthology.orchestra.run.smithi069.stdout:Total 16 MB/s | 37 MB 00:02 2024-03-22T07:05:03.265 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-03-22T07:05:03.290 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-03-22T07:05:03.290 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-03-22T07:05:03.362 INFO:teuthology.orchestra.run.smithi063.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.4 MB/s | 1.0 MB 00:00 2024-03-22T07:05:03.365 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:05:03.366 INFO:teuthology.orchestra.run.smithi063.stdout:Total 1.1 MB/s | 2.0 MB 00:01 2024-03-22T07:05:03.366 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-03-22T07:05:03.382 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-03-22T07:05:03.382 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-03-22T07:05:03.485 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-03-22T07:05:03.486 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-03-22T07:05:03.738 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-03-22T07:05:03.738 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-03-22T07:05:04.013 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-03-22T07:05:04.197 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-22T07:05:04.246 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-22T07:05:04.311 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-22T07:05:04.346 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-22T07:05:04.388 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-22T07:05:04.431 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-03-22T07:05:04.486 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-22T07:05:04.509 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-22T07:05:04.542 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-22T07:05:04.602 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-22T07:05:04.636 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-22T07:05:04.656 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-22T07:05:04.670 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-22T07:05:04.847 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-22T07:05:04.878 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-22T07:05:04.953 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-22T07:05:05.412 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-22T07:05:05.460 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-22T07:05:05.494 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-22T07:05:05.540 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-22T07:05:05.563 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 13/13 2024-03-22T07:05:05.606 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-22T07:05:05.722 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-22T07:05:06.336 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 13/13 2024-03-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 1/13 2024-03-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-22T07:05:06.337 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-22T07:05:06.338 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-22T07:05:06.714 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-22T07:05:06.714 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:06.715 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-03-22T07:05:06.715 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T07:05:06.715 INFO:teuthology.orchestra.run.smithi063.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-22T07:05:06.715 INFO:teuthology.orchestra.run.smithi063.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-22T07:05:06.715 INFO:teuthology.orchestra.run.smithi063.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-22T07:05:06.715 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-22T07:05:06.715 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-22T07:05:06.715 INFO:teuthology.orchestra.run.smithi063.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-22T07:05:06.715 INFO:teuthology.orchestra.run.smithi063.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-22T07:05:06.715 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-22T07:05:06.715 INFO:teuthology.orchestra.run.smithi063.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-22T07:05:06.715 INFO:teuthology.orchestra.run.smithi063.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-22T07:05:06.716 INFO:teuthology.orchestra.run.smithi063.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-22T07:05:06.716 INFO:teuthology.orchestra.run.smithi063.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-22T07:05:06.716 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:06.716 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:05:06.962 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-mgr-cephadm 2024-03-22T07:05:06.967 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-22T07:05:07.537 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:34 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:05:07.693 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:05:07.694 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-03-22T07:05:07.694 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-03-22T07:05:07.695 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-03-22T07:05:07.695 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-03-22T07:05:07.695 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-cephadm noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 123 k 2024-03-22T07:05:07.695 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-03-22T07:05:07.695 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-22T07:05:07.695 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-22T07:05:07.695 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-22T07:05:07.695 INFO:teuthology.orchestra.run.smithi063.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-22T07:05:07.695 INFO:teuthology.orchestra.run.smithi063.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-22T07:05:07.695 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:07.695 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-03-22T07:05:07.695 INFO:teuthology.orchestra.run.smithi063.stdout:======================================================================================= 2024-03-22T07:05:07.695 INFO:teuthology.orchestra.run.smithi063.stdout:Install 6 Packages 2024-03-22T07:05:07.696 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:07.696 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 6.8 M 2024-03-22T07:05:07.696 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 33 M 2024-03-22T07:05:07.696 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-03-22T07:05:07.986 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-22T07:05:08.027 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 12/12 2024-03-22T07:05:08.179 INFO:teuthology.orchestra.run.smithi063.stdout:(1/6): ceph-mgr-cephadm-17.2.7-726.g9c4e0c34.el 592 kB/s | 123 kB 00:00 2024-03-22T07:05:08.312 INFO:teuthology.orchestra.run.smithi063.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 725 kB/s | 248 kB 00:00 2024-03-22T07:05:08.528 INFO:teuthology.orchestra.run.smithi063.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 5.8 MB 00:00 2024-03-22T07:05:08.729 INFO:teuthology.orchestra.run.smithi063.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 138 kB/s | 58 kB 00:00 2024-03-22T07:05:08.754 INFO:teuthology.orchestra.run.smithi063.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 952 kB/s | 548 kB 00:00 2024-03-22T07:05:08.812 INFO:teuthology.orchestra.run.smithi063.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 181 kB/s | 51 kB 00:00 2024-03-22T07:05:08.815 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:05:08.815 INFO:teuthology.orchestra.run.smithi063.stdout:Total 6.1 MB/s | 6.8 MB 00:01 2024-03-22T07:05:08.828 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-03-22T07:05:08.839 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-03-22T07:05:08.839 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-03-22T07:05:08.931 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-03-22T07:05:08.932 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-03-22T07:05:09.233 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-03-22T07:05:09.404 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-22T07:05:09.465 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-22T07:05:10.022 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-22T07:05:10.110 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-22T07:05:10.164 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-22T07:05:10.192 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 6/6 2024-03-22T07:05:10.500 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 12/12 2024-03-22T07:05:10.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c 1/12 2024-03-22T07:05:10.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-22T07:05:10.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-22T07:05:10.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-22T07:05:10.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-22T07:05:10.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-22T07:05:10.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-22T07:05:10.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-22T07:05:10.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-22T07:05:10.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-22T07:05:10.502 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-22T07:05:10.927 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-22T07:05:10.927 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:10.928 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-03-22T07:05:10.928 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T07:05:10.928 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-22T07:05:10.928 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-22T07:05:10.928 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-22T07:05:10.928 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-22T07:05:10.928 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-22T07:05:10.928 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-22T07:05:10.928 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-22T07:05:10.928 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-22T07:05:10.928 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-22T07:05:10.928 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-22T07:05:10.928 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-22T07:05:10.929 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:10.929 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:05:10.953 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 6/6 2024-03-22T07:05:10.953 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/6 2024-03-22T07:05:10.953 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-22T07:05:10.953 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-22T07:05:10.953 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-22T07:05:10.953 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-22T07:05:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-22T07:05:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:11.236 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-03-22T07:05:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T07:05:11.236 INFO:teuthology.orchestra.run.smithi063.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-22T07:05:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-22T07:05:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-22T07:05:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-22T07:05:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-22T07:05:11.237 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:11.237 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:05:11.299 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-mgr-rook 2024-03-22T07:05:11.492 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install ceph-fuse 2024-03-22T07:05:11.871 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:16 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:05:12.022 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:05:12.024 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-03-22T07:05:12.024 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-03-22T07:05:12.024 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-03-22T07:05:12.024 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-03-22T07:05:12.024 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 47 k 2024-03-22T07:05:12.024 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-03-22T07:05:12.024 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-22T07:05:12.024 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-22T07:05:12.024 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-22T07:05:12.024 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-22T07:05:12.025 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-22T07:05:12.025 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-22T07:05:12.025 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-22T07:05:12.025 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-22T07:05:12.025 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-22T07:05:12.025 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-22T07:05:12.025 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-22T07:05:12.025 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-22T07:05:12.025 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:12.025 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-03-22T07:05:12.025 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-03-22T07:05:12.025 INFO:teuthology.orchestra.run.smithi069.stdout:Install 13 Packages 2024-03-22T07:05:12.025 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:12.026 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 2.0 M 2024-03-22T07:05:12.026 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 26 M 2024-03-22T07:05:12.026 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-03-22T07:05:12.066 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:39 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:05:12.209 INFO:teuthology.orchestra.run.smithi069.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 137 kB/s | 18 kB 00:00 2024-03-22T07:05:12.223 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:05:12.224 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:05:12.224 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repository Size 2024-03-22T07:05:12.224 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:05:12.224 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-03-22T07:05:12.224 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-fuse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 812 k 2024-03-22T07:05:12.224 INFO:teuthology.orchestra.run.smithi063.stdout:Installing dependencies: 2024-03-22T07:05:12.224 INFO:teuthology.orchestra.run.smithi063.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-22T07:05:12.225 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:12.225 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-03-22T07:05:12.225 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:05:12.225 INFO:teuthology.orchestra.run.smithi063.stdout:Install 2 Packages 2024-03-22T07:05:12.225 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:12.225 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 891 k 2024-03-22T07:05:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 2.6 M 2024-03-22T07:05:12.226 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-03-22T07:05:12.243 INFO:teuthology.orchestra.run.smithi069.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 147 kB/s | 24 kB 00:00 2024-03-22T07:05:12.302 INFO:teuthology.orchestra.run.smithi069.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 174 kB 00:00 2024-03-22T07:05:12.327 INFO:teuthology.orchestra.run.smithi069.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2024-03-22T07:05:12.352 INFO:teuthology.orchestra.run.smithi069.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-03-22T07:05:12.378 INFO:teuthology.orchestra.run.smithi069.stdout:(6/13): ceph-mgr-rook-17.2.7-726.g9c4e0c34.el9. 158 kB/s | 47 kB 00:00 2024-03-22T07:05:12.380 INFO:teuthology.orchestra.run.smithi063.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 509 kB/s | 79 kB 00:00 2024-03-22T07:05:12.403 INFO:teuthology.orchestra.run.smithi069.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 569 kB/s | 43 kB 00:00 2024-03-22T07:05:12.520 INFO:teuthology.orchestra.run.smithi069.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 129 kB/s | 15 kB 00:00 2024-03-22T07:05:12.545 INFO:teuthology.orchestra.run.smithi069.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 167 kB/s | 32 kB 00:00 2024-03-22T07:05:12.555 INFO:teuthology.orchestra.run.smithi063.stdout:(2/2): ceph-fuse-17.2.7-726.g9c4e0c34.el9.x86_6 2.4 MB/s | 812 kB 00:00 2024-03-22T07:05:12.556 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:05:12.557 INFO:teuthology.orchestra.run.smithi063.stdout:Total 2.6 MB/s | 891 kB 00:00 2024-03-22T07:05:12.557 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-03-22T07:05:12.577 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-03-22T07:05:12.578 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-03-22T07:05:12.629 INFO:teuthology.orchestra.run.smithi069.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 900 kB/s | 203 kB 00:00 2024-03-22T07:05:12.651 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-03-22T07:05:12.651 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-03-22T07:05:12.654 INFO:teuthology.orchestra.run.smithi069.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-03-22T07:05:12.679 INFO:teuthology.orchestra.run.smithi069.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-03-22T07:05:12.799 INFO:teuthology.orchestra.run.smithi069.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.7 MB/s | 1.0 MB 00:00 2024-03-22T07:05:12.801 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:05:12.801 INFO:teuthology.orchestra.run.smithi069.stdout:Total 2.6 MB/s | 2.0 MB 00:00 2024-03-22T07:05:12.801 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-03-22T07:05:12.814 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-03-22T07:05:12.814 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-03-22T07:05:12.836 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-03-22T07:05:12.902 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-03-22T07:05:12.903 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-03-22T07:05:12.963 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-22T07:05:13.032 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/2 2024-03-22T07:05:13.414 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-03-22T07:05:13.585 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-22T07:05:13.639 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-22T07:05:13.694 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-22T07:05:13.723 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-22T07:05:13.752 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/2 2024-03-22T07:05:13.752 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/2 2024-03-22T07:05:13.764 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-22T07:05:13.859 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-22T07:05:13.968 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-22T07:05:14.024 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-22T07:05:14.066 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-22T07:05:14.066 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:14.066 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-03-22T07:05:14.066 INFO:teuthology.orchestra.run.smithi063.stdout: ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-22T07:05:14.066 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:14.066 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:05:14.338 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install librados-devel 2024-03-22T07:05:14.742 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-22T07:05:14.788 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-22T07:05:14.821 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-22T07:05:14.867 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-22T07:05:14.892 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 13/13 2024-03-22T07:05:14.919 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:41 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:05:15.074 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:05:15.075 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:05:15.075 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repo Size 2024-03-22T07:05:15.075 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:05:15.075 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-03-22T07:05:15.075 INFO:teuthology.orchestra.run.smithi063.stdout: librados-devel x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 126 k 2024-03-22T07:05:15.075 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:15.076 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-03-22T07:05:15.076 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:05:15.076 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-03-22T07:05:15.076 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:15.076 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 126 k 2024-03-22T07:05:15.076 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 448 k 2024-03-22T07:05:15.076 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-03-22T07:05:15.293 INFO:teuthology.orchestra.run.smithi063.stdout:librados-devel-17.2.7-726.g9c4e0c34.el9.x86_64. 584 kB/s | 126 kB 00:00 2024-03-22T07:05:15.293 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:05:15.294 INFO:teuthology.orchestra.run.smithi063.stdout:Total 578 kB/s | 126 kB 00:00 2024-03-22T07:05:15.294 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-03-22T07:05:15.300 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-03-22T07:05:15.300 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-03-22T07:05:15.356 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-03-22T07:05:15.357 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-03-22T07:05:15.518 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-03-22T07:05:15.629 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:15.752 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 13/13 2024-03-22T07:05:15.752 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 1/13 2024-03-22T07:05:15.752 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-22T07:05:15.752 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-22T07:05:15.752 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-22T07:05:15.753 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-22T07:05:15.753 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-22T07:05:15.753 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-22T07:05:15.753 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-22T07:05:15.753 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-22T07:05:15.753 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-22T07:05:15.753 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-22T07:05:15.753 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-22T07:05:16.066 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:16.093 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-22T07:05:16.093 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:16.093 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-03-22T07:05:16.093 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T07:05:16.093 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-22T07:05:16.093 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-22T07:05:16.093 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-22T07:05:16.093 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-22T07:05:16.093 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-22T07:05:16.093 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-22T07:05:16.093 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-22T07:05:16.093 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-22T07:05:16.093 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-22T07:05:16.093 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-22T07:05:16.094 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-22T07:05:16.094 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-22T07:05:16.094 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:16.094 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:05:16.346 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-mgr-cephadm 2024-03-22T07:05:16.369 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:16.369 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:16.369 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-03-22T07:05:16.369 INFO:teuthology.orchestra.run.smithi063.stdout: librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:05:16.370 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:16.370 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:05:16.606 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install libcephfs2 2024-03-22T07:05:16.936 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:21 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:05:17.089 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:05:17.090 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-03-22T07:05:17.090 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-03-22T07:05:17.090 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-03-22T07:05:17.090 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-03-22T07:05:17.090 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm noarch 2:17.2.7-726.g9c4e0c34.el9 ceph-noarch 123 k 2024-03-22T07:05:17.090 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-03-22T07:05:17.090 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-22T07:05:17.090 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-22T07:05:17.090 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-22T07:05:17.090 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-22T07:05:17.090 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-22T07:05:17.091 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:17.091 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-03-22T07:05:17.091 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-03-22T07:05:17.091 INFO:teuthology.orchestra.run.smithi069.stdout:Install 6 Packages 2024-03-22T07:05:17.091 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:17.091 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 6.8 M 2024-03-22T07:05:17.091 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 33 M 2024-03-22T07:05:17.092 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-03-22T07:05:17.197 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:44 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:05:17.301 INFO:teuthology.orchestra.run.smithi063.stdout:Package libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T07:05:17.355 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:05:17.356 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-03-22T07:05:17.356 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:05:17.455 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install libcephfs-devel 2024-03-22T07:05:17.549 INFO:teuthology.orchestra.run.smithi069.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-03-22T07:05:17.649 INFO:teuthology.orchestra.run.smithi069.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2024-03-22T07:05:17.941 INFO:teuthology.orchestra.run.smithi069.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 198 kB/s | 58 kB 00:00 2024-03-22T07:05:18.037 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:45 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:05:18.049 INFO:teuthology.orchestra.run.smithi069.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 470 kB/s | 51 kB 00:00 2024-03-22T07:05:18.074 INFO:teuthology.orchestra.run.smithi069.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.0 MB/s | 548 kB 00:00 2024-03-22T07:05:18.192 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:05:18.193 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:05:18.193 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repo Size 2024-03-22T07:05:18.193 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:05:18.194 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-03-22T07:05:18.194 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-devel x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 25 k 2024-03-22T07:05:18.194 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:18.194 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-03-22T07:05:18.194 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:05:18.194 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-03-22T07:05:18.194 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:18.194 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 25 k 2024-03-22T07:05:18.194 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 110 k 2024-03-22T07:05:18.194 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-03-22T07:05:18.364 INFO:teuthology.orchestra.run.smithi063.stdout:libcephfs-devel-17.2.7-726.g9c4e0c34.el9.x86_64 147 kB/s | 25 kB 00:00 2024-03-22T07:05:18.365 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:05:18.365 INFO:teuthology.orchestra.run.smithi063.stdout:Total 145 kB/s | 25 kB 00:00 2024-03-22T07:05:18.365 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-03-22T07:05:18.369 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-03-22T07:05:18.369 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-03-22T07:05:18.370 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-03-22T07:05:18.371 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-03-22T07:05:18.461 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-03-22T07:05:18.572 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:18.950 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:19.209 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:19.209 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:19.209 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-03-22T07:05:19.209 INFO:teuthology.orchestra.run.smithi063.stdout: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:05:19.209 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:19.210 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:05:19.379 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install librados2 2024-03-22T07:05:19.775 INFO:teuthology.orchestra.run.smithi069.stdout:(6/6): ceph-mgr-cephadm-17.2.7-726.g9c4e0c34.el 51 kB/s | 123 kB 00:02 2024-03-22T07:05:19.778 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:05:19.778 INFO:teuthology.orchestra.run.smithi069.stdout:Total 2.5 MB/s | 6.8 MB 00:02 2024-03-22T07:05:19.793 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-03-22T07:05:19.804 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-03-22T07:05:19.804 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-03-22T07:05:19.895 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-03-22T07:05:19.895 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-03-22T07:05:19.951 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:46 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:05:20.051 INFO:teuthology.orchestra.run.smithi063.stdout:Package librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T07:05:20.107 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:05:20.108 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-03-22T07:05:20.108 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:05:20.178 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install librbd1 2024-03-22T07:05:20.194 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-03-22T07:05:20.364 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-22T07:05:20.417 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-22T07:05:20.750 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:47 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:05:20.850 INFO:teuthology.orchestra.run.smithi063.stdout:Package librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T07:05:20.906 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:05:20.908 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-03-22T07:05:20.908 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:05:20.927 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-22T07:05:20.990 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install python3-rados 2024-03-22T07:05:21.011 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-22T07:05:21.067 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-22T07:05:21.096 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 6/6 2024-03-22T07:05:21.558 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:48 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:05:21.658 INFO:teuthology.orchestra.run.smithi063.stdout:Package python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T07:05:21.714 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:05:21.715 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-03-22T07:05:21.715 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:05:21.790 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install python3-rgw 2024-03-22T07:05:21.853 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 6/6 2024-03-22T07:05:21.853 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 1/6 2024-03-22T07:05:21.853 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-22T07:05:21.853 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-22T07:05:21.853 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-22T07:05:21.853 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-22T07:05:22.160 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-22T07:05:22.160 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:22.160 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-03-22T07:05:22.160 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm-2:17.2.7-726.g9c4e0c34.el9.noarch 2024-03-22T07:05:22.160 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-22T07:05:22.161 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-22T07:05:22.161 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-22T07:05:22.161 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-22T07:05:22.161 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-22T07:05:22.161 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:22.161 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:05:22.380 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:49 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:05:22.434 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-fuse 2024-03-22T07:05:22.491 INFO:teuthology.orchestra.run.smithi063.stdout:Package python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T07:05:22.547 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:05:22.548 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-03-22T07:05:22.548 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:05:22.621 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install python3-cephfs 2024-03-22T07:05:22.999 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:27 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:05:23.148 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:05:23.149 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:05:23.149 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-03-22T07:05:23.149 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:05:23.149 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-03-22T07:05:23.149 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 812 k 2024-03-22T07:05:23.149 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-03-22T07:05:23.149 INFO:teuthology.orchestra.run.smithi069.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-22T07:05:23.149 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:23.149 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-03-22T07:05:23.150 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:05:23.150 INFO:teuthology.orchestra.run.smithi069.stdout:Install 2 Packages 2024-03-22T07:05:23.150 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:23.150 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 891 k 2024-03-22T07:05:23.150 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 2.6 M 2024-03-22T07:05:23.150 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-03-22T07:05:23.197 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:50 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:05:23.297 INFO:teuthology.orchestra.run.smithi063.stdout:Package python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T07:05:23.352 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:05:23.353 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-03-22T07:05:23.353 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:05:23.376 INFO:teuthology.orchestra.run.smithi069.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 347 kB/s | 79 kB 00:00 2024-03-22T07:05:23.431 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install python3-rbd 2024-03-22T07:05:23.451 INFO:teuthology.orchestra.run.smithi069.stdout:(2/2): ceph-fuse-17.2.7-726.g9c4e0c34.el9.x86_6 2.6 MB/s | 812 kB 00:00 2024-03-22T07:05:23.453 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:05:23.453 INFO:teuthology.orchestra.run.smithi069.stdout:Total 2.9 MB/s | 891 kB 00:00 2024-03-22T07:05:23.453 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-03-22T07:05:23.472 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-03-22T07:05:23.473 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-03-22T07:05:23.543 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-03-22T07:05:23.544 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-03-22T07:05:23.725 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-03-22T07:05:23.864 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-22T07:05:23.934 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/2 2024-03-22T07:05:24.001 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:51 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:05:24.101 INFO:teuthology.orchestra.run.smithi063.stdout:Package python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T07:05:24.155 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:05:24.156 INFO:teuthology.orchestra.run.smithi063.stdout:Nothing to do. 2024-03-22T07:05:24.156 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:05:24.231 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install rbd-fuse 2024-03-22T07:05:24.620 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2/2 2024-03-22T07:05:24.620 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/2 2024-03-22T07:05:24.823 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:51 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:05:24.917 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-22T07:05:24.917 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:24.917 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-03-22T07:05:24.917 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-22T07:05:24.917 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:24.917 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:05:24.990 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:05:24.991 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:05:24.991 INFO:teuthology.orchestra.run.smithi063.stdout: Package Architecture Version Repository Size 2024-03-22T07:05:24.991 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:05:24.991 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-03-22T07:05:24.991 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-fuse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 82 k 2024-03-22T07:05:24.991 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:24.991 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-03-22T07:05:24.991 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:05:24.991 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-03-22T07:05:24.991 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:24.992 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 82 k 2024-03-22T07:05:24.992 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 222 k 2024-03-22T07:05:24.992 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-03-22T07:05:25.165 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install librados-devel 2024-03-22T07:05:25.201 INFO:teuthology.orchestra.run.smithi063.stdout:rbd-fuse-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 391 kB/s | 82 kB 00:00 2024-03-22T07:05:25.201 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:05:25.202 INFO:teuthology.orchestra.run.smithi063.stdout:Total 388 kB/s | 82 kB 00:00 2024-03-22T07:05:25.202 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-03-22T07:05:25.209 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-03-22T07:05:25.209 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-03-22T07:05:25.266 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-03-22T07:05:25.266 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-03-22T07:05:25.404 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-03-22T07:05:25.502 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:25.736 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:30 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:05:25.888 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:05:25.889 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:05:25.889 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repo Size 2024-03-22T07:05:25.889 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:05:25.889 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-03-22T07:05:25.889 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 126 k 2024-03-22T07:05:25.889 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:25.890 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-03-22T07:05:25.890 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:05:25.890 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package 2024-03-22T07:05:25.890 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:25.890 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 126 k 2024-03-22T07:05:25.890 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 448 k 2024-03-22T07:05:25.890 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-03-22T07:05:25.892 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:26.176 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:26.176 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:26.176 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-03-22T07:05:26.176 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:05:26.176 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:26.176 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:05:26.222 INFO:teuthology.orchestra.run.smithi069.stdout:librados-devel-17.2.7-726.g9c4e0c34.el9.x86_64. 380 kB/s | 126 kB 00:00 2024-03-22T07:05:26.223 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:05:26.223 INFO:teuthology.orchestra.run.smithi069.stdout:Total 379 kB/s | 126 kB 00:00 2024-03-22T07:05:26.223 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-03-22T07:05:26.228 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-03-22T07:05:26.228 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-03-22T07:05:26.283 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-03-22T07:05:26.284 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-03-22T07:05:26.436 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install rbd-mirror 2024-03-22T07:05:26.465 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-03-22T07:05:26.570 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:27.004 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:27.007 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:54 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:05:27.165 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:05:27.166 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:05:27.166 INFO:teuthology.orchestra.run.smithi063.stdout: Package Arch Version Repo Size 2024-03-22T07:05:27.166 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:05:27.166 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-03-22T07:05:27.166 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-mirror x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.9 M 2024-03-22T07:05:27.166 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:27.166 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-03-22T07:05:27.166 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:05:27.166 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-03-22T07:05:27.166 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:27.166 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 2.9 M 2024-03-22T07:05:27.166 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 12 M 2024-03-22T07:05:27.166 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-03-22T07:05:27.366 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:27.366 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:27.366 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-03-22T07:05:27.366 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:05:27.366 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:27.366 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:05:27.641 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install libcephfs2 2024-03-22T07:05:27.846 INFO:teuthology.orchestra.run.smithi063.stdout:rbd-mirror-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 4.3 MB/s | 2.9 MB 00:00 2024-03-22T07:05:27.846 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:05:27.847 INFO:teuthology.orchestra.run.smithi063.stdout:Total 4.3 MB/s | 2.9 MB 00:00 2024-03-22T07:05:27.847 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-03-22T07:05:27.857 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-03-22T07:05:27.857 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-03-22T07:05:27.934 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-03-22T07:05:27.934 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-03-22T07:05:28.210 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:33 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:05:28.311 INFO:teuthology.orchestra.run.smithi069.stdout:Package libcephfs2-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T07:05:28.362 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-03-22T07:05:28.365 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:05:28.366 INFO:teuthology.orchestra.run.smithi069.stdout:Nothing to do. 2024-03-22T07:05:28.366 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:05:28.422 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:28.445 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:28.446 INFO:teuthology.orchestra.run.smithi063.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T07:05:28.446 INFO:teuthology.orchestra.run.smithi063.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-22T07:05:28.446 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-22T07:05:28.446 INFO:teuthology.orchestra.run.smithi063.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-22T07:05:28.446 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:28.447 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install libcephfs-devel 2024-03-22T07:05:29.007 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:34 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:05:29.155 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:05:29.156 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:05:29.156 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repo Size 2024-03-22T07:05:29.156 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:05:29.156 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-03-22T07:05:29.156 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 25 k 2024-03-22T07:05:29.156 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:29.156 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-03-22T07:05:29.156 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:05:29.156 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package 2024-03-22T07:05:29.156 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:29.157 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 25 k 2024-03-22T07:05:29.157 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 110 k 2024-03-22T07:05:29.157 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-03-22T07:05:29.451 INFO:teuthology.orchestra.run.smithi069.stdout:libcephfs-devel-17.2.7-726.g9c4e0c34.el9.x86_64 85 kB/s | 25 kB 00:00 2024-03-22T07:05:29.452 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:05:29.452 INFO:teuthology.orchestra.run.smithi069.stdout:Total 84 kB/s | 25 kB 00:00 2024-03-22T07:05:29.452 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-03-22T07:05:29.455 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-03-22T07:05:29.455 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-03-22T07:05:29.457 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-03-22T07:05:29.457 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-03-22T07:05:29.536 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-03-22T07:05:29.542 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:29.542 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:29.542 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-03-22T07:05:29.542 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:05:29.542 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:29.542 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:05:29.641 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:29.775 DEBUG:teuthology.orchestra.run.smithi063:> sudo yum -y install rbd-nbd 2024-03-22T07:05:30.018 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:30.291 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:30.291 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:30.291 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-03-22T07:05:30.291 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:05:30.292 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:30.292 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:05:30.366 INFO:teuthology.orchestra.run.smithi063.stdout:Last metadata expiration check: 0:01:57 ago on Fri 22 Mar 2024 07:03:33 AM UTC. 2024-03-22T07:05:30.469 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install librados2 2024-03-22T07:05:30.522 INFO:teuthology.orchestra.run.smithi063.stdout:Dependencies resolved. 2024-03-22T07:05:30.523 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:05:30.523 INFO:teuthology.orchestra.run.smithi063.stdout: Package Architecture Version Repository Size 2024-03-22T07:05:30.523 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:05:30.523 INFO:teuthology.orchestra.run.smithi063.stdout:Installing: 2024-03-22T07:05:30.523 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-nbd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 165 k 2024-03-22T07:05:30.523 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:30.523 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction Summary 2024-03-22T07:05:30.523 INFO:teuthology.orchestra.run.smithi063.stdout:================================================================================ 2024-03-22T07:05:30.523 INFO:teuthology.orchestra.run.smithi063.stdout:Install 1 Package 2024-03-22T07:05:30.524 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:30.524 INFO:teuthology.orchestra.run.smithi063.stdout:Total download size: 165 k 2024-03-22T07:05:30.524 INFO:teuthology.orchestra.run.smithi063.stdout:Installed size: 477 k 2024-03-22T07:05:30.524 INFO:teuthology.orchestra.run.smithi063.stdout:Downloading Packages: 2024-03-22T07:05:30.748 INFO:teuthology.orchestra.run.smithi063.stdout:rbd-nbd-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 736 kB/s | 165 kB 00:00 2024-03-22T07:05:30.749 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:05:30.749 INFO:teuthology.orchestra.run.smithi063.stdout:Total 731 kB/s | 165 kB 00:00 2024-03-22T07:05:30.749 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction check 2024-03-22T07:05:30.757 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction check succeeded. 2024-03-22T07:05:30.757 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction test 2024-03-22T07:05:30.815 INFO:teuthology.orchestra.run.smithi063.stdout:Transaction test succeeded. 2024-03-22T07:05:30.816 INFO:teuthology.orchestra.run.smithi063.stdout:Running transaction 2024-03-22T07:05:30.963 INFO:teuthology.orchestra.run.smithi063.stdout: Preparing : 1/1 2024-03-22T07:05:31.032 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:36 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:05:31.074 INFO:teuthology.orchestra.run.smithi063.stdout: Installing : rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:31.129 INFO:teuthology.orchestra.run.smithi069.stdout:Package librados2-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T07:05:31.180 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:05:31.181 INFO:teuthology.orchestra.run.smithi069.stdout:Nothing to do. 2024-03-22T07:05:31.181 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:05:31.267 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install librbd1 2024-03-22T07:05:31.456 INFO:teuthology.orchestra.run.smithi063.stdout: Running scriptlet: rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:31.734 INFO:teuthology.orchestra.run.smithi063.stdout: Verifying : rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:31.734 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:31.734 INFO:teuthology.orchestra.run.smithi063.stdout:Installed: 2024-03-22T07:05:31.734 INFO:teuthology.orchestra.run.smithi063.stdout: rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:05:31.734 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:05:31.734 INFO:teuthology.orchestra.run.smithi063.stdout:Complete! 2024-03-22T07:05:31.829 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:36 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:05:31.927 INFO:teuthology.orchestra.run.smithi069.stdout:Package librbd1-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T07:05:31.973 DEBUG:teuthology.parallel:result is None 2024-03-22T07:05:31.980 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:05:31.982 INFO:teuthology.orchestra.run.smithi069.stdout:Nothing to do. 2024-03-22T07:05:31.982 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:05:32.064 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install python3-rados 2024-03-22T07:05:32.624 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:37 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:05:32.720 INFO:teuthology.orchestra.run.smithi069.stdout:Package python3-rados-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T07:05:32.773 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:05:32.775 INFO:teuthology.orchestra.run.smithi069.stdout:Nothing to do. 2024-03-22T07:05:32.775 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:05:32.853 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install python3-rgw 2024-03-22T07:05:33.408 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:38 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:05:33.505 INFO:teuthology.orchestra.run.smithi069.stdout:Package python3-rgw-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T07:05:33.556 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:05:33.557 INFO:teuthology.orchestra.run.smithi069.stdout:Nothing to do. 2024-03-22T07:05:33.557 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:05:33.650 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install python3-cephfs 2024-03-22T07:05:34.214 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:39 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:05:34.311 INFO:teuthology.orchestra.run.smithi069.stdout:Package python3-cephfs-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T07:05:34.363 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:05:34.364 INFO:teuthology.orchestra.run.smithi069.stdout:Nothing to do. 2024-03-22T07:05:34.364 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:05:34.450 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install python3-rbd 2024-03-22T07:05:35.003 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:40 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:05:35.100 INFO:teuthology.orchestra.run.smithi069.stdout:Package python3-rbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 is already installed. 2024-03-22T07:05:35.152 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:05:35.153 INFO:teuthology.orchestra.run.smithi069.stdout:Nothing to do. 2024-03-22T07:05:35.153 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:05:35.237 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install rbd-fuse 2024-03-22T07:05:35.795 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:40 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:05:35.944 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:05:35.945 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:05:35.945 INFO:teuthology.orchestra.run.smithi069.stdout: Package Architecture Version Repository Size 2024-03-22T07:05:35.945 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:05:35.945 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-03-22T07:05:35.945 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 82 k 2024-03-22T07:05:35.945 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:35.945 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-03-22T07:05:35.946 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:05:35.946 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package 2024-03-22T07:05:35.946 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:35.946 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 82 k 2024-03-22T07:05:35.946 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 222 k 2024-03-22T07:05:35.946 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-03-22T07:05:36.262 INFO:teuthology.orchestra.run.smithi069.stdout:rbd-fuse-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 258 kB/s | 82 kB 00:00 2024-03-22T07:05:36.263 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:05:36.263 INFO:teuthology.orchestra.run.smithi069.stdout:Total 257 kB/s | 82 kB 00:00 2024-03-22T07:05:36.263 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-03-22T07:05:36.270 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-03-22T07:05:36.270 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-03-22T07:05:36.323 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-03-22T07:05:36.323 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-03-22T07:05:36.457 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-03-22T07:05:36.551 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:36.971 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:37.247 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:37.247 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:37.247 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-03-22T07:05:37.247 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:05:37.248 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:37.248 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:05:37.516 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install rbd-mirror 2024-03-22T07:05:38.087 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:43 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:05:38.240 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:05:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:05:38.241 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repo Size 2024-03-22T07:05:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:05:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-03-22T07:05:38.241 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 2.9 M 2024-03-22T07:05:38.241 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-03-22T07:05:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:05:38.241 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package 2024-03-22T07:05:38.242 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:38.242 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 2.9 M 2024-03-22T07:05:38.242 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 12 M 2024-03-22T07:05:38.242 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-03-22T07:05:38.790 INFO:teuthology.orchestra.run.smithi069.stdout:rbd-mirror-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 5.4 MB/s | 2.9 MB 00:00 2024-03-22T07:05:38.790 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:05:38.791 INFO:teuthology.orchestra.run.smithi069.stdout:Total 5.3 MB/s | 2.9 MB 00:00 2024-03-22T07:05:38.791 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-03-22T07:05:38.800 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-03-22T07:05:38.800 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-03-22T07:05:38.871 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-03-22T07:05:38.872 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-03-22T07:05:39.283 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-03-22T07:05:39.349 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:39.374 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:39.374 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-22T07:05:39.374 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-22T07:05:39.374 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-22T07:05:39.374 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-22T07:05:39.374 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:40.478 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:40.478 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:40.478 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-03-22T07:05:40.478 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:05:40.478 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:40.478 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:05:40.735 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install rbd-nbd 2024-03-22T07:05:41.308 INFO:teuthology.orchestra.run.smithi069.stdout:Last metadata expiration check: 0:01:46 ago on Fri 22 Mar 2024 07:03:55 AM UTC. 2024-03-22T07:05:41.456 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-03-22T07:05:41.457 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:05:41.457 INFO:teuthology.orchestra.run.smithi069.stdout: Package Architecture Version Repository Size 2024-03-22T07:05:41.457 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:05:41.457 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-03-22T07:05:41.457 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd x86_64 2:17.2.7-726.g9c4e0c34.el9 ceph 165 k 2024-03-22T07:05:41.457 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:41.457 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-03-22T07:05:41.458 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-03-22T07:05:41.458 INFO:teuthology.orchestra.run.smithi069.stdout:Install 1 Package 2024-03-22T07:05:41.458 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:41.458 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 165 k 2024-03-22T07:05:41.458 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 477 k 2024-03-22T07:05:41.458 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-03-22T07:05:41.810 INFO:teuthology.orchestra.run.smithi069.stdout:rbd-nbd-17.2.7-726.g9c4e0c34.el9.x86_64.rpm 468 kB/s | 165 kB 00:00 2024-03-22T07:05:41.811 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:05:41.811 INFO:teuthology.orchestra.run.smithi069.stdout:Total 466 kB/s | 165 kB 00:00 2024-03-22T07:05:41.811 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-03-22T07:05:41.819 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-03-22T07:05:41.819 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-03-22T07:05:41.874 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-03-22T07:05:41.875 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-03-22T07:05:42.024 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-03-22T07:05:42.116 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:42.543 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:42.808 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 1/1 2024-03-22T07:05:42.808 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:42.808 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-03-22T07:05:42.808 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd-2:17.2.7-726.g9c4e0c34.el9.x86_64 2024-03-22T07:05:42.808 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:05:42.808 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-03-22T07:05:43.083 DEBUG:teuthology.parallel:result is None 2024-03-22T07:05:43.083 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T07:05:43.083 INFO:teuthology.packaging:ref: None 2024-03-22T07:05:43.083 INFO:teuthology.packaging:tag: None 2024-03-22T07:05:43.084 INFO:teuthology.packaging:branch: quincy 2024-03-22T07:05:43.084 INFO:teuthology.packaging:sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:05:43.084 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-22T07:05:43.472 DEBUG:teuthology.orchestra.run.smithi063:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-22T07:05:43.497 INFO:teuthology.orchestra.run.smithi063.stdout:17.2.7-726.g9c4e0c34.el9 2024-03-22T07:05:43.497 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-726.g9c4e0c34.el9 2024-03-22T07:05:43.497 INFO:teuthology.task.install:The correct ceph version 17.2.7-726.g9c4e0c34 is installed. 2024-03-22T07:05:43.499 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-22T07:05:43.499 INFO:teuthology.packaging:ref: None 2024-03-22T07:05:43.499 INFO:teuthology.packaging:tag: None 2024-03-22T07:05:43.499 INFO:teuthology.packaging:branch: quincy 2024-03-22T07:05:43.499 INFO:teuthology.packaging:sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:05:43.499 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-03-22T07:05:43.837 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-22T07:05:43.859 INFO:teuthology.orchestra.run.smithi069.stdout:17.2.7-726.g9c4e0c34.el9 2024-03-22T07:05:43.859 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-726.g9c4e0c34.el9 2024-03-22T07:05:43.859 INFO:teuthology.task.install:The correct ceph version 17.2.7-726.g9c4e0c34 is installed. 2024-03-22T07:05:43.861 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-22T07:05:43.861 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T07:05:43.861 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-22T07:05:43.893 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-22T07:05:43.893 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-22T07:05:43.928 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-22T07:05:43.928 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T07:05:43.928 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/daemon-helper 2024-03-22T07:05:43.965 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-22T07:05:44.034 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-22T07:05:44.034 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/daemon-helper 2024-03-22T07:05:44.062 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-22T07:05:44.129 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-22T07:05:44.129 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T07:05:44.129 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-22T07:05:44.159 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-22T07:05:44.228 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-22T07:05:44.228 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-22T07:05:44.255 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-22T07:05:44.323 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-22T07:05:44.323 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T07:05:44.323 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/usr/bin/stdin-killer 2024-03-22T07:05:44.351 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-22T07:05:44.420 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-22T07:05:44.420 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/stdin-killer 2024-03-22T07:05:44.448 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-22T07:05:44.514 INFO:teuthology.run_tasks:Running task print... 2024-03-22T07:05:44.524 INFO:teuthology.task.print:**** done install task... 2024-03-22T07:05:44.524 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-22T07:05:44.630 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'db0330b1e4e2470d52b750e251e55a522b4f7d69'} 2024-03-22T07:05:44.630 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-03-22T07:05:44.631 INFO:tasks.cephadm:Cluster fsid is 9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:05:44.631 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-22T07:05:44.631 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-03-22T07:05:44.631 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi063': '172.21.15.63', 'mon.smithi069': '172.21.15.69'} 2024-03-22T07:05:44.631 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-22T07:05:44.632 DEBUG:teuthology.orchestra.run.smithi063:> sudo hostname $(hostname -s) 2024-03-22T07:05:44.661 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname $(hostname -s) 2024-03-22T07:05:44.688 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-03-22T07:05:44.689 DEBUG:teuthology.orchestra.run.smithi063:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-22T07:05:44.867 INFO:teuthology.orchestra.run.smithi063.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Mar 22 07:05 /home/ubuntu/cephtest/cephadm 2024-03-22T07:05:44.867 DEBUG:teuthology.orchestra.run.smithi069:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-22T07:05:45.035 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Mar 22 07:05 /home/ubuntu/cephtest/cephadm 2024-03-22T07:05:45.035 DEBUG:teuthology.orchestra.run.smithi063:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-22T07:05:45.052 DEBUG:teuthology.orchestra.run.smithi069:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-22T07:05:45.077 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-03-22T07:05:45.077 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-03-22T07:05:45.094 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-03-22T07:05:45.982 INFO:teuthology.orchestra.run.smithi069.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-22T07:05:45.998 INFO:teuthology.orchestra.run.smithi063.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-22T07:06:28.911 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:06:28.912 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph_version": "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)", 2024-03-22T07:06:28.912 INFO:teuthology.orchestra.run.smithi063.stdout: "image_id": "837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de", 2024-03-22T07:06:28.912 INFO:teuthology.orchestra.run.smithi063.stdout: "repo_digests": [ 2024-03-22T07:06:28.912 INFO:teuthology.orchestra.run.smithi063.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232" 2024-03-22T07:06:28.912 INFO:teuthology.orchestra.run.smithi063.stdout: ] 2024-03-22T07:06:28.912 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:06:29.105 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-03-22T07:06:29.106 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_version": "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)", 2024-03-22T07:06:29.106 INFO:teuthology.orchestra.run.smithi069.stdout: "image_id": "837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de", 2024-03-22T07:06:29.106 INFO:teuthology.orchestra.run.smithi069.stdout: "repo_digests": [ 2024-03-22T07:06:29.106 INFO:teuthology.orchestra.run.smithi069.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232" 2024-03-22T07:06:29.106 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-03-22T07:06:29.106 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-03-22T07:06:29.125 DEBUG:teuthology.orchestra.run.smithi063:> sudo mkdir -p /etc/ceph 2024-03-22T07:06:29.155 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /etc/ceph 2024-03-22T07:06:29.185 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 777 /etc/ceph 2024-03-22T07:06:29.222 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 777 /etc/ceph 2024-03-22T07:06:29.251 INFO:tasks.cephadm:Writing seed config... 2024-03-22T07:06:29.252 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-03-22T07:06:29.252 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-03-22T07:06:29.252 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-22T07:06:29.252 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-22T07:06:29.252 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-22T07:06:29.252 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-22T07:06:29.252 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-22T07:06:29.252 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-22T07:06:29.252 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-22T07:06:29.252 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-22T07:06:29.252 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-22T07:06:29.253 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-22T07:06:29.253 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-22T07:06:29.253 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-22T07:06:29.253 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-22T07:06:29.253 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-22T07:06:29.253 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-22T07:06:29.253 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-22T07:06:29.253 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-22T07:06:29.253 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-22T07:06:29.253 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-22T07:06:29.253 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-22T07:06:29.253 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-22T07:06:29.253 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-03-22T07:06:29.254 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-22T07:06:29.254 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-22T07:06:29.254 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-22T07:06:29.254 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-22T07:06:29.254 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-22T07:06:29.254 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-22T07:06:29.254 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-22T07:06:29.254 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-22T07:06:29.254 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-22T07:06:29.254 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-22T07:06:29.254 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-22T07:06:29.254 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-22T07:06:29.255 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-22T07:06:29.255 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-22T07:06:29.255 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-22T07:06:29.255 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-22T07:06:29.255 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T07:06:29.256 DEBUG:teuthology.orchestra.run.smithi063:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-22T07:06:29.277 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9a0ba9d6-e81a-11ee-95cd-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-22T07:06:29.278 DEBUG:teuthology.orchestra.run.smithi063:mon.smithi063> sudo journalctl -f -n 0 -u ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mon.smithi063.service 2024-03-22T07:06:29.319 INFO:tasks.cephadm:Bootstrapping... 2024-03-22T07:06:29.320 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.63 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-22T07:06:29.529 INFO:teuthology.orchestra.run.smithi063.stdout:-------------------------------------------------------------------------------- 2024-03-22T07:06:29.530 INFO:teuthology.orchestra.run.smithi063.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', '9a0ba9d6-e81a-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.63', '--skip-admin-label'] 2024-03-22T07:06:29.554 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-22T07:06:29.555 INFO:teuthology.orchestra.run.smithi063.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-22T07:06:29.555 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying podman|docker is present... 2024-03-22T07:06:29.579 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-22T07:06:29.579 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying lvm2 is present... 2024-03-22T07:06:29.579 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying time synchronization is in place... 2024-03-22T07:06:29.586 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-22T07:06:29.586 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-22T07:06:29.593 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-22T07:06:29.593 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-03-22T07:06:29.599 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout enabled 2024-03-22T07:06:29.606 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout active 2024-03-22T07:06:29.606 INFO:teuthology.orchestra.run.smithi063.stdout:Unit chronyd.service is enabled and running 2024-03-22T07:06:29.606 INFO:teuthology.orchestra.run.smithi063.stdout:Repeating the final host check... 2024-03-22T07:06:29.628 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-22T07:06:29.628 INFO:teuthology.orchestra.run.smithi063.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-22T07:06:29.628 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl is present 2024-03-22T07:06:29.628 INFO:teuthology.orchestra.run.smithi063.stdout:lvcreate is present 2024-03-22T07:06:29.635 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-22T07:06:29.635 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-22T07:06:29.641 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-22T07:06:29.641 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-03-22T07:06:29.647 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout enabled 2024-03-22T07:06:29.654 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout active 2024-03-22T07:06:29.654 INFO:teuthology.orchestra.run.smithi063.stdout:Unit chronyd.service is enabled and running 2024-03-22T07:06:29.654 INFO:teuthology.orchestra.run.smithi063.stdout:Host looks OK 2024-03-22T07:06:29.654 INFO:teuthology.orchestra.run.smithi063.stdout:Cluster fsid: 9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:06:29.654 INFO:teuthology.orchestra.run.smithi063.stdout:Acquiring lock 140320593241184 on /run/cephadm/9a0ba9d6-e81a-11ee-95cd-87774f69a715.lock 2024-03-22T07:06:29.654 INFO:teuthology.orchestra.run.smithi063.stdout:Lock 140320593241184 acquired on /run/cephadm/9a0ba9d6-e81a-11ee-95cd-87774f69a715.lock 2024-03-22T07:06:29.655 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying IP 172.21.15.63 port 3300 ... 2024-03-22T07:06:29.655 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying IP 172.21.15.63 port 6789 ... 2024-03-22T07:06:29.655 INFO:teuthology.orchestra.run.smithi063.stdout:Base mon IP(s) is [172.21.15.63:3300, 172.21.15.63:6789], mon addrv is [v2:172.21.15.63:3300,v1:172.21.15.63:6789] 2024-03-22T07:06:29.658 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.63 metric 100 2024-03-22T07:06:29.659 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.63 metric 100 2024-03-22T07:06:29.661 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-22T07:06:29.661 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-22T07:06:29.662 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1790sec hoplimit 64 pref medium 2024-03-22T07:06:29.664 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-22T07:06:29.664 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-22T07:06:29.664 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-22T07:06:29.664 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-22T07:06:29.664 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:feb6:f8d7/64 scope link 2024-03-22T07:06:29.664 INFO:teuthology.orchestra.run.smithi063.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-22T07:06:29.665 INFO:teuthology.orchestra.run.smithi063.stdout:Mon IP `172.21.15.63` is in CIDR network `172.21.0.0/20` 2024-03-22T07:06:29.665 INFO:teuthology.orchestra.run.smithi063.stdout:Mon IP `172.21.15.63` is in CIDR network `172.21.0.0/20` 2024-03-22T07:06:29.665 INFO:teuthology.orchestra.run.smithi063.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-22T07:06:29.665 INFO:teuthology.orchestra.run.smithi063.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-22T07:06:29.666 INFO:teuthology.orchestra.run.smithi063.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-22T07:06:30.170 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stdout 837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de 2024-03-22T07:06:30.170 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-03-22T07:06:30.170 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Getting image source signatures 2024-03-22T07:06:30.170 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Copying blob sha256:601ec453fe24458667695dda21742a4f3ea719c31ad8aa8ec48bf585cc42a7d7 2024-03-22T07:06:30.170 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-22T07:06:30.170 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Copying config sha256:837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de 2024-03-22T07:06:30.171 INFO:teuthology.orchestra.run.smithi063.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-22T07:06:31.050 INFO:teuthology.orchestra.run.smithi063.stdout:ceph: stdout ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable) 2024-03-22T07:06:31.050 INFO:teuthology.orchestra.run.smithi063.stdout:Ceph version: ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable) 2024-03-22T07:06:31.050 INFO:teuthology.orchestra.run.smithi063.stdout:Extracting ceph user uid/gid from container image... 2024-03-22T07:06:31.988 INFO:teuthology.orchestra.run.smithi063.stdout:stat: stdout 167 167 2024-03-22T07:06:31.989 INFO:teuthology.orchestra.run.smithi063.stdout:Creating initial keys... 2024-03-22T07:06:32.978 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph-authtool: stdout AQD4Lf1lo6iMEBAAJDoz6C7hR6Tor7exFTS8qA== 2024-03-22T07:06:33.816 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph-authtool: stdout AQD5Lf1lEU6IEhAAtdOie47fdWvFtwSBnpa+UQ== 2024-03-22T07:06:34.864 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph-authtool: stdout AQD6Lf1ljDDhCBAAzFkoU0fji550qi+F4jd+Lw== 2024-03-22T07:06:34.865 INFO:teuthology.orchestra.run.smithi063.stdout:Creating initial monmap... 2024-03-22T07:06:35.744 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-22T07:06:35.744 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-03-22T07:06:35.744 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:06:35.744 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-22T07:06:35.744 INFO:teuthology.orchestra.run.smithi063.stdout:monmaptool for smithi063 [v2:172.21.15.63:3300,v1:172.21.15.63:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-22T07:06:35.745 INFO:teuthology.orchestra.run.smithi063.stdout:setting min_mon_release = octopus 2024-03-22T07:06:35.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: set fsid to 9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:06:35.745 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-22T07:06:35.745 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:06:35.745 INFO:teuthology.orchestra.run.smithi063.stdout:Creating mon... 2024-03-22T07:06:36.700 INFO:teuthology.orchestra.run.smithi063.stdout:create mon.smithi063 on 2024-03-22T07:06:36.901 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-22T07:06:37.044 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-22T07:06:37.200 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715.target. 2024-03-22T07:06:37.200 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715.target. 2024-03-22T07:06:37.376 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mon.smithi063 2024-03-22T07:06:37.376 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mon.smithi063.service: Unit ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mon.smithi063.service not loaded. 2024-03-22T07:06:37.549 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715.target.wants/ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mon.smithi063.service → /etc/systemd/system/ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@.service. 2024-03-22T07:06:38.057 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T07:06:38.057 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-03-22T07:06:38.064 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T07:06:38.064 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-03-22T07:06:38.064 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-03-22T07:06:38.064 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to enable service . firewalld.service is not available 2024-03-22T07:06:38.065 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mon to start... 2024-03-22T07:06:38.065 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mon... 2024-03-22T07:06:39.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout cluster: 2024-03-22T07:06:39.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout id: 9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:06:39.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-22T07:06:39.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-03-22T07:06:39.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout services: 2024-03-22T07:06:39.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi063 (age 0.545291s) 2024-03-22T07:06:39.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-22T07:06:39.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-22T07:06:39.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-03-22T07:06:39.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout data: 2024-03-22T07:06:39.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-22T07:06:39.027 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-22T07:06:39.028 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-22T07:06:39.028 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout pgs: 2024-03-22T07:06:39.028 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-03-22T07:06:39.028 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.469+0000 7f81f5833700 1 Processor -- start 2024-03-22T07:06:39.028 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.469+0000 7f81f5833700 1 -- start start 2024-03-22T07:06:39.028 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.469+0000 7f81f5833700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81f01015d0 0x7f81f01019e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:39.028 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.469+0000 7f81f5833700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f0101f20 con 0x7f81f01015d0 2024-03-22T07:06:39.028 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.470+0000 7f81eeffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81f01015d0 0x7f81f01019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:39.028 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.470+0000 7f81eeffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81f01015d0 0x7f81f01019e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39878/0 (socket says 172.21.15.63:39878) 2024-03-22T07:06:39.028 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.470+0000 7f81eeffd700 1 -- 172.21.15.63:0/1352301355 learned_addr learned my addr 172.21.15.63:0/1352301355 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:06:39.028 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.470+0000 7f81eeffd700 1 -- 172.21.15.63:0/1352301355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f0102060 con 0x7f81f01015d0 2024-03-22T07:06:39.028 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.470+0000 7f81eeffd700 1 --2- 172.21.15.63:0/1352301355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81f01015d0 0x7f81f01019e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f81e00098e0 tx=0x7f81e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6523d040937f5b server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:39.028 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.470+0000 7f81edffb700 1 -- 172.21.15.63:0/1352301355 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81e000b860 con 0x7f81f01015d0 2024-03-22T07:06:39.028 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.470+0000 7f81edffb700 1 -- 172.21.15.63:0/1352301355 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e000b9c0 con 0x7f81f01015d0 2024-03-22T07:06:39.029 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.470+0000 7f81edffb700 1 -- 172.21.15.63:0/1352301355 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81e000bc90 con 0x7f81f01015d0 2024-03-22T07:06:39.029 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.471+0000 7f81f5833700 1 -- 172.21.15.63:0/1352301355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81f01015d0 msgr2=0x7f81f01019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:39.029 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.471+0000 7f81f5833700 1 --2- 172.21.15.63:0/1352301355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81f01015d0 0x7f81f01019e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f81e00098e0 tx=0x7f81e0009250 comp rx=0 tx=0).stop 2024-03-22T07:06:39.029 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.471+0000 7f81f5833700 1 -- 172.21.15.63:0/1352301355 shutdown_connections 2024-03-22T07:06:39.029 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.471+0000 7f81f5833700 1 --2- 172.21.15.63:0/1352301355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81f01015d0 0x7f81f01019e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:39.029 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.471+0000 7f81f5833700 1 -- 172.21.15.63:0/1352301355 >> 172.21.15.63:0/1352301355 conn(0x7f81f00fcd50 msgr2=0x7f81f00ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:39.029 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.471+0000 7f81f5833700 1 -- 172.21.15.63:0/1352301355 shutdown_connections 2024-03-22T07:06:39.029 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.471+0000 7f81f5833700 1 -- 172.21.15.63:0/1352301355 wait complete. 2024-03-22T07:06:39.029 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.472+0000 7f81f5833700 1 Processor -- start 2024-03-22T07:06:39.029 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.472+0000 7f81f5833700 1 -- start start 2024-03-22T07:06:39.029 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.472+0000 7f81f5833700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81f01015d0 0x7f81f0071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:39.029 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.472+0000 7f81f5833700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f0113a50 con 0x7f81f01015d0 2024-03-22T07:06:39.029 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.472+0000 7f81eeffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81f01015d0 0x7f81f0071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:39.029 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.472+0000 7f81eeffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81f01015d0 0x7f81f0071200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39892/0 (socket says 172.21.15.63:39892) 2024-03-22T07:06:39.030 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.472+0000 7f81eeffd700 1 -- 172.21.15.63:0/3303569365 learned_addr learned my addr 172.21.15.63:0/3303569365 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:06:39.030 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.473+0000 7f81eeffd700 1 -- 172.21.15.63:0/3303569365 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e00094f0 con 0x7f81f01015d0 2024-03-22T07:06:39.030 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.473+0000 7f81eeffd700 1 --2- 172.21.15.63:0/3303569365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81f01015d0 0x7f81f0071200 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f81e0003080 tx=0x7f81e0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:39.030 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.473+0000 7f81d77fe700 1 -- 172.21.15.63:0/3303569365 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81e001c5c0 con 0x7f81f01015d0 2024-03-22T07:06:39.030 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.473+0000 7f81d77fe700 1 -- 172.21.15.63:0/3303569365 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e001c720 con 0x7f81f01015d0 2024-03-22T07:06:39.030 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.473+0000 7f81d77fe700 1 -- 172.21.15.63:0/3303569365 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81e001ca30 con 0x7f81f01015d0 2024-03-22T07:06:39.030 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.473+0000 7f81f5833700 1 -- 172.21.15.63:0/3303569365 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f0113b90 con 0x7f81f01015d0 2024-03-22T07:06:39.030 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.473+0000 7f81f5833700 1 -- 172.21.15.63:0/3303569365 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f0114020 con 0x7f81f01015d0 2024-03-22T07:06:39.030 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.473+0000 7f81d77fe700 1 -- 172.21.15.63:0/3303569365 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f81e001a450 con 0x7f81f01015d0 2024-03-22T07:06:39.030 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.473+0000 7f81d77fe700 1 -- 172.21.15.63:0/3303569365 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f81e002e790 con 0x7f81f01015d0 2024-03-22T07:06:39.030 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.474+0000 7f81f5833700 1 -- 172.21.15.63:0/3303569365 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f005f5f0 con 0x7f81f01015d0 2024-03-22T07:06:39.030 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.476+0000 7f81d77fe700 1 -- 172.21.15.63:0/3303569365 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f81e001a890 con 0x7f81f01015d0 2024-03-22T07:06:39.030 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.548+0000 7f81f5833700 1 -- 172.21.15.63:0/3303569365 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f81f01093a0 con 0x7f81f01015d0 2024-03-22T07:06:39.030 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.548+0000 7f81d77fe700 1 -- 172.21.15.63:0/3303569365 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f81e001aa30 con 0x7f81f01015d0 2024-03-22T07:06:39.030 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.549+0000 7f81f5833700 1 -- 172.21.15.63:0/3303569365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81f01015d0 msgr2=0x7f81f0071200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:39.031 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.549+0000 7f81f5833700 1 --2- 172.21.15.63:0/3303569365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81f01015d0 0x7f81f0071200 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f81e0003080 tx=0x7f81e0016000 comp rx=0 tx=0).stop 2024-03-22T07:06:39.031 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.549+0000 7f81f5833700 1 -- 172.21.15.63:0/3303569365 shutdown_connections 2024-03-22T07:06:39.031 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.549+0000 7f81f5833700 1 --2- 172.21.15.63:0/3303569365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81f01015d0 0x7f81f0071200 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:39.031 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.549+0000 7f81f5833700 1 -- 172.21.15.63:0/3303569365 >> 172.21.15.63:0/3303569365 conn(0x7f81f00fcd50 msgr2=0x7f81f0104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:39.031 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.549+0000 7f81f5833700 1 -- 172.21.15.63:0/3303569365 shutdown_connections 2024-03-22T07:06:39.031 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:38.549+0000 7f81f5833700 1 -- 172.21.15.63:0/3303569365 wait complete. 2024-03-22T07:06:39.031 INFO:teuthology.orchestra.run.smithi063.stdout:mon is available 2024-03-22T07:06:39.031 INFO:teuthology.orchestra.run.smithi063.stdout:Assimilating anything we can from ceph.conf... 2024-03-22T07:06:40.133 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-03-22T07:06:40.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [global] 2024-03-22T07:06:40.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout fsid = 9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:06:40.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.63:3300,v1:172.21.15.63:6789] 2024-03-22T07:06:40.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-22T07:06:40.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-22T07:06:40.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-22T07:06:40.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-22T07:06:40.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-03-22T07:06:40.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-22T07:06:40.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-22T07:06:40.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-03-22T07:06:40.134 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [osd] 2024-03-22T07:06:40.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-22T07:06:40.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-22T07:06:40.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-22T07:06:40.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-22T07:06:40.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-22T07:06:40.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.435+0000 7f5787f8c700 1 Processor -- start 2024-03-22T07:06:40.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.435+0000 7f5787f8c700 1 -- start start 2024-03-22T07:06:40.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.435+0000 7f5787f8c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57801015b0 0x7f57801019c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:40.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.435+0000 7f5787f8c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5780101f00 con 0x7f57801015b0 2024-03-22T07:06:40.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.435+0000 7f5785d28700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57801015b0 0x7f57801019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:40.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.435+0000 7f5785d28700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57801015b0 0x7f57801019c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39908/0 (socket says 172.21.15.63:39908) 2024-03-22T07:06:40.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.435+0000 7f5785d28700 1 -- 172.21.15.63:0/4027393390 learned_addr learned my addr 172.21.15.63:0/4027393390 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:06:40.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.436+0000 7f5785d28700 1 -- 172.21.15.63:0/4027393390 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5780102040 con 0x7f57801015b0 2024-03-22T07:06:40.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.436+0000 7f5785d28700 1 --2- 172.21.15.63:0/4027393390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57801015b0 0x7f57801019c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f577c0098e0 tx=0x7f577c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be905f42f69ef96 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:40.135 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.436+0000 7f5784d26700 1 -- 172.21.15.63:0/4027393390 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f577c00b860 con 0x7f57801015b0 2024-03-22T07:06:40.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.436+0000 7f5784d26700 1 -- 172.21.15.63:0/4027393390 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577c00b9c0 con 0x7f57801015b0 2024-03-22T07:06:40.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.436+0000 7f5784d26700 1 -- 172.21.15.63:0/4027393390 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f577c00bc90 con 0x7f57801015b0 2024-03-22T07:06:40.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.436+0000 7f5787f8c700 1 -- 172.21.15.63:0/4027393390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57801015b0 msgr2=0x7f57801019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:40.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.436+0000 7f5787f8c700 1 --2- 172.21.15.63:0/4027393390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57801015b0 0x7f57801019c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f577c0098e0 tx=0x7f577c009250 comp rx=0 tx=0).stop 2024-03-22T07:06:40.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.437+0000 7f5787f8c700 1 -- 172.21.15.63:0/4027393390 shutdown_connections 2024-03-22T07:06:40.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.437+0000 7f5787f8c700 1 --2- 172.21.15.63:0/4027393390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57801015b0 0x7f57801019c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:40.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.437+0000 7f5787f8c700 1 -- 172.21.15.63:0/4027393390 >> 172.21.15.63:0/4027393390 conn(0x7f57800fcd50 msgr2=0x7f57800ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:40.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.437+0000 7f5787f8c700 1 -- 172.21.15.63:0/4027393390 shutdown_connections 2024-03-22T07:06:40.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.437+0000 7f5787f8c700 1 -- 172.21.15.63:0/4027393390 wait complete. 2024-03-22T07:06:40.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.437+0000 7f5787f8c700 1 Processor -- start 2024-03-22T07:06:40.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.437+0000 7f5787f8c700 1 -- start start 2024-03-22T07:06:40.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.437+0000 7f5787f8c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57801015b0 0x7f5780110d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:40.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.437+0000 7f5787f8c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5780111280 con 0x7f57801015b0 2024-03-22T07:06:40.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.437+0000 7f5785d28700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57801015b0 0x7f5780110d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:40.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.437+0000 7f5785d28700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57801015b0 0x7f5780110d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39924/0 (socket says 172.21.15.63:39924) 2024-03-22T07:06:40.136 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.437+0000 7f5785d28700 1 -- 172.21.15.63:0/2553636770 learned_addr learned my addr 172.21.15.63:0/2553636770 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:06:40.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.438+0000 7f5785d28700 1 -- 172.21.15.63:0/2553636770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c0094f0 con 0x7f57801015b0 2024-03-22T07:06:40.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.438+0000 7f5785d28700 1 --2- 172.21.15.63:0/2553636770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57801015b0 0x7f5780110d40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f577c003080 tx=0x7f577c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:40.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.438+0000 7f57767fc700 1 -- 172.21.15.63:0/2553636770 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f577c01c5c0 con 0x7f57801015b0 2024-03-22T07:06:40.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.438+0000 7f57767fc700 1 -- 172.21.15.63:0/2553636770 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577c01c720 con 0x7f57801015b0 2024-03-22T07:06:40.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.438+0000 7f5787f8c700 1 -- 172.21.15.63:0/2553636770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5780111480 con 0x7f57801015b0 2024-03-22T07:06:40.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.438+0000 7f57767fc700 1 -- 172.21.15.63:0/2553636770 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f577c01ca30 con 0x7f57801015b0 2024-03-22T07:06:40.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.438+0000 7f5787f8c700 1 -- 172.21.15.63:0/2553636770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5780111ff0 con 0x7f57801015b0 2024-03-22T07:06:40.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.438+0000 7f57767fc700 1 -- 172.21.15.63:0/2553636770 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f577c01a450 con 0x7f57801015b0 2024-03-22T07:06:40.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.438+0000 7f57767fc700 1 -- 172.21.15.63:0/2553636770 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f577c02e790 con 0x7f57801015b0 2024-03-22T07:06:40.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.439+0000 7f5787f8c700 1 -- 172.21.15.63:0/2553636770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f578005f5f0 con 0x7f57801015b0 2024-03-22T07:06:40.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.441+0000 7f57767fc700 1 -- 172.21.15.63:0/2553636770 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f577c01a890 con 0x7f57801015b0 2024-03-22T07:06:40.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.510+0000 7f5787f8c700 1 -- 172.21.15.63:0/2553636770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f5780102ad0 con 0x7f57801015b0 2024-03-22T07:06:40.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.514+0000 7f57767fc700 1 -- 172.21.15.63:0/2553636770 <== mon.0 v2:172.21.15.63:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f577c02ee80 con 0x7f57801015b0 2024-03-22T07:06:40.137 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.514+0000 7f57767fc700 1 -- 172.21.15.63:0/2553636770 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f5780102ad0 con 0x7f57801015b0 2024-03-22T07:06:40.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.515+0000 7f5787f8c700 1 -- 172.21.15.63:0/2553636770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57801015b0 msgr2=0x7f5780110d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:40.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.515+0000 7f5787f8c700 1 --2- 172.21.15.63:0/2553636770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57801015b0 0x7f5780110d40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f577c003080 tx=0x7f577c016000 comp rx=0 tx=0).stop 2024-03-22T07:06:40.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.515+0000 7f5787f8c700 1 -- 172.21.15.63:0/2553636770 shutdown_connections 2024-03-22T07:06:40.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.515+0000 7f5787f8c700 1 --2- 172.21.15.63:0/2553636770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57801015b0 0x7f5780110d40 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:40.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.515+0000 7f5787f8c700 1 -- 172.21.15.63:0/2553636770 >> 172.21.15.63:0/2553636770 conn(0x7f57800fcd50 msgr2=0x7f57800767d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:40.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.515+0000 7f5787f8c700 1 -- 172.21.15.63:0/2553636770 shutdown_connections 2024-03-22T07:06:40.138 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:39.515+0000 7f5787f8c700 1 -- 172.21.15.63:0/2553636770 wait complete. 2024-03-22T07:06:40.138 INFO:teuthology.orchestra.run.smithi063.stdout:Generating new minimal ceph.conf... 2024-03-22T07:06:41.117 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.572+0000 7f41c4bd8700 1 Processor -- start 2024-03-22T07:06:41.118 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.572+0000 7f41c4bd8700 1 -- start start 2024-03-22T07:06:41.118 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.572+0000 7f41c4bd8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41c0076b30 0x7f41c0105430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:41.118 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.572+0000 7f41c4bd8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c0105970 con 0x7f41c0076b30 2024-03-22T07:06:41.118 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.573+0000 7f41be59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41c0076b30 0x7f41c0105430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:41.118 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.573+0000 7f41be59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41c0076b30 0x7f41c0105430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39940/0 (socket says 172.21.15.63:39940) 2024-03-22T07:06:41.118 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.573+0000 7f41be59c700 1 -- 172.21.15.63:0/3878868380 learned_addr learned my addr 172.21.15.63:0/3878868380 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:06:41.122 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.573+0000 7f41be59c700 1 -- 172.21.15.63:0/3878868380 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c0105ab0 con 0x7f41c0076b30 2024-03-22T07:06:41.122 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.573+0000 7f41be59c700 1 --2- 172.21.15.63:0/3878868380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41c0076b30 0x7f41c0105430 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f41a80098e0 tx=0x7f41a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bd7207ee20ce28 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:41.122 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.573+0000 7f41bd59a700 1 -- 172.21.15.63:0/3878868380 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41a800b860 con 0x7f41c0076b30 2024-03-22T07:06:41.122 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.573+0000 7f41bd59a700 1 -- 172.21.15.63:0/3878868380 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f41a800be50 con 0x7f41c0076b30 2024-03-22T07:06:41.122 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.573+0000 7f41bd59a700 1 -- 172.21.15.63:0/3878868380 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41a8010380 con 0x7f41c0076b30 2024-03-22T07:06:41.123 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.574+0000 7f41c4bd8700 1 -- 172.21.15.63:0/3878868380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41c0076b30 msgr2=0x7f41c0105430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:41.123 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.574+0000 7f41c4bd8700 1 --2- 172.21.15.63:0/3878868380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41c0076b30 0x7f41c0105430 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f41a80098e0 tx=0x7f41a8009250 comp rx=0 tx=0).stop 2024-03-22T07:06:41.123 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.574+0000 7f41c4bd8700 1 -- 172.21.15.63:0/3878868380 shutdown_connections 2024-03-22T07:06:41.123 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.574+0000 7f41c4bd8700 1 --2- 172.21.15.63:0/3878868380 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41c0076b30 0x7f41c0105430 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:41.123 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.574+0000 7f41c4bd8700 1 -- 172.21.15.63:0/3878868380 >> 172.21.15.63:0/3878868380 conn(0x7f41c00fce40 msgr2=0x7f41c00ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:41.123 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.574+0000 7f41c4bd8700 1 -- 172.21.15.63:0/3878868380 shutdown_connections 2024-03-22T07:06:41.123 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.574+0000 7f41c4bd8700 1 -- 172.21.15.63:0/3878868380 wait complete. 2024-03-22T07:06:41.123 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.574+0000 7f41c4bd8700 1 Processor -- start 2024-03-22T07:06:41.123 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.574+0000 7f41c4bd8700 1 -- start start 2024-03-22T07:06:41.123 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.575+0000 7f41c4bd8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41c0076b30 0x7f41c0102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:41.123 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.575+0000 7f41c4bd8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c01032c0 con 0x7f41c0076b30 2024-03-22T07:06:41.123 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.575+0000 7f41be59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41c0076b30 0x7f41c0102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:41.123 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.575+0000 7f41be59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41c0076b30 0x7f41c0102d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39954/0 (socket says 172.21.15.63:39954) 2024-03-22T07:06:41.123 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.575+0000 7f41be59c700 1 -- 172.21.15.63:0/1525503747 learned_addr learned my addr 172.21.15.63:0/1525503747 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:06:41.124 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.575+0000 7f41be59c700 1 -- 172.21.15.63:0/1525503747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a80094f0 con 0x7f41c0076b30 2024-03-22T07:06:41.124 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.575+0000 7f41be59c700 1 --2- 172.21.15.63:0/1525503747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41c0076b30 0x7f41c0102d80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f41a80090b0 tx=0x7f41a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:41.124 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.575+0000 7f41b6ffd700 1 -- 172.21.15.63:0/1525503747 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41a8010760 con 0x7f41c0076b30 2024-03-22T07:06:41.124 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.575+0000 7f41c4bd8700 1 -- 172.21.15.63:0/1525503747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c0101330 con 0x7f41c0076b30 2024-03-22T07:06:41.124 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.575+0000 7f41b6ffd700 1 -- 172.21.15.63:0/1525503747 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f41a8010d50 con 0x7f41c0076b30 2024-03-22T07:06:41.124 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.575+0000 7f41b6ffd700 1 -- 172.21.15.63:0/1525503747 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41a801ace0 con 0x7f41c0076b30 2024-03-22T07:06:41.124 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.575+0000 7f41c4bd8700 1 -- 172.21.15.63:0/1525503747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c0101820 con 0x7f41c0076b30 2024-03-22T07:06:41.124 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.576+0000 7f41b6ffd700 1 -- 172.21.15.63:0/1525503747 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f41a802d670 con 0x7f41c0076b30 2024-03-22T07:06:41.124 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.576+0000 7f41b6ffd700 1 -- 172.21.15.63:0/1525503747 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f41a80247d0 con 0x7f41c0076b30 2024-03-22T07:06:41.124 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.576+0000 7f41c4bd8700 1 -- 172.21.15.63:0/1525503747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c004bed0 con 0x7f41c0076b30 2024-03-22T07:06:41.124 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.579+0000 7f41b6ffd700 1 -- 172.21.15.63:0/1525503747 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f41a8029070 con 0x7f41c0076b30 2024-03-22T07:06:41.124 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.649+0000 7f41c4bd8700 1 -- 172.21.15.63:0/1525503747 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f41c0101a60 con 0x7f41c0076b30 2024-03-22T07:06:41.124 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.649+0000 7f41b6ffd700 1 -- 172.21.15.63:0/1525503747 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f41a80108c0 con 0x7f41c0076b30 2024-03-22T07:06:41.124 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.650+0000 7f41c4bd8700 1 -- 172.21.15.63:0/1525503747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41c0076b30 msgr2=0x7f41c0102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:41.124 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.650+0000 7f41c4bd8700 1 --2- 172.21.15.63:0/1525503747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41c0076b30 0x7f41c0102d80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f41a80090b0 tx=0x7f41a8009250 comp rx=0 tx=0).stop 2024-03-22T07:06:41.125 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.650+0000 7f41c4bd8700 1 -- 172.21.15.63:0/1525503747 shutdown_connections 2024-03-22T07:06:41.125 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.650+0000 7f41c4bd8700 1 --2- 172.21.15.63:0/1525503747 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f41c0076b30 0x7f41c0102d80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:41.125 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.650+0000 7f41c4bd8700 1 -- 172.21.15.63:0/1525503747 >> 172.21.15.63:0/1525503747 conn(0x7f41c00fce40 msgr2=0x7f41c00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:41.125 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.650+0000 7f41c4bd8700 1 -- 172.21.15.63:0/1525503747 shutdown_connections 2024-03-22T07:06:41.125 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:40.650+0000 7f41c4bd8700 1 -- 172.21.15.63:0/1525503747 wait complete. 2024-03-22T07:06:41.125 INFO:teuthology.orchestra.run.smithi063.stdout:Restarting the monitor... 2024-03-22T07:06:41.368 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:41 smithi063 systemd[1]: Stopping Ceph mon.smithi063 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:06:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:41 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063[28825]: 2024-03-22T07:06:41.368+0000 7f2b3d6ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi063 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:06:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:41 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063[28825]: 2024-03-22T07:06:41.368+0000 7f2b3d6ea700 -1 mon.smithi063@0(leader) e1 *** Got Signal Terminated *** 2024-03-22T07:06:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:41 smithi063 podman[29073]: 2024-03-22 07:06:41.481091341 +0000 UTC m=+0.200865908 container died f329a04ff5f9d2ce3d52cc909bd5aa5d4494f671207bb131edd83a65b5313091 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-22T07:06:42.118 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:41 smithi063 podman[29073]: 2024-03-22 07:06:41.818289665 +0000 UTC m=+0.538064233 container cleanup f329a04ff5f9d2ce3d52cc909bd5aa5d4494f671207bb131edd83a65b5313091 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063, RELEASE=quincy-9c4e0c3, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-22T07:06:42.119 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:41 smithi063 bash[29073]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063 2024-03-22T07:06:42.119 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:41 smithi063 podman[29087]: 2024-03-22 07:06:41.994187646 +0000 UTC m=+0.508440217 container remove f329a04ff5f9d2ce3d52cc909bd5aa5d4494f671207bb131edd83a65b5313091 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=quincy-9c4e0c3, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-22T07:06:42.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mon.smithi063.service: Deactivated successfully. 2024-03-22T07:06:42.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 systemd[1]: Stopped Ceph mon.smithi063 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:06:42.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 systemd[1]: Starting Ceph mon.smithi063 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:06:42.684 INFO:teuthology.orchestra.run.smithi063.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-22T07:06:42.715 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 podman[29181]: 2024-03-22 07:06:42.445173212 +0000 UTC m=+0.110322691 container create f92a5bb6caeef0bdbdc34d9d2b881f196c82b11c21e6f81e8f225dea9312f40b (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=quincy-9c4e0c3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:06:42.715 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 podman[29181]: 2024-03-22 07:06:42.352057635 +0000 UTC m=+0.017207130 image pull 837124da804ffab28571a3e286aff1c534a701f228c0d3d7a309fcb128aca1de quay.ceph.io/ceph-ci/ceph:quincy 2024-03-22T07:06:42.715 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 podman[29181]: 2024-03-22 07:06:42.614573624 +0000 UTC m=+0.279723104 container init f92a5bb6caeef0bdbdc34d9d2b881f196c82b11c21e6f81e8f225dea9312f40b (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063, RELEASE=quincy-9c4e0c3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-22T07:06:42.715 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 podman[29181]: 2024-03-22 07:06:42.617288414 +0000 UTC m=+0.282437896 container start f92a5bb6caeef0bdbdc34d9d2b881f196c82b11c21e6f81e8f225dea9312f40b (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=quincy-9c4e0c3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-22T07:06:42.715 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: set uid:gid to 167:167 (ceph:ceph) 2024-03-22T07:06:42.715 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable), process ceph-mon, pid 2 2024-03-22T07:06:42.715 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: pidfile_write: ignore empty --pid-file 2024-03-22T07:06:42.715 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: load: jerasure load: lrc 2024-03-22T07:06:42.716 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: RocksDB version: 6.15.5 2024-03-22T07:06:42.716 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-03-22T07:06:42.716 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Compile date Mar 21 2024 2024-03-22T07:06:42.716 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: DB SUMMARY 2024-03-22T07:06:42.716 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: DB Session ID: TRG0X869EHG1DBJ138GP 2024-03-22T07:06:42.716 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: CURRENT file: CURRENT 2024-03-22T07:06:42.716 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: IDENTITY file: IDENTITY 2024-03-22T07:06:42.716 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-03-22T07:06:42.716 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi063/store.db dir, Total Num: 1, files: 000008.sst 2024-03-22T07:06:42.716 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi063/store.db: 000010.log size: 97282 ; 2024-03-22T07:06:42.716 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.error_if_exists: 0 2024-03-22T07:06:42.716 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.create_if_missing: 0 2024-03-22T07:06:42.716 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.paranoid_checks: 1 2024-03-22T07:06:42.717 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-22T07:06:42.717 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.env: 0x55d8d98aca80 2024-03-22T07:06:42.717 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.fs: Posix File System 2024-03-22T07:06:42.717 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.info_log: 0x55d8db7f5c00 2024-03-22T07:06:42.717 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_file_opening_threads: 16 2024-03-22T07:06:42.718 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.statistics: (nil) 2024-03-22T07:06:42.718 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.use_fsync: 0 2024-03-22T07:06:42.718 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_log_file_size: 0 2024-03-22T07:06:42.718 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-22T07:06:42.718 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-22T07:06:42.718 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.keep_log_file_num: 1000 2024-03-22T07:06:42.718 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.recycle_log_file_num: 0 2024-03-22T07:06:42.718 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.allow_fallocate: 1 2024-03-22T07:06:42.718 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.allow_mmap_reads: 0 2024-03-22T07:06:42.719 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.allow_mmap_writes: 0 2024-03-22T07:06:42.719 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.use_direct_reads: 0 2024-03-22T07:06:42.719 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-22T07:06:42.719 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.create_missing_column_families: 0 2024-03-22T07:06:42.719 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.db_log_dir: 2024-03-22T07:06:42.719 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi063/store.db 2024-03-22T07:06:42.719 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-22T07:06:42.719 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-22T07:06:42.719 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-22T07:06:42.719 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-22T07:06:42.719 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-22T07:06:42.719 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-22T07:06:42.719 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.advise_random_on_open: 1 2024-03-22T07:06:42.719 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.db_write_buffer_size: 0 2024-03-22T07:06:42.720 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.write_buffer_manager: 0x55d8dc56aab0 2024-03-22T07:06:42.720 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-22T07:06:42.720 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-03-22T07:06:42.720 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-22T07:06:42.720 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-22T07:06:42.720 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.rate_limiter: (nil) 2024-03-22T07:06:42.720 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-22T07:06:42.720 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.wal_recovery_mode: 2 2024-03-22T07:06:42.720 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.enable_thread_tracking: 0 2024-03-22T07:06:42.720 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.enable_pipelined_write: 0 2024-03-22T07:06:42.720 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.unordered_write: 0 2024-03-22T07:06:42.720 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-22T07:06:42.720 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-22T07:06:42.721 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-22T07:06:42.721 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-22T07:06:42.721 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.row_cache: None 2024-03-22T07:06:42.721 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.wal_filter: None 2024-03-22T07:06:42.721 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-22T07:06:42.721 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.allow_ingest_behind: 0 2024-03-22T07:06:42.721 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.preserve_deletes: 0 2024-03-22T07:06:42.721 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.two_write_queues: 0 2024-03-22T07:06:42.721 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.manual_wal_flush: 0 2024-03-22T07:06:42.721 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.atomic_flush: 0 2024-03-22T07:06:42.721 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-22T07:06:42.721 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-22T07:06:42.721 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-22T07:06:42.721 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.log_readahead_size: 0 2024-03-22T07:06:42.721 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-22T07:06:42.721 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.best_efforts_recovery: 0 2024-03-22T07:06:42.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-22T07:06:42.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-22T07:06:42.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.allow_data_in_errors: 0 2024-03-22T07:06:42.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.db_host_id: __hostname__ 2024-03-22T07:06:42.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_background_jobs: 2 2024-03-22T07:06:42.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_background_compactions: -1 2024-03-22T07:06:42.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_subcompactions: 1 2024-03-22T07:06:42.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-22T07:06:42.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-22T07:06:42.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-22T07:06:42.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_total_wal_size: 0 2024-03-22T07:06:42.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-22T07:06:42.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-22T07:06:42.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-22T07:06:42.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-22T07:06:42.722 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_open_files: -1 2024-03-22T07:06:42.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.bytes_per_sync: 0 2024-03-22T07:06:42.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-22T07:06:42.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-22T07:06:42.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compaction_readahead_size: 0 2024-03-22T07:06:42.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_background_flushes: -1 2024-03-22T07:06:42.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Compression algorithms supported: 2024-03-22T07:06:42.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-22T07:06:42.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: kZSTD supported: 0 2024-03-22T07:06:42.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: kXpressCompression supported: 0 2024-03-22T07:06:42.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: kLZ4HCCompression supported: 1 2024-03-22T07:06:42.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: kZlibCompression supported: 1 2024-03-22T07:06:42.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: kSnappyCompression supported: 1 2024-03-22T07:06:42.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: kLZ4Compression supported: 1 2024-03-22T07:06:42.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: kBZip2Compression supported: 0 2024-03-22T07:06:42.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-22T07:06:42.723 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi063/store.db/MANIFEST-000009 2024-03-22T07:06:42.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-03-22T07:06:42.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-22T07:06:42.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.merge_operator: 2024-03-22T07:06:42.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compaction_filter: None 2024-03-22T07:06:42.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compaction_filter_factory: None 2024-03-22T07:06:42.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.sst_partitioner_factory: None 2024-03-22T07:06:42.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-22T07:06:42.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-22T07:06:42.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d8db84af58) 2024-03-22T07:06:42.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: cache_index_and_filter_blocks: 1 2024-03-22T07:06:42.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-22T07:06:42.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-22T07:06:42.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: pin_top_level_index_and_filter: 1 2024-03-22T07:06:42.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: index_type: 0 2024-03-22T07:06:42.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: data_block_index_type: 0 2024-03-22T07:06:42.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: index_shortening: 1 2024-03-22T07:06:42.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-22T07:06:42.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: hash_index_allow_collision: 1 2024-03-22T07:06:42.725 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: checksum: 1 2024-03-22T07:06:42.725 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: no_block_cache: 0 2024-03-22T07:06:42.725 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache: 0x55d8db87d090 2024-03-22T07:06:42.725 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache_name: BinnedLRUCache 2024-03-22T07:06:42.725 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache_options: 2024-03-22T07:06:42.725 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: capacity : 536870912 2024-03-22T07:06:42.725 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: num_shard_bits : 4 2024-03-22T07:06:42.725 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: strict_capacity_limit : 0 2024-03-22T07:06:42.725 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: high_pri_pool_ratio: 0.000 2024-03-22T07:06:42.725 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache_compressed: (nil) 2024-03-22T07:06:42.725 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: persistent_cache: (nil) 2024-03-22T07:06:42.725 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_size: 4096 2024-03-22T07:06:42.725 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_size_deviation: 10 2024-03-22T07:06:42.725 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_restart_interval: 16 2024-03-22T07:06:42.725 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: index_block_restart_interval: 1 2024-03-22T07:06:42.725 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: metadata_block_size: 4096 2024-03-22T07:06:42.725 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: partition_filters: 0 2024-03-22T07:06:42.726 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: use_delta_encoding: 1 2024-03-22T07:06:42.726 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-03-22T07:06:42.726 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: whole_key_filtering: 1 2024-03-22T07:06:42.727 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: verify_compression: 0 2024-03-22T07:06:42.727 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: read_amp_bytes_per_bit: 0 2024-03-22T07:06:42.727 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: format_version: 4 2024-03-22T07:06:42.727 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: enable_index_compression: 1 2024-03-22T07:06:42.727 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_align: 0 2024-03-22T07:06:42.727 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.write_buffer_size: 33554432 2024-03-22T07:06:42.727 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_write_buffer_number: 2 2024-03-22T07:06:42.727 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compression: NoCompression 2024-03-22T07:06:42.727 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.bottommost_compression: Disabled 2024-03-22T07:06:42.728 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.prefix_extractor: nullptr 2024-03-22T07:06:42.728 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-22T07:06:42.728 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.num_levels: 7 2024-03-22T07:06:42.728 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-22T07:06:42.728 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-22T07:06:42.728 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-22T07:06:42.728 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-22T07:06:42.728 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-22T07:06:42.728 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-22T07:06:42.728 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-22T07:06:42.728 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-22T07:06:42.728 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-22T07:06:42.728 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-22T07:06:42.728 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-22T07:06:42.728 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compression_opts.level: 32767 2024-03-22T07:06:42.729 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compression_opts.strategy: 0 2024-03-22T07:06:42.729 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-22T07:06:42.729 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-22T07:06:42.729 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-22T07:06:42.729 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compression_opts.enabled: false 2024-03-22T07:06:42.729 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-22T07:06:42.729 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-22T07:06:42.729 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-22T07:06:42.729 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.target_file_size_base: 67108864 2024-03-22T07:06:42.729 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-22T07:06:42.729 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-22T07:06:42.729 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-22T07:06:42.729 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-22T07:06:42.729 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-22T07:06:42.729 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-22T07:06:42.729 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-22T07:06:42.729 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-22T07:06:42.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-22T07:06:42.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-22T07:06:42.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-22T07:06:42.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-22T07:06:42.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-22T07:06:42.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.arena_block_size: 4194304 2024-03-22T07:06:42.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-22T07:06:42.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-22T07:06:42.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-03-22T07:06:42.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.disable_auto_compactions: 0 2024-03-22T07:06:42.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-22T07:06:42.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-22T07:06:42.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-22T07:06:42.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-22T07:06:42.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-22T07:06:42.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-22T07:06:42.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-22T07:06:42.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-22T07:06:42.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-22T07:06:42.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-22T07:06:42.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.table_properties_collectors: 2024-03-22T07:06:42.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.inplace_update_support: 0 2024-03-22T07:06:42.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-22T07:06:42.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-22T07:06:42.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-22T07:06:42.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-22T07:06:42.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.bloom_locality: 0 2024-03-22T07:06:42.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.max_successive_merges: 0 2024-03-22T07:06:42.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-22T07:06:42.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.paranoid_file_checks: 0 2024-03-22T07:06:42.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.force_consistency_checks: 1 2024-03-22T07:06:42.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.report_bg_io_stats: 0 2024-03-22T07:06:42.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.ttl: 2592000 2024-03-22T07:06:42.732 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-22T07:06:42.732 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.enable_blob_files: false 2024-03-22T07:06:42.732 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.min_blob_size: 0 2024-03-22T07:06:42.732 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.blob_file_size: 268435456 2024-03-22T07:06:42.732 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-22T07:06:42.732 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-22T07:06:42.732 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-22T07:06:42.732 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi063/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-22T07:06:42.732 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-03-22T07:06:42.732 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-03-22T07:06:42.732 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711091202642855, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-03-22T07:06:42.732 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-03-22T07:06:42.732 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-03-22T07:06:42.732 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711091202643588, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 92554, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 90649, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13356, "raw_average_key_size": 49, "raw_value_size": 83364, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 269, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1711091202, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "02b32545-ecc8-447d-b56f-1f610bd5dcf1", "db_session_id": "TRG0X869EHG1DBJ138GP"}} 2024-03-22T07:06:42.732 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-03-22T07:06:42.732 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711091202643913, "job": 1, "event": "recovery_finished"} 2024-03-22T07:06:42.733 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi063/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-22T07:06:42.733 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55d8db811180 2024-03-22T07:06:42.733 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: DB pointer 0x55d8db8e6000 2024-03-22T07:06:42.733 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-03-22T07:06:42.733 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-03-22T07:06:42.733 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** DB Stats ** 2024-03-22T07:06:42.733 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T07:06:42.733 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-22T07:06:42.733 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T07:06:42.733 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T07:06:42.733 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-22T07:06:42.733 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-03-22T07:06:42.733 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T07:06:42.733 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-03-22T07:06:42.733 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** Compaction Stats [default] ** 2024-03-22T07:06:42.733 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-22T07:06:42.734 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T07:06:42.734 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: L0 2/0 92.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 178.3 0.00 0.00 1 0.000 0 0 2024-03-22T07:06:42.734 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Sum 2/0 92.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 178.3 0.00 0.00 1 0.000 0 0 2024-03-22T07:06:42.734 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 178.3 0.00 0.00 1 0.000 0 0 2024-03-22T07:06:42.734 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-03-22T07:06:42.734 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** Compaction Stats [default] ** 2024-03-22T07:06:42.734 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-22T07:06:42.734 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T07:06:42.734 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 178.3 0.00 0.00 1 0.000 0 0 2024-03-22T07:06:42.734 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T07:06:42.734 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-22T07:06:42.734 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-22T07:06:42.734 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-22T07:06:42.734 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-22T07:06:42.734 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-22T07:06:42.734 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative compaction: 0.00 GB write, 30.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T07:06:42.735 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval compaction: 0.00 GB write, 30.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T07:06:42.735 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-22T07:06:42.735 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-03-22T07:06:42.735 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-22T07:06:42.735 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-03-22T07:06:42.735 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** Compaction Stats [default] ** 2024-03-22T07:06:42.735 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-22T07:06:42.735 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T07:06:42.735 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: L0 2/0 92.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 178.3 0.00 0.00 1 0.000 0 0 2024-03-22T07:06:42.735 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Sum 2/0 92.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 178.3 0.00 0.00 1 0.000 0 0 2024-03-22T07:06:42.735 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-03-22T07:06:42.735 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-03-22T07:06:42.735 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** Compaction Stats [default] ** 2024-03-22T07:06:42.735 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-22T07:06:42.735 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T07:06:42.735 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 178.3 0.00 0.00 1 0.000 0 0 2024-03-22T07:06:42.735 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T07:06:42.736 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-22T07:06:42.736 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-22T07:06:42.736 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-22T07:06:42.736 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-22T07:06:42.736 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-22T07:06:42.736 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative compaction: 0.00 GB write, 30.05 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T07:06:42.736 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T07:06:42.736 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-22T07:06:42.736 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-03-22T07:06:42.736 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-22T07:06:42.736 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: starting mon.smithi063 rank 0 at public addrs [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] at bind addrs [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi063 fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:06:42.736 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: mon.smithi063@-1(???) e1 preinit fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:06:42.737 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: mon.smithi063@-1(???).mds e1 new map 2024-03-22T07:06:42.737 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: mon.smithi063@-1(???).mds e1 print_map 2024-03-22T07:06:42.737 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: e1 2024-03-22T07:06:42.737 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:06:42.737 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:06:42.737 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: legacy client fscid: -1 2024-03-22T07:06:42.737 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-03-22T07:06:42.737 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: No filesystems configured 2024-03-22T07:06:42.737 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: mon.smithi063@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-22T07:06:42.737 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: mon.smithi063@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-22T07:06:42.737 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: mon.smithi063@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-22T07:06:42.737 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: mon.smithi063@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-22T07:06:42.737 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: mon.smithi063@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-22T07:06:42.737 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 bash[29181]: f92a5bb6caeef0bdbdc34d9d2b881f196c82b11c21e6f81e8f225dea9312f40b 2024-03-22T07:06:42.737 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 systemd[1]: Started Ceph mon.smithi063 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:06:42.737 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: mon.smithi063 is new leader, mons smithi063 in quorum (ranks 0) 2024-03-22T07:06:42.738 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: monmap e1: 1 mons at {smithi063=[v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T07:06:42.738 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: fsmap 2024-03-22T07:06:42.738 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: osdmap e1: 0 total, 0 up, 0 in 2024-03-22T07:06:42.738 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:42 smithi063 ceph-mon[29200]: mgrmap e1: no daemons active 2024-03-22T07:06:43.773 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.142+0000 7f6d96c0d700 1 Processor -- start 2024-03-22T07:06:43.778 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.142+0000 7f6d96c0d700 1 -- start start 2024-03-22T07:06:43.778 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.142+0000 7f6d96c0d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90102770 0x7f6d90102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:43.778 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.142+0000 7f6d96c0d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d901030c0 con 0x7f6d90102770 2024-03-22T07:06:43.778 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.143+0000 7f6d949a9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90102770 0x7f6d90102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:43.778 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.143+0000 7f6d949a9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90102770 0x7f6d90102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39966/0 (socket says 172.21.15.63:39966) 2024-03-22T07:06:43.778 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.143+0000 7f6d949a9700 1 -- 172.21.15.63:0/1117003428 learned_addr learned my addr 172.21.15.63:0/1117003428 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:06:43.778 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.143+0000 7f6d949a9700 1 -- 172.21.15.63:0/1117003428 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d90103200 con 0x7f6d90102770 2024-03-22T07:06:43.778 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.143+0000 7f6d949a9700 1 --2- 172.21.15.63:0/1117003428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90102770 0x7f6d90102b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6d840098e0 tx=0x7f6d84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1e3ad18eee4c server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:43.778 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.143+0000 7f6d8f7fe700 1 -- 172.21.15.63:0/1117003428 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d8400b860 con 0x7f6d90102770 2024-03-22T07:06:43.779 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.143+0000 7f6d8f7fe700 1 -- 172.21.15.63:0/1117003428 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f6d8400be50 con 0x7f6d90102770 2024-03-22T07:06:43.779 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.143+0000 7f6d8f7fe700 1 -- 172.21.15.63:0/1117003428 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d84010380 con 0x7f6d90102770 2024-03-22T07:06:43.779 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.144+0000 7f6d96c0d700 1 -- 172.21.15.63:0/1117003428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90102770 msgr2=0x7f6d90102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:43.779 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.144+0000 7f6d96c0d700 1 --2- 172.21.15.63:0/1117003428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90102770 0x7f6d90102b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6d840098e0 tx=0x7f6d84009250 comp rx=0 tx=0).stop 2024-03-22T07:06:43.779 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.144+0000 7f6d96c0d700 1 -- 172.21.15.63:0/1117003428 shutdown_connections 2024-03-22T07:06:43.779 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.144+0000 7f6d96c0d700 1 --2- 172.21.15.63:0/1117003428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90102770 0x7f6d90102b80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:43.779 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.144+0000 7f6d96c0d700 1 -- 172.21.15.63:0/1117003428 >> 172.21.15.63:0/1117003428 conn(0x7f6d900fde90 msgr2=0x7f6d90100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:43.779 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.144+0000 7f6d96c0d700 1 -- 172.21.15.63:0/1117003428 shutdown_connections 2024-03-22T07:06:43.779 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.144+0000 7f6d96c0d700 1 -- 172.21.15.63:0/1117003428 wait complete. 2024-03-22T07:06:43.779 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.144+0000 7f6d96c0d700 1 Processor -- start 2024-03-22T07:06:43.779 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.144+0000 7f6d96c0d700 1 -- start start 2024-03-22T07:06:43.779 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.145+0000 7f6d96c0d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90102770 0x7f6d90078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:43.779 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.145+0000 7f6d96c0d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d90079110 con 0x7f6d90102770 2024-03-22T07:06:43.779 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.145+0000 7f6d949a9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90102770 0x7f6d90078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:43.780 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.145+0000 7f6d949a9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90102770 0x7f6d90078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39972/0 (socket says 172.21.15.63:39972) 2024-03-22T07:06:43.780 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.145+0000 7f6d949a9700 1 -- 172.21.15.63:0/2901875930 learned_addr learned my addr 172.21.15.63:0/2901875930 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:06:43.780 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.145+0000 7f6d949a9700 1 -- 172.21.15.63:0/2901875930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d840094f0 con 0x7f6d90102770 2024-03-22T07:06:43.780 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.145+0000 7f6d949a9700 1 --2- 172.21.15.63:0/2901875930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90102770 0x7f6d90078bd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400b440 tx=0x7f6d84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:43.780 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.145+0000 7f6d8d7fa700 1 -- 172.21.15.63:0/2901875930 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d84010760 con 0x7f6d90102770 2024-03-22T07:06:43.780 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.145+0000 7f6d8d7fa700 1 -- 172.21.15.63:0/2901875930 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f6d84010d50 con 0x7f6d90102770 2024-03-22T07:06:43.780 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.145+0000 7f6d96c0d700 1 -- 172.21.15.63:0/2901875930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d90077180 con 0x7f6d90102770 2024-03-22T07:06:43.780 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.145+0000 7f6d8d7fa700 1 -- 172.21.15.63:0/2901875930 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d8401ae20 con 0x7f6d90102770 2024-03-22T07:06:43.780 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.145+0000 7f6d96c0d700 1 -- 172.21.15.63:0/2901875930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d90077670 con 0x7f6d90102770 2024-03-22T07:06:43.780 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.146+0000 7f6d8d7fa700 1 -- 172.21.15.63:0/2901875930 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f6d8402e6c0 con 0x7f6d90102770 2024-03-22T07:06:43.780 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.146+0000 7f6d8d7fa700 1 -- 172.21.15.63:0/2901875930 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f6d84024950 con 0x7f6d90102770 2024-03-22T07:06:43.780 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.146+0000 7f6d96c0d700 1 -- 172.21.15.63:0/2901875930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d9004bed0 con 0x7f6d90102770 2024-03-22T07:06:43.780 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.148+0000 7f6d8d7fa700 1 -- 172.21.15.63:0/2901875930 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f6d84029070 con 0x7f6d90102770 2024-03-22T07:06:43.780 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.217+0000 7f6d96c0d700 1 -- 172.21.15.63:0/2901875930 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f6d900778b0 con 0x7f6d90102770 2024-03-22T07:06:43.780 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.221+0000 7f6d8d7fa700 1 -- 172.21.15.63:0/2901875930 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6d8401a6a0 con 0x7f6d90102770 2024-03-22T07:06:43.781 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.222+0000 7f6d96c0d700 1 -- 172.21.15.63:0/2901875930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90102770 msgr2=0x7f6d90078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:43.781 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.222+0000 7f6d96c0d700 1 --2- 172.21.15.63:0/2901875930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90102770 0x7f6d90078bd0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400b440 tx=0x7f6d84009250 comp rx=0 tx=0).stop 2024-03-22T07:06:43.781 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.222+0000 7f6d96c0d700 1 -- 172.21.15.63:0/2901875930 shutdown_connections 2024-03-22T07:06:43.781 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.222+0000 7f6d96c0d700 1 --2- 172.21.15.63:0/2901875930 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6d90102770 0x7f6d90078bd0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:43.781 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.222+0000 7f6d96c0d700 1 -- 172.21.15.63:0/2901875930 >> 172.21.15.63:0/2901875930 conn(0x7f6d900fde90 msgr2=0x7f6d900feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:43.781 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.222+0000 7f6d96c0d700 1 -- 172.21.15.63:0/2901875930 shutdown_connections 2024-03-22T07:06:43.781 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:43.222+0000 7f6d96c0d700 1 -- 172.21.15.63:0/2901875930 wait complete. 2024-03-22T07:06:43.781 INFO:teuthology.orchestra.run.smithi063.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-22T07:06:43.781 INFO:teuthology.orchestra.run.smithi063.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-22T07:06:43.781 INFO:teuthology.orchestra.run.smithi063.stdout:Creating mgr... 2024-03-22T07:06:43.781 INFO:teuthology.orchestra.run.smithi063.stdout:Verifying port 9283 ... 2024-03-22T07:06:43.948 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mgr.smithi063.mnxayh 2024-03-22T07:06:43.949 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mgr.smithi063.mnxayh.service: Unit ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mgr.smithi063.mnxayh.service not loaded. 2024-03-22T07:06:44.094 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715.target.wants/ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mgr.smithi063.mnxayh.service → /etc/systemd/system/ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@.service. 2024-03-22T07:06:44.480 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:44 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2901875930' entity='client.admin' 2024-03-22T07:06:44.638 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T07:06:44.638 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-03-22T07:06:44.645 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T07:06:44.645 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-03-22T07:06:44.645 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-03-22T07:06:44.645 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to enable service . firewalld.service is not available 2024-03-22T07:06:44.654 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T07:06:44.654 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-03-22T07:06:44.662 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T07:06:44.662 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-03-22T07:06:44.662 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-03-22T07:06:44.662 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-03-22T07:06:44.662 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr to start... 2024-03-22T07:06:44.662 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr... 2024-03-22T07:06:45.455 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:45 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2384595598' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T07:06:45.709 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-03-22T07:06:45.711 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-03-22T07:06:45.711 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "9a0ba9d6-e81a-11ee-95cd-87774f69a715", 2024-03-22T07:06:45.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T07:06:45.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T07:06:45.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T07:06:45.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T07:06:45.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:45.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T07:06:45.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T07:06:45.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-03-22T07:06:45.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-03-22T07:06:45.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T07:06:45.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "smithi063" 2024-03-22T07:06:45.712 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-03-22T07:06:45.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-22T07:06:45.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T07:06:45.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T07:06:45.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-22T07:06:45.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T07:06:45.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:45.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T07:06:45.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T07:06:45.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T07:06:45.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T07:06:45.713 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T07:06:45.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T07:06:45.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T07:06:45.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T07:06:45.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:45.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T07:06:45.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T07:06:45.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T07:06:45.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T07:06:45.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T07:06:45.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T07:06:45.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T07:06:45.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T07:06:45.722 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T07:06:45.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:45.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T07:06:45.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T07:06:45.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T07:06:45.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T07:06:45.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:45.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T07:06:45.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-22T07:06:45.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T07:06:45.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T07:06:45.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T07:06:45.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T07:06:45.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T07:06:45.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-03-22T07:06:45.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T07:06:45.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:45.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T07:06:45.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T07:06:45.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T07:06:38.005342+0000", 2024-03-22T07:06:45.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T07:06:45.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:45.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T07:06:45.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-03-22T07:06:45.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.076+0000 7f013a427700 1 Processor -- start 2024-03-22T07:06:45.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.076+0000 7f013a427700 1 -- start start 2024-03-22T07:06:45.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.076+0000 7f013a427700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0134100b80 0x7f0134100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:45.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.076+0000 7f013a427700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01341014f0 con 0x7f0134100b80 2024-03-22T07:06:45.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.076+0000 7f0133fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0134100b80 0x7f0134100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:45.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.076+0000 7f0133fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0134100b80 0x7f0134100fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43514/0 (socket says 172.21.15.63:43514) 2024-03-22T07:06:45.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.076+0000 7f0133fff700 1 -- 172.21.15.63:0/2136942450 learned_addr learned my addr 172.21.15.63:0/2136942450 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:06:45.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.076+0000 7f0133fff700 1 -- 172.21.15.63:0/2136942450 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0134101630 con 0x7f0134100b80 2024-03-22T07:06:45.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.077+0000 7f0133fff700 1 --2- 172.21.15.63:0/2136942450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0134100b80 0x7f0134100fb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f01240098e0 tx=0x7f0124009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b326e140ae83e319 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:45.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.077+0000 7f0132ffd700 1 -- 172.21.15.63:0/2136942450 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f012400b860 con 0x7f0134100b80 2024-03-22T07:06:45.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.077+0000 7f0132ffd700 1 -- 172.21.15.63:0/2136942450 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f012400be50 con 0x7f0134100b80 2024-03-22T07:06:45.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.077+0000 7f0132ffd700 1 -- 172.21.15.63:0/2136942450 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0124010380 con 0x7f0134100b80 2024-03-22T07:06:45.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.077+0000 7f013a427700 1 -- 172.21.15.63:0/2136942450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0134100b80 msgr2=0x7f0134100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:45.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.077+0000 7f013a427700 1 --2- 172.21.15.63:0/2136942450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0134100b80 0x7f0134100fb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f01240098e0 tx=0x7f0124009250 comp rx=0 tx=0).stop 2024-03-22T07:06:45.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.078+0000 7f013a427700 1 -- 172.21.15.63:0/2136942450 shutdown_connections 2024-03-22T07:06:45.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.078+0000 7f013a427700 1 --2- 172.21.15.63:0/2136942450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0134100b80 0x7f0134100fb0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:45.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.078+0000 7f013a427700 1 -- 172.21.15.63:0/2136942450 >> 172.21.15.63:0/2136942450 conn(0x7f01340fc560 msgr2=0x7f01340fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:45.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.078+0000 7f013a427700 1 -- 172.21.15.63:0/2136942450 shutdown_connections 2024-03-22T07:06:45.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.078+0000 7f013a427700 1 -- 172.21.15.63:0/2136942450 wait complete. 2024-03-22T07:06:45.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.078+0000 7f013a427700 1 Processor -- start 2024-03-22T07:06:45.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.078+0000 7f013a427700 1 -- start start 2024-03-22T07:06:45.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.078+0000 7f013a427700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0134100b80 0x7f01341035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:45.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.079+0000 7f013a427700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0134103af0 con 0x7f0134100b80 2024-03-22T07:06:45.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.079+0000 7f0133fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0134100b80 0x7f01341035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:45.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.079+0000 7f0133fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0134100b80 0x7f01341035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43526/0 (socket says 172.21.15.63:43526) 2024-03-22T07:06:45.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.079+0000 7f0133fff700 1 -- 172.21.15.63:0/2384595598 learned_addr learned my addr 172.21.15.63:0/2384595598 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:06:45.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.079+0000 7f0133fff700 1 -- 172.21.15.63:0/2384595598 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01240094f0 con 0x7f0134100b80 2024-03-22T07:06:45.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.079+0000 7f0133fff700 1 --2- 172.21.15.63:0/2384595598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0134100b80 0x7f01341035b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0124011bd0 tx=0x7f0124009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:45.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.079+0000 7f0130ff9700 1 -- 172.21.15.63:0/2384595598 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0124010760 con 0x7f0134100b80 2024-03-22T07:06:45.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.079+0000 7f0130ff9700 1 -- 172.21.15.63:0/2384595598 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0124010d50 con 0x7f0134100b80 2024-03-22T07:06:45.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.079+0000 7f0130ff9700 1 -- 172.21.15.63:0/2384595598 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f012401ad30 con 0x7f0134100b80 2024-03-22T07:06:45.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.079+0000 7f013a427700 1 -- 172.21.15.63:0/2384595598 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0134101b60 con 0x7f0134100b80 2024-03-22T07:06:45.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.079+0000 7f013a427700 1 -- 172.21.15.63:0/2384595598 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0134102050 con 0x7f0134100b80 2024-03-22T07:06:45.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.080+0000 7f0130ff9700 1 -- 172.21.15.63:0/2384595598 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f012402d670 con 0x7f0134100b80 2024-03-22T07:06:45.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.080+0000 7f0130ff9700 1 -- 172.21.15.63:0/2384595598 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f01240247d0 con 0x7f0134100b80 2024-03-22T07:06:45.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.080+0000 7f013a427700 1 -- 172.21.15.63:0/2384595598 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f013404bed0 con 0x7f0134100b80 2024-03-22T07:06:45.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.083+0000 7f0130ff9700 1 -- 172.21.15.63:0/2384595598 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f0124029070 con 0x7f0134100b80 2024-03-22T07:06:45.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.156+0000 7f013a427700 1 -- 172.21.15.63:0/2384595598 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0134102290 con 0x7f0134100b80 2024-03-22T07:06:45.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.156+0000 7f0130ff9700 1 -- 172.21.15.63:0/2384595598 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f012401a430 con 0x7f0134100b80 2024-03-22T07:06:45.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.157+0000 7f013a427700 1 -- 172.21.15.63:0/2384595598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0134100b80 msgr2=0x7f01341035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:45.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.157+0000 7f013a427700 1 --2- 172.21.15.63:0/2384595598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0134100b80 0x7f01341035b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0124011bd0 tx=0x7f0124009250 comp rx=0 tx=0).stop 2024-03-22T07:06:45.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.157+0000 7f013a427700 1 -- 172.21.15.63:0/2384595598 shutdown_connections 2024-03-22T07:06:45.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.157+0000 7f013a427700 1 --2- 172.21.15.63:0/2384595598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0134100b80 0x7f01341035b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:45.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.157+0000 7f013a427700 1 -- 172.21.15.63:0/2384595598 >> 172.21.15.63:0/2384595598 conn(0x7f01340fc560 msgr2=0x7f01340fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:45.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.158+0000 7f013a427700 1 -- 172.21.15.63:0/2384595598 shutdown_connections 2024-03-22T07:06:45.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:45.158+0000 7f013a427700 1 -- 172.21.15.63:0/2384595598 wait complete. 2024-03-22T07:06:45.727 INFO:teuthology.orchestra.run.smithi063.stdout:mgr not available, waiting (1/15)... 2024-03-22T07:06:48.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:48 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3167403009' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T07:06:48.836 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-03-22T07:06:48.837 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-03-22T07:06:48.837 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "9a0ba9d6-e81a-11ee-95cd-87774f69a715", 2024-03-22T07:06:48.837 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T07:06:48.837 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T07:06:48.837 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T07:06:48.837 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T07:06:48.838 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:48.838 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T07:06:48.838 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T07:06:48.838 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-03-22T07:06:48.838 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-03-22T07:06:48.839 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T07:06:48.839 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "smithi063" 2024-03-22T07:06:48.839 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-03-22T07:06:48.839 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-22T07:06:48.839 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T07:06:48.839 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T07:06:48.839 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-22T07:06:48.839 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T07:06:48.839 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:48.840 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T07:06:48.840 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T07:06:48.840 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T07:06:48.840 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T07:06:48.840 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T07:06:48.840 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T07:06:48.840 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T07:06:48.840 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T07:06:48.840 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:48.840 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T07:06:48.840 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T07:06:48.840 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T07:06:48.840 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T07:06:48.840 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T07:06:48.840 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T07:06:48.841 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T07:06:48.841 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T07:06:48.841 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T07:06:48.841 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:48.841 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T07:06:48.841 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T07:06:48.841 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T07:06:48.841 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T07:06:48.841 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:48.841 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T07:06:48.841 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-22T07:06:48.841 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T07:06:48.841 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T07:06:48.841 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T07:06:48.841 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T07:06:48.842 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T07:06:48.842 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-03-22T07:06:48.842 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T07:06:48.842 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:48.842 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T07:06:48.842 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T07:06:48.842 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T07:06:38.005342+0000", 2024-03-22T07:06:48.842 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T07:06:48.842 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:48.842 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T07:06:48.842 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-03-22T07:06:48.842 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.139+0000 7fa0cbab7700 1 Processor -- start 2024-03-22T07:06:48.842 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.139+0000 7fa0cbab7700 1 -- start start 2024-03-22T07:06:48.842 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.140+0000 7fa0cbab7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0c4076b70 0x7fa0c41053f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:48.842 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.140+0000 7fa0cbab7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4105930 con 0x7fa0c4076b70 2024-03-22T07:06:48.843 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.140+0000 7fa0c9853700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0c4076b70 0x7fa0c41053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:48.843 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.140+0000 7fa0c9853700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0c4076b70 0x7fa0c41053f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43536/0 (socket says 172.21.15.63:43536) 2024-03-22T07:06:48.843 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.140+0000 7fa0c9853700 1 -- 172.21.15.63:0/2698343684 learned_addr learned my addr 172.21.15.63:0/2698343684 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:06:48.843 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.140+0000 7fa0c9853700 1 -- 172.21.15.63:0/2698343684 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c4105a70 con 0x7fa0c4076b70 2024-03-22T07:06:48.843 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.140+0000 7fa0c9853700 1 --2- 172.21.15.63:0/2698343684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0c4076b70 0x7fa0c41053f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00098e0 tx=0x7fa0c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90899c4f32c5fba3 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:48.843 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.140+0000 7fa0c8851700 1 -- 172.21.15.63:0/2698343684 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0c000b860 con 0x7fa0c4076b70 2024-03-22T07:06:48.843 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.140+0000 7fa0c8851700 1 -- 172.21.15.63:0/2698343684 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa0c000be50 con 0x7fa0c4076b70 2024-03-22T07:06:48.843 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.141+0000 7fa0c8851700 1 -- 172.21.15.63:0/2698343684 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0c0010380 con 0x7fa0c4076b70 2024-03-22T07:06:48.843 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.141+0000 7fa0cbab7700 1 -- 172.21.15.63:0/2698343684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0c4076b70 msgr2=0x7fa0c41053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:48.843 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.141+0000 7fa0cbab7700 1 --2- 172.21.15.63:0/2698343684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0c4076b70 0x7fa0c41053f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00098e0 tx=0x7fa0c0009250 comp rx=0 tx=0).stop 2024-03-22T07:06:48.843 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.141+0000 7fa0cbab7700 1 -- 172.21.15.63:0/2698343684 shutdown_connections 2024-03-22T07:06:48.843 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.141+0000 7fa0cbab7700 1 --2- 172.21.15.63:0/2698343684 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0c4076b70 0x7fa0c41053f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.843 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.141+0000 7fa0cbab7700 1 -- 172.21.15.63:0/2698343684 >> 172.21.15.63:0/2698343684 conn(0x7fa0c40fce40 msgr2=0x7fa0c40ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:48.843 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.141+0000 7fa0cbab7700 1 -- 172.21.15.63:0/2698343684 shutdown_connections 2024-03-22T07:06:48.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.141+0000 7fa0cbab7700 1 -- 172.21.15.63:0/2698343684 wait complete. 2024-03-22T07:06:48.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.142+0000 7fa0cbab7700 1 Processor -- start 2024-03-22T07:06:48.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.142+0000 7fa0cbab7700 1 -- start start 2024-03-22T07:06:48.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.142+0000 7fa0cbab7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0c4076b70 0x7fa0c41115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:48.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.142+0000 7fa0cbab7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4111b10 con 0x7fa0c4076b70 2024-03-22T07:06:48.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.142+0000 7fa0c9853700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0c4076b70 0x7fa0c41115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:48.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.142+0000 7fa0c9853700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0c4076b70 0x7fa0c41115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43538/0 (socket says 172.21.15.63:43538) 2024-03-22T07:06:48.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.142+0000 7fa0c9853700 1 -- 172.21.15.63:0/3167403009 learned_addr learned my addr 172.21.15.63:0/3167403009 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:06:48.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.142+0000 7fa0c9853700 1 -- 172.21.15.63:0/3167403009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c00094f0 con 0x7fa0c4076b70 2024-03-22T07:06:48.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.142+0000 7fa0c9853700 1 --2- 172.21.15.63:0/3167403009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0c4076b70 0x7fa0c41115d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00090b0 tx=0x7fa0c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:48.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.143+0000 7fa0ba7fc700 1 -- 172.21.15.63:0/3167403009 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0c0010760 con 0x7fa0c4076b70 2024-03-22T07:06:48.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.143+0000 7fa0ba7fc700 1 -- 172.21.15.63:0/3167403009 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa0c0010d50 con 0x7fa0c4076b70 2024-03-22T07:06:48.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.143+0000 7fa0ba7fc700 1 -- 172.21.15.63:0/3167403009 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0c001ace0 con 0x7fa0c4076b70 2024-03-22T07:06:48.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.143+0000 7fa0cbab7700 1 -- 172.21.15.63:0/3167403009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c4111d10 con 0x7fa0c4076b70 2024-03-22T07:06:48.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.143+0000 7fa0cbab7700 1 -- 172.21.15.63:0/3167403009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c4112880 con 0x7fa0c4076b70 2024-03-22T07:06:48.844 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.143+0000 7fa0ba7fc700 1 -- 172.21.15.63:0/3167403009 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fa0c0024670 con 0x7fa0c4076b70 2024-03-22T07:06:48.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.143+0000 7fa0ba7fc700 1 -- 172.21.15.63:0/3167403009 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fa0c0024c70 con 0x7fa0c4076b70 2024-03-22T07:06:48.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.144+0000 7fa0cbab7700 1 -- 172.21.15.63:0/3167403009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c405f5f0 con 0x7fa0c4076b70 2024-03-22T07:06:48.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.146+0000 7fa0ba7fc700 1 -- 172.21.15.63:0/3167403009 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fa0c0029070 con 0x7fa0c4076b70 2024-03-22T07:06:48.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.218+0000 7fa0cbab7700 1 -- 172.21.15.63:0/3167403009 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa0c404bed0 con 0x7fa0c4076b70 2024-03-22T07:06:48.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.218+0000 7fa0ba7fc700 1 -- 172.21.15.63:0/3167403009 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fa0c0010ec0 con 0x7fa0c4076b70 2024-03-22T07:06:48.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.219+0000 7fa0cbab7700 1 -- 172.21.15.63:0/3167403009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0c4076b70 msgr2=0x7fa0c41115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:48.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.219+0000 7fa0cbab7700 1 --2- 172.21.15.63:0/3167403009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0c4076b70 0x7fa0c41115d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00090b0 tx=0x7fa0c0009250 comp rx=0 tx=0).stop 2024-03-22T07:06:48.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.219+0000 7fa0cbab7700 1 -- 172.21.15.63:0/3167403009 shutdown_connections 2024-03-22T07:06:48.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.219+0000 7fa0cbab7700 1 --2- 172.21.15.63:0/3167403009 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa0c4076b70 0x7fa0c41115d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:48.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.219+0000 7fa0cbab7700 1 -- 172.21.15.63:0/3167403009 >> 172.21.15.63:0/3167403009 conn(0x7fa0c40fce40 msgr2=0x7fa0c40fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:48.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.219+0000 7fa0cbab7700 1 -- 172.21.15.63:0/3167403009 shutdown_connections 2024-03-22T07:06:48.845 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:48.219+0000 7fa0cbab7700 1 -- 172.21.15.63:0/3167403009 wait complete. 2024-03-22T07:06:48.845 INFO:teuthology.orchestra.run.smithi063.stdout:mgr not available, waiting (2/15)... 2024-03-22T07:06:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:51 smithi063 ceph-mon[29200]: Activating manager daemon smithi063.mnxayh 2024-03-22T07:06:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:51 smithi063 ceph-mon[29200]: mgrmap e2: smithi063.mnxayh(active, starting, since 0.00290847s) 2024-03-22T07:06:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:51 smithi063 ceph-mon[29200]: from='mgr.14100 172.21.15.63:0/2726337620' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:06:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:51 smithi063 ceph-mon[29200]: from='mgr.14100 172.21.15.63:0/2726337620' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:06:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:51 smithi063 ceph-mon[29200]: from='mgr.14100 172.21.15.63:0/2726337620' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:06:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:51 smithi063 ceph-mon[29200]: from='mgr.14100 172.21.15.63:0/2726337620' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:06:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:51 smithi063 ceph-mon[29200]: from='mgr.14100 172.21.15.63:0/2726337620' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr metadata", "who": "smithi063.mnxayh", "id": "smithi063.mnxayh"}]: dispatch 2024-03-22T07:06:51.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:51 smithi063 ceph-mon[29200]: Manager daemon smithi063.mnxayh is now available 2024-03-22T07:06:51.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:51 smithi063 ceph-mon[29200]: from='mgr.14100 172.21.15.63:0/2726337620' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.mnxayh/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:06:51.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:51 smithi063 ceph-mon[29200]: from='mgr.14100 172.21.15.63:0/2726337620' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.mnxayh/trash_purge_schedule"}]: dispatch 2024-03-22T07:06:51.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:51 smithi063 ceph-mon[29200]: from='mgr.14100 172.21.15.63:0/2726337620' entity='mgr.smithi063.mnxayh' 2024-03-22T07:06:51.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:51 smithi063 ceph-mon[29200]: from='mgr.14100 172.21.15.63:0/2726337620' entity='mgr.smithi063.mnxayh' 2024-03-22T07:06:51.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:51 smithi063 ceph-mon[29200]: from='mgr.14100 172.21.15.63:0/2726337620' entity='mgr.smithi063.mnxayh' 2024-03-22T07:06:51.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-03-22T07:06:51.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-03-22T07:06:51.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "9a0ba9d6-e81a-11ee-95cd-87774f69a715", 2024-03-22T07:06:51.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T07:06:51.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T07:06:51.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T07:06:51.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T07:06:51.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:51.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T07:06:51.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T07:06:51.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-03-22T07:06:51.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-03-22T07:06:51.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T07:06:51.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "smithi063" 2024-03-22T07:06:51.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-03-22T07:06:51.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-03-22T07:06:51.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T07:06:51.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T07:06:51.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-22T07:06:51.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T07:06:51.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:51.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T07:06:51.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T07:06:51.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T07:06:51.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T07:06:51.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T07:06:51.967 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T07:06:51.967 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T07:06:51.967 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T07:06:51.968 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:51.968 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T07:06:51.968 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T07:06:51.968 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T07:06:51.968 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T07:06:51.968 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T07:06:51.968 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T07:06:51.968 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T07:06:51.968 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T07:06:51.968 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T07:06:51.968 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:51.968 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T07:06:51.968 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T07:06:51.968 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T07:06:51.969 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T07:06:51.969 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:51.969 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T07:06:51.969 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-22T07:06:51.969 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T07:06:51.969 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T07:06:51.969 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T07:06:51.969 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T07:06:51.969 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T07:06:51.969 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-03-22T07:06:51.969 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T07:06:51.969 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:51.969 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T07:06:51.969 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T07:06:51.969 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T07:06:38.005342+0000", 2024-03-22T07:06:51.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T07:06:51.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:06:51.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T07:06:51.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-03-22T07:06:51.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.295+0000 7f99aca6b700 1 Processor -- start 2024-03-22T07:06:51.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.295+0000 7f99aca6b700 1 -- start start 2024-03-22T07:06:51.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.296+0000 7f99aca6b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a8102730 0x7f99a8102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:51.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.296+0000 7f99aca6b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8103080 con 0x7f99a8102730 2024-03-22T07:06:51.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.296+0000 7f99a659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a8102730 0x7f99a8102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:51.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.296+0000 7f99a659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a8102730 0x7f99a8102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43544/0 (socket says 172.21.15.63:43544) 2024-03-22T07:06:51.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.296+0000 7f99a659c700 1 -- 172.21.15.63:0/1056186174 learned_addr learned my addr 172.21.15.63:0/1056186174 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:06:51.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.296+0000 7f99a659c700 1 -- 172.21.15.63:0/1056186174 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a81031c0 con 0x7f99a8102730 2024-03-22T07:06:51.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.296+0000 7f99a659c700 1 --2- 172.21.15.63:0/1056186174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a8102730 0x7f99a8102b40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f99980098e0 tx=0x7f9998009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab3f4444e040658 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:51.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.297+0000 7f99a559a700 1 -- 172.21.15.63:0/1056186174 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f999800b860 con 0x7f99a8102730 2024-03-22T07:06:51.970 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.297+0000 7f99a559a700 1 -- 172.21.15.63:0/1056186174 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f999800be50 con 0x7f99a8102730 2024-03-22T07:06:51.971 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.297+0000 7f99a559a700 1 -- 172.21.15.63:0/1056186174 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9998010380 con 0x7f99a8102730 2024-03-22T07:06:51.971 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.297+0000 7f99aca6b700 1 -- 172.21.15.63:0/1056186174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a8102730 msgr2=0x7f99a8102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:51.971 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.297+0000 7f99aca6b700 1 --2- 172.21.15.63:0/1056186174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a8102730 0x7f99a8102b40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f99980098e0 tx=0x7f9998009250 comp rx=0 tx=0).stop 2024-03-22T07:06:51.971 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.297+0000 7f99aca6b700 1 -- 172.21.15.63:0/1056186174 shutdown_connections 2024-03-22T07:06:51.971 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.297+0000 7f99aca6b700 1 --2- 172.21.15.63:0/1056186174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a8102730 0x7f99a8102b40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:51.971 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.297+0000 7f99aca6b700 1 -- 172.21.15.63:0/1056186174 >> 172.21.15.63:0/1056186174 conn(0x7f99a80fdf10 msgr2=0x7f99a8100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:51.971 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.297+0000 7f99aca6b700 1 -- 172.21.15.63:0/1056186174 shutdown_connections 2024-03-22T07:06:51.971 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.297+0000 7f99aca6b700 1 -- 172.21.15.63:0/1056186174 wait complete. 2024-03-22T07:06:51.971 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.298+0000 7f99aca6b700 1 Processor -- start 2024-03-22T07:06:51.971 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.298+0000 7f99aca6b700 1 -- start start 2024-03-22T07:06:51.971 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.298+0000 7f99aca6b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a8102730 0x7f99a81115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:06:51.971 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.298+0000 7f99aca6b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8111b30 con 0x7f99a8102730 2024-03-22T07:06:51.971 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.298+0000 7f99a659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a8102730 0x7f99a81115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:06:51.971 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.298+0000 7f99a659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a8102730 0x7f99a81115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:43556/0 (socket says 172.21.15.63:43556) 2024-03-22T07:06:51.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.298+0000 7f99a659c700 1 -- 172.21.15.63:0/3800138736 learned_addr learned my addr 172.21.15.63:0/3800138736 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:06:51.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.298+0000 7f99a659c700 1 -- 172.21.15.63:0/3800138736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99980094f0 con 0x7f99a8102730 2024-03-22T07:06:51.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.298+0000 7f99a659c700 1 --2- 172.21.15.63:0/3800138736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a8102730 0x7f99a81115f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9998006fd0 tx=0x7f9998009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:06:51.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.299+0000 7f9996ffd700 1 -- 172.21.15.63:0/3800138736 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99980107c0 con 0x7f99a8102730 2024-03-22T07:06:51.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.299+0000 7f9996ffd700 1 -- 172.21.15.63:0/3800138736 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9998010db0 con 0x7f99a8102730 2024-03-22T07:06:51.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.299+0000 7f99aca6b700 1 -- 172.21.15.63:0/3800138736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a8111d30 con 0x7f99a8102730 2024-03-22T07:06:51.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.299+0000 7f9996ffd700 1 -- 172.21.15.63:0/3800138736 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f999801ada0 con 0x7f99a8102730 2024-03-22T07:06:51.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.299+0000 7f99aca6b700 1 -- 172.21.15.63:0/3800138736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a81128a0 con 0x7f99a8102730 2024-03-22T07:06:51.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.299+0000 7f9996ffd700 1 -- 172.21.15.63:0/3800138736 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f99980196d0 con 0x7f99a8102730 2024-03-22T07:06:51.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.299+0000 7f9996ffd700 1 -- 172.21.15.63:0/3800138736 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9998024920 con 0x7f99a8102730 2024-03-22T07:06:51.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.300+0000 7f99aca6b700 1 -- 172.21.15.63:0/3800138736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a81121c0 con 0x7f99a8102730 2024-03-22T07:06:51.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.302+0000 7f9996ffd700 1 -- 172.21.15.63:0/3800138736 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f9998029060 con 0x7f99a8102730 2024-03-22T07:06:51.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.304+0000 7f9996ffd700 1 -- 172.21.15.63:0/3800138736 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 2) v1 ==== 43124+0+0 (secure 0 0 0) 0x7f9998010920 con 0x7f99a8102730 2024-03-22T07:06:51.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.384+0000 7f99aca6b700 1 -- 172.21.15.63:0/3800138736 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f99a804bed0 con 0x7f99a8102730 2024-03-22T07:06:51.972 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.385+0000 7f9996ffd700 1 -- 172.21.15.63:0/3800138736 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f9998020030 con 0x7f99a8102730 2024-03-22T07:06:51.973 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.386+0000 7f99aca6b700 1 -- 172.21.15.63:0/3800138736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a8102730 msgr2=0x7f99a81115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:06:51.973 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.386+0000 7f99aca6b700 1 --2- 172.21.15.63:0/3800138736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a8102730 0x7f99a81115f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9998006fd0 tx=0x7f9998009250 comp rx=0 tx=0).stop 2024-03-22T07:06:51.973 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.386+0000 7f99aca6b700 1 -- 172.21.15.63:0/3800138736 shutdown_connections 2024-03-22T07:06:51.973 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.386+0000 7f99aca6b700 1 --2- 172.21.15.63:0/3800138736 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f99a8102730 0x7f99a81115f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:06:51.973 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.386+0000 7f99aca6b700 1 -- 172.21.15.63:0/3800138736 >> 172.21.15.63:0/3800138736 conn(0x7f99a80fdf10 msgr2=0x7f99a80feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:06:51.973 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.386+0000 7f99aca6b700 1 -- 172.21.15.63:0/3800138736 shutdown_connections 2024-03-22T07:06:51.973 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:51.386+0000 7f99aca6b700 1 -- 172.21.15.63:0/3800138736 wait complete. 2024-03-22T07:06:51.973 INFO:teuthology.orchestra.run.smithi063.stdout:mgr not available, waiting (3/15)... 2024-03-22T07:06:52.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:52 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3800138736' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T07:06:52.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:52 smithi063 ceph-mon[29200]: mgrmap e3: smithi063.mnxayh(active, since 1.00697s) 2024-03-22T07:06:54.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:54 smithi063 ceph-mon[29200]: mgrmap e4: smithi063.mnxayh(active, since 2s) 2024-03-22T07:06:57.641 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:06:57 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3509249795' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-22T07:07:01.859 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-03-22T07:07:01.860 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-03-22T07:07:01.860 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsid": "9a0ba9d6-e81a-11ee-95cd-87774f69a715", 2024-03-22T07:07:01.860 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "health": { 2024-03-22T07:07:01.860 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-22T07:07:01.860 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-22T07:07:01.860 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-22T07:07:01.860 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:07:01.860 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-22T07:07:01.860 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-22T07:07:01.860 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 0 2024-03-22T07:07:01.860 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-03-22T07:07:01.860 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-22T07:07:01.860 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "smithi063" 2024-03-22T07:07:01.861 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-03-22T07:07:01.861 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-22T07:07:01.861 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-22T07:07:01.861 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T07:07:01.861 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-03-22T07:07:01.861 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-22T07:07:01.861 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:07:01.861 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-22T07:07:01.861 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T07:07:01.861 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-22T07:07:01.861 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-22T07:07:01.861 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-22T07:07:01.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-22T07:07:01.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-22T07:07:01.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-22T07:07:01.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:07:01.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-22T07:07:01.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-22T07:07:01.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-22T07:07:01.870 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-22T07:07:01.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-22T07:07:01.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-22T07:07:01.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-22T07:07:01.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-22T07:07:01.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-22T07:07:01.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:07:01.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-22T07:07:01.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T07:07:01.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-22T07:07:01.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-22T07:07:01.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:07:01.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-22T07:07:01.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-22T07:07:01.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-22T07:07:01.871 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-22T07:07:01.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-22T07:07:01.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-22T07:07:01.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "restful" 2024-03-22T07:07:01.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ], 2024-03-22T07:07:01.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T07:07:01.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:07:01.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-22T07:07:01.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-22T07:07:01.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "modified": "2024-03-22T07:06:38.005342+0000", 2024-03-22T07:07:01.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-22T07:07:01.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout }, 2024-03-22T07:07:01.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-22T07:07:01.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-03-22T07:07:01.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.220+0000 7f553b92c700 1 Processor -- start 2024-03-22T07:07:01.872 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.220+0000 7f553b92c700 1 -- start start 2024-03-22T07:07:01.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.220+0000 7f553b92c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55341009a0 0x7f5534100db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:01.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.220+0000 7f553b92c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55341012f0 con 0x7f55341009a0 2024-03-22T07:07:01.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.221+0000 7f55396c8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55341009a0 0x7f5534100db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:01.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.221+0000 7f55396c8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55341009a0 0x7f5534100db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55414/0 (socket says 172.21.15.63:55414) 2024-03-22T07:07:01.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.221+0000 7f55396c8700 1 -- 172.21.15.63:0/1498047953 learned_addr learned my addr 172.21.15.63:0/1498047953 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:01.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.221+0000 7f55396c8700 1 -- 172.21.15.63:0/1498047953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5534101430 con 0x7f55341009a0 2024-03-22T07:07:01.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.221+0000 7f55396c8700 1 --2- 172.21.15.63:0/1498047953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55341009a0 0x7f5534100db0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f55240098e0 tx=0x7f5524009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db76fb755318144 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:01.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.221+0000 7f552bfff700 1 -- 172.21.15.63:0/1498047953 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f552400b860 con 0x7f55341009a0 2024-03-22T07:07:01.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.221+0000 7f552bfff700 1 -- 172.21.15.63:0/1498047953 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f552400be50 con 0x7f55341009a0 2024-03-22T07:07:01.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.221+0000 7f552bfff700 1 -- 172.21.15.63:0/1498047953 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5524010380 con 0x7f55341009a0 2024-03-22T07:07:01.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.222+0000 7f553b92c700 1 -- 172.21.15.63:0/1498047953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55341009a0 msgr2=0x7f5534100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:01.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.222+0000 7f553b92c700 1 --2- 172.21.15.63:0/1498047953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55341009a0 0x7f5534100db0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f55240098e0 tx=0x7f5524009250 comp rx=0 tx=0).stop 2024-03-22T07:07:01.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.222+0000 7f553b92c700 1 -- 172.21.15.63:0/1498047953 shutdown_connections 2024-03-22T07:07:01.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.222+0000 7f553b92c700 1 --2- 172.21.15.63:0/1498047953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55341009a0 0x7f5534100db0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:01.873 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.222+0000 7f553b92c700 1 -- 172.21.15.63:0/1498047953 >> 172.21.15.63:0/1498047953 conn(0x7f55340fc4a0 msgr2=0x7f55340fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:01.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.222+0000 7f553b92c700 1 -- 172.21.15.63:0/1498047953 shutdown_connections 2024-03-22T07:07:01.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.222+0000 7f553b92c700 1 -- 172.21.15.63:0/1498047953 wait complete. 2024-03-22T07:07:01.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.222+0000 7f553b92c700 1 Processor -- start 2024-03-22T07:07:01.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.222+0000 7f553b92c700 1 -- start start 2024-03-22T07:07:01.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.222+0000 7f553b92c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55341009a0 0x7f5534111560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:01.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.222+0000 7f553b92c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5534111aa0 con 0x7f55341009a0 2024-03-22T07:07:01.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.223+0000 7f55396c8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55341009a0 0x7f5534111560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:01.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.223+0000 7f55396c8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55341009a0 0x7f5534111560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55418/0 (socket says 172.21.15.63:55418) 2024-03-22T07:07:01.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.223+0000 7f55396c8700 1 -- 172.21.15.63:0/3509249795 learned_addr learned my addr 172.21.15.63:0/3509249795 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:01.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.223+0000 7f55396c8700 1 -- 172.21.15.63:0/3509249795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55240094f0 con 0x7f55341009a0 2024-03-22T07:07:01.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.223+0000 7f55396c8700 1 --2- 172.21.15.63:0/3509249795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55341009a0 0x7f5534111560 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5524000c00 tx=0x7f5524009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:01.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.223+0000 7f5529ffb700 1 -- 172.21.15.63:0/3509249795 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f55240107c0 con 0x7f55341009a0 2024-03-22T07:07:01.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.223+0000 7f5529ffb700 1 -- 172.21.15.63:0/3509249795 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5524010db0 con 0x7f55341009a0 2024-03-22T07:07:01.874 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.223+0000 7f553b92c700 1 -- 172.21.15.63:0/3509249795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5534111ca0 con 0x7f55341009a0 2024-03-22T07:07:01.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.223+0000 7f5529ffb700 1 -- 172.21.15.63:0/3509249795 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f552401ad00 con 0x7f55341009a0 2024-03-22T07:07:01.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.223+0000 7f553b92c700 1 -- 172.21.15.63:0/3509249795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5534112810 con 0x7f55341009a0 2024-03-22T07:07:01.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.224+0000 7f5529ffb700 1 -- 172.21.15.63:0/3509249795 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7f55240193c0 con 0x7f55341009a0 2024-03-22T07:07:01.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.224+0000 7f553b92c700 1 -- 172.21.15.63:0/3509249795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f553405f5f0 con 0x7f55341009a0 2024-03-22T07:07:01.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.225+0000 7f5529ffb700 1 --2- 172.21.15.63:0/3509249795 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f5520036790 0x7f5520038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:01.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.225+0000 7f5529ffb700 1 -- 172.21.15.63:0/3509249795 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f552404a6b0 con 0x7f55341009a0 2024-03-22T07:07:01.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.225+0000 7f5538ec7700 1 --2- 172.21.15.63:0/3509249795 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f5520036790 0x7f5520038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:01.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.225+0000 7f5538ec7700 1 --2- 172.21.15.63:0/3509249795 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f5520036790 0x7f5520038c40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5530009990 tx=0x7f5530006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:01.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.229+0000 7f5529ffb700 1 -- 172.21.15.63:0/3509249795 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5524019670 con 0x7f55341009a0 2024-03-22T07:07:01.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.506+0000 7f553b92c700 1 -- 172.21.15.63:0/3509249795 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f553404bed0 con 0x7f55341009a0 2024-03-22T07:07:01.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.506+0000 7f5529ffb700 1 -- 172.21.15.63:0/3509249795 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f5524022340 con 0x7f55341009a0 2024-03-22T07:07:01.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.508+0000 7f553b92c700 1 -- 172.21.15.63:0/3509249795 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f5520036790 msgr2=0x7f5520038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:01.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.508+0000 7f553b92c700 1 --2- 172.21.15.63:0/3509249795 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f5520036790 0x7f5520038c40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5530009990 tx=0x7f5530006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:01.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.508+0000 7f553b92c700 1 -- 172.21.15.63:0/3509249795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55341009a0 msgr2=0x7f5534111560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:01.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.508+0000 7f553b92c700 1 --2- 172.21.15.63:0/3509249795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55341009a0 0x7f5534111560 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5524000c00 tx=0x7f5524009250 comp rx=0 tx=0).stop 2024-03-22T07:07:01.875 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.509+0000 7f553b92c700 1 -- 172.21.15.63:0/3509249795 shutdown_connections 2024-03-22T07:07:01.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.509+0000 7f553b92c700 1 --2- 172.21.15.63:0/3509249795 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f5520036790 0x7f5520038c40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:01.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.509+0000 7f553b92c700 1 --2- 172.21.15.63:0/3509249795 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f55341009a0 0x7f5534111560 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:01.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.509+0000 7f553b92c700 1 -- 172.21.15.63:0/3509249795 >> 172.21.15.63:0/3509249795 conn(0x7f55340fc4a0 msgr2=0x7f55340fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:01.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.509+0000 7f553b92c700 1 -- 172.21.15.63:0/3509249795 shutdown_connections 2024-03-22T07:07:01.876 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:06:57.509+0000 7f553b92c700 1 -- 172.21.15.63:0/3509249795 wait complete. 2024-03-22T07:07:01.876 INFO:teuthology.orchestra.run.smithi063.stdout:mgr is available 2024-03-22T07:07:02.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:02 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/734653405' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-22T07:07:03.110 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-03-22T07:07:03.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [global] 2024-03-22T07:07:03.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout fsid = 9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:07:03.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-22T07:07:03.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-22T07:07:03.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-22T07:07:03.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-22T07:07:03.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-03-22T07:07:03.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-22T07:07:03.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-22T07:07:03.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 2024-03-22T07:07:03.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout [osd] 2024-03-22T07:07:03.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-22T07:07:03.111 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-22T07:07:03.112 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-22T07:07:03.112 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-22T07:07:03.112 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-22T07:07:03.112 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.340+0000 7fe5dcb79700 1 Processor -- start 2024-03-22T07:07:03.112 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.340+0000 7fe5dcb79700 1 -- start start 2024-03-22T07:07:03.112 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.340+0000 7fe5dcb79700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d8102710 0x7fe5d8102b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:03.112 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.341+0000 7fe5dcb79700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d8103060 con 0x7fe5d8102710 2024-03-22T07:07:03.112 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.341+0000 7fe5d659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d8102710 0x7fe5d8102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:03.112 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.341+0000 7fe5d659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d8102710 0x7fe5d8102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55426/0 (socket says 172.21.15.63:55426) 2024-03-22T07:07:03.112 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.341+0000 7fe5d659c700 1 -- 172.21.15.63:0/2976213021 learned_addr learned my addr 172.21.15.63:0/2976213021 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:03.112 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.341+0000 7fe5d659c700 1 -- 172.21.15.63:0/2976213021 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d81031a0 con 0x7fe5d8102710 2024-03-22T07:07:03.112 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.341+0000 7fe5d659c700 1 --2- 172.21.15.63:0/2976213021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d8102710 0x7fe5d8102b20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009e50 tx=0x7fe5c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3011716c52160378 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:03.112 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.341+0000 7fe5d559a700 1 -- 172.21.15.63:0/2976213021 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5c80092e0 con 0x7fe5d8102710 2024-03-22T07:07:03.112 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.342+0000 7fe5d559a700 1 -- 172.21.15.63:0/2976213021 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fe5c8012430 con 0x7fe5d8102710 2024-03-22T07:07:03.112 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.342+0000 7fe5d559a700 1 -- 172.21.15.63:0/2976213021 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5c80075c0 con 0x7fe5d8102710 2024-03-22T07:07:03.112 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.342+0000 7fe5dcb79700 1 -- 172.21.15.63:0/2976213021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d8102710 msgr2=0x7fe5d8102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:03.113 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.342+0000 7fe5dcb79700 1 --2- 172.21.15.63:0/2976213021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d8102710 0x7fe5d8102b20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009e50 tx=0x7fe5c8009840 comp rx=0 tx=0).stop 2024-03-22T07:07:03.113 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.342+0000 7fe5dcb79700 1 -- 172.21.15.63:0/2976213021 shutdown_connections 2024-03-22T07:07:03.113 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.342+0000 7fe5dcb79700 1 --2- 172.21.15.63:0/2976213021 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d8102710 0x7fe5d8102b20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:03.113 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.342+0000 7fe5dcb79700 1 -- 172.21.15.63:0/2976213021 >> 172.21.15.63:0/2976213021 conn(0x7fe5d80fde50 msgr2=0x7fe5d81002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:03.113 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.342+0000 7fe5dcb79700 1 -- 172.21.15.63:0/2976213021 shutdown_connections 2024-03-22T07:07:03.113 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.342+0000 7fe5dcb79700 1 -- 172.21.15.63:0/2976213021 wait complete. 2024-03-22T07:07:03.113 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.343+0000 7fe5dcb79700 1 Processor -- start 2024-03-22T07:07:03.113 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.343+0000 7fe5dcb79700 1 -- start start 2024-03-22T07:07:03.113 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.343+0000 7fe5dcb79700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d8102710 0x7fe5d8111590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:03.113 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.343+0000 7fe5dcb79700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d8111ad0 con 0x7fe5d8102710 2024-03-22T07:07:03.113 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.343+0000 7fe5d659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d8102710 0x7fe5d8111590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:03.113 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.343+0000 7fe5d659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d8102710 0x7fe5d8111590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55438/0 (socket says 172.21.15.63:55438) 2024-03-22T07:07:03.113 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.343+0000 7fe5d659c700 1 -- 172.21.15.63:0/734653405 learned_addr learned my addr 172.21.15.63:0/734653405 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:03.113 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.343+0000 7fe5d659c700 1 -- 172.21.15.63:0/734653405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c8009ae0 con 0x7fe5d8102710 2024-03-22T07:07:03.113 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.344+0000 7fe5d659c700 1 --2- 172.21.15.63:0/734653405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d8102710 0x7fe5d8111590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009fd0 tx=0x7fe5c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:03.113 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.344+0000 7fe5ceffd700 1 -- 172.21.15.63:0/734653405 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5c80092e0 con 0x7fe5d8102710 2024-03-22T07:07:03.114 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.344+0000 7fe5ceffd700 1 -- 172.21.15.63:0/734653405 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fe5c8007cc0 con 0x7fe5d8102710 2024-03-22T07:07:03.114 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.344+0000 7fe5ceffd700 1 -- 172.21.15.63:0/734653405 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5c801c9a0 con 0x7fe5d8102710 2024-03-22T07:07:03.114 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.344+0000 7fe5dcb79700 1 -- 172.21.15.63:0/734653405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d8111cd0 con 0x7fe5d8102710 2024-03-22T07:07:03.114 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.344+0000 7fe5dcb79700 1 -- 172.21.15.63:0/734653405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d8112840 con 0x7fe5d8102710 2024-03-22T07:07:03.114 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.345+0000 7fe5ceffd700 1 -- 172.21.15.63:0/734653405 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7fe5c8019070 con 0x7fe5d8102710 2024-03-22T07:07:03.114 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.345+0000 7fe5ceffd700 1 --2- 172.21.15.63:0/734653405 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fe5c4036790 0x7fe5c4038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:03.114 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.345+0000 7fe5dcb79700 1 -- 172.21.15.63:0/734653405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d805f5f0 con 0x7fe5d8102710 2024-03-22T07:07:03.114 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.345+0000 7fe5ceffd700 1 -- 172.21.15.63:0/734653405 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fe5c8049ce0 con 0x7fe5d8102710 2024-03-22T07:07:03.114 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.345+0000 7fe5d5d9b700 1 --2- 172.21.15.63:0/734653405 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fe5c4036790 0x7fe5c4038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:03.114 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.345+0000 7fe5d5d9b700 1 --2- 172.21.15.63:0/734653405 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fe5c4036790 0x7fe5c4038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009990 tx=0x7fe5c0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:03.114 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.350+0000 7fe5ceffd700 1 -- 172.21.15.63:0/734653405 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe5c8020070 con 0x7fe5d8102710 2024-03-22T07:07:03.114 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.542+0000 7fe5dcb79700 1 -- 172.21.15.63:0/734653405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fe5d8112be0 con 0x7fe5d8102710 2024-03-22T07:07:03.114 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.543+0000 7fe5ceffd700 1 -- 172.21.15.63:0/734653405 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fe5c802a020 con 0x7fe5d8102710 2024-03-22T07:07:03.114 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.545+0000 7fe5dcb79700 1 -- 172.21.15.63:0/734653405 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fe5c4036790 msgr2=0x7fe5c4038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:03.114 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.545+0000 7fe5dcb79700 1 --2- 172.21.15.63:0/734653405 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fe5c4036790 0x7fe5c4038c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009990 tx=0x7fe5c0006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:03.115 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.545+0000 7fe5dcb79700 1 -- 172.21.15.63:0/734653405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d8102710 msgr2=0x7fe5d8111590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:03.115 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.545+0000 7fe5dcb79700 1 --2- 172.21.15.63:0/734653405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d8102710 0x7fe5d8111590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009fd0 tx=0x7fe5c8009840 comp rx=0 tx=0).stop 2024-03-22T07:07:03.115 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.545+0000 7fe5dcb79700 1 -- 172.21.15.63:0/734653405 shutdown_connections 2024-03-22T07:07:03.115 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.545+0000 7fe5dcb79700 1 --2- 172.21.15.63:0/734653405 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fe5c4036790 0x7fe5c4038c40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:03.115 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.545+0000 7fe5dcb79700 1 --2- 172.21.15.63:0/734653405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5d8102710 0x7fe5d8111590 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:03.115 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.545+0000 7fe5dcb79700 1 -- 172.21.15.63:0/734653405 >> 172.21.15.63:0/734653405 conn(0x7fe5d80fde50 msgr2=0x7fe5d80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:03.115 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.545+0000 7fe5dcb79700 1 -- 172.21.15.63:0/734653405 shutdown_connections 2024-03-22T07:07:03.115 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:02.546+0000 7fe5dcb79700 1 -- 172.21.15.63:0/734653405 wait complete. 2024-03-22T07:07:03.115 INFO:teuthology.orchestra.run.smithi063.stdout:Enabling cephadm module... 2024-03-22T07:07:04.083 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:03 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2894607367' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-22T07:07:04.349 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.472+0000 7fa112c70700 1 Processor -- start 2024-03-22T07:07:04.349 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.472+0000 7fa112c70700 1 -- start start 2024-03-22T07:07:04.349 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.472+0000 7fa112c70700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa10c102740 0x7fa10c102b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:04.349 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.472+0000 7fa112c70700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c103090 con 0x7fa10c102740 2024-03-22T07:07:04.349 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.473+0000 7fa110a0c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa10c102740 0x7fa10c102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:04.349 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.473+0000 7fa110a0c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa10c102740 0x7fa10c102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55454/0 (socket says 172.21.15.63:55454) 2024-03-22T07:07:04.349 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.473+0000 7fa110a0c700 1 -- 172.21.15.63:0/1055542294 learned_addr learned my addr 172.21.15.63:0/1055542294 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:04.349 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.473+0000 7fa110a0c700 1 -- 172.21.15.63:0/1055542294 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c1031d0 con 0x7fa10c102740 2024-03-22T07:07:04.349 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.473+0000 7fa110a0c700 1 --2- 172.21.15.63:0/1055542294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa10c102740 0x7fa10c102b50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0098e0 tx=0x7fa0fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c51a09854ecb7e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:04.349 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.473+0000 7fa10b7fe700 1 -- 172.21.15.63:0/1055542294 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0fc00b860 con 0x7fa10c102740 2024-03-22T07:07:04.349 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.473+0000 7fa10b7fe700 1 -- 172.21.15.63:0/1055542294 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa0fc00be50 con 0x7fa10c102740 2024-03-22T07:07:04.350 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.473+0000 7fa10b7fe700 1 -- 172.21.15.63:0/1055542294 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0fc010380 con 0x7fa10c102740 2024-03-22T07:07:04.350 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.474+0000 7fa112c70700 1 -- 172.21.15.63:0/1055542294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa10c102740 msgr2=0x7fa10c102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:04.350 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.474+0000 7fa112c70700 1 --2- 172.21.15.63:0/1055542294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa10c102740 0x7fa10c102b50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0098e0 tx=0x7fa0fc009250 comp rx=0 tx=0).stop 2024-03-22T07:07:04.350 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.474+0000 7fa112c70700 1 -- 172.21.15.63:0/1055542294 shutdown_connections 2024-03-22T07:07:04.350 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.474+0000 7fa112c70700 1 --2- 172.21.15.63:0/1055542294 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa10c102740 0x7fa10c102b50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:04.350 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.474+0000 7fa112c70700 1 -- 172.21.15.63:0/1055542294 >> 172.21.15.63:0/1055542294 conn(0x7fa10c0fdea0 msgr2=0x7fa10c1002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:04.350 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.474+0000 7fa112c70700 1 -- 172.21.15.63:0/1055542294 shutdown_connections 2024-03-22T07:07:04.350 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.474+0000 7fa112c70700 1 -- 172.21.15.63:0/1055542294 wait complete. 2024-03-22T07:07:04.350 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.474+0000 7fa112c70700 1 Processor -- start 2024-03-22T07:07:04.350 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.474+0000 7fa112c70700 1 -- start start 2024-03-22T07:07:04.350 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.475+0000 7fa112c70700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa10c102740 0x7fa10c078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:04.350 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.475+0000 7fa112c70700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c079110 con 0x7fa10c102740 2024-03-22T07:07:04.350 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.475+0000 7fa110a0c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa10c102740 0x7fa10c078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:04.350 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.475+0000 7fa110a0c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa10c102740 0x7fa10c078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55464/0 (socket says 172.21.15.63:55464) 2024-03-22T07:07:04.351 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.475+0000 7fa110a0c700 1 -- 172.21.15.63:0/2894607367 learned_addr learned my addr 172.21.15.63:0/2894607367 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:04.351 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.475+0000 7fa110a0c700 1 -- 172.21.15.63:0/2894607367 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc0094f0 con 0x7fa10c102740 2024-03-22T07:07:04.351 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.475+0000 7fa110a0c700 1 --2- 172.21.15.63:0/2894607367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa10c102740 0x7fa10c078bd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00ba90 tx=0x7fa0fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:04.351 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.475+0000 7fa1097fa700 1 -- 172.21.15.63:0/2894607367 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0fc0107c0 con 0x7fa10c102740 2024-03-22T07:07:04.351 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.475+0000 7fa1097fa700 1 -- 172.21.15.63:0/2894607367 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa0fc010db0 con 0x7fa10c102740 2024-03-22T07:07:04.351 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.475+0000 7fa1097fa700 1 -- 172.21.15.63:0/2894607367 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa0fc01add0 con 0x7fa10c102740 2024-03-22T07:07:04.351 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.475+0000 7fa112c70700 1 -- 172.21.15.63:0/2894607367 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10c077180 con 0x7fa10c102740 2024-03-22T07:07:04.351 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.475+0000 7fa112c70700 1 -- 172.21.15.63:0/2894607367 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10c077670 con 0x7fa10c102740 2024-03-22T07:07:04.351 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.476+0000 7fa1097fa700 1 -- 172.21.15.63:0/2894607367 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 4) v1 ==== 43357+0+0 (secure 0 0 0) 0x7fa0fc010920 con 0x7fa10c102740 2024-03-22T07:07:04.351 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.476+0000 7fa1097fa700 1 --2- 172.21.15.63:0/2894607367 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fa0f4036790 0x7fa0f4038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:04.351 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.476+0000 7fa1097fa700 1 -- 172.21.15.63:0/2894607367 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fa0fc04a570 con 0x7fa10c102740 2024-03-22T07:07:04.351 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.476+0000 7fa112c70700 1 -- 172.21.15.63:0/2894607367 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa10c04bed0 con 0x7fa10c102740 2024-03-22T07:07:04.351 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.477+0000 7fa10bfff700 1 --2- 172.21.15.63:0/2894607367 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fa0f4036790 0x7fa0f4038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:04.351 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.477+0000 7fa10bfff700 1 --2- 172.21.15.63:0/2894607367 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fa0f4036790 0x7fa0f4038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa100009990 tx=0x7fa100006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:04.351 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.481+0000 7fa1097fa700 1 -- 172.21.15.63:0/2894607367 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa0fc01a650 con 0x7fa10c102740 2024-03-22T07:07:04.352 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.728+0000 7fa112c70700 1 -- 172.21.15.63:0/2894607367 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fa10c077b40 con 0x7fa10c102740 2024-03-22T07:07:04.352 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.781+0000 7fa1097fa700 1 -- 172.21.15.63:0/2894607367 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fa0fc02e440 con 0x7fa10c102740 2024-03-22T07:07:04.352 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.782+0000 7fa1097fa700 1 -- 172.21.15.63:0/2894607367 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fa0fc018350 con 0x7fa10c102740 2024-03-22T07:07:04.352 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.784+0000 7fa112c70700 1 -- 172.21.15.63:0/2894607367 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fa0f4036790 msgr2=0x7fa0f4038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:04.352 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.784+0000 7fa112c70700 1 --2- 172.21.15.63:0/2894607367 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fa0f4036790 0x7fa0f4038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa100009990 tx=0x7fa100006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:04.352 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.784+0000 7fa112c70700 1 -- 172.21.15.63:0/2894607367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa10c102740 msgr2=0x7fa10c078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:04.352 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.784+0000 7fa112c70700 1 --2- 172.21.15.63:0/2894607367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa10c102740 0x7fa10c078bd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00ba90 tx=0x7fa0fc009250 comp rx=0 tx=0).stop 2024-03-22T07:07:04.352 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.784+0000 7fa112c70700 1 -- 172.21.15.63:0/2894607367 shutdown_connections 2024-03-22T07:07:04.352 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.784+0000 7fa112c70700 1 --2- 172.21.15.63:0/2894607367 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fa0f4036790 0x7fa0f4038c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:04.352 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.784+0000 7fa112c70700 1 --2- 172.21.15.63:0/2894607367 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa10c102740 0x7fa10c078bd0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:04.352 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.784+0000 7fa112c70700 1 -- 172.21.15.63:0/2894607367 >> 172.21.15.63:0/2894607367 conn(0x7fa10c0fdea0 msgr2=0x7fa10c0fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:04.352 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.784+0000 7fa112c70700 1 -- 172.21.15.63:0/2894607367 shutdown_connections 2024-03-22T07:07:04.352 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:03.784+0000 7fa112c70700 1 -- 172.21.15.63:0/2894607367 wait complete. 2024-03-22T07:07:04.964 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:04 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2894607367' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-22T07:07:04.964 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:04 smithi063 ceph-mon[29200]: mgrmap e5: smithi063.mnxayh(active, since 12s) 2024-03-22T07:07:05.723 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-03-22T07:07:05.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-22T07:07:05.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-22T07:07:05.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "active_name": "smithi063.mnxayh", 2024-03-22T07:07:05.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-22T07:07:05.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-03-22T07:07:05.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.758+0000 7fddeca1a700 1 Processor -- start 2024-03-22T07:07:05.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.758+0000 7fddeca1a700 1 -- start start 2024-03-22T07:07:05.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.758+0000 7fddeca1a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdde8102bb0 0x7fdde8102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:05.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.758+0000 7fddeca1a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde8103500 con 0x7fdde8102bb0 2024-03-22T07:07:05.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.758+0000 7fdde659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdde8102bb0 0x7fdde8102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:05.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.759+0000 7fdde659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdde8102bb0 0x7fdde8102fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60256/0 (socket says 172.21.15.63:60256) 2024-03-22T07:07:05.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.759+0000 7fdde659c700 1 -- 172.21.15.63:0/1553471113 learned_addr learned my addr 172.21.15.63:0/1553471113 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:05.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.759+0000 7fdde659c700 1 -- 172.21.15.63:0/1553471113 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde8103640 con 0x7fdde8102bb0 2024-03-22T07:07:05.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.759+0000 7fdde659c700 1 --2- 172.21.15.63:0/1553471113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdde8102bb0 0x7fdde8102fc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0098e0 tx=0x7fdddc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec9b8871fbe157f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:05.724 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.759+0000 7fdde559a700 1 -- 172.21.15.63:0/1553471113 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdddc00b860 con 0x7fdde8102bb0 2024-03-22T07:07:05.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.759+0000 7fdde559a700 1 -- 172.21.15.63:0/1553471113 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fdddc00be50 con 0x7fdde8102bb0 2024-03-22T07:07:05.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.759+0000 7fdde559a700 1 -- 172.21.15.63:0/1553471113 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdddc010380 con 0x7fdde8102bb0 2024-03-22T07:07:05.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.760+0000 7fddeca1a700 1 -- 172.21.15.63:0/1553471113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdde8102bb0 msgr2=0x7fdde8102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:05.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.760+0000 7fddeca1a700 1 --2- 172.21.15.63:0/1553471113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdde8102bb0 0x7fdde8102fc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0098e0 tx=0x7fdddc009250 comp rx=0 tx=0).stop 2024-03-22T07:07:05.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.760+0000 7fddeca1a700 1 -- 172.21.15.63:0/1553471113 shutdown_connections 2024-03-22T07:07:05.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.760+0000 7fddeca1a700 1 --2- 172.21.15.63:0/1553471113 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdde8102bb0 0x7fdde8102fc0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:05.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.760+0000 7fddeca1a700 1 -- 172.21.15.63:0/1553471113 >> 172.21.15.63:0/1553471113 conn(0x7fdde80fc500 msgr2=0x7fdde80fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:05.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.760+0000 7fddeca1a700 1 -- 172.21.15.63:0/1553471113 shutdown_connections 2024-03-22T07:07:05.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.760+0000 7fddeca1a700 1 -- 172.21.15.63:0/1553471113 wait complete. 2024-03-22T07:07:05.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.761+0000 7fddeca1a700 1 Processor -- start 2024-03-22T07:07:05.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.761+0000 7fddeca1a700 1 -- start start 2024-03-22T07:07:05.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.761+0000 7fddeca1a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdde8102bb0 0x7fdde8101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:05.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.761+0000 7fddeca1a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde8114260 con 0x7fdde8102bb0 2024-03-22T07:07:05.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.761+0000 7fdde659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdde8102bb0 0x7fdde8101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:05.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.761+0000 7fdde659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdde8102bb0 0x7fdde8101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60270/0 (socket says 172.21.15.63:60270) 2024-03-22T07:07:05.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.761+0000 7fdde659c700 1 -- 172.21.15.63:0/3371129743 learned_addr learned my addr 172.21.15.63:0/3371129743 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:05.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.761+0000 7fdde659c700 1 -- 172.21.15.63:0/3371129743 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc0094f0 con 0x7fdde8102bb0 2024-03-22T07:07:05.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.761+0000 7fdde659c700 1 --2- 172.21.15.63:0/3371129743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdde8102bb0 0x7fdde8101370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdddc003050 tx=0x7fdddc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:05.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.762+0000 7fddd6ffd700 1 -- 172.21.15.63:0/3371129743 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdddc010760 con 0x7fdde8102bb0 2024-03-22T07:07:05.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.762+0000 7fddd6ffd700 1 -- 172.21.15.63:0/3371129743 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fdddc010d50 con 0x7fdde8102bb0 2024-03-22T07:07:05.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.762+0000 7fddd6ffd700 1 -- 172.21.15.63:0/3371129743 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdddc01ae20 con 0x7fdde8102bb0 2024-03-22T07:07:05.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.762+0000 7fddeca1a700 1 -- 172.21.15.63:0/3371129743 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde81143a0 con 0x7fdde8102bb0 2024-03-22T07:07:05.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.762+0000 7fddeca1a700 1 -- 172.21.15.63:0/3371129743 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde8114890 con 0x7fdde8102bb0 2024-03-22T07:07:05.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.763+0000 7fddeca1a700 1 -- 172.21.15.63:0/3371129743 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde804bed0 con 0x7fdde8102bb0 2024-03-22T07:07:05.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.763+0000 7fddd6ffd700 1 -- 172.21.15.63:0/3371129743 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fdddc02e440 con 0x7fdde8102bb0 2024-03-22T07:07:05.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.763+0000 7fddd6ffd700 1 --2- 172.21.15.63:0/3371129743 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fddd00367e0 0x7fddd0038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:05.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.763+0000 7fddd6ffd700 1 -- 172.21.15.63:0/3371129743 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fdddc04a610 con 0x7fdde8102bb0 2024-03-22T07:07:05.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.763+0000 7fdde5d9b700 1 -- 172.21.15.63:0/3371129743 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fddd00367e0 msgr2=0x7fddd0038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2616455474 2024-03-22T07:07:05.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.763+0000 7fdde5d9b700 1 --2- 172.21.15.63:0/3371129743 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fddd00367e0 0x7fddd0038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:07:05.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.768+0000 7fddd6ffd700 1 -- 172.21.15.63:0/3371129743 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdddc018350 con 0x7fdde8102bb0 2024-03-22T07:07:05.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.964+0000 7fdde5d9b700 1 -- 172.21.15.63:0/3371129743 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fddd00367e0 msgr2=0x7fddd0038c90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2616455474 2024-03-22T07:07:05.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:04.964+0000 7fdde5d9b700 1 --2- 172.21.15.63:0/3371129743 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fddd00367e0 0x7fddd0038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T07:07:05.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:05.017+0000 7fddeca1a700 1 -- 172.21.15.63:0/3371129743 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fdde8114a20 con 0x7fdde8102bb0 2024-03-22T07:07:05.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:05.017+0000 7fddd6ffd700 1 -- 172.21.15.63:0/3371129743 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7fdddc023ab0 con 0x7fdde8102bb0 2024-03-22T07:07:05.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:05.019+0000 7fddeca1a700 1 -- 172.21.15.63:0/3371129743 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fddd00367e0 msgr2=0x7fddd0038c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:07:05.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:05.019+0000 7fddeca1a700 1 --2- 172.21.15.63:0/3371129743 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fddd00367e0 0x7fddd0038c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:05.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:05.019+0000 7fddeca1a700 1 -- 172.21.15.63:0/3371129743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdde8102bb0 msgr2=0x7fdde8101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:05.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:05.019+0000 7fddeca1a700 1 --2- 172.21.15.63:0/3371129743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdde8102bb0 0x7fdde8101370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdddc003050 tx=0x7fdddc009250 comp rx=0 tx=0).stop 2024-03-22T07:07:05.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:05.019+0000 7fddeca1a700 1 -- 172.21.15.63:0/3371129743 shutdown_connections 2024-03-22T07:07:05.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:05.019+0000 7fddeca1a700 1 --2- 172.21.15.63:0/3371129743 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7fddd00367e0 0x7fddd0038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:05.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:05.019+0000 7fddeca1a700 1 --2- 172.21.15.63:0/3371129743 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdde8102bb0 0x7fdde8101370 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:05.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:05.020+0000 7fddeca1a700 1 -- 172.21.15.63:0/3371129743 >> 172.21.15.63:0/3371129743 conn(0x7fdde80fc500 msgr2=0x7fdde80fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:05.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:05.020+0000 7fddeca1a700 1 -- 172.21.15.63:0/3371129743 shutdown_connections 2024-03-22T07:07:05.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:05.020+0000 7fddeca1a700 1 -- 172.21.15.63:0/3371129743 wait complete. 2024-03-22T07:07:05.728 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for the mgr to restart... 2024-03-22T07:07:05.728 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr epoch 5... 2024-03-22T07:07:05.913 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:05 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3371129743' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-22T07:07:10.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:10 smithi063 ceph-mon[29200]: Active manager daemon smithi063.mnxayh restarted 2024-03-22T07:07:10.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:10 smithi063 ceph-mon[29200]: Activating manager daemon smithi063.mnxayh 2024-03-22T07:07:10.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:10 smithi063 ceph-mon[29200]: osdmap e2: 0 total, 0 up, 0 in 2024-03-22T07:07:10.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:10 smithi063 ceph-mon[29200]: mgrmap e6: smithi063.mnxayh(active, starting, since 0.00326864s) 2024-03-22T07:07:10.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:10 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:07:10.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:10 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr metadata", "who": "smithi063.mnxayh", "id": "smithi063.mnxayh"}]: dispatch 2024-03-22T07:07:10.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:10 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:07:10.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:10 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:07:10.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:10 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:07:10.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:10 smithi063 ceph-mon[29200]: Manager daemon smithi063.mnxayh is now available 2024-03-22T07:07:10.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:10 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:10.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:10 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:10.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:10 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:07:10.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:10 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:07:10.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:10 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:07:11.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:11 smithi063 ceph-mon[29200]: Found migration_current of "None". Setting to last migration. 2024-03-22T07:07:11.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:11 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.mnxayh/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:07:11.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:11 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.mnxayh/trash_purge_schedule"}]: dispatch 2024-03-22T07:07:11.732 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:11 smithi063 ceph-mon[29200]: mgrmap e7: smithi063.mnxayh(active, since 1.00504s) 2024-03-22T07:07:12.093 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-03-22T07:07:12.093 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-22T07:07:12.093 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-22T07:07:12.093 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-03-22T07:07:12.093 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.853+0000 7f6f604b7700 1 Processor -- start 2024-03-22T07:07:12.093 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.854+0000 7f6f604b7700 1 -- start start 2024-03-22T07:07:12.093 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.854+0000 7f6f604b7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f58102720 0x7f6f58102b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:12.093 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.854+0000 7f6f604b7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f58103070 con 0x7f6f58102720 2024-03-22T07:07:12.094 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.854+0000 7f6f5e253700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f58102720 0x7f6f58102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:12.094 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.854+0000 7f6f5e253700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f58102720 0x7f6f58102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60274/0 (socket says 172.21.15.63:60274) 2024-03-22T07:07:12.094 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.854+0000 7f6f5e253700 1 -- 172.21.15.63:0/1216941417 learned_addr learned my addr 172.21.15.63:0/1216941417 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:12.094 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.854+0000 7f6f5e253700 1 -- 172.21.15.63:0/1216941417 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f581031b0 con 0x7f6f58102720 2024-03-22T07:07:12.094 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.855+0000 7f6f5e253700 1 --2- 172.21.15.63:0/1216941417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f58102720 0x7f6f58102b30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6f540098e0 tx=0x7f6f54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd867db73b5b3f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:12.094 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.855+0000 7f6f5d251700 1 -- 172.21.15.63:0/1216941417 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6f5400b860 con 0x7f6f58102720 2024-03-22T07:07:12.094 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.855+0000 7f6f5d251700 1 -- 172.21.15.63:0/1216941417 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f6f5400be50 con 0x7f6f58102720 2024-03-22T07:07:12.094 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.855+0000 7f6f5d251700 1 -- 172.21.15.63:0/1216941417 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6f54010380 con 0x7f6f58102720 2024-03-22T07:07:12.094 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.855+0000 7f6f604b7700 1 -- 172.21.15.63:0/1216941417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f58102720 msgr2=0x7f6f58102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:12.094 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.855+0000 7f6f604b7700 1 --2- 172.21.15.63:0/1216941417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f58102720 0x7f6f58102b30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6f540098e0 tx=0x7f6f54009250 comp rx=0 tx=0).stop 2024-03-22T07:07:12.094 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.855+0000 7f6f604b7700 1 -- 172.21.15.63:0/1216941417 shutdown_connections 2024-03-22T07:07:12.094 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.856+0000 7f6f604b7700 1 --2- 172.21.15.63:0/1216941417 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f58102720 0x7f6f58102b30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:12.094 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.856+0000 7f6f604b7700 1 -- 172.21.15.63:0/1216941417 >> 172.21.15.63:0/1216941417 conn(0x7f6f580fde40 msgr2=0x7f6f581002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:12.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.856+0000 7f6f604b7700 1 -- 172.21.15.63:0/1216941417 shutdown_connections 2024-03-22T07:07:12.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.856+0000 7f6f604b7700 1 -- 172.21.15.63:0/1216941417 wait complete. 2024-03-22T07:07:12.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.856+0000 7f6f604b7700 1 Processor -- start 2024-03-22T07:07:12.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.856+0000 7f6f604b7700 1 -- start start 2024-03-22T07:07:12.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.856+0000 7f6f604b7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f58102720 0x7f6f58111550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:12.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.856+0000 7f6f604b7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f58111a90 con 0x7f6f58102720 2024-03-22T07:07:12.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.856+0000 7f6f5e253700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f58102720 0x7f6f58111550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:12.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.856+0000 7f6f5e253700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f58102720 0x7f6f58111550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60278/0 (socket says 172.21.15.63:60278) 2024-03-22T07:07:12.095 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.856+0000 7f6f5e253700 1 -- 172.21.15.63:0/468042602 learned_addr learned my addr 172.21.15.63:0/468042602 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:12.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.857+0000 7f6f5e253700 1 -- 172.21.15.63:0/468042602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f540094f0 con 0x7f6f58102720 2024-03-22T07:07:12.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.857+0000 7f6f5e253700 1 --2- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f58102720 0x7f6f58111550 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400b440 tx=0x7f6f54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:12.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.857+0000 7f6f4effd700 1 -- 172.21.15.63:0/468042602 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6f54010760 con 0x7f6f58102720 2024-03-22T07:07:12.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.857+0000 7f6f4effd700 1 -- 172.21.15.63:0/468042602 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f6f54010d50 con 0x7f6f58102720 2024-03-22T07:07:12.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.857+0000 7f6f4effd700 1 -- 172.21.15.63:0/468042602 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6f5401ae20 con 0x7f6f58102720 2024-03-22T07:07:12.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.857+0000 7f6f604b7700 1 -- 172.21.15.63:0/468042602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f58111c90 con 0x7f6f58102720 2024-03-22T07:07:12.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.857+0000 7f6f604b7700 1 -- 172.21.15.63:0/468042602 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f58112800 con 0x7f6f58102720 2024-03-22T07:07:12.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.858+0000 7f6f4effd700 1 -- 172.21.15.63:0/468042602 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 5) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f6f5402e440 con 0x7f6f58102720 2024-03-22T07:07:12.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.858+0000 7f6f4effd700 1 --2- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f6f44036770 0x7f6f44038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:12.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.858+0000 7f6f4effd700 1 -- 172.21.15.63:0/468042602 --> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f44039330 con 0x7f6f44036770 2024-03-22T07:07:12.096 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.858+0000 7f6f4effd700 1 -- 172.21.15.63:0/468042602 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f6f5404a610 con 0x7f6f58102720 2024-03-22T07:07:12.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.858+0000 7f6f5da52700 1 -- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f6f44036770 msgr2=0x7f6f44038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2616455474 2024-03-22T07:07:12.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:06.858+0000 7f6f5da52700 1 --2- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f6f44036770 0x7f6f44038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:07:12.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:07.059+0000 7f6f5da52700 1 -- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f6f44036770 msgr2=0x7f6f44038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2616455474 2024-03-22T07:07:12.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:07.059+0000 7f6f5da52700 1 --2- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f6f44036770 0x7f6f44038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T07:07:12.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:07.460+0000 7f6f5da52700 1 -- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f6f44036770 msgr2=0x7f6f44038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2616455474 2024-03-22T07:07:12.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:07.460+0000 7f6f5da52700 1 --2- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f6f44036770 0x7f6f44038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-22T07:07:12.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:08.261+0000 7f6f5da52700 1 -- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f6f44036770 msgr2=0x7f6f44038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2616455474 2024-03-22T07:07:12.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:08.261+0000 7f6f5da52700 1 --2- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f6f44036770 0x7f6f44038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-22T07:07:12.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:09.862+0000 7f6f5da52700 1 -- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f6f44036770 msgr2=0x7f6f44038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2616455474 2024-03-22T07:07:12.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:09.862+0000 7f6f5da52700 1 --2- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f6f44036770 0x7f6f44038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-22T07:07:12.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:10.419+0000 7f6f4effd700 1 -- 172.21.15.63:0/468042602 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mgrmap(e 6) v1 ==== 43135+0+0 (secure 0 0 0) 0x7f6f5402e790 con 0x7f6f58102720 2024-03-22T07:07:12.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:10.419+0000 7f6f4effd700 1 -- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f6f44036770 msgr2=0x7f6f44038c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:07:12.097 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:10.419+0000 7f6f4effd700 1 --2- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f6f44036770 0x7f6f44038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:12.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.420+0000 7f6f4effd700 1 -- 172.21.15.63:0/468042602 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 7) v1 ==== 43262+0+0 (secure 0 0 0) 0x7f6f5404c070 con 0x7f6f58102720 2024-03-22T07:07:12.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.420+0000 7f6f4effd700 1 --2- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f6f44039ff0 0x7f6f4403c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:12.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.420+0000 7f6f4effd700 1 -- 172.21.15.63:0/468042602 --> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f44039330 con 0x7f6f44039ff0 2024-03-22T07:07:12.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.420+0000 7f6f5da52700 1 --2- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f6f44039ff0 0x7f6f4403c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:12.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.421+0000 7f6f5da52700 1 --2- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f6f44039ff0 0x7f6f4403c3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6f480039d0 tx=0x7f6f48009350 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:12.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.421+0000 7f6f4effd700 1 -- 172.21.15.63:0/468042602 <== mgr.14120 v2:172.21.15.63:6800/480818549 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f6f44039330 con 0x7f6f44039ff0 2024-03-22T07:07:12.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.427+0000 7f6f604b7700 1 -- 172.21.15.63:0/468042602 --> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f6f58112120 con 0x7f6f44039ff0 2024-03-22T07:07:12.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.427+0000 7f6f4effd700 1 -- 172.21.15.63:0/468042602 <== mgr.14120 v2:172.21.15.63:6800/480818549 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f6f58112120 con 0x7f6f44039ff0 2024-03-22T07:07:12.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.428+0000 7f6f604b7700 1 -- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f6f44039ff0 msgr2=0x7f6f4403c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:12.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.428+0000 7f6f604b7700 1 --2- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f6f44039ff0 0x7f6f4403c3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6f480039d0 tx=0x7f6f48009350 comp rx=0 tx=0).stop 2024-03-22T07:07:12.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.428+0000 7f6f604b7700 1 -- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f58102720 msgr2=0x7f6f58111550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:12.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.428+0000 7f6f604b7700 1 --2- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f58102720 0x7f6f58111550 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6f5400b440 tx=0x7f6f54009250 comp rx=0 tx=0).stop 2024-03-22T07:07:12.098 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.428+0000 7f6f604b7700 1 -- 172.21.15.63:0/468042602 shutdown_connections 2024-03-22T07:07:12.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.428+0000 7f6f604b7700 1 --2- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f6f44039ff0 0x7f6f4403c3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:12.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.428+0000 7f6f604b7700 1 --2- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6f58102720 0x7f6f58111550 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:12.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.428+0000 7f6f604b7700 1 --2- 172.21.15.63:0/468042602 >> [v2:172.21.15.63:6800/2616455474,v1:172.21.15.63:6801/2616455474] conn(0x7f6f44036770 0x7f6f44038c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:12.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.428+0000 7f6f604b7700 1 -- 172.21.15.63:0/468042602 >> 172.21.15.63:0/468042602 conn(0x7f6f580fde40 msgr2=0x7f6f580fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:12.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.428+0000 7f6f604b7700 1 -- 172.21.15.63:0/468042602 shutdown_connections 2024-03-22T07:07:12.099 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:11.428+0000 7f6f604b7700 1 -- 172.21.15.63:0/468042602 wait complete. 2024-03-22T07:07:12.099 INFO:teuthology.orchestra.run.smithi063.stdout:mgr epoch 5 is available 2024-03-22T07:07:12.099 INFO:teuthology.orchestra.run.smithi063.stdout:Setting orchestrator backend to cephadm... 2024-03-22T07:07:12.718 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:12 smithi063 ceph-mon[29200]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-22T07:07:12.718 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:12 smithi063 ceph-mon[29200]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-22T07:07:13.303 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.500+0000 7f84d1388700 1 Processor -- start 2024-03-22T07:07:13.304 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.500+0000 7f84d1388700 1 -- start start 2024-03-22T07:07:13.304 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.500+0000 7f84d1388700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc076be0 0x7f84cc105310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:13.304 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.500+0000 7f84d1388700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc105850 con 0x7f84cc076be0 2024-03-22T07:07:13.304 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.500+0000 7f84caffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc076be0 0x7f84cc105310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:13.304 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.500+0000 7f84caffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc076be0 0x7f84cc105310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60366/0 (socket says 172.21.15.63:60366) 2024-03-22T07:07:13.304 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.500+0000 7f84caffd700 1 -- 172.21.15.63:0/3975443359 learned_addr learned my addr 172.21.15.63:0/3975443359 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:13.304 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.501+0000 7f84caffd700 1 -- 172.21.15.63:0/3975443359 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84cc105990 con 0x7f84cc076be0 2024-03-22T07:07:13.304 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.501+0000 7f84caffd700 1 --2- 172.21.15.63:0/3975443359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc076be0 0x7f84cc105310 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f84c00098e0 tx=0x7f84c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd91793fdd1c2fda server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:13.304 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.501+0000 7f84c9ffb700 1 -- 172.21.15.63:0/3975443359 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84c000b860 con 0x7f84cc076be0 2024-03-22T07:07:13.304 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.501+0000 7f84c9ffb700 1 -- 172.21.15.63:0/3975443359 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f84c000be50 con 0x7f84cc076be0 2024-03-22T07:07:13.304 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.501+0000 7f84c9ffb700 1 -- 172.21.15.63:0/3975443359 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84c0010380 con 0x7f84cc076be0 2024-03-22T07:07:13.304 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.501+0000 7f84d1388700 1 -- 172.21.15.63:0/3975443359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc076be0 msgr2=0x7f84cc105310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:13.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.501+0000 7f84d1388700 1 --2- 172.21.15.63:0/3975443359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc076be0 0x7f84cc105310 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f84c00098e0 tx=0x7f84c0009250 comp rx=0 tx=0).stop 2024-03-22T07:07:13.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.501+0000 7f84d1388700 1 -- 172.21.15.63:0/3975443359 shutdown_connections 2024-03-22T07:07:13.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.501+0000 7f84d1388700 1 --2- 172.21.15.63:0/3975443359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc076be0 0x7f84cc105310 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:13.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.501+0000 7f84d1388700 1 -- 172.21.15.63:0/3975443359 >> 172.21.15.63:0/3975443359 conn(0x7f84cc0fcdc0 msgr2=0x7f84cc0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:13.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.502+0000 7f84d1388700 1 -- 172.21.15.63:0/3975443359 shutdown_connections 2024-03-22T07:07:13.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.502+0000 7f84d1388700 1 -- 172.21.15.63:0/3975443359 wait complete. 2024-03-22T07:07:13.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.502+0000 7f84d1388700 1 Processor -- start 2024-03-22T07:07:13.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.502+0000 7f84d1388700 1 -- start start 2024-03-22T07:07:13.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.502+0000 7f84d1388700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc076be0 0x7f84cc102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:13.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.502+0000 7f84d1388700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc1031a0 con 0x7f84cc076be0 2024-03-22T07:07:13.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.503+0000 7f84caffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc076be0 0x7f84cc102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:13.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.503+0000 7f84caffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc076be0 0x7f84cc102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60372/0 (socket says 172.21.15.63:60372) 2024-03-22T07:07:13.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.503+0000 7f84caffd700 1 -- 172.21.15.63:0/1008839770 learned_addr learned my addr 172.21.15.63:0/1008839770 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:13.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.503+0000 7f84caffd700 1 -- 172.21.15.63:0/1008839770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c00094f0 con 0x7f84cc076be0 2024-03-22T07:07:13.305 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.503+0000 7f84caffd700 1 --2- 172.21.15.63:0/1008839770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc076be0 0x7f84cc102c60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f84c0003050 tx=0x7f84c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:13.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.503+0000 7f84b37fe700 1 -- 172.21.15.63:0/1008839770 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84c0010760 con 0x7f84cc076be0 2024-03-22T07:07:13.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.503+0000 7f84d1388700 1 -- 172.21.15.63:0/1008839770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84cc101210 con 0x7f84cc076be0 2024-03-22T07:07:13.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.503+0000 7f84b37fe700 1 -- 172.21.15.63:0/1008839770 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f84c0010d50 con 0x7f84cc076be0 2024-03-22T07:07:13.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.503+0000 7f84b37fe700 1 -- 172.21.15.63:0/1008839770 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84c001ae20 con 0x7f84cc076be0 2024-03-22T07:07:13.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.503+0000 7f84d1388700 1 -- 172.21.15.63:0/1008839770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84cc101700 con 0x7f84cc076be0 2024-03-22T07:07:13.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.504+0000 7f84b37fe700 1 -- 172.21.15.63:0/1008839770 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f84c00108c0 con 0x7f84cc076be0 2024-03-22T07:07:13.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.504+0000 7f84b37fe700 1 --2- 172.21.15.63:0/1008839770 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f84b40367e0 0x7f84b4038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:13.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.504+0000 7f84d1388700 1 -- 172.21.15.63:0/1008839770 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84cc04bed0 con 0x7f84cc076be0 2024-03-22T07:07:13.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.504+0000 7f84b37fe700 1 -- 172.21.15.63:0/1008839770 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f84c004a4d0 con 0x7f84cc076be0 2024-03-22T07:07:13.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.504+0000 7f84ca7fc700 1 --2- 172.21.15.63:0/1008839770 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f84b40367e0 0x7f84b4038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:13.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.505+0000 7f84ca7fc700 1 --2- 172.21.15.63:0/1008839770 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f84b40367e0 0x7f84b4038c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009990 tx=0x7f84bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:13.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.509+0000 7f84b37fe700 1 -- 172.21.15.63:0/1008839770 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f84c001a430 con 0x7f84cc076be0 2024-03-22T07:07:13.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.717+0000 7f84d1388700 1 -- 172.21.15.63:0/1008839770 --> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f84cc101e10 con 0x7f84b40367e0 2024-03-22T07:07:13.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.726+0000 7f84b37fe700 1 -- 172.21.15.63:0/1008839770 <== mgr.14120 v2:172.21.15.63:6800/480818549 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f84cc101e10 con 0x7f84b40367e0 2024-03-22T07:07:13.306 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.729+0000 7f84d1388700 1 -- 172.21.15.63:0/1008839770 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f84b40367e0 msgr2=0x7f84b4038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:13.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.729+0000 7f84d1388700 1 --2- 172.21.15.63:0/1008839770 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f84b40367e0 0x7f84b4038c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009990 tx=0x7f84bc006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:13.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.729+0000 7f84d1388700 1 -- 172.21.15.63:0/1008839770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc076be0 msgr2=0x7f84cc102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:13.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.729+0000 7f84d1388700 1 --2- 172.21.15.63:0/1008839770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc076be0 0x7f84cc102c60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f84c0003050 tx=0x7f84c0009250 comp rx=0 tx=0).stop 2024-03-22T07:07:13.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.729+0000 7f84d1388700 1 -- 172.21.15.63:0/1008839770 shutdown_connections 2024-03-22T07:07:13.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.729+0000 7f84d1388700 1 --2- 172.21.15.63:0/1008839770 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f84b40367e0 0x7f84b4038c90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:13.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.729+0000 7f84d1388700 1 --2- 172.21.15.63:0/1008839770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84cc076be0 0x7f84cc102c60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:13.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.729+0000 7f84d1388700 1 -- 172.21.15.63:0/1008839770 >> 172.21.15.63:0/1008839770 conn(0x7f84cc0fcdc0 msgr2=0x7f84cc0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:13.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.729+0000 7f84d1388700 1 -- 172.21.15.63:0/1008839770 shutdown_connections 2024-03-22T07:07:13.307 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:12.730+0000 7f84d1388700 1 -- 172.21.15.63:0/1008839770 wait complete. 2024-03-22T07:07:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:13 smithi063 ceph-mon[29200]: mgrmap e8: smithi063.mnxayh(active, since 2s) 2024-03-22T07:07:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:13 smithi063 ceph-mon[29200]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:13 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:13 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:07:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:13 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:13 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:07:14.405 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-22T07:07:14.405 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.734+0000 7efe65ee6700 1 Processor -- start 2024-03-22T07:07:14.405 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.734+0000 7efe65ee6700 1 -- start start 2024-03-22T07:07:14.405 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.734+0000 7efe65ee6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe60102730 0x7efe60102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:14.405 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.734+0000 7efe65ee6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe60103080 con 0x7efe60102730 2024-03-22T07:07:14.405 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.734+0000 7efe5f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe60102730 0x7efe60102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:14.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.734+0000 7efe5f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe60102730 0x7efe60102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60382/0 (socket says 172.21.15.63:60382) 2024-03-22T07:07:14.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.734+0000 7efe5f7fe700 1 -- 172.21.15.63:0/2118461349 learned_addr learned my addr 172.21.15.63:0/2118461349 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:14.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.734+0000 7efe5f7fe700 1 -- 172.21.15.63:0/2118461349 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe601031c0 con 0x7efe60102730 2024-03-22T07:07:14.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.734+0000 7efe5f7fe700 1 --2- 172.21.15.63:0/2118461349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe60102730 0x7efe60102b40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efe540098e0 tx=0x7efe54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df69f3c3cc81881 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:14.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.735+0000 7efe5e7fc700 1 -- 172.21.15.63:0/2118461349 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe5400b860 con 0x7efe60102730 2024-03-22T07:07:14.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.735+0000 7efe5e7fc700 1 -- 172.21.15.63:0/2118461349 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7efe5400be50 con 0x7efe60102730 2024-03-22T07:07:14.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.735+0000 7efe5e7fc700 1 -- 172.21.15.63:0/2118461349 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe54010380 con 0x7efe60102730 2024-03-22T07:07:14.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.735+0000 7efe65ee6700 1 -- 172.21.15.63:0/2118461349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe60102730 msgr2=0x7efe60102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:14.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.735+0000 7efe65ee6700 1 --2- 172.21.15.63:0/2118461349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe60102730 0x7efe60102b40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7efe540098e0 tx=0x7efe54009250 comp rx=0 tx=0).stop 2024-03-22T07:07:14.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.735+0000 7efe65ee6700 1 -- 172.21.15.63:0/2118461349 shutdown_connections 2024-03-22T07:07:14.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.735+0000 7efe65ee6700 1 --2- 172.21.15.63:0/2118461349 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe60102730 0x7efe60102b40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:14.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.735+0000 7efe65ee6700 1 -- 172.21.15.63:0/2118461349 >> 172.21.15.63:0/2118461349 conn(0x7efe600fdf10 msgr2=0x7efe60100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:14.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.735+0000 7efe65ee6700 1 -- 172.21.15.63:0/2118461349 shutdown_connections 2024-03-22T07:07:14.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.735+0000 7efe65ee6700 1 -- 172.21.15.63:0/2118461349 wait complete. 2024-03-22T07:07:14.406 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.736+0000 7efe65ee6700 1 Processor -- start 2024-03-22T07:07:14.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.736+0000 7efe65ee6700 1 -- start start 2024-03-22T07:07:14.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.736+0000 7efe65ee6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe60102730 0x7efe60078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:14.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.736+0000 7efe65ee6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe60079110 con 0x7efe60102730 2024-03-22T07:07:14.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.736+0000 7efe5f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe60102730 0x7efe60078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:14.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.736+0000 7efe5f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe60102730 0x7efe60078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60392/0 (socket says 172.21.15.63:60392) 2024-03-22T07:07:14.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.736+0000 7efe5f7fe700 1 -- 172.21.15.63:0/1316013436 learned_addr learned my addr 172.21.15.63:0/1316013436 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:14.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.736+0000 7efe5f7fe700 1 -- 172.21.15.63:0/1316013436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe540094f0 con 0x7efe60102730 2024-03-22T07:07:14.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.737+0000 7efe5f7fe700 1 --2- 172.21.15.63:0/1316013436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe60102730 0x7efe60078bd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efe54006fd0 tx=0x7efe54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:14.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.737+0000 7efe47fff700 1 -- 172.21.15.63:0/1316013436 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe540107c0 con 0x7efe60102730 2024-03-22T07:07:14.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.737+0000 7efe47fff700 1 -- 172.21.15.63:0/1316013436 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7efe54010db0 con 0x7efe60102730 2024-03-22T07:07:14.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.737+0000 7efe65ee6700 1 -- 172.21.15.63:0/1316013436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe60077180 con 0x7efe60102730 2024-03-22T07:07:14.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.737+0000 7efe65ee6700 1 -- 172.21.15.63:0/1316013436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe60077670 con 0x7efe60102730 2024-03-22T07:07:14.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.737+0000 7efe47fff700 1 -- 172.21.15.63:0/1316013436 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe5401ada0 con 0x7efe60102730 2024-03-22T07:07:14.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.738+0000 7efe47fff700 1 -- 172.21.15.63:0/1316013436 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7efe54010920 con 0x7efe60102730 2024-03-22T07:07:14.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.738+0000 7efe65ee6700 1 -- 172.21.15.63:0/1316013436 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6004bed0 con 0x7efe60102730 2024-03-22T07:07:14.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.738+0000 7efe47fff700 1 --2- 172.21.15.63:0/1316013436 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7efe48036790 0x7efe48038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:14.407 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.738+0000 7efe47fff700 1 -- 172.21.15.63:0/1316013436 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7efe54049d50 con 0x7efe60102730 2024-03-22T07:07:14.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.738+0000 7efe5effd700 1 --2- 172.21.15.63:0/1316013436 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7efe48036790 0x7efe48038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:14.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.738+0000 7efe5effd700 1 --2- 172.21.15.63:0/1316013436 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7efe48036790 0x7efe48038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efe50009990 tx=0x7efe50006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:14.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.743+0000 7efe47fff700 1 -- 172.21.15.63:0/1316013436 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7efe5404d030 con 0x7efe60102730 2024-03-22T07:07:14.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.939+0000 7efe65ee6700 1 -- 172.21.15.63:0/1316013436 --> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7efe60077d70 con 0x7efe48036790 2024-03-22T07:07:14.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.939+0000 7efe47fff700 1 -- 172.21.15.63:0/1316013436 <== mgr.14120 v2:172.21.15.63:6800/480818549 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7efe60077d70 con 0x7efe48036790 2024-03-22T07:07:14.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.941+0000 7efe65ee6700 1 -- 172.21.15.63:0/1316013436 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7efe48036790 msgr2=0x7efe48038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:14.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.941+0000 7efe65ee6700 1 --2- 172.21.15.63:0/1316013436 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7efe48036790 0x7efe48038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efe50009990 tx=0x7efe50006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:14.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.941+0000 7efe65ee6700 1 -- 172.21.15.63:0/1316013436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe60102730 msgr2=0x7efe60078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:14.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.941+0000 7efe65ee6700 1 --2- 172.21.15.63:0/1316013436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe60102730 0x7efe60078bd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efe54006fd0 tx=0x7efe54009250 comp rx=0 tx=0).stop 2024-03-22T07:07:14.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.941+0000 7efe65ee6700 1 -- 172.21.15.63:0/1316013436 shutdown_connections 2024-03-22T07:07:14.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.941+0000 7efe65ee6700 1 --2- 172.21.15.63:0/1316013436 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7efe48036790 0x7efe48038c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:14.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.941+0000 7efe65ee6700 1 --2- 172.21.15.63:0/1316013436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efe60102730 0x7efe60078bd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:14.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.941+0000 7efe65ee6700 1 -- 172.21.15.63:0/1316013436 >> 172.21.15.63:0/1316013436 conn(0x7efe600fdf10 msgr2=0x7efe600feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:14.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.941+0000 7efe65ee6700 1 -- 172.21.15.63:0/1316013436 shutdown_connections 2024-03-22T07:07:14.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:13.941+0000 7efe65ee6700 1 -- 172.21.15.63:0/1316013436 wait complete. 2024-03-22T07:07:14.409 INFO:teuthology.orchestra.run.smithi063.stdout:Generating ssh key... 2024-03-22T07:07:14.714 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:14 smithi063 ceph-mon[29200]: [22/Mar/2024:07:07:13] ENGINE Bus STARTING 2024-03-22T07:07:14.714 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:14 smithi063 ceph-mon[29200]: [22/Mar/2024:07:07:13] ENGINE Serving on https://172.21.15.63:7150 2024-03-22T07:07:14.715 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:14 smithi063 ceph-mon[29200]: [22/Mar/2024:07:07:13] ENGINE Bus STARTED 2024-03-22T07:07:15.542 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:15 smithi063 ceph-mon[29200]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:15.542 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:15 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:15.542 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:15 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:15.800 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.810+0000 7fd825be2700 1 Processor -- start 2024-03-22T07:07:15.800 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.810+0000 7fd825be2700 1 -- start start 2024-03-22T07:07:15.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.811+0000 7fd825be2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8201026b0 0x7fd820102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:15.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.811+0000 7fd825be2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd820103000 con 0x7fd8201026b0 2024-03-22T07:07:15.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.811+0000 7fd81f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8201026b0 0x7fd820102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:15.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.811+0000 7fd81f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8201026b0 0x7fd820102ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52000/0 (socket says 172.21.15.63:52000) 2024-03-22T07:07:15.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.811+0000 7fd81f7fe700 1 -- 172.21.15.63:0/344960497 learned_addr learned my addr 172.21.15.63:0/344960497 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:15.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.811+0000 7fd81f7fe700 1 -- 172.21.15.63:0/344960497 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd820103140 con 0x7fd8201026b0 2024-03-22T07:07:15.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.811+0000 7fd81f7fe700 1 --2- 172.21.15.63:0/344960497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8201026b0 0x7fd820102ac0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd8100098e0 tx=0x7fd810009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281b406a89d7cab9 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:15.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.811+0000 7fd81e7fc700 1 -- 172.21.15.63:0/344960497 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd81000b860 con 0x7fd8201026b0 2024-03-22T07:07:15.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.811+0000 7fd81e7fc700 1 -- 172.21.15.63:0/344960497 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd81000be50 con 0x7fd8201026b0 2024-03-22T07:07:15.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.811+0000 7fd81e7fc700 1 -- 172.21.15.63:0/344960497 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd810010380 con 0x7fd8201026b0 2024-03-22T07:07:15.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.812+0000 7fd825be2700 1 -- 172.21.15.63:0/344960497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8201026b0 msgr2=0x7fd820102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:15.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.812+0000 7fd825be2700 1 --2- 172.21.15.63:0/344960497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8201026b0 0x7fd820102ac0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd8100098e0 tx=0x7fd810009250 comp rx=0 tx=0).stop 2024-03-22T07:07:15.801 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.812+0000 7fd825be2700 1 -- 172.21.15.63:0/344960497 shutdown_connections 2024-03-22T07:07:15.802 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.812+0000 7fd825be2700 1 --2- 172.21.15.63:0/344960497 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8201026b0 0x7fd820102ac0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:15.802 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.812+0000 7fd825be2700 1 -- 172.21.15.63:0/344960497 >> 172.21.15.63:0/344960497 conn(0x7fd8200fde70 msgr2=0x7fd820100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:15.802 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.812+0000 7fd825be2700 1 -- 172.21.15.63:0/344960497 shutdown_connections 2024-03-22T07:07:15.802 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.812+0000 7fd825be2700 1 -- 172.21.15.63:0/344960497 wait complete. 2024-03-22T07:07:15.802 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.812+0000 7fd825be2700 1 Processor -- start 2024-03-22T07:07:15.802 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.813+0000 7fd825be2700 1 -- start start 2024-03-22T07:07:15.802 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.813+0000 7fd825be2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8201026b0 0x7fd820111540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:15.802 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.813+0000 7fd825be2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd820111a80 con 0x7fd8201026b0 2024-03-22T07:07:15.802 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.813+0000 7fd81f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8201026b0 0x7fd820111540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:15.802 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.813+0000 7fd81f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8201026b0 0x7fd820111540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52010/0 (socket says 172.21.15.63:52010) 2024-03-22T07:07:15.802 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.813+0000 7fd81f7fe700 1 -- 172.21.15.63:0/4160716108 learned_addr learned my addr 172.21.15.63:0/4160716108 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:15.802 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.813+0000 7fd81f7fe700 1 -- 172.21.15.63:0/4160716108 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8100094f0 con 0x7fd8201026b0 2024-03-22T07:07:15.802 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.813+0000 7fd81f7fe700 1 --2- 172.21.15.63:0/4160716108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8201026b0 0x7fd820111540 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd81000b440 tx=0x7fd810009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:15.802 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.813+0000 7fd807fff700 1 -- 172.21.15.63:0/4160716108 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd810010760 con 0x7fd8201026b0 2024-03-22T07:07:15.803 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.813+0000 7fd825be2700 1 -- 172.21.15.63:0/4160716108 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd820111c80 con 0x7fd8201026b0 2024-03-22T07:07:15.803 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.813+0000 7fd807fff700 1 -- 172.21.15.63:0/4160716108 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd810010d50 con 0x7fd8201026b0 2024-03-22T07:07:15.803 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.813+0000 7fd807fff700 1 -- 172.21.15.63:0/4160716108 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd81001ae20 con 0x7fd8201026b0 2024-03-22T07:07:15.803 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.814+0000 7fd825be2700 1 -- 172.21.15.63:0/4160716108 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8201127f0 con 0x7fd8201026b0 2024-03-22T07:07:15.803 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.814+0000 7fd807fff700 1 -- 172.21.15.63:0/4160716108 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fd81002e440 con 0x7fd8201026b0 2024-03-22T07:07:15.803 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.814+0000 7fd825be2700 1 -- 172.21.15.63:0/4160716108 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd82005f5f0 con 0x7fd8201026b0 2024-03-22T07:07:15.803 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.814+0000 7fd807fff700 1 --2- 172.21.15.63:0/4160716108 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fd808036770 0x7fd808038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:15.803 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.815+0000 7fd807fff700 1 -- 172.21.15.63:0/4160716108 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd81004a630 con 0x7fd8201026b0 2024-03-22T07:07:15.803 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.815+0000 7fd81effd700 1 --2- 172.21.15.63:0/4160716108 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fd808036770 0x7fd808038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:15.803 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.815+0000 7fd81effd700 1 --2- 172.21.15.63:0/4160716108 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fd808036770 0x7fd808038c20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd814009990 tx=0x7fd814006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:15.803 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:14.819+0000 7fd807fff700 1 -- 172.21.15.63:0/4160716108 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd810018940 con 0x7fd8201026b0 2024-03-22T07:07:15.803 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:15.014+0000 7fd825be2700 1 -- 172.21.15.63:0/4160716108 --> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8200ff0e0 con 0x7fd808036770 2024-03-22T07:07:15.803 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:15.285+0000 7fd807fff700 1 -- 172.21.15.63:0/4160716108 <== mgr.14120 v2:172.21.15.63:6800/480818549 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd8200ff0e0 con 0x7fd808036770 2024-03-22T07:07:15.804 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:15.287+0000 7fd825be2700 1 -- 172.21.15.63:0/4160716108 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fd808036770 msgr2=0x7fd808038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:15.804 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:15.287+0000 7fd825be2700 1 --2- 172.21.15.63:0/4160716108 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fd808036770 0x7fd808038c20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd814009990 tx=0x7fd814006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:15.804 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:15.287+0000 7fd825be2700 1 -- 172.21.15.63:0/4160716108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8201026b0 msgr2=0x7fd820111540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:15.804 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:15.287+0000 7fd825be2700 1 --2- 172.21.15.63:0/4160716108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8201026b0 0x7fd820111540 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd81000b440 tx=0x7fd810009250 comp rx=0 tx=0).stop 2024-03-22T07:07:15.804 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:15.288+0000 7fd825be2700 1 -- 172.21.15.63:0/4160716108 shutdown_connections 2024-03-22T07:07:15.804 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:15.288+0000 7fd825be2700 1 --2- 172.21.15.63:0/4160716108 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fd808036770 0x7fd808038c20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:15.804 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:15.288+0000 7fd825be2700 1 --2- 172.21.15.63:0/4160716108 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd8201026b0 0x7fd820111540 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:15.804 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:15.288+0000 7fd825be2700 1 -- 172.21.15.63:0/4160716108 >> 172.21.15.63:0/4160716108 conn(0x7fd8200fde70 msgr2=0x7fd8200fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:15.804 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:15.288+0000 7fd825be2700 1 -- 172.21.15.63:0/4160716108 shutdown_connections 2024-03-22T07:07:15.804 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:15.288+0000 7fd825be2700 1 -- 172.21.15.63:0/4160716108 wait complete. 2024-03-22T07:07:16.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:16 smithi063 ceph-mon[29200]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:16.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:16 smithi063 ceph-mon[29200]: Generating ssh key... 2024-03-22T07:07:16.957 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSXTVRL4AK7J0O4BEHH/BCPSmyKTf294IiJuyEVljae8krjCePgETJQ3GFUSUudhKV2ePgmI1IzQ/ubAlK6mFZGyRh+B9ClXZCFnrospYzvA9o/2slnDRjWuHj7lEfm2rx58BC93R6VMHW7jnYGQRvxBmbO84cklb372J9TnvVXIhY/nCiTptgy+1Wa8KGjzNSXxFxa88mSQgV3jQW/88DS+a4u0MSofXZYVG0c8FC7Hjl2aDrECJVlY654W7+z2Q5dKwKuDl/0Wgl6l6KrF6p2HDxhGnlj/cR21OXefutGqzKcDltn2sVln8XOZxZmG6vUt7o/0KUZN0s49Jip9S6awIpqVHG2IRNmlVX6V+NtF6FG2GfXVmhQWjI3iw/pjQPFGtgdiSQLCCQg4tZi4EmMSWoVJpUIgEAxF1W+y+YO4llIOIB0/NhZEQOEAqhT1v/QB+nsCV4Q4vODBkGaXSJSgoSyh3xDED7tOSJjIC+A/BPEK0/TAH6zlVWr5iDFUs= ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:07:16.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.168+0000 7f1f39d61700 1 Processor -- start 2024-03-22T07:07:16.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.168+0000 7f1f39d61700 1 -- start start 2024-03-22T07:07:16.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.168+0000 7f1f39d61700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f34076b90 0x7f1f341053d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:16.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.168+0000 7f1f39d61700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f34105910 con 0x7f1f34076b90 2024-03-22T07:07:16.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.168+0000 7f1f337fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f34076b90 0x7f1f341053d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:16.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.168+0000 7f1f337fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f34076b90 0x7f1f341053d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52024/0 (socket says 172.21.15.63:52024) 2024-03-22T07:07:16.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.168+0000 7f1f337fe700 1 -- 172.21.15.63:0/3890982137 learned_addr learned my addr 172.21.15.63:0/3890982137 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:16.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.168+0000 7f1f337fe700 1 -- 172.21.15.63:0/3890982137 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f34105a50 con 0x7f1f34076b90 2024-03-22T07:07:16.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.169+0000 7f1f337fe700 1 --2- 172.21.15.63:0/3890982137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f34076b90 0x7f1f341053d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f240098e0 tx=0x7f1f24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1805b37cdc7f336d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:16.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.169+0000 7f1f327fc700 1 -- 172.21.15.63:0/3890982137 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f2400b8f0 con 0x7f1f34076b90 2024-03-22T07:07:16.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.169+0000 7f1f327fc700 1 -- 172.21.15.63:0/3890982137 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1f24011420 con 0x7f1f34076b90 2024-03-22T07:07:16.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.169+0000 7f1f327fc700 1 -- 172.21.15.63:0/3890982137 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f24010540 con 0x7f1f34076b90 2024-03-22T07:07:16.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.169+0000 7f1f39d61700 1 -- 172.21.15.63:0/3890982137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f34076b90 msgr2=0x7f1f341053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:16.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.169+0000 7f1f39d61700 1 --2- 172.21.15.63:0/3890982137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f34076b90 0x7f1f341053d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f240098e0 tx=0x7f1f24009250 comp rx=0 tx=0).stop 2024-03-22T07:07:16.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.169+0000 7f1f39d61700 1 -- 172.21.15.63:0/3890982137 shutdown_connections 2024-03-22T07:07:16.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.169+0000 7f1f39d61700 1 --2- 172.21.15.63:0/3890982137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f34076b90 0x7f1f341053d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:16.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.169+0000 7f1f39d61700 1 -- 172.21.15.63:0/3890982137 >> 172.21.15.63:0/3890982137 conn(0x7f1f340fce40 msgr2=0x7f1f340ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:16.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.169+0000 7f1f39d61700 1 -- 172.21.15.63:0/3890982137 shutdown_connections 2024-03-22T07:07:16.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.169+0000 7f1f39d61700 1 -- 172.21.15.63:0/3890982137 wait complete. 2024-03-22T07:07:16.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.170+0000 7f1f39d61700 1 Processor -- start 2024-03-22T07:07:16.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.170+0000 7f1f39d61700 1 -- start start 2024-03-22T07:07:16.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.170+0000 7f1f39d61700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f34076b90 0x7f1f34111640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:16.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.170+0000 7f1f39d61700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f34111b80 con 0x7f1f34076b90 2024-03-22T07:07:16.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.170+0000 7f1f337fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f34076b90 0x7f1f34111640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:16.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.170+0000 7f1f337fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f34076b90 0x7f1f34111640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52040/0 (socket says 172.21.15.63:52040) 2024-03-22T07:07:16.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.170+0000 7f1f337fe700 1 -- 172.21.15.63:0/559596938 learned_addr learned my addr 172.21.15.63:0/559596938 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:16.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.171+0000 7f1f337fe700 1 -- 172.21.15.63:0/559596938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f240094f0 con 0x7f1f34076b90 2024-03-22T07:07:16.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.171+0000 7f1f337fe700 1 --2- 172.21.15.63:0/559596938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f34076b90 0x7f1f34111640 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1f24011db0 tx=0x7f1f24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:16.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.171+0000 7f1f1bfff700 1 -- 172.21.15.63:0/559596938 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f240107b0 con 0x7f1f34076b90 2024-03-22T07:07:16.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.171+0000 7f1f39d61700 1 -- 172.21.15.63:0/559596938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f34111d80 con 0x7f1f34076b90 2024-03-22T07:07:16.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.171+0000 7f1f1bfff700 1 -- 172.21.15.63:0/559596938 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1f24010da0 con 0x7f1f34076b90 2024-03-22T07:07:16.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.171+0000 7f1f1bfff700 1 -- 172.21.15.63:0/559596938 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1f2401ad60 con 0x7f1f34076b90 2024-03-22T07:07:16.962 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.171+0000 7f1f39d61700 1 -- 172.21.15.63:0/559596938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f341128f0 con 0x7f1f34076b90 2024-03-22T07:07:16.962 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.172+0000 7f1f1bfff700 1 -- 172.21.15.63:0/559596938 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f1f2402e440 con 0x7f1f34076b90 2024-03-22T07:07:16.962 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.172+0000 7f1f1bfff700 1 --2- 172.21.15.63:0/559596938 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f1f1c036790 0x7f1f1c038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:16.962 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.172+0000 7f1f1bfff700 1 -- 172.21.15.63:0/559596938 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f1f2404acb0 con 0x7f1f34076b90 2024-03-22T07:07:16.962 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.172+0000 7f1f39d61700 1 -- 172.21.15.63:0/559596938 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3405f5f0 con 0x7f1f34076b90 2024-03-22T07:07:16.962 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.172+0000 7f1f32ffd700 1 --2- 172.21.15.63:0/559596938 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f1f1c036790 0x7f1f1c038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:16.962 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.173+0000 7f1f32ffd700 1 --2- 172.21.15.63:0/559596938 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f1f1c036790 0x7f1f1c038c40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009990 tx=0x7f1f28006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:16.962 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.177+0000 7f1f1bfff700 1 -- 172.21.15.63:0/559596938 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1f24024440 con 0x7f1f34076b90 2024-03-22T07:07:16.962 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.375+0000 7f1f39d61700 1 -- 172.21.15.63:0/559596938 --> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f3405f5f0 con 0x7f1f1c036790 2024-03-22T07:07:16.963 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.376+0000 7f1f1bfff700 1 -- 172.21.15.63:0/559596938 <== mgr.14120 v2:172.21.15.63:6800/480818549 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f1f3405f5f0 con 0x7f1f1c036790 2024-03-22T07:07:16.963 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.378+0000 7f1f39d61700 1 -- 172.21.15.63:0/559596938 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f1f1c036790 msgr2=0x7f1f1c038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:16.963 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.378+0000 7f1f39d61700 1 --2- 172.21.15.63:0/559596938 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f1f1c036790 0x7f1f1c038c40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009990 tx=0x7f1f28006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:16.963 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.378+0000 7f1f39d61700 1 -- 172.21.15.63:0/559596938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f34076b90 msgr2=0x7f1f34111640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:16.963 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.378+0000 7f1f39d61700 1 --2- 172.21.15.63:0/559596938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f34076b90 0x7f1f34111640 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1f24011db0 tx=0x7f1f24009250 comp rx=0 tx=0).stop 2024-03-22T07:07:16.963 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.378+0000 7f1f39d61700 1 -- 172.21.15.63:0/559596938 shutdown_connections 2024-03-22T07:07:16.963 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.378+0000 7f1f39d61700 1 --2- 172.21.15.63:0/559596938 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f1f1c036790 0x7f1f1c038c40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:16.963 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.378+0000 7f1f39d61700 1 --2- 172.21.15.63:0/559596938 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1f34076b90 0x7f1f34111640 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:16.963 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.378+0000 7f1f39d61700 1 -- 172.21.15.63:0/559596938 >> 172.21.15.63:0/559596938 conn(0x7f1f340fce40 msgr2=0x7f1f340fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:16.963 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.378+0000 7f1f39d61700 1 -- 172.21.15.63:0/559596938 shutdown_connections 2024-03-22T07:07:16.963 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:16.378+0000 7f1f39d61700 1 -- 172.21.15.63:0/559596938 wait complete. 2024-03-22T07:07:16.963 INFO:teuthology.orchestra.run.smithi063.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-22T07:07:16.963 INFO:teuthology.orchestra.run.smithi063.stdout:Adding key to root@localhost authorized_keys... 2024-03-22T07:07:16.963 INFO:teuthology.orchestra.run.smithi063.stdout:Adding host smithi063... 2024-03-22T07:07:17.754 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:17 smithi063 ceph-mon[29200]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:18.694 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:18 smithi063 ceph-mon[29200]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi063", "addr": "172.21.15.63", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:19.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:19 smithi063 ceph-mon[29200]: Deploying cephadm binary to smithi063 2024-03-22T07:07:20.224 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Added host 'smithi063' with addr '172.21.15.63' 2024-03-22T07:07:20.224 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.318+0000 7f3add0d4700 1 Processor -- start 2024-03-22T07:07:20.224 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.318+0000 7f3add0d4700 1 -- start start 2024-03-22T07:07:20.224 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.318+0000 7f3add0d4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ad8102740 0x7f3ad8102b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:20.224 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.318+0000 7f3add0d4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad8103090 con 0x7f3ad8102740 2024-03-22T07:07:20.224 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.318+0000 7f3ad6d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ad8102740 0x7f3ad8102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:20.224 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.318+0000 7f3ad6d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ad8102740 0x7f3ad8102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52052/0 (socket says 172.21.15.63:52052) 2024-03-22T07:07:20.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.318+0000 7f3ad6d9d700 1 -- 172.21.15.63:0/4293977414 learned_addr learned my addr 172.21.15.63:0/4293977414 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:20.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.318+0000 7f3ad6d9d700 1 -- 172.21.15.63:0/4293977414 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad81031d0 con 0x7f3ad8102740 2024-03-22T07:07:20.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.319+0000 7f3ad6d9d700 1 --2- 172.21.15.63:0/4293977414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ad8102740 0x7f3ad8102b50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80098e0 tx=0x7f3ac8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5472fabe3f8d53ca server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:20.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.319+0000 7f3ad5d9b700 1 -- 172.21.15.63:0/4293977414 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ac800b860 con 0x7f3ad8102740 2024-03-22T07:07:20.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.319+0000 7f3ad5d9b700 1 -- 172.21.15.63:0/4293977414 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3ac800be50 con 0x7f3ad8102740 2024-03-22T07:07:20.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.319+0000 7f3ad5d9b700 1 -- 172.21.15.63:0/4293977414 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ac8010380 con 0x7f3ad8102740 2024-03-22T07:07:20.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.319+0000 7f3add0d4700 1 -- 172.21.15.63:0/4293977414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ad8102740 msgr2=0x7f3ad8102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:20.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.319+0000 7f3add0d4700 1 --2- 172.21.15.63:0/4293977414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ad8102740 0x7f3ad8102b50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80098e0 tx=0x7f3ac8009250 comp rx=0 tx=0).stop 2024-03-22T07:07:20.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.319+0000 7f3add0d4700 1 -- 172.21.15.63:0/4293977414 shutdown_connections 2024-03-22T07:07:20.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.319+0000 7f3add0d4700 1 --2- 172.21.15.63:0/4293977414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ad8102740 0x7f3ad8102b50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:20.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.319+0000 7f3add0d4700 1 -- 172.21.15.63:0/4293977414 >> 172.21.15.63:0/4293977414 conn(0x7f3ad80fdea0 msgr2=0x7f3ad81002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:20.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.319+0000 7f3add0d4700 1 -- 172.21.15.63:0/4293977414 shutdown_connections 2024-03-22T07:07:20.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.320+0000 7f3add0d4700 1 -- 172.21.15.63:0/4293977414 wait complete. 2024-03-22T07:07:20.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.320+0000 7f3add0d4700 1 Processor -- start 2024-03-22T07:07:20.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.320+0000 7f3add0d4700 1 -- start start 2024-03-22T07:07:20.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.320+0000 7f3add0d4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ad8102740 0x7f3ad81115b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:20.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.320+0000 7f3add0d4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad8111af0 con 0x7f3ad8102740 2024-03-22T07:07:20.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.320+0000 7f3ad6d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ad8102740 0x7f3ad81115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:20.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.320+0000 7f3ad6d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ad8102740 0x7f3ad81115b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52062/0 (socket says 172.21.15.63:52062) 2024-03-22T07:07:20.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.320+0000 7f3ad6d9d700 1 -- 172.21.15.63:0/2393216610 learned_addr learned my addr 172.21.15.63:0/2393216610 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:20.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.321+0000 7f3ad6d9d700 1 -- 172.21.15.63:0/2393216610 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac80094f0 con 0x7f3ad8102740 2024-03-22T07:07:20.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.321+0000 7f3ad6d9d700 1 --2- 172.21.15.63:0/2393216610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ad8102740 0x7f3ad81115b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800ba90 tx=0x7f3ac8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:20.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.321+0000 7f3abf7fe700 1 -- 172.21.15.63:0/2393216610 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ac80107c0 con 0x7f3ad8102740 2024-03-22T07:07:20.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.321+0000 7f3abf7fe700 1 -- 172.21.15.63:0/2393216610 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3ac8010db0 con 0x7f3ad8102740 2024-03-22T07:07:20.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.321+0000 7f3add0d4700 1 -- 172.21.15.63:0/2393216610 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad8111cf0 con 0x7f3ad8102740 2024-03-22T07:07:20.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.321+0000 7f3abf7fe700 1 -- 172.21.15.63:0/2393216610 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ac801add0 con 0x7f3ad8102740 2024-03-22T07:07:20.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.321+0000 7f3add0d4700 1 -- 172.21.15.63:0/2393216610 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad8112860 con 0x7f3ad8102740 2024-03-22T07:07:20.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.322+0000 7f3abf7fe700 1 -- 172.21.15.63:0/2393216610 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f3ac802e440 con 0x7f3ad8102740 2024-03-22T07:07:20.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.322+0000 7f3abf7fe700 1 --2- 172.21.15.63:0/2393216610 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f3ac00367e0 0x7f3ac0038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:20.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.322+0000 7f3add0d4700 1 -- 172.21.15.63:0/2393216610 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad805f5f0 con 0x7f3ad8102740 2024-03-22T07:07:20.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.322+0000 7f3abf7fe700 1 -- 172.21.15.63:0/2393216610 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f3ac804a720 con 0x7f3ad8102740 2024-03-22T07:07:20.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.322+0000 7f3ad659c700 1 --2- 172.21.15.63:0/2393216610 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f3ac00367e0 0x7f3ac0038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:20.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.323+0000 7f3ad659c700 1 --2- 172.21.15.63:0/2393216610 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f3ac00367e0 0x7f3ac0038c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009990 tx=0x7f3acc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:20.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.327+0000 7f3abf7fe700 1 -- 172.21.15.63:0/2393216610 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3ac8023b40 con 0x7f3ad8102740 2024-03-22T07:07:20.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:17.528+0000 7f3add0d4700 1 -- 172.21.15.63:0/2393216610 --> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi063", "addr": "172.21.15.63", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ad80ff120 con 0x7f3ac00367e0 2024-03-22T07:07:20.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:19.612+0000 7f3abf7fe700 1 -- 172.21.15.63:0/2393216610 <== mgr.14120 v2:172.21.15.63:6800/480818549 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f3ad80ff120 con 0x7f3ac00367e0 2024-03-22T07:07:20.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:19.615+0000 7f3add0d4700 1 -- 172.21.15.63:0/2393216610 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f3ac00367e0 msgr2=0x7f3ac0038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:20.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:19.615+0000 7f3add0d4700 1 --2- 172.21.15.63:0/2393216610 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f3ac00367e0 0x7f3ac0038c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009990 tx=0x7f3acc006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:20.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:19.615+0000 7f3add0d4700 1 -- 172.21.15.63:0/2393216610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ad8102740 msgr2=0x7f3ad81115b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:20.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:19.615+0000 7f3add0d4700 1 --2- 172.21.15.63:0/2393216610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ad8102740 0x7f3ad81115b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800ba90 tx=0x7f3ac8009250 comp rx=0 tx=0).stop 2024-03-22T07:07:20.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:19.615+0000 7f3add0d4700 1 -- 172.21.15.63:0/2393216610 shutdown_connections 2024-03-22T07:07:20.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:19.615+0000 7f3add0d4700 1 --2- 172.21.15.63:0/2393216610 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f3ac00367e0 0x7f3ac0038c90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:20.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:19.615+0000 7f3add0d4700 1 --2- 172.21.15.63:0/2393216610 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3ad8102740 0x7f3ad81115b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:20.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:19.615+0000 7f3add0d4700 1 -- 172.21.15.63:0/2393216610 >> 172.21.15.63:0/2393216610 conn(0x7f3ad80fdea0 msgr2=0x7f3ad80fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:20.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:19.615+0000 7f3add0d4700 1 -- 172.21.15.63:0/2393216610 shutdown_connections 2024-03-22T07:07:20.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:19.615+0000 7f3add0d4700 1 -- 172.21.15.63:0/2393216610 wait complete. 2024-03-22T07:07:20.228 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying mon service with default placement... 2024-03-22T07:07:20.864 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:20 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:20.864 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:20 smithi063 ceph-mon[29200]: Added host smithi063 2024-03-22T07:07:20.864 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:20 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:07:21.400 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-22T07:07:21.400 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.627+0000 7fa818bda700 1 Processor -- start 2024-03-22T07:07:21.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.627+0000 7fa818bda700 1 -- start start 2024-03-22T07:07:21.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.627+0000 7fa818bda700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa814076bb0 0x7fa8141053b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:21.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.627+0000 7fa818bda700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8141058f0 con 0x7fa814076bb0 2024-03-22T07:07:21.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.627+0000 7fa81259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa814076bb0 0x7fa8141053b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:21.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.628+0000 7fa81259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa814076bb0 0x7fa8141053b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52078/0 (socket says 172.21.15.63:52078) 2024-03-22T07:07:21.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.628+0000 7fa81259c700 1 -- 172.21.15.63:0/2111077588 learned_addr learned my addr 172.21.15.63:0/2111077588 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:21.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.628+0000 7fa81259c700 1 -- 172.21.15.63:0/2111077588 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa814105a30 con 0x7fa814076bb0 2024-03-22T07:07:21.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.628+0000 7fa81259c700 1 --2- 172.21.15.63:0/2111077588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa814076bb0 0x7fa8141053b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa8040098e0 tx=0x7fa804009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329a58519bcff343 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:21.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.628+0000 7fa81159a700 1 -- 172.21.15.63:0/2111077588 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa80400b860 con 0x7fa814076bb0 2024-03-22T07:07:21.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.628+0000 7fa81159a700 1 -- 172.21.15.63:0/2111077588 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa80400be50 con 0x7fa814076bb0 2024-03-22T07:07:21.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.628+0000 7fa81159a700 1 -- 172.21.15.63:0/2111077588 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa804010380 con 0x7fa814076bb0 2024-03-22T07:07:21.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.629+0000 7fa818bda700 1 -- 172.21.15.63:0/2111077588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa814076bb0 msgr2=0x7fa8141053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:21.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.629+0000 7fa818bda700 1 --2- 172.21.15.63:0/2111077588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa814076bb0 0x7fa8141053b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa8040098e0 tx=0x7fa804009250 comp rx=0 tx=0).stop 2024-03-22T07:07:21.401 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.629+0000 7fa818bda700 1 -- 172.21.15.63:0/2111077588 shutdown_connections 2024-03-22T07:07:21.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.629+0000 7fa818bda700 1 --2- 172.21.15.63:0/2111077588 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa814076bb0 0x7fa8141053b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:21.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.629+0000 7fa818bda700 1 -- 172.21.15.63:0/2111077588 >> 172.21.15.63:0/2111077588 conn(0x7fa8140fce40 msgr2=0x7fa8140ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:21.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.629+0000 7fa818bda700 1 -- 172.21.15.63:0/2111077588 shutdown_connections 2024-03-22T07:07:21.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.629+0000 7fa818bda700 1 -- 172.21.15.63:0/2111077588 wait complete. 2024-03-22T07:07:21.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.629+0000 7fa818bda700 1 Processor -- start 2024-03-22T07:07:21.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.629+0000 7fa818bda700 1 -- start start 2024-03-22T07:07:21.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.629+0000 7fa818bda700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa814076bb0 0x7fa814102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:21.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.629+0000 7fa818bda700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa814103240 con 0x7fa814076bb0 2024-03-22T07:07:21.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.630+0000 7fa81259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa814076bb0 0x7fa814102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:21.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.630+0000 7fa81259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa814076bb0 0x7fa814102d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52082/0 (socket says 172.21.15.63:52082) 2024-03-22T07:07:21.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.630+0000 7fa81259c700 1 -- 172.21.15.63:0/2845205676 learned_addr learned my addr 172.21.15.63:0/2845205676 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:21.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.630+0000 7fa81259c700 1 -- 172.21.15.63:0/2845205676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8040094f0 con 0x7fa814076bb0 2024-03-22T07:07:21.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.630+0000 7fa81259c700 1 --2- 172.21.15.63:0/2845205676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa814076bb0 0x7fa814102d00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa804011bd0 tx=0x7fa804009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:21.402 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.630+0000 7fa802ffd700 1 -- 172.21.15.63:0/2845205676 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa8040107c0 con 0x7fa814076bb0 2024-03-22T07:07:21.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.630+0000 7fa802ffd700 1 -- 172.21.15.63:0/2845205676 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa804010db0 con 0x7fa814076bb0 2024-03-22T07:07:21.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.630+0000 7fa818bda700 1 -- 172.21.15.63:0/2845205676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8141012b0 con 0x7fa814076bb0 2024-03-22T07:07:21.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.630+0000 7fa802ffd700 1 -- 172.21.15.63:0/2845205676 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa80401ad60 con 0x7fa814076bb0 2024-03-22T07:07:21.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.630+0000 7fa818bda700 1 -- 172.21.15.63:0/2845205676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8141017a0 con 0x7fa814076bb0 2024-03-22T07:07:21.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.631+0000 7fa802ffd700 1 -- 172.21.15.63:0/2845205676 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fa80401a5c0 con 0x7fa814076bb0 2024-03-22T07:07:21.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.631+0000 7fa802ffd700 1 --2- 172.21.15.63:0/2845205676 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fa7fc0367e0 0x7fa7fc038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:21.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.631+0000 7fa818bda700 1 -- 172.21.15.63:0/2845205676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa81404bed0 con 0x7fa814076bb0 2024-03-22T07:07:21.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.631+0000 7fa802ffd700 1 -- 172.21.15.63:0/2845205676 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fa80404ab20 con 0x7fa814076bb0 2024-03-22T07:07:21.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.631+0000 7fa811d9b700 1 --2- 172.21.15.63:0/2845205676 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fa7fc0367e0 0x7fa7fc038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:21.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.632+0000 7fa811d9b700 1 --2- 172.21.15.63:0/2845205676 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fa7fc0367e0 0x7fa7fc038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa808009990 tx=0x7fa808006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:21.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.636+0000 7fa802ffd700 1 -- 172.21.15.63:0/2845205676 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa804023df0 con 0x7fa814076bb0 2024-03-22T07:07:21.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.855+0000 7fa818bda700 1 -- 172.21.15.63:0/2845205676 --> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fa814101eb0 con 0x7fa7fc0367e0 2024-03-22T07:07:21.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.863+0000 7fa802ffd700 1 -- 172.21.15.63:0/2845205676 <== mgr.14120 v2:172.21.15.63:6800/480818549 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa814101eb0 con 0x7fa7fc0367e0 2024-03-22T07:07:21.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.865+0000 7fa818bda700 1 -- 172.21.15.63:0/2845205676 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fa7fc0367e0 msgr2=0x7fa7fc038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:21.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.865+0000 7fa818bda700 1 --2- 172.21.15.63:0/2845205676 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fa7fc0367e0 0x7fa7fc038c90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa808009990 tx=0x7fa808006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:21.403 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.865+0000 7fa818bda700 1 -- 172.21.15.63:0/2845205676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa814076bb0 msgr2=0x7fa814102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:21.404 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.865+0000 7fa818bda700 1 --2- 172.21.15.63:0/2845205676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa814076bb0 0x7fa814102d00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa804011bd0 tx=0x7fa804009250 comp rx=0 tx=0).stop 2024-03-22T07:07:21.404 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.865+0000 7fa818bda700 1 -- 172.21.15.63:0/2845205676 shutdown_connections 2024-03-22T07:07:21.404 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.865+0000 7fa818bda700 1 --2- 172.21.15.63:0/2845205676 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fa7fc0367e0 0x7fa7fc038c90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:21.404 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.865+0000 7fa818bda700 1 --2- 172.21.15.63:0/2845205676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa814076bb0 0x7fa814102d00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:21.404 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.865+0000 7fa818bda700 1 -- 172.21.15.63:0/2845205676 >> 172.21.15.63:0/2845205676 conn(0x7fa8140fce40 msgr2=0x7fa8140fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:21.404 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.865+0000 7fa818bda700 1 -- 172.21.15.63:0/2845205676 shutdown_connections 2024-03-22T07:07:21.404 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:20.865+0000 7fa818bda700 1 -- 172.21.15.63:0/2845205676 wait complete. 2024-03-22T07:07:21.404 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying mgr service with default placement... 2024-03-22T07:07:22.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:21 smithi063 ceph-mon[29200]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:22.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:21 smithi063 ceph-mon[29200]: Saving service mon spec with placement count:5 2024-03-22T07:07:22.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:21 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:23.255 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-22T07:07:23.256 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.372+0000 7f84e9695700 1 Processor -- start 2024-03-22T07:07:23.256 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.372+0000 7f84e9695700 1 -- start start 2024-03-22T07:07:23.256 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.372+0000 7f84e9695700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84e4071360 0x7f84e41052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:23.256 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.372+0000 7f84e9695700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e4105820 con 0x7f84e4071360 2024-03-22T07:07:23.256 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.372+0000 7f84e2ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84e4071360 0x7f84e41052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:23.256 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.372+0000 7f84e2ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84e4071360 0x7f84e41052e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52092/0 (socket says 172.21.15.63:52092) 2024-03-22T07:07:23.256 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.372+0000 7f84e2ffd700 1 -- 172.21.15.63:0/1446408614 learned_addr learned my addr 172.21.15.63:0/1446408614 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:23.256 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.372+0000 7f84e2ffd700 1 -- 172.21.15.63:0/1446408614 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e4105960 con 0x7f84e4071360 2024-03-22T07:07:23.256 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.373+0000 7f84e2ffd700 1 --2- 172.21.15.63:0/1446408614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84e4071360 0x7f84e41052e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f84d40098e0 tx=0x7f84d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938eb83caf8a99f server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:23.256 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.373+0000 7f84e1ffb700 1 -- 172.21.15.63:0/1446408614 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84d400b860 con 0x7f84e4071360 2024-03-22T07:07:23.256 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.373+0000 7f84e1ffb700 1 -- 172.21.15.63:0/1446408614 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f84d400be50 con 0x7f84e4071360 2024-03-22T07:07:23.256 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.373+0000 7f84e1ffb700 1 -- 172.21.15.63:0/1446408614 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84d4010380 con 0x7f84e4071360 2024-03-22T07:07:23.256 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.373+0000 7f84e9695700 1 -- 172.21.15.63:0/1446408614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84e4071360 msgr2=0x7f84e41052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:23.256 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.373+0000 7f84e9695700 1 --2- 172.21.15.63:0/1446408614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84e4071360 0x7f84e41052e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f84d40098e0 tx=0x7f84d4009250 comp rx=0 tx=0).stop 2024-03-22T07:07:23.257 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.373+0000 7f84e9695700 1 -- 172.21.15.63:0/1446408614 shutdown_connections 2024-03-22T07:07:23.257 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.373+0000 7f84e9695700 1 --2- 172.21.15.63:0/1446408614 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84e4071360 0x7f84e41052e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:23.257 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.373+0000 7f84e9695700 1 -- 172.21.15.63:0/1446408614 >> 172.21.15.63:0/1446408614 conn(0x7f84e40fcd50 msgr2=0x7f84e40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:23.257 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.373+0000 7f84e9695700 1 -- 172.21.15.63:0/1446408614 shutdown_connections 2024-03-22T07:07:23.257 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.373+0000 7f84e9695700 1 -- 172.21.15.63:0/1446408614 wait complete. 2024-03-22T07:07:23.257 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.374+0000 7f84e9695700 1 Processor -- start 2024-03-22T07:07:23.257 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.374+0000 7f84e9695700 1 -- start start 2024-03-22T07:07:23.257 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.374+0000 7f84e9695700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84e4071360 0x7f84e4102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:23.257 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.374+0000 7f84e9695700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e4103170 con 0x7f84e4071360 2024-03-22T07:07:23.257 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.374+0000 7f84e2ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84e4071360 0x7f84e4102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:23.257 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.375+0000 7f84e2ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84e4071360 0x7f84e4102c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52100/0 (socket says 172.21.15.63:52100) 2024-03-22T07:07:23.257 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.375+0000 7f84e2ffd700 1 -- 172.21.15.63:0/587357211 learned_addr learned my addr 172.21.15.63:0/587357211 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:23.257 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.375+0000 7f84e2ffd700 1 -- 172.21.15.63:0/587357211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d40094f0 con 0x7f84e4071360 2024-03-22T07:07:23.257 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.375+0000 7f84e2ffd700 1 --2- 172.21.15.63:0/587357211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84e4071360 0x7f84e4102c30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f84d40090b0 tx=0x7f84d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:23.257 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.375+0000 7f84cb7fe700 1 -- 172.21.15.63:0/587357211 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84d4010760 con 0x7f84e4071360 2024-03-22T07:07:23.258 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.375+0000 7f84e9695700 1 -- 172.21.15.63:0/587357211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e41011e0 con 0x7f84e4071360 2024-03-22T07:07:23.258 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.375+0000 7f84cb7fe700 1 -- 172.21.15.63:0/587357211 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f84d4010d50 con 0x7f84e4071360 2024-03-22T07:07:23.258 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.375+0000 7f84cb7fe700 1 -- 172.21.15.63:0/587357211 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f84d401ace0 con 0x7f84e4071360 2024-03-22T07:07:23.258 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.375+0000 7f84e9695700 1 -- 172.21.15.63:0/587357211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e41016d0 con 0x7f84e4071360 2024-03-22T07:07:23.258 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.376+0000 7f84cb7fe700 1 -- 172.21.15.63:0/587357211 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f84d402d3f0 con 0x7f84e4071360 2024-03-22T07:07:23.258 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.376+0000 7f84cb7fe700 1 --2- 172.21.15.63:0/587357211 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f84cc036790 0x7f84cc038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:23.258 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.376+0000 7f84cb7fe700 1 -- 172.21.15.63:0/587357211 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f84d404a200 con 0x7f84e4071360 2024-03-22T07:07:23.258 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.376+0000 7f84e9695700 1 -- 172.21.15.63:0/587357211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e404bed0 con 0x7f84e4071360 2024-03-22T07:07:23.258 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.376+0000 7f84e27fc700 1 --2- 172.21.15.63:0/587357211 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f84cc036790 0x7f84cc038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:23.258 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.377+0000 7f84e27fc700 1 --2- 172.21.15.63:0/587357211 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f84cc036790 0x7f84cc038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009990 tx=0x7f84d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:23.258 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.381+0000 7f84cb7fe700 1 -- 172.21.15.63:0/587357211 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f84d402b8e0 con 0x7f84e4071360 2024-03-22T07:07:23.258 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.588+0000 7f84e9695700 1 -- 172.21.15.63:0/587357211 --> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f84e4101ce0 con 0x7f84cc036790 2024-03-22T07:07:23.258 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.592+0000 7f84cb7fe700 1 -- 172.21.15.63:0/587357211 <== mgr.14120 v2:172.21.15.63:6800/480818549 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f84e4101ce0 con 0x7f84cc036790 2024-03-22T07:07:23.258 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.594+0000 7f84e9695700 1 -- 172.21.15.63:0/587357211 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f84cc036790 msgr2=0x7f84cc038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:23.258 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.594+0000 7f84e9695700 1 --2- 172.21.15.63:0/587357211 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f84cc036790 0x7f84cc038c40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009990 tx=0x7f84d8006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:23.259 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.594+0000 7f84e9695700 1 -- 172.21.15.63:0/587357211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84e4071360 msgr2=0x7f84e4102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:23.259 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.594+0000 7f84e9695700 1 --2- 172.21.15.63:0/587357211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84e4071360 0x7f84e4102c30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f84d40090b0 tx=0x7f84d4009250 comp rx=0 tx=0).stop 2024-03-22T07:07:23.259 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.594+0000 7f84e9695700 1 -- 172.21.15.63:0/587357211 shutdown_connections 2024-03-22T07:07:23.259 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.594+0000 7f84e9695700 1 --2- 172.21.15.63:0/587357211 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f84cc036790 0x7f84cc038c40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:23.259 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.594+0000 7f84e9695700 1 --2- 172.21.15.63:0/587357211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f84e4071360 0x7f84e4102c30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:23.259 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.594+0000 7f84e9695700 1 -- 172.21.15.63:0/587357211 >> 172.21.15.63:0/587357211 conn(0x7f84e40fcd50 msgr2=0x7f84e40fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:23.259 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.594+0000 7f84e9695700 1 -- 172.21.15.63:0/587357211 shutdown_connections 2024-03-22T07:07:23.259 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:22.594+0000 7f84e9695700 1 -- 172.21.15.63:0/587357211 wait complete. 2024-03-22T07:07:23.259 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying crash service with default placement... 2024-03-22T07:07:23.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:23 smithi063 ceph-mon[29200]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:23.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:23 smithi063 ceph-mon[29200]: Saving service mgr spec with placement count:2 2024-03-22T07:07:23.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:23 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:23.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:23 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:23.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:23 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:24.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-03-22T07:07:24.958 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.666+0000 7ff408f98700 1 Processor -- start 2024-03-22T07:07:24.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.666+0000 7ff408f98700 1 -- start start 2024-03-22T07:07:24.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.666+0000 7ff408f98700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff404078ce0 0x7ff4040771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:24.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.666+0000 7ff408f98700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040790f0 con 0x7ff404078ce0 2024-03-22T07:07:24.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.667+0000 7ff40259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff404078ce0 0x7ff4040771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:24.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.667+0000 7ff40259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff404078ce0 0x7ff4040771e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52116/0 (socket says 172.21.15.63:52116) 2024-03-22T07:07:24.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.667+0000 7ff40259c700 1 -- 172.21.15.63:0/2444936449 learned_addr learned my addr 172.21.15.63:0/2444936449 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:24.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.667+0000 7ff40259c700 1 -- 172.21.15.63:0/2444936449 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff404077720 con 0x7ff404078ce0 2024-03-22T07:07:24.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.667+0000 7ff40259c700 1 --2- 172.21.15.63:0/2444936449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff404078ce0 0x7ff4040771e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40098e0 tx=0x7ff3f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568f6740e74b7b2d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:24.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.667+0000 7ff40159a700 1 -- 172.21.15.63:0/2444936449 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3f400b860 con 0x7ff404078ce0 2024-03-22T07:07:24.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.667+0000 7ff40159a700 1 -- 172.21.15.63:0/2444936449 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3f400be90 con 0x7ff404078ce0 2024-03-22T07:07:24.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.667+0000 7ff40159a700 1 -- 172.21.15.63:0/2444936449 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3f40103c0 con 0x7ff404078ce0 2024-03-22T07:07:24.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.668+0000 7ff408f98700 1 -- 172.21.15.63:0/2444936449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff404078ce0 msgr2=0x7ff4040771e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:24.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.668+0000 7ff408f98700 1 --2- 172.21.15.63:0/2444936449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff404078ce0 0x7ff4040771e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40098e0 tx=0x7ff3f4009250 comp rx=0 tx=0).stop 2024-03-22T07:07:24.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.668+0000 7ff408f98700 1 -- 172.21.15.63:0/2444936449 shutdown_connections 2024-03-22T07:07:24.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.668+0000 7ff408f98700 1 --2- 172.21.15.63:0/2444936449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff404078ce0 0x7ff4040771e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:24.959 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.668+0000 7ff408f98700 1 -- 172.21.15.63:0/2444936449 >> 172.21.15.63:0/2444936449 conn(0x7ff4040fdf10 msgr2=0x7ff404100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:24.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.668+0000 7ff408f98700 1 -- 172.21.15.63:0/2444936449 shutdown_connections 2024-03-22T07:07:24.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.668+0000 7ff408f98700 1 -- 172.21.15.63:0/2444936449 wait complete. 2024-03-22T07:07:24.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.669+0000 7ff408f98700 1 Processor -- start 2024-03-22T07:07:24.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.669+0000 7ff408f98700 1 -- start start 2024-03-22T07:07:24.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.669+0000 7ff408f98700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff404078ce0 0x7ff4041137f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:24.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.669+0000 7ff408f98700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff404113d30 con 0x7ff404078ce0 2024-03-22T07:07:24.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.669+0000 7ff40259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff404078ce0 0x7ff4041137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:24.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.669+0000 7ff40259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff404078ce0 0x7ff4041137f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52124/0 (socket says 172.21.15.63:52124) 2024-03-22T07:07:24.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.669+0000 7ff40259c700 1 -- 172.21.15.63:0/56284390 learned_addr learned my addr 172.21.15.63:0/56284390 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:24.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.669+0000 7ff40259c700 1 -- 172.21.15.63:0/56284390 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f40094f0 con 0x7ff404078ce0 2024-03-22T07:07:24.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.669+0000 7ff40259c700 1 --2- 172.21.15.63:0/56284390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff404078ce0 0x7ff4041137f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40094c0 tx=0x7ff3f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:24.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.669+0000 7ff3f2ffd700 1 -- 172.21.15.63:0/56284390 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3f4010890 con 0x7ff404078ce0 2024-03-22T07:07:24.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.669+0000 7ff3f2ffd700 1 -- 172.21.15.63:0/56284390 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3f4010ec0 con 0x7ff404078ce0 2024-03-22T07:07:24.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.669+0000 7ff408f98700 1 -- 172.21.15.63:0/56284390 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff404113f30 con 0x7ff404078ce0 2024-03-22T07:07:24.960 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.670+0000 7ff3f2ffd700 1 -- 172.21.15.63:0/56284390 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3f40243f0 con 0x7ff404078ce0 2024-03-22T07:07:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.670+0000 7ff408f98700 1 -- 172.21.15.63:0/56284390 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff404114aa0 con 0x7ff404078ce0 2024-03-22T07:07:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.670+0000 7ff3f2ffd700 1 -- 172.21.15.63:0/56284390 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7ff3f402d3f0 con 0x7ff404078ce0 2024-03-22T07:07:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.671+0000 7ff3f2ffd700 1 --2- 172.21.15.63:0/56284390 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7ff3ec0367e0 0x7ff3ec038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.671+0000 7ff3f2ffd700 1 -- 172.21.15.63:0/56284390 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff3f404a290 con 0x7ff404078ce0 2024-03-22T07:07:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.671+0000 7ff408f98700 1 -- 172.21.15.63:0/56284390 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40405f5f0 con 0x7ff404078ce0 2024-03-22T07:07:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.671+0000 7ff401d9b700 1 --2- 172.21.15.63:0/56284390 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7ff3ec0367e0 0x7ff3ec038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.671+0000 7ff401d9b700 1 --2- 172.21.15.63:0/56284390 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7ff3ec0367e0 0x7ff3ec038c90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009990 tx=0x7ff3f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.675+0000 7ff3f2ffd700 1 -- 172.21.15.63:0/56284390 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff3f402b8e0 con 0x7ff404078ce0 2024-03-22T07:07:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.903+0000 7ff408f98700 1 -- 172.21.15.63:0/56284390 --> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4040ff240 con 0x7ff3ec0367e0 2024-03-22T07:07:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.907+0000 7ff3f2ffd700 1 -- 172.21.15.63:0/56284390 <== mgr.14120 v2:172.21.15.63:6800/480818549 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7ff4040ff240 con 0x7ff3ec0367e0 2024-03-22T07:07:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.909+0000 7ff408f98700 1 -- 172.21.15.63:0/56284390 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7ff3ec0367e0 msgr2=0x7ff3ec038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.909+0000 7ff408f98700 1 --2- 172.21.15.63:0/56284390 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7ff3ec0367e0 0x7ff3ec038c90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009990 tx=0x7ff3f8006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.909+0000 7ff408f98700 1 -- 172.21.15.63:0/56284390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff404078ce0 msgr2=0x7ff4041137f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.909+0000 7ff408f98700 1 --2- 172.21.15.63:0/56284390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff404078ce0 0x7ff4041137f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40094c0 tx=0x7ff3f4009250 comp rx=0 tx=0).stop 2024-03-22T07:07:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.909+0000 7ff408f98700 1 -- 172.21.15.63:0/56284390 shutdown_connections 2024-03-22T07:07:24.961 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.909+0000 7ff408f98700 1 --2- 172.21.15.63:0/56284390 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7ff3ec0367e0 0x7ff3ec038c90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:24.962 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.909+0000 7ff408f98700 1 --2- 172.21.15.63:0/56284390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff404078ce0 0x7ff4041137f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:24.962 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.909+0000 7ff408f98700 1 -- 172.21.15.63:0/56284390 >> 172.21.15.63:0/56284390 conn(0x7ff4040fdf10 msgr2=0x7ff4040fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:24.962 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.909+0000 7ff408f98700 1 -- 172.21.15.63:0/56284390 shutdown_connections 2024-03-22T07:07:24.962 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:23.909+0000 7ff408f98700 1 -- 172.21.15.63:0/56284390 wait complete. 2024-03-22T07:07:24.962 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying prometheus service with default placement... 2024-03-22T07:07:25.159 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:24 smithi063 ceph-mon[29200]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:25.159 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:24 smithi063 ceph-mon[29200]: Saving service crash spec with placement * 2024-03-22T07:07:25.159 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:24 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:26.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-03-22T07:07:26.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.363+0000 7fc11b8f7700 1 Processor -- start 2024-03-22T07:07:26.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.364+0000 7fc11b8f7700 1 -- start start 2024-03-22T07:07:26.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.364+0000 7fc11b8f7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114102730 0x7fc114102b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:26.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.364+0000 7fc11b8f7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc114103080 con 0x7fc114102730 2024-03-22T07:07:26.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.364+0000 7fc119693700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114102730 0x7fc114102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:26.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.364+0000 7fc119693700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114102730 0x7fc114102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56396/0 (socket says 172.21.15.63:56396) 2024-03-22T07:07:26.214 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.364+0000 7fc119693700 1 -- 172.21.15.63:0/2530871280 learned_addr learned my addr 172.21.15.63:0/2530871280 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:26.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.364+0000 7fc119693700 1 -- 172.21.15.63:0/2530871280 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1141031c0 con 0x7fc114102730 2024-03-22T07:07:26.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.364+0000 7fc119693700 1 --2- 172.21.15.63:0/2530871280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114102730 0x7fc114102b40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc1040098e0 tx=0x7fc104009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e62cbbcb6f92ee server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:26.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.365+0000 7fc10bfff700 1 -- 172.21.15.63:0/2530871280 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc10400b860 con 0x7fc114102730 2024-03-22T07:07:26.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.365+0000 7fc10bfff700 1 -- 172.21.15.63:0/2530871280 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc10400be90 con 0x7fc114102730 2024-03-22T07:07:26.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.365+0000 7fc10bfff700 1 -- 172.21.15.63:0/2530871280 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc1040103c0 con 0x7fc114102730 2024-03-22T07:07:26.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.365+0000 7fc11b8f7700 1 -- 172.21.15.63:0/2530871280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114102730 msgr2=0x7fc114102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:26.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.365+0000 7fc11b8f7700 1 --2- 172.21.15.63:0/2530871280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114102730 0x7fc114102b40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc1040098e0 tx=0x7fc104009250 comp rx=0 tx=0).stop 2024-03-22T07:07:26.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.365+0000 7fc11b8f7700 1 -- 172.21.15.63:0/2530871280 shutdown_connections 2024-03-22T07:07:26.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.365+0000 7fc11b8f7700 1 --2- 172.21.15.63:0/2530871280 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114102730 0x7fc114102b40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:26.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.365+0000 7fc11b8f7700 1 -- 172.21.15.63:0/2530871280 >> 172.21.15.63:0/2530871280 conn(0x7fc1140fdf10 msgr2=0x7fc114100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:26.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.365+0000 7fc11b8f7700 1 -- 172.21.15.63:0/2530871280 shutdown_connections 2024-03-22T07:07:26.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.365+0000 7fc11b8f7700 1 -- 172.21.15.63:0/2530871280 wait complete. 2024-03-22T07:07:26.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.366+0000 7fc11b8f7700 1 Processor -- start 2024-03-22T07:07:26.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.366+0000 7fc11b8f7700 1 -- start start 2024-03-22T07:07:26.215 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.366+0000 7fc11b8f7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114102730 0x7fc114111630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.366+0000 7fc11b8f7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc114111b70 con 0x7fc114102730 2024-03-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.366+0000 7fc119693700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114102730 0x7fc114111630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.366+0000 7fc119693700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114102730 0x7fc114111630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56404/0 (socket says 172.21.15.63:56404) 2024-03-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.366+0000 7fc119693700 1 -- 172.21.15.63:0/1914035625 learned_addr learned my addr 172.21.15.63:0/1914035625 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.366+0000 7fc119693700 1 -- 172.21.15.63:0/1914035625 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1040094f0 con 0x7fc114102730 2024-03-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.367+0000 7fc119693700 1 --2- 172.21.15.63:0/1914035625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114102730 0x7fc114111630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc104006fd0 tx=0x7fc104009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.367+0000 7fc109ffb700 1 -- 172.21.15.63:0/1914035625 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc104010630 con 0x7fc114102730 2024-03-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.367+0000 7fc109ffb700 1 -- 172.21.15.63:0/1914035625 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc104010c60 con 0x7fc114102730 2024-03-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.367+0000 7fc11b8f7700 1 -- 172.21.15.63:0/1914035625 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc114111d70 con 0x7fc114102730 2024-03-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.367+0000 7fc109ffb700 1 -- 172.21.15.63:0/1914035625 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc10401ad40 con 0x7fc114102730 2024-03-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.367+0000 7fc11b8f7700 1 -- 172.21.15.63:0/1914035625 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1141128e0 con 0x7fc114102730 2024-03-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.368+0000 7fc109ffb700 1 -- 172.21.15.63:0/1914035625 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fc1040193c0 con 0x7fc114102730 2024-03-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.368+0000 7fc11b8f7700 1 -- 172.21.15.63:0/1914035625 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11405f5f0 con 0x7fc114102730 2024-03-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.368+0000 7fc109ffb700 1 --2- 172.21.15.63:0/1914035625 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fc100036790 0x7fc100038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.368+0000 7fc109ffb700 1 -- 172.21.15.63:0/1914035625 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc10404a070 con 0x7fc114102730 2024-03-22T07:07:26.216 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.368+0000 7fc118e92700 1 --2- 172.21.15.63:0/1914035625 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fc100036790 0x7fc100038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:26.217 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.368+0000 7fc118e92700 1 --2- 172.21.15.63:0/1914035625 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fc100036790 0x7fc100038c40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc110009990 tx=0x7fc110006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:26.217 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.373+0000 7fc109ffb700 1 -- 172.21.15.63:0/1914035625 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc10407c190 con 0x7fc114102730 2024-03-22T07:07:26.217 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.582+0000 7fc11b8f7700 1 -- 172.21.15.63:0/1914035625 --> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1140ff2d0 con 0x7fc100036790 2024-03-22T07:07:26.217 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.588+0000 7fc109ffb700 1 -- 172.21.15.63:0/1914035625 <== mgr.14120 v2:172.21.15.63:6800/480818549 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fc1140ff2d0 con 0x7fc100036790 2024-03-22T07:07:26.217 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.590+0000 7fc11b8f7700 1 -- 172.21.15.63:0/1914035625 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fc100036790 msgr2=0x7fc100038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:26.217 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.590+0000 7fc11b8f7700 1 --2- 172.21.15.63:0/1914035625 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fc100036790 0x7fc100038c40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc110009990 tx=0x7fc110006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:26.217 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.590+0000 7fc11b8f7700 1 -- 172.21.15.63:0/1914035625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114102730 msgr2=0x7fc114111630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:26.217 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.590+0000 7fc11b8f7700 1 --2- 172.21.15.63:0/1914035625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114102730 0x7fc114111630 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc104006fd0 tx=0x7fc104009250 comp rx=0 tx=0).stop 2024-03-22T07:07:26.217 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.590+0000 7fc11b8f7700 1 -- 172.21.15.63:0/1914035625 shutdown_connections 2024-03-22T07:07:26.217 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.590+0000 7fc11b8f7700 1 --2- 172.21.15.63:0/1914035625 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fc100036790 0x7fc100038c40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:26.217 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.590+0000 7fc11b8f7700 1 --2- 172.21.15.63:0/1914035625 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc114102730 0x7fc114111630 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:26.217 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.590+0000 7fc11b8f7700 1 -- 172.21.15.63:0/1914035625 >> 172.21.15.63:0/1914035625 conn(0x7fc1140fdf10 msgr2=0x7fc1140feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:26.217 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.590+0000 7fc11b8f7700 1 -- 172.21.15.63:0/1914035625 shutdown_connections 2024-03-22T07:07:26.217 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:25.590+0000 7fc11b8f7700 1 -- 172.21.15.63:0/1914035625 wait complete. 2024-03-22T07:07:26.217 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying grafana service with default placement... 2024-03-22T07:07:26.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:26 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:26.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:26 smithi063 ceph-mon[29200]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:26.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:26 smithi063 ceph-mon[29200]: Saving service prometheus spec with placement count:1 2024-03-22T07:07:26.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:26 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:26.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:26 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:27.766 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-03-22T07:07:27.766 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.681+0000 7f30d80a5700 1 Processor -- start 2024-03-22T07:07:27.766 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.681+0000 7f30d80a5700 1 -- start start 2024-03-22T07:07:27.766 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.682+0000 7f30d80a5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d0102720 0x7f30d0102b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:27.766 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.682+0000 7f30d80a5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d0103070 con 0x7f30d0102720 2024-03-22T07:07:27.766 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.682+0000 7f30d5e41700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d0102720 0x7f30d0102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:27.766 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.682+0000 7f30d5e41700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d0102720 0x7f30d0102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56418/0 (socket says 172.21.15.63:56418) 2024-03-22T07:07:27.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.682+0000 7f30d5e41700 1 -- 172.21.15.63:0/1755250621 learned_addr learned my addr 172.21.15.63:0/1755250621 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:27.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.682+0000 7f30d5e41700 1 -- 172.21.15.63:0/1755250621 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d01031b0 con 0x7f30d0102720 2024-03-22T07:07:27.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.682+0000 7f30d5e41700 1 --2- 172.21.15.63:0/1755250621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d0102720 0x7f30d0102b30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0098e0 tx=0x7f30cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf14d6a165dd8060 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:27.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.682+0000 7f30d4e3f700 1 -- 172.21.15.63:0/1755250621 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f30cc00b860 con 0x7f30d0102720 2024-03-22T07:07:27.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.683+0000 7f30d4e3f700 1 -- 172.21.15.63:0/1755250621 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30cc00be90 con 0x7f30d0102720 2024-03-22T07:07:27.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.683+0000 7f30d4e3f700 1 -- 172.21.15.63:0/1755250621 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f30cc0103c0 con 0x7f30d0102720 2024-03-22T07:07:27.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.683+0000 7f30d80a5700 1 -- 172.21.15.63:0/1755250621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d0102720 msgr2=0x7f30d0102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:27.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.683+0000 7f30d80a5700 1 --2- 172.21.15.63:0/1755250621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d0102720 0x7f30d0102b30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0098e0 tx=0x7f30cc009250 comp rx=0 tx=0).stop 2024-03-22T07:07:27.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.683+0000 7f30d80a5700 1 -- 172.21.15.63:0/1755250621 shutdown_connections 2024-03-22T07:07:27.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.683+0000 7f30d80a5700 1 --2- 172.21.15.63:0/1755250621 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d0102720 0x7f30d0102b30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:27.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.683+0000 7f30d80a5700 1 -- 172.21.15.63:0/1755250621 >> 172.21.15.63:0/1755250621 conn(0x7f30d00fdee0 msgr2=0x7f30d01002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:27.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.683+0000 7f30d80a5700 1 -- 172.21.15.63:0/1755250621 shutdown_connections 2024-03-22T07:07:27.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.683+0000 7f30d80a5700 1 -- 172.21.15.63:0/1755250621 wait complete. 2024-03-22T07:07:27.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.684+0000 7f30d80a5700 1 Processor -- start 2024-03-22T07:07:27.767 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.684+0000 7f30d80a5700 1 -- start start 2024-03-22T07:07:27.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.684+0000 7f30d80a5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d0102720 0x7f30d0078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:27.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.684+0000 7f30d80a5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d0079110 con 0x7f30d0102720 2024-03-22T07:07:27.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.684+0000 7f30d5e41700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d0102720 0x7f30d0078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:27.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.684+0000 7f30d5e41700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d0102720 0x7f30d0078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56432/0 (socket says 172.21.15.63:56432) 2024-03-22T07:07:27.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.684+0000 7f30d5e41700 1 -- 172.21.15.63:0/1079167475 learned_addr learned my addr 172.21.15.63:0/1079167475 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:27.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.684+0000 7f30d5e41700 1 -- 172.21.15.63:0/1079167475 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30cc0094f0 con 0x7f30d0102720 2024-03-22T07:07:27.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.684+0000 7f30d5e41700 1 --2- 172.21.15.63:0/1079167475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d0102720 0x7f30d0078bd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0090b0 tx=0x7f30cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:27.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.685+0000 7f30c67fc700 1 -- 172.21.15.63:0/1079167475 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f30cc010840 con 0x7f30d0102720 2024-03-22T07:07:27.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.685+0000 7f30c67fc700 1 -- 172.21.15.63:0/1079167475 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30cc010e70 con 0x7f30d0102720 2024-03-22T07:07:27.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.685+0000 7f30d80a5700 1 -- 172.21.15.63:0/1079167475 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d0077180 con 0x7f30d0102720 2024-03-22T07:07:27.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.685+0000 7f30c67fc700 1 -- 172.21.15.63:0/1079167475 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f30cc024e40 con 0x7f30d0102720 2024-03-22T07:07:27.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.685+0000 7f30d80a5700 1 -- 172.21.15.63:0/1079167475 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d0077670 con 0x7f30d0102720 2024-03-22T07:07:27.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.685+0000 7f30c67fc700 1 -- 172.21.15.63:0/1079167475 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f30cc01a4c0 con 0x7f30d0102720 2024-03-22T07:07:27.768 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.686+0000 7f30c67fc700 1 --2- 172.21.15.63:0/1079167475 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f30bc036790 0x7f30bc038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:27.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.686+0000 7f30d80a5700 1 -- 172.21.15.63:0/1079167475 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d004bed0 con 0x7f30d0102720 2024-03-22T07:07:27.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.686+0000 7f30c67fc700 1 -- 172.21.15.63:0/1079167475 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f30cc04a8e0 con 0x7f30d0102720 2024-03-22T07:07:27.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.686+0000 7f30d5640700 1 --2- 172.21.15.63:0/1079167475 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f30bc036790 0x7f30bc038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:27.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.686+0000 7f30d5640700 1 --2- 172.21.15.63:0/1079167475 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f30bc036790 0x7f30bc038c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009990 tx=0x7f30c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:27.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.690+0000 7f30c67fc700 1 -- 172.21.15.63:0/1079167475 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f30cc018710 con 0x7f30d0102720 2024-03-22T07:07:27.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.907+0000 7f30d80a5700 1 -- 172.21.15.63:0/1079167475 --> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f30d0077e30 con 0x7f30bc036790 2024-03-22T07:07:27.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.911+0000 7f30c67fc700 1 -- 172.21.15.63:0/1079167475 <== mgr.14120 v2:172.21.15.63:6800/480818549 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f30d0077e30 con 0x7f30bc036790 2024-03-22T07:07:27.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.913+0000 7f30d80a5700 1 -- 172.21.15.63:0/1079167475 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f30bc036790 msgr2=0x7f30bc038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:27.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.913+0000 7f30d80a5700 1 --2- 172.21.15.63:0/1079167475 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f30bc036790 0x7f30bc038c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009990 tx=0x7f30c0006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:27.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.913+0000 7f30d80a5700 1 -- 172.21.15.63:0/1079167475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d0102720 msgr2=0x7f30d0078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:27.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.913+0000 7f30d80a5700 1 --2- 172.21.15.63:0/1079167475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d0102720 0x7f30d0078bd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0090b0 tx=0x7f30cc009250 comp rx=0 tx=0).stop 2024-03-22T07:07:27.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.913+0000 7f30d80a5700 1 -- 172.21.15.63:0/1079167475 shutdown_connections 2024-03-22T07:07:27.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.913+0000 7f30d80a5700 1 --2- 172.21.15.63:0/1079167475 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f30bc036790 0x7f30bc038c40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:27.769 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.913+0000 7f30d80a5700 1 --2- 172.21.15.63:0/1079167475 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30d0102720 0x7f30d0078bd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:27.770 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.913+0000 7f30d80a5700 1 -- 172.21.15.63:0/1079167475 >> 172.21.15.63:0/1079167475 conn(0x7f30d00fdee0 msgr2=0x7f30d00fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:27.770 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.913+0000 7f30d80a5700 1 -- 172.21.15.63:0/1079167475 shutdown_connections 2024-03-22T07:07:27.770 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:26.913+0000 7f30d80a5700 1 -- 172.21.15.63:0/1079167475 wait complete. 2024-03-22T07:07:27.770 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying node-exporter service with default placement... 2024-03-22T07:07:28.018 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:27 smithi063 ceph-mon[29200]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:28.018 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:27 smithi063 ceph-mon[29200]: Saving service grafana spec with placement count:1 2024-03-22T07:07:28.018 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:27 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:29.200 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-03-22T07:07:29.200 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.201+0000 7fa72d7e9700 1 Processor -- start 2024-03-22T07:07:29.200 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.201+0000 7fa72d7e9700 1 -- start start 2024-03-22T07:07:29.201 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.201+0000 7fa72d7e9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa728076b30 0x7fa728105430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:29.201 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.201+0000 7fa72d7e9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa728105970 con 0x7fa728076b30 2024-03-22T07:07:29.201 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.202+0000 7fa726ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa728076b30 0x7fa728105430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:29.201 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.202+0000 7fa726ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa728076b30 0x7fa728105430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56438/0 (socket says 172.21.15.63:56438) 2024-03-22T07:07:29.201 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.202+0000 7fa726ffd700 1 -- 172.21.15.63:0/2704225717 learned_addr learned my addr 172.21.15.63:0/2704225717 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:29.201 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.202+0000 7fa726ffd700 1 -- 172.21.15.63:0/2704225717 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa728105ab0 con 0x7fa728076b30 2024-03-22T07:07:29.201 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.202+0000 7fa726ffd700 1 --2- 172.21.15.63:0/2704225717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa728076b30 0x7fa728105430 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7180098e0 tx=0x7fa718009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b775d39254fb3e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:29.201 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.202+0000 7fa725ffb700 1 -- 172.21.15.63:0/2704225717 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa71800b860 con 0x7fa728076b30 2024-03-22T07:07:29.201 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.202+0000 7fa725ffb700 1 -- 172.21.15.63:0/2704225717 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa71800be90 con 0x7fa728076b30 2024-03-22T07:07:29.201 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.202+0000 7fa725ffb700 1 -- 172.21.15.63:0/2704225717 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa7180103c0 con 0x7fa728076b30 2024-03-22T07:07:29.201 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.203+0000 7fa72d7e9700 1 -- 172.21.15.63:0/2704225717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa728076b30 msgr2=0x7fa728105430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:29.201 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.203+0000 7fa72d7e9700 1 --2- 172.21.15.63:0/2704225717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa728076b30 0x7fa728105430 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa7180098e0 tx=0x7fa718009250 comp rx=0 tx=0).stop 2024-03-22T07:07:29.201 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.203+0000 7fa72d7e9700 1 -- 172.21.15.63:0/2704225717 shutdown_connections 2024-03-22T07:07:29.202 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.203+0000 7fa72d7e9700 1 --2- 172.21.15.63:0/2704225717 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa728076b30 0x7fa728105430 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:29.202 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.203+0000 7fa72d7e9700 1 -- 172.21.15.63:0/2704225717 >> 172.21.15.63:0/2704225717 conn(0x7fa7280fce40 msgr2=0x7fa7280ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:29.202 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.203+0000 7fa72d7e9700 1 -- 172.21.15.63:0/2704225717 shutdown_connections 2024-03-22T07:07:29.202 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.203+0000 7fa72d7e9700 1 -- 172.21.15.63:0/2704225717 wait complete. 2024-03-22T07:07:29.202 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.204+0000 7fa72d7e9700 1 Processor -- start 2024-03-22T07:07:29.202 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.204+0000 7fa72d7e9700 1 -- start start 2024-03-22T07:07:29.202 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.204+0000 7fa72d7e9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa728076b30 0x7fa728111620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:29.202 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.204+0000 7fa72d7e9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa728111b60 con 0x7fa728076b30 2024-03-22T07:07:29.202 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.204+0000 7fa726ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa728076b30 0x7fa728111620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:29.202 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.204+0000 7fa726ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa728076b30 0x7fa728111620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56444/0 (socket says 172.21.15.63:56444) 2024-03-22T07:07:29.202 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.204+0000 7fa726ffd700 1 -- 172.21.15.63:0/4016269455 learned_addr learned my addr 172.21.15.63:0/4016269455 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:29.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.204+0000 7fa726ffd700 1 -- 172.21.15.63:0/4016269455 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7180094f0 con 0x7fa728076b30 2024-03-22T07:07:29.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.205+0000 7fa726ffd700 1 --2- 172.21.15.63:0/4016269455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa728076b30 0x7fa728111620 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa7180105b0 tx=0x7fa718009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:29.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.205+0000 7fa70f7fe700 1 -- 172.21.15.63:0/4016269455 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa718010840 con 0x7fa728076b30 2024-03-22T07:07:29.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.205+0000 7fa70f7fe700 1 -- 172.21.15.63:0/4016269455 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa718010e70 con 0x7fa728076b30 2024-03-22T07:07:29.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.205+0000 7fa72d7e9700 1 -- 172.21.15.63:0/4016269455 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa728111d60 con 0x7fa728076b30 2024-03-22T07:07:29.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.205+0000 7fa70f7fe700 1 -- 172.21.15.63:0/4016269455 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa71801ae30 con 0x7fa728076b30 2024-03-22T07:07:29.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.205+0000 7fa72d7e9700 1 -- 172.21.15.63:0/4016269455 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7281128d0 con 0x7fa728076b30 2024-03-22T07:07:29.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.206+0000 7fa70f7fe700 1 -- 172.21.15.63:0/4016269455 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fa71802d3f0 con 0x7fa728076b30 2024-03-22T07:07:29.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.206+0000 7fa72d7e9700 1 -- 172.21.15.63:0/4016269455 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72804bed0 con 0x7fa728076b30 2024-03-22T07:07:29.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.206+0000 7fa70f7fe700 1 --2- 172.21.15.63:0/4016269455 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fa710036790 0x7fa710038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:29.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.206+0000 7fa70f7fe700 1 -- 172.21.15.63:0/4016269455 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fa71804a210 con 0x7fa728076b30 2024-03-22T07:07:29.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.206+0000 7fa7267fc700 1 --2- 172.21.15.63:0/4016269455 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fa710036790 0x7fa710038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:29.203 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.207+0000 7fa7267fc700 1 --2- 172.21.15.63:0/4016269455 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fa710036790 0x7fa710038c40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009990 tx=0x7fa71c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:29.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.211+0000 7fa70f7fe700 1 -- 172.21.15.63:0/4016269455 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa71802b8e0 con 0x7fa728076b30 2024-03-22T07:07:29.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.426+0000 7fa72d7e9700 1 -- 172.21.15.63:0/4016269455 --> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7280fe230 con 0x7fa710036790 2024-03-22T07:07:29.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.431+0000 7fa70f7fe700 1 -- 172.21.15.63:0/4016269455 <== mgr.14120 v2:172.21.15.63:6800/480818549 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fa7280fe230 con 0x7fa710036790 2024-03-22T07:07:29.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.433+0000 7fa72d7e9700 1 -- 172.21.15.63:0/4016269455 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fa710036790 msgr2=0x7fa710038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:29.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.433+0000 7fa72d7e9700 1 --2- 172.21.15.63:0/4016269455 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fa710036790 0x7fa710038c40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009990 tx=0x7fa71c006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:29.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.433+0000 7fa72d7e9700 1 -- 172.21.15.63:0/4016269455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa728076b30 msgr2=0x7fa728111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:29.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.433+0000 7fa72d7e9700 1 --2- 172.21.15.63:0/4016269455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa728076b30 0x7fa728111620 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa7180105b0 tx=0x7fa718009250 comp rx=0 tx=0).stop 2024-03-22T07:07:29.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.433+0000 7fa72d7e9700 1 -- 172.21.15.63:0/4016269455 shutdown_connections 2024-03-22T07:07:29.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.433+0000 7fa72d7e9700 1 --2- 172.21.15.63:0/4016269455 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fa710036790 0x7fa710038c40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:29.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.433+0000 7fa72d7e9700 1 --2- 172.21.15.63:0/4016269455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa728076b30 0x7fa728111620 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:29.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.433+0000 7fa72d7e9700 1 -- 172.21.15.63:0/4016269455 >> 172.21.15.63:0/4016269455 conn(0x7fa7280fce40 msgr2=0x7fa7280fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:29.204 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.433+0000 7fa72d7e9700 1 -- 172.21.15.63:0/4016269455 shutdown_connections 2024-03-22T07:07:29.205 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:28.433+0000 7fa72d7e9700 1 -- 172.21.15.63:0/4016269455 wait complete. 2024-03-22T07:07:29.205 INFO:teuthology.orchestra.run.smithi063.stdout:Deploying alertmanager service with default placement... 2024-03-22T07:07:29.527 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:29 smithi063 ceph-mon[29200]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:29.528 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:29 smithi063 ceph-mon[29200]: Saving service node-exporter spec with placement * 2024-03-22T07:07:29.528 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:29 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:30.725 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-03-22T07:07:30.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.903+0000 7f27808e7700 1 Processor -- start 2024-03-22T07:07:30.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.903+0000 7f27808e7700 1 -- start start 2024-03-22T07:07:30.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.903+0000 7f27808e7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277c1016e0 0x7f277c101af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:30.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.903+0000 7f27808e7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c102030 con 0x7f277c1016e0 2024-03-22T07:07:30.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.903+0000 7f277a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277c1016e0 0x7f277c101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:30.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.904+0000 7f277a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277c1016e0 0x7f277c101af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56450/0 (socket says 172.21.15.63:56450) 2024-03-22T07:07:30.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.904+0000 7f277a59c700 1 -- 172.21.15.63:0/259038824 learned_addr learned my addr 172.21.15.63:0/259038824 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:30.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.904+0000 7f277a59c700 1 -- 172.21.15.63:0/259038824 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277c102170 con 0x7f277c1016e0 2024-03-22T07:07:30.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.904+0000 7f277a59c700 1 --2- 172.21.15.63:0/259038824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277c1016e0 0x7f277c101af0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f276c00bb80 tx=0x7f276c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45a195d9d09e9e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:30.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.904+0000 7f277959a700 1 -- 172.21.15.63:0/259038824 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f276c00dd00 con 0x7f277c1016e0 2024-03-22T07:07:30.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.904+0000 7f277959a700 1 -- 172.21.15.63:0/259038824 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f276c00de60 con 0x7f277c1016e0 2024-03-22T07:07:30.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.904+0000 7f277959a700 1 -- 172.21.15.63:0/259038824 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f276c018920 con 0x7f277c1016e0 2024-03-22T07:07:30.726 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.905+0000 7f27808e7700 1 -- 172.21.15.63:0/259038824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277c1016e0 msgr2=0x7f277c101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.905+0000 7f27808e7700 1 --2- 172.21.15.63:0/259038824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277c1016e0 0x7f277c101af0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f276c00bb80 tx=0x7f276c00b4f0 comp rx=0 tx=0).stop 2024-03-22T07:07:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.905+0000 7f27808e7700 1 -- 172.21.15.63:0/259038824 shutdown_connections 2024-03-22T07:07:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.905+0000 7f27808e7700 1 --2- 172.21.15.63:0/259038824 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277c1016e0 0x7f277c101af0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.905+0000 7f27808e7700 1 -- 172.21.15.63:0/259038824 >> 172.21.15.63:0/259038824 conn(0x7f277c0fce40 msgr2=0x7f277c0ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.905+0000 7f27808e7700 1 -- 172.21.15.63:0/259038824 shutdown_connections 2024-03-22T07:07:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.905+0000 7f27808e7700 1 -- 172.21.15.63:0/259038824 wait complete. 2024-03-22T07:07:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.905+0000 7f27808e7700 1 Processor -- start 2024-03-22T07:07:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.905+0000 7f27808e7700 1 -- start start 2024-03-22T07:07:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.905+0000 7f27808e7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277c1016e0 0x7f277c113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.905+0000 7f27808e7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c113cc0 con 0x7f277c1016e0 2024-03-22T07:07:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.905+0000 7f277a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277c1016e0 0x7f277c113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.905+0000 7f277a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277c1016e0 0x7f277c113780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56464/0 (socket says 172.21.15.63:56464) 2024-03-22T07:07:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.905+0000 7f277a59c700 1 -- 172.21.15.63:0/3202044144 learned_addr learned my addr 172.21.15.63:0/3202044144 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.906+0000 7f277a59c700 1 -- 172.21.15.63:0/3202044144 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276c00b790 con 0x7f277c1016e0 2024-03-22T07:07:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.906+0000 7f277a59c700 1 --2- 172.21.15.63:0/3202044144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277c1016e0 0x7f277c113780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f276c0106a0 tx=0x7f276c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.906+0000 7f276affd700 1 -- 172.21.15.63:0/3202044144 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f276c00dc70 con 0x7f277c1016e0 2024-03-22T07:07:30.727 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.906+0000 7f276affd700 1 -- 172.21.15.63:0/3202044144 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f276c010c30 con 0x7f277c1016e0 2024-03-22T07:07:30.728 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.906+0000 7f27808e7700 1 -- 172.21.15.63:0/3202044144 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277c113ec0 con 0x7f277c1016e0 2024-03-22T07:07:30.728 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.906+0000 7f276affd700 1 -- 172.21.15.63:0/3202044144 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f276c021550 con 0x7f277c1016e0 2024-03-22T07:07:30.728 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.906+0000 7f27808e7700 1 -- 172.21.15.63:0/3202044144 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277c114a30 con 0x7f277c1016e0 2024-03-22T07:07:30.728 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.907+0000 7f276affd700 1 -- 172.21.15.63:0/3202044144 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f276c0217c0 con 0x7f277c1016e0 2024-03-22T07:07:30.728 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.907+0000 7f27808e7700 1 -- 172.21.15.63:0/3202044144 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277c05f5f0 con 0x7f277c1016e0 2024-03-22T07:07:30.728 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.907+0000 7f276affd700 1 --2- 172.21.15.63:0/3202044144 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f27640367e0 0x7f2764038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:30.728 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.907+0000 7f276affd700 1 -- 172.21.15.63:0/3202044144 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f276c014070 con 0x7f277c1016e0 2024-03-22T07:07:30.728 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.907+0000 7f2779d9b700 1 --2- 172.21.15.63:0/3202044144 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f27640367e0 0x7f2764038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:30.728 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.908+0000 7f2779d9b700 1 --2- 172.21.15.63:0/3202044144 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f27640367e0 0x7f2764038c90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2770009990 tx=0x7f2770006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:30.728 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:29.912+0000 7f276affd700 1 -- 172.21.15.63:0/3202044144 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f276c00e930 con 0x7f277c1016e0 2024-03-22T07:07:30.728 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:30.120+0000 7f27808e7700 1 -- 172.21.15.63:0/3202044144 --> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f277c114e40 con 0x7f27640367e0 2024-03-22T07:07:30.728 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:30.124+0000 7f276affd700 1 -- 172.21.15.63:0/3202044144 <== mgr.14120 v2:172.21.15.63:6800/480818549 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f277c114e40 con 0x7f27640367e0 2024-03-22T07:07:30.728 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:30.126+0000 7f27808e7700 1 -- 172.21.15.63:0/3202044144 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f27640367e0 msgr2=0x7f2764038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:30.728 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:30.126+0000 7f27808e7700 1 --2- 172.21.15.63:0/3202044144 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f27640367e0 0x7f2764038c90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2770009990 tx=0x7f2770006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:30.729 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:30.126+0000 7f27808e7700 1 -- 172.21.15.63:0/3202044144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277c1016e0 msgr2=0x7f277c113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:30.729 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:30.126+0000 7f27808e7700 1 --2- 172.21.15.63:0/3202044144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277c1016e0 0x7f277c113780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f276c0106a0 tx=0x7f276c00b4f0 comp rx=0 tx=0).stop 2024-03-22T07:07:30.729 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:30.126+0000 7f27808e7700 1 -- 172.21.15.63:0/3202044144 shutdown_connections 2024-03-22T07:07:30.729 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:30.126+0000 7f27808e7700 1 --2- 172.21.15.63:0/3202044144 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f27640367e0 0x7f2764038c90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:30.729 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:30.126+0000 7f27808e7700 1 --2- 172.21.15.63:0/3202044144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f277c1016e0 0x7f277c113780 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:30.729 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:30.126+0000 7f27808e7700 1 -- 172.21.15.63:0/3202044144 >> 172.21.15.63:0/3202044144 conn(0x7f277c0fce40 msgr2=0x7f277c109f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:30.729 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:30.126+0000 7f27808e7700 1 -- 172.21.15.63:0/3202044144 shutdown_connections 2024-03-22T07:07:30.729 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:30.126+0000 7f27808e7700 1 -- 172.21.15.63:0/3202044144 wait complete. 2024-03-22T07:07:31.336 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:31 smithi063 ceph-mon[29200]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:31.336 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:31 smithi063 ceph-mon[29200]: Saving service alertmanager spec with placement count:1 2024-03-22T07:07:31.336 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:31 smithi063 ceph-mon[29200]: from='mgr.14120 172.21.15.63:0/198929843' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:31.336 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:31 smithi063 ceph-mon[29200]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:07:31.917 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.131+0000 7fbd0d829700 1 Processor -- start 2024-03-22T07:07:31.917 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.131+0000 7fbd0d829700 1 -- start start 2024-03-22T07:07:31.917 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.131+0000 7fbd0d829700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd08078ca0 0x7fbd08077200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:31.917 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.131+0000 7fbd0d829700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd080790b0 con 0x7fbd08078ca0 2024-03-22T07:07:31.917 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.132+0000 7fbd06ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd08078ca0 0x7fbd08077200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:31.917 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.132+0000 7fbd06ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd08078ca0 0x7fbd08077200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56468/0 (socket says 172.21.15.63:56468) 2024-03-22T07:07:31.917 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.132+0000 7fbd06ffd700 1 -- 172.21.15.63:0/1283149535 learned_addr learned my addr 172.21.15.63:0/1283149535 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:31.917 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.132+0000 7fbd06ffd700 1 -- 172.21.15.63:0/1283149535 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd08077740 con 0x7fbd08078ca0 2024-03-22T07:07:31.917 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.132+0000 7fbd06ffd700 1 --2- 172.21.15.63:0/1283149535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd08078ca0 0x7fbd08077200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009e50 tx=0x7fbd00009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5d224bfb7c928e server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:31.918 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.132+0000 7fbd05ffb700 1 -- 172.21.15.63:0/1283149535 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd000092e0 con 0x7fbd08078ca0 2024-03-22T07:07:31.918 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.132+0000 7fbd05ffb700 1 -- 172.21.15.63:0/1283149535 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd00012430 con 0x7fbd08078ca0 2024-03-22T07:07:31.918 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.132+0000 7fbd05ffb700 1 -- 172.21.15.63:0/1283149535 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd000075a0 con 0x7fbd08078ca0 2024-03-22T07:07:31.918 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.133+0000 7fbd0d829700 1 -- 172.21.15.63:0/1283149535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd08078ca0 msgr2=0x7fbd08077200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:31.918 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.133+0000 7fbd0d829700 1 --2- 172.21.15.63:0/1283149535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd08078ca0 0x7fbd08077200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009e50 tx=0x7fbd00009840 comp rx=0 tx=0).stop 2024-03-22T07:07:31.918 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.133+0000 7fbd0d829700 1 -- 172.21.15.63:0/1283149535 shutdown_connections 2024-03-22T07:07:31.918 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.133+0000 7fbd0d829700 1 --2- 172.21.15.63:0/1283149535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd08078ca0 0x7fbd08077200 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:31.918 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.133+0000 7fbd0d829700 1 -- 172.21.15.63:0/1283149535 >> 172.21.15.63:0/1283149535 conn(0x7fbd080fde40 msgr2=0x7fbd081002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:31.918 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.133+0000 7fbd0d829700 1 -- 172.21.15.63:0/1283149535 shutdown_connections 2024-03-22T07:07:31.918 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.133+0000 7fbd0d829700 1 -- 172.21.15.63:0/1283149535 wait complete. 2024-03-22T07:07:31.918 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.134+0000 7fbd0d829700 1 Processor -- start 2024-03-22T07:07:31.918 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.134+0000 7fbd0d829700 1 -- start start 2024-03-22T07:07:31.918 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.134+0000 7fbd0d829700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd08078ca0 0x7fbd081137f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:31.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.134+0000 7fbd0d829700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd08113d30 con 0x7fbd08078ca0 2024-03-22T07:07:31.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.134+0000 7fbd06ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd08078ca0 0x7fbd081137f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:31.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.134+0000 7fbd06ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd08078ca0 0x7fbd081137f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56482/0 (socket says 172.21.15.63:56482) 2024-03-22T07:07:31.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.134+0000 7fbd06ffd700 1 -- 172.21.15.63:0/37869086 learned_addr learned my addr 172.21.15.63:0/37869086 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:31.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.134+0000 7fbd06ffd700 1 -- 172.21.15.63:0/37869086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd00009ae0 con 0x7fbd08078ca0 2024-03-22T07:07:31.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.134+0000 7fbd06ffd700 1 --2- 172.21.15.63:0/37869086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd08078ca0 0x7fbd081137f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009fd0 tx=0x7fbd00009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:31.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.134+0000 7fbcff7fe700 1 -- 172.21.15.63:0/37869086 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd000092e0 con 0x7fbd08078ca0 2024-03-22T07:07:31.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.134+0000 7fbcff7fe700 1 -- 172.21.15.63:0/37869086 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd00007ce0 con 0x7fbd08078ca0 2024-03-22T07:07:31.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.134+0000 7fbd0d829700 1 -- 172.21.15.63:0/37869086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd08113f30 con 0x7fbd08078ca0 2024-03-22T07:07:31.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.134+0000 7fbcff7fe700 1 -- 172.21.15.63:0/37869086 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbd000259c0 con 0x7fbd08078ca0 2024-03-22T07:07:31.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.135+0000 7fbd0d829700 1 -- 172.21.15.63:0/37869086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd08114aa0 con 0x7fbd08078ca0 2024-03-22T07:07:31.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.136+0000 7fbcff7fe700 1 -- 172.21.15.63:0/37869086 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fbd00019050 con 0x7fbd08078ca0 2024-03-22T07:07:31.919 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.136+0000 7fbd0d829700 1 -- 172.21.15.63:0/37869086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd0805f5f0 con 0x7fbd08078ca0 2024-03-22T07:07:31.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.136+0000 7fbcff7fe700 1 --2- 172.21.15.63:0/37869086 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fbcf4036790 0x7fbcf4038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:31.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.136+0000 7fbcff7fe700 1 -- 172.21.15.63:0/37869086 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fbd0004a060 con 0x7fbd08078ca0 2024-03-22T07:07:31.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.136+0000 7fbd067fc700 1 --2- 172.21.15.63:0/37869086 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fbcf4036790 0x7fbcf4038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:31.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.136+0000 7fbd067fc700 1 --2- 172.21.15.63:0/37869086 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fbcf4036790 0x7fbcf4038c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009990 tx=0x7fbcf0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:31.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.141+0000 7fbcff7fe700 1 -- 172.21.15.63:0/37869086 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbd0002d950 con 0x7fbd08078ca0 2024-03-22T07:07:31.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.335+0000 7fbd0d829700 1 -- 172.21.15.63:0/37869086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fbd0804bed0 con 0x7fbd08078ca0 2024-03-22T07:07:31.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.338+0000 7fbcff7fe700 1 -- 172.21.15.63:0/37869086 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fbd00020030 con 0x7fbd08078ca0 2024-03-22T07:07:31.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.340+0000 7fbd0d829700 1 -- 172.21.15.63:0/37869086 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fbcf4036790 msgr2=0x7fbcf4038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:31.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.340+0000 7fbd0d829700 1 --2- 172.21.15.63:0/37869086 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fbcf4036790 0x7fbcf4038c40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0009990 tx=0x7fbcf0006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:31.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.340+0000 7fbd0d829700 1 -- 172.21.15.63:0/37869086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd08078ca0 msgr2=0x7fbd081137f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:31.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.340+0000 7fbd0d829700 1 --2- 172.21.15.63:0/37869086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd08078ca0 0x7fbd081137f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009fd0 tx=0x7fbd00009840 comp rx=0 tx=0).stop 2024-03-22T07:07:31.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.340+0000 7fbd0d829700 1 -- 172.21.15.63:0/37869086 shutdown_connections 2024-03-22T07:07:31.920 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.340+0000 7fbd0d829700 1 --2- 172.21.15.63:0/37869086 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fbcf4036790 0x7fbcf4038c40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:31.921 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.340+0000 7fbd0d829700 1 --2- 172.21.15.63:0/37869086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd08078ca0 0x7fbd081137f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:31.921 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.340+0000 7fbd0d829700 1 -- 172.21.15.63:0/37869086 >> 172.21.15.63:0/37869086 conn(0x7fbd080fde40 msgr2=0x7fbd080fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:31.921 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.341+0000 7fbd0d829700 1 -- 172.21.15.63:0/37869086 shutdown_connections 2024-03-22T07:07:31.921 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:31.341+0000 7fbd0d829700 1 -- 172.21.15.63:0/37869086 wait complete. 2024-03-22T07:07:32.507 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:32 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/37869086' entity='client.admin' 2024-03-22T07:07:33.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.300+0000 7fc241bfd700 1 Processor -- start 2024-03-22T07:07:33.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.300+0000 7fc241bfd700 1 -- start start 2024-03-22T07:07:33.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.300+0000 7fc241bfd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c078c50 0x7fc23c077200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:33.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.300+0000 7fc241bfd700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c079060 con 0x7fc23c078c50 2024-03-22T07:07:33.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.300+0000 7fc23b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c078c50 0x7fc23c077200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:33.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.300+0000 7fc23b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c078c50 0x7fc23c077200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56488/0 (socket says 172.21.15.63:56488) 2024-03-22T07:07:33.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.300+0000 7fc23b7fe700 1 -- 172.21.15.63:0/4281131207 learned_addr learned my addr 172.21.15.63:0/4281131207 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:33.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.301+0000 7fc23b7fe700 1 -- 172.21.15.63:0/4281131207 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c077740 con 0x7fc23c078c50 2024-03-22T07:07:33.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.301+0000 7fc23b7fe700 1 --2- 172.21.15.63:0/4281131207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c078c50 0x7fc23c077200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0098e0 tx=0x7fc22c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21752b9ba12fd311 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:33.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.301+0000 7fc23a7fc700 1 -- 172.21.15.63:0/4281131207 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc22c00b860 con 0x7fc23c078c50 2024-03-22T07:07:33.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.301+0000 7fc23a7fc700 1 -- 172.21.15.63:0/4281131207 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc22c00be90 con 0x7fc23c078c50 2024-03-22T07:07:33.164 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.301+0000 7fc23a7fc700 1 -- 172.21.15.63:0/4281131207 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc22c0103c0 con 0x7fc23c078c50 2024-03-22T07:07:33.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.301+0000 7fc241bfd700 1 -- 172.21.15.63:0/4281131207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c078c50 msgr2=0x7fc23c077200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:33.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.301+0000 7fc241bfd700 1 --2- 172.21.15.63:0/4281131207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c078c50 0x7fc23c077200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0098e0 tx=0x7fc22c009250 comp rx=0 tx=0).stop 2024-03-22T07:07:33.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.301+0000 7fc241bfd700 1 -- 172.21.15.63:0/4281131207 shutdown_connections 2024-03-22T07:07:33.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.301+0000 7fc241bfd700 1 --2- 172.21.15.63:0/4281131207 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c078c50 0x7fc23c077200 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:33.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.301+0000 7fc241bfd700 1 -- 172.21.15.63:0/4281131207 >> 172.21.15.63:0/4281131207 conn(0x7fc23c0fde90 msgr2=0x7fc23c100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:33.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.301+0000 7fc241bfd700 1 -- 172.21.15.63:0/4281131207 shutdown_connections 2024-03-22T07:07:33.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.302+0000 7fc241bfd700 1 -- 172.21.15.63:0/4281131207 wait complete. 2024-03-22T07:07:33.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.302+0000 7fc241bfd700 1 Processor -- start 2024-03-22T07:07:33.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.302+0000 7fc241bfd700 1 -- start start 2024-03-22T07:07:33.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.302+0000 7fc241bfd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c078c50 0x7fc23c113820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:33.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.302+0000 7fc241bfd700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c113d60 con 0x7fc23c078c50 2024-03-22T07:07:33.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.302+0000 7fc23b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c078c50 0x7fc23c113820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:33.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.302+0000 7fc23b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c078c50 0x7fc23c113820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56498/0 (socket says 172.21.15.63:56498) 2024-03-22T07:07:33.165 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.302+0000 7fc23b7fe700 1 -- 172.21.15.63:0/3726333461 learned_addr learned my addr 172.21.15.63:0/3726333461 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:33.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.303+0000 7fc23b7fe700 1 -- 172.21.15.63:0/3726333461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c0094f0 con 0x7fc23c078c50 2024-03-22T07:07:33.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.303+0000 7fc23b7fe700 1 --2- 172.21.15.63:0/3726333461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c078c50 0x7fc23c113820 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0094c0 tx=0x7fc22c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:33.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.303+0000 7fc223fff700 1 -- 172.21.15.63:0/3726333461 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc22c010890 con 0x7fc23c078c50 2024-03-22T07:07:33.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.303+0000 7fc223fff700 1 -- 172.21.15.63:0/3726333461 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc22c010ec0 con 0x7fc23c078c50 2024-03-22T07:07:33.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.303+0000 7fc241bfd700 1 -- 172.21.15.63:0/3726333461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23c113f60 con 0x7fc23c078c50 2024-03-22T07:07:33.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.303+0000 7fc223fff700 1 -- 172.21.15.63:0/3726333461 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc22c0243f0 con 0x7fc23c078c50 2024-03-22T07:07:33.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.303+0000 7fc241bfd700 1 -- 172.21.15.63:0/3726333461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23c114ad0 con 0x7fc23c078c50 2024-03-22T07:07:33.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.304+0000 7fc223fff700 1 -- 172.21.15.63:0/3726333461 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7fc22c02d3f0 con 0x7fc23c078c50 2024-03-22T07:07:33.166 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.304+0000 7fc223fff700 1 --2- 172.21.15.63:0/3726333461 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fc2240367e0 0x7fc224038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:33.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.304+0000 7fc241bfd700 1 -- 172.21.15.63:0/3726333461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc23c05f5f0 con 0x7fc23c078c50 2024-03-22T07:07:33.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.304+0000 7fc223fff700 1 -- 172.21.15.63:0/3726333461 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc22c04a220 con 0x7fc23c078c50 2024-03-22T07:07:33.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.304+0000 7fc23affd700 1 --2- 172.21.15.63:0/3726333461 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fc2240367e0 0x7fc224038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:33.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.305+0000 7fc23affd700 1 --2- 172.21.15.63:0/3726333461 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fc2240367e0 0x7fc224038c90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc230009990 tx=0x7fc230006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:33.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.309+0000 7fc223fff700 1 -- 172.21.15.63:0/3726333461 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc22c02b8e0 con 0x7fc23c078c50 2024-03-22T07:07:33.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.506+0000 7fc241bfd700 1 -- 172.21.15.63:0/3726333461 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fc23c04bed0 con 0x7fc23c078c50 2024-03-22T07:07:33.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.511+0000 7fc223fff700 1 -- 172.21.15.63:0/3726333461 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fc22c02b2f0 con 0x7fc23c078c50 2024-03-22T07:07:33.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.513+0000 7fc241bfd700 1 -- 172.21.15.63:0/3726333461 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fc2240367e0 msgr2=0x7fc224038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:33.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.513+0000 7fc241bfd700 1 --2- 172.21.15.63:0/3726333461 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fc2240367e0 0x7fc224038c90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc230009990 tx=0x7fc230006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:33.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.513+0000 7fc241bfd700 1 -- 172.21.15.63:0/3726333461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c078c50 msgr2=0x7fc23c113820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:33.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.513+0000 7fc241bfd700 1 --2- 172.21.15.63:0/3726333461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c078c50 0x7fc23c113820 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0094c0 tx=0x7fc22c009250 comp rx=0 tx=0).stop 2024-03-22T07:07:33.167 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.513+0000 7fc241bfd700 1 -- 172.21.15.63:0/3726333461 shutdown_connections 2024-03-22T07:07:33.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.513+0000 7fc241bfd700 1 --2- 172.21.15.63:0/3726333461 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fc2240367e0 0x7fc224038c90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:33.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.513+0000 7fc241bfd700 1 --2- 172.21.15.63:0/3726333461 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc23c078c50 0x7fc23c113820 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:33.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.513+0000 7fc241bfd700 1 -- 172.21.15.63:0/3726333461 >> 172.21.15.63:0/3726333461 conn(0x7fc23c0fde90 msgr2=0x7fc23c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:33.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.513+0000 7fc241bfd700 1 -- 172.21.15.63:0/3726333461 shutdown_connections 2024-03-22T07:07:33.168 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:32.513+0000 7fc241bfd700 1 -- 172.21.15.63:0/3726333461 wait complete. 2024-03-22T07:07:33.168 INFO:teuthology.orchestra.run.smithi063.stdout:Enabling the dashboard module... 2024-03-22T07:07:33.809 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:33 smithi063 ceph-mon[29200]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:07:33.809 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:33 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3726333461' entity='client.admin' 2024-03-22T07:07:34.829 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:34 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/1483921182' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-22T07:07:35.104 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.551+0000 7f6e70d53700 1 Processor -- start 2024-03-22T07:07:35.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.551+0000 7f6e70d53700 1 -- start start 2024-03-22T07:07:35.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.551+0000 7f6e70d53700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c101610 0x7f6e6c101a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:35.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.551+0000 7f6e70d53700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c101f60 con 0x7f6e6c101610 2024-03-22T07:07:35.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.551+0000 7f6e6a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c101610 0x7f6e6c101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:35.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.551+0000 7f6e6a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c101610 0x7f6e6c101a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56510/0 (socket says 172.21.15.63:56510) 2024-03-22T07:07:35.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.551+0000 7f6e6a59c700 1 -- 172.21.15.63:0/1781449765 learned_addr learned my addr 172.21.15.63:0/1781449765 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:35.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.551+0000 7f6e6a59c700 1 -- 172.21.15.63:0/1781449765 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c1020a0 con 0x7f6e6c101610 2024-03-22T07:07:35.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.552+0000 7f6e6a59c700 1 --2- 172.21.15.63:0/1781449765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c101610 0x7f6e6c101a20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0098e0 tx=0x7f6e5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675a5b1e1cc062ee server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:35.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.552+0000 7f6e6959a700 1 -- 172.21.15.63:0/1781449765 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e5c00b860 con 0x7f6e6c101610 2024-03-22T07:07:35.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.552+0000 7f6e6959a700 1 -- 172.21.15.63:0/1781449765 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e5c00be90 con 0x7f6e6c101610 2024-03-22T07:07:35.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.552+0000 7f6e6959a700 1 -- 172.21.15.63:0/1781449765 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e5c0103c0 con 0x7f6e6c101610 2024-03-22T07:07:35.105 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.552+0000 7f6e70d53700 1 -- 172.21.15.63:0/1781449765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c101610 msgr2=0x7f6e6c101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:35.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.552+0000 7f6e70d53700 1 --2- 172.21.15.63:0/1781449765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c101610 0x7f6e6c101a20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0098e0 tx=0x7f6e5c009250 comp rx=0 tx=0).stop 2024-03-22T07:07:35.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.552+0000 7f6e70d53700 1 -- 172.21.15.63:0/1781449765 shutdown_connections 2024-03-22T07:07:35.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.552+0000 7f6e70d53700 1 --2- 172.21.15.63:0/1781449765 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c101610 0x7f6e6c101a20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:35.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.552+0000 7f6e70d53700 1 -- 172.21.15.63:0/1781449765 >> 172.21.15.63:0/1781449765 conn(0x7f6e6c0fcd90 msgr2=0x7f6e6c0ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:35.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.552+0000 7f6e70d53700 1 -- 172.21.15.63:0/1781449765 shutdown_connections 2024-03-22T07:07:35.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.552+0000 7f6e70d53700 1 -- 172.21.15.63:0/1781449765 wait complete. 2024-03-22T07:07:35.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.553+0000 7f6e70d53700 1 Processor -- start 2024-03-22T07:07:35.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.553+0000 7f6e70d53700 1 -- start start 2024-03-22T07:07:35.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.553+0000 7f6e70d53700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c101610 0x7f6e6c1115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:35.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.553+0000 7f6e70d53700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c111b20 con 0x7f6e6c101610 2024-03-22T07:07:35.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.553+0000 7f6e6a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c101610 0x7f6e6c1115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:35.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.553+0000 7f6e6a59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c101610 0x7f6e6c1115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56518/0 (socket says 172.21.15.63:56518) 2024-03-22T07:07:35.106 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.553+0000 7f6e6a59c700 1 -- 172.21.15.63:0/1483921182 learned_addr learned my addr 172.21.15.63:0/1483921182 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:35.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.553+0000 7f6e6a59c700 1 -- 172.21.15.63:0/1483921182 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c0094f0 con 0x7f6e6c101610 2024-03-22T07:07:35.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.553+0000 7f6e6a59c700 1 --2- 172.21.15.63:0/1483921182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c101610 0x7f6e6c1115e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0090b0 tx=0x7f6e5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:35.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.554+0000 7f6e5affd700 1 -- 172.21.15.63:0/1483921182 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e5c010840 con 0x7f6e6c101610 2024-03-22T07:07:35.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.554+0000 7f6e5affd700 1 -- 172.21.15.63:0/1483921182 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e5c010e70 con 0x7f6e6c101610 2024-03-22T07:07:35.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.554+0000 7f6e70d53700 1 -- 172.21.15.63:0/1483921182 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c111d20 con 0x7f6e6c101610 2024-03-22T07:07:35.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.554+0000 7f6e5affd700 1 -- 172.21.15.63:0/1483921182 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6e5c024e40 con 0x7f6e6c101610 2024-03-22T07:07:35.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.554+0000 7f6e70d53700 1 -- 172.21.15.63:0/1483921182 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c112890 con 0x7f6e6c101610 2024-03-22T07:07:35.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.554+0000 7f6e5affd700 1 -- 172.21.15.63:0/1483921182 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 8) v1 ==== 43368+0+0 (secure 0 0 0) 0x7f6e5c0109a0 con 0x7f6e6c101610 2024-03-22T07:07:35.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.554+0000 7f6e70d53700 1 -- 172.21.15.63:0/1483921182 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6c05f5f0 con 0x7f6e6c101610 2024-03-22T07:07:35.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.555+0000 7f6e5affd700 1 --2- 172.21.15.63:0/1483921182 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f6e54036790 0x7f6e54038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:35.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.555+0000 7f6e5affd700 1 -- 172.21.15.63:0/1483921182 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f6e5c04a760 con 0x7f6e6c101610 2024-03-22T07:07:35.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.555+0000 7f6e69d9b700 1 --2- 172.21.15.63:0/1483921182 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f6e54036790 0x7f6e54038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:35.107 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.555+0000 7f6e69d9b700 1 --2- 172.21.15.63:0/1483921182 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f6e54036790 0x7f6e54038c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009990 tx=0x7f6e60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:35.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.559+0000 7f6e5affd700 1 -- 172.21.15.63:0/1483921182 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6e5c024430 con 0x7f6e6c101610 2024-03-22T07:07:35.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:33.808+0000 7f6e70d53700 1 -- 172.21.15.63:0/1483921182 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f6e6c04bed0 con 0x7f6e6c101610 2024-03-22T07:07:35.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:34.518+0000 7f6e5affd700 1 -- 172.21.15.63:0/1483921182 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f6e5c02e440 con 0x7f6e6c101610 2024-03-22T07:07:35.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:34.520+0000 7f6e5affd700 1 -- 172.21.15.63:0/1483921182 <== mon.0 v2:172.21.15.63:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f6e5c018530 con 0x7f6e6c101610 2024-03-22T07:07:35.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:34.522+0000 7f6e70d53700 1 -- 172.21.15.63:0/1483921182 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f6e54036790 msgr2=0x7f6e54038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:35.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:34.522+0000 7f6e70d53700 1 --2- 172.21.15.63:0/1483921182 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f6e54036790 0x7f6e54038c40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009990 tx=0x7f6e60006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:35.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:34.522+0000 7f6e70d53700 1 -- 172.21.15.63:0/1483921182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c101610 msgr2=0x7f6e6c1115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:35.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:34.522+0000 7f6e70d53700 1 --2- 172.21.15.63:0/1483921182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c101610 0x7f6e6c1115e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0090b0 tx=0x7f6e5c009250 comp rx=0 tx=0).stop 2024-03-22T07:07:35.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:34.522+0000 7f6e70d53700 1 -- 172.21.15.63:0/1483921182 shutdown_connections 2024-03-22T07:07:35.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:34.522+0000 7f6e70d53700 1 --2- 172.21.15.63:0/1483921182 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f6e54036790 0x7f6e54038c40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:35.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:34.523+0000 7f6e70d53700 1 --2- 172.21.15.63:0/1483921182 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c101610 0x7f6e6c1115e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:35.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:34.523+0000 7f6e70d53700 1 -- 172.21.15.63:0/1483921182 >> 172.21.15.63:0/1483921182 conn(0x7f6e6c0fcd90 msgr2=0x7f6e6c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:35.108 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:34.523+0000 7f6e70d53700 1 -- 172.21.15.63:0/1483921182 shutdown_connections 2024-03-22T07:07:35.109 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:34.523+0000 7f6e70d53700 1 -- 172.21.15.63:0/1483921182 wait complete. 2024-03-22T07:07:35.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:35 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/1483921182' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-22T07:07:35.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:35 smithi063 ceph-mon[29200]: mgrmap e9: smithi063.mnxayh(active, since 24s) 2024-03-22T07:07:36.370 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-03-22T07:07:36.370 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-22T07:07:36.370 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-22T07:07:36.370 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "active_name": "smithi063.mnxayh", 2024-03-22T07:07:36.370 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-22T07:07:36.370 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-03-22T07:07:36.370 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.529+0000 7fce9b3e1700 1 Processor -- start 2024-03-22T07:07:36.370 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.529+0000 7fce9b3e1700 1 -- start start 2024-03-22T07:07:36.370 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.529+0000 7fce9b3e1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce941015d0 0x7fce941019e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:36.370 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.529+0000 7fce9b3e1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce94101f20 con 0x7fce941015d0 2024-03-22T07:07:36.370 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.530+0000 7fce9917d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce941015d0 0x7fce941019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:36.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.530+0000 7fce9917d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce941015d0 0x7fce941019e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49738/0 (socket says 172.21.15.63:49738) 2024-03-22T07:07:36.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.530+0000 7fce9917d700 1 -- 172.21.15.63:0/1830930662 learned_addr learned my addr 172.21.15.63:0/1830930662 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:36.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.530+0000 7fce9917d700 1 -- 172.21.15.63:0/1830930662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce94102060 con 0x7fce941015d0 2024-03-22T07:07:36.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.530+0000 7fce9917d700 1 --2- 172.21.15.63:0/1830930662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce941015d0 0x7fce941019e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fce8400bb80 tx=0x7fce8400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26524abb3030fd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:36.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.530+0000 7fce8bfff700 1 -- 172.21.15.63:0/1830930662 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce8400dd00 con 0x7fce941015d0 2024-03-22T07:07:36.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.530+0000 7fce8bfff700 1 -- 172.21.15.63:0/1830930662 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce8400de60 con 0x7fce941015d0 2024-03-22T07:07:36.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.530+0000 7fce8bfff700 1 -- 172.21.15.63:0/1830930662 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce84018920 con 0x7fce941015d0 2024-03-22T07:07:36.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.531+0000 7fce9b3e1700 1 -- 172.21.15.63:0/1830930662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce941015d0 msgr2=0x7fce941019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:36.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.531+0000 7fce9b3e1700 1 --2- 172.21.15.63:0/1830930662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce941015d0 0x7fce941019e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fce8400bb80 tx=0x7fce8400b4f0 comp rx=0 tx=0).stop 2024-03-22T07:07:36.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.531+0000 7fce9b3e1700 1 -- 172.21.15.63:0/1830930662 shutdown_connections 2024-03-22T07:07:36.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.531+0000 7fce9b3e1700 1 --2- 172.21.15.63:0/1830930662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce941015d0 0x7fce941019e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:36.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.531+0000 7fce9b3e1700 1 -- 172.21.15.63:0/1830930662 >> 172.21.15.63:0/1830930662 conn(0x7fce940fcd50 msgr2=0x7fce940ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:36.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.531+0000 7fce9b3e1700 1 -- 172.21.15.63:0/1830930662 shutdown_connections 2024-03-22T07:07:36.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.531+0000 7fce9b3e1700 1 -- 172.21.15.63:0/1830930662 wait complete. 2024-03-22T07:07:36.371 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.532+0000 7fce9b3e1700 1 Processor -- start 2024-03-22T07:07:36.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.532+0000 7fce9b3e1700 1 -- start start 2024-03-22T07:07:36.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.532+0000 7fce9b3e1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce941015d0 0x7fce9410f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:36.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.532+0000 7fce9b3e1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9410f8e0 con 0x7fce941015d0 2024-03-22T07:07:36.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.532+0000 7fce9917d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce941015d0 0x7fce9410f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:36.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.532+0000 7fce9917d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce941015d0 0x7fce9410f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49754/0 (socket says 172.21.15.63:49754) 2024-03-22T07:07:36.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.532+0000 7fce9917d700 1 -- 172.21.15.63:0/523220543 learned_addr learned my addr 172.21.15.63:0/523220543 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:36.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.532+0000 7fce9917d700 1 -- 172.21.15.63:0/523220543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8400b790 con 0x7fce941015d0 2024-03-22T07:07:36.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.532+0000 7fce9917d700 1 --2- 172.21.15.63:0/523220543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce941015d0 0x7fce9410f3a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fce84006b00 tx=0x7fce8400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:36.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.532+0000 7fce89ffb700 1 -- 172.21.15.63:0/523220543 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce84010b80 con 0x7fce941015d0 2024-03-22T07:07:36.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.533+0000 7fce89ffb700 1 -- 172.21.15.63:0/523220543 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce8401d050 con 0x7fce941015d0 2024-03-22T07:07:36.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.533+0000 7fce9b3e1700 1 -- 172.21.15.63:0/523220543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce9410fae0 con 0x7fce941015d0 2024-03-22T07:07:36.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.533+0000 7fce89ffb700 1 -- 172.21.15.63:0/523220543 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fce84022590 con 0x7fce941015d0 2024-03-22T07:07:36.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.533+0000 7fce9b3e1700 1 -- 172.21.15.63:0/523220543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce94110650 con 0x7fce941015d0 2024-03-22T07:07:36.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.533+0000 7fce89ffb700 1 -- 172.21.15.63:0/523220543 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fce840226f0 con 0x7fce941015d0 2024-03-22T07:07:36.372 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.534+0000 7fce89ffb700 1 --2- 172.21.15.63:0/523220543 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fce80036830 0x7fce80038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:36.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.534+0000 7fce89ffb700 1 -- 172.21.15.63:0/523220543 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fce84014070 con 0x7fce941015d0 2024-03-22T07:07:36.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.534+0000 7fce9897c700 1 -- 172.21.15.63:0/523220543 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fce80036830 msgr2=0x7fce80038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/480818549 2024-03-22T07:07:36.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.534+0000 7fce9897c700 1 --2- 172.21.15.63:0/523220543 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fce80036830 0x7fce80038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:07:36.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.534+0000 7fce9b3e1700 1 -- 172.21.15.63:0/523220543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce9405f5f0 con 0x7fce941015d0 2024-03-22T07:07:36.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.542+0000 7fce89ffb700 1 -- 172.21.15.63:0/523220543 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fce84022a40 con 0x7fce941015d0 2024-03-22T07:07:36.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.735+0000 7fce9897c700 1 -- 172.21.15.63:0/523220543 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fce80036830 msgr2=0x7fce80038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/480818549 2024-03-22T07:07:36.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.735+0000 7fce9897c700 1 --2- 172.21.15.63:0/523220543 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fce80036830 0x7fce80038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T07:07:36.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.792+0000 7fce9b3e1700 1 -- 172.21.15.63:0/523220543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fce941110c0 con 0x7fce941015d0 2024-03-22T07:07:36.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.793+0000 7fce89ffb700 1 -- 172.21.15.63:0/523220543 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7fce84021b20 con 0x7fce941015d0 2024-03-22T07:07:36.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.795+0000 7fce9b3e1700 1 -- 172.21.15.63:0/523220543 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fce80036830 msgr2=0x7fce80038ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:07:36.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.795+0000 7fce9b3e1700 1 --2- 172.21.15.63:0/523220543 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fce80036830 0x7fce80038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:36.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.795+0000 7fce9b3e1700 1 -- 172.21.15.63:0/523220543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce941015d0 msgr2=0x7fce9410f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:36.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.795+0000 7fce9b3e1700 1 --2- 172.21.15.63:0/523220543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce941015d0 0x7fce9410f3a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fce84006b00 tx=0x7fce8400b4f0 comp rx=0 tx=0).stop 2024-03-22T07:07:36.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.795+0000 7fce9b3e1700 1 -- 172.21.15.63:0/523220543 shutdown_connections 2024-03-22T07:07:36.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.795+0000 7fce9b3e1700 1 --2- 172.21.15.63:0/523220543 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7fce80036830 0x7fce80038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:36.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.795+0000 7fce9b3e1700 1 --2- 172.21.15.63:0/523220543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce941015d0 0x7fce9410f3a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:36.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.795+0000 7fce9b3e1700 1 -- 172.21.15.63:0/523220543 >> 172.21.15.63:0/523220543 conn(0x7fce940fcd50 msgr2=0x7fce94105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:36.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.795+0000 7fce9b3e1700 1 -- 172.21.15.63:0/523220543 shutdown_connections 2024-03-22T07:07:36.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:35.795+0000 7fce9b3e1700 1 -- 172.21.15.63:0/523220543 wait complete. 2024-03-22T07:07:36.374 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for the mgr to restart... 2024-03-22T07:07:36.374 INFO:teuthology.orchestra.run.smithi063.stdout:Waiting for mgr epoch 9... 2024-03-22T07:07:36.617 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:36 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/523220543' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-22T07:07:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:41 smithi063 ceph-mon[29200]: Active manager daemon smithi063.mnxayh restarted 2024-03-22T07:07:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:41 smithi063 ceph-mon[29200]: Activating manager daemon smithi063.mnxayh 2024-03-22T07:07:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:41 smithi063 ceph-mon[29200]: osdmap e3: 0 total, 0 up, 0 in 2024-03-22T07:07:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:41 smithi063 ceph-mon[29200]: mgrmap e10: smithi063.mnxayh(active, starting, since 0.0032703s) 2024-03-22T07:07:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:41 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:07:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:41 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr metadata", "who": "smithi063.mnxayh", "id": "smithi063.mnxayh"}]: dispatch 2024-03-22T07:07:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:41 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:07:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:41 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:07:41.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:41 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:07:41.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:41 smithi063 ceph-mon[29200]: Manager daemon smithi063.mnxayh is now available 2024-03-22T07:07:41.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:41 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:07:41.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:41 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:07:42.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:42 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.mnxayh/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:07:42.667 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:42 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.mnxayh/trash_purge_schedule"}]: dispatch 2024-03-22T07:07:42.667 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:42 smithi063 ceph-mon[29200]: mgrmap e11: smithi063.mnxayh(active, since 1.00577s) 2024-03-22T07:07:42.932 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout { 2024-03-22T07:07:42.932 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-22T07:07:42.932 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-22T07:07:42.932 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout } 2024-03-22T07:07:42.932 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.772+0000 7f0840189700 1 Processor -- start 2024-03-22T07:07:42.932 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.772+0000 7f0840189700 1 -- start start 2024-03-22T07:07:42.932 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.772+0000 7f0840189700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0838076c00 0x7f08381052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:42.932 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.772+0000 7f0840189700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0838105810 con 0x7f0838076c00 2024-03-22T07:07:42.932 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.773+0000 7f083df25700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0838076c00 0x7f08381052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:42.932 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.773+0000 7f083df25700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0838076c00 0x7f08381052d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49760/0 (socket says 172.21.15.63:49760) 2024-03-22T07:07:42.932 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.773+0000 7f083df25700 1 -- 172.21.15.63:0/3751822273 learned_addr learned my addr 172.21.15.63:0/3751822273 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:42.932 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.773+0000 7f083df25700 1 -- 172.21.15.63:0/3751822273 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0838105950 con 0x7f0838076c00 2024-03-22T07:07:42.932 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.773+0000 7f083df25700 1 --2- 172.21.15.63:0/3751822273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0838076c00 0x7f08381052d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f08280098e0 tx=0x7f0828009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e57741b85b309d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:42.933 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.773+0000 7f083cf23700 1 -- 172.21.15.63:0/3751822273 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f082800b860 con 0x7f0838076c00 2024-03-22T07:07:42.933 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.773+0000 7f083cf23700 1 -- 172.21.15.63:0/3751822273 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f082800be90 con 0x7f0838076c00 2024-03-22T07:07:42.933 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.773+0000 7f083cf23700 1 -- 172.21.15.63:0/3751822273 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08280103c0 con 0x7f0838076c00 2024-03-22T07:07:42.933 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.774+0000 7f0840189700 1 -- 172.21.15.63:0/3751822273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0838076c00 msgr2=0x7f08381052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:42.933 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.774+0000 7f0840189700 1 --2- 172.21.15.63:0/3751822273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0838076c00 0x7f08381052d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f08280098e0 tx=0x7f0828009250 comp rx=0 tx=0).stop 2024-03-22T07:07:42.933 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.774+0000 7f0840189700 1 -- 172.21.15.63:0/3751822273 shutdown_connections 2024-03-22T07:07:42.933 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.774+0000 7f0840189700 1 --2- 172.21.15.63:0/3751822273 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0838076c00 0x7f08381052d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:42.933 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.774+0000 7f0840189700 1 -- 172.21.15.63:0/3751822273 >> 172.21.15.63:0/3751822273 conn(0x7f08380fcd80 msgr2=0x7f08380ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:42.933 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.774+0000 7f0840189700 1 -- 172.21.15.63:0/3751822273 shutdown_connections 2024-03-22T07:07:42.933 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.774+0000 7f0840189700 1 -- 172.21.15.63:0/3751822273 wait complete. 2024-03-22T07:07:42.933 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.775+0000 7f0840189700 1 Processor -- start 2024-03-22T07:07:42.933 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.775+0000 7f0840189700 1 -- start start 2024-03-22T07:07:42.933 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.775+0000 7f0840189700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0838076c00 0x7f0838102c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:42.933 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.775+0000 7f0840189700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0838103160 con 0x7f0838076c00 2024-03-22T07:07:42.933 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.775+0000 7f083df25700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0838076c00 0x7f0838102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:42.933 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.775+0000 7f083df25700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0838076c00 0x7f0838102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49776/0 (socket says 172.21.15.63:49776) 2024-03-22T07:07:42.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.775+0000 7f083df25700 1 -- 172.21.15.63:0/256473240 learned_addr learned my addr 172.21.15.63:0/256473240 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:42.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.775+0000 7f083df25700 1 -- 172.21.15.63:0/256473240 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08280094f0 con 0x7f0838076c00 2024-03-22T07:07:42.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.775+0000 7f083df25700 1 --2- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0838076c00 0x7f0838102c20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0828006d40 tx=0x7f0828009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:42.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.776+0000 7f082e7fc700 1 -- 172.21.15.63:0/256473240 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0828010840 con 0x7f0838076c00 2024-03-22T07:07:42.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.776+0000 7f0840189700 1 -- 172.21.15.63:0/256473240 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08381011d0 con 0x7f0838076c00 2024-03-22T07:07:42.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.776+0000 7f082e7fc700 1 -- 172.21.15.63:0/256473240 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0828010e70 con 0x7f0838076c00 2024-03-22T07:07:42.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.776+0000 7f082e7fc700 1 -- 172.21.15.63:0/256473240 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0828024ea0 con 0x7f0838076c00 2024-03-22T07:07:42.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.776+0000 7f0840189700 1 -- 172.21.15.63:0/256473240 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08381016c0 con 0x7f0838076c00 2024-03-22T07:07:42.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.776+0000 7f082e7fc700 1 -- 172.21.15.63:0/256473240 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 9) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f08280109a0 con 0x7f0838076c00 2024-03-22T07:07:42.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.777+0000 7f082e7fc700 1 --2- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f0824036790 0x7f0824038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:42.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.777+0000 7f082e7fc700 1 -- 172.21.15.63:0/256473240 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f082804a710 con 0x7f0838076c00 2024-03-22T07:07:42.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.777+0000 7f083d724700 1 -- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f0824036790 msgr2=0x7f0824038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/480818549 2024-03-22T07:07:42.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.777+0000 7f083d724700 1 --2- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f0824036790 0x7f0824038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:07:42.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.777+0000 7f0840189700 1 -- 172.21.15.63:0/256473240 --> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f083804bed0 con 0x7f0824036790 2024-03-22T07:07:42.934 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.978+0000 7f083d724700 1 -- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f0824036790 msgr2=0x7f0824038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/480818549 2024-03-22T07:07:42.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:36.978+0000 7f083d724700 1 --2- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f0824036790 0x7f0824038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T07:07:42.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:37.379+0000 7f083d724700 1 -- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f0824036790 msgr2=0x7f0824038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/480818549 2024-03-22T07:07:42.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:37.379+0000 7f083d724700 1 --2- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f0824036790 0x7f0824038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-22T07:07:42.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:38.180+0000 7f083d724700 1 -- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f0824036790 msgr2=0x7f0824038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/480818549 2024-03-22T07:07:42.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:38.180+0000 7f083d724700 1 --2- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f0824036790 0x7f0824038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-22T07:07:42.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:39.781+0000 7f083d724700 1 -- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f0824036790 msgr2=0x7f0824038c40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/480818549 2024-03-22T07:07:42.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:39.781+0000 7f083d724700 1 --2- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f0824036790 0x7f0824038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-22T07:07:42.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:41.263+0000 7f082e7fc700 1 -- 172.21.15.63:0/256473240 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mgrmap(e 10) v1 ==== 43148+0+0 (secure 0 0 0) 0x7f082802cd80 con 0x7f0838076c00 2024-03-22T07:07:42.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:41.263+0000 7f082e7fc700 1 -- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f0824036790 msgr2=0x7f0824038c40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:07:42.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:41.263+0000 7f082e7fc700 1 --2- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f0824036790 0x7f0824038c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:42.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.264+0000 7f082e7fc700 1 -- 172.21.15.63:0/256473240 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f082802e7c0 con 0x7f0838076c00 2024-03-22T07:07:42.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.264+0000 7f082e7fc700 1 --2- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f0824039ff0 0x7f082403c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:42.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.264+0000 7f082e7fc700 1 -- 172.21.15.63:0/256473240 --> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f083804bed0 con 0x7f0824039ff0 2024-03-22T07:07:42.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.265+0000 7f083d724700 1 --2- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f0824039ff0 0x7f082403c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:42.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.265+0000 7f083d724700 1 --2- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f0824039ff0 0x7f082403c3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f08340039d0 tx=0x7f08340092f0 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:42.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.266+0000 7f082e7fc700 1 -- 172.21.15.63:0/256473240 <== mgr.14162 v2:172.21.15.63:6800/2296442540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f083804bed0 con 0x7f0824039ff0 2024-03-22T07:07:42.935 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.273+0000 7f0840189700 1 -- 172.21.15.63:0/256473240 --> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f0838106730 con 0x7f0824039ff0 2024-03-22T07:07:42.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.273+0000 7f082e7fc700 1 -- 172.21.15.63:0/256473240 <== mgr.14162 v2:172.21.15.63:6800/2296442540 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f0838106730 con 0x7f0824039ff0 2024-03-22T07:07:42.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.273+0000 7f0840189700 1 -- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f0824039ff0 msgr2=0x7f082403c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:42.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.273+0000 7f0840189700 1 --2- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f0824039ff0 0x7f082403c3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f08340039d0 tx=0x7f08340092f0 comp rx=0 tx=0).stop 2024-03-22T07:07:42.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.273+0000 7f0840189700 1 -- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0838076c00 msgr2=0x7f0838102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:42.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.273+0000 7f0840189700 1 --2- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0838076c00 0x7f0838102c20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0828006d40 tx=0x7f0828009250 comp rx=0 tx=0).stop 2024-03-22T07:07:42.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.273+0000 7f0840189700 1 -- 172.21.15.63:0/256473240 shutdown_connections 2024-03-22T07:07:42.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.273+0000 7f0840189700 1 --2- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f0824039ff0 0x7f082403c3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:42.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.273+0000 7f0840189700 1 --2- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0838076c00 0x7f0838102c20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:42.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.273+0000 7f0840189700 1 --2- 172.21.15.63:0/256473240 >> [v2:172.21.15.63:6800/480818549,v1:172.21.15.63:6801/480818549] conn(0x7f0824036790 0x7f0824038c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:42.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.273+0000 7f0840189700 1 -- 172.21.15.63:0/256473240 >> 172.21.15.63:0/256473240 conn(0x7f08380fcd80 msgr2=0x7f08380fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:42.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.273+0000 7f0840189700 1 -- 172.21.15.63:0/256473240 shutdown_connections 2024-03-22T07:07:42.936 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:42.273+0000 7f0840189700 1 -- 172.21.15.63:0/256473240 wait complete. 2024-03-22T07:07:42.936 INFO:teuthology.orchestra.run.smithi063.stdout:mgr epoch 9 is available 2024-03-22T07:07:42.936 INFO:teuthology.orchestra.run.smithi063.stdout:Generating a dashboard self-signed certificate... 2024-03-22T07:07:43.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:43 smithi063 ceph-mon[29200]: [22/Mar/2024:07:07:42] ENGINE Bus STARTING 2024-03-22T07:07:43.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:43 smithi063 ceph-mon[29200]: [22/Mar/2024:07:07:42] ENGINE Serving on https://172.21.15.63:7150 2024-03-22T07:07:43.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:43 smithi063 ceph-mon[29200]: [22/Mar/2024:07:07:42] ENGINE Bus STARTED 2024-03-22T07:07:43.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:43 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:43.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:43 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:43.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:43 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:44.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-22T07:07:44.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.318+0000 7f4e79309700 1 Processor -- start 2024-03-22T07:07:44.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.318+0000 7f4e79309700 1 -- start start 2024-03-22T07:07:44.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.318+0000 7f4e79309700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e74101660 0x7f4e74101a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:44.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.319+0000 7f4e79309700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e74101fb0 con 0x7f4e74101660 2024-03-22T07:07:44.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.319+0000 7f4e72ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e74101660 0x7f4e74101a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:44.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.319+0000 7f4e72ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e74101660 0x7f4e74101a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49862/0 (socket says 172.21.15.63:49862) 2024-03-22T07:07:44.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.319+0000 7f4e72ffd700 1 -- 172.21.15.63:0/2255344504 learned_addr learned my addr 172.21.15.63:0/2255344504 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:44.373 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.319+0000 7f4e72ffd700 1 -- 172.21.15.63:0/2255344504 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e741020f0 con 0x7f4e74101660 2024-03-22T07:07:44.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.319+0000 7f4e72ffd700 1 --2- 172.21.15.63:0/2255344504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e74101660 0x7f4e74101a70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009e50 tx=0x7f4e6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa21b7a98ec44a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:44.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.319+0000 7f4e71ffb700 1 -- 172.21.15.63:0/2255344504 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e6c0092e0 con 0x7f4e74101660 2024-03-22T07:07:44.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.319+0000 7f4e71ffb700 1 -- 172.21.15.63:0/2255344504 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e6c012430 con 0x7f4e74101660 2024-03-22T07:07:44.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.319+0000 7f4e71ffb700 1 -- 172.21.15.63:0/2255344504 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e6c0075a0 con 0x7f4e74101660 2024-03-22T07:07:44.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.320+0000 7f4e79309700 1 -- 172.21.15.63:0/2255344504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e74101660 msgr2=0x7f4e74101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:44.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.320+0000 7f4e79309700 1 --2- 172.21.15.63:0/2255344504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e74101660 0x7f4e74101a70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009e50 tx=0x7f4e6c009840 comp rx=0 tx=0).stop 2024-03-22T07:07:44.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.320+0000 7f4e79309700 1 -- 172.21.15.63:0/2255344504 shutdown_connections 2024-03-22T07:07:44.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.320+0000 7f4e79309700 1 --2- 172.21.15.63:0/2255344504 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e74101660 0x7f4e74101a70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:44.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.320+0000 7f4e79309700 1 -- 172.21.15.63:0/2255344504 >> 172.21.15.63:0/2255344504 conn(0x7f4e740fcde0 msgr2=0x7f4e740ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:44.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.320+0000 7f4e79309700 1 -- 172.21.15.63:0/2255344504 shutdown_connections 2024-03-22T07:07:44.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.320+0000 7f4e79309700 1 -- 172.21.15.63:0/2255344504 wait complete. 2024-03-22T07:07:44.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.321+0000 7f4e79309700 1 Processor -- start 2024-03-22T07:07:44.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.321+0000 7f4e79309700 1 -- start start 2024-03-22T07:07:44.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.321+0000 7f4e79309700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e74101660 0x7f4e741137a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:44.374 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.321+0000 7f4e79309700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e74113ce0 con 0x7f4e74101660 2024-03-22T07:07:44.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.321+0000 7f4e72ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e74101660 0x7f4e741137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:44.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.321+0000 7f4e72ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e74101660 0x7f4e741137a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49868/0 (socket says 172.21.15.63:49868) 2024-03-22T07:07:44.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.321+0000 7f4e72ffd700 1 -- 172.21.15.63:0/2827119595 learned_addr learned my addr 172.21.15.63:0/2827119595 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:44.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.322+0000 7f4e72ffd700 1 -- 172.21.15.63:0/2827119595 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c009ae0 con 0x7f4e74101660 2024-03-22T07:07:44.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.322+0000 7f4e72ffd700 1 --2- 172.21.15.63:0/2827119595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e74101660 0x7f4e741137a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0127d0 tx=0x7f4e6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:44.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.322+0000 7f4e6b7fe700 1 -- 172.21.15.63:0/2827119595 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e6c0092e0 con 0x7f4e74101660 2024-03-22T07:07:44.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.322+0000 7f4e6b7fe700 1 -- 172.21.15.63:0/2827119595 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e6c007d30 con 0x7f4e74101660 2024-03-22T07:07:44.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.322+0000 7f4e6b7fe700 1 -- 172.21.15.63:0/2827119595 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4e6c025960 con 0x7f4e74101660 2024-03-22T07:07:44.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.322+0000 7f4e79309700 1 -- 172.21.15.63:0/2827119595 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e74113ee0 con 0x7f4e74101660 2024-03-22T07:07:44.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.322+0000 7f4e79309700 1 -- 172.21.15.63:0/2827119595 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e74114a50 con 0x7f4e74101660 2024-03-22T07:07:44.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.323+0000 7f4e6b7fe700 1 -- 172.21.15.63:0/2827119595 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 11) v1 ==== 43275+0+0 (secure 0 0 0) 0x7f4e6c019050 con 0x7f4e74101660 2024-03-22T07:07:44.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.323+0000 7f4e79309700 1 -- 172.21.15.63:0/2827119595 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e7405f5f0 con 0x7f4e74101660 2024-03-22T07:07:44.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.323+0000 7f4e6b7fe700 1 --2- 172.21.15.63:0/2827119595 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f4e60036700 0x7f4e60038bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:44.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.323+0000 7f4e6b7fe700 1 -- 172.21.15.63:0/2827119595 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4e6c049f60 con 0x7f4e74101660 2024-03-22T07:07:44.375 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.324+0000 7f4e727fc700 1 --2- 172.21.15.63:0/2827119595 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f4e60036700 0x7f4e60038bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:44.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.324+0000 7f4e727fc700 1 --2- 172.21.15.63:0/2827119595 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f4e60036700 0x7f4e60038bb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009990 tx=0x7f4e5c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:44.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.328+0000 7f4e6b7fe700 1 -- 172.21.15.63:0/2827119595 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4e6c07c190 con 0x7f4e74101660 2024-03-22T07:07:44.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.530+0000 7f4e79309700 1 -- 172.21.15.63:0/2827119595 --> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e74114e60 con 0x7f4e60036700 2024-03-22T07:07:44.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.577+0000 7f4e6b7fe700 1 -- 172.21.15.63:0/2827119595 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7f4e6c01c500 con 0x7f4e74101660 2024-03-22T07:07:44.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.659+0000 7f4e6b7fe700 1 -- 172.21.15.63:0/2827119595 <== mgr.14162 v2:172.21.15.63:6800/2296442540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f4e74114e60 con 0x7f4e60036700 2024-03-22T07:07:44.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.661+0000 7f4e79309700 1 -- 172.21.15.63:0/2827119595 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f4e60036700 msgr2=0x7f4e60038bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:44.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.661+0000 7f4e79309700 1 --2- 172.21.15.63:0/2827119595 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f4e60036700 0x7f4e60038bb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009990 tx=0x7f4e5c006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:44.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.661+0000 7f4e79309700 1 -- 172.21.15.63:0/2827119595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e74101660 msgr2=0x7f4e741137a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:44.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.661+0000 7f4e79309700 1 --2- 172.21.15.63:0/2827119595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e74101660 0x7f4e741137a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0127d0 tx=0x7f4e6c009840 comp rx=0 tx=0).stop 2024-03-22T07:07:44.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.661+0000 7f4e79309700 1 -- 172.21.15.63:0/2827119595 shutdown_connections 2024-03-22T07:07:44.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.661+0000 7f4e79309700 1 --2- 172.21.15.63:0/2827119595 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f4e60036700 0x7f4e60038bb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:44.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.661+0000 7f4e79309700 1 --2- 172.21.15.63:0/2827119595 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4e74101660 0x7f4e741137a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:44.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.661+0000 7f4e79309700 1 -- 172.21.15.63:0/2827119595 >> 172.21.15.63:0/2827119595 conn(0x7f4e740fcde0 msgr2=0x7f4e74105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:44.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.662+0000 7f4e79309700 1 -- 172.21.15.63:0/2827119595 shutdown_connections 2024-03-22T07:07:44.376 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:43.662+0000 7f4e79309700 1 -- 172.21.15.63:0/2827119595 wait complete. 2024-03-22T07:07:44.377 INFO:teuthology.orchestra.run.smithi063.stdout:Creating initial admin user... 2024-03-22T07:07:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:44 smithi063 ceph-mon[29200]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:44 smithi063 ceph-mon[29200]: mgrmap e12: smithi063.mnxayh(active, since 2s) 2024-03-22T07:07:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:44 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:44 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:46.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$T5eKrbJfP4/PMbpI34xGeu0vK/MMf1zBSu6TqfSo48wmnv.rmBw8e", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711091265, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-22T07:07:46.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.809+0000 7fdf3944f700 1 Processor -- start 2024-03-22T07:07:46.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.809+0000 7fdf3944f700 1 -- start start 2024-03-22T07:07:46.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.809+0000 7fdf3944f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf34076ad0 0x7fdf34105410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:46.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.809+0000 7fdf3944f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf34076ee0 con 0x7fdf34076ad0 2024-03-22T07:07:46.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.809+0000 7fdf32ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf34076ad0 0x7fdf34105410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:46.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.809+0000 7fdf32ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf34076ad0 0x7fdf34105410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49608/0 (socket says 172.21.15.63:49608) 2024-03-22T07:07:46.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.809+0000 7fdf32ffd700 1 -- 172.21.15.63:0/1854291519 learned_addr learned my addr 172.21.15.63:0/1854291519 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:46.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.810+0000 7fdf32ffd700 1 -- 172.21.15.63:0/1854291519 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf34105950 con 0x7fdf34076ad0 2024-03-22T07:07:46.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.810+0000 7fdf32ffd700 1 --2- 172.21.15.63:0/1854291519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf34076ad0 0x7fdf34105410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0098e0 tx=0x7fdf2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80514ea0b2f37b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:46.225 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.810+0000 7fdf31ffb700 1 -- 172.21.15.63:0/1854291519 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf2c00b860 con 0x7fdf34076ad0 2024-03-22T07:07:46.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.810+0000 7fdf31ffb700 1 -- 172.21.15.63:0/1854291519 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf2c00be90 con 0x7fdf34076ad0 2024-03-22T07:07:46.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.810+0000 7fdf31ffb700 1 -- 172.21.15.63:0/1854291519 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf2c0103c0 con 0x7fdf34076ad0 2024-03-22T07:07:46.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.811+0000 7fdf3944f700 1 -- 172.21.15.63:0/1854291519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf34076ad0 msgr2=0x7fdf34105410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:46.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.811+0000 7fdf3944f700 1 --2- 172.21.15.63:0/1854291519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf34076ad0 0x7fdf34105410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0098e0 tx=0x7fdf2c009250 comp rx=0 tx=0).stop 2024-03-22T07:07:46.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.811+0000 7fdf3944f700 1 -- 172.21.15.63:0/1854291519 shutdown_connections 2024-03-22T07:07:46.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.811+0000 7fdf3944f700 1 --2- 172.21.15.63:0/1854291519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf34076ad0 0x7fdf34105410 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:46.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.811+0000 7fdf3944f700 1 -- 172.21.15.63:0/1854291519 >> 172.21.15.63:0/1854291519 conn(0x7fdf340fcde0 msgr2=0x7fdf340ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:46.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.811+0000 7fdf3944f700 1 -- 172.21.15.63:0/1854291519 shutdown_connections 2024-03-22T07:07:46.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.811+0000 7fdf3944f700 1 -- 172.21.15.63:0/1854291519 wait complete. 2024-03-22T07:07:46.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.811+0000 7fdf3944f700 1 Processor -- start 2024-03-22T07:07:46.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.811+0000 7fdf3944f700 1 -- start start 2024-03-22T07:07:46.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.812+0000 7fdf3944f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf34076ad0 0x7fdf34111580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:46.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.812+0000 7fdf3944f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf34111ac0 con 0x7fdf34076ad0 2024-03-22T07:07:46.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.812+0000 7fdf32ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf34076ad0 0x7fdf34111580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:46.226 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.812+0000 7fdf32ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf34076ad0 0x7fdf34111580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49614/0 (socket says 172.21.15.63:49614) 2024-03-22T07:07:46.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.812+0000 7fdf32ffd700 1 -- 172.21.15.63:0/314181018 learned_addr learned my addr 172.21.15.63:0/314181018 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:46.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.812+0000 7fdf32ffd700 1 -- 172.21.15.63:0/314181018 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c0094f0 con 0x7fdf34076ad0 2024-03-22T07:07:46.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.812+0000 7fdf32ffd700 1 --2- 172.21.15.63:0/314181018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf34076ad0 0x7fdf34111580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c006d40 tx=0x7fdf2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:46.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.812+0000 7fdf2b7fe700 1 -- 172.21.15.63:0/314181018 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf2c010840 con 0x7fdf34076ad0 2024-03-22T07:07:46.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.812+0000 7fdf2b7fe700 1 -- 172.21.15.63:0/314181018 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf2c010e70 con 0x7fdf34076ad0 2024-03-22T07:07:46.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.812+0000 7fdf3944f700 1 -- 172.21.15.63:0/314181018 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf34111cc0 con 0x7fdf34076ad0 2024-03-22T07:07:46.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.812+0000 7fdf2b7fe700 1 -- 172.21.15.63:0/314181018 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdf2c024ea0 con 0x7fdf34076ad0 2024-03-22T07:07:46.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.812+0000 7fdf3944f700 1 -- 172.21.15.63:0/314181018 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf34112830 con 0x7fdf34076ad0 2024-03-22T07:07:46.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.813+0000 7fdf2b7fe700 1 -- 172.21.15.63:0/314181018 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fdf2c024780 con 0x7fdf34076ad0 2024-03-22T07:07:46.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.813+0000 7fdf2b7fe700 1 --2- 172.21.15.63:0/314181018 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fdf20036790 0x7fdf20038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:46.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.813+0000 7fdf2b7fe700 1 -- 172.21.15.63:0/314181018 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fdf2c04a800 con 0x7fdf34076ad0 2024-03-22T07:07:46.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.813+0000 7fdf3944f700 1 -- 172.21.15.63:0/314181018 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3405f5f0 con 0x7fdf34076ad0 2024-03-22T07:07:46.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.814+0000 7fdf327fc700 1 --2- 172.21.15.63:0/314181018 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fdf20036790 0x7fdf20038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:46.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.814+0000 7fdf327fc700 1 --2- 172.21.15.63:0/314181018 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fdf20036790 0x7fdf20038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009990 tx=0x7fdf1c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:46.227 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:44.818+0000 7fdf2b7fe700 1 -- 172.21.15.63:0/314181018 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdf2c02c930 con 0x7fdf34076ad0 2024-03-22T07:07:46.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:45.031+0000 7fdf3944f700 1 -- 172.21.15.63:0/314181018 --> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fdf3404bed0 con 0x7fdf20036790 2024-03-22T07:07:46.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:45.285+0000 7fdf2b7fe700 1 -- 172.21.15.63:0/314181018 <== mgr.14162 v2:172.21.15.63:6800/2296442540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fdf3404bed0 con 0x7fdf20036790 2024-03-22T07:07:46.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:45.287+0000 7fdf3944f700 1 -- 172.21.15.63:0/314181018 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fdf20036790 msgr2=0x7fdf20038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:46.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:45.287+0000 7fdf3944f700 1 --2- 172.21.15.63:0/314181018 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fdf20036790 0x7fdf20038c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009990 tx=0x7fdf1c006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:46.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:45.287+0000 7fdf3944f700 1 -- 172.21.15.63:0/314181018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf34076ad0 msgr2=0x7fdf34111580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:46.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:45.287+0000 7fdf3944f700 1 --2- 172.21.15.63:0/314181018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf34076ad0 0x7fdf34111580 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c006d40 tx=0x7fdf2c009250 comp rx=0 tx=0).stop 2024-03-22T07:07:46.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:45.287+0000 7fdf3944f700 1 -- 172.21.15.63:0/314181018 shutdown_connections 2024-03-22T07:07:46.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:45.287+0000 7fdf3944f700 1 --2- 172.21.15.63:0/314181018 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fdf20036790 0x7fdf20038c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:46.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:45.287+0000 7fdf3944f700 1 --2- 172.21.15.63:0/314181018 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf34076ad0 0x7fdf34111580 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:46.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:45.287+0000 7fdf3944f700 1 -- 172.21.15.63:0/314181018 >> 172.21.15.63:0/314181018 conn(0x7fdf340fcde0 msgr2=0x7fdf340fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:46.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:45.287+0000 7fdf3944f700 1 -- 172.21.15.63:0/314181018 shutdown_connections 2024-03-22T07:07:46.228 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:45.288+0000 7fdf3944f700 1 -- 172.21.15.63:0/314181018 wait complete. 2024-03-22T07:07:46.228 INFO:teuthology.orchestra.run.smithi063.stdout:Fetching dashboard port number... 2024-03-22T07:07:46.346 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:46 smithi063 ceph-mon[29200]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:46.347 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:46 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:47.398 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:47 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2277172201' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-22T07:07:47.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stdout 8443 2024-03-22T07:07:47.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.643+0000 7faa4ed33700 1 Processor -- start 2024-03-22T07:07:47.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.643+0000 7faa4ed33700 1 -- start start 2024-03-22T07:07:47.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.644+0000 7faa4ed33700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa48101620 0x7faa48101a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:47.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.644+0000 7faa4ed33700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa48101f70 con 0x7faa48101620 2024-03-22T07:07:47.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.644+0000 7faa4cacf700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa48101620 0x7faa48101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:47.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.644+0000 7faa4cacf700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa48101620 0x7faa48101a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49622/0 (socket says 172.21.15.63:49622) 2024-03-22T07:07:47.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.644+0000 7faa4cacf700 1 -- 172.21.15.63:0/3019087487 learned_addr learned my addr 172.21.15.63:0/3019087487 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:47.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.644+0000 7faa4cacf700 1 -- 172.21.15.63:0/3019087487 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa481020b0 con 0x7faa48101620 2024-03-22T07:07:47.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.644+0000 7faa4cacf700 1 --2- 172.21.15.63:0/3019087487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa48101620 0x7faa48101a30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faa380098e0 tx=0x7faa38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c4b47ee93cf6eb server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:47.408 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.644+0000 7faa477fe700 1 -- 172.21.15.63:0/3019087487 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa3800b860 con 0x7faa48101620 2024-03-22T07:07:47.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.644+0000 7faa477fe700 1 -- 172.21.15.63:0/3019087487 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa3800be90 con 0x7faa48101620 2024-03-22T07:07:47.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.644+0000 7faa477fe700 1 -- 172.21.15.63:0/3019087487 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa380103c0 con 0x7faa48101620 2024-03-22T07:07:47.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.645+0000 7faa4ed33700 1 -- 172.21.15.63:0/3019087487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa48101620 msgr2=0x7faa48101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:47.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.645+0000 7faa4ed33700 1 --2- 172.21.15.63:0/3019087487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa48101620 0x7faa48101a30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faa380098e0 tx=0x7faa38009250 comp rx=0 tx=0).stop 2024-03-22T07:07:47.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.645+0000 7faa4ed33700 1 -- 172.21.15.63:0/3019087487 shutdown_connections 2024-03-22T07:07:47.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.645+0000 7faa4ed33700 1 --2- 172.21.15.63:0/3019087487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa48101620 0x7faa48101a30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:47.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.645+0000 7faa4ed33700 1 -- 172.21.15.63:0/3019087487 >> 172.21.15.63:0/3019087487 conn(0x7faa480fcdc0 msgr2=0x7faa480ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:47.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.645+0000 7faa4ed33700 1 -- 172.21.15.63:0/3019087487 shutdown_connections 2024-03-22T07:07:47.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.645+0000 7faa4ed33700 1 -- 172.21.15.63:0/3019087487 wait complete. 2024-03-22T07:07:47.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.646+0000 7faa4ed33700 1 Processor -- start 2024-03-22T07:07:47.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.646+0000 7faa4ed33700 1 -- start start 2024-03-22T07:07:47.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.646+0000 7faa4ed33700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa48101620 0x7faa48111590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:47.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.646+0000 7faa4ed33700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa48111ad0 con 0x7faa48101620 2024-03-22T07:07:47.409 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.646+0000 7faa4cacf700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa48101620 0x7faa48111590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:47.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.646+0000 7faa4cacf700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa48101620 0x7faa48111590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49634/0 (socket says 172.21.15.63:49634) 2024-03-22T07:07:47.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.646+0000 7faa4cacf700 1 -- 172.21.15.63:0/2277172201 learned_addr learned my addr 172.21.15.63:0/2277172201 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:47.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.646+0000 7faa4cacf700 1 -- 172.21.15.63:0/2277172201 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa380094f0 con 0x7faa48101620 2024-03-22T07:07:47.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.647+0000 7faa4cacf700 1 --2- 172.21.15.63:0/2277172201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa48101620 0x7faa48111590 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faa380090b0 tx=0x7faa38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:47.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.647+0000 7faa457fa700 1 -- 172.21.15.63:0/2277172201 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa38010840 con 0x7faa48101620 2024-03-22T07:07:47.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.647+0000 7faa457fa700 1 -- 172.21.15.63:0/2277172201 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa38010e70 con 0x7faa48101620 2024-03-22T07:07:47.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.647+0000 7faa4ed33700 1 -- 172.21.15.63:0/2277172201 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa48111cd0 con 0x7faa48101620 2024-03-22T07:07:47.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.647+0000 7faa457fa700 1 -- 172.21.15.63:0/2277172201 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faa38024e40 con 0x7faa48101620 2024-03-22T07:07:47.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.647+0000 7faa4ed33700 1 -- 172.21.15.63:0/2277172201 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa48112840 con 0x7faa48101620 2024-03-22T07:07:47.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.648+0000 7faa457fa700 1 -- 172.21.15.63:0/2277172201 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7faa3801a530 con 0x7faa48101620 2024-03-22T07:07:47.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.648+0000 7faa4ed33700 1 -- 172.21.15.63:0/2277172201 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4805f5f0 con 0x7faa48101620 2024-03-22T07:07:47.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.648+0000 7faa457fa700 1 --2- 172.21.15.63:0/2277172201 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7faa34036790 0x7faa34038c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:47.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.648+0000 7faa457fa700 1 -- 172.21.15.63:0/2277172201 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7faa3804aa60 con 0x7faa48101620 2024-03-22T07:07:47.410 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.648+0000 7faa47fff700 1 --2- 172.21.15.63:0/2277172201 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7faa34036790 0x7faa34038c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:47.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.648+0000 7faa47fff700 1 --2- 172.21.15.63:0/2277172201 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7faa34036790 0x7faa34038c40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faa30009990 tx=0x7faa30006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:47.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.652+0000 7faa457fa700 1 -- 172.21.15.63:0/2277172201 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7faa3802e440 con 0x7faa48101620 2024-03-22T07:07:47.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.844+0000 7faa4ed33700 1 -- 172.21.15.63:0/2277172201 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7faa4804bed0 con 0x7faa48101620 2024-03-22T07:07:47.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.844+0000 7faa457fa700 1 -- 172.21.15.63:0/2277172201 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7faa38023b40 con 0x7faa48101620 2024-03-22T07:07:47.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.846+0000 7faa4ed33700 1 -- 172.21.15.63:0/2277172201 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7faa34036790 msgr2=0x7faa34038c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:47.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.846+0000 7faa4ed33700 1 --2- 172.21.15.63:0/2277172201 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7faa34036790 0x7faa34038c40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faa30009990 tx=0x7faa30006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:47.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.846+0000 7faa4ed33700 1 -- 172.21.15.63:0/2277172201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa48101620 msgr2=0x7faa48111590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:47.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.846+0000 7faa4ed33700 1 --2- 172.21.15.63:0/2277172201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa48101620 0x7faa48111590 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faa380090b0 tx=0x7faa38009250 comp rx=0 tx=0).stop 2024-03-22T07:07:47.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.846+0000 7faa4ed33700 1 -- 172.21.15.63:0/2277172201 shutdown_connections 2024-03-22T07:07:47.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.846+0000 7faa4ed33700 1 --2- 172.21.15.63:0/2277172201 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7faa34036790 0x7faa34038c40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:47.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.846+0000 7faa4ed33700 1 --2- 172.21.15.63:0/2277172201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faa48101620 0x7faa48111590 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:47.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.846+0000 7faa4ed33700 1 -- 172.21.15.63:0/2277172201 >> 172.21.15.63:0/2277172201 conn(0x7faa480fcdc0 msgr2=0x7faa480fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:47.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.847+0000 7faa4ed33700 1 -- 172.21.15.63:0/2277172201 shutdown_connections 2024-03-22T07:07:47.411 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:46.847+0000 7faa4ed33700 1 -- 172.21.15.63:0/2277172201 wait complete. 2024-03-22T07:07:47.412 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-22T07:07:47.412 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout disabled 2024-03-22T07:07:47.419 INFO:teuthology.orchestra.run.smithi063.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-22T07:07:47.419 INFO:teuthology.orchestra.run.smithi063.stdout:systemctl: stdout inactive 2024-03-22T07:07:47.419 INFO:teuthology.orchestra.run.smithi063.stdout:firewalld.service is not enabled 2024-03-22T07:07:47.419 INFO:teuthology.orchestra.run.smithi063.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-22T07:07:47.421 INFO:teuthology.orchestra.run.smithi063.stdout:Ceph Dashboard is now available at: 2024-03-22T07:07:47.421 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:07:47.421 INFO:teuthology.orchestra.run.smithi063.stdout: URL: https://smithi063.front.sepia.ceph.com:8443/ 2024-03-22T07:07:47.421 INFO:teuthology.orchestra.run.smithi063.stdout: User: admin 2024-03-22T07:07:47.421 INFO:teuthology.orchestra.run.smithi063.stdout: Password: rt22s3ff4n 2024-03-22T07:07:47.421 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:07:47.421 INFO:teuthology.orchestra.run.smithi063.stdout:Saving cluster configuration to /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config directory 2024-03-22T07:07:47.421 INFO:teuthology.orchestra.run.smithi063.stdout:Enabling autotune for osd_memory_target 2024-03-22T07:07:48.462 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.800+0000 7fb789be8700 1 Processor -- start 2024-03-22T07:07:48.462 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.801+0000 7fb789be8700 1 -- start start 2024-03-22T07:07:48.462 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.801+0000 7fb789be8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb784102770 0x7fb784102b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:48.462 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.801+0000 7fb789be8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7841030c0 con 0x7fb784102770 2024-03-22T07:07:48.462 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.801+0000 7fb7837fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb784102770 0x7fb784102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:48.462 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.801+0000 7fb7837fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb784102770 0x7fb784102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49648/0 (socket says 172.21.15.63:49648) 2024-03-22T07:07:48.462 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.801+0000 7fb7837fe700 1 -- 172.21.15.63:0/1533459030 learned_addr learned my addr 172.21.15.63:0/1533459030 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:48.462 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.801+0000 7fb7837fe700 1 -- 172.21.15.63:0/1533459030 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb784103200 con 0x7fb784102770 2024-03-22T07:07:48.462 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.801+0000 7fb7837fe700 1 --2- 172.21.15.63:0/1533459030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb784102770 0x7fb784102b80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0098e0 tx=0x7fb77c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33be714fe9096980 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:48.463 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.801+0000 7fb7827fc700 1 -- 172.21.15.63:0/1533459030 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb77c00b860 con 0x7fb784102770 2024-03-22T07:07:48.463 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.802+0000 7fb7827fc700 1 -- 172.21.15.63:0/1533459030 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb77c00be90 con 0x7fb784102770 2024-03-22T07:07:48.463 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.802+0000 7fb7827fc700 1 -- 172.21.15.63:0/1533459030 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb77c0103c0 con 0x7fb784102770 2024-03-22T07:07:48.463 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.802+0000 7fb789be8700 1 -- 172.21.15.63:0/1533459030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb784102770 msgr2=0x7fb784102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:48.463 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.802+0000 7fb789be8700 1 --2- 172.21.15.63:0/1533459030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb784102770 0x7fb784102b80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0098e0 tx=0x7fb77c009250 comp rx=0 tx=0).stop 2024-03-22T07:07:48.463 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.802+0000 7fb789be8700 1 -- 172.21.15.63:0/1533459030 shutdown_connections 2024-03-22T07:07:48.463 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.802+0000 7fb789be8700 1 --2- 172.21.15.63:0/1533459030 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb784102770 0x7fb784102b80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:48.463 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.802+0000 7fb789be8700 1 -- 172.21.15.63:0/1533459030 >> 172.21.15.63:0/1533459030 conn(0x7fb7840fde90 msgr2=0x7fb784100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:48.463 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.802+0000 7fb789be8700 1 -- 172.21.15.63:0/1533459030 shutdown_connections 2024-03-22T07:07:48.463 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.802+0000 7fb789be8700 1 -- 172.21.15.63:0/1533459030 wait complete. 2024-03-22T07:07:48.463 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.803+0000 7fb789be8700 1 Processor -- start 2024-03-22T07:07:48.463 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.803+0000 7fb789be8700 1 -- start start 2024-03-22T07:07:48.463 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.803+0000 7fb789be8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb784102770 0x7fb7841115d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:48.464 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.803+0000 7fb789be8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb784111b10 con 0x7fb784102770 2024-03-22T07:07:48.464 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.803+0000 7fb7837fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb784102770 0x7fb7841115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:48.464 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.803+0000 7fb7837fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb784102770 0x7fb7841115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49654/0 (socket says 172.21.15.63:49654) 2024-03-22T07:07:48.464 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.803+0000 7fb7837fe700 1 -- 172.21.15.63:0/2510874727 learned_addr learned my addr 172.21.15.63:0/2510874727 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:48.464 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.803+0000 7fb7837fe700 1 -- 172.21.15.63:0/2510874727 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77c0094f0 con 0x7fb784102770 2024-03-22T07:07:48.464 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.804+0000 7fb7837fe700 1 --2- 172.21.15.63:0/2510874727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb784102770 0x7fb7841115d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb77c011680 tx=0x7fb77c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:48.464 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.804+0000 7fb77bfff700 1 -- 172.21.15.63:0/2510874727 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb77c010890 con 0x7fb784102770 2024-03-22T07:07:48.464 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.804+0000 7fb77bfff700 1 -- 172.21.15.63:0/2510874727 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb77c010ec0 con 0x7fb784102770 2024-03-22T07:07:48.464 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.804+0000 7fb789be8700 1 -- 172.21.15.63:0/2510874727 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb784111d10 con 0x7fb784102770 2024-03-22T07:07:48.464 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.804+0000 7fb77bfff700 1 -- 172.21.15.63:0/2510874727 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb77c0243f0 con 0x7fb784102770 2024-03-22T07:07:48.464 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.804+0000 7fb789be8700 1 -- 172.21.15.63:0/2510874727 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb784112880 con 0x7fb784102770 2024-03-22T07:07:48.464 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.805+0000 7fb77bfff700 1 -- 172.21.15.63:0/2510874727 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 12) v1 ==== 43381+0+0 (secure 0 0 0) 0x7fb77c02d3f0 con 0x7fb784102770 2024-03-22T07:07:48.464 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.805+0000 7fb77bfff700 1 --2- 172.21.15.63:0/2510874727 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fb770036830 0x7fb770038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:48.465 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.805+0000 7fb77bfff700 1 -- 172.21.15.63:0/2510874727 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb77c04a570 con 0x7fb784102770 2024-03-22T07:07:48.465 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.805+0000 7fb789be8700 1 -- 172.21.15.63:0/2510874727 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb78405f5f0 con 0x7fb784102770 2024-03-22T07:07:48.465 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.805+0000 7fb782ffd700 1 --2- 172.21.15.63:0/2510874727 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fb770036830 0x7fb770038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:48.465 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.805+0000 7fb782ffd700 1 --2- 172.21.15.63:0/2510874727 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fb770036830 0x7fb770038ce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009990 tx=0x7fb76c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:48.465 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:47.810+0000 7fb77bfff700 1 -- 172.21.15.63:0/2510874727 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb77c02b8e0 con 0x7fb784102770 2024-03-22T07:07:48.465 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.005+0000 7fb789be8700 1 -- 172.21.15.63:0/2510874727 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fb78404bed0 con 0x7fb784102770 2024-03-22T07:07:48.465 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.006+0000 7fb77bfff700 1 -- 172.21.15.63:0/2510874727 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fb77c02c350 con 0x7fb784102770 2024-03-22T07:07:48.465 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.008+0000 7fb789be8700 1 -- 172.21.15.63:0/2510874727 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fb770036830 msgr2=0x7fb770038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:48.465 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.008+0000 7fb789be8700 1 --2- 172.21.15.63:0/2510874727 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fb770036830 0x7fb770038ce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009990 tx=0x7fb76c006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:48.465 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.008+0000 7fb789be8700 1 -- 172.21.15.63:0/2510874727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb784102770 msgr2=0x7fb7841115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:48.465 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.008+0000 7fb789be8700 1 --2- 172.21.15.63:0/2510874727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb784102770 0x7fb7841115d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb77c011680 tx=0x7fb77c009250 comp rx=0 tx=0).stop 2024-03-22T07:07:48.465 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.008+0000 7fb789be8700 1 -- 172.21.15.63:0/2510874727 shutdown_connections 2024-03-22T07:07:48.466 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.008+0000 7fb789be8700 1 --2- 172.21.15.63:0/2510874727 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fb770036830 0x7fb770038ce0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:48.466 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.008+0000 7fb789be8700 1 --2- 172.21.15.63:0/2510874727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb784102770 0x7fb7841115d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:48.466 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.008+0000 7fb789be8700 1 -- 172.21.15.63:0/2510874727 >> 172.21.15.63:0/2510874727 conn(0x7fb7840fde90 msgr2=0x7fb7840feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:48.466 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.008+0000 7fb789be8700 1 -- 172.21.15.63:0/2510874727 shutdown_connections 2024-03-22T07:07:48.466 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.008+0000 7fb789be8700 1 -- 172.21.15.63:0/2510874727 wait complete. 2024-03-22T07:07:49.567 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:49 smithi063 ceph-mon[29200]: mgrmap e13: smithi063.mnxayh(active, since 7s) 2024-03-22T07:07:49.568 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:49 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/868581690' entity='client.admin' 2024-03-22T07:07:49.784 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.897+0000 7fb2bc7e2700 1 Processor -- start 2024-03-22T07:07:49.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.897+0000 7fb2bc7e2700 1 -- start start 2024-03-22T07:07:49.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.898+0000 7fb2bc7e2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b4102740 0x7fb2b4102b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:49.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.898+0000 7fb2bc7e2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b4103090 con 0x7fb2b4102740 2024-03-22T07:07:49.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.898+0000 7fb2ba57e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b4102740 0x7fb2b4102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:49.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.898+0000 7fb2ba57e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b4102740 0x7fb2b4102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49666/0 (socket says 172.21.15.63:49666) 2024-03-22T07:07:49.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.898+0000 7fb2ba57e700 1 -- 172.21.15.63:0/1854438256 learned_addr learned my addr 172.21.15.63:0/1854438256 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:49.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.898+0000 7fb2ba57e700 1 -- 172.21.15.63:0/1854438256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b41031d0 con 0x7fb2b4102740 2024-03-22T07:07:49.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.898+0000 7fb2ba57e700 1 --2- 172.21.15.63:0/1854438256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b4102740 0x7fb2b4102b50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00098e0 tx=0x7fb2b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303f613b6114427d server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:49.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.899+0000 7fb2b957c700 1 -- 172.21.15.63:0/1854438256 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2b000b860 con 0x7fb2b4102740 2024-03-22T07:07:49.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.899+0000 7fb2b957c700 1 -- 172.21.15.63:0/1854438256 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2b000be90 con 0x7fb2b4102740 2024-03-22T07:07:49.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.899+0000 7fb2b957c700 1 -- 172.21.15.63:0/1854438256 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2b00103c0 con 0x7fb2b4102740 2024-03-22T07:07:49.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.899+0000 7fb2bc7e2700 1 -- 172.21.15.63:0/1854438256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b4102740 msgr2=0x7fb2b4102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:49.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.899+0000 7fb2bc7e2700 1 --2- 172.21.15.63:0/1854438256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b4102740 0x7fb2b4102b50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00098e0 tx=0x7fb2b0009250 comp rx=0 tx=0).stop 2024-03-22T07:07:49.785 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.899+0000 7fb2bc7e2700 1 -- 172.21.15.63:0/1854438256 shutdown_connections 2024-03-22T07:07:49.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.899+0000 7fb2bc7e2700 1 --2- 172.21.15.63:0/1854438256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b4102740 0x7fb2b4102b50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:49.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.899+0000 7fb2bc7e2700 1 -- 172.21.15.63:0/1854438256 >> 172.21.15.63:0/1854438256 conn(0x7fb2b40fdea0 msgr2=0x7fb2b41002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:49.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.899+0000 7fb2bc7e2700 1 -- 172.21.15.63:0/1854438256 shutdown_connections 2024-03-22T07:07:49.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.899+0000 7fb2bc7e2700 1 -- 172.21.15.63:0/1854438256 wait complete. 2024-03-22T07:07:49.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.900+0000 7fb2bc7e2700 1 Processor -- start 2024-03-22T07:07:49.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.900+0000 7fb2bc7e2700 1 -- start start 2024-03-22T07:07:49.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.900+0000 7fb2bc7e2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b4102740 0x7fb2b4078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:49.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.900+0000 7fb2bc7e2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b4079110 con 0x7fb2b4102740 2024-03-22T07:07:49.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.900+0000 7fb2ba57e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b4102740 0x7fb2b4078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:49.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.900+0000 7fb2ba57e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b4102740 0x7fb2b4078bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49676/0 (socket says 172.21.15.63:49676) 2024-03-22T07:07:49.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.900+0000 7fb2ba57e700 1 -- 172.21.15.63:0/868581690 learned_addr learned my addr 172.21.15.63:0/868581690 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:49.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.900+0000 7fb2ba57e700 1 -- 172.21.15.63:0/868581690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b00094f0 con 0x7fb2b4102740 2024-03-22T07:07:49.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.900+0000 7fb2ba57e700 1 --2- 172.21.15.63:0/868581690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b4102740 0x7fb2b4078bd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0011c10 tx=0x7fb2b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:49.786 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.901+0000 7fb2aaffd700 1 -- 172.21.15.63:0/868581690 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2b0010890 con 0x7fb2b4102740 2024-03-22T07:07:49.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.901+0000 7fb2aaffd700 1 -- 172.21.15.63:0/868581690 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2b0010ec0 con 0x7fb2b4102740 2024-03-22T07:07:49.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.901+0000 7fb2bc7e2700 1 -- 172.21.15.63:0/868581690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b4077180 con 0x7fb2b4102740 2024-03-22T07:07:49.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.901+0000 7fb2aaffd700 1 -- 172.21.15.63:0/868581690 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb2b00243f0 con 0x7fb2b4102740 2024-03-22T07:07:49.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.901+0000 7fb2bc7e2700 1 -- 172.21.15.63:0/868581690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b4077670 con 0x7fb2b4102740 2024-03-22T07:07:49.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.901+0000 7fb2aaffd700 1 -- 172.21.15.63:0/868581690 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fb2b002d3f0 con 0x7fb2b4102740 2024-03-22T07:07:49.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.902+0000 7fb2bc7e2700 1 -- 172.21.15.63:0/868581690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b404bed0 con 0x7fb2b4102740 2024-03-22T07:07:49.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.902+0000 7fb2aaffd700 1 --2- 172.21.15.63:0/868581690 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fb2a0036880 0x7fb2a0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:49.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.902+0000 7fb2aaffd700 1 -- 172.21.15.63:0/868581690 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb2b004a560 con 0x7fb2b4102740 2024-03-22T07:07:49.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.902+0000 7fb2b9d7d700 1 --2- 172.21.15.63:0/868581690 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fb2a0036880 0x7fb2a0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:49.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.902+0000 7fb2b9d7d700 1 --2- 172.21.15.63:0/868581690 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fb2a0036880 0x7fb2a0038d30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009990 tx=0x7fb2a4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:49.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:48.906+0000 7fb2aaffd700 1 -- 172.21.15.63:0/868581690 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb2b0018d30 con 0x7fb2b4102740 2024-03-22T07:07:49.787 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:49.209+0000 7fb2bc7e2700 1 -- 172.21.15.63:0/868581690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fb2b4077af0 con 0x7fb2b4102740 2024-03-22T07:07:49.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:49.212+0000 7fb2aaffd700 1 -- 172.21.15.63:0/868581690 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7fb2b0018b50 con 0x7fb2b4102740 2024-03-22T07:07:49.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:49.214+0000 7fb2bc7e2700 1 -- 172.21.15.63:0/868581690 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fb2a0036880 msgr2=0x7fb2a0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:49.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:49.214+0000 7fb2bc7e2700 1 --2- 172.21.15.63:0/868581690 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fb2a0036880 0x7fb2a0038d30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009990 tx=0x7fb2a4006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:49.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:49.214+0000 7fb2bc7e2700 1 -- 172.21.15.63:0/868581690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b4102740 msgr2=0x7fb2b4078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:49.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:49.214+0000 7fb2bc7e2700 1 --2- 172.21.15.63:0/868581690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b4102740 0x7fb2b4078bd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0011c10 tx=0x7fb2b0009250 comp rx=0 tx=0).stop 2024-03-22T07:07:49.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:49.214+0000 7fb2bc7e2700 1 -- 172.21.15.63:0/868581690 shutdown_connections 2024-03-22T07:07:49.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:49.214+0000 7fb2bc7e2700 1 --2- 172.21.15.63:0/868581690 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fb2a0036880 0x7fb2a0038d30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:49.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:49.214+0000 7fb2bc7e2700 1 --2- 172.21.15.63:0/868581690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb2b4102740 0x7fb2b4078bd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:49.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:49.214+0000 7fb2bc7e2700 1 -- 172.21.15.63:0/868581690 >> 172.21.15.63:0/868581690 conn(0x7fb2b40fdea0 msgr2=0x7fb2b40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:49.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:49.214+0000 7fb2bc7e2700 1 -- 172.21.15.63:0/868581690 shutdown_connections 2024-03-22T07:07:49.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr 2024-03-22T07:07:49.214+0000 7fb2bc7e2700 1 -- 172.21.15.63:0/868581690 wait complete. 2024-03-22T07:07:49.788 INFO:teuthology.orchestra.run.smithi063.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-22T07:07:49.788 INFO:teuthology.orchestra.run.smithi063.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-22T07:07:49.788 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:07:49.788 INFO:teuthology.orchestra.run.smithi063.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-22T07:07:49.788 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:07:49.789 INFO:teuthology.orchestra.run.smithi063.stdout:Or, if you are only running a single cluster on this host: 2024-03-22T07:07:49.789 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:07:49.789 INFO:teuthology.orchestra.run.smithi063.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-22T07:07:49.789 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:07:49.789 INFO:teuthology.orchestra.run.smithi063.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-22T07:07:49.789 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:07:49.789 INFO:teuthology.orchestra.run.smithi063.stdout: ceph telemetry on 2024-03-22T07:07:49.789 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:07:49.789 INFO:teuthology.orchestra.run.smithi063.stdout:For more information see: 2024-03-22T07:07:49.789 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:07:49.789 INFO:teuthology.orchestra.run.smithi063.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-03-22T07:07:49.789 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:07:49.789 INFO:teuthology.orchestra.run.smithi063.stdout:Bootstrap complete. 2024-03-22T07:07:49.820 INFO:tasks.cephadm:Fetching config... 2024-03-22T07:07:49.821 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T07:07:49.821 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-22T07:07:49.877 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-22T07:07:49.877 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T07:07:49.877 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-22T07:07:49.932 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-22T07:07:49.932 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T07:07:49.932 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/keyring of=/dev/stdout 2024-03-22T07:07:49.998 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-22T07:07:49.998 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T07:07:49.998 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-22T07:07:50.052 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-22T07:07:50.052 DEBUG:teuthology.orchestra.run.smithi063:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSXTVRL4AK7J0O4BEHH/BCPSmyKTf294IiJuyEVljae8krjCePgETJQ3GFUSUudhKV2ePgmI1IzQ/ubAlK6mFZGyRh+B9ClXZCFnrospYzvA9o/2slnDRjWuHj7lEfm2rx58BC93R6VMHW7jnYGQRvxBmbO84cklb372J9TnvVXIhY/nCiTptgy+1Wa8KGjzNSXxFxa88mSQgV3jQW/88DS+a4u0MSofXZYVG0c8FC7Hjl2aDrECJVlY654W7+z2Q5dKwKuDl/0Wgl6l6KrF6p2HDxhGnlj/cR21OXefutGqzKcDltn2sVln8XOZxZmG6vUt7o/0KUZN0s49Jip9S6awIpqVHG2IRNmlVX6V+NtF6FG2GfXVmhQWjI3iw/pjQPFGtgdiSQLCCQg4tZi4EmMSWoVJpUIgEAxF1W+y+YO4llIOIB0/NhZEQOEAqhT1v/QB+nsCV4Q4vODBkGaXSJSgoSyh3xDED7tOSJjIC+A/BPEK0/TAH6zlVWr5iDFUs= ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-22T07:07:50.132 INFO:teuthology.orchestra.run.smithi063.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSXTVRL4AK7J0O4BEHH/BCPSmyKTf294IiJuyEVljae8krjCePgETJQ3GFUSUudhKV2ePgmI1IzQ/ubAlK6mFZGyRh+B9ClXZCFnrospYzvA9o/2slnDRjWuHj7lEfm2rx58BC93R6VMHW7jnYGQRvxBmbO84cklb372J9TnvVXIhY/nCiTptgy+1Wa8KGjzNSXxFxa88mSQgV3jQW/88DS+a4u0MSofXZYVG0c8FC7Hjl2aDrECJVlY654W7+z2Q5dKwKuDl/0Wgl6l6KrF6p2HDxhGnlj/cR21OXefutGqzKcDltn2sVln8XOZxZmG6vUt7o/0KUZN0s49Jip9S6awIpqVHG2IRNmlVX6V+NtF6FG2GfXVmhQWjI3iw/pjQPFGtgdiSQLCCQg4tZi4EmMSWoVJpUIgEAxF1W+y+YO4llIOIB0/NhZEQOEAqhT1v/QB+nsCV4Q4vODBkGaXSJSgoSyh3xDED7tOSJjIC+A/BPEK0/TAH6zlVWr5iDFUs= ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:07:50.146 DEBUG:teuthology.orchestra.run.smithi069:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSXTVRL4AK7J0O4BEHH/BCPSmyKTf294IiJuyEVljae8krjCePgETJQ3GFUSUudhKV2ePgmI1IzQ/ubAlK6mFZGyRh+B9ClXZCFnrospYzvA9o/2slnDRjWuHj7lEfm2rx58BC93R6VMHW7jnYGQRvxBmbO84cklb372J9TnvVXIhY/nCiTptgy+1Wa8KGjzNSXxFxa88mSQgV3jQW/88DS+a4u0MSofXZYVG0c8FC7Hjl2aDrECJVlY654W7+z2Q5dKwKuDl/0Wgl6l6KrF6p2HDxhGnlj/cR21OXefutGqzKcDltn2sVln8XOZxZmG6vUt7o/0KUZN0s49Jip9S6awIpqVHG2IRNmlVX6V+NtF6FG2GfXVmhQWjI3iw/pjQPFGtgdiSQLCCQg4tZi4EmMSWoVJpUIgEAxF1W+y+YO4llIOIB0/NhZEQOEAqhT1v/QB+nsCV4Q4vODBkGaXSJSgoSyh3xDED7tOSJjIC+A/BPEK0/TAH6zlVWr5iDFUs= ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-22T07:07:50.192 INFO:teuthology.orchestra.run.smithi069.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDSXTVRL4AK7J0O4BEHH/BCPSmyKTf294IiJuyEVljae8krjCePgETJQ3GFUSUudhKV2ePgmI1IzQ/ubAlK6mFZGyRh+B9ClXZCFnrospYzvA9o/2slnDRjWuHj7lEfm2rx58BC93R6VMHW7jnYGQRvxBmbO84cklb372J9TnvVXIhY/nCiTptgy+1Wa8KGjzNSXxFxa88mSQgV3jQW/88DS+a4u0MSofXZYVG0c8FC7Hjl2aDrECJVlY654W7+z2Q5dKwKuDl/0Wgl6l6KrF6p2HDxhGnlj/cR21OXefutGqzKcDltn2sVln8XOZxZmG6vUt7o/0KUZN0s49Jip9S6awIpqVHG2IRNmlVX6V+NtF6FG2GfXVmhQWjI3iw/pjQPFGtgdiSQLCCQg4tZi4EmMSWoVJpUIgEAxF1W+y+YO4llIOIB0/NhZEQOEAqhT1v/QB+nsCV4Q4vODBkGaXSJSgoSyh3xDED7tOSJjIC+A/BPEK0/TAH6zlVWr5iDFUs= ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:07:50.207 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-22T07:07:50.465 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:07:52.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.873+0000 7ff8b339a700 1 -- 172.21.15.63:0/4066943181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8ac066980 msgr2=0x7ff8ac0ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:52.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.873+0000 7ff8b339a700 1 --2- 172.21.15.63:0/4066943181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8ac066980 0x7ff8ac0ff6b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff8a80048c0 tx=0x7ff8a8009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:52.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.873+0000 7ff8b339a700 1 -- 172.21.15.63:0/4066943181 shutdown_connections 2024-03-22T07:07:52.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.873+0000 7ff8b339a700 1 --2- 172.21.15.63:0/4066943181 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8ac066980 0x7ff8ac0ff6b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:52.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.873+0000 7ff8b339a700 1 -- 172.21.15.63:0/4066943181 >> 172.21.15.63:0/4066943181 conn(0x7ff8ac0f70c0 msgr2=0x7ff8ac0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:52.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.874+0000 7ff8b339a700 1 -- 172.21.15.63:0/4066943181 shutdown_connections 2024-03-22T07:07:52.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.874+0000 7ff8b339a700 1 -- 172.21.15.63:0/4066943181 wait complete. 2024-03-22T07:07:52.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.874+0000 7ff8b339a700 1 Processor -- start 2024-03-22T07:07:52.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.874+0000 7ff8b339a700 1 -- start start 2024-03-22T07:07:52.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.874+0000 7ff8b339a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8ac066980 0x7ff8ac10d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:52.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.874+0000 7ff8b339a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac10d630 con 0x7ff8ac066980 2024-03-22T07:07:52.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.874+0000 7ff8b1136700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8ac066980 0x7ff8ac10d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:52.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.874+0000 7ff8b1136700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8ac066980 0x7ff8ac10d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49704/0 (socket says 172.21.15.63:49704) 2024-03-22T07:07:52.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.874+0000 7ff8b1136700 1 -- 172.21.15.63:0/2779527829 learned_addr learned my addr 172.21.15.63:0/2779527829 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:52.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.875+0000 7ff8b1136700 1 -- 172.21.15.63:0/2779527829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a8005040 con 0x7ff8ac066980 2024-03-22T07:07:52.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.875+0000 7ff8b1136700 1 --2- 172.21.15.63:0/2779527829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8ac066980 0x7ff8ac10d0f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009680 tx=0x7ff8a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:52.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.875+0000 7ff8a1ffb700 1 -- 172.21.15.63:0/2779527829 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8a800bed0 con 0x7ff8ac066980 2024-03-22T07:07:52.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.875+0000 7ff8b339a700 1 -- 172.21.15.63:0/2779527829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ac10d830 con 0x7ff8ac066980 2024-03-22T07:07:52.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.875+0000 7ff8a1ffb700 1 -- 172.21.15.63:0/2779527829 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8a800b5c0 con 0x7ff8ac066980 2024-03-22T07:07:52.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.875+0000 7ff8a1ffb700 1 -- 172.21.15.63:0/2779527829 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8a801c5b0 con 0x7ff8ac066980 2024-03-22T07:07:52.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.875+0000 7ff8b339a700 1 -- 172.21.15.63:0/2779527829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ac10e3a0 con 0x7ff8ac066980 2024-03-22T07:07:52.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.876+0000 7ff8a1ffb700 1 -- 172.21.15.63:0/2779527829 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7ff8a801c710 con 0x7ff8ac066980 2024-03-22T07:07:52.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.876+0000 7ff8b339a700 1 -- 172.21.15.63:0/2779527829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8ac102f60 con 0x7ff8ac066980 2024-03-22T07:07:52.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.876+0000 7ff8a1ffb700 1 --2- 172.21.15.63:0/2779527829 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7ff898036880 0x7ff898038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:52.877 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.876+0000 7ff8a1ffb700 1 -- 172.21.15.63:0/2779527829 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff8a8049a50 con 0x7ff8ac066980 2024-03-22T07:07:52.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.876+0000 7ff8b0935700 1 --2- 172.21.15.63:0/2779527829 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7ff898036880 0x7ff898038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:52.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.877+0000 7ff8b0935700 1 --2- 172.21.15.63:0/2779527829 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7ff898036880 0x7ff898038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009990 tx=0x7ff89c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:52.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:52.881+0000 7ff8a1ffb700 1 -- 172.21.15.63:0/2779527829 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff8a8025400 con 0x7ff8ac066980 2024-03-22T07:07:53.077 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:53.077+0000 7ff8b339a700 1 -- 172.21.15.63:0/2779527829 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7ff8ac10ee70 con 0x7ff8ac066980 2024-03-22T07:07:53.081 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:53.081+0000 7ff8a1ffb700 1 -- 172.21.15.63:0/2779527829 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7ff8a8011370 con 0x7ff8ac066980 2024-03-22T07:07:53.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:53.084+0000 7ff8b339a700 1 -- 172.21.15.63:0/2779527829 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7ff898036880 msgr2=0x7ff898038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:53.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:53.084+0000 7ff8b339a700 1 --2- 172.21.15.63:0/2779527829 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7ff898036880 0x7ff898038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009990 tx=0x7ff89c006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:53.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:53.084+0000 7ff8b339a700 1 -- 172.21.15.63:0/2779527829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8ac066980 msgr2=0x7ff8ac10d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:53.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:53.084+0000 7ff8b339a700 1 --2- 172.21.15.63:0/2779527829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8ac066980 0x7ff8ac10d0f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8009680 tx=0x7ff8a8009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:53.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:53.084+0000 7ff8b339a700 1 -- 172.21.15.63:0/2779527829 shutdown_connections 2024-03-22T07:07:53.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:53.084+0000 7ff8b339a700 1 --2- 172.21.15.63:0/2779527829 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7ff898036880 0x7ff898038d30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:53.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:53.084+0000 7ff8b339a700 1 --2- 172.21.15.63:0/2779527829 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8ac066980 0x7ff8ac10d0f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:53.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:53.084+0000 7ff8b339a700 1 -- 172.21.15.63:0/2779527829 >> 172.21.15.63:0/2779527829 conn(0x7ff8ac0f70c0 msgr2=0x7ff8ac103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:53.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:53.085+0000 7ff8b339a700 1 -- 172.21.15.63:0/2779527829 shutdown_connections 2024-03-22T07:07:53.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:53.085+0000 7ff8b339a700 1 -- 172.21.15.63:0/2779527829 wait complete. 2024-03-22T07:07:53.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:53 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:53.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:53 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:53.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:53 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd/host:smithi063", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:07:53.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:53 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:53.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:53 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:07:53.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:53 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-22T07:07:53.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:53 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:07:53.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:53 smithi063 ceph-mon[29200]: Deploying daemon crash.smithi063 on smithi063 2024-03-22T07:07:53.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:53 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2779527829' entity='client.admin' 2024-03-22T07:07:53.799 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-22T07:07:53.799 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-22T07:07:54.056 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:07:56.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.287+0000 7f0ed6366700 1 -- 172.21.15.63:0/3133049745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ed0066980 msgr2=0x7f0ed00ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:56.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.287+0000 7f0ed6366700 1 --2- 172.21.15.63:0/3133049745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ed0066980 0x7f0ed00ff6b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00048c0 tx=0x7f0ec0009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:56.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.287+0000 7f0ed6366700 1 -- 172.21.15.63:0/3133049745 shutdown_connections 2024-03-22T07:07:56.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.287+0000 7f0ed6366700 1 --2- 172.21.15.63:0/3133049745 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ed0066980 0x7f0ed00ff6b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:56.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.287+0000 7f0ed6366700 1 -- 172.21.15.63:0/3133049745 >> 172.21.15.63:0/3133049745 conn(0x7f0ed00f70c0 msgr2=0x7f0ed00f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:56.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.287+0000 7f0ed6366700 1 -- 172.21.15.63:0/3133049745 shutdown_connections 2024-03-22T07:07:56.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.287+0000 7f0ed6366700 1 -- 172.21.15.63:0/3133049745 wait complete. 2024-03-22T07:07:56.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.288+0000 7f0ed6366700 1 Processor -- start 2024-03-22T07:07:56.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.288+0000 7f0ed6366700 1 -- start start 2024-03-22T07:07:56.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.288+0000 7f0ed6366700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ed0066980 0x7f0ed010d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:56.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.288+0000 7f0ed6366700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed010d610 con 0x7f0ed0066980 2024-03-22T07:07:56.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.288+0000 7f0ecffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ed0066980 0x7f0ed010d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:56.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.288+0000 7f0ecffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ed0066980 0x7f0ed010d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57282/0 (socket says 172.21.15.63:57282) 2024-03-22T07:07:56.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.288+0000 7f0ecffff700 1 -- 172.21.15.63:0/1625593000 learned_addr learned my addr 172.21.15.63:0/1625593000 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:56.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.288+0000 7f0ecffff700 1 -- 172.21.15.63:0/1625593000 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec0005040 con 0x7f0ed0066980 2024-03-22T07:07:56.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.289+0000 7f0ecffff700 1 --2- 172.21.15.63:0/1625593000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ed0066980 0x7f0ed010d0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000b900 tx=0x7f0ec0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:56.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.289+0000 7f0eccff9700 1 -- 172.21.15.63:0/1625593000 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ec000b5c0 con 0x7f0ed0066980 2024-03-22T07:07:56.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.289+0000 7f0ed6366700 1 -- 172.21.15.63:0/1625593000 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed010d810 con 0x7f0ed0066980 2024-03-22T07:07:56.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.289+0000 7f0eccff9700 1 -- 172.21.15.63:0/1625593000 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ec000be70 con 0x7f0ed0066980 2024-03-22T07:07:56.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.289+0000 7f0eccff9700 1 -- 172.21.15.63:0/1625593000 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ec001c590 con 0x7f0ed0066980 2024-03-22T07:07:56.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.289+0000 7f0ed6366700 1 -- 172.21.15.63:0/1625593000 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed010e380 con 0x7f0ed0066980 2024-03-22T07:07:56.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.290+0000 7f0eccff9700 1 -- 172.21.15.63:0/1625593000 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f0ec0013420 con 0x7f0ed0066980 2024-03-22T07:07:56.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.290+0000 7f0eccff9700 1 --2- 172.21.15.63:0/1625593000 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f0eb8036880 0x7f0eb8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:56.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.290+0000 7f0ed6366700 1 -- 172.21.15.63:0/1625593000 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed0102f60 con 0x7f0ed0066980 2024-03-22T07:07:56.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.290+0000 7f0eccff9700 1 -- 172.21.15.63:0/1625593000 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0ec0049380 con 0x7f0ed0066980 2024-03-22T07:07:56.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.290+0000 7f0ecf7fe700 1 --2- 172.21.15.63:0/1625593000 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f0eb8036880 0x7f0eb8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:56.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.290+0000 7f0ecf7fe700 1 --2- 172.21.15.63:0/1625593000 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f0eb8036880 0x7f0eb8038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009990 tx=0x7f0ec4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:56.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.295+0000 7f0eccff9700 1 -- 172.21.15.63:0/1625593000 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0ec0021030 con 0x7f0ed0066980 2024-03-22T07:07:56.495 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.495+0000 7f0ed6366700 1 -- 172.21.15.63:0/1625593000 --> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ed010e680 con 0x7f0eb8036880 2024-03-22T07:07:56.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.498+0000 7f0eccff9700 1 -- 172.21.15.63:0/1625593000 <== mgr.14162 v2:172.21.15.63:6800/2296442540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0ed010e680 con 0x7f0eb8036880 2024-03-22T07:07:56.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.500+0000 7f0ed6366700 1 -- 172.21.15.63:0/1625593000 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f0eb8036880 msgr2=0x7f0eb8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:56.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.500+0000 7f0ed6366700 1 --2- 172.21.15.63:0/1625593000 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f0eb8036880 0x7f0eb8038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4009990 tx=0x7f0ec4006e40 comp rx=0 tx=0).stop 2024-03-22T07:07:56.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.500+0000 7f0ed6366700 1 -- 172.21.15.63:0/1625593000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ed0066980 msgr2=0x7f0ed010d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:56.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.500+0000 7f0ed6366700 1 --2- 172.21.15.63:0/1625593000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ed0066980 0x7f0ed010d0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000b900 tx=0x7f0ec0009e30 comp rx=0 tx=0).stop 2024-03-22T07:07:56.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.500+0000 7f0ed6366700 1 -- 172.21.15.63:0/1625593000 shutdown_connections 2024-03-22T07:07:56.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.500+0000 7f0ed6366700 1 --2- 172.21.15.63:0/1625593000 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f0eb8036880 0x7f0eb8038d30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:56.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.500+0000 7f0ed6366700 1 --2- 172.21.15.63:0/1625593000 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0ed0066980 0x7f0ed010d0d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:56.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.500+0000 7f0ed6366700 1 -- 172.21.15.63:0/1625593000 >> 172.21.15.63:0/1625593000 conn(0x7f0ed00f70c0 msgr2=0x7f0ed0103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:56.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.500+0000 7f0ed6366700 1 -- 172.21.15.63:0/1625593000 shutdown_connections 2024-03-22T07:07:56.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:56.501+0000 7f0ed6366700 1 -- 172.21.15.63:0/1625593000 wait complete. 2024-03-22T07:07:57.480 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi069 2024-03-22T07:07:57.481 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-22T07:07:57.481 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/etc/ceph/ceph.conf 2024-03-22T07:07:57.499 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-22T07:07:57.499 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:07:57.557 INFO:tasks.cephadm:Adding host smithi069 to orchestrator... 2024-03-22T07:07:57.557 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph orch host add smithi069 2024-03-22T07:07:57.708 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:57 smithi063 ceph-mon[29200]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:07:57.708 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:57 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:58.098 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:07:59.249 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:59 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:59.249 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:59 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:59.249 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:59 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:59.249 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:59 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:07:59.249 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:07:59 smithi063 ceph-mon[29200]: Deploying daemon node-exporter.smithi063 on smithi063 2024-03-22T07:07:59.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.981+0000 7f5703e05700 1 -- 172.21.15.63:0/3089832722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56fc071e20 msgr2=0x7f56fc070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:07:59.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.981+0000 7f5703e05700 1 --2- 172.21.15.63:0/3089832722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56fc071e20 0x7f56fc070330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009c60 tx=0x7f56f4004d10 comp rx=0 tx=0).stop 2024-03-22T07:07:59.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.981+0000 7f5703e05700 1 -- 172.21.15.63:0/3089832722 shutdown_connections 2024-03-22T07:07:59.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.981+0000 7f5703e05700 1 --2- 172.21.15.63:0/3089832722 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56fc071e20 0x7f56fc070330 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:07:59.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.981+0000 7f5703e05700 1 -- 172.21.15.63:0/3089832722 >> 172.21.15.63:0/3089832722 conn(0x7f56fc0f81d0 msgr2=0x7f56fc0fa5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:07:59.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.981+0000 7f5703e05700 1 -- 172.21.15.63:0/3089832722 shutdown_connections 2024-03-22T07:07:59.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.981+0000 7f5703e05700 1 -- 172.21.15.63:0/3089832722 wait complete. 2024-03-22T07:07:59.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.981+0000 7f5703e05700 1 Processor -- start 2024-03-22T07:07:59.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.982+0000 7f5703e05700 1 -- start start 2024-03-22T07:07:59.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.982+0000 7f5703e05700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56fc071e20 0x7f56fc10f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:59.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.982+0000 7f5703e05700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc10f7d0 con 0x7f56fc071e20 2024-03-22T07:07:59.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.982+0000 7f5701ba1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56fc071e20 0x7f56fc10f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:59.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.982+0000 7f5701ba1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56fc071e20 0x7f56fc10f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:57310/0 (socket says 172.21.15.63:57310) 2024-03-22T07:07:59.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.982+0000 7f5701ba1700 1 -- 172.21.15.63:0/524283856 learned_addr learned my addr 172.21.15.63:0/524283856 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:07:59.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.982+0000 7f5701ba1700 1 -- 172.21.15.63:0/524283856 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f4009940 con 0x7f56fc071e20 2024-03-22T07:07:59.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.982+0000 7f5701ba1700 1 --2- 172.21.15.63:0/524283856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56fc071e20 0x7f56fc10f290 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f56f40097b0 tx=0x7f56f4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:59.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.982+0000 7f56fa7fc700 1 -- 172.21.15.63:0/524283856 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f56f4012410 con 0x7f56fc071e20 2024-03-22T07:07:59.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.982+0000 7f56fa7fc700 1 -- 172.21.15.63:0/524283856 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56f4012a40 con 0x7f56fc071e20 2024-03-22T07:07:59.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.982+0000 7f5703e05700 1 -- 172.21.15.63:0/524283856 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56fc10f9d0 con 0x7f56fc071e20 2024-03-22T07:07:59.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.982+0000 7f56fa7fc700 1 -- 172.21.15.63:0/524283856 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f56f401ba20 con 0x7f56fc071e20 2024-03-22T07:07:59.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.982+0000 7f5703e05700 1 -- 172.21.15.63:0/524283856 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56fc110540 con 0x7f56fc071e20 2024-03-22T07:07:59.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.983+0000 7f56fa7fc700 1 -- 172.21.15.63:0/524283856 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f56f4019040 con 0x7f56fc071e20 2024-03-22T07:07:59.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.983+0000 7f56fa7fc700 1 --2- 172.21.15.63:0/524283856 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f56e4036880 0x7f56e4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:07:59.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.983+0000 7f5703e05700 1 -- 172.21.15.63:0/524283856 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56fc105150 con 0x7f56fc071e20 2024-03-22T07:07:59.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.983+0000 7f56fa7fc700 1 -- 172.21.15.63:0/524283856 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f56f4049eb0 con 0x7f56fc071e20 2024-03-22T07:07:59.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.984+0000 7f57013a0700 1 --2- 172.21.15.63:0/524283856 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f56e4036880 0x7f56e4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:07:59.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.984+0000 7f57013a0700 1 --2- 172.21.15.63:0/524283856 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f56e4036880 0x7f56e4038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009990 tx=0x7f56f0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:07:59.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:07:59.988+0000 7f56fa7fc700 1 -- 172.21.15.63:0/524283856 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f56f402b8b0 con 0x7f56fc071e20 2024-03-22T07:08:00.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:00.189+0000 7f5703e05700 1 -- 172.21.15.63:0/524283856 --> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi069", "target": ["mon-mgr", ""]}) v1 -- 0x7f56fc05f690 con 0x7f56e4036880 2024-03-22T07:08:01.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:00 smithi063 ceph-mon[29200]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi069", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:08:02.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:02.373+0000 7f56fa7fc700 1 -- 172.21.15.63:0/524283856 <== mgr.14162 v2:172.21.15.63:6800/2296442540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f56fc05f690 con 0x7f56e4036880 2024-03-22T07:08:02.374 INFO:teuthology.orchestra.run.smithi063.stdout:Added host 'smithi069' with addr '172.21.15.69' 2024-03-22T07:08:02.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:02.375+0000 7f5703e05700 1 -- 172.21.15.63:0/524283856 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f56e4036880 msgr2=0x7f56e4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:02.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:02.375+0000 7f5703e05700 1 --2- 172.21.15.63:0/524283856 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f56e4036880 0x7f56e4038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009990 tx=0x7f56f0006e40 comp rx=0 tx=0).stop 2024-03-22T07:08:02.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:02.375+0000 7f5703e05700 1 -- 172.21.15.63:0/524283856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56fc071e20 msgr2=0x7f56fc10f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:02.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:02.375+0000 7f5703e05700 1 --2- 172.21.15.63:0/524283856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56fc071e20 0x7f56fc10f290 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f56f40097b0 tx=0x7f56f4004d10 comp rx=0 tx=0).stop 2024-03-22T07:08:02.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:02.375+0000 7f5703e05700 1 -- 172.21.15.63:0/524283856 shutdown_connections 2024-03-22T07:08:02.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:02.375+0000 7f5703e05700 1 --2- 172.21.15.63:0/524283856 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f56e4036880 0x7f56e4038d30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:02.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:02.375+0000 7f5703e05700 1 --2- 172.21.15.63:0/524283856 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56fc071e20 0x7f56fc10f290 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:02.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:02.375+0000 7f5703e05700 1 -- 172.21.15.63:0/524283856 >> 172.21.15.63:0/524283856 conn(0x7f56fc0f81d0 msgr2=0x7f56fc105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:02.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:02.375+0000 7f5703e05700 1 -- 172.21.15.63:0/524283856 shutdown_connections 2024-03-22T07:08:02.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:02.375+0000 7f5703e05700 1 -- 172.21.15.63:0/524283856 wait complete. 2024-03-22T07:08:02.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:02 smithi063 ceph-mon[29200]: Deploying cephadm binary to smithi069 2024-03-22T07:08:02.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:02 smithi063 ceph-mon[29200]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:08:02.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:02 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:03.100 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-22T07:08:03.368 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:08:03.633 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:03 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:03.633 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:03 smithi063 ceph-mon[29200]: Added host smithi069 2024-03-22T07:08:03.633 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:03 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:04.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:04 smithi063 ceph-mon[29200]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:08:04.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:04 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:04.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:04 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:04.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:04 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:04.554 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:04 smithi063 ceph-mon[29200]: Deploying daemon alertmanager.smithi063 on smithi063 2024-03-22T07:08:04.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.770+0000 7fd778fe6700 1 -- 172.21.15.63:0/1843968519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7740668e0 msgr2=0x7fd7740ff6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:04.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.770+0000 7fd778fe6700 1 --2- 172.21.15.63:0/1843968519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7740668e0 0x7fd7740ff6c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd7640048c0 tx=0x7fd764009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:04.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.770+0000 7fd778fe6700 1 -- 172.21.15.63:0/1843968519 shutdown_connections 2024-03-22T07:08:04.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.770+0000 7fd778fe6700 1 --2- 172.21.15.63:0/1843968519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7740668e0 0x7fd7740ff6c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:04.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.770+0000 7fd778fe6700 1 -- 172.21.15.63:0/1843968519 >> 172.21.15.63:0/1843968519 conn(0x7fd7740f7090 msgr2=0x7fd7740f94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:04.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.770+0000 7fd778fe6700 1 -- 172.21.15.63:0/1843968519 shutdown_connections 2024-03-22T07:08:04.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.770+0000 7fd778fe6700 1 -- 172.21.15.63:0/1843968519 wait complete. 2024-03-22T07:08:04.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.771+0000 7fd778fe6700 1 Processor -- start 2024-03-22T07:08:04.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.771+0000 7fd778fe6700 1 -- start start 2024-03-22T07:08:04.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.771+0000 7fd778fe6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7740668e0 0x7fd77410d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:04.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.771+0000 7fd778fe6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77410d5e0 con 0x7fd7740668e0 2024-03-22T07:08:04.772 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.771+0000 7fd77259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7740668e0 0x7fd77410d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.771+0000 7fd77259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7740668e0 0x7fd77410d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35274/0 (socket says 172.21.15.63:35274) 2024-03-22T07:08:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.771+0000 7fd77259c700 1 -- 172.21.15.63:0/774039070 learned_addr learned my addr 172.21.15.63:0/774039070 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:08:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.772+0000 7fd77259c700 1 -- 172.21.15.63:0/774039070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd764005040 con 0x7fd7740668e0 2024-03-22T07:08:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.772+0000 7fd77259c700 1 --2- 172.21.15.63:0/774039070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7740668e0 0x7fd77410d0a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd76400b900 tx=0x7fd764009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.772+0000 7fd762ffd700 1 -- 172.21.15.63:0/774039070 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd76400b5c0 con 0x7fd7740668e0 2024-03-22T07:08:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.772+0000 7fd762ffd700 1 -- 172.21.15.63:0/774039070 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd76400be70 con 0x7fd7740668e0 2024-03-22T07:08:04.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.772+0000 7fd778fe6700 1 -- 172.21.15.63:0/774039070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77410d7e0 con 0x7fd7740668e0 2024-03-22T07:08:04.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.772+0000 7fd762ffd700 1 -- 172.21.15.63:0/774039070 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd76401c590 con 0x7fd7740668e0 2024-03-22T07:08:04.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.772+0000 7fd778fe6700 1 -- 172.21.15.63:0/774039070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77410e350 con 0x7fd7740668e0 2024-03-22T07:08:04.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.773+0000 7fd762ffd700 1 -- 172.21.15.63:0/774039070 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fd764013420 con 0x7fd7740668e0 2024-03-22T07:08:04.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.773+0000 7fd762ffd700 1 --2- 172.21.15.63:0/774039070 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fd75c0368d0 0x7fd75c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:04.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.773+0000 7fd778fe6700 1 -- 172.21.15.63:0/774039070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd774102f50 con 0x7fd7740668e0 2024-03-22T07:08:04.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.773+0000 7fd762ffd700 1 -- 172.21.15.63:0/774039070 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd764049300 con 0x7fd7740668e0 2024-03-22T07:08:04.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.773+0000 7fd771d9b700 1 --2- 172.21.15.63:0/774039070 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fd75c0368d0 0x7fd75c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:04.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.774+0000 7fd771d9b700 1 --2- 172.21.15.63:0/774039070 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fd75c0368d0 0x7fd75c038d80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd768009990 tx=0x7fd768006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:04.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.778+0000 7fd762ffd700 1 -- 172.21.15.63:0/774039070 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd76401b350 con 0x7fd7740668e0 2024-03-22T07:08:04.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.982+0000 7fd778fe6700 1 -- 172.21.15.63:0/774039070 --> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd77405f690 con 0x7fd75c0368d0 2024-03-22T07:08:04.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.984+0000 7fd762ffd700 1 -- 172.21.15.63:0/774039070 <== mgr.14162 v2:172.21.15.63:6800/2296442540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7fd77405f690 con 0x7fd75c0368d0 2024-03-22T07:08:04.984 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:08:04.984 INFO:teuthology.orchestra.run.smithi063.stdout:[{"addr": "172.21.15.63", "hostname": "smithi063", "labels": [], "status": ""}, {"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}] 2024-03-22T07:08:04.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.986+0000 7fd778fe6700 1 -- 172.21.15.63:0/774039070 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fd75c0368d0 msgr2=0x7fd75c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:04.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.986+0000 7fd778fe6700 1 --2- 172.21.15.63:0/774039070 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fd75c0368d0 0x7fd75c038d80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd768009990 tx=0x7fd768006e40 comp rx=0 tx=0).stop 2024-03-22T07:08:04.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.986+0000 7fd778fe6700 1 -- 172.21.15.63:0/774039070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7740668e0 msgr2=0x7fd77410d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:04.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.986+0000 7fd778fe6700 1 --2- 172.21.15.63:0/774039070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7740668e0 0x7fd77410d0a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd76400b900 tx=0x7fd764009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:04.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.986+0000 7fd778fe6700 1 -- 172.21.15.63:0/774039070 shutdown_connections 2024-03-22T07:08:04.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.986+0000 7fd778fe6700 1 --2- 172.21.15.63:0/774039070 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fd75c0368d0 0x7fd75c038d80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:04.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.986+0000 7fd778fe6700 1 --2- 172.21.15.63:0/774039070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7740668e0 0x7fd77410d0a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:04.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.986+0000 7fd778fe6700 1 -- 172.21.15.63:0/774039070 >> 172.21.15.63:0/774039070 conn(0x7fd7740f7090 msgr2=0x7fd774103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:04.986 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.986+0000 7fd778fe6700 1 -- 172.21.15.63:0/774039070 shutdown_connections 2024-03-22T07:08:04.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:04.986+0000 7fd778fe6700 1 -- 172.21.15.63:0/774039070 wait complete. 2024-03-22T07:08:05.929 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-22T07:08:05.930 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-22T07:08:06.644 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:08:06.765 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:06 smithi063 ceph-mon[29200]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T07:08:06.765 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:06 smithi063 ceph-mon[29200]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:08:06.765 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:06 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:08.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:08 smithi063 ceph-mon[29200]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:08:09.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.246+0000 7f5817fdf700 1 -- 172.21.15.63:0/599966495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58100fb960 msgr2=0x7f58100fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:09.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.246+0000 7f5817fdf700 1 --2- 172.21.15.63:0/599966495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58100fb960 0x7f58100fbd70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f580c0048c0 tx=0x7f580c009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:09.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.246+0000 7f5817fdf700 1 -- 172.21.15.63:0/599966495 shutdown_connections 2024-03-22T07:08:09.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.246+0000 7f5817fdf700 1 --2- 172.21.15.63:0/599966495 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58100fb960 0x7f58100fbd70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:09.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.246+0000 7f5817fdf700 1 -- 172.21.15.63:0/599966495 >> 172.21.15.63:0/599966495 conn(0x7f58100f70c0 msgr2=0x7f58100f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:09.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.246+0000 7f5817fdf700 1 -- 172.21.15.63:0/599966495 shutdown_connections 2024-03-22T07:08:09.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.246+0000 7f5817fdf700 1 -- 172.21.15.63:0/599966495 wait complete. 2024-03-22T07:08:09.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.247+0000 7f5817fdf700 1 Processor -- start 2024-03-22T07:08:09.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.247+0000 7f5817fdf700 1 -- start start 2024-03-22T07:08:09.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.247+0000 7f5817fdf700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58100fb960 0x7f58100f9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:09.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.247+0000 7f5817fdf700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58100f9590 con 0x7f58100fb960 2024-03-22T07:08:09.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.247+0000 7f5815d7b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58100fb960 0x7f58100f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:09.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.247+0000 7f5815d7b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58100fb960 0x7f58100f9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35290/0 (socket says 172.21.15.63:35290) 2024-03-22T07:08:09.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.247+0000 7f5815d7b700 1 -- 172.21.15.63:0/2586958819 learned_addr learned my addr 172.21.15.63:0/2586958819 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:08:09.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.247+0000 7f5815d7b700 1 -- 172.21.15.63:0/2586958819 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c005040 con 0x7f58100fb960 2024-03-22T07:08:09.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.247+0000 7f5815d7b700 1 --2- 172.21.15.63:0/2586958819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58100fb960 0x7f58100f9050 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f580c0096b0 tx=0x7f580c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:09.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.248+0000 7f58067fc700 1 -- 172.21.15.63:0/2586958819 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f580c00bed0 con 0x7f58100fb960 2024-03-22T07:08:09.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.248+0000 7f58067fc700 1 -- 172.21.15.63:0/2586958819 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f580c00b5c0 con 0x7f58100fb960 2024-03-22T07:08:09.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.248+0000 7f5817fdf700 1 -- 172.21.15.63:0/2586958819 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58100f7660 con 0x7f58100fb960 2024-03-22T07:08:09.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.248+0000 7f58067fc700 1 -- 172.21.15.63:0/2586958819 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f580c01c5b0 con 0x7f58100fb960 2024-03-22T07:08:09.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.248+0000 7f5817fdf700 1 -- 172.21.15.63:0/2586958819 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58100f7b50 con 0x7f58100fb960 2024-03-22T07:08:09.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.249+0000 7f58067fc700 1 -- 172.21.15.63:0/2586958819 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f580c013420 con 0x7f58100fb960 2024-03-22T07:08:09.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.249+0000 7f5817fdf700 1 -- 172.21.15.63:0/2586958819 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581005f5f0 con 0x7f58100fb960 2024-03-22T07:08:09.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.249+0000 7f58067fc700 1 --2- 172.21.15.63:0/2586958819 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f57fc036880 0x7f57fc038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:09.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.249+0000 7f58067fc700 1 -- 172.21.15.63:0/2586958819 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f580c049a10 con 0x7f58100fb960 2024-03-22T07:08:09.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.249+0000 7f581557a700 1 --2- 172.21.15.63:0/2586958819 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f57fc036880 0x7f57fc038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:09.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.249+0000 7f581557a700 1 --2- 172.21.15.63:0/2586958819 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f57fc036880 0x7f57fc038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5800009990 tx=0x7f5800006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:09.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.254+0000 7f58067fc700 1 -- 172.21.15.63:0/2586958819 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f580c013a80 con 0x7f58100fb960 2024-03-22T07:08:09.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:09.461+0000 7f5817fdf700 1 -- 172.21.15.63:0/2586958819 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f581004bed0 con 0x7f58100fb960 2024-03-22T07:08:10.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:10.389+0000 7f58067fc700 1 -- 172.21.15.63:0/2586958819 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f580c011370 con 0x7f58100fb960 2024-03-22T07:08:10.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:10.392+0000 7f5817fdf700 1 -- 172.21.15.63:0/2586958819 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f57fc036880 msgr2=0x7f57fc038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:10.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:10.392+0000 7f5817fdf700 1 --2- 172.21.15.63:0/2586958819 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f57fc036880 0x7f57fc038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5800009990 tx=0x7f5800006e40 comp rx=0 tx=0).stop 2024-03-22T07:08:10.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:10.392+0000 7f5817fdf700 1 -- 172.21.15.63:0/2586958819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58100fb960 msgr2=0x7f58100f9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:10.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:10.392+0000 7f5817fdf700 1 --2- 172.21.15.63:0/2586958819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58100fb960 0x7f58100f9050 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f580c0096b0 tx=0x7f580c009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:10.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:10.393+0000 7f5817fdf700 1 -- 172.21.15.63:0/2586958819 shutdown_connections 2024-03-22T07:08:10.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:10.393+0000 7f5817fdf700 1 --2- 172.21.15.63:0/2586958819 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f57fc036880 0x7f57fc038d30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:10.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:10.393+0000 7f5817fdf700 1 --2- 172.21.15.63:0/2586958819 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f58100fb960 0x7f58100f9050 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:10.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:10.393+0000 7f5817fdf700 1 -- 172.21.15.63:0/2586958819 >> 172.21.15.63:0/2586958819 conn(0x7f58100f70c0 msgr2=0x7f5810065d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:10.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:10.393+0000 7f5817fdf700 1 -- 172.21.15.63:0/2586958819 shutdown_connections 2024-03-22T07:08:10.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:08:10.393+0000 7f5817fdf700 1 -- 172.21.15.63:0/2586958819 wait complete. 2024-03-22T07:08:10.394 INFO:teuthology.orchestra.run.smithi063.stderr:adjusted tunables profile to default 2024-03-22T07:08:10.505 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:10 smithi063 ceph-mon[29200]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:08:10.505 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:10 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2586958819' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-22T07:08:11.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:11 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2586958819' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-22T07:08:11.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:11 smithi063 ceph-mon[29200]: osdmap e4: 0 total, 0 up, 0 in 2024-03-22T07:08:12.559 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:12 smithi063 ceph-mon[29200]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:08:12.587 INFO:tasks.cephadm:Adding mon.smithi063 on smithi063 2024-03-22T07:08:12.587 INFO:tasks.cephadm:Adding mon.smithi069 on smithi069 2024-03-22T07:08:12.587 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph orch apply mon '2;smithi063:172.21.15.63=smithi063;smithi069:172.21.15.69=smithi069' 2024-03-22T07:08:12.838 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:12.884 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:14 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:14 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:14 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:14 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:14 smithi063 ceph-mon[29200]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:08:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:14 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:14 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:14 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-22T07:08:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:14 smithi063 ceph-mon[29200]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-22T07:08:14.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:14 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:14.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:14 smithi063 ceph-mon[29200]: Deploying daemon grafana.smithi063 on smithi063 2024-03-22T07:08:14.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.650+0000 7fab0144d700 1 -- 172.21.15.69:0/1741309285 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faafc0fb960 msgr2=0x7faafc0fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:14.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.650+0000 7fab0144d700 1 --2- 172.21.15.69:0/1741309285 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faafc0fb960 0x7faafc0fbd70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faaec0048c0 tx=0x7faaec009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:14.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.650+0000 7fab0144d700 1 -- 172.21.15.69:0/1741309285 shutdown_connections 2024-03-22T07:08:14.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.650+0000 7fab0144d700 1 --2- 172.21.15.69:0/1741309285 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faafc0fb960 0x7faafc0fbd70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:14.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.650+0000 7fab0144d700 1 -- 172.21.15.69:0/1741309285 >> 172.21.15.69:0/1741309285 conn(0x7faafc0f70c0 msgr2=0x7faafc0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:14.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.650+0000 7fab0144d700 1 -- 172.21.15.69:0/1741309285 shutdown_connections 2024-03-22T07:08:14.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.650+0000 7fab0144d700 1 -- 172.21.15.69:0/1741309285 wait complete. 2024-03-22T07:08:14.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.650+0000 7fab0144d700 1 Processor -- start 2024-03-22T07:08:14.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.651+0000 7fab0144d700 1 -- start start 2024-03-22T07:08:14.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.651+0000 7fab0144d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faafc0fb960 0x7faafc10f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:14.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.651+0000 7fab0144d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc10f830 con 0x7faafc0fb960 2024-03-22T07:08:14.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.651+0000 7faafaffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faafc0fb960 0x7faafc10f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:14.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.651+0000 7faafaffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faafc0fb960 0x7faafc10f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.69:41404/0 (socket says 172.21.15.69:41404) 2024-03-22T07:08:14.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.651+0000 7faafaffd700 1 -- 172.21.15.69:0/143477633 learned_addr learned my addr 172.21.15.69:0/143477633 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:08:14.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.651+0000 7faafaffd700 1 -- 172.21.15.69:0/143477633 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaec005040 con 0x7faafc0fb960 2024-03-22T07:08:14.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.652+0000 7faafaffd700 1 --2- 172.21.15.69:0/143477633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faafc0fb960 0x7faafc10f2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faaec009c90 tx=0x7faaec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:14.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.652+0000 7faaf3fff700 1 -- 172.21.15.69:0/143477633 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faaec004e60 con 0x7faafc0fb960 2024-03-22T07:08:14.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.652+0000 7fab0144d700 1 -- 172.21.15.69:0/143477633 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faafc10fa30 con 0x7faafc0fb960 2024-03-22T07:08:14.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.652+0000 7faaf3fff700 1 -- 172.21.15.69:0/143477633 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faaec009830 con 0x7faafc0fb960 2024-03-22T07:08:14.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.652+0000 7faaf3fff700 1 -- 172.21.15.69:0/143477633 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faaec022640 con 0x7faafc0fb960 2024-03-22T07:08:14.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.652+0000 7fab0144d700 1 -- 172.21.15.69:0/143477633 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faafc1105a0 con 0x7faafc0fb960 2024-03-22T07:08:14.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.653+0000 7faaf3fff700 1 -- 172.21.15.69:0/143477633 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7faaec011480 con 0x7faafc0fb960 2024-03-22T07:08:14.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.653+0000 7fab0144d700 1 -- 172.21.15.69:0/143477633 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faafc100d10 con 0x7faafc0fb960 2024-03-22T07:08:14.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.653+0000 7faaf3fff700 1 --2- 172.21.15.69:0/143477633 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7faae8036880 0x7faae8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:14.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.653+0000 7faaf3fff700 1 -- 172.21.15.69:0/143477633 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7faaec049fb0 con 0x7faafc0fb960 2024-03-22T07:08:14.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.653+0000 7faaf27fc700 1 --2- 172.21.15.69:0/143477633 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7faae8036880 0x7faae8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:14.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.653+0000 7faaf27fc700 1 --2- 172.21.15.69:0/143477633 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7faae8036880 0x7faae8038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faae4009990 tx=0x7faae4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:14.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.659+0000 7faaf3fff700 1 -- 172.21.15.69:0/143477633 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7faaec011a50 con 0x7faafc0fb960 2024-03-22T07:08:14.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.853+0000 7fab0144d700 1 -- 172.21.15.69:0/143477633 --> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi063:172.21.15.63=smithi063;smithi069:172.21.15.69=smithi069", "target": ["mon-mgr", ""]}) v1 -- 0x7faafc110890 con 0x7faae8036880 2024-03-22T07:08:14.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.859+0000 7faaf3fff700 1 -- 172.21.15.69:0/143477633 <== mgr.14162 v2:172.21.15.63:6800/2296442540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7faafc110890 con 0x7faae8036880 2024-03-22T07:08:14.859 INFO:teuthology.orchestra.run.smithi069.stdout:Scheduled mon update... 2024-03-22T07:08:14.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.861+0000 7fab0144d700 1 -- 172.21.15.69:0/143477633 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7faae8036880 msgr2=0x7faae8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:14.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.861+0000 7fab0144d700 1 --2- 172.21.15.69:0/143477633 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7faae8036880 0x7faae8038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faae4009990 tx=0x7faae4006e40 comp rx=0 tx=0).stop 2024-03-22T07:08:14.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.861+0000 7fab0144d700 1 -- 172.21.15.69:0/143477633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faafc0fb960 msgr2=0x7faafc10f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:14.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.861+0000 7fab0144d700 1 --2- 172.21.15.69:0/143477633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faafc0fb960 0x7faafc10f2f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faaec009c90 tx=0x7faaec009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:14.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.861+0000 7fab0144d700 1 -- 172.21.15.69:0/143477633 shutdown_connections 2024-03-22T07:08:14.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.861+0000 7fab0144d700 1 --2- 172.21.15.69:0/143477633 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7faae8036880 0x7faae8038d30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:14.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.861+0000 7fab0144d700 1 --2- 172.21.15.69:0/143477633 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7faafc0fb960 0x7faafc10f2f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:14.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.861+0000 7fab0144d700 1 -- 172.21.15.69:0/143477633 >> 172.21.15.69:0/143477633 conn(0x7faafc0f70c0 msgr2=0x7faafc0f7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:14.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.861+0000 7fab0144d700 1 -- 172.21.15.69:0/143477633 shutdown_connections 2024-03-22T07:08:14.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:14.861+0000 7fab0144d700 1 -- 172.21.15.69:0/143477633 wait complete. 2024-03-22T07:08:15.429 DEBUG:teuthology.orchestra.run.smithi069:mon.smithi069> sudo journalctl -f -n 0 -u ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mon.smithi069.service 2024-03-22T07:08:15.432 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:08:15.432 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:08:15.708 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:15.756 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:16.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:15 smithi063 ceph-mon[29200]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi063:172.21.15.63=smithi063;smithi069:172.21.15.69=smithi069", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:08:16.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:15 smithi063 ceph-mon[29200]: Saving service mon spec with placement smithi063:172.21.15.63=smithi063;smithi069:172.21.15.69=smithi069;count:2 2024-03-22T07:08:16.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:15 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:16.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:15 smithi063 ceph-mon[29200]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:08:17.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.245+0000 7f3237997700 1 -- 172.21.15.69:0/171816644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32300fc9e0 msgr2=0x7f32300fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:17.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.245+0000 7f3237997700 1 --2- 172.21.15.69:0/171816644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32300fc9e0 0x7f32300fcdf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3228009c60 tx=0x7f3228004d10 comp rx=0 tx=0).stop 2024-03-22T07:08:17.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.245+0000 7f3237997700 1 -- 172.21.15.69:0/171816644 shutdown_connections 2024-03-22T07:08:17.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.245+0000 7f3237997700 1 --2- 172.21.15.69:0/171816644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32300fc9e0 0x7f32300fcdf0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:17.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.245+0000 7f3237997700 1 -- 172.21.15.69:0/171816644 >> 172.21.15.69:0/171816644 conn(0x7f32300f8120 msgr2=0x7f32300fa570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:17.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.245+0000 7f3237997700 1 -- 172.21.15.69:0/171816644 shutdown_connections 2024-03-22T07:08:17.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.245+0000 7f3237997700 1 -- 172.21.15.69:0/171816644 wait complete. 2024-03-22T07:08:17.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.246+0000 7f3237997700 1 Processor -- start 2024-03-22T07:08:17.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.246+0000 7f3237997700 1 -- start start 2024-03-22T07:08:17.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.246+0000 7f3237997700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32300fc9e0 0x7f323010d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:17.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.246+0000 7f3237997700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323010d550 con 0x7f32300fc9e0 2024-03-22T07:08:17.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.246+0000 7f3235733700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32300fc9e0 0x7f323010d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:17.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.246+0000 7f3235733700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32300fc9e0 0x7f323010d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.69:41420/0 (socket says 172.21.15.69:41420) 2024-03-22T07:08:17.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.246+0000 7f3235733700 1 -- 172.21.15.69:0/293944724 learned_addr learned my addr 172.21.15.69:0/293944724 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:08:17.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.246+0000 7f3235733700 1 -- 172.21.15.69:0/293944724 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3228009940 con 0x7f32300fc9e0 2024-03-22T07:08:17.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.246+0000 7f3235733700 1 --2- 172.21.15.69:0/293944724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32300fc9e0 0x7f323010d010 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3228009500 tx=0x7f3228004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:17.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.247+0000 7f3225ffb700 1 -- 172.21.15.69:0/293944724 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3228007c80 con 0x7f32300fc9e0 2024-03-22T07:08:17.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.247+0000 7f3225ffb700 1 -- 172.21.15.69:0/293944724 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f322801d910 con 0x7f32300fc9e0 2024-03-22T07:08:17.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.247+0000 7f3237997700 1 -- 172.21.15.69:0/293944724 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323010d750 con 0x7f32300fc9e0 2024-03-22T07:08:17.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.247+0000 7f3225ffb700 1 -- 172.21.15.69:0/293944724 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32280129d0 con 0x7f32300fc9e0 2024-03-22T07:08:17.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.247+0000 7f3237997700 1 -- 172.21.15.69:0/293944724 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323010e2c0 con 0x7f32300fc9e0 2024-03-22T07:08:17.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.248+0000 7f3225ffb700 1 -- 172.21.15.69:0/293944724 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f3228019040 con 0x7f32300fc9e0 2024-03-22T07:08:17.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.248+0000 7f3237997700 1 -- 172.21.15.69:0/293944724 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3230102ef0 con 0x7f32300fc9e0 2024-03-22T07:08:17.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.248+0000 7f3225ffb700 1 --2- 172.21.15.69:0/293944724 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f321c0368d0 0x7f321c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:17.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.248+0000 7f3225ffb700 1 -- 172.21.15.69:0/293944724 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f322804a3d0 con 0x7f32300fc9e0 2024-03-22T07:08:17.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.248+0000 7f3234f32700 1 --2- 172.21.15.69:0/293944724 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f321c0368d0 0x7f321c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:17.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.248+0000 7f3234f32700 1 --2- 172.21.15.69:0/293944724 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f321c0368d0 0x7f321c038d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3220009990 tx=0x7f3220006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:17.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.253+0000 7f3225ffb700 1 -- 172.21.15.69:0/293944724 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f322802eb40 con 0x7f32300fc9e0 2024-03-22T07:08:17.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.512+0000 7f3237997700 1 -- 172.21.15.69:0/293944724 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f323010ec20 con 0x7f32300fc9e0 2024-03-22T07:08:17.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.512+0000 7f3225ffb700 1 -- 172.21.15.69:0/293944724 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f3228017020 con 0x7f32300fc9e0 2024-03-22T07:08:17.512 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:08:17.512 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","modified":"2024-03-22T07:06:35.169231Z","created":"2024-03-22T07:06:35.169231Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:08:17.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.515+0000 7f3237997700 1 -- 172.21.15.69:0/293944724 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f321c0368d0 msgr2=0x7f321c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:17.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.515+0000 7f3237997700 1 --2- 172.21.15.69:0/293944724 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f321c0368d0 0x7f321c038d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3220009990 tx=0x7f3220006e40 comp rx=0 tx=0).stop 2024-03-22T07:08:17.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.515+0000 7f3237997700 1 -- 172.21.15.69:0/293944724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32300fc9e0 msgr2=0x7f323010d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:17.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.515+0000 7f3237997700 1 --2- 172.21.15.69:0/293944724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32300fc9e0 0x7f323010d010 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3228009500 tx=0x7f3228004d10 comp rx=0 tx=0).stop 2024-03-22T07:08:17.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.515+0000 7f3237997700 1 -- 172.21.15.69:0/293944724 shutdown_connections 2024-03-22T07:08:17.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.515+0000 7f3237997700 1 --2- 172.21.15.69:0/293944724 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f321c0368d0 0x7f321c038d80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:17.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.515+0000 7f3237997700 1 --2- 172.21.15.69:0/293944724 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f32300fc9e0 0x7f323010d010 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:17.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.515+0000 7f3237997700 1 -- 172.21.15.69:0/293944724 >> 172.21.15.69:0/293944724 conn(0x7f32300f8120 msgr2=0x7f3230103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:17.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.515+0000 7f3237997700 1 -- 172.21.15.69:0/293944724 shutdown_connections 2024-03-22T07:08:17.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:17.515+0000 7f3237997700 1 -- 172.21.15.69:0/293944724 wait complete. 2024-03-22T07:08:17.516 INFO:teuthology.orchestra.run.smithi069.stderr:dumped monmap epoch 1 2024-03-22T07:08:17.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:17 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:18.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:18 smithi063 ceph-mon[29200]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:08:18.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:18 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/293944724' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:08:19.074 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:08:19.075 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:08:19.323 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:19.369 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:20.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:20 smithi063 ceph-mon[29200]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:08:20.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.798+0000 7fa3169d6700 1 -- 172.21.15.69:0/1401766896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3100fca40 msgr2=0x7fa3100fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:20.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.798+0000 7fa3169d6700 1 --2- 172.21.15.69:0/1401766896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3100fca40 0x7fa3100fce50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa3000048c0 tx=0x7fa300009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:20.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.798+0000 7fa3169d6700 1 -- 172.21.15.69:0/1401766896 shutdown_connections 2024-03-22T07:08:20.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.798+0000 7fa3169d6700 1 --2- 172.21.15.69:0/1401766896 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3100fca40 0x7fa3100fce50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:20.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.798+0000 7fa3169d6700 1 -- 172.21.15.69:0/1401766896 >> 172.21.15.69:0/1401766896 conn(0x7fa3100f81c0 msgr2=0x7fa3100fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:20.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.798+0000 7fa3169d6700 1 -- 172.21.15.69:0/1401766896 shutdown_connections 2024-03-22T07:08:20.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.798+0000 7fa3169d6700 1 -- 172.21.15.69:0/1401766896 wait complete. 2024-03-22T07:08:20.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.798+0000 7fa3169d6700 1 Processor -- start 2024-03-22T07:08:20.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.799+0000 7fa3169d6700 1 -- start start 2024-03-22T07:08:20.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.799+0000 7fa3169d6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3100fca40 0x7fa310071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:20.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.799+0000 7fa3169d6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3100722a0 con 0x7fa3100fca40 2024-03-22T07:08:20.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.799+0000 7fa30ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3100fca40 0x7fa310071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:20.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.799+0000 7fa30ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3100fca40 0x7fa310071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.69:34778/0 (socket says 172.21.15.69:34778) 2024-03-22T07:08:20.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.799+0000 7fa30ffff700 1 -- 172.21.15.69:0/3399898238 learned_addr learned my addr 172.21.15.69:0/3399898238 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:08:20.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.799+0000 7fa30ffff700 1 -- 172.21.15.69:0/3399898238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa300005040 con 0x7fa3100fca40 2024-03-22T07:08:20.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.799+0000 7fa30ffff700 1 --2- 172.21.15.69:0/3399898238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3100fca40 0x7fa310071d60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa3000096b0 tx=0x7fa300009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:20.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.799+0000 7fa30cff9700 1 -- 172.21.15.69:0/3399898238 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa30000bed0 con 0x7fa3100fca40 2024-03-22T07:08:20.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.799+0000 7fa30cff9700 1 -- 172.21.15.69:0/3399898238 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa30000b5c0 con 0x7fa3100fca40 2024-03-22T07:08:20.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.799+0000 7fa3169d6700 1 -- 172.21.15.69:0/3399898238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa310070370 con 0x7fa3100fca40 2024-03-22T07:08:20.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.799+0000 7fa30cff9700 1 -- 172.21.15.69:0/3399898238 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa30001c5b0 con 0x7fa3100fca40 2024-03-22T07:08:20.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.799+0000 7fa3169d6700 1 -- 172.21.15.69:0/3399898238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa310070860 con 0x7fa3100fca40 2024-03-22T07:08:20.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.800+0000 7fa30cff9700 1 -- 172.21.15.69:0/3399898238 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fa30001c820 con 0x7fa3100fca40 2024-03-22T07:08:20.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.800+0000 7fa3169d6700 1 -- 172.21.15.69:0/3399898238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31005f5f0 con 0x7fa3100fca40 2024-03-22T07:08:20.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.800+0000 7fa30cff9700 1 --2- 172.21.15.69:0/3399898238 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fa2f8036880 0x7fa2f8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:20.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.800+0000 7fa30cff9700 1 -- 172.21.15.69:0/3399898238 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fa300049d10 con 0x7fa3100fca40 2024-03-22T07:08:20.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.801+0000 7fa30f7fe700 1 --2- 172.21.15.69:0/3399898238 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fa2f8036880 0x7fa2f8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:20.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.801+0000 7fa30f7fe700 1 --2- 172.21.15.69:0/3399898238 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fa2f8036880 0x7fa2f8038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa304009990 tx=0x7fa304006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:20.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:20.806+0000 7fa30cff9700 1 -- 172.21.15.69:0/3399898238 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa300023300 con 0x7fa3100fca40 2024-03-22T07:08:21.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:21.068+0000 7fa3169d6700 1 -- 172.21.15.69:0/3399898238 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa310106420 con 0x7fa3100fca40 2024-03-22T07:08:21.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:21.069+0000 7fa30cff9700 1 -- 172.21.15.69:0/3399898238 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fa300011b10 con 0x7fa3100fca40 2024-03-22T07:08:21.069 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:08:21.069 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","modified":"2024-03-22T07:06:35.169231Z","created":"2024-03-22T07:06:35.169231Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:08:21.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:21.071+0000 7fa3169d6700 1 -- 172.21.15.69:0/3399898238 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fa2f8036880 msgr2=0x7fa2f8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:21.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:21.071+0000 7fa3169d6700 1 --2- 172.21.15.69:0/3399898238 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fa2f8036880 0x7fa2f8038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa304009990 tx=0x7fa304006e40 comp rx=0 tx=0).stop 2024-03-22T07:08:21.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:21.071+0000 7fa3169d6700 1 -- 172.21.15.69:0/3399898238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3100fca40 msgr2=0x7fa310071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:21.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:21.071+0000 7fa3169d6700 1 --2- 172.21.15.69:0/3399898238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3100fca40 0x7fa310071d60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa3000096b0 tx=0x7fa300009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:21.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:21.071+0000 7fa3169d6700 1 -- 172.21.15.69:0/3399898238 shutdown_connections 2024-03-22T07:08:21.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:21.071+0000 7fa3169d6700 1 --2- 172.21.15.69:0/3399898238 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fa2f8036880 0x7fa2f8038d30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:21.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:21.071+0000 7fa3169d6700 1 --2- 172.21.15.69:0/3399898238 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3100fca40 0x7fa310071d60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:21.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:21.071+0000 7fa3169d6700 1 -- 172.21.15.69:0/3399898238 >> 172.21.15.69:0/3399898238 conn(0x7fa3100f81c0 msgr2=0x7fa310103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:21.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:21.071+0000 7fa3169d6700 1 -- 172.21.15.69:0/3399898238 shutdown_connections 2024-03-22T07:08:21.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:21.071+0000 7fa3169d6700 1 -- 172.21.15.69:0/3399898238 wait complete. 2024-03-22T07:08:21.072 INFO:teuthology.orchestra.run.smithi069.stderr:dumped monmap epoch 1 2024-03-22T07:08:21.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:21 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/3399898238' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:08:22.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:22 smithi063 ceph-mon[29200]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:08:22.672 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:08:22.672 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:08:22.911 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:22.957 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:24.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:24 smithi063 ceph-mon[29200]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:08:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.401+0000 7f18804b0700 1 -- 172.21.15.69:0/216650096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1878066a40 msgr2=0x7f18780ff650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.401+0000 7f18804b0700 1 --2- 172.21.15.69:0/216650096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1878066a40 0x7f18780ff650 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1874009c60 tx=0x7f1874004d10 comp rx=0 tx=0).stop 2024-03-22T07:08:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.401+0000 7f18804b0700 1 -- 172.21.15.69:0/216650096 shutdown_connections 2024-03-22T07:08:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.401+0000 7f18804b0700 1 --2- 172.21.15.69:0/216650096 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1878066a40 0x7f18780ff650 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.401+0000 7f18804b0700 1 -- 172.21.15.69:0/216650096 >> 172.21.15.69:0/216650096 conn(0x7f18780f70c0 msgr2=0x7f18780f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.401+0000 7f18804b0700 1 -- 172.21.15.69:0/216650096 shutdown_connections 2024-03-22T07:08:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.401+0000 7f18804b0700 1 -- 172.21.15.69:0/216650096 wait complete. 2024-03-22T07:08:24.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.401+0000 7f18804b0700 1 Processor -- start 2024-03-22T07:08:24.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.401+0000 7f18804b0700 1 -- start start 2024-03-22T07:08:24.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.402+0000 7f18804b0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1878066a40 0x7f187810d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:24.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.402+0000 7f18804b0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187810d610 con 0x7f1878066a40 2024-03-22T07:08:24.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.402+0000 7f187e24c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1878066a40 0x7f187810d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:24.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.402+0000 7f187e24c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1878066a40 0x7f187810d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.69:34790/0 (socket says 172.21.15.69:34790) 2024-03-22T07:08:24.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.402+0000 7f187e24c700 1 -- 172.21.15.69:0/2272108453 learned_addr learned my addr 172.21.15.69:0/2272108453 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:08:24.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.402+0000 7f187e24c700 1 -- 172.21.15.69:0/2272108453 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1874009940 con 0x7f1878066a40 2024-03-22T07:08:24.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.402+0000 7f187e24c700 1 --2- 172.21.15.69:0/2272108453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1878066a40 0x7f187810d0d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f18740097b0 tx=0x7f1874004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:24.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.402+0000 7f186effd700 1 -- 172.21.15.69:0/2272108453 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1874012410 con 0x7f1878066a40 2024-03-22T07:08:24.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.402+0000 7f186effd700 1 -- 172.21.15.69:0/2272108453 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1874012a40 con 0x7f1878066a40 2024-03-22T07:08:24.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.402+0000 7f186effd700 1 -- 172.21.15.69:0/2272108453 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f187401ba20 con 0x7f1878066a40 2024-03-22T07:08:24.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.402+0000 7f18804b0700 1 -- 172.21.15.69:0/2272108453 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187810d810 con 0x7f1878066a40 2024-03-22T07:08:24.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.402+0000 7f18804b0700 1 -- 172.21.15.69:0/2272108453 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187810e380 con 0x7f1878066a40 2024-03-22T07:08:24.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.403+0000 7f18804b0700 1 -- 172.21.15.69:0/2272108453 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1878102f60 con 0x7f1878066a40 2024-03-22T07:08:24.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.403+0000 7f186effd700 1 -- 172.21.15.69:0/2272108453 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f1874019040 con 0x7f1878066a40 2024-03-22T07:08:24.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.404+0000 7f186effd700 1 --2- 172.21.15.69:0/2272108453 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f1864036880 0x7f1864038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:24.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.404+0000 7f186effd700 1 -- 172.21.15.69:0/2272108453 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f1874049d10 con 0x7f1878066a40 2024-03-22T07:08:24.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.404+0000 7f187da4b700 1 --2- 172.21.15.69:0/2272108453 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f1864036880 0x7f1864038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:24.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.404+0000 7f187da4b700 1 --2- 172.21.15.69:0/2272108453 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f1864036880 0x7f1864038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1868009990 tx=0x7f1868006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:24.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.409+0000 7f186effd700 1 -- 172.21.15.69:0/2272108453 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f187404f020 con 0x7f1878066a40 2024-03-22T07:08:24.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.671+0000 7f18804b0700 1 -- 172.21.15.69:0/2272108453 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f187810ed90 con 0x7f1878066a40 2024-03-22T07:08:24.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.672+0000 7f186effd700 1 -- 172.21.15.69:0/2272108453 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f1874017090 con 0x7f1878066a40 2024-03-22T07:08:24.671 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:08:24.672 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","modified":"2024-03-22T07:06:35.169231Z","created":"2024-03-22T07:06:35.169231Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:08:24.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.673+0000 7f18804b0700 1 -- 172.21.15.69:0/2272108453 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f1864036880 msgr2=0x7f1864038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:24.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.673+0000 7f18804b0700 1 --2- 172.21.15.69:0/2272108453 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f1864036880 0x7f1864038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1868009990 tx=0x7f1868006e40 comp rx=0 tx=0).stop 2024-03-22T07:08:24.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.673+0000 7f18804b0700 1 -- 172.21.15.69:0/2272108453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1878066a40 msgr2=0x7f187810d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:24.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.673+0000 7f18804b0700 1 --2- 172.21.15.69:0/2272108453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1878066a40 0x7f187810d0d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f18740097b0 tx=0x7f1874004d10 comp rx=0 tx=0).stop 2024-03-22T07:08:24.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.673+0000 7f18804b0700 1 -- 172.21.15.69:0/2272108453 shutdown_connections 2024-03-22T07:08:24.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.673+0000 7f18804b0700 1 --2- 172.21.15.69:0/2272108453 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f1864036880 0x7f1864038d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:24.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.673+0000 7f18804b0700 1 --2- 172.21.15.69:0/2272108453 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1878066a40 0x7f187810d0d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:24.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.674+0000 7f18804b0700 1 -- 172.21.15.69:0/2272108453 >> 172.21.15.69:0/2272108453 conn(0x7f18780f70c0 msgr2=0x7f1878103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:24.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.674+0000 7f18804b0700 1 -- 172.21.15.69:0/2272108453 shutdown_connections 2024-03-22T07:08:24.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:24.674+0000 7f18804b0700 1 -- 172.21.15.69:0/2272108453 wait complete. 2024-03-22T07:08:24.674 INFO:teuthology.orchestra.run.smithi069.stderr:dumped monmap epoch 1 2024-03-22T07:08:25.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:25 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/2272108453' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:08:26.290 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:08:26.291 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:08:26.353 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:26 smithi063 ceph-mon[29200]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:08:26.353 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:26 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:26.353 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:26 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:26.353 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:26 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:26.353 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:26 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:26.353 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:26 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:26.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:26 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:26.354 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:26 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:26.536 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:26.581 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:27.399 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:27 smithi063 ceph-mon[29200]: Deploying daemon prometheus.smithi063 on smithi063 2024-03-22T07:08:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.026+0000 7f2817fde700 1 -- 172.21.15.69:0/3977881879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28100fd050 msgr2=0x7f28100fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.026+0000 7f2817fde700 1 --2- 172.21.15.69:0/3977881879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28100fd050 0x7f28100fd460 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f28000048c0 tx=0x7f2800009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.026+0000 7f2817fde700 1 -- 172.21.15.69:0/3977881879 shutdown_connections 2024-03-22T07:08:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.026+0000 7f2817fde700 1 --2- 172.21.15.69:0/3977881879 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28100fd050 0x7f28100fd460 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.026+0000 7f2817fde700 1 -- 172.21.15.69:0/3977881879 >> 172.21.15.69:0/3977881879 conn(0x7f28100f70d0 msgr2=0x7f28100f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.026+0000 7f2817fde700 1 -- 172.21.15.69:0/3977881879 shutdown_connections 2024-03-22T07:08:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.026+0000 7f2817fde700 1 -- 172.21.15.69:0/3977881879 wait complete. 2024-03-22T07:08:28.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.027+0000 7f2817fde700 1 Processor -- start 2024-03-22T07:08:28.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.027+0000 7f2817fde700 1 -- start start 2024-03-22T07:08:28.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.027+0000 7f2817fde700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28100fd050 0x7f281010d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:28.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.027+0000 7f2817fde700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281010d600 con 0x7f28100fd050 2024-03-22T07:08:28.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.027+0000 7f2815d7a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28100fd050 0x7f281010d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:28.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.027+0000 7f2815d7a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28100fd050 0x7f281010d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.69:34804/0 (socket says 172.21.15.69:34804) 2024-03-22T07:08:28.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.027+0000 7f2815d7a700 1 -- 172.21.15.69:0/934692766 learned_addr learned my addr 172.21.15.69:0/934692766 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:08:28.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.027+0000 7f2815d7a700 1 -- 172.21.15.69:0/934692766 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2800005040 con 0x7f28100fd050 2024-03-22T07:08:28.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.028+0000 7f2815d7a700 1 --2- 172.21.15.69:0/934692766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28100fd050 0x7f281010d0c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2800009680 tx=0x7f2800009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:28.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.028+0000 7f28067fc700 1 -- 172.21.15.69:0/934692766 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f280000b5c0 con 0x7f28100fd050 2024-03-22T07:08:28.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.028+0000 7f28067fc700 1 -- 172.21.15.69:0/934692766 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f280000be70 con 0x7f28100fd050 2024-03-22T07:08:28.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.028+0000 7f2817fde700 1 -- 172.21.15.69:0/934692766 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281010d800 con 0x7f28100fd050 2024-03-22T07:08:28.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.028+0000 7f28067fc700 1 -- 172.21.15.69:0/934692766 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f28000135a0 con 0x7f28100fd050 2024-03-22T07:08:28.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.028+0000 7f2817fde700 1 -- 172.21.15.69:0/934692766 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281010e370 con 0x7f28100fd050 2024-03-22T07:08:28.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.029+0000 7f28067fc700 1 -- 172.21.15.69:0/934692766 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f2800013860 con 0x7f28100fd050 2024-03-22T07:08:28.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.029+0000 7f2817fde700 1 -- 172.21.15.69:0/934692766 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2810101e70 con 0x7f28100fd050 2024-03-22T07:08:28.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.029+0000 7f28067fc700 1 --2- 172.21.15.69:0/934692766 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f27fc036880 0x7f27fc038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:28.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.029+0000 7f28067fc700 1 -- 172.21.15.69:0/934692766 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f28000494e0 con 0x7f28100fd050 2024-03-22T07:08:28.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.029+0000 7f2815579700 1 --2- 172.21.15.69:0/934692766 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f27fc036880 0x7f27fc038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:28.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.029+0000 7f2815579700 1 --2- 172.21.15.69:0/934692766 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f27fc036880 0x7f27fc038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f280c009990 tx=0x7f280c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:28.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.034+0000 7f28067fc700 1 -- 172.21.15.69:0/934692766 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2800011940 con 0x7f28100fd050 2024-03-22T07:08:28.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.293+0000 7f2817fde700 1 -- 172.21.15.69:0/934692766 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f281010e5b0 con 0x7f28100fd050 2024-03-22T07:08:28.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.294+0000 7f28067fc700 1 -- 172.21.15.69:0/934692766 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f2800011370 con 0x7f28100fd050 2024-03-22T07:08:28.294 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:08:28.294 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","modified":"2024-03-22T07:06:35.169231Z","created":"2024-03-22T07:06:35.169231Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:08:28.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.295+0000 7f2817fde700 1 -- 172.21.15.69:0/934692766 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f27fc036880 msgr2=0x7f27fc038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:28.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.295+0000 7f2817fde700 1 --2- 172.21.15.69:0/934692766 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f27fc036880 0x7f27fc038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f280c009990 tx=0x7f280c006e40 comp rx=0 tx=0).stop 2024-03-22T07:08:28.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.295+0000 7f2817fde700 1 -- 172.21.15.69:0/934692766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28100fd050 msgr2=0x7f281010d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:28.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.295+0000 7f2817fde700 1 --2- 172.21.15.69:0/934692766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28100fd050 0x7f281010d0c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2800009680 tx=0x7f2800009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:28.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.295+0000 7f2817fde700 1 -- 172.21.15.69:0/934692766 shutdown_connections 2024-03-22T07:08:28.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.295+0000 7f2817fde700 1 --2- 172.21.15.69:0/934692766 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f27fc036880 0x7f27fc038d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:28.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.295+0000 7f2817fde700 1 --2- 172.21.15.69:0/934692766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f28100fd050 0x7f281010d0c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:28.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.295+0000 7f2817fde700 1 -- 172.21.15.69:0/934692766 >> 172.21.15.69:0/934692766 conn(0x7f28100f70d0 msgr2=0x7f2810065070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:28.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.296+0000 7f2817fde700 1 -- 172.21.15.69:0/934692766 shutdown_connections 2024-03-22T07:08:28.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:28.296+0000 7f2817fde700 1 -- 172.21.15.69:0/934692766 wait complete. 2024-03-22T07:08:28.297 INFO:teuthology.orchestra.run.smithi069.stderr:dumped monmap epoch 1 2024-03-22T07:08:28.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:28 smithi063 ceph-mon[29200]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:08:28.671 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:28 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/934692766' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:08:29.836 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:08:29.836 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:08:30.084 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:30.130 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:30.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:30 smithi063 ceph-mon[29200]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:08:31.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.518+0000 7f6487a6a700 1 -- 172.21.15.69:0/1424743499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64800fd040 msgr2=0x7f64800fd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:31.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.518+0000 7f6487a6a700 1 --2- 172.21.15.69:0/1424743499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64800fd040 0x7f64800fd450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f64700048c0 tx=0x7f6470009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:31.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.518+0000 7f6487a6a700 1 -- 172.21.15.69:0/1424743499 shutdown_connections 2024-03-22T07:08:31.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.518+0000 7f6487a6a700 1 --2- 172.21.15.69:0/1424743499 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64800fd040 0x7f64800fd450 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:31.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.518+0000 7f6487a6a700 1 -- 172.21.15.69:0/1424743499 >> 172.21.15.69:0/1424743499 conn(0x7f64800f70c0 msgr2=0x7f64800f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:31.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.518+0000 7f6487a6a700 1 -- 172.21.15.69:0/1424743499 shutdown_connections 2024-03-22T07:08:31.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.518+0000 7f6487a6a700 1 -- 172.21.15.69:0/1424743499 wait complete. 2024-03-22T07:08:31.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.519+0000 7f6487a6a700 1 Processor -- start 2024-03-22T07:08:31.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.519+0000 7f6487a6a700 1 -- start start 2024-03-22T07:08:31.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.519+0000 7f6487a6a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64800fd040 0x7f648010d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:31.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.519+0000 7f6487a6a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648010d5b0 con 0x7f64800fd040 2024-03-22T07:08:31.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.519+0000 7f6485806700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64800fd040 0x7f648010d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:31.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.519+0000 7f6485806700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64800fd040 0x7f648010d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.69:39362/0 (socket says 172.21.15.69:39362) 2024-03-22T07:08:31.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.519+0000 7f6485806700 1 -- 172.21.15.69:0/2209115338 learned_addr learned my addr 172.21.15.69:0/2209115338 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:08:31.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.519+0000 7f6485806700 1 -- 172.21.15.69:0/2209115338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6470005040 con 0x7f64800fd040 2024-03-22T07:08:31.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.520+0000 7f6485806700 1 --2- 172.21.15.69:0/2209115338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64800fd040 0x7f648010d070 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6470009680 tx=0x7f6470009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:31.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.520+0000 7f64767fc700 1 -- 172.21.15.69:0/2209115338 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f647000b5c0 con 0x7f64800fd040 2024-03-22T07:08:31.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.520+0000 7f6487a6a700 1 -- 172.21.15.69:0/2209115338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648010d7b0 con 0x7f64800fd040 2024-03-22T07:08:31.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.520+0000 7f64767fc700 1 -- 172.21.15.69:0/2209115338 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f647000be70 con 0x7f64800fd040 2024-03-22T07:08:31.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.520+0000 7f64767fc700 1 -- 172.21.15.69:0/2209115338 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f64700135a0 con 0x7f64800fd040 2024-03-22T07:08:31.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.520+0000 7f6487a6a700 1 -- 172.21.15.69:0/2209115338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648010e320 con 0x7f64800fd040 2024-03-22T07:08:31.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.520+0000 7f64767fc700 1 -- 172.21.15.69:0/2209115338 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7f647001c430 con 0x7f64800fd040 2024-03-22T07:08:31.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.521+0000 7f6487a6a700 1 -- 172.21.15.69:0/2209115338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6480101e60 con 0x7f64800fd040 2024-03-22T07:08:31.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.521+0000 7f64767fc700 1 --2- 172.21.15.69:0/2209115338 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f646c036880 0x7f646c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:31.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.521+0000 7f64767fc700 1 -- 172.21.15.69:0/2209115338 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f6470049420 con 0x7f64800fd040 2024-03-22T07:08:31.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.521+0000 7f6485005700 1 --2- 172.21.15.69:0/2209115338 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f646c036880 0x7f646c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:31.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.521+0000 7f6485005700 1 --2- 172.21.15.69:0/2209115338 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f646c036880 0x7f646c038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f647c009990 tx=0x7f647c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:31.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.526+0000 7f64767fc700 1 -- 172.21.15.69:0/2209115338 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6470011bd0 con 0x7f64800fd040 2024-03-22T07:08:31.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.787+0000 7f6487a6a700 1 -- 172.21.15.69:0/2209115338 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f648010e560 con 0x7f64800fd040 2024-03-22T07:08:31.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.787+0000 7f64767fc700 1 -- 172.21.15.69:0/2209115338 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f6470021070 con 0x7f64800fd040 2024-03-22T07:08:31.787 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:08:31.788 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","modified":"2024-03-22T07:06:35.169231Z","created":"2024-03-22T07:06:35.169231Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:08:31.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.789+0000 7f6487a6a700 1 -- 172.21.15.69:0/2209115338 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f646c036880 msgr2=0x7f646c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:31.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.789+0000 7f6487a6a700 1 --2- 172.21.15.69:0/2209115338 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f646c036880 0x7f646c038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f647c009990 tx=0x7f647c006e40 comp rx=0 tx=0).stop 2024-03-22T07:08:31.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.790+0000 7f6487a6a700 1 -- 172.21.15.69:0/2209115338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64800fd040 msgr2=0x7f648010d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:31.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.790+0000 7f6487a6a700 1 --2- 172.21.15.69:0/2209115338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64800fd040 0x7f648010d070 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6470009680 tx=0x7f6470009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:31.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.790+0000 7f6487a6a700 1 -- 172.21.15.69:0/2209115338 shutdown_connections 2024-03-22T07:08:31.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.790+0000 7f6487a6a700 1 --2- 172.21.15.69:0/2209115338 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f646c036880 0x7f646c038d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:31.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.790+0000 7f6487a6a700 1 --2- 172.21.15.69:0/2209115338 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64800fd040 0x7f648010d070 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:31.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.790+0000 7f6487a6a700 1 -- 172.21.15.69:0/2209115338 >> 172.21.15.69:0/2209115338 conn(0x7f64800f70c0 msgr2=0x7f6480065900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:31.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.790+0000 7f6487a6a700 1 -- 172.21.15.69:0/2209115338 shutdown_connections 2024-03-22T07:08:31.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:31.790+0000 7f6487a6a700 1 -- 172.21.15.69:0/2209115338 wait complete. 2024-03-22T07:08:31.791 INFO:teuthology.orchestra.run.smithi069.stderr:dumped monmap epoch 1 2024-03-22T07:08:32.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:32 smithi063 ceph-mon[29200]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:08:32.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:32 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/2209115338' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:08:33.531 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:08:33.532 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:08:33.779 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:33.825 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:34.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:34 smithi063 ceph-mon[29200]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:08:35.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.297+0000 7fe3c2dfd700 1 -- 172.21.15.69:0/2431588680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3bc0fcf70 msgr2=0x7fe3bc0fd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.297+0000 7fe3c2dfd700 1 --2- 172.21.15.69:0/2431588680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3bc0fcf70 0x7fe3bc0fd380 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40048c0 tx=0x7fe3b4009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.298+0000 7fe3c2dfd700 1 -- 172.21.15.69:0/2431588680 shutdown_connections 2024-03-22T07:08:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.298+0000 7fe3c2dfd700 1 --2- 172.21.15.69:0/2431588680 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3bc0fcf70 0x7fe3bc0fd380 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.298+0000 7fe3c2dfd700 1 -- 172.21.15.69:0/2431588680 >> 172.21.15.69:0/2431588680 conn(0x7fe3bc0f6850 msgr2=0x7fe3bc0f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.298+0000 7fe3c2dfd700 1 -- 172.21.15.69:0/2431588680 shutdown_connections 2024-03-22T07:08:35.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.298+0000 7fe3c2dfd700 1 -- 172.21.15.69:0/2431588680 wait complete. 2024-03-22T07:08:35.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.298+0000 7fe3c2dfd700 1 Processor -- start 2024-03-22T07:08:35.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.298+0000 7fe3c2dfd700 1 -- start start 2024-03-22T07:08:35.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.298+0000 7fe3c2dfd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3bc0fcf70 0x7fe3bc0fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:35.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.298+0000 7fe3c2dfd700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc10fde0 con 0x7fe3bc0fcf70 2024-03-22T07:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.299+0000 7fe3c0b99700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3bc0fcf70 0x7fe3bc0fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.299+0000 7fe3c0b99700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3bc0fcf70 0x7fe3bc0fb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.69:39372/0 (socket says 172.21.15.69:39372) 2024-03-22T07:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.299+0000 7fe3c0b99700 1 -- 172.21.15.69:0/3222876248 learned_addr learned my addr 172.21.15.69:0/3222876248 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.299+0000 7fe3c0b99700 1 -- 172.21.15.69:0/3222876248 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b4005040 con 0x7fe3bc0fcf70 2024-03-22T07:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.299+0000 7fe3c0b99700 1 --2- 172.21.15.69:0/3222876248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3bc0fcf70 0x7fe3bc0fb720 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40096b0 tx=0x7fe3b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.299+0000 7fe3b97fa700 1 -- 172.21.15.69:0/3222876248 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe3b400bed0 con 0x7fe3bc0fcf70 2024-03-22T07:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.299+0000 7fe3c2dfd700 1 -- 172.21.15.69:0/3222876248 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3bc10ff20 con 0x7fe3bc0fcf70 2024-03-22T07:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.299+0000 7fe3b97fa700 1 -- 172.21.15.69:0/3222876248 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3b400b5c0 con 0x7fe3bc0fcf70 2024-03-22T07:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.299+0000 7fe3c2dfd700 1 -- 172.21.15.69:0/3222876248 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3bc110410 con 0x7fe3bc0fcf70 2024-03-22T07:08:35.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.299+0000 7fe3b97fa700 1 -- 172.21.15.69:0/3222876248 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe3b401c5b0 con 0x7fe3bc0fcf70 2024-03-22T07:08:35.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.300+0000 7fe3c2dfd700 1 -- 172.21.15.69:0/3222876248 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3bc105920 con 0x7fe3bc0fcf70 2024-03-22T07:08:35.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.300+0000 7fe3b97fa700 1 -- 172.21.15.69:0/3222876248 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 13) v1 ==== 43424+0+0 (secure 0 0 0) 0x7fe3b4013a80 con 0x7fe3bc0fcf70 2024-03-22T07:08:35.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.300+0000 7fe3b97fa700 1 --2- 172.21.15.69:0/3222876248 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fe3a80368d0 0x7fe3a8038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:35.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.300+0000 7fe3b97fa700 1 -- 172.21.15.69:0/3222876248 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe3b4049a80 con 0x7fe3bc0fcf70 2024-03-22T07:08:35.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.301+0000 7fe3bbfff700 1 --2- 172.21.15.69:0/3222876248 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fe3a80368d0 0x7fe3a8038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:35.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.301+0000 7fe3bbfff700 1 --2- 172.21.15.69:0/3222876248 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fe3a80368d0 0x7fe3a8038d80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00ad90 tx=0x7fe3ac009410 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:35.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.305+0000 7fe3b97fa700 1 -- 172.21.15.69:0/3222876248 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe3b4011550 con 0x7fe3bc0fcf70 2024-03-22T07:08:35.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.564+0000 7fe3c2dfd700 1 -- 172.21.15.69:0/3222876248 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe3bc05f5f0 con 0x7fe3bc0fcf70 2024-03-22T07:08:35.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.565+0000 7fe3b97fa700 1 -- 172.21.15.69:0/3222876248 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fe3b4011370 con 0x7fe3bc0fcf70 2024-03-22T07:08:35.565 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:08:35.565 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","modified":"2024-03-22T07:06:35.169231Z","created":"2024-03-22T07:06:35.169231Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:08:35.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.567+0000 7fe3c2dfd700 1 -- 172.21.15.69:0/3222876248 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fe3a80368d0 msgr2=0x7fe3a8038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:35.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.567+0000 7fe3c2dfd700 1 --2- 172.21.15.69:0/3222876248 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fe3a80368d0 0x7fe3a8038d80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00ad90 tx=0x7fe3ac009410 comp rx=0 tx=0).stop 2024-03-22T07:08:35.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.567+0000 7fe3c2dfd700 1 -- 172.21.15.69:0/3222876248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3bc0fcf70 msgr2=0x7fe3bc0fb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:35.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.567+0000 7fe3c2dfd700 1 --2- 172.21.15.69:0/3222876248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3bc0fcf70 0x7fe3bc0fb720 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40096b0 tx=0x7fe3b4009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:35.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.567+0000 7fe3c2dfd700 1 -- 172.21.15.69:0/3222876248 shutdown_connections 2024-03-22T07:08:35.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.567+0000 7fe3c2dfd700 1 --2- 172.21.15.69:0/3222876248 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7fe3a80368d0 0x7fe3a8038d80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:35.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.567+0000 7fe3c2dfd700 1 --2- 172.21.15.69:0/3222876248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe3bc0fcf70 0x7fe3bc0fb720 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:35.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.567+0000 7fe3c2dfd700 1 -- 172.21.15.69:0/3222876248 >> 172.21.15.69:0/3222876248 conn(0x7fe3bc0f6850 msgr2=0x7fe3bc102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:35.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.567+0000 7fe3c2dfd700 1 -- 172.21.15.69:0/3222876248 shutdown_connections 2024-03-22T07:08:35.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:35.567+0000 7fe3c2dfd700 1 -- 172.21.15.69:0/3222876248 wait complete. 2024-03-22T07:08:35.567 INFO:teuthology.orchestra.run.smithi069.stderr:dumped monmap epoch 1 2024-03-22T07:08:36.425 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:36 smithi063 ceph-mon[29200]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:08:36.425 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:36 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/3222876248' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:08:37.173 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:08:37.174 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:08:37.422 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:37.469 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:37 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:37 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:37 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:37 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-22T07:08:38.767 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:38 smithi063 ceph-mon[29200]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-22T07:08:38.768 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:38 smithi063 ceph-mon[29200]: mgrmap e14: smithi063.mnxayh(active, since 56s) 2024-03-22T07:08:38.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.923+0000 7f2f621f4700 1 -- 172.21.15.69:0/2187581796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5c0fca60 msgr2=0x7f2f5c0fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:38.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.923+0000 7f2f621f4700 1 --2- 172.21.15.69:0/2187581796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5c0fca60 0x7f2f5c0fce70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0048c0 tx=0x7f2f4c009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:38.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.924+0000 7f2f621f4700 1 -- 172.21.15.69:0/2187581796 shutdown_connections 2024-03-22T07:08:38.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.924+0000 7f2f621f4700 1 --2- 172.21.15.69:0/2187581796 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5c0fca60 0x7f2f5c0fce70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:38.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.924+0000 7f2f621f4700 1 -- 172.21.15.69:0/2187581796 >> 172.21.15.69:0/2187581796 conn(0x7f2f5c0f81c0 msgr2=0x7f2f5c0fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:38.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.924+0000 7f2f621f4700 1 -- 172.21.15.69:0/2187581796 shutdown_connections 2024-03-22T07:08:38.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.924+0000 7f2f621f4700 1 -- 172.21.15.69:0/2187581796 wait complete. 2024-03-22T07:08:38.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.924+0000 7f2f621f4700 1 Processor -- start 2024-03-22T07:08:38.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.924+0000 7f2f621f4700 1 -- start start 2024-03-22T07:08:38.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.924+0000 7f2f621f4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5c0fca60 0x7f2f5c10d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:38.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.924+0000 7f2f621f4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c10d5d0 con 0x7f2f5c0fca60 2024-03-22T07:08:38.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.925+0000 7f2f5b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5c0fca60 0x7f2f5c10d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:38.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.925+0000 7f2f5b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5c0fca60 0x7f2f5c10d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.69:39386/0 (socket says 172.21.15.69:39386) 2024-03-22T07:08:38.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.925+0000 7f2f5b7fe700 1 -- 172.21.15.69:0/3184526927 learned_addr learned my addr 172.21.15.69:0/3184526927 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:08:38.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.925+0000 7f2f5b7fe700 1 -- 172.21.15.69:0/3184526927 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c005040 con 0x7f2f5c0fca60 2024-03-22T07:08:38.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.925+0000 7f2f5b7fe700 1 --2- 172.21.15.69:0/3184526927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5c0fca60 0x7f2f5c10d090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009680 tx=0x7f2f4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:38.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.925+0000 7f2f43fff700 1 -- 172.21.15.69:0/3184526927 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2f4c00b5c0 con 0x7f2f5c0fca60 2024-03-22T07:08:38.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.925+0000 7f2f43fff700 1 -- 172.21.15.69:0/3184526927 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f4c00be70 con 0x7f2f5c0fca60 2024-03-22T07:08:38.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.925+0000 7f2f621f4700 1 -- 172.21.15.69:0/3184526927 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f5c10d7d0 con 0x7f2f5c0fca60 2024-03-22T07:08:38.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.925+0000 7f2f43fff700 1 -- 172.21.15.69:0/3184526927 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2f4c0135a0 con 0x7f2f5c0fca60 2024-03-22T07:08:38.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.925+0000 7f2f621f4700 1 -- 172.21.15.69:0/3184526927 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f5c10e340 con 0x7f2f5c0fca60 2024-03-22T07:08:38.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.926+0000 7f2f43fff700 1 -- 172.21.15.69:0/3184526927 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7f2f4c01c430 con 0x7f2f5c0fca60 2024-03-22T07:08:38.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.926+0000 7f2f621f4700 1 -- 172.21.15.69:0/3184526927 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f5c102f70 con 0x7f2f5c0fca60 2024-03-22T07:08:38.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.926+0000 7f2f43fff700 1 --2- 172.21.15.69:0/3184526927 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f2f440368d0 0x7f2f44038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:38.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.926+0000 7f2f43fff700 1 -- 172.21.15.69:0/3184526927 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2f4c0494b0 con 0x7f2f5c0fca60 2024-03-22T07:08:38.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.926+0000 7f2f5affd700 1 -- 172.21.15.69:0/3184526927 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f2f440368d0 msgr2=0x7f2f44038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2296442540 2024-03-22T07:08:38.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.926+0000 7f2f5affd700 1 --2- 172.21.15.69:0/3184526927 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f2f440368d0 0x7f2f44038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:08:38.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:38.931+0000 7f2f43fff700 1 -- 172.21.15.69:0/3184526927 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2f4c0117e0 con 0x7f2f5c0fca60 2024-03-22T07:08:39.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:39.127+0000 7f2f5affd700 1 -- 172.21.15.69:0/3184526927 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f2f440368d0 msgr2=0x7f2f44038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2296442540 2024-03-22T07:08:39.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:39.127+0000 7f2f5affd700 1 --2- 172.21.15.69:0/3184526927 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f2f440368d0 0x7f2f44038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T07:08:39.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:39.192+0000 7f2f621f4700 1 -- 172.21.15.69:0/3184526927 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2f5c10ed50 con 0x7f2f5c0fca60 2024-03-22T07:08:39.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:39.193+0000 7f2f43fff700 1 -- 172.21.15.69:0/3184526927 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f2f4c021070 con 0x7f2f5c0fca60 2024-03-22T07:08:39.193 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:08:39.193 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","modified":"2024-03-22T07:06:35.169231Z","created":"2024-03-22T07:06:35.169231Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:08:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:39.195+0000 7f2f621f4700 1 -- 172.21.15.69:0/3184526927 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f2f440368d0 msgr2=0x7f2f44038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:08:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:39.195+0000 7f2f621f4700 1 --2- 172.21.15.69:0/3184526927 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f2f440368d0 0x7f2f44038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:39.195+0000 7f2f621f4700 1 -- 172.21.15.69:0/3184526927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5c0fca60 msgr2=0x7f2f5c10d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:39.195+0000 7f2f621f4700 1 --2- 172.21.15.69:0/3184526927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5c0fca60 0x7f2f5c10d090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009680 tx=0x7f2f4c009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:39.195+0000 7f2f621f4700 1 -- 172.21.15.69:0/3184526927 shutdown_connections 2024-03-22T07:08:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:39.195+0000 7f2f621f4700 1 --2- 172.21.15.69:0/3184526927 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f2f440368d0 0x7f2f44038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:39.195+0000 7f2f621f4700 1 --2- 172.21.15.69:0/3184526927 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2f5c0fca60 0x7f2f5c10d090 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:39.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:39.195+0000 7f2f621f4700 1 -- 172.21.15.69:0/3184526927 >> 172.21.15.69:0/3184526927 conn(0x7f2f5c0f81c0 msgr2=0x7f2f5c103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:39.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:39.195+0000 7f2f621f4700 1 -- 172.21.15.69:0/3184526927 shutdown_connections 2024-03-22T07:08:39.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:39.195+0000 7f2f621f4700 1 -- 172.21.15.69:0/3184526927 wait complete. 2024-03-22T07:08:39.196 INFO:teuthology.orchestra.run.smithi069.stderr:dumped monmap epoch 1 2024-03-22T07:08:39.614 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:39 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/3184526927' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:08:40.783 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:08:40.783 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:08:41.033 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:41.080 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:42.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.461+0000 7f14d30f5700 1 -- 172.21.15.69:0/2711004337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14cc0fca60 msgr2=0x7f14cc0fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:42.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.461+0000 7f14d30f5700 1 --2- 172.21.15.69:0/2711004337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14cc0fca60 0x7f14cc0fce70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f14c00048c0 tx=0x7f14c0009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:42.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.461+0000 7f14d30f5700 1 -- 172.21.15.69:0/2711004337 shutdown_connections 2024-03-22T07:08:42.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.461+0000 7f14d30f5700 1 --2- 172.21.15.69:0/2711004337 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14cc0fca60 0x7f14cc0fce70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:42.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.461+0000 7f14d30f5700 1 -- 172.21.15.69:0/2711004337 >> 172.21.15.69:0/2711004337 conn(0x7f14cc0f81c0 msgr2=0x7f14cc0fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:42.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.461+0000 7f14d30f5700 1 -- 172.21.15.69:0/2711004337 shutdown_connections 2024-03-22T07:08:42.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.461+0000 7f14d30f5700 1 -- 172.21.15.69:0/2711004337 wait complete. 2024-03-22T07:08:42.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.461+0000 7f14d30f5700 1 Processor -- start 2024-03-22T07:08:42.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.461+0000 7f14d30f5700 1 -- start start 2024-03-22T07:08:42.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.462+0000 7f14d30f5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14cc0fca60 0x7f14cc10d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:42.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.462+0000 7f14d30f5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc10d5d0 con 0x7f14cc0fca60 2024-03-22T07:08:42.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.462+0000 7f14d0e91700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14cc0fca60 0x7f14cc10d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:42.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.462+0000 7f14d0e91700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14cc0fca60 0x7f14cc10d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.69:43148/0 (socket says 172.21.15.69:43148) 2024-03-22T07:08:42.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.462+0000 7f14d0e91700 1 -- 172.21.15.69:0/3143386876 learned_addr learned my addr 172.21.15.69:0/3143386876 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:08:42.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.462+0000 7f14d0e91700 1 -- 172.21.15.69:0/3143386876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c0005040 con 0x7f14cc0fca60 2024-03-22T07:08:42.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.462+0000 7f14d0e91700 1 --2- 172.21.15.69:0/3143386876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14cc0fca60 0x7f14cc10d090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f14c00096b0 tx=0x7f14c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:42.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.462+0000 7f14c97fa700 1 -- 172.21.15.69:0/3143386876 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f14c000bed0 con 0x7f14cc0fca60 2024-03-22T07:08:42.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.462+0000 7f14c97fa700 1 -- 172.21.15.69:0/3143386876 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f14c000b5c0 con 0x7f14cc0fca60 2024-03-22T07:08:42.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.462+0000 7f14d30f5700 1 -- 172.21.15.69:0/3143386876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14cc10d7d0 con 0x7f14cc0fca60 2024-03-22T07:08:42.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.462+0000 7f14c97fa700 1 -- 172.21.15.69:0/3143386876 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f14c001c5b0 con 0x7f14cc0fca60 2024-03-22T07:08:42.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.462+0000 7f14d30f5700 1 -- 172.21.15.69:0/3143386876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14cc10e340 con 0x7f14cc0fca60 2024-03-22T07:08:42.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.463+0000 7f14c97fa700 1 -- 172.21.15.69:0/3143386876 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 14) v1 ==== 43438+0+0 (secure 0 0 0) 0x7f14c0013420 con 0x7f14cc0fca60 2024-03-22T07:08:42.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.463+0000 7f14d30f5700 1 -- 172.21.15.69:0/3143386876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14cc102f70 con 0x7f14cc0fca60 2024-03-22T07:08:42.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.463+0000 7f14c97fa700 1 --2- 172.21.15.69:0/3143386876 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f14bc036920 0x7f14bc038dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:42.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.464+0000 7f14c97fa700 1 -- 172.21.15.69:0/3143386876 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f14c0049a50 con 0x7f14cc0fca60 2024-03-22T07:08:42.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.464+0000 7f14cbfff700 1 -- 172.21.15.69:0/3143386876 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f14bc036920 msgr2=0x7f14bc038dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2296442540 2024-03-22T07:08:42.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.464+0000 7f14cbfff700 1 --2- 172.21.15.69:0/3143386876 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f14bc036920 0x7f14bc038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:08:42.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.468+0000 7f14c97fa700 1 -- 172.21.15.69:0/3143386876 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f14c0025400 con 0x7f14cc0fca60 2024-03-22T07:08:42.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.665+0000 7f14cbfff700 1 -- 172.21.15.69:0/3143386876 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f14bc036920 msgr2=0x7f14bc038dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.63:6800/2296442540 2024-03-22T07:08:42.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.665+0000 7f14cbfff700 1 --2- 172.21.15.69:0/3143386876 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f14bc036920 0x7f14bc038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-22T07:08:42.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.729+0000 7f14d30f5700 1 -- 172.21.15.69:0/3143386876 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f14cc10ed50 con 0x7f14cc0fca60 2024-03-22T07:08:42.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.730+0000 7f14c97fa700 1 -- 172.21.15.69:0/3143386876 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f14c0004d30 con 0x7f14cc0fca60 2024-03-22T07:08:42.731 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:08:42.731 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","modified":"2024-03-22T07:06:35.169231Z","created":"2024-03-22T07:06:35.169231Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:08:42.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.731+0000 7f14d30f5700 1 -- 172.21.15.69:0/3143386876 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f14bc036920 msgr2=0x7f14bc038dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:08:42.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.731+0000 7f14d30f5700 1 --2- 172.21.15.69:0/3143386876 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f14bc036920 0x7f14bc038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:42.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.731+0000 7f14d30f5700 1 -- 172.21.15.69:0/3143386876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14cc0fca60 msgr2=0x7f14cc10d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:42.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.731+0000 7f14d30f5700 1 --2- 172.21.15.69:0/3143386876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14cc0fca60 0x7f14cc10d090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f14c00096b0 tx=0x7f14c0009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:42.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.731+0000 7f14d30f5700 1 -- 172.21.15.69:0/3143386876 shutdown_connections 2024-03-22T07:08:42.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.731+0000 7f14d30f5700 1 --2- 172.21.15.69:0/3143386876 >> [v2:172.21.15.63:6800/2296442540,v1:172.21.15.63:6801/2296442540] conn(0x7f14bc036920 0x7f14bc038dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:42.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.732+0000 7f14d30f5700 1 --2- 172.21.15.69:0/3143386876 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14cc0fca60 0x7f14cc10d090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:42.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.732+0000 7f14d30f5700 1 -- 172.21.15.69:0/3143386876 >> 172.21.15.69:0/3143386876 conn(0x7f14cc0f81c0 msgr2=0x7f14cc103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:42.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.732+0000 7f14d30f5700 1 -- 172.21.15.69:0/3143386876 shutdown_connections 2024-03-22T07:08:42.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:42.732+0000 7f14d30f5700 1 -- 172.21.15.69:0/3143386876 wait complete. 2024-03-22T07:08:42.734 INFO:teuthology.orchestra.run.smithi069.stderr:dumped monmap epoch 1 2024-03-22T07:08:42.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:42 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/3143386876' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:08:44.253 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:08:44.254 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:08:44.509 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:44.556 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:44.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:44 smithi063 ceph-mon[29200]: Active manager daemon smithi063.mnxayh restarted 2024-03-22T07:08:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:44 smithi063 ceph-mon[29200]: Activating manager daemon smithi063.mnxayh 2024-03-22T07:08:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:44 smithi063 ceph-mon[29200]: osdmap e5: 0 total, 0 up, 0 in 2024-03-22T07:08:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:44 smithi063 ceph-mon[29200]: mgrmap e15: smithi063.mnxayh(active, starting, since 0.00515753s) 2024-03-22T07:08:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:08:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr metadata", "who": "smithi063.mnxayh", "id": "smithi063.mnxayh"}]: dispatch 2024-03-22T07:08:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:08:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:08:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:08:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:44 smithi063 ceph-mon[29200]: Manager daemon smithi063.mnxayh is now available 2024-03-22T07:08:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:08:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:08:45.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:45 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:45.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:45 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:08:45.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:45 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.mnxayh/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:08:45.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:45 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.mnxayh/trash_purge_schedule"}]: dispatch 2024-03-22T07:08:45.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:45 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:45.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:45 smithi063 ceph-mon[29200]: mgrmap e16: smithi063.mnxayh(active, since 1.00536s) 2024-03-22T07:08:46.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.046+0000 7f5911d07700 1 -- 172.21.15.69:0/1361021483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f590c0fb960 msgr2=0x7f590c0fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:46.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.046+0000 7f5911d07700 1 --2- 172.21.15.69:0/1361021483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f590c0fb960 0x7f590c0fbd70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0048c0 tx=0x7f58fc009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:46.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.046+0000 7f5911d07700 1 -- 172.21.15.69:0/1361021483 shutdown_connections 2024-03-22T07:08:46.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.046+0000 7f5911d07700 1 --2- 172.21.15.69:0/1361021483 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f590c0fb960 0x7f590c0fbd70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:46.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.046+0000 7f5911d07700 1 -- 172.21.15.69:0/1361021483 >> 172.21.15.69:0/1361021483 conn(0x7f590c0f70c0 msgr2=0x7f590c0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.046+0000 7f5911d07700 1 -- 172.21.15.69:0/1361021483 shutdown_connections 2024-03-22T07:08:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.046+0000 7f5911d07700 1 -- 172.21.15.69:0/1361021483 wait complete. 2024-03-22T07:08:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.046+0000 7f5911d07700 1 Processor -- start 2024-03-22T07:08:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.046+0000 7f5911d07700 1 -- start start 2024-03-22T07:08:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.046+0000 7f5911d07700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f590c0fb960 0x7f590c10d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:46.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.046+0000 7f5911d07700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c10d630 con 0x7f590c0fb960 2024-03-22T07:08:46.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.047+0000 7f590b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f590c0fb960 0x7f590c10d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:46.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.047+0000 7f590b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f590c0fb960 0x7f590c10d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.69:43172/0 (socket says 172.21.15.69:43172) 2024-03-22T07:08:46.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.047+0000 7f590b7fe700 1 -- 172.21.15.69:0/3051513311 learned_addr learned my addr 172.21.15.69:0/3051513311 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:08:46.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.047+0000 7f590b7fe700 1 -- 172.21.15.69:0/3051513311 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58fc005040 con 0x7f590c0fb960 2024-03-22T07:08:46.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.047+0000 7f590b7fe700 1 --2- 172.21.15.69:0/3051513311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f590c0fb960 0x7f590c10d0f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009300 tx=0x7f58fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:46.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.047+0000 7f58f3fff700 1 -- 172.21.15.69:0/3051513311 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f58fc004e60 con 0x7f590c0fb960 2024-03-22T07:08:46.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.047+0000 7f58f3fff700 1 -- 172.21.15.69:0/3051513311 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58fc009680 con 0x7f590c0fb960 2024-03-22T07:08:46.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.047+0000 7f5911d07700 1 -- 172.21.15.69:0/3051513311 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590c10d830 con 0x7f590c0fb960 2024-03-22T07:08:46.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.047+0000 7f58f3fff700 1 -- 172.21.15.69:0/3051513311 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f58fc01c3f0 con 0x7f590c0fb960 2024-03-22T07:08:46.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.047+0000 7f5911d07700 1 -- 172.21.15.69:0/3051513311 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590c10e3a0 con 0x7f590c0fb960 2024-03-22T07:08:46.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.048+0000 7f58f3fff700 1 -- 172.21.15.69:0/3051513311 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 16) v1 ==== 43289+0+0 (secure 0 0 0) 0x7f58fc026440 con 0x7f590c0fb960 2024-03-22T07:08:46.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.048+0000 7f58f3fff700 1 --2- 172.21.15.69:0/3051513311 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f58f4036710 0x7f58f4038bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:46.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.048+0000 7f58f3fff700 1 -- 172.21.15.69:0/3051513311 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f58fc049320 con 0x7f590c0fb960 2024-03-22T07:08:46.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.048+0000 7f5911d07700 1 -- 172.21.15.69:0/3051513311 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590c101e70 con 0x7f590c0fb960 2024-03-22T07:08:46.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.049+0000 7f590affd700 1 --2- 172.21.15.69:0/3051513311 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f58f4036710 0x7f58f4038bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:46.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.049+0000 7f590affd700 1 --2- 172.21.15.69:0/3051513311 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f58f4036710 0x7f58f4038bc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5900009990 tx=0x7f5900006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:46.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.053+0000 7f58f3fff700 1 -- 172.21.15.69:0/3051513311 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f58fc021070 con 0x7f590c0fb960 2024-03-22T07:08:46.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.320+0000 7f5911d07700 1 -- 172.21.15.69:0/3051513311 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f590c1017a0 con 0x7f590c0fb960 2024-03-22T07:08:46.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.321+0000 7f58f3fff700 1 -- 172.21.15.69:0/3051513311 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f58fc0123c0 con 0x7f590c0fb960 2024-03-22T07:08:46.322 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:08:46.322 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","modified":"2024-03-22T07:06:35.169231Z","created":"2024-03-22T07:06:35.169231Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:08:46.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.323+0000 7f5911d07700 1 -- 172.21.15.69:0/3051513311 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f58f4036710 msgr2=0x7f58f4038bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:46.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.323+0000 7f5911d07700 1 --2- 172.21.15.69:0/3051513311 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f58f4036710 0x7f58f4038bc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5900009990 tx=0x7f5900006e40 comp rx=0 tx=0).stop 2024-03-22T07:08:46.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.323+0000 7f5911d07700 1 -- 172.21.15.69:0/3051513311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f590c0fb960 msgr2=0x7f590c10d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:46.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.323+0000 7f5911d07700 1 --2- 172.21.15.69:0/3051513311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f590c0fb960 0x7f590c10d0f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009300 tx=0x7f58fc009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:46.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.323+0000 7f5911d07700 1 -- 172.21.15.69:0/3051513311 shutdown_connections 2024-03-22T07:08:46.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.323+0000 7f5911d07700 1 --2- 172.21.15.69:0/3051513311 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f58f4036710 0x7f58f4038bc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:46.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.323+0000 7f5911d07700 1 --2- 172.21.15.69:0/3051513311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f590c0fb960 0x7f590c10d0f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:46.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.323+0000 7f5911d07700 1 -- 172.21.15.69:0/3051513311 >> 172.21.15.69:0/3051513311 conn(0x7f590c0f70c0 msgr2=0x7f590c065d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:46.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.323+0000 7f5911d07700 1 -- 172.21.15.69:0/3051513311 shutdown_connections 2024-03-22T07:08:46.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:46.323+0000 7f5911d07700 1 -- 172.21.15.69:0/3051513311 wait complete. 2024-03-22T07:08:46.325 INFO:teuthology.orchestra.run.smithi069.stderr:dumped monmap epoch 1 2024-03-22T07:08:46.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:46 smithi063 ceph-mon[29200]: [22/Mar/2024:07:08:45] ENGINE Bus STARTING 2024-03-22T07:08:46.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:46 smithi063 ceph-mon[29200]: [22/Mar/2024:07:08:45] ENGINE Serving on https://172.21.15.63:7150 2024-03-22T07:08:46.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:46 smithi063 ceph-mon[29200]: [22/Mar/2024:07:08:45] ENGINE Bus STARTED 2024-03-22T07:08:46.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:46 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:46.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:46 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:46.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:46 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/3051513311' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:08:47.879 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:08:47.879 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:08:48.211 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:48.541 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-22T07:08:48.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:48 smithi063 ceph-mon[29200]: mgrmap e17: smithi063.mnxayh(active, since 2s) 2024-03-22T07:08:48.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:48 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:49.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:49.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:49.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:49.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:49.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd/host:smithi063", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:08:50.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.294+0000 7fc3177be700 1 -- 172.21.15.69:0/2126845117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc310066640 msgr2=0x7fc3100ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:50.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.294+0000 7fc3177be700 1 --2- 172.21.15.69:0/2126845117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc310066640 0x7fc3100ff6f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc3080048c0 tx=0x7fc308009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:50.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.294+0000 7fc3177be700 1 -- 172.21.15.69:0/2126845117 shutdown_connections 2024-03-22T07:08:50.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.294+0000 7fc3177be700 1 --2- 172.21.15.69:0/2126845117 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc310066640 0x7fc3100ff6f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:50.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.294+0000 7fc3177be700 1 -- 172.21.15.69:0/2126845117 >> 172.21.15.69:0/2126845117 conn(0x7fc3100f70c0 msgr2=0x7fc3100f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:50.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.294+0000 7fc3177be700 1 -- 172.21.15.69:0/2126845117 shutdown_connections 2024-03-22T07:08:50.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.294+0000 7fc3177be700 1 -- 172.21.15.69:0/2126845117 wait complete. 2024-03-22T07:08:50.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.295+0000 7fc3177be700 1 Processor -- start 2024-03-22T07:08:50.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.295+0000 7fc3177be700 1 -- start start 2024-03-22T07:08:50.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.295+0000 7fc3177be700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc310066640 0x7fc31010d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:50.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.295+0000 7fc3177be700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31010d600 con 0x7fc310066640 2024-03-22T07:08:50.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.296+0000 7fc31555a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc310066640 0x7fc31010d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:50.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.296+0000 7fc31555a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc310066640 0x7fc31010d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.69:43186/0 (socket says 172.21.15.69:43186) 2024-03-22T07:08:50.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.296+0000 7fc31555a700 1 -- 172.21.15.69:0/2864023619 learned_addr learned my addr 172.21.15.69:0/2864023619 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:08:50.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.296+0000 7fc31555a700 1 -- 172.21.15.69:0/2864023619 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc308005040 con 0x7fc310066640 2024-03-22T07:08:50.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.296+0000 7fc31555a700 1 --2- 172.21.15.69:0/2864023619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc310066640 0x7fc31010d0c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc308009680 tx=0x7fc308009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:50.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.296+0000 7fc305ffb700 1 -- 172.21.15.69:0/2864023619 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc30800bed0 con 0x7fc310066640 2024-03-22T07:08:50.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.296+0000 7fc3177be700 1 -- 172.21.15.69:0/2864023619 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc31010d800 con 0x7fc310066640 2024-03-22T07:08:50.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.296+0000 7fc305ffb700 1 -- 172.21.15.69:0/2864023619 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc30800b5c0 con 0x7fc310066640 2024-03-22T07:08:50.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.296+0000 7fc305ffb700 1 -- 172.21.15.69:0/2864023619 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc30801c5b0 con 0x7fc310066640 2024-03-22T07:08:50.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.296+0000 7fc3177be700 1 -- 172.21.15.69:0/2864023619 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc31010e370 con 0x7fc310066640 2024-03-22T07:08:50.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.297+0000 7fc305ffb700 1 -- 172.21.15.69:0/2864023619 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7fc30801c710 con 0x7fc310066640 2024-03-22T07:08:50.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.297+0000 7fc3177be700 1 -- 172.21.15.69:0/2864023619 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc310102f50 con 0x7fc310066640 2024-03-22T07:08:50.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.298+0000 7fc305ffb700 1 --2- 172.21.15.69:0/2864023619 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fc2fc036970 0x7fc2fc038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:50.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.298+0000 7fc305ffb700 1 -- 172.21.15.69:0/2864023619 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fc308049c00 con 0x7fc310066640 2024-03-22T07:08:50.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.298+0000 7fc314d59700 1 --2- 172.21.15.69:0/2864023619 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fc2fc036970 0x7fc2fc038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:50.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.298+0000 7fc314d59700 1 --2- 172.21.15.69:0/2864023619 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fc2fc036970 0x7fc2fc038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc300009990 tx=0x7fc300006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:50.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.303+0000 7fc305ffb700 1 -- 172.21.15.69:0/2864023619 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc308011550 con 0x7fc310066640 2024-03-22T07:08:50.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.608+0000 7fc3177be700 1 -- 172.21.15.69:0/2864023619 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc31010ed80 con 0x7fc310066640 2024-03-22T07:08:50.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.609+0000 7fc305ffb700 1 -- 172.21.15.69:0/2864023619 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fc308004d30 con 0x7fc310066640 2024-03-22T07:08:50.609 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:08:50.609 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","modified":"2024-03-22T07:06:35.169231Z","created":"2024-03-22T07:06:35.169231Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:08:50.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.611+0000 7fc3177be700 1 -- 172.21.15.69:0/2864023619 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fc2fc036970 msgr2=0x7fc2fc038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:50.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.611+0000 7fc3177be700 1 --2- 172.21.15.69:0/2864023619 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fc2fc036970 0x7fc2fc038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc300009990 tx=0x7fc300006e40 comp rx=0 tx=0).stop 2024-03-22T07:08:50.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.611+0000 7fc3177be700 1 -- 172.21.15.69:0/2864023619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc310066640 msgr2=0x7fc31010d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:50.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.611+0000 7fc3177be700 1 --2- 172.21.15.69:0/2864023619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc310066640 0x7fc31010d0c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc308009680 tx=0x7fc308009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:50.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.611+0000 7fc3177be700 1 -- 172.21.15.69:0/2864023619 shutdown_connections 2024-03-22T07:08:50.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.611+0000 7fc3177be700 1 --2- 172.21.15.69:0/2864023619 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fc2fc036970 0x7fc2fc038e20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:50.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.611+0000 7fc3177be700 1 --2- 172.21.15.69:0/2864023619 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc310066640 0x7fc31010d0c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:50.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.611+0000 7fc3177be700 1 -- 172.21.15.69:0/2864023619 >> 172.21.15.69:0/2864023619 conn(0x7fc3100f70c0 msgr2=0x7fc310103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:50.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.611+0000 7fc3177be700 1 -- 172.21.15.69:0/2864023619 shutdown_connections 2024-03-22T07:08:50.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:50.611+0000 7fc3177be700 1 -- 172.21.15.69:0/2864023619 wait complete. 2024-03-22T07:08:50.612 INFO:teuthology.orchestra.run.smithi069.stderr:dumped monmap epoch 1 2024-03-22T07:08:51.122 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:50 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/2864023619' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:08:52.859 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:08:52.860 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:08:52.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:52 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:52.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:52 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:52.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:52 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:52.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:52 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:52.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:52 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd/host:smithi069", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:08:52.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:52 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:08:52.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:52 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:08:52.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:52 smithi063 ceph-mon[29200]: Updating smithi063:/etc/ceph/ceph.conf 2024-03-22T07:08:52.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:52 smithi063 ceph-mon[29200]: Updating smithi069:/etc/ceph/ceph.conf 2024-03-22T07:08:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:52 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:52 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:52 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:52 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:52 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:52 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi069", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:08:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:52 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi069", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-22T07:08:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:52 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:08:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:08:53.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:53 smithi063 ceph-mon[29200]: Updating smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:08:53.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:53 smithi063 ceph-mon[29200]: Updating smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:08:53.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:53 smithi063 ceph-mon[29200]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:08:53.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:53 smithi063 ceph-mon[29200]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:08:53.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:53 smithi063 ceph-mon[29200]: Updating smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:08:53.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:53 smithi063 ceph-mon[29200]: Updating smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:08:53.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:53 smithi063 ceph-mon[29200]: Deploying daemon crash.smithi069 on smithi069 2024-03-22T07:08:55.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.120+0000 7f0a706f9700 1 -- 172.21.15.69:0/3871021713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a68102ae0 msgr2=0x7f0a68102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:55.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.120+0000 7f0a706f9700 1 --2- 172.21.15.69:0/3871021713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a68102ae0 0x7f0a68102eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0a640048c0 tx=0x7f0a64009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:55.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.120+0000 7f0a706f9700 1 -- 172.21.15.69:0/3871021713 shutdown_connections 2024-03-22T07:08:55.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.120+0000 7f0a706f9700 1 --2- 172.21.15.69:0/3871021713 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a68102ae0 0x7f0a68102eb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:55.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.120+0000 7f0a706f9700 1 -- 172.21.15.69:0/3871021713 >> 172.21.15.69:0/3871021713 conn(0x7f0a680f85e0 msgr2=0x7f0a680fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:55.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.120+0000 7f0a706f9700 1 -- 172.21.15.69:0/3871021713 shutdown_connections 2024-03-22T07:08:55.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.120+0000 7f0a706f9700 1 -- 172.21.15.69:0/3871021713 wait complete. 2024-03-22T07:08:55.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.121+0000 7f0a706f9700 1 Processor -- start 2024-03-22T07:08:55.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.121+0000 7f0a706f9700 1 -- start start 2024-03-22T07:08:55.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.121+0000 7f0a706f9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a68102ae0 0x7f0a681131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:55.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.121+0000 7f0a706f9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a68113720 con 0x7f0a68102ae0 2024-03-22T07:08:55.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.121+0000 7f0a6e495700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a68102ae0 0x7f0a681131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:55.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.121+0000 7f0a6e495700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a68102ae0 0x7f0a681131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.69:35044/0 (socket says 172.21.15.69:35044) 2024-03-22T07:08:55.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.121+0000 7f0a6e495700 1 -- 172.21.15.69:0/780988051 learned_addr learned my addr 172.21.15.69:0/780988051 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:08:55.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.121+0000 7f0a6e495700 1 -- 172.21.15.69:0/780988051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a64005040 con 0x7f0a68102ae0 2024-03-22T07:08:55.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.122+0000 7f0a6e495700 1 --2- 172.21.15.69:0/780988051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a68102ae0 0x7f0a681131e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400b820 tx=0x7f0a64009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:55.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.122+0000 7f0a5effd700 1 -- 172.21.15.69:0/780988051 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0a640093c0 con 0x7f0a68102ae0 2024-03-22T07:08:55.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.122+0000 7f0a5effd700 1 -- 172.21.15.69:0/780988051 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a64013420 con 0x7f0a68102ae0 2024-03-22T07:08:55.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.122+0000 7f0a706f9700 1 -- 172.21.15.69:0/780988051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a68113920 con 0x7f0a68102ae0 2024-03-22T07:08:55.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.122+0000 7f0a5effd700 1 -- 172.21.15.69:0/780988051 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0a6401c5b0 con 0x7f0a68102ae0 2024-03-22T07:08:55.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.122+0000 7f0a706f9700 1 -- 172.21.15.69:0/780988051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a68113d90 con 0x7f0a68102ae0 2024-03-22T07:08:55.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.123+0000 7f0a5effd700 1 -- 172.21.15.69:0/780988051 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f0a64013a90 con 0x7f0a68102ae0 2024-03-22T07:08:55.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.123+0000 7f0a706f9700 1 -- 172.21.15.69:0/780988051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a680fe1e0 con 0x7f0a68102ae0 2024-03-22T07:08:55.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.123+0000 7f0a5effd700 1 --2- 172.21.15.69:0/780988051 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0a54036970 0x7f0a54038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:55.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.123+0000 7f0a5effd700 1 -- 172.21.15.69:0/780988051 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f0a64049a90 con 0x7f0a68102ae0 2024-03-22T07:08:55.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.123+0000 7f0a6dc94700 1 --2- 172.21.15.69:0/780988051 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0a54036970 0x7f0a54038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:55.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.124+0000 7f0a6dc94700 1 --2- 172.21.15.69:0/780988051 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0a54036970 0x7f0a54038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a58009990 tx=0x7f0a58006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:55.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.128+0000 7f0a5effd700 1 -- 172.21.15.69:0/780988051 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0a64011960 con 0x7f0a68102ae0 2024-03-22T07:08:55.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.420+0000 7f0a706f9700 1 -- 172.21.15.69:0/780988051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0a6804bed0 con 0x7f0a68102ae0 2024-03-22T07:08:55.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.421+0000 7f0a5effd700 1 -- 172.21.15.69:0/780988051 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f0a64011370 con 0x7f0a68102ae0 2024-03-22T07:08:55.421 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:08:55.421 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","modified":"2024-03-22T07:06:35.169231Z","created":"2024-03-22T07:06:35.169231Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:08:55.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.423+0000 7f0a706f9700 1 -- 172.21.15.69:0/780988051 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0a54036970 msgr2=0x7f0a54038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:55.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.423+0000 7f0a706f9700 1 --2- 172.21.15.69:0/780988051 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0a54036970 0x7f0a54038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a58009990 tx=0x7f0a58006e40 comp rx=0 tx=0).stop 2024-03-22T07:08:55.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.423+0000 7f0a706f9700 1 -- 172.21.15.69:0/780988051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a68102ae0 msgr2=0x7f0a681131e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:55.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.423+0000 7f0a706f9700 1 --2- 172.21.15.69:0/780988051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a68102ae0 0x7f0a681131e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400b820 tx=0x7f0a64009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:55.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.423+0000 7f0a706f9700 1 -- 172.21.15.69:0/780988051 shutdown_connections 2024-03-22T07:08:55.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.423+0000 7f0a706f9700 1 --2- 172.21.15.69:0/780988051 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0a54036970 0x7f0a54038e20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:55.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.423+0000 7f0a706f9700 1 --2- 172.21.15.69:0/780988051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0a68102ae0 0x7f0a681131e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:55.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.423+0000 7f0a706f9700 1 -- 172.21.15.69:0/780988051 >> 172.21.15.69:0/780988051 conn(0x7f0a680f85e0 msgr2=0x7f0a680f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:55.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.423+0000 7f0a706f9700 1 -- 172.21.15.69:0/780988051 shutdown_connections 2024-03-22T07:08:55.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:55.423+0000 7f0a706f9700 1 -- 172.21.15.69:0/780988051 wait complete. 2024-03-22T07:08:55.424 INFO:teuthology.orchestra.run.smithi069.stderr:dumped monmap epoch 1 2024-03-22T07:08:55.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:55 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/780988051' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:08:56.927 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:08:56.927 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:08:57.187 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:08:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:57 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:57 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:57 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:57 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:08:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:08:57 smithi063 ceph-mon[29200]: Deploying daemon node-exporter.smithi069 on smithi069 2024-03-22T07:08:59.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.559+0000 7fa251811700 1 -- 172.21.15.69:0/2425700720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa24c102bd0 msgr2=0x7fa24c102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:59.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.559+0000 7fa251811700 1 --2- 172.21.15.69:0/2425700720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa24c102bd0 0x7fa24c102fa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa23c0048c0 tx=0x7fa23c009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:59.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.559+0000 7fa251811700 1 -- 172.21.15.69:0/2425700720 shutdown_connections 2024-03-22T07:08:59.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.559+0000 7fa251811700 1 --2- 172.21.15.69:0/2425700720 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa24c102bd0 0x7fa24c102fa0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:59.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.559+0000 7fa251811700 1 -- 172.21.15.69:0/2425700720 >> 172.21.15.69:0/2425700720 conn(0x7fa24c0f85e0 msgr2=0x7fa24c0fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:59.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.559+0000 7fa251811700 1 -- 172.21.15.69:0/2425700720 shutdown_connections 2024-03-22T07:08:59.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.559+0000 7fa251811700 1 -- 172.21.15.69:0/2425700720 wait complete. 2024-03-22T07:08:59.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.560+0000 7fa251811700 1 Processor -- start 2024-03-22T07:08:59.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.560+0000 7fa251811700 1 -- start start 2024-03-22T07:08:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.560+0000 7fa251811700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa24c102bd0 0x7fa24c113180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.560+0000 7fa251811700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c1136c0 con 0x7fa24c102bd0 2024-03-22T07:08:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.560+0000 7fa24affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa24c102bd0 0x7fa24c113180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.560+0000 7fa24affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa24c102bd0 0x7fa24c113180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.69:35072/0 (socket says 172.21.15.69:35072) 2024-03-22T07:08:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.560+0000 7fa24affd700 1 -- 172.21.15.69:0/3806770256 learned_addr learned my addr 172.21.15.69:0/3806770256 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:08:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.561+0000 7fa24affd700 1 -- 172.21.15.69:0/3806770256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23c005040 con 0x7fa24c102bd0 2024-03-22T07:08:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.561+0000 7fa24affd700 1 --2- 172.21.15.69:0/3806770256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa24c102bd0 0x7fa24c113180 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa23c00b820 tx=0x7fa23c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.561+0000 7fa2337fe700 1 -- 172.21.15.69:0/3806770256 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa23c0093c0 con 0x7fa24c102bd0 2024-03-22T07:08:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.561+0000 7fa2337fe700 1 -- 172.21.15.69:0/3806770256 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa23c013420 con 0x7fa24c102bd0 2024-03-22T07:08:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.561+0000 7fa251811700 1 -- 172.21.15.69:0/3806770256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa24c1138c0 con 0x7fa24c102bd0 2024-03-22T07:08:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.561+0000 7fa2337fe700 1 -- 172.21.15.69:0/3806770256 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa23c01c5b0 con 0x7fa24c102bd0 2024-03-22T07:08:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.561+0000 7fa251811700 1 -- 172.21.15.69:0/3806770256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa24c113db0 con 0x7fa24c102bd0 2024-03-22T07:08:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.562+0000 7fa2337fe700 1 -- 172.21.15.69:0/3806770256 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7fa23c013a90 con 0x7fa24c102bd0 2024-03-22T07:08:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.562+0000 7fa251811700 1 -- 172.21.15.69:0/3806770256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa24c05f5f0 con 0x7fa24c102bd0 2024-03-22T07:08:59.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.562+0000 7fa2337fe700 1 --2- 172.21.15.69:0/3806770256 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa234036970 0x7fa234038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:08:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.562+0000 7fa2337fe700 1 -- 172.21.15.69:0/3806770256 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fa23c049a70 con 0x7fa24c102bd0 2024-03-22T07:08:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.562+0000 7fa24a7fc700 1 --2- 172.21.15.69:0/3806770256 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa234036970 0x7fa234038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:08:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.563+0000 7fa24a7fc700 1 --2- 172.21.15.69:0/3806770256 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa234036970 0x7fa234038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa240009990 tx=0x7fa240006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:08:59.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.567+0000 7fa2337fe700 1 -- 172.21.15.69:0/3806770256 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa23c011960 con 0x7fa24c102bd0 2024-03-22T07:08:59.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.831+0000 7fa251811700 1 -- 172.21.15.69:0/3806770256 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa24c10e880 con 0x7fa24c102bd0 2024-03-22T07:08:59.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.831+0000 7fa2337fe700 1 -- 172.21.15.69:0/3806770256 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fa23c011370 con 0x7fa24c102bd0 2024-03-22T07:08:59.831 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:08:59.831 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","modified":"2024-03-22T07:06:35.169231Z","created":"2024-03-22T07:06:35.169231Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:08:59.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.833+0000 7fa251811700 1 -- 172.21.15.69:0/3806770256 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa234036970 msgr2=0x7fa234038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:59.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.833+0000 7fa251811700 1 --2- 172.21.15.69:0/3806770256 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa234036970 0x7fa234038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa240009990 tx=0x7fa240006e40 comp rx=0 tx=0).stop 2024-03-22T07:08:59.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.833+0000 7fa251811700 1 -- 172.21.15.69:0/3806770256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa24c102bd0 msgr2=0x7fa24c113180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:08:59.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.833+0000 7fa251811700 1 --2- 172.21.15.69:0/3806770256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa24c102bd0 0x7fa24c113180 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa23c00b820 tx=0x7fa23c009e30 comp rx=0 tx=0).stop 2024-03-22T07:08:59.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.833+0000 7fa251811700 1 -- 172.21.15.69:0/3806770256 shutdown_connections 2024-03-22T07:08:59.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.833+0000 7fa251811700 1 --2- 172.21.15.69:0/3806770256 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa234036970 0x7fa234038e20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:59.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.833+0000 7fa251811700 1 --2- 172.21.15.69:0/3806770256 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa24c102bd0 0x7fa24c113180 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:08:59.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.833+0000 7fa251811700 1 -- 172.21.15.69:0/3806770256 >> 172.21.15.69:0/3806770256 conn(0x7fa24c0f85e0 msgr2=0x7fa24c0f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:08:59.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.833+0000 7fa251811700 1 -- 172.21.15.69:0/3806770256 shutdown_connections 2024-03-22T07:08:59.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:08:59.833+0000 7fa251811700 1 -- 172.21.15.69:0/3806770256 wait complete. 2024-03-22T07:08:59.834 INFO:teuthology.orchestra.run.smithi069.stderr:dumped monmap epoch 1 2024-03-22T07:09:00.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:00 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:00.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:00 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:09:00.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:00 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/3806770256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:09:01.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:01 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:01.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:01 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:01.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:01 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:01.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:01 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:01.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:01 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi069.gfziwh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:09:01.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:01 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi069.gfziwh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-22T07:09:01.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:01 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:09:01.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:01 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:01.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:01 smithi063 ceph-mon[29200]: Deploying daemon mgr.smithi069.gfziwh on smithi069 2024-03-22T07:09:01.762 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:09:01.762 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:09:02.128 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:09:03.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.741+0000 7f5f2cab0700 1 -- 172.21.15.69:0/722609763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f281019d0 msgr2=0x7f5f28101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:03.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.741+0000 7f5f2cab0700 1 --2- 172.21.15.69:0/722609763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f281019d0 0x7f5f28101da0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009c60 tx=0x7f5f20004d10 comp rx=0 tx=0).stop 2024-03-22T07:09:03.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.741+0000 7f5f2cab0700 1 -- 172.21.15.69:0/722609763 shutdown_connections 2024-03-22T07:09:03.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.741+0000 7f5f2cab0700 1 --2- 172.21.15.69:0/722609763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f281019d0 0x7f5f28101da0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:03.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.741+0000 7f5f2cab0700 1 -- 172.21.15.69:0/722609763 >> 172.21.15.69:0/722609763 conn(0x7f5f28071800 msgr2=0x7f5f28071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:03.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.741+0000 7f5f2cab0700 1 -- 172.21.15.69:0/722609763 shutdown_connections 2024-03-22T07:09:03.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.741+0000 7f5f2cab0700 1 -- 172.21.15.69:0/722609763 wait complete. 2024-03-22T07:09:03.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.742+0000 7f5f2cab0700 1 Processor -- start 2024-03-22T07:09:03.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.742+0000 7f5f2cab0700 1 -- start start 2024-03-22T07:09:03.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.742+0000 7f5f2cab0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f281019d0 0x7f5f2810e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:03.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.742+0000 7f5f2cab0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2810e630 con 0x7f5f281019d0 2024-03-22T07:09:03.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.742+0000 7f5f2659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f281019d0 0x7f5f2810e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:03.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.742+0000 7f5f2659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f281019d0 0x7f5f2810e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.69:41556/0 (socket says 172.21.15.69:41556) 2024-03-22T07:09:03.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.742+0000 7f5f2659c700 1 -- 172.21.15.69:0/2607940850 learned_addr learned my addr 172.21.15.69:0/2607940850 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:09:03.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.743+0000 7f5f2659c700 1 -- 172.21.15.69:0/2607940850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f20009940 con 0x7f5f281019d0 2024-03-22T07:09:03.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.743+0000 7f5f2659c700 1 --2- 172.21.15.69:0/2607940850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f281019d0 0x7f5f2810e0f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000b740 tx=0x7f5f20004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:03.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.743+0000 7f5f1effd700 1 -- 172.21.15.69:0/2607940850 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5f20007b30 con 0x7f5f281019d0 2024-03-22T07:09:03.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.743+0000 7f5f2cab0700 1 -- 172.21.15.69:0/2607940850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2810e830 con 0x7f5f281019d0 2024-03-22T07:09:03.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.743+0000 7f5f1effd700 1 -- 172.21.15.69:0/2607940850 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f2001d910 con 0x7f5f281019d0 2024-03-22T07:09:03.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.743+0000 7f5f1effd700 1 -- 172.21.15.69:0/2607940850 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5f200128b0 con 0x7f5f281019d0 2024-03-22T07:09:03.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.743+0000 7f5f2cab0700 1 -- 172.21.15.69:0/2607940850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2810ed20 con 0x7f5f281019d0 2024-03-22T07:09:03.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.744+0000 7f5f2cab0700 1 -- 172.21.15.69:0/2607940850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f28100c40 con 0x7f5f281019d0 2024-03-22T07:09:03.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.744+0000 7f5f1effd700 1 -- 172.21.15.69:0/2607940850 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 17) v1 ==== 43481+0+0 (secure 0 0 0) 0x7f5f20019040 con 0x7f5f281019d0 2024-03-22T07:09:03.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.744+0000 7f5f1effd700 1 --2- 172.21.15.69:0/2607940850 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f5f14036970 0x7f5f14038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:03.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.744+0000 7f5f1effd700 1 -- 172.21.15.69:0/2607940850 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f5f2004a3f0 con 0x7f5f281019d0 2024-03-22T07:09:03.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.744+0000 7f5f25d9b700 1 --2- 172.21.15.69:0/2607940850 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f5f14036970 0x7f5f14038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:03.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.745+0000 7f5f25d9b700 1 --2- 172.21.15.69:0/2607940850 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f5f14036970 0x7f5f14038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009990 tx=0x7f5f10006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:03.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:03.749+0000 7f5f1effd700 1 -- 172.21.15.69:0/2607940850 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5f2002ed20 con 0x7f5f281019d0 2024-03-22T07:09:04.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:04.023+0000 7f5f2cab0700 1 -- 172.21.15.69:0/2607940850 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5f28102ad0 con 0x7f5f281019d0 2024-03-22T07:09:04.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:04.023+0000 7f5f1effd700 1 -- 172.21.15.69:0/2607940850 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f5f28102ad0 con 0x7f5f281019d0 2024-03-22T07:09:04.024 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:09:04.024 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","modified":"2024-03-22T07:06:35.169231Z","created":"2024-03-22T07:06:35.169231Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-22T07:09:04.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:04.025+0000 7f5f2cab0700 1 -- 172.21.15.69:0/2607940850 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f5f14036970 msgr2=0x7f5f14038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:04.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:04.025+0000 7f5f2cab0700 1 --2- 172.21.15.69:0/2607940850 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f5f14036970 0x7f5f14038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009990 tx=0x7f5f10006e40 comp rx=0 tx=0).stop 2024-03-22T07:09:04.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:04.025+0000 7f5f2cab0700 1 -- 172.21.15.69:0/2607940850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f281019d0 msgr2=0x7f5f2810e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:04.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:04.025+0000 7f5f2cab0700 1 --2- 172.21.15.69:0/2607940850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f281019d0 0x7f5f2810e0f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000b740 tx=0x7f5f20004d10 comp rx=0 tx=0).stop 2024-03-22T07:09:04.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:04.025+0000 7f5f2cab0700 1 -- 172.21.15.69:0/2607940850 shutdown_connections 2024-03-22T07:09:04.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:04.025+0000 7f5f2cab0700 1 --2- 172.21.15.69:0/2607940850 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f5f14036970 0x7f5f14038e20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:04.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:04.025+0000 7f5f2cab0700 1 --2- 172.21.15.69:0/2607940850 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5f281019d0 0x7f5f2810e0f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:04.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:04.025+0000 7f5f2cab0700 1 -- 172.21.15.69:0/2607940850 >> 172.21.15.69:0/2607940850 conn(0x7f5f28071800 msgr2=0x7f5f281035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:04.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:04.025+0000 7f5f2cab0700 1 -- 172.21.15.69:0/2607940850 shutdown_connections 2024-03-22T07:09:04.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:04.025+0000 7f5f2cab0700 1 -- 172.21.15.69:0/2607940850 wait complete. 2024-03-22T07:09:04.026 INFO:teuthology.orchestra.run.smithi069.stderr:dumped monmap epoch 1 2024-03-22T07:09:04.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:04 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/2607940850' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:09:05.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:05 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:05.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:05 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:05.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:05 smithi063 ceph-mon[29200]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:05.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:05 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:05.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:05 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:05.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:05 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:09:05.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:05 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:05.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:05 smithi063 ceph-mon[29200]: Deploying daemon mon.smithi069 on smithi069 2024-03-22T07:09:05.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:05 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:05.752 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-22T07:09:05.752 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-22T07:09:06.964 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi069/config 2024-03-22T07:09:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:07 smithi063 ceph-mon[29200]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:08.709 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 podman[36384]: 2024-03-22 07:09:08.689825279 +0000 UTC m=+0.954605547 container start 95f6103f760352f41574150c918cb0bb72c0c36d938ccef5e7764ceecdb35a07 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi069, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=quincy-9c4e0c3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-22T07:09:08.709 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: set uid:gid to 167:167 (ceph:ceph) 2024-03-22T07:09:08.709 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable), process ceph-mon, pid 2 2024-03-22T07:09:08.709 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pidfile_write: ignore empty --pid-file 2024-03-22T07:09:09.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: load: jerasure load: lrc 2024-03-22T07:09:09.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: RocksDB version: 6.15.5 2024-03-22T07:09:09.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-03-22T07:09:09.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Compile date Mar 21 2024 2024-03-22T07:09:09.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: DB SUMMARY 2024-03-22T07:09:09.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: DB Session ID: KROEJX3YOJTQHOUZUDRV 2024-03-22T07:09:09.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: CURRENT file: CURRENT 2024-03-22T07:09:09.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: IDENTITY file: IDENTITY 2024-03-22T07:09:09.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-03-22T07:09:09.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi069/store.db dir, Total Num: 0, files: 2024-03-22T07:09:09.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi069/store.db: 000004.log size: 511 ; 2024-03-22T07:09:09.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.error_if_exists: 0 2024-03-22T07:09:09.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.create_if_missing: 0 2024-03-22T07:09:09.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.paranoid_checks: 1 2024-03-22T07:09:09.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-22T07:09:09.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.env: 0x55a82c144a80 2024-03-22T07:09:09.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.fs: Posix File System 2024-03-22T07:09:09.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.info_log: 0x55a82cf8dc00 2024-03-22T07:09:09.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_file_opening_threads: 16 2024-03-22T07:09:09.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.statistics: (nil) 2024-03-22T07:09:09.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.use_fsync: 0 2024-03-22T07:09:09.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_log_file_size: 0 2024-03-22T07:09:09.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-22T07:09:09.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-22T07:09:09.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.keep_log_file_num: 1000 2024-03-22T07:09:09.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.recycle_log_file_num: 0 2024-03-22T07:09:09.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.allow_fallocate: 1 2024-03-22T07:09:09.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.allow_mmap_reads: 0 2024-03-22T07:09:09.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.allow_mmap_writes: 0 2024-03-22T07:09:09.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.use_direct_reads: 0 2024-03-22T07:09:09.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-22T07:09:09.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.create_missing_column_families: 0 2024-03-22T07:09:09.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.db_log_dir: 2024-03-22T07:09:09.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi069/store.db 2024-03-22T07:09:09.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-22T07:09:09.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-22T07:09:09.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-22T07:09:09.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-22T07:09:09.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-22T07:09:09.033 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-22T07:09:09.033 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.advise_random_on_open: 1 2024-03-22T07:09:09.033 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.db_write_buffer_size: 0 2024-03-22T07:09:09.033 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.write_buffer_manager: 0x55a82dd02ab0 2024-03-22T07:09:09.033 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-22T07:09:09.033 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-03-22T07:09:09.033 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-22T07:09:09.033 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-22T07:09:09.033 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.rate_limiter: (nil) 2024-03-22T07:09:09.034 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-22T07:09:09.034 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.wal_recovery_mode: 2 2024-03-22T07:09:09.034 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.enable_thread_tracking: 0 2024-03-22T07:09:09.034 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.enable_pipelined_write: 0 2024-03-22T07:09:09.034 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.unordered_write: 0 2024-03-22T07:09:09.034 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-22T07:09:09.034 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-22T07:09:09.034 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-22T07:09:09.034 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-22T07:09:09.034 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.row_cache: None 2024-03-22T07:09:09.034 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.wal_filter: None 2024-03-22T07:09:09.034 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-22T07:09:09.034 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.allow_ingest_behind: 0 2024-03-22T07:09:09.034 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.preserve_deletes: 0 2024-03-22T07:09:09.034 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.two_write_queues: 0 2024-03-22T07:09:09.035 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.manual_wal_flush: 0 2024-03-22T07:09:09.035 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.atomic_flush: 0 2024-03-22T07:09:09.035 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-22T07:09:09.035 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-22T07:09:09.035 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-22T07:09:09.035 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.log_readahead_size: 0 2024-03-22T07:09:09.035 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-22T07:09:09.035 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.best_efforts_recovery: 0 2024-03-22T07:09:09.035 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-22T07:09:09.035 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-22T07:09:09.035 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.allow_data_in_errors: 0 2024-03-22T07:09:09.035 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.db_host_id: __hostname__ 2024-03-22T07:09:09.035 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_background_jobs: 2 2024-03-22T07:09:09.035 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_background_compactions: -1 2024-03-22T07:09:09.035 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_subcompactions: 1 2024-03-22T07:09:09.035 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-22T07:09:09.036 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-22T07:09:09.036 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-22T07:09:09.036 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_total_wal_size: 0 2024-03-22T07:09:09.036 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-22T07:09:09.036 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-22T07:09:09.036 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-22T07:09:09.036 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-22T07:09:09.036 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_open_files: -1 2024-03-22T07:09:09.036 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.bytes_per_sync: 0 2024-03-22T07:09:09.036 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-22T07:09:09.036 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-22T07:09:09.036 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compaction_readahead_size: 0 2024-03-22T07:09:09.036 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_background_flushes: -1 2024-03-22T07:09:09.036 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Compression algorithms supported: 2024-03-22T07:09:09.036 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-22T07:09:09.036 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: kZSTD supported: 0 2024-03-22T07:09:09.036 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: kXpressCompression supported: 0 2024-03-22T07:09:09.037 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: kLZ4HCCompression supported: 1 2024-03-22T07:09:09.037 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: kZlibCompression supported: 1 2024-03-22T07:09:09.037 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: kSnappyCompression supported: 1 2024-03-22T07:09:09.037 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: kLZ4Compression supported: 1 2024-03-22T07:09:09.037 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: kBZip2Compression supported: 0 2024-03-22T07:09:09.037 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-22T07:09:09.037 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi069/store.db/MANIFEST-000003 2024-03-22T07:09:09.037 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-03-22T07:09:09.037 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-22T07:09:09.037 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.merge_operator: 2024-03-22T07:09:09.037 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compaction_filter: None 2024-03-22T07:09:09.037 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compaction_filter_factory: None 2024-03-22T07:09:09.037 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.sst_partitioner_factory: None 2024-03-22T07:09:09.037 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-22T07:09:09.037 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-22T07:09:09.037 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a82cfe2f58) 2024-03-22T07:09:09.037 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: cache_index_and_filter_blocks: 1 2024-03-22T07:09:09.038 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-22T07:09:09.038 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-22T07:09:09.038 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: pin_top_level_index_and_filter: 1 2024-03-22T07:09:09.038 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: index_type: 0 2024-03-22T07:09:09.038 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: data_block_index_type: 0 2024-03-22T07:09:09.038 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: index_shortening: 1 2024-03-22T07:09:09.038 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-22T07:09:09.038 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: hash_index_allow_collision: 1 2024-03-22T07:09:09.038 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: checksum: 1 2024-03-22T07:09:09.038 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: no_block_cache: 0 2024-03-22T07:09:09.038 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: block_cache: 0x55a82d015090 2024-03-22T07:09:09.038 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: block_cache_name: BinnedLRUCache 2024-03-22T07:09:09.038 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: block_cache_options: 2024-03-22T07:09:09.038 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: capacity : 536870912 2024-03-22T07:09:09.038 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: num_shard_bits : 4 2024-03-22T07:09:09.038 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: strict_capacity_limit : 0 2024-03-22T07:09:09.039 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: high_pri_pool_ratio: 0.000 2024-03-22T07:09:09.039 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: block_cache_compressed: (nil) 2024-03-22T07:09:09.039 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: persistent_cache: (nil) 2024-03-22T07:09:09.039 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: block_size: 4096 2024-03-22T07:09:09.039 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: block_size_deviation: 10 2024-03-22T07:09:09.039 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: block_restart_interval: 16 2024-03-22T07:09:09.039 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: index_block_restart_interval: 1 2024-03-22T07:09:09.039 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: metadata_block_size: 4096 2024-03-22T07:09:09.039 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: partition_filters: 0 2024-03-22T07:09:09.039 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: use_delta_encoding: 1 2024-03-22T07:09:09.039 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-03-22T07:09:09.039 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: whole_key_filtering: 1 2024-03-22T07:09:09.039 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: verify_compression: 0 2024-03-22T07:09:09.039 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: read_amp_bytes_per_bit: 0 2024-03-22T07:09:09.039 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: format_version: 4 2024-03-22T07:09:09.039 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: enable_index_compression: 1 2024-03-22T07:09:09.039 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: block_align: 0 2024-03-22T07:09:09.039 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.write_buffer_size: 33554432 2024-03-22T07:09:09.040 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_write_buffer_number: 2 2024-03-22T07:09:09.040 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compression: NoCompression 2024-03-22T07:09:09.040 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.bottommost_compression: Disabled 2024-03-22T07:09:09.040 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.prefix_extractor: nullptr 2024-03-22T07:09:09.040 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-22T07:09:09.040 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.num_levels: 7 2024-03-22T07:09:09.040 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-22T07:09:09.040 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-22T07:09:09.040 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-22T07:09:09.040 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-22T07:09:09.040 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-22T07:09:09.040 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-22T07:09:09.040 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-22T07:09:09.040 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-22T07:09:09.040 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-22T07:09:09.040 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-22T07:09:09.040 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-22T07:09:09.041 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compression_opts.level: 32767 2024-03-22T07:09:09.041 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compression_opts.strategy: 0 2024-03-22T07:09:09.041 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-22T07:09:09.041 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-22T07:09:09.041 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-22T07:09:09.041 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compression_opts.enabled: false 2024-03-22T07:09:09.041 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-22T07:09:09.041 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-22T07:09:09.041 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-22T07:09:09.041 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.target_file_size_base: 67108864 2024-03-22T07:09:09.041 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-22T07:09:09.041 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-22T07:09:09.041 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-22T07:09:09.041 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-22T07:09:09.041 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-22T07:09:09.041 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-22T07:09:09.041 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-22T07:09:09.041 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-22T07:09:09.042 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-22T07:09:09.042 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-22T07:09:09.042 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-22T07:09:09.042 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-22T07:09:09.042 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-22T07:09:09.042 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.arena_block_size: 4194304 2024-03-22T07:09:09.042 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-22T07:09:09.042 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-22T07:09:09.042 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-03-22T07:09:09.042 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.disable_auto_compactions: 0 2024-03-22T07:09:09.042 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-22T07:09:09.042 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-22T07:09:09.042 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-22T07:09:09.042 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-22T07:09:09.042 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-22T07:09:09.042 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-22T07:09:09.042 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-22T07:09:09.043 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-22T07:09:09.043 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-22T07:09:09.043 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-22T07:09:09.043 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.table_properties_collectors: 2024-03-22T07:09:09.043 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.inplace_update_support: 0 2024-03-22T07:09:09.043 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-22T07:09:09.043 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-22T07:09:09.043 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-22T07:09:09.043 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-22T07:09:09.043 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.bloom_locality: 0 2024-03-22T07:09:09.043 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.max_successive_merges: 0 2024-03-22T07:09:09.043 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-22T07:09:09.043 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.paranoid_file_checks: 0 2024-03-22T07:09:09.043 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.force_consistency_checks: 1 2024-03-22T07:09:09.043 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.report_bg_io_stats: 0 2024-03-22T07:09:09.044 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.ttl: 2592000 2024-03-22T07:09:09.044 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-22T07:09:09.044 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.enable_blob_files: false 2024-03-22T07:09:09.044 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.min_blob_size: 0 2024-03-22T07:09:09.044 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.blob_file_size: 268435456 2024-03-22T07:09:09.044 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-22T07:09:09.044 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-22T07:09:09.044 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-22T07:09:09.044 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi069/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-22T07:09:09.044 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-03-22T07:09:09.044 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-03-22T07:09:09.044 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711091348715282, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-22T07:09:09.044 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-03-22T07:09:09.044 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-03-22T07:09:09.044 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711091348715577, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1545, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1711091348, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "6504fbbe-97cc-47e0-a0dd-5d7a6c3f3001", "db_session_id": "KROEJX3YOJTQHOUZUDRV"}} 2024-03-22T07:09:09.044 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-03-22T07:09:09.045 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711091348715878, "job": 1, "event": "recovery_finished"} 2024-03-22T07:09:09.045 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi069/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-22T07:09:09.045 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55a82cfa9180 2024-03-22T07:09:09.045 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: DB pointer 0x55a82d07e000 2024-03-22T07:09:09.045 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-03-22T07:09:09.045 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-03-22T07:09:09.045 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: ** DB Stats ** 2024-03-22T07:09:09.045 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T07:09:09.045 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-22T07:09:09.045 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T07:09:09.045 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T07:09:09.045 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-22T07:09:09.045 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-03-22T07:09:09.045 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T07:09:09.045 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: 2024-03-22T07:09:09.046 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: ** Compaction Stats [default] ** 2024-03-22T07:09:09.046 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-22T07:09:09.046 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T07:09:09.046 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-03-22T07:09:09.046 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-03-22T07:09:09.046 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-03-22T07:09:09.046 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: 2024-03-22T07:09:09.046 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: ** Compaction Stats [default] ** 2024-03-22T07:09:09.046 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-22T07:09:09.046 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T07:09:09.046 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-03-22T07:09:09.046 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T07:09:09.046 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-22T07:09:09.046 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-22T07:09:09.046 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-22T07:09:09.047 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-22T07:09:09.047 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-22T07:09:09.047 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Cumulative compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T07:09:09.047 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Interval compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T07:09:09.047 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-22T07:09:09.047 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: 2024-03-22T07:09:09.047 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-22T07:09:09.047 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: 2024-03-22T07:09:09.047 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: ** Compaction Stats [default] ** 2024-03-22T07:09:09.047 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-22T07:09:09.047 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T07:09:09.047 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-03-22T07:09:09.047 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-03-22T07:09:09.047 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-03-22T07:09:09.047 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: 2024-03-22T07:09:09.048 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: ** Compaction Stats [default] ** 2024-03-22T07:09:09.048 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-03-22T07:09:09.048 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T07:09:09.048 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-03-22T07:09:09.048 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T07:09:09.048 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-22T07:09:09.048 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-22T07:09:09.048 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-22T07:09:09.048 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-22T07:09:09.048 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-22T07:09:09.048 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Cumulative compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T07:09:09.048 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T07:09:09.048 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-22T07:09:09.048 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: 2024-03-22T07:09:09.048 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-22T07:09:09.049 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mon.smithi069 does not exist in monmap, will attempt to join an existing cluster 2024-03-22T07:09:09.049 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: using public_addr v2:172.21.15.69:0/0 -> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] 2024-03-22T07:09:09.049 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: starting mon.smithi069 rank -1 at public addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] at bind addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi069 fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:09:09.049 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mon.smithi069@-1(???) e0 preinit fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:09:09.049 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mon.smithi069@-1(synchronizing).mds e1 new map 2024-03-22T07:09:09.049 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mon.smithi069@-1(synchronizing).mds e1 print_map 2024-03-22T07:09:09.049 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: e1 2024-03-22T07:09:09.049 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:09:09.049 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:09:09.049 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: legacy client fscid: -1 2024-03-22T07:09:09.049 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: 2024-03-22T07:09:09.049 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: No filesystems configured 2024-03-22T07:09:09.049 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mon.smithi069@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-22T07:09:09.049 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mon.smithi069@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-22T07:09:09.049 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mon.smithi069@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-22T07:09:09.049 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mon.smithi069@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-22T07:09:09.049 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mon.smithi069@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-22T07:09:09.049 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mon.smithi069@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-22T07:09:09.050 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mon.smithi069@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-03-22T07:09:09.050 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mon.smithi069@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-22T07:09:09.050 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mon.smithi069@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-22T07:09:09.050 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mon.smithi069@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-22T07:09:09.050 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mon.smithi069@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-22T07:09:09.050 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Deploying cephadm binary to smithi069 2024-03-22T07:09:09.050 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.050 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.050 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.050 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Added host smithi069 2024-03-22T07:09:09.050 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.050 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.050 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.050 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.050 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.050 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Deploying daemon alertmanager.smithi063 on smithi063 2024-03-22T07:09:09.050 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T07:09:09.051 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.051 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.051 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.051 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.051 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/2586958819' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-22T07:09:09.051 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/2586958819' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-22T07:09:09.051 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: osdmap e4: 0 total, 0 up, 0 in 2024-03-22T07:09:09.051 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.051 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.051 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.051 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.051 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.051 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.051 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.051 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.051 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-22T07:09:09.051 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-22T07:09:09.052 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.052 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Deploying daemon grafana.smithi063 on smithi063 2024-03-22T07:09:09.052 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi063:172.21.15.63=smithi063;smithi069:172.21.15.69=smithi069", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:09.052 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Saving service mon spec with placement smithi063:172.21.15.63=smithi063;smithi069:172.21.15.69=smithi069;count:2 2024-03-22T07:09:09.052 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.052 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.052 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.052 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.052 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/293944724' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:09:09.052 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.052 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/3399898238' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:09:09.052 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.052 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.052 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/2272108453' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:09:09.052 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.052 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.053 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.053 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.053 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.053 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.053 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.053 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.053 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Deploying daemon prometheus.smithi063 on smithi063 2024-03-22T07:09:09.054 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.054 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/934692766' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:09:09.054 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.054 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.054 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/2209115338' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:09:09.054 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.054 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.054 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/3222876248' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:09:09.054 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.054 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.054 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.054 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-22T07:09:09.054 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14162 172.21.15.63:0/4225820256' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-22T07:09:09.054 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mgrmap e14: smithi063.mnxayh(active, since 56s) 2024-03-22T07:09:09.054 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/3184526927' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:09:09.055 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/3143386876' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:09:09.055 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Active manager daemon smithi063.mnxayh restarted 2024-03-22T07:09:09.055 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Activating manager daemon smithi063.mnxayh 2024-03-22T07:09:09.055 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: osdmap e5: 0 total, 0 up, 0 in 2024-03-22T07:09:09.055 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mgrmap e15: smithi063.mnxayh(active, starting, since 0.00515753s) 2024-03-22T07:09:09.055 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:09:09.055 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr metadata", "who": "smithi063.mnxayh", "id": "smithi063.mnxayh"}]: dispatch 2024-03-22T07:09:09.055 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:09:09.055 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:09:09.055 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:09:09.055 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Manager daemon smithi063.mnxayh is now available 2024-03-22T07:09:09.055 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:09:09.055 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:09:09.055 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.055 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:09:09.056 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.mnxayh/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:09:09.056 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.mnxayh/trash_purge_schedule"}]: dispatch 2024-03-22T07:09:09.056 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.056 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mgrmap e16: smithi063.mnxayh(active, since 1.00536s) 2024-03-22T07:09:09.056 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: [22/Mar/2024:07:08:45] ENGINE Bus STARTING 2024-03-22T07:09:09.056 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: [22/Mar/2024:07:08:45] ENGINE Serving on https://172.21.15.63:7150 2024-03-22T07:09:09.056 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: [22/Mar/2024:07:08:45] ENGINE Bus STARTED 2024-03-22T07:09:09.056 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.056 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.056 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/3051513311' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:09:09.056 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mgrmap e17: smithi063.mnxayh(active, since 2s) 2024-03-22T07:09:09.056 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.056 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.056 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.056 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.056 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.056 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd/host:smithi063", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:09:09.056 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/2864023619' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:09:09.057 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.057 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.057 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.057 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.057 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd/host:smithi069", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:09:09.057 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:09.057 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:09:09.057 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Updating smithi063:/etc/ceph/ceph.conf 2024-03-22T07:09:09.057 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Updating smithi069:/etc/ceph/ceph.conf 2024-03-22T07:09:09.057 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.057 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.057 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.057 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.057 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.057 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi069", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:09:09.057 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi069", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-22T07:09:09.057 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:09.058 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Updating smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:09:09.058 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Updating smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:09:09.058 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:09:09.058 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:09:09.058 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Updating smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:09:09.058 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Updating smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:09:09.058 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Deploying daemon crash.smithi069 on smithi069 2024-03-22T07:09:09.058 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/780988051' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:09:09.058 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.058 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.058 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.058 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.058 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Deploying daemon node-exporter.smithi069 on smithi069 2024-03-22T07:09:09.058 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.058 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:09:09.058 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/3806770256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:09:09.058 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.059 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.059 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.059 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.059 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi069.gfziwh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:09:09.059 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi069.gfziwh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-22T07:09:09.059 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:09:09.059 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:09.059 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Deploying daemon mgr.smithi069.gfziwh on smithi069 2024-03-22T07:09:09.059 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/2607940850' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:09:09.059 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.059 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.059 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.059 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.059 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.059 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:09:09.059 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:09.060 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: Deploying daemon mon.smithi069 on smithi069 2024-03-22T07:09:09.060 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:09.060 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:09.060 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:08 smithi069 ceph-mon[36441]: mon.smithi069@-1(synchronizing).paxosservice(auth 1..6) refresh upgraded, format 0 -> 3 2024-03-22T07:09:09.060 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:09 smithi069 bash[36384]: 95f6103f760352f41574150c918cb0bb72c0c36d938ccef5e7764ceecdb35a07 2024-03-22T07:09:09.060 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:09 smithi069 systemd[1]: Started Ceph mon.smithi069 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:09:14.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:09:14.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: mon.smithi063 calling monitor election 2024-03-22T07:09:14.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:09:14.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:09:14.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:14.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:09:14.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: mon.smithi069 calling monitor election 2024-03-22T07:09:14.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.69:0/2795324938' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/crt"}]: dispatch 2024-03-22T07:09:14.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:09:14.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:14.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:09:14.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:09:14.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: mon.smithi063 is new leader, mons smithi063,smithi069 in quorum (ranks 0,1) 2024-03-22T07:09:14.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: monmap e2: 2 mons at {smithi063=[v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0],smithi069=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T07:09:14.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: fsmap 2024-03-22T07:09:14.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: osdmap e5: 0 total, 0 up, 0 in 2024-03-22T07:09:14.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: mgrmap e17: smithi063.mnxayh(active, since 29s) 2024-03-22T07:09:14.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: overall HEALTH_OK 2024-03-22T07:09:14.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: Standby manager daemon smithi069.gfziwh started 2024-03-22T07:09:14.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.69:0/2795324938' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:09:14.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.69:0/2795324938' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/key"}]: dispatch 2024-03-22T07:09:14.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:14.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.69:0/2795324938' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:09:14.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:14.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:14.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:14.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:13 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:14.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:09:14.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: mon.smithi063 calling monitor election 2024-03-22T07:09:14.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:09:14.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:09:14.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:14.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:09:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: mon.smithi069 calling monitor election 2024-03-22T07:09:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/2795324938' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/crt"}]: dispatch 2024-03-22T07:09:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:09:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:09:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:09:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: mon.smithi063 is new leader, mons smithi063,smithi069 in quorum (ranks 0,1) 2024-03-22T07:09:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: monmap e2: 2 mons at {smithi063=[v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0],smithi069=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T07:09:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: fsmap 2024-03-22T07:09:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: osdmap e5: 0 total, 0 up, 0 in 2024-03-22T07:09:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: mgrmap e17: smithi063.mnxayh(active, since 29s) 2024-03-22T07:09:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: overall HEALTH_OK 2024-03-22T07:09:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: Standby manager daemon smithi069.gfziwh started 2024-03-22T07:09:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/2795324938' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:09:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/2795324938' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/key"}]: dispatch 2024-03-22T07:09:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/2795324938' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:09:14.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:14.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:14.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:14.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:13 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:14.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.903+0000 7f1cc75a2700 1 -- 172.21.15.69:0/1918324288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cac0061f0 msgr2=0x7f1cac008610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:14.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.903+0000 7f1cc75a2700 1 --2- 172.21.15.69:0/1918324288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cac0061f0 0x7f1cac008610 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00fdc50 tx=0x7f1cbc002040 comp rx=0 tx=0).stop 2024-03-22T07:09:14.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.903+0000 7f1cc75a2700 1 -- 172.21.15.69:0/1918324288 shutdown_connections 2024-03-22T07:09:14.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.903+0000 7f1cc75a2700 1 --2- 172.21.15.69:0/1918324288 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cac0061f0 0x7f1cac008610 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:14.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.903+0000 7f1cc75a2700 1 -- 172.21.15.69:0/1918324288 >> 172.21.15.69:0/1918324288 conn(0x7f1cc00f8730 msgr2=0x7f1cc00fab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:14.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.903+0000 7f1cc75a2700 1 -- 172.21.15.69:0/1918324288 shutdown_connections 2024-03-22T07:09:14.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.903+0000 7f1cc75a2700 1 -- 172.21.15.69:0/1918324288 wait complete. 2024-03-22T07:09:14.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.904+0000 7f1cc75a2700 1 Processor -- start 2024-03-22T07:09:14.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.904+0000 7f1cc75a2700 1 -- start start 2024-03-22T07:09:14.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.904+0000 7f1cc75a2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cac0061f0 0x7f1cc0113390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:14.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.904+0000 7f1cc75a2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cc01138d0 0x7f1cc010d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:14.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.904+0000 7f1cc75a2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc0113ee0 con 0x7f1cc01138d0 2024-03-22T07:09:14.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.904+0000 7f1cc75a2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc010d830 con 0x7f1cac0061f0 2024-03-22T07:09:14.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.904+0000 7f1cc533e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cac0061f0 0x7f1cc0113390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:14.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.904+0000 7f1cc533e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cac0061f0 0x7f1cc0113390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56724/0 (socket says 172.21.15.69:56724) 2024-03-22T07:09:14.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.904+0000 7f1cc533e700 1 -- 172.21.15.69:0/2046965082 learned_addr learned my addr 172.21.15.69:0/2046965082 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:09:14.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.904+0000 7f1cc533e700 1 -- 172.21.15.69:0/2046965082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cac0061f0 msgr2=0x7f1cc0113390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-22T07:09:14.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.904+0000 7f1cc533e700 1 -- 172.21.15.69:0/2046965082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cac0061f0 msgr2=0x7f1cc0113390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T07:09:14.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.904+0000 7f1cc533e700 1 --2- 172.21.15.69:0/2046965082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cac0061f0 0x7f1cc0113390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-22T07:09:14.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.904+0000 7f1cc533e700 1 --2- 172.21.15.69:0/2046965082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cac0061f0 0x7f1cc0113390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:09:14.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.905+0000 7f1cc4b3d700 1 --2- 172.21.15.69:0/2046965082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cc01138d0 0x7f1cc010d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:14.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.905+0000 7f1cc4b3d700 1 -- 172.21.15.69:0/2046965082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cac0061f0 msgr2=0x7f1cc0113390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:09:14.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.905+0000 7f1cc4b3d700 1 --2- 172.21.15.69:0/2046965082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cac0061f0 0x7f1cc0113390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:14.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.905+0000 7f1cc4b3d700 1 -- 172.21.15.69:0/2046965082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc005040 con 0x7f1cc01138d0 2024-03-22T07:09:14.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.905+0000 7f1cc4b3d700 1 --2- 172.21.15.69:0/2046965082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cc01138d0 0x7f1cc010d2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000b790 tx=0x7f1cb00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:14.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.906+0000 7f1cb5ffb700 1 -- 172.21.15.69:0/2046965082 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cb0011810 con 0x7f1cc01138d0 2024-03-22T07:09:14.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.906+0000 7f1cb5ffb700 1 -- 172.21.15.69:0/2046965082 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1cb0011e40 con 0x7f1cc01138d0 2024-03-22T07:09:14.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.906+0000 7f1cc75a2700 1 -- 172.21.15.69:0/2046965082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc010db10 con 0x7f1cc01138d0 2024-03-22T07:09:14.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.906+0000 7f1cb5ffb700 1 -- 172.21.15.69:0/2046965082 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cb000f2e0 con 0x7f1cc01138d0 2024-03-22T07:09:14.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.906+0000 7f1cc75a2700 1 -- 172.21.15.69:0/2046965082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc010e050 con 0x7f1cc01138d0 2024-03-22T07:09:14.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.907+0000 7f1cc75a2700 1 -- 172.21.15.69:0/2046965082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc00fe2d0 con 0x7f1cc01138d0 2024-03-22T07:09:14.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.907+0000 7f1cb5ffb700 1 -- 172.21.15.69:0/2046965082 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f1cb000f480 con 0x7f1cc01138d0 2024-03-22T07:09:14.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.907+0000 7f1cb5ffb700 1 --2- 172.21.15.69:0/2046965082 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f1ca4069370 0x7f1ca406b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:14.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.908+0000 7f1cb5ffb700 1 -- 172.21.15.69:0/2046965082 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f1cb0086370 con 0x7f1cc01138d0 2024-03-22T07:09:14.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.908+0000 7f1cc533e700 1 --2- 172.21.15.69:0/2046965082 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f1ca4069370 0x7f1ca406b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:14.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.908+0000 7f1cc533e700 1 --2- 172.21.15.69:0/2046965082 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f1ca4069370 0x7f1ca406b820 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc004f50 tx=0x7f1cbc002040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:14.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:14.912+0000 7f1cb5ffb700 1 -- 172.21.15.69:0/2046965082 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1cb00570f0 con 0x7f1cc01138d0 2024-03-22T07:09:15.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:14 smithi063 ceph-mon[29200]: mgrmap e18: smithi063.mnxayh(active, since 29s), standbys: smithi069.gfziwh 2024-03-22T07:09:15.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:14 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr metadata", "who": "smithi069.gfziwh", "id": "smithi069.gfziwh"}]: dispatch 2024-03-22T07:09:15.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:14 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:09:15.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:14 smithi063 ceph-mon[29200]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:15.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:14 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:09:15.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:14 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:15.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:14 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:09:15.140 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:14 smithi069 ceph-mon[36441]: mgrmap e18: smithi063.mnxayh(active, since 29s), standbys: smithi069.gfziwh 2024-03-22T07:09:15.140 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:14 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr metadata", "who": "smithi069.gfziwh", "id": "smithi069.gfziwh"}]: dispatch 2024-03-22T07:09:15.140 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:14 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:09:15.140 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:14 smithi069 ceph-mon[36441]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:15.140 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:14 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:09:15.140 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:14 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:15.141 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:14 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:09:15.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:15.192+0000 7f1cc75a2700 1 -- 172.21.15.69:0/2046965082 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1cc01048a0 con 0x7f1cc01138d0 2024-03-22T07:09:15.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:15.192+0000 7f1cb5ffb700 1 -- 172.21.15.69:0/2046965082 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1013 (secure 0 0 0) 0x7f1cb0015410 con 0x7f1cc01138d0 2024-03-22T07:09:15.192 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-03-22T07:09:15.193 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":2,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","modified":"2024-03-22T07:09:08.740333Z","created":"2024-03-22T07:06:35.169231Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi063","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:3300","nonce":0},{"type":"v1","addr":"172.21.15.63:6789","nonce":0}]},"addr":"172.21.15.63:6789/0","public_addr":"172.21.15.63:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi069","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-22T07:09:15.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:15.194+0000 7f1cc75a2700 1 -- 172.21.15.69:0/2046965082 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f1ca4069370 msgr2=0x7f1ca406b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:15.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:15.194+0000 7f1cc75a2700 1 --2- 172.21.15.69:0/2046965082 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f1ca4069370 0x7f1ca406b820 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc004f50 tx=0x7f1cbc002040 comp rx=0 tx=0).stop 2024-03-22T07:09:15.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:15.194+0000 7f1cc75a2700 1 -- 172.21.15.69:0/2046965082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cc01138d0 msgr2=0x7f1cc010d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:15.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:15.194+0000 7f1cc75a2700 1 --2- 172.21.15.69:0/2046965082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cc01138d0 0x7f1cc010d2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000b790 tx=0x7f1cb00091a0 comp rx=0 tx=0).stop 2024-03-22T07:09:15.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:15.194+0000 7f1cc75a2700 1 -- 172.21.15.69:0/2046965082 shutdown_connections 2024-03-22T07:09:15.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:15.194+0000 7f1cc75a2700 1 --2- 172.21.15.69:0/2046965082 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f1ca4069370 0x7f1ca406b820 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:15.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:15.194+0000 7f1cc75a2700 1 --2- 172.21.15.69:0/2046965082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cac0061f0 0x7f1cc0113390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:15.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:15.194+0000 7f1cc75a2700 1 --2- 172.21.15.69:0/2046965082 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cc01138d0 0x7f1cc010d2f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:15.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:15.194+0000 7f1cc75a2700 1 -- 172.21.15.69:0/2046965082 >> 172.21.15.69:0/2046965082 conn(0x7f1cc00f8730 msgr2=0x7f1cc0106190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:15.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:15.194+0000 7f1cc75a2700 1 -- 172.21.15.69:0/2046965082 shutdown_connections 2024-03-22T07:09:15.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:09:15.195+0000 7f1cc75a2700 1 -- 172.21.15.69:0/2046965082 wait complete. 2024-03-22T07:09:15.195 INFO:teuthology.orchestra.run.smithi069.stderr:dumped monmap epoch 2 2024-03-22T07:09:15.851 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-22T07:09:15.851 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-22T07:09:16.121 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:09:16.122 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:15 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/2046965082' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:09:16.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:15 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/2046965082' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-22T07:09:17.467 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:17 smithi063 ceph-mon[29200]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:17.467 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:17 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:17.467 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:17 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:17.467 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:17 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:17.468 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:17 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:09:17.468 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:17 smithi063 ceph-mon[29200]: Updating smithi063:/etc/ceph/ceph.conf 2024-03-22T07:09:17.468 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:17 smithi063 ceph-mon[29200]: Updating smithi069:/etc/ceph/ceph.conf 2024-03-22T07:09:17.468 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:17 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:17.468 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:17 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:17.468 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:17 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:17.468 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:17 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:17.468 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:17 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:17.468 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:17 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:09:17.468 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:17 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:09:17.468 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:17 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:17.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:17 smithi069 ceph-mon[36441]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:17.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:17 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:17.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:17 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:17.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:17 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:17.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:17 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:09:17.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:17 smithi069 ceph-mon[36441]: Updating smithi063:/etc/ceph/ceph.conf 2024-03-22T07:09:17.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:17 smithi069 ceph-mon[36441]: Updating smithi069:/etc/ceph/ceph.conf 2024-03-22T07:09:17.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:17 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:17.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:17 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:17.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:17 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:17.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:17 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:17.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:17 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:17.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:17 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:09:17.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:17 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:09:17.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:17 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:18.570 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:18 smithi063 ceph-mon[29200]: Updating smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:09:18.570 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:18 smithi063 ceph-mon[29200]: Updating smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:09:18.570 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:18 smithi063 ceph-mon[29200]: Reconfiguring mon.smithi063 (unknown last config time)... 2024-03-22T07:09:18.570 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:18 smithi063 ceph-mon[29200]: Reconfiguring daemon mon.smithi063 on smithi063 2024-03-22T07:09:18.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:18 smithi069 ceph-mon[36441]: Updating smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:09:18.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:18 smithi069 ceph-mon[36441]: Updating smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:09:18.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:18 smithi069 ceph-mon[36441]: Reconfiguring mon.smithi063 (unknown last config time)... 2024-03-22T07:09:18.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:18 smithi069 ceph-mon[36441]: Reconfiguring daemon mon.smithi063 on smithi063 2024-03-22T07:09:18.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.780+0000 7f6807422700 1 -- 172.21.15.63:0/3081171218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68001037a0 msgr2=0x7f68000fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:18.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.780+0000 7f6807422700 1 --2- 172.21.15.63:0/3081171218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68001037a0 0x7f68000fce10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f67f0004940 tx=0x7f67f0009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:18.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.780+0000 7f6807422700 1 -- 172.21.15.63:0/3081171218 shutdown_connections 2024-03-22T07:09:18.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.780+0000 7f6807422700 1 --2- 172.21.15.63:0/3081171218 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68001037a0 0x7f68000fce10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:18.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.781+0000 7f6807422700 1 --2- 172.21.15.63:0/3081171218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6800102e00 0x7f68001031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:18.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.781+0000 7f6807422700 1 -- 172.21.15.63:0/3081171218 >> 172.21.15.63:0/3081171218 conn(0x7f68000f89b0 msgr2=0x7f68000fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:18.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.781+0000 7f6807422700 1 -- 172.21.15.63:0/3081171218 shutdown_connections 2024-03-22T07:09:18.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.781+0000 7f6807422700 1 -- 172.21.15.63:0/3081171218 wait complete. 2024-03-22T07:09:18.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.781+0000 7f6807422700 1 Processor -- start 2024-03-22T07:09:18.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.781+0000 7f6807422700 1 -- start start 2024-03-22T07:09:18.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.781+0000 7f6807422700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6800102e00 0x7f68000702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:18.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.781+0000 7f6807422700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68001037a0 0x7f6800070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:18.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.781+0000 7f6807422700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6800072210 con 0x7f68001037a0 2024-03-22T07:09:18.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.781+0000 7f6807422700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6800070d40 con 0x7f6800102e00 2024-03-22T07:09:18.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.781+0000 7f68049bd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68001037a0 0x7f6800070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:18.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.782+0000 7f68049bd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68001037a0 0x7f6800070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34504/0 (socket says 172.21.15.63:34504) 2024-03-22T07:09:18.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.782+0000 7f68049bd700 1 -- 172.21.15.63:0/1343714259 learned_addr learned my addr 172.21.15.63:0/1343714259 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:09:18.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.782+0000 7f68051be700 1 --2- 172.21.15.63:0/1343714259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6800102e00 0x7f68000702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:18.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.782+0000 7f68049bd700 1 -- 172.21.15.63:0/1343714259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6800102e00 msgr2=0x7f68000702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:18.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.782+0000 7f68049bd700 1 --2- 172.21.15.63:0/1343714259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6800102e00 0x7f68000702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:18.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.782+0000 7f68049bd700 1 -- 172.21.15.63:0/1343714259 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f0005040 con 0x7f68001037a0 2024-03-22T07:09:18.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.782+0000 7f68049bd700 1 --2- 172.21.15.63:0/1343714259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68001037a0 0x7f6800070800 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009100 tx=0x7f67f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:18.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.782+0000 7f67f5ffb700 1 -- 172.21.15.63:0/1343714259 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67f000e040 con 0x7f68001037a0 2024-03-22T07:09:18.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.782+0000 7f6807422700 1 -- 172.21.15.63:0/1343714259 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6800070fc0 con 0x7f68001037a0 2024-03-22T07:09:18.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.782+0000 7f67f5ffb700 1 -- 172.21.15.63:0/1343714259 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67f0009af0 con 0x7f68001037a0 2024-03-22T07:09:18.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.782+0000 7f67f5ffb700 1 -- 172.21.15.63:0/1343714259 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67f001c3f0 con 0x7f68001037a0 2024-03-22T07:09:18.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.783+0000 7f6807422700 1 -- 172.21.15.63:0/1343714259 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680011a0d0 con 0x7f68001037a0 2024-03-22T07:09:18.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.783+0000 7f6807422700 1 -- 172.21.15.63:0/1343714259 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68001052d0 con 0x7f68001037a0 2024-03-22T07:09:18.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.784+0000 7f67f5ffb700 1 -- 172.21.15.63:0/1343714259 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f67f000baf0 con 0x7f68001037a0 2024-03-22T07:09:18.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.784+0000 7f67f5ffb700 1 --2- 172.21.15.63:0/1343714259 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f67ec069210 0x7f67ec06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:18.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.784+0000 7f67f5ffb700 1 -- 172.21.15.63:0/1343714259 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f67f0087300 con 0x7f68001037a0 2024-03-22T07:09:18.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.784+0000 7f68051be700 1 --2- 172.21.15.63:0/1343714259 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f67ec069210 0x7f67ec06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:18.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.785+0000 7f68051be700 1 --2- 172.21.15.63:0/1343714259 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f67ec069210 0x7f67ec06b6c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f67fc004eb0 tx=0x7f67fc00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:18.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.789+0000 7f67f5ffb700 1 -- 172.21.15.63:0/1343714259 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f67f0054710 con 0x7f68001037a0 2024-03-22T07:09:18.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.990+0000 7f6807422700 1 -- 172.21.15.63:0/1343714259 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f6800109ee0 con 0x7f68001037a0 2024-03-22T07:09:18.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.990+0000 7f67f5ffb700 1 -- 172.21.15.63:0/1343714259 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+223 (secure 0 0 0) 0x7f67f0086ab0 con 0x7f68001037a0 2024-03-22T07:09:18.990 INFO:teuthology.orchestra.run.smithi063.stdout:# minimal ceph.conf for 9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:09:18.990 INFO:teuthology.orchestra.run.smithi063.stdout:[global] 2024-03-22T07:09:18.990 INFO:teuthology.orchestra.run.smithi063.stdout: fsid = 9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:09:18.990 INFO:teuthology.orchestra.run.smithi063.stdout: mon_host = [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] 2024-03-22T07:09:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.992+0000 7f6807422700 1 -- 172.21.15.63:0/1343714259 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f67ec069210 msgr2=0x7f67ec06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.992+0000 7f6807422700 1 --2- 172.21.15.63:0/1343714259 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f67ec069210 0x7f67ec06b6c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f67fc004eb0 tx=0x7f67fc00b040 comp rx=0 tx=0).stop 2024-03-22T07:09:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.992+0000 7f6807422700 1 -- 172.21.15.63:0/1343714259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68001037a0 msgr2=0x7f6800070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.992+0000 7f6807422700 1 --2- 172.21.15.63:0/1343714259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68001037a0 0x7f6800070800 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009100 tx=0x7f67f0009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.992+0000 7f6807422700 1 -- 172.21.15.63:0/1343714259 shutdown_connections 2024-03-22T07:09:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.992+0000 7f6807422700 1 --2- 172.21.15.63:0/1343714259 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f67ec069210 0x7f67ec06b6c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.992+0000 7f6807422700 1 --2- 172.21.15.63:0/1343714259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6800102e00 0x7f68000702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.992+0000 7f6807422700 1 --2- 172.21.15.63:0/1343714259 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68001037a0 0x7f6800070800 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.992+0000 7f6807422700 1 -- 172.21.15.63:0/1343714259 >> 172.21.15.63:0/1343714259 conn(0x7f68000f89b0 msgr2=0x7f68001021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.992+0000 7f6807422700 1 -- 172.21.15.63:0/1343714259 shutdown_connections 2024-03-22T07:09:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:18.992+0000 7f6807422700 1 -- 172.21.15.63:0/1343714259 wait complete. 2024-03-22T07:09:19.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:19 smithi063 ceph-mon[29200]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:19.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:19 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:19.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:19 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:19.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:19 smithi063 ceph-mon[29200]: Reconfiguring mgr.smithi063.mnxayh (unknown last config time)... 2024-03-22T07:09:19.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:19 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi063.mnxayh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:09:19.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:19 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:09:19.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:19 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:19.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:19 smithi063 ceph-mon[29200]: Reconfiguring daemon mgr.smithi063.mnxayh on smithi063 2024-03-22T07:09:19.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:19 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/1343714259' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:19.620 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-22T07:09:19.620 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T07:09:19.620 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.conf 2024-03-22T07:09:19.650 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T07:09:19.650 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:09:19.717 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-22T07:09:19.717 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.conf 2024-03-22T07:09:19.747 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:19 smithi069 ceph-mon[36441]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:19.747 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:19 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:19.747 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:19 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:19.747 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:19 smithi069 ceph-mon[36441]: Reconfiguring mgr.smithi063.mnxayh (unknown last config time)... 2024-03-22T07:09:19.747 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:19 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi063.mnxayh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:09:19.747 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:19 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:09:19.747 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:19 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:19.747 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:19 smithi069 ceph-mon[36441]: Reconfiguring daemon mgr.smithi063.mnxayh on smithi063 2024-03-22T07:09:19.747 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:19 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/1343714259' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:19.752 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-22T07:09:19.752 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:09:19.823 INFO:tasks.cephadm:Deploying OSDs... 2024-03-22T07:09:19.823 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T07:09:19.823 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/scratch_devs of=/dev/stdout 2024-03-22T07:09:19.838 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-22T07:09:19.838 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_1 2024-03-22T07:09:19.893 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-22T07:09:19.893 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T07:09:19.893 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-03-22T07:09:19.893 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:09:19.893 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T07:09:19.893 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-03-22 07:08:35.896198607 +0000 2024-03-22T07:09:19.893 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-03-22 07:02:51.026669811 +0000 2024-03-22T07:09:19.894 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-03-22 07:02:51.026669811 +0000 2024-03-22T07:09:19.894 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-03-22 07:02:51.026669811 +0000 2024-03-22T07:09:19.894 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-22T07:09:19.958 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-03-22T07:09:19.959 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-03-22T07:09:19.959 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000242884 s, 2.1 MB/s 2024-03-22T07:09:19.960 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-22T07:09:20.015 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_2 2024-03-22T07:09:20.069 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-22T07:09:20.069 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T07:09:20.070 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-22T07:09:20.070 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:09:20.070 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T07:09:20.070 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-03-22 07:08:35.897198596 +0000 2024-03-22T07:09:20.070 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-03-22 07:02:51.365666356 +0000 2024-03-22T07:09:20.070 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-03-22 07:02:51.365666356 +0000 2024-03-22T07:09:20.070 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-03-22 07:02:51.365666356 +0000 2024-03-22T07:09:20.070 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-22T07:09:20.135 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-03-22T07:09:20.135 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-03-22T07:09:20.135 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000174654 s, 2.9 MB/s 2024-03-22T07:09:20.136 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-22T07:09:20.191 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_3 2024-03-22T07:09:20.246 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-22T07:09:20.246 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T07:09:20.246 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-03-22T07:09:20.246 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:09:20.246 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T07:09:20.246 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-03-22 07:08:35.897198596 +0000 2024-03-22T07:09:20.246 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-03-22 07:02:51.716662779 +0000 2024-03-22T07:09:20.246 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-03-22 07:02:51.716662779 +0000 2024-03-22T07:09:20.246 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-03-22 07:02:51.716662779 +0000 2024-03-22T07:09:20.247 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-22T07:09:20.312 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-03-22T07:09:20.312 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-03-22T07:09:20.312 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000187065 s, 2.7 MB/s 2024-03-22T07:09:20.313 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-22T07:09:20.369 DEBUG:teuthology.orchestra.run.smithi063:> stat /dev/vg_nvme/lv_4 2024-03-22T07:09:20.423 INFO:teuthology.orchestra.run.smithi063.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-22T07:09:20.423 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T07:09:20.423 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-22T07:09:20.423 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:09:20.423 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T07:09:20.423 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 2024-03-22 07:08:35.898198586 +0000 2024-03-22T07:09:20.423 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-03-22 07:02:52.072659151 +0000 2024-03-22T07:09:20.423 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-03-22 07:02:52.072659151 +0000 2024-03-22T07:09:20.423 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-03-22 07:02:52.072659151 +0000 2024-03-22T07:09:20.423 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-22T07:09:20.488 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records in 2024-03-22T07:09:20.488 INFO:teuthology.orchestra.run.smithi063.stderr:1+0 records out 2024-03-22T07:09:20.488 INFO:teuthology.orchestra.run.smithi063.stderr:512 bytes copied, 0.000199448 s, 2.6 MB/s 2024-03-22T07:09:20.490 DEBUG:teuthology.orchestra.run.smithi063:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-22T07:09:20.545 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-22T07:09:20.545 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/scratch_devs of=/dev/stdout 2024-03-22T07:09:20.563 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-22T07:09:20.563 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_1 2024-03-22T07:09:20.621 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-22T07:09:20.621 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T07:09:20.621 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-22T07:09:20.621 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:09:20.621 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T07:09:20.621 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-03-22 07:09:07.230358654 +0000 2024-03-22T07:09:20.621 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-03-22 07:02:45.875825482 +0000 2024-03-22T07:09:20.621 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-03-22 07:02:45.875825482 +0000 2024-03-22T07:09:20.621 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-03-22 07:02:45.875825482 +0000 2024-03-22T07:09:20.621 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-22T07:09:20.687 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-03-22T07:09:20.687 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-03-22T07:09:20.687 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000201463 s, 2.5 MB/s 2024-03-22T07:09:20.688 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-22T07:09:20.744 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_2 2024-03-22T07:09:20.800 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-22T07:09:20.800 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T07:09:20.800 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-03-22T07:09:20.800 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:09:20.800 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T07:09:20.801 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-03-22 07:09:07.230358654 +0000 2024-03-22T07:09:20.801 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-03-22 07:02:46.221817541 +0000 2024-03-22T07:09:20.801 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-03-22 07:02:46.221817541 +0000 2024-03-22T07:09:20.801 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-03-22 07:02:46.221817541 +0000 2024-03-22T07:09:20.801 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-22T07:09:20.866 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-03-22T07:09:20.866 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-03-22T07:09:20.866 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000210486 s, 2.4 MB/s 2024-03-22T07:09:20.867 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-22T07:09:20.924 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_3 2024-03-22T07:09:20.979 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-22T07:09:20.979 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T07:09:20.979 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-03-22T07:09:20.979 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:09:20.980 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T07:09:20.980 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-03-22 07:09:07.231358631 +0000 2024-03-22T07:09:20.980 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-03-22 07:02:46.552809944 +0000 2024-03-22T07:09:20.980 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-03-22 07:02:46.552809944 +0000 2024-03-22T07:09:20.980 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-03-22 07:02:46.552809944 +0000 2024-03-22T07:09:20.980 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-22T07:09:21.046 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-03-22T07:09:21.046 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-03-22T07:09:21.046 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000197115 s, 2.6 MB/s 2024-03-22T07:09:21.047 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-22T07:09:21.103 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_4 2024-03-22T07:09:21.158 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-22T07:09:21.158 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-22T07:09:21.158 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-22T07:09:21.159 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:09:21.159 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-03-22T07:09:21.159 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-03-22 07:09:07.231358631 +0000 2024-03-22T07:09:21.159 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-03-22 07:02:47.057798353 +0000 2024-03-22T07:09:21.159 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-03-22 07:02:47.057798353 +0000 2024-03-22T07:09:21.159 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-03-22 07:02:47.057798353 +0000 2024-03-22T07:09:21.159 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-22T07:09:21.227 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-03-22T07:09:21.227 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-03-22T07:09:21.227 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000198786 s, 2.6 MB/s 2024-03-22T07:09:21.227 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-22T07:09:21.283 INFO:tasks.cephadm:Deploying osd.0 on smithi063 with /dev/vg_nvme/lv_4... 2024-03-22T07:09:21.283 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-22T07:09:21.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:21 smithi069 ceph-mon[36441]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:21.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:21 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:21.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:21 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:21.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:21 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:09:21.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:21 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:21.547 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:09:21.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:21 smithi063 ceph-mon[29200]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:21.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:21 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:21.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:21 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:21.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:21 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:09:21.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:21 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:22.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:22 smithi063 ceph-mon[29200]: Reconfiguring crash.smithi063 (monmap changed)... 2024-03-22T07:09:22.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:22 smithi063 ceph-mon[29200]: Reconfiguring daemon crash.smithi063 on smithi063 2024-03-22T07:09:22.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:22 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:22.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:22 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:22.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:22 smithi069 ceph-mon[36441]: Reconfiguring crash.smithi063 (monmap changed)... 2024-03-22T07:09:22.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:22 smithi069 ceph-mon[36441]: Reconfiguring daemon crash.smithi063 on smithi063 2024-03-22T07:09:22.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:22 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:22.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:22 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:23.434 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:23 smithi063 ceph-mon[29200]: Reconfiguring alertmanager.smithi063 (dependencies changed)... 2024-03-22T07:09:23.434 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:23 smithi063 ceph-mon[29200]: Reconfiguring daemon alertmanager.smithi063 on smithi063 2024-03-22T07:09:23.434 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:23 smithi063 ceph-mon[29200]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:23.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:23 smithi069 ceph-mon[36441]: Reconfiguring alertmanager.smithi063 (dependencies changed)... 2024-03-22T07:09:23.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:23 smithi069 ceph-mon[36441]: Reconfiguring daemon alertmanager.smithi063 on smithi063 2024-03-22T07:09:23.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:23 smithi069 ceph-mon[36441]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:24.998 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-22T07:09:24.998 INFO:teuthology.orchestra.run.smithi063.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-22T07:09:24.998 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10+0 records in 2024-03-22T07:09:24.998 INFO:teuthology.orchestra.run.smithi063.stderr:10+0 records out 2024-03-22T07:09:24.999 INFO:teuthology.orchestra.run.smithi063.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126905 s, 826 MB/s 2024-03-22T07:09:24.999 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping successful for: 2024-03-22T07:09:25.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:25 smithi063 ceph-mon[29200]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:25.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:25 smithi069 ceph-mon[36441]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:25.986 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi063:vg_nvme/lv_4 2024-03-22T07:09:26.590 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:09:27.613 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:27 smithi063 ceph-mon[29200]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:27.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:27 smithi069 ceph-mon[36441]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:28.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.383+0000 7fadd8dad700 1 -- 172.21.15.63:0/4211561599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadd40fff30 msgr2=0x7fadd4102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:28.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.383+0000 7fadd8dad700 1 --2- 172.21.15.63:0/4211561599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadd40fff30 0x7fadd4102350 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fadc8004940 tx=0x7fadc8009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:28.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.384+0000 7fadd8dad700 1 -- 172.21.15.63:0/4211561599 shutdown_connections 2024-03-22T07:09:28.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.384+0000 7fadd8dad700 1 --2- 172.21.15.63:0/4211561599 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadd40fff30 0x7fadd4102350 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:28.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.384+0000 7fadd8dad700 1 --2- 172.21.15.63:0/4211561599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadd4066f90 0x7fadd40ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:28.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.384+0000 7fadd8dad700 1 -- 172.21.15.63:0/4211561599 >> 172.21.15.63:0/4211561599 conn(0x7fadd40f7480 msgr2=0x7fadd40f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:28.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.384+0000 7fadd8dad700 1 -- 172.21.15.63:0/4211561599 shutdown_connections 2024-03-22T07:09:28.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.384+0000 7fadd8dad700 1 -- 172.21.15.63:0/4211561599 wait complete. 2024-03-22T07:09:28.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.384+0000 7fadd8dad700 1 Processor -- start 2024-03-22T07:09:28.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.384+0000 7fadd8dad700 1 -- start start 2024-03-22T07:09:28.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.384+0000 7fadd8dad700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadd4066f90 0x7fadd410d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:28.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.384+0000 7fadd8dad700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadd40fff30 0x7fadd410def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:28.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.384+0000 7fadd8dad700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd410e580 con 0x7fadd40fff30 2024-03-22T07:09:28.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.384+0000 7fadd8dad700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd4112ed0 con 0x7fadd4066f90 2024-03-22T07:09:28.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.384+0000 7fadd1d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadd40fff30 0x7fadd410def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:28.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.384+0000 7fadd1d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadd40fff30 0x7fadd410def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55606/0 (socket says 172.21.15.63:55606) 2024-03-22T07:09:28.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.384+0000 7fadd1d9b700 1 -- 172.21.15.63:0/4009221355 learned_addr learned my addr 172.21.15.63:0/4009221355 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:09:28.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.385+0000 7fadd1d9b700 1 -- 172.21.15.63:0/4009221355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadd4066f90 msgr2=0x7fadd410d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:28.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.385+0000 7fadd1d9b700 1 --2- 172.21.15.63:0/4009221355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadd4066f90 0x7fadd410d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:28.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.385+0000 7fadd1d9b700 1 -- 172.21.15.63:0/4009221355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc8005040 con 0x7fadd40fff30 2024-03-22T07:09:28.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.385+0000 7fadd1d9b700 1 --2- 172.21.15.63:0/4009221355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadd40fff30 0x7fadd410def0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009150 tx=0x7fadc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:28.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.385+0000 7fadc2ffd700 1 -- 172.21.15.63:0/4009221355 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc800e040 con 0x7fadd40fff30 2024-03-22T07:09:28.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.385+0000 7fadd8dad700 1 -- 172.21.15.63:0/4009221355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd41130d0 con 0x7fadd40fff30 2024-03-22T07:09:28.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.385+0000 7fadc2ffd700 1 -- 172.21.15.63:0/4009221355 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadc800bcc0 con 0x7fadd40fff30 2024-03-22T07:09:28.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.385+0000 7fadc2ffd700 1 -- 172.21.15.63:0/4009221355 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc801c630 con 0x7fadd40fff30 2024-03-22T07:09:28.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.385+0000 7fadd8dad700 1 -- 172.21.15.63:0/4009221355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd4113640 con 0x7fadd40fff30 2024-03-22T07:09:28.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.386+0000 7fadd8dad700 1 -- 172.21.15.63:0/4009221355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd404c710 con 0x7fadd40fff30 2024-03-22T07:09:28.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.387+0000 7fadc2ffd700 1 -- 172.21.15.63:0/4009221355 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7fadc8013420 con 0x7fadd40fff30 2024-03-22T07:09:28.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.387+0000 7fadc2ffd700 1 --2- 172.21.15.63:0/4009221355 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fadbc069210 0x7fadbc06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:09:28.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.387+0000 7fadc2ffd700 1 -- 172.21.15.63:0/4009221355 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fadc8086e80 con 0x7fadd40fff30 2024-03-22T07:09:28.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.387+0000 7fadd259c700 1 --2- 172.21.15.63:0/4009221355 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fadbc069210 0x7fadbc06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:09:28.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.388+0000 7fadd259c700 1 --2- 172.21.15.63:0/4009221355 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fadbc069210 0x7fadbc06b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fadc4006fd0 tx=0x7fadc4006db0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:09:28.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.392+0000 7fadc2ffd700 1 -- 172.21.15.63:0/4009221355 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fadc8057d20 con 0x7fadd40fff30 2024-03-22T07:09:28.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:28.598+0000 7fadd8dad700 1 -- 172.21.15.63:0/4009221355 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fadd40f8f70 con 0x7fadbc069210 2024-03-22T07:09:29.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:29 smithi063 ceph-mon[29200]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:29.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:29 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:09:29.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:29 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:09:29.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:29 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:29.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:29 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:09:29.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:29 smithi069 ceph-mon[36441]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:29.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:29 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:09:29.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:29 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:09:29.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:29 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:29.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:29 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:09:30.420 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:30 smithi063 ceph-mon[29200]: from='client.14242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:30.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:30 smithi069 ceph-mon[36441]: from='client.14242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:09:31.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:31 smithi069 ceph-mon[36441]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:31.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:31 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:31.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:31 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:31.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:31 smithi063 ceph-mon[29200]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:31.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:31 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:31.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:31 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:32.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:32 smithi063 ceph-mon[29200]: Reconfiguring grafana.smithi063 (dependencies changed)... 2024-03-22T07:09:32.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:32 smithi063 ceph-mon[29200]: Reconfiguring daemon grafana.smithi063 on smithi063 2024-03-22T07:09:32.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:32 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/412478749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33ade431-7f97-43c7-bf9d-962c12d3ba7b"}]: dispatch 2024-03-22T07:09:32.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:32 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/412478749' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33ade431-7f97-43c7-bf9d-962c12d3ba7b"}]': finished 2024-03-22T07:09:32.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:32 smithi063 ceph-mon[29200]: osdmap e6: 1 total, 0 up, 1 in 2024-03-22T07:09:32.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:32 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:09:32.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:32 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/269457785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:09:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:32 smithi069 ceph-mon[36441]: Reconfiguring grafana.smithi063 (dependencies changed)... 2024-03-22T07:09:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:32 smithi069 ceph-mon[36441]: Reconfiguring daemon grafana.smithi063 on smithi063 2024-03-22T07:09:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:32 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/412478749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33ade431-7f97-43c7-bf9d-962c12d3ba7b"}]: dispatch 2024-03-22T07:09:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:32 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/412478749' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33ade431-7f97-43c7-bf9d-962c12d3ba7b"}]': finished 2024-03-22T07:09:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:32 smithi069 ceph-mon[36441]: osdmap e6: 1 total, 0 up, 1 in 2024-03-22T07:09:32.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:32 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:09:32.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:32 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/269457785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:09:33.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:33 smithi063 ceph-mon[29200]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:33 smithi069 ceph-mon[36441]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:35.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:35 smithi063 ceph-mon[29200]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:35.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:35 smithi069 ceph-mon[36441]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:37.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:37 smithi063 ceph-mon[29200]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:37.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:37 smithi069 ceph-mon[36441]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:39.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:39 smithi063 ceph-mon[29200]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:39.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:39 smithi069 ceph-mon[36441]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:41 smithi063 ceph-mon[29200]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:41 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:41 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:41.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:41 smithi069 ceph-mon[36441]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:41.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:41 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:41.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:41 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:42.587 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:42 smithi063 ceph-mon[29200]: Reconfiguring prometheus.smithi063 (dependencies changed)... 2024-03-22T07:09:42.587 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:42 smithi063 ceph-mon[29200]: Reconfiguring daemon prometheus.smithi063 on smithi063 2024-03-22T07:09:42.587 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:42 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T07:09:42.588 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:42 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:42.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:42 smithi069 ceph-mon[36441]: Reconfiguring prometheus.smithi063 (dependencies changed)... 2024-03-22T07:09:42.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:42 smithi069 ceph-mon[36441]: Reconfiguring daemon prometheus.smithi063 on smithi063 2024-03-22T07:09:42.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:42 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T07:09:42.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:42 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:43.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:43 smithi063 ceph-mon[29200]: Deploying daemon osd.0 on smithi063 2024-03-22T07:09:43.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:43 smithi063 ceph-mon[29200]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:43.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:43 smithi069 ceph-mon[36441]: Deploying daemon osd.0 on smithi063 2024-03-22T07:09:43.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:43 smithi069 ceph-mon[36441]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:44.653 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:09:44.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:44 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:09:45.539 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:45 smithi063 ceph-mon[29200]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:45.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:45 smithi069 ceph-mon[36441]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:47.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:47 smithi063 ceph-mon[29200]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:47.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:47 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:47.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:47 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:47.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:47 smithi063 ceph-mon[29200]: Reconfiguring crash.smithi069 (monmap changed)... 2024-03-22T07:09:47.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:47 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi069", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:09:47.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:47 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:47.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:47 smithi063 ceph-mon[29200]: Reconfiguring daemon crash.smithi069 on smithi069 2024-03-22T07:09:47.574 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:47 smithi069 ceph-mon[36441]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:47.574 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:47 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:47.574 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:47 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:47.575 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:47 smithi069 ceph-mon[36441]: Reconfiguring crash.smithi069 (monmap changed)... 2024-03-22T07:09:47.575 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:47 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi069", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:09:47.575 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:47 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:47.575 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:47 smithi069 ceph-mon[36441]: Reconfiguring daemon crash.smithi069 on smithi069 2024-03-22T07:09:48.634 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:48 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:48.634 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:48 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:48.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:48 smithi063 ceph-mon[29200]: Reconfiguring mgr.smithi069.gfziwh (monmap changed)... 2024-03-22T07:09:48.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:48 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi069.gfziwh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:09:48.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:48 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:09:48.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:48 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:48.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:48 smithi063 ceph-mon[29200]: Reconfiguring daemon mgr.smithi069.gfziwh on smithi069 2024-03-22T07:09:48.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:48 smithi063 ceph-mon[29200]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:48.897 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:48 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:48.897 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:48 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:48.897 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:48 smithi069 ceph-mon[36441]: Reconfiguring mgr.smithi069.gfziwh (monmap changed)... 2024-03-22T07:09:48.897 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:48 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi069.gfziwh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:09:48.897 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:48 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:09:48.897 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:48 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:48.897 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:48 smithi069 ceph-mon[36441]: Reconfiguring daemon mgr.smithi069.gfziwh on smithi069 2024-03-22T07:09:48.897 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:48 smithi069 ceph-mon[36441]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:50.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:50.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:50.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:49 smithi069 ceph-mon[36441]: Reconfiguring mon.smithi069 (monmap changed)... 2024-03-22T07:09:50.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:09:50.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:09:50.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:50.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:49 smithi069 ceph-mon[36441]: Reconfiguring daemon mon.smithi069 on smithi069 2024-03-22T07:09:50.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:50.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:50.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:49 smithi063 ceph-mon[29200]: Reconfiguring mon.smithi069 (monmap changed)... 2024-03-22T07:09:50.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:09:50.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:09:50.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:09:50.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:49 smithi063 ceph-mon[29200]: Reconfiguring daemon mon.smithi069 on smithi069 2024-03-22T07:09:51.462 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:51.463 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:51.463 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:51.463 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:09:51.463 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:09:51.463 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063.front.sepia.ceph.com:9093"}]: dispatch 2024-03-22T07:09:51.463 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063.front.sepia.ceph.com:9093"}]: dispatch 2024-03-22T07:09:51.463 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:51.463 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:09:51.463 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:09:51.463 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063.front.sepia.ceph.com:3000"}]: dispatch 2024-03-22T07:09:51.463 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063.front.sepia.ceph.com:3000"}]: dispatch 2024-03-22T07:09:51.464 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:51.464 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:09:51.464 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:09:51.464 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063.front.sepia.ceph.com:9095"}]: dispatch 2024-03-22T07:09:51.464 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063.front.sepia.ceph.com:9095"}]: dispatch 2024-03-22T07:09:51.464 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:51.464 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:09:51.464 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:51.464 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:51.724 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:51.725 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:51.725 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:51.725 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:09:51.725 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:09:51.725 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063.front.sepia.ceph.com:9093"}]: dispatch 2024-03-22T07:09:51.725 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi063.front.sepia.ceph.com:9093"}]: dispatch 2024-03-22T07:09:51.725 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:51.725 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:09:51.725 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:09:51.725 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063.front.sepia.ceph.com:3000"}]: dispatch 2024-03-22T07:09:51.725 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi063.front.sepia.ceph.com:3000"}]: dispatch 2024-03-22T07:09:51.725 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:51.725 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:09:51.726 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:09:51.726 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063.front.sepia.ceph.com:9095"}]: dispatch 2024-03-22T07:09:51.726 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi063.front.sepia.ceph.com:9095"}]: dispatch 2024-03-22T07:09:51.726 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:51.726 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:09:51.726 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:51.726 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:53.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:53 smithi063 ceph-mon[29200]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:53.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:53 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:53.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:53 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:53.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:53 smithi069 ceph-mon[36441]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:53.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:53 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:53.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:53 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:55.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:55.065+0000 7fadc2ffd700 1 -- 172.21.15.63:0/4009221355 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fadd40f8f70 con 0x7fadbc069210 2024-03-22T07:09:55.065 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 0 on host 'smithi063' 2024-03-22T07:09:55.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:55.067+0000 7fadd8dad700 1 -- 172.21.15.63:0/4009221355 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fadbc069210 msgr2=0x7fadbc06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:55.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:55.067+0000 7fadd8dad700 1 --2- 172.21.15.63:0/4009221355 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fadbc069210 0x7fadbc06b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fadc4006fd0 tx=0x7fadc4006db0 comp rx=0 tx=0).stop 2024-03-22T07:09:55.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:55.067+0000 7fadd8dad700 1 -- 172.21.15.63:0/4009221355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadd40fff30 msgr2=0x7fadd410def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:09:55.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:55.067+0000 7fadd8dad700 1 --2- 172.21.15.63:0/4009221355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadd40fff30 0x7fadd410def0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009150 tx=0x7fadc8009e30 comp rx=0 tx=0).stop 2024-03-22T07:09:55.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:55.067+0000 7fadd8dad700 1 -- 172.21.15.63:0/4009221355 shutdown_connections 2024-03-22T07:09:55.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:55.067+0000 7fadd8dad700 1 --2- 172.21.15.63:0/4009221355 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fadbc069210 0x7fadbc06b6c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:55.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:55.067+0000 7fadd8dad700 1 --2- 172.21.15.63:0/4009221355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadd4066f90 0x7fadd410d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:55.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:55.067+0000 7fadd8dad700 1 --2- 172.21.15.63:0/4009221355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadd40fff30 0x7fadd410def0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:09:55.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:55.067+0000 7fadd8dad700 1 -- 172.21.15.63:0/4009221355 >> 172.21.15.63:0/4009221355 conn(0x7fadd40f7480 msgr2=0x7fadd40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:09:55.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:55.067+0000 7fadd8dad700 1 -- 172.21.15.63:0/4009221355 shutdown_connections 2024-03-22T07:09:55.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:09:55.067+0000 7fadd8dad700 1 -- 172.21.15.63:0/4009221355 wait complete. 2024-03-22T07:09:55.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:55 smithi063 ceph-mon[29200]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:55.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:55 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:55.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:55 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:55.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:55 smithi063 ceph-mon[29200]: from='osd.0 [v2:172.21.15.63:6802/3791141232,v1:172.21.15.63:6803/3791141232]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T07:09:55.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:55 smithi069 ceph-mon[36441]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:55.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:55 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:55.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:55 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:55.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:55 smithi069 ceph-mon[36441]: from='osd.0 [v2:172.21.15.63:6802/3791141232,v1:172.21.15.63:6803/3791141232]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T07:09:55.987 DEBUG:teuthology.orchestra.run.smithi063:osd.0> sudo journalctl -f -n 0 -u ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.0.service 2024-03-22T07:09:55.989 INFO:tasks.cephadm:Deploying osd.1 on smithi063 with /dev/vg_nvme/lv_3... 2024-03-22T07:09:55.990 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-22T07:09:56.352 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:09:57.311 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:57 smithi063 ceph-mon[29200]: from='osd.0 [v2:172.21.15.63:6802/3791141232,v1:172.21.15.63:6803/3791141232]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T07:09:57.311 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:57 smithi063 ceph-mon[29200]: osdmap e7: 1 total, 0 up, 1 in 2024-03-22T07:09:57.311 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:57 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:09:57.312 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:57 smithi063 ceph-mon[29200]: from='osd.0 [v2:172.21.15.63:6802/3791141232,v1:172.21.15.63:6803/3791141232]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-03-22T07:09:57.312 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:57 smithi063 ceph-mon[29200]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:57.312 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:09:57 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0[47343]: 2024-03-22T07:09:57.058+0000 7f6c891e5700 -1 osd.0 0 waiting for initial osdmap 2024-03-22T07:09:57.312 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:09:57 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0[47343]: 2024-03-22T07:09:57.061+0000 7f6c81fdc700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:09:57.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:57 smithi069 ceph-mon[36441]: from='osd.0 [v2:172.21.15.63:6802/3791141232,v1:172.21.15.63:6803/3791141232]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T07:09:57.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:57 smithi069 ceph-mon[36441]: osdmap e7: 1 total, 0 up, 1 in 2024-03-22T07:09:57.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:57 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:09:57.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:57 smithi069 ceph-mon[36441]: from='osd.0 [v2:172.21.15.63:6802/3791141232,v1:172.21.15.63:6803/3791141232]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-03-22T07:09:57.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:57 smithi069 ceph-mon[36441]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:58.220 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:58 smithi063 ceph-mon[29200]: from='osd.0 [v2:172.21.15.63:6802/3791141232,v1:172.21.15.63:6803/3791141232]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-03-22T07:09:58.220 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:58 smithi063 ceph-mon[29200]: osdmap e8: 1 total, 0 up, 1 in 2024-03-22T07:09:58.220 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:09:58.220 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:09:58.220 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:09:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:58 smithi069 ceph-mon[36441]: from='osd.0 [v2:172.21.15.63:6802/3791141232,v1:172.21.15.63:6803/3791141232]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-03-22T07:09:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:58 smithi069 ceph-mon[36441]: osdmap e8: 1 total, 0 up, 1 in 2024-03-22T07:09:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:09:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:09:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:09:59.135 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-22T07:09:59.135 INFO:teuthology.orchestra.run.smithi063.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-22T07:09:59.135 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10+0 records in 2024-03-22T07:09:59.135 INFO:teuthology.orchestra.run.smithi063.stderr:10+0 records out 2024-03-22T07:09:59.135 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125944 s, 833 MB/s 2024-03-22T07:09:59.136 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping successful for: 2024-03-22T07:09:59.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:59 smithi063 ceph-mon[29200]: purged_snaps scrub starts 2024-03-22T07:09:59.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:59 smithi063 ceph-mon[29200]: purged_snaps scrub ok 2024-03-22T07:09:59.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:59 smithi063 ceph-mon[29200]: osd.0 [v2:172.21.15.63:6802/3791141232,v1:172.21.15.63:6803/3791141232] boot 2024-03-22T07:09:59.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:59 smithi063 ceph-mon[29200]: osdmap e9: 1 total, 1 up, 1 in 2024-03-22T07:09:59.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:59 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:09:59.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:59 smithi063 ceph-mon[29200]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:59.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:59 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:59.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:09:59 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:59.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:59 smithi069 ceph-mon[36441]: purged_snaps scrub starts 2024-03-22T07:09:59.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:59 smithi069 ceph-mon[36441]: purged_snaps scrub ok 2024-03-22T07:09:59.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:59 smithi069 ceph-mon[36441]: osd.0 [v2:172.21.15.63:6802/3791141232,v1:172.21.15.63:6803/3791141232] boot 2024-03-22T07:09:59.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:59 smithi069 ceph-mon[36441]: osdmap e9: 1 total, 1 up, 1 in 2024-03-22T07:09:59.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:59 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:09:59.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:59 smithi069 ceph-mon[36441]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-22T07:09:59.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:59 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:59.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:09:59 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:09:59.800 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi063:vg_nvme/lv_3 2024-03-22T07:10:00.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:00 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:10:00.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:00 smithi063 ceph-mon[29200]: osdmap e10: 1 total, 1 up, 1 in 2024-03-22T07:10:00.363 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:00 smithi063 ceph-mon[29200]: overall HEALTH_OK 2024-03-22T07:10:00.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:00 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:10:00.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:00 smithi069 ceph-mon[36441]: osdmap e10: 1 total, 1 up, 1 in 2024-03-22T07:10:00.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:00 smithi069 ceph-mon[36441]: overall HEALTH_OK 2024-03-22T07:10:01.173 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:01 smithi063 ceph-mon[29200]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:01.412 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:10:01.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:01 smithi069 ceph-mon[36441]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:03.325 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:03 smithi063 ceph-mon[29200]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:03.325 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:03 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:03.325 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:03 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:03.325 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:03 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:10:03.326 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:03 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:03.326 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:03 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:03.326 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:03 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:10:03.326 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:03 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:03.326 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:03 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:03.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.523+0000 7f6e71b44700 1 -- 172.21.15.63:0/3682894935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c0fce70 msgr2=0x7f6e6c0fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:03.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.523+0000 7f6e71b44700 1 --2- 172.21.15.63:0/3682894935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c0fce70 0x7f6e6c0fd280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c004940 tx=0x7f6e5c009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:03.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.523+0000 7f6e71b44700 1 -- 172.21.15.63:0/3682894935 shutdown_connections 2024-03-22T07:10:03.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.523+0000 7f6e71b44700 1 --2- 172.21.15.63:0/3682894935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e6c0fdfe0 0x7f6e6c0fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:03.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.523+0000 7f6e71b44700 1 --2- 172.21.15.63:0/3682894935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c0fce70 0x7f6e6c0fd280 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:03.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.523+0000 7f6e71b44700 1 -- 172.21.15.63:0/3682894935 >> 172.21.15.63:0/3682894935 conn(0x7f6e6c0f8550 msgr2=0x7f6e6c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:03.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.523+0000 7f6e71b44700 1 -- 172.21.15.63:0/3682894935 shutdown_connections 2024-03-22T07:10:03.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.523+0000 7f6e71b44700 1 -- 172.21.15.63:0/3682894935 wait complete. 2024-03-22T07:10:03.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.523+0000 7f6e71b44700 1 Processor -- start 2024-03-22T07:10:03.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.524+0000 7f6e71b44700 1 -- start start 2024-03-22T07:10:03.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.524+0000 7f6e71b44700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c0fce70 0x7f6e6c10fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:03.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.524+0000 7f6e71b44700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e6c0fdfe0 0x7f6e6c110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:03.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.524+0000 7f6e71b44700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c110690 con 0x7f6e6c0fce70 2024-03-22T07:10:03.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.524+0000 7f6e71b44700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c110800 con 0x7f6e6c0fdfe0 2024-03-22T07:10:03.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.524+0000 7f6e6b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c0fce70 0x7f6e6c10fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:03.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.524+0000 7f6e6b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c0fce70 0x7f6e6c10fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33420/0 (socket says 172.21.15.63:33420) 2024-03-22T07:10:03.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.524+0000 7f6e6b7fe700 1 -- 172.21.15.63:0/2990706950 learned_addr learned my addr 172.21.15.63:0/2990706950 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:10:03.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.524+0000 7f6e62ffd700 1 --2- 172.21.15.63:0/2990706950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e6c0fdfe0 0x7f6e6c110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:03.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.524+0000 7f6e6b7fe700 1 -- 172.21.15.63:0/2990706950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e6c0fdfe0 msgr2=0x7f6e6c110090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:03.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.524+0000 7f6e6b7fe700 1 --2- 172.21.15.63:0/2990706950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e6c0fdfe0 0x7f6e6c110090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:03.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.524+0000 7f6e6b7fe700 1 -- 172.21.15.63:0/2990706950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c005040 con 0x7f6e6c0fce70 2024-03-22T07:10:03.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.524+0000 7f6e62ffd700 1 --2- 172.21.15.63:0/2990706950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e6c0fdfe0 0x7f6e6c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:10:03.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.524+0000 7f6e6b7fe700 1 --2- 172.21.15.63:0/2990706950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c0fce70 0x7f6e6c10fb50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c013870 tx=0x7f6e5c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:03.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.525+0000 7f6e68ff9700 1 -- 172.21.15.63:0/2990706950 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e5c00e070 con 0x7f6e6c0fce70 2024-03-22T07:10:03.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.525+0000 7f6e68ff9700 1 -- 172.21.15.63:0/2990706950 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e5c013de0 con 0x7f6e6c0fce70 2024-03-22T07:10:03.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.525+0000 7f6e71b44700 1 -- 172.21.15.63:0/2990706950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c115200 con 0x7f6e6c0fce70 2024-03-22T07:10:03.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.525+0000 7f6e68ff9700 1 -- 172.21.15.63:0/2990706950 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e5c01cea0 con 0x7f6e6c0fce70 2024-03-22T07:10:03.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.525+0000 7f6e71b44700 1 -- 172.21.15.63:0/2990706950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c1156e0 con 0x7f6e6c0fce70 2024-03-22T07:10:03.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.526+0000 7f6e71b44700 1 -- 172.21.15.63:0/2990706950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6c101970 con 0x7f6e6c0fce70 2024-03-22T07:10:03.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.526+0000 7f6e68ff9700 1 -- 172.21.15.63:0/2990706950 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f6e5c01c690 con 0x7f6e6c0fce70 2024-03-22T07:10:03.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.526+0000 7f6e68ff9700 1 --2- 172.21.15.63:0/2990706950 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f6e58069210 0x7f6e5806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:03.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.527+0000 7f6e62ffd700 1 --2- 172.21.15.63:0/2990706950 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f6e58069210 0x7f6e5806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:03.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.527+0000 7f6e68ff9700 1 -- 172.21.15.63:0/2990706950 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7f6e5c0879e0 con 0x7f6e6c0fce70 2024-03-22T07:10:03.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.527+0000 7f6e62ffd700 1 --2- 172.21.15.63:0/2990706950 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f6e58069210 0x7f6e5806b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6e54004fd0 tx=0x7f6e5400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:03.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:03 smithi069 ceph-mon[36441]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:03.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:03 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:03.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:03 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:03.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:03 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:10:03.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:03 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:03.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:03 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:03.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:03 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:10:03.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:03 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:03.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:03 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:03.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.531+0000 7f6e68ff9700 1 -- 172.21.15.63:0/2990706950 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f6e5c058620 con 0x7f6e6c0fce70 2024-03-22T07:10:03.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:03.735+0000 7f6e71b44700 1 -- 172.21.15.63:0/2990706950 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e6c02b340 con 0x7f6e58069210 2024-03-22T07:10:04.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:04 smithi069 ceph-mon[36441]: Detected new or changed devices on smithi063 2024-03-22T07:10:04.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:04 smithi069 ceph-mon[36441]: Adjusting osd_memory_target on smithi063 to 12864M 2024-03-22T07:10:04.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:04 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:10:04.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:04 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:10:04.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:04 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:04.559 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:04 smithi063 ceph-mon[29200]: Detected new or changed devices on smithi063 2024-03-22T07:10:04.560 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:04 smithi063 ceph-mon[29200]: Adjusting osd_memory_target on smithi063 to 12864M 2024-03-22T07:10:04.560 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:04 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:10:04.560 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:04 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:10:04.560 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:04 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:05.616 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:05 smithi063 ceph-mon[29200]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:05.616 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:05 smithi063 ceph-mon[29200]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:05.616 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:05 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:05.616 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:05 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:05.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:05 smithi069 ceph-mon[36441]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:05.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:05 smithi069 ceph-mon[36441]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:05.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:05 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:05.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:05 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:07 smithi063 ceph-mon[29200]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:07 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/1652151051' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0643ddb6-db9d-464c-8236-3aea2893de83"}]: dispatch 2024-03-22T07:10:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:07 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/1652151051' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0643ddb6-db9d-464c-8236-3aea2893de83"}]': finished 2024-03-22T07:10:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:07 smithi063 ceph-mon[29200]: osdmap e11: 2 total, 1 up, 2 in 2024-03-22T07:10:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:07 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:10:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:07 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/208508789' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:10:07.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:07 smithi069 ceph-mon[36441]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:07.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:07 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/1652151051' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0643ddb6-db9d-464c-8236-3aea2893de83"}]: dispatch 2024-03-22T07:10:07.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:07 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/1652151051' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0643ddb6-db9d-464c-8236-3aea2893de83"}]': finished 2024-03-22T07:10:07.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:07 smithi069 ceph-mon[36441]: osdmap e11: 2 total, 1 up, 2 in 2024-03-22T07:10:07.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:07 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:10:07.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:07 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/208508789' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:10:09.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:09 smithi069 ceph-mon[36441]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:09.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:09 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:09.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:09 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:09.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:09 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:09.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:09 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:10:09.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:09 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:09.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:09 smithi063 ceph-mon[29200]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:09.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:09 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:09.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:09 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:09.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:09 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:09.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:09 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:10:09.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:09 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:11.749 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:11 smithi063 ceph-mon[29200]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:11.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:11 smithi069 ceph-mon[36441]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:13.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:13 smithi069 ceph-mon[36441]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:13.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:13 smithi063 ceph-mon[29200]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:14.599 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:14 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T07:10:14.599 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:14 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:14.599 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:14 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:10:14.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:14 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T07:10:14.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:14 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:14.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:14 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:10:15.614 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:15 smithi063 ceph-mon[29200]: Deploying daemon osd.1 on smithi063 2024-03-22T07:10:15.614 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:15 smithi063 ceph-mon[29200]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:15.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:15 smithi069 ceph-mon[36441]: Deploying daemon osd.1 on smithi063 2024-03-22T07:10:15.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:15 smithi069 ceph-mon[36441]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:17.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:17 smithi063 ceph-mon[29200]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:17.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:17 smithi069 ceph-mon[36441]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:19.773 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:19 smithi069 ceph-mon[36441]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:19.773 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:19 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:19.773 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:19 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:19.773 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:19 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:19.842 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:19 smithi063 ceph-mon[29200]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:19.842 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:19 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:19.842 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:19 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:19.843 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:19 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:21.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:21 smithi069 ceph-mon[36441]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:21.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:21 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:21.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:21 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:21.789 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:21 smithi063 ceph-mon[29200]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:21.789 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:21 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:21.790 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:21 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:22.988 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 1 on host 'smithi063' 2024-03-22T07:10:22.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:22.988+0000 7f6e68ff9700 1 -- 172.21.15.63:0/2990706950 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6e6c02b340 con 0x7f6e58069210 2024-03-22T07:10:22.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:22.990+0000 7f6e71b44700 1 -- 172.21.15.63:0/2990706950 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f6e58069210 msgr2=0x7f6e5806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:22.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:22.990+0000 7f6e71b44700 1 --2- 172.21.15.63:0/2990706950 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f6e58069210 0x7f6e5806b6c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6e54004fd0 tx=0x7f6e5400b040 comp rx=0 tx=0).stop 2024-03-22T07:10:22.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:22.990+0000 7f6e71b44700 1 -- 172.21.15.63:0/2990706950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c0fce70 msgr2=0x7f6e6c10fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:22.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:22.990+0000 7f6e71b44700 1 --2- 172.21.15.63:0/2990706950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c0fce70 0x7f6e6c10fb50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c013870 tx=0x7f6e5c009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:22.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:22.990+0000 7f6e71b44700 1 -- 172.21.15.63:0/2990706950 shutdown_connections 2024-03-22T07:10:22.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:22.990+0000 7f6e71b44700 1 --2- 172.21.15.63:0/2990706950 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f6e58069210 0x7f6e5806b6c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:22.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:22.990+0000 7f6e71b44700 1 --2- 172.21.15.63:0/2990706950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e6c0fce70 0x7f6e6c10fb50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:22.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:22.990+0000 7f6e71b44700 1 --2- 172.21.15.63:0/2990706950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e6c0fdfe0 0x7f6e6c110090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:22.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:22.990+0000 7f6e71b44700 1 -- 172.21.15.63:0/2990706950 >> 172.21.15.63:0/2990706950 conn(0x7f6e6c0f8550 msgr2=0x7f6e6c0f9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:22.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:22.991+0000 7f6e71b44700 1 -- 172.21.15.63:0/2990706950 shutdown_connections 2024-03-22T07:10:22.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:22.991+0000 7f6e71b44700 1 -- 172.21.15.63:0/2990706950 wait complete. 2024-03-22T07:10:23.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:23 smithi069 ceph-mon[36441]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:23.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:23 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:23.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:23 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:23.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:23 smithi069 ceph-mon[36441]: from='osd.1 [v2:172.21.15.63:6810/1043983969,v1:172.21.15.63:6811/1043983969]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T07:10:23.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:23 smithi063 ceph-mon[29200]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:23.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:23 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:23.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:23 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:23.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:23 smithi063 ceph-mon[29200]: from='osd.1 [v2:172.21.15.63:6810/1043983969,v1:172.21.15.63:6811/1043983969]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T07:10:24.256 DEBUG:teuthology.orchestra.run.smithi063:osd.1> sudo journalctl -f -n 0 -u ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.1.service 2024-03-22T07:10:24.258 INFO:tasks.cephadm:Deploying osd.2 on smithi063 with /dev/vg_nvme/lv_2... 2024-03-22T07:10:24.258 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-22T07:10:24.595 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:10:25.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:24 smithi069 ceph-mon[36441]: from='osd.1 [v2:172.21.15.63:6810/1043983969,v1:172.21.15.63:6811/1043983969]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T07:10:25.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:24 smithi069 ceph-mon[36441]: osdmap e12: 2 total, 1 up, 2 in 2024-03-22T07:10:25.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:24 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:10:25.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:24 smithi069 ceph-mon[36441]: from='osd.1 [v2:172.21.15.63:6810/1043983969,v1:172.21.15.63:6811/1043983969]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-03-22T07:10:25.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:24 smithi069 ceph-mon[36441]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:25.286 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:24 smithi063 ceph-mon[29200]: from='osd.1 [v2:172.21.15.63:6810/1043983969,v1:172.21.15.63:6811/1043983969]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T07:10:25.286 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:24 smithi063 ceph-mon[29200]: osdmap e12: 2 total, 1 up, 2 in 2024-03-22T07:10:25.286 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:24 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:10:25.286 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:24 smithi063 ceph-mon[29200]: from='osd.1 [v2:172.21.15.63:6810/1043983969,v1:172.21.15.63:6811/1043983969]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-03-22T07:10:25.287 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:24 smithi063 ceph-mon[29200]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:25.287 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:10:24 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1[54588]: 2024-03-22T07:10:24.979+0000 7f81e1eec700 -1 osd.1 0 waiting for initial osdmap 2024-03-22T07:10:25.287 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:10:24 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1[54588]: 2024-03-22T07:10:24.984+0000 7f81dcce7700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:10:26.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:25 smithi069 ceph-mon[36441]: from='osd.1 [v2:172.21.15.63:6810/1043983969,v1:172.21.15.63:6811/1043983969]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-03-22T07:10:26.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:25 smithi069 ceph-mon[36441]: osdmap e13: 2 total, 1 up, 2 in 2024-03-22T07:10:26.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:25 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:10:26.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:25 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:10:26.347 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:25 smithi063 ceph-mon[29200]: from='osd.1 [v2:172.21.15.63:6810/1043983969,v1:172.21.15.63:6811/1043983969]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-03-22T07:10:26.347 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:25 smithi063 ceph-mon[29200]: osdmap e13: 2 total, 1 up, 2 in 2024-03-22T07:10:26.347 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:25 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:10:26.347 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:25 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:10:27.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:26 smithi069 ceph-mon[36441]: purged_snaps scrub starts 2024-03-22T07:10:27.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:26 smithi069 ceph-mon[36441]: purged_snaps scrub ok 2024-03-22T07:10:27.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:26 smithi069 ceph-mon[36441]: osd.1 [v2:172.21.15.63:6810/1043983969,v1:172.21.15.63:6811/1043983969] boot 2024-03-22T07:10:27.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:26 smithi069 ceph-mon[36441]: osdmap e14: 2 total, 2 up, 2 in 2024-03-22T07:10:27.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:26 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:10:27.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:26 smithi069 ceph-mon[36441]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:27.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:26 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:27.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:26 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:27.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:26 smithi063 ceph-mon[29200]: purged_snaps scrub starts 2024-03-22T07:10:27.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:26 smithi063 ceph-mon[29200]: purged_snaps scrub ok 2024-03-22T07:10:27.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:26 smithi063 ceph-mon[29200]: osd.1 [v2:172.21.15.63:6810/1043983969,v1:172.21.15.63:6811/1043983969] boot 2024-03-22T07:10:27.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:26 smithi063 ceph-mon[29200]: osdmap e14: 2 total, 2 up, 2 in 2024-03-22T07:10:27.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:26 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:10:27.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:26 smithi063 ceph-mon[29200]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-03-22T07:10:27.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:26 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:27.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:26 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:27.615 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-22T07:10:27.615 INFO:teuthology.orchestra.run.smithi063.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-22T07:10:27.615 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 10+0 records in 2024-03-22T07:10:27.615 INFO:teuthology.orchestra.run.smithi063.stderr:10+0 records out 2024-03-22T07:10:27.615 INFO:teuthology.orchestra.run.smithi063.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0118302 s, 886 MB/s 2024-03-22T07:10:27.615 INFO:teuthology.orchestra.run.smithi063.stderr: stderr: 2024-03-22T07:10:27.616 INFO:teuthology.orchestra.run.smithi063.stderr:--> Zapping successful for: 2024-03-22T07:10:28.276 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:27 smithi063 ceph-mon[29200]: osdmap e15: 2 total, 2 up, 2 in 2024-03-22T07:10:28.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:27 smithi069 ceph-mon[36441]: osdmap e15: 2 total, 2 up, 2 in 2024-03-22T07:10:28.539 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi063:vg_nvme/lv_2 2024-03-22T07:10:28.914 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:10:29.122 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:28 smithi063 ceph-mon[29200]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:29.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:28 smithi069 ceph-mon[36441]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:30.122 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:29 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:10:30.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:29 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:10:31.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:31 smithi063 ceph-mon[29200]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:31.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:31 smithi069 ceph-mon[36441]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:31.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.431+0000 7f9e52764700 1 -- 172.21.15.63:0/3845369215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e4c0fbcd0 msgr2=0x7f9e4c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:31.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.431+0000 7f9e52764700 1 --2- 172.21.15.63:0/3845369215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e4c0fbcd0 0x7f9e4c0fc0e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9e40004940 tx=0x7f9e40009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:31.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.431+0000 7f9e52764700 1 -- 172.21.15.63:0/3845369215 shutdown_connections 2024-03-22T07:10:31.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.431+0000 7f9e52764700 1 --2- 172.21.15.63:0/3845369215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e4c0fcf80 0x7f9e4c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:31.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.431+0000 7f9e52764700 1 --2- 172.21.15.63:0/3845369215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e4c0fbcd0 0x7f9e4c0fc0e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:31.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.431+0000 7f9e52764700 1 -- 172.21.15.63:0/3845369215 >> 172.21.15.63:0/3845369215 conn(0x7f9e4c0f7430 msgr2=0x7f9e4c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:31.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.431+0000 7f9e52764700 1 -- 172.21.15.63:0/3845369215 shutdown_connections 2024-03-22T07:10:31.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.431+0000 7f9e52764700 1 -- 172.21.15.63:0/3845369215 wait complete. 2024-03-22T07:10:31.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.432+0000 7f9e52764700 1 Processor -- start 2024-03-22T07:10:31.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.432+0000 7f9e52764700 1 -- start start 2024-03-22T07:10:31.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.432+0000 7f9e52764700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e4c0fbcd0 0x7f9e4c10da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:31.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.432+0000 7f9e52764700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e4c0fcf80 0x7f9e4c10df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:31.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.432+0000 7f9e52764700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c10e5f0 con 0x7f9e4c0fbcd0 2024-03-22T07:10:31.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.432+0000 7f9e52764700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c112ef0 con 0x7f9e4c0fcf80 2024-03-22T07:10:31.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.432+0000 7f9e4bfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e4c0fbcd0 0x7f9e4c10da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:31.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.432+0000 7f9e4bfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e4c0fbcd0 0x7f9e4c10da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:55256/0 (socket says 172.21.15.63:55256) 2024-03-22T07:10:31.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.432+0000 7f9e4bfff700 1 -- 172.21.15.63:0/2845109099 learned_addr learned my addr 172.21.15.63:0/2845109099 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:10:31.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.432+0000 7f9e4bfff700 1 -- 172.21.15.63:0/2845109099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e4c0fcf80 msgr2=0x7f9e4c10df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:31.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.432+0000 7f9e4b7fe700 1 --2- 172.21.15.63:0/2845109099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e4c0fcf80 0x7f9e4c10df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:31.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.433+0000 7f9e4bfff700 1 --2- 172.21.15.63:0/2845109099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e4c0fcf80 0x7f9e4c10df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:31.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.433+0000 7f9e4bfff700 1 -- 172.21.15.63:0/2845109099 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e40005040 con 0x7f9e4c0fbcd0 2024-03-22T07:10:31.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.433+0000 7f9e4b7fe700 1 --2- 172.21.15.63:0/2845109099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e4c0fcf80 0x7f9e4c10df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:10:31.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.433+0000 7f9e4bfff700 1 --2- 172.21.15.63:0/2845109099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e4c0fbcd0 0x7f9e4c10da20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009150 tx=0x7f9e40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:31.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.433+0000 7f9e48ff9700 1 -- 172.21.15.63:0/2845109099 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e4000e040 con 0x7f9e4c0fbcd0 2024-03-22T07:10:31.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.433+0000 7f9e52764700 1 -- 172.21.15.63:0/2845109099 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4c1130f0 con 0x7f9e4c0fbcd0 2024-03-22T07:10:31.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.433+0000 7f9e48ff9700 1 -- 172.21.15.63:0/2845109099 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e4000bcc0 con 0x7f9e4c0fbcd0 2024-03-22T07:10:31.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.433+0000 7f9e48ff9700 1 -- 172.21.15.63:0/2845109099 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e4001c630 con 0x7f9e4c0fbcd0 2024-03-22T07:10:31.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.433+0000 7f9e52764700 1 -- 172.21.15.63:0/2845109099 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4c113660 con 0x7f9e4c0fbcd0 2024-03-22T07:10:31.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.434+0000 7f9e52764700 1 -- 172.21.15.63:0/2845109099 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4c101b10 con 0x7f9e4c0fbcd0 2024-03-22T07:10:31.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.434+0000 7f9e48ff9700 1 -- 172.21.15.63:0/2845109099 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f9e40013420 con 0x7f9e4c0fbcd0 2024-03-22T07:10:31.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.435+0000 7f9e48ff9700 1 --2- 172.21.15.63:0/2845109099 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f9e34069210 0x7f9e3406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:31.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.435+0000 7f9e48ff9700 1 -- 172.21.15.63:0/2845109099 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f9e40087290 con 0x7f9e4c0fbcd0 2024-03-22T07:10:31.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.435+0000 7f9e4b7fe700 1 --2- 172.21.15.63:0/2845109099 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f9e34069210 0x7f9e3406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:31.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.435+0000 7f9e4b7fe700 1 --2- 172.21.15.63:0/2845109099 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f9e34069210 0x7f9e3406b6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c006fd0 tx=0x7f9e3c008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:31.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.439+0000 7f9e48ff9700 1 -- 172.21.15.63:0/2845109099 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9e40057d20 con 0x7f9e4c0fbcd0 2024-03-22T07:10:31.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:31.649+0000 7f9e52764700 1 -- 172.21.15.63:0/2845109099 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e4c02b310 con 0x7f9e34069210 2024-03-22T07:10:32.563 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:32 smithi063 ceph-mon[29200]: Detected new or changed devices on smithi063 2024-03-22T07:10:32.563 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:32 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:32.563 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:32 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:32.563 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:32 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:10:32.563 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:32 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:10:32.563 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:32 smithi063 ceph-mon[29200]: Adjusting osd_memory_target on smithi063 to 6432M 2024-03-22T07:10:32.564 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:32 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:32.564 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:32 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:32.564 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:32 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:10:32.564 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:32 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:32.564 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:32 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:32.564 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:32 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:10:32.564 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:32 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:10:32.564 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:32 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:32 smithi069 ceph-mon[36441]: Detected new or changed devices on smithi063 2024-03-22T07:10:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:32 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:32 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:32.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:32 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:10:32.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:32 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:10:32.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:32 smithi069 ceph-mon[36441]: Adjusting osd_memory_target on smithi063 to 6432M 2024-03-22T07:10:32.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:32 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:32.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:32 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:32.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:32 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:10:32.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:32 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:32.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:32 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:32.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:32 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:10:32.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:32 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:10:32.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:32 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:33.568 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:33 smithi063 ceph-mon[29200]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:33.569 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:33 smithi063 ceph-mon[29200]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:33 smithi069 ceph-mon[36441]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi063:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:33 smithi069 ceph-mon[36441]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:35.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:35 smithi063 ceph-mon[29200]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:35.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:35 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3099780099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "140b09d0-1fb6-4a3b-873a-048b797c0180"}]: dispatch 2024-03-22T07:10:35.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:35 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3099780099' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "140b09d0-1fb6-4a3b-873a-048b797c0180"}]': finished 2024-03-22T07:10:35.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:35 smithi063 ceph-mon[29200]: osdmap e16: 3 total, 2 up, 3 in 2024-03-22T07:10:35.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:35 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:10:35.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:35 smithi069 ceph-mon[36441]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:35.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:35 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/3099780099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "140b09d0-1fb6-4a3b-873a-048b797c0180"}]: dispatch 2024-03-22T07:10:35.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:35 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/3099780099' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "140b09d0-1fb6-4a3b-873a-048b797c0180"}]': finished 2024-03-22T07:10:35.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:35 smithi069 ceph-mon[36441]: osdmap e16: 3 total, 2 up, 3 in 2024-03-22T07:10:35.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:35 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:10:36.654 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:36 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/4040423340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:10:36.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:36 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/4040423340' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:10:37.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:37 smithi069 ceph-mon[36441]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:37.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:37 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:37.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:37 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:37.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:37 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:37.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:37 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:10:37.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:37 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:37 smithi063 ceph-mon[29200]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:37 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:37 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:37 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:37 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:10:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:37 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:39.684 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:39 smithi063 ceph-mon[29200]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:39.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:39 smithi069 ceph-mon[36441]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:41.648 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:41 smithi063 ceph-mon[29200]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:41.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:41 smithi069 ceph-mon[36441]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:43.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:43 smithi063 ceph-mon[29200]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:43.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:43 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T07:10:43.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:43 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:43.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:43 smithi069 ceph-mon[36441]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:43.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:43 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T07:10:43.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:43 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:44.614 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:44 smithi063 ceph-mon[29200]: Deploying daemon osd.2 on smithi063 2024-03-22T07:10:44.615 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:10:44.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:44 smithi069 ceph-mon[36441]: Deploying daemon osd.2 on smithi063 2024-03-22T07:10:44.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:44 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:10:45.502 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:45 smithi063 ceph-mon[29200]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:45.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:45 smithi069 ceph-mon[36441]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:47.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:47 smithi069 ceph-mon[36441]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:47.779 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:47 smithi063 ceph-mon[29200]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:49.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:49.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:49.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:49.362 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:49 smithi063 ceph-mon[29200]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:49.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:49.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:49.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:49.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:49 smithi069 ceph-mon[36441]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:51.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:51 smithi069 ceph-mon[36441]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:51 smithi063 ceph-mon[29200]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:52.056 INFO:teuthology.orchestra.run.smithi063.stdout:Created osd(s) 2 on host 'smithi063' 2024-03-22T07:10:52.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:52.055+0000 7f9e48ff9700 1 -- 172.21.15.63:0/2845109099 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9e4c02b310 con 0x7f9e34069210 2024-03-22T07:10:52.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:52.058+0000 7f9e52764700 1 -- 172.21.15.63:0/2845109099 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f9e34069210 msgr2=0x7f9e3406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:52.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:52.058+0000 7f9e52764700 1 --2- 172.21.15.63:0/2845109099 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f9e34069210 0x7f9e3406b6c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c006fd0 tx=0x7f9e3c008040 comp rx=0 tx=0).stop 2024-03-22T07:10:52.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:52.058+0000 7f9e52764700 1 -- 172.21.15.63:0/2845109099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e4c0fbcd0 msgr2=0x7f9e4c10da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:52.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:52.058+0000 7f9e52764700 1 --2- 172.21.15.63:0/2845109099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e4c0fbcd0 0x7f9e4c10da20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009150 tx=0x7f9e40009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:52.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:52.058+0000 7f9e52764700 1 -- 172.21.15.63:0/2845109099 shutdown_connections 2024-03-22T07:10:52.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:52.058+0000 7f9e52764700 1 --2- 172.21.15.63:0/2845109099 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f9e34069210 0x7f9e3406b6c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:52.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:52.058+0000 7f9e52764700 1 --2- 172.21.15.63:0/2845109099 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9e4c0fbcd0 0x7f9e4c10da20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:52.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:52.058+0000 7f9e52764700 1 --2- 172.21.15.63:0/2845109099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e4c0fcf80 0x7f9e4c10df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:52.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:52.058+0000 7f9e52764700 1 -- 172.21.15.63:0/2845109099 >> 172.21.15.63:0/2845109099 conn(0x7f9e4c0f7430 msgr2=0x7f9e4c066820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:52.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:52.058+0000 7f9e52764700 1 -- 172.21.15.63:0/2845109099 shutdown_connections 2024-03-22T07:10:52.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:10:52.058+0000 7f9e52764700 1 -- 172.21.15.63:0/2845109099 wait complete. 2024-03-22T07:10:52.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:52 smithi063 ceph-mon[29200]: from='osd.2 [v2:172.21.15.63:6818/3603088351,v1:172.21.15.63:6819/3603088351]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T07:10:52.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:52 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:52.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:52 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:52.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:52 smithi069 ceph-mon[36441]: from='osd.2 [v2:172.21.15.63:6818/3603088351,v1:172.21.15.63:6819/3603088351]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T07:10:52.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:52 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:52.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:52 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:53.307 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:53 smithi063 ceph-mon[29200]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:53.307 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:53 smithi063 ceph-mon[29200]: from='osd.2 [v2:172.21.15.63:6818/3603088351,v1:172.21.15.63:6819/3603088351]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T07:10:53.307 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:53 smithi063 ceph-mon[29200]: osdmap e17: 3 total, 2 up, 3 in 2024-03-22T07:10:53.307 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:53 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:10:53.308 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:53 smithi063 ceph-mon[29200]: from='osd.2 [v2:172.21.15.63:6818/3603088351,v1:172.21.15.63:6819/3603088351]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-03-22T07:10:53.309 DEBUG:teuthology.orchestra.run.smithi063:osd.2> sudo journalctl -f -n 0 -u ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.2.service 2024-03-22T07:10:53.311 INFO:tasks.cephadm:Deploying osd.3 on smithi069 with /dev/vg_nvme/lv_4... 2024-03-22T07:10:53.311 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-22T07:10:53.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:53 smithi069 ceph-mon[36441]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:53.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:53 smithi069 ceph-mon[36441]: from='osd.2 [v2:172.21.15.63:6818/3603088351,v1:172.21.15.63:6819/3603088351]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T07:10:53.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:53 smithi069 ceph-mon[36441]: osdmap e17: 3 total, 2 up, 3 in 2024-03-22T07:10:53.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:53 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:10:53.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:53 smithi069 ceph-mon[36441]: from='osd.2 [v2:172.21.15.63:6818/3603088351,v1:172.21.15.63:6819/3603088351]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-03-22T07:10:53.572 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi069/config 2024-03-22T07:10:54.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:54 smithi063 ceph-mon[29200]: from='osd.2 [v2:172.21.15.63:6818/3603088351,v1:172.21.15.63:6819/3603088351]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-03-22T07:10:54.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:54 smithi063 ceph-mon[29200]: osdmap e18: 3 total, 2 up, 3 in 2024-03-22T07:10:54.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:54 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:10:54.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:54 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:10:54.648 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:54 smithi069 ceph-mon[36441]: from='osd.2 [v2:172.21.15.63:6818/3603088351,v1:172.21.15.63:6819/3603088351]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]': finished 2024-03-22T07:10:54.648 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:54 smithi069 ceph-mon[36441]: osdmap e18: 3 total, 2 up, 3 in 2024-03-22T07:10:54.648 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:54 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:10:54.648 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:54 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:10:55.322 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-22T07:10:55.322 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-22T07:10:55.323 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-03-22T07:10:55.323 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-03-22T07:10:55.323 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113354 s, 925 MB/s 2024-03-22T07:10:55.323 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 2024-03-22T07:10:55.323 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-03-22T07:10:55.583 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:55 smithi063 ceph-mon[29200]: purged_snaps scrub starts 2024-03-22T07:10:55.583 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:55 smithi063 ceph-mon[29200]: purged_snaps scrub ok 2024-03-22T07:10:55.583 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:55 smithi063 ceph-mon[29200]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:55.583 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:55 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:10:55.583 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:55 smithi063 ceph-mon[29200]: osd.2 [v2:172.21.15.63:6818/3603088351,v1:172.21.15.63:6819/3603088351] boot 2024-03-22T07:10:55.583 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:55 smithi063 ceph-mon[29200]: osdmap e19: 3 total, 3 up, 3 in 2024-03-22T07:10:55.583 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:55 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:10:55.583 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:55 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:55.583 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:55 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:55.619 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:55 smithi069 ceph-mon[36441]: purged_snaps scrub starts 2024-03-22T07:10:55.620 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:55 smithi069 ceph-mon[36441]: purged_snaps scrub ok 2024-03-22T07:10:55.620 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:55 smithi069 ceph-mon[36441]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-03-22T07:10:55.620 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:55 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:10:55.620 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:55 smithi069 ceph-mon[36441]: osd.2 [v2:172.21.15.63:6818/3603088351,v1:172.21.15.63:6819/3603088351] boot 2024-03-22T07:10:55.620 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:55 smithi069 ceph-mon[36441]: osdmap e19: 3 total, 3 up, 3 in 2024-03-22T07:10:55.620 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:55 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:10:55.620 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:55 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:55.620 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:55 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:55.969 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi069:vg_nvme/lv_4 2024-03-22T07:10:56.219 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi069/config 2024-03-22T07:10:56.840 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:56 smithi063 ceph-mon[29200]: osdmap e20: 3 total, 3 up, 3 in 2024-03-22T07:10:56.840 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:56 smithi063 ceph-mon[29200]: pgmap v74: 0 pgs: ; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:10:56.840 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:56 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-22T07:10:57.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:56 smithi069 ceph-mon[36441]: osdmap e20: 3 total, 3 up, 3 in 2024-03-22T07:10:57.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:56 smithi069 ceph-mon[36441]: pgmap v74: 0 pgs: ; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:10:57.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:56 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-03-22T07:10:57.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.679+0000 7f94c55b5700 1 -- 172.21.15.69:0/1844749677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94c00fbce0 msgr2=0x7f94c00fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:57.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.679+0000 7f94c55b5700 1 --2- 172.21.15.69:0/1844749677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94c00fbce0 0x7f94c00fc0f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f94b0004940 tx=0x7f94b0009e30 comp rx=0 tx=0).stop 2024-03-22T07:10:57.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.679+0000 7f94c55b5700 1 -- 172.21.15.69:0/1844749677 shutdown_connections 2024-03-22T07:10:57.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.679+0000 7f94c55b5700 1 --2- 172.21.15.69:0/1844749677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94c00fcf90 0x7f94c00fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:57.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.679+0000 7f94c55b5700 1 --2- 172.21.15.69:0/1844749677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94c00fbce0 0x7f94c00fc0f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:57.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.679+0000 7f94c55b5700 1 -- 172.21.15.69:0/1844749677 >> 172.21.15.69:0/1844749677 conn(0x7f94c00f7440 msgr2=0x7f94c00f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:10:57.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.679+0000 7f94c55b5700 1 -- 172.21.15.69:0/1844749677 shutdown_connections 2024-03-22T07:10:57.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.679+0000 7f94c55b5700 1 -- 172.21.15.69:0/1844749677 wait complete. 2024-03-22T07:10:57.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.680+0000 7f94c55b5700 1 Processor -- start 2024-03-22T07:10:57.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.680+0000 7f94c55b5700 1 -- start start 2024-03-22T07:10:57.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.680+0000 7f94c55b5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94c00fbce0 0x7f94c0066d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:57.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.680+0000 7f94c55b5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94c00fcf90 0x7f94c0112740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:57.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.680+0000 7f94c55b5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c0112c80 con 0x7f94c00fcf90 2024-03-22T07:10:57.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.680+0000 7f94c55b5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c0112dc0 con 0x7f94c00fbce0 2024-03-22T07:10:57.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.680+0000 7f94beffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94c00fbce0 0x7f94c0066d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:57.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.680+0000 7f94beffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94c00fbce0 0x7f94c0066d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47280/0 (socket says 172.21.15.69:47280) 2024-03-22T07:10:57.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.680+0000 7f94beffd700 1 -- 172.21.15.69:0/3800039364 learned_addr learned my addr 172.21.15.69:0/3800039364 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:10:57.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.680+0000 7f94be7fc700 1 --2- 172.21.15.69:0/3800039364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94c00fcf90 0x7f94c0112740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:57.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.680+0000 7f94beffd700 1 -- 172.21.15.69:0/3800039364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94c00fcf90 msgr2=0x7f94c0112740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:10:57.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.680+0000 7f94beffd700 1 --2- 172.21.15.69:0/3800039364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94c00fcf90 0x7f94c0112740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:10:57.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.680+0000 7f94beffd700 1 -- 172.21.15.69:0/3800039364 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b0005040 con 0x7f94c00fbce0 2024-03-22T07:10:57.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.680+0000 7f94beffd700 1 --2- 172.21.15.69:0/3800039364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94c00fbce0 0x7f94c0066d00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f94b0013870 tx=0x7f94b0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:57.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.681+0000 7f94a77fe700 1 -- 172.21.15.69:0/3800039364 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94b0021070 con 0x7f94c00fbce0 2024-03-22T07:10:57.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.681+0000 7f94a77fe700 1 -- 172.21.15.69:0/3800039364 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94b001c400 con 0x7f94c00fbce0 2024-03-22T07:10:57.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.681+0000 7f94c55b5700 1 -- 172.21.15.69:0/3800039364 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c0112fc0 con 0x7f94c00fbce0 2024-03-22T07:10:57.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.681+0000 7f94a77fe700 1 -- 172.21.15.69:0/3800039364 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94b000e070 con 0x7f94c00fbce0 2024-03-22T07:10:57.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.681+0000 7f94c55b5700 1 -- 172.21.15.69:0/3800039364 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c0113420 con 0x7f94c00fbce0 2024-03-22T07:10:57.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.682+0000 7f94c55b5700 1 -- 172.21.15.69:0/3800039364 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c004c710 con 0x7f94c00fbce0 2024-03-22T07:10:57.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.682+0000 7f94a77fe700 1 -- 172.21.15.69:0/3800039364 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 18) v1 ==== 86565+0+0 (secure 0 0 0) 0x7f94b000b7d0 con 0x7f94c00fbce0 2024-03-22T07:10:57.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.682+0000 7f94a77fe700 1 --2- 172.21.15.69:0/3800039364 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f94a8069210 0x7f94a806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:10:57.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.682+0000 7f94a77fe700 1 -- 172.21.15.69:0/3800039364 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3217+0+0 (secure 0 0 0) 0x7f94b0087b40 con 0x7f94c00fbce0 2024-03-22T07:10:57.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.683+0000 7f94be7fc700 1 --2- 172.21.15.69:0/3800039364 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f94a8069210 0x7f94a806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:10:57.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.683+0000 7f94be7fc700 1 --2- 172.21.15.69:0/3800039364 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f94a8069210 0x7f94a806b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f94b4004600 tx=0x7f94b400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:10:57.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.687+0000 7f94a77fe700 1 -- 172.21.15.69:0/3800039364 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f94b00582a0 con 0x7f94c00fbce0 2024-03-22T07:10:57.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:57 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-22T07:10:57.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:57 smithi063 ceph-mon[29200]: osdmap e21: 3 total, 3 up, 3 in 2024-03-22T07:10:57.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:57 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-22T07:10:57.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:57.882+0000 7f94c55b5700 1 -- 172.21.15.69:0/3800039364 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f94c0113640 con 0x7f94a8069210 2024-03-22T07:10:58.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:57 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-03-22T07:10:58.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:57 smithi069 ceph-mon[36441]: osdmap e21: 3 total, 3 up, 3 in 2024-03-22T07:10:58.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:57 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-22T07:10:58.870 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:10:58 smithi063 sudo[67118]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T07:10:58.871 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:10:58 smithi063 sudo[67118]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:10:58.871 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:10:58 smithi063 sudo[67118]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:10:58.871 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:10:58 smithi063 sudo[67118]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:10:58.871 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-22T07:10:58.871 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: osdmap e22: 3 total, 3 up, 3 in 2024-03-22T07:10:58.871 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:58.871 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:10:58.871 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:10:58.871 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:58.871 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: Detected new or changed devices on smithi063 2024-03-22T07:10:58.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:58.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:58.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:10:58.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:10:58.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:10:58.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: Adjusting osd_memory_target on smithi063 to 4288M 2024-03-22T07:10:58.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:58.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:58.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:10:58.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:58.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:58.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:58 smithi063 ceph-mon[29200]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:10:59.122 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:10:58 smithi063 sudo[67140]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T07:10:59.123 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:10:58 smithi063 sudo[67140]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:10:59.123 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:10:58 smithi063 sudo[67140]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:10:59.123 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:10:58 smithi063 sudo[67140]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:10:59.187 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-22T07:10:59.187 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: osdmap e22: 3 total, 3 up, 3 in 2024-03-22T07:10:59.187 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:10:59.187 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:10:59.187 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:10:59.187 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:59.187 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: Detected new or changed devices on smithi063 2024-03-22T07:10:59.188 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:59.188 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:59.188 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:10:59.188 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:10:59.188 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:10:59.188 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: Adjusting osd_memory_target on smithi063 to 4288M 2024-03-22T07:10:59.188 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:59.188 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:10:59.188 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:10:59.188 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:59.188 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:10:59.188 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:58 smithi069 ceph-mon[36441]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:10:59.428 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:10:59 smithi063 sudo[67143]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T07:10:59.428 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:10:59 smithi063 sudo[67143]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:10:59.428 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:10:59 smithi063 sudo[67143]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:10:59.428 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:10:59 smithi063 sudo[67143]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:10:59.686 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:10:59 smithi063 sudo[67226]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T07:10:59.686 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:10:59 smithi063 sudo[67226]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:10:59.686 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:10:59 smithi063 sudo[67226]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:10:59.686 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:10:59 smithi063 sudo[67226]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:10:59.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:10:59.904+0000 7f94a77fe700 1 -- 172.21.15.69:0/3800039364 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f94b001ca20 con 0x7f94c00fbce0 2024-03-22T07:10:59.956 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:10:59 smithi063 sudo[67251]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T07:10:59.956 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:10:59 smithi063 sudo[67251]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:10:59.956 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:10:59 smithi063 sudo[67251]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:10:59.956 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:10:59 smithi063 sudo[67251]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:10:59.957 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:59 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:10:59.957 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:59 smithi063 ceph-mon[29200]: osdmap e23: 3 total, 3 up, 3 in 2024-03-22T07:10:59.957 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:10:59 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:11:00.155 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:59 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:00.155 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:59 smithi069 ceph-mon[36441]: osdmap e23: 3 total, 3 up, 3 in 2024-03-22T07:11:00.155 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:10:59 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:11:00.211 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:10:59 smithi063 sudo[67254]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T07:11:00.211 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:10:59 smithi063 sudo[67254]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:11:00.211 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:10:59 smithi063 sudo[67254]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:11:00.211 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:10:59 smithi063 sudo[67254]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:11:00.477 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 sudo[67302]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T07:11:00.477 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 sudo[67302]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:11:00.478 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 sudo[67302]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:11:00.478 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 sudo[67302]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:11:00.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 sudo[67332]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T07:11:00.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 sudo[67332]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:11:00.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 sudo[67332]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:11:00.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 sudo[67332]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:11:00.984 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 sudo[39791]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-22T07:11:00.985 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 sudo[39791]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:11:00.985 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 sudo[39791]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:11:00.985 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 sudo[39791]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:11:01.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 sudo[39794]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-22T07:11:01.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 sudo[39794]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-22T07:11:01.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 sudo[39794]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-22T07:11:01.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 sudo[39794]: pam_unix(sudo:session): session closed for user root 2024-03-22T07:11:01.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 ceph-mon[36441]: mgrmap e19: smithi063.mnxayh(active, since 2m), standbys: smithi069.gfziwh 2024-03-22T07:11:01.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 ceph-mon[36441]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T07:11:01.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 ceph-mon[36441]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:01.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/1804940667' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d92b77e-6bcc-4585-96f1-72251303cc9b"}]: dispatch 2024-03-22T07:11:01.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 ceph-mon[36441]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T07:11:01.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:11:01.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 ceph-mon[36441]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d92b77e-6bcc-4585-96f1-72251303cc9b"}]: dispatch 2024-03-22T07:11:01.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:11:01.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 ceph-mon[36441]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d92b77e-6bcc-4585-96f1-72251303cc9b"}]': finished 2024-03-22T07:11:01.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 ceph-mon[36441]: osdmap e24: 4 total, 3 up, 4 in 2024-03-22T07:11:01.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:11:01.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:11:01.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:00 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:11:01.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 ceph-mon[29200]: mgrmap e19: smithi063.mnxayh(active, since 2m), standbys: smithi069.gfziwh 2024-03-22T07:11:01.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 ceph-mon[29200]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T07:11:01.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 ceph-mon[29200]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:01.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/1804940667' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d92b77e-6bcc-4585-96f1-72251303cc9b"}]: dispatch 2024-03-22T07:11:01.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 ceph-mon[29200]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T07:11:01.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:11:01.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 ceph-mon[29200]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d92b77e-6bcc-4585-96f1-72251303cc9b"}]: dispatch 2024-03-22T07:11:01.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:11:01.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 ceph-mon[29200]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d92b77e-6bcc-4585-96f1-72251303cc9b"}]': finished 2024-03-22T07:11:01.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 ceph-mon[29200]: osdmap e24: 4 total, 3 up, 4 in 2024-03-22T07:11:01.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:11:01.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:11:01.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:00 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:11:02.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:02 smithi069 ceph-mon[36441]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T07:11:02.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:02 smithi069 ceph-mon[36441]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T07:11:02.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:02 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:02.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:02 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/3788189342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:11:02.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:02 smithi063 ceph-mon[29200]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-22T07:11:02.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:02 smithi063 ceph-mon[29200]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-22T07:11:02.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:02 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:02.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:02 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/3788189342' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:11:03.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:03 smithi063 ceph-mon[29200]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:03.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:03 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:03.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:03 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:03.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:03 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:03.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:03 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:03.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:03 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:03.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:03 smithi069 ceph-mon[36441]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:03.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:03 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:03.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:03 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:03.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:03 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:03.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:03 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:03.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:03 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:05.436 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:05 smithi069 ceph-mon[36441]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:05.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:05 smithi063 ceph-mon[29200]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 875 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:07 smithi063 ceph-mon[29200]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:07.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:07 smithi069 ceph-mon[36441]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:08.517 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T07:11:08.518 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:08.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:08 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T07:11:08.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:08 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:09.521 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:09 smithi069 ceph-mon[36441]: Deploying daemon osd.3 on smithi069 2024-03-22T07:11:09.521 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:09 smithi069 ceph-mon[36441]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:09.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:09 smithi063 ceph-mon[29200]: Deploying daemon osd.3 on smithi069 2024-03-22T07:11:09.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:09 smithi063 ceph-mon[29200]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:11.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:11 smithi063 ceph-mon[29200]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:11.629 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:11 smithi069 ceph-mon[36441]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:13.404 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:13 smithi069 ceph-mon[36441]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:13.404 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:13 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:13.404 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:13 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:13.404 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:13 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:13 smithi063 ceph-mon[29200]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:13 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:13 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:13 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:14 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:11:14.777 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:14 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:11:15.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:15 smithi069 ceph-mon[36441]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:15.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:15 smithi063 ceph-mon[29200]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:15.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:15.897+0000 7f94a77fe700 1 -- 172.21.15.69:0/3800039364 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f94c0113640 con 0x7f94a8069210 2024-03-22T07:11:15.897 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 3 on host 'smithi069' 2024-03-22T07:11:15.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:15.899+0000 7f94c55b5700 1 -- 172.21.15.69:0/3800039364 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f94a8069210 msgr2=0x7f94a806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:15.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:15.899+0000 7f94c55b5700 1 --2- 172.21.15.69:0/3800039364 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f94a8069210 0x7f94a806b6c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f94b4004600 tx=0x7f94b400b040 comp rx=0 tx=0).stop 2024-03-22T07:11:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:15.899+0000 7f94c55b5700 1 -- 172.21.15.69:0/3800039364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94c00fbce0 msgr2=0x7f94c0066d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:15.899+0000 7f94c55b5700 1 --2- 172.21.15.69:0/3800039364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94c00fbce0 0x7f94c0066d00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f94b0013870 tx=0x7f94b0009e30 comp rx=0 tx=0).stop 2024-03-22T07:11:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:15.900+0000 7f94c55b5700 1 -- 172.21.15.69:0/3800039364 shutdown_connections 2024-03-22T07:11:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:15.900+0000 7f94c55b5700 1 --2- 172.21.15.69:0/3800039364 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f94a8069210 0x7f94a806b6c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:15.900+0000 7f94c55b5700 1 --2- 172.21.15.69:0/3800039364 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94c00fbce0 0x7f94c0066d00 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:15.900+0000 7f94c55b5700 1 --2- 172.21.15.69:0/3800039364 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f94c00fcf90 0x7f94c0112740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:15.900+0000 7f94c55b5700 1 -- 172.21.15.69:0/3800039364 >> 172.21.15.69:0/3800039364 conn(0x7f94c00f7440 msgr2=0x7f94c00f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:11:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:15.900+0000 7f94c55b5700 1 -- 172.21.15.69:0/3800039364 shutdown_connections 2024-03-22T07:11:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:15.900+0000 7f94c55b5700 1 -- 172.21.15.69:0/3800039364 wait complete. 2024-03-22T07:11:16.978 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:16 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:16.978 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:16 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:16.978 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:16 smithi069 ceph-mon[36441]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:16.978 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:16 smithi069 ceph-mon[36441]: from='osd.3 [v2:172.21.15.69:6800/1811566075,v1:172.21.15.69:6801/1811566075]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T07:11:16.978 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:16 smithi069 ceph-mon[36441]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T07:11:17.001 DEBUG:teuthology.orchestra.run.smithi069:osd.3> sudo journalctl -f -n 0 -u ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.3.service 2024-03-22T07:11:17.004 INFO:tasks.cephadm:Deploying osd.4 on smithi069 with /dev/vg_nvme/lv_3... 2024-03-22T07:11:17.004 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-22T07:11:17.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:16 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:17.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:16 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:17.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:16 smithi063 ceph-mon[29200]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:17.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:16 smithi063 ceph-mon[29200]: from='osd.3 [v2:172.21.15.69:6800/1811566075,v1:172.21.15.69:6801/1811566075]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T07:11:17.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:16 smithi063 ceph-mon[29200]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T07:11:17.307 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi069/config 2024-03-22T07:11:18.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:17 smithi069 ceph-mon[36441]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T07:11:18.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:17 smithi069 ceph-mon[36441]: osdmap e25: 4 total, 3 up, 4 in 2024-03-22T07:11:18.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:17 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:11:18.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:17 smithi069 ceph-mon[36441]: from='osd.3 [v2:172.21.15.69:6800/1811566075,v1:172.21.15.69:6801/1811566075]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:11:18.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:17 smithi069 ceph-mon[36441]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:11:18.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:17 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:18.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:17 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:18.030 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:11:17 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3[41863]: 2024-03-22T07:11:17.868+0000 7f14b5950700 -1 osd.3 0 waiting for initial osdmap 2024-03-22T07:11:18.030 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:11:17 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3[41863]: 2024-03-22T07:11:17.874+0000 7f14af749700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:11:18.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:17 smithi063 ceph-mon[29200]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T07:11:18.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:17 smithi063 ceph-mon[29200]: osdmap e25: 4 total, 3 up, 4 in 2024-03-22T07:11:18.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:17 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:11:18.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:17 smithi063 ceph-mon[29200]: from='osd.3 [v2:172.21.15.69:6800/1811566075,v1:172.21.15.69:6801/1811566075]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:11:18.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:17 smithi063 ceph-mon[29200]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:11:18.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:17 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:18.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:17 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:19.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:18 smithi069 ceph-mon[36441]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-03-22T07:11:19.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:18 smithi069 ceph-mon[36441]: osdmap e26: 4 total, 3 up, 4 in 2024-03-22T07:11:19.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:18 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:11:19.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:18 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:11:19.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:18 smithi069 ceph-mon[36441]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:19.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:18 smithi063 ceph-mon[29200]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-03-22T07:11:19.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:18 smithi063 ceph-mon[29200]: osdmap e26: 4 total, 3 up, 4 in 2024-03-22T07:11:19.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:18 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:11:19.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:18 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:11:19.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:18 smithi063 ceph-mon[29200]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-03-22T07:11:19.171 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-22T07:11:19.171 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-22T07:11:19.171 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-03-22T07:11:19.171 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-03-22T07:11:19.171 INFO:teuthology.orchestra.run.smithi069.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0135295 s, 775 MB/s 2024-03-22T07:11:19.172 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-03-22T07:11:20.086 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi069:vg_nvme/lv_3 2024-03-22T07:11:20.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:19 smithi063 ceph-mon[29200]: purged_snaps scrub starts 2024-03-22T07:11:20.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:19 smithi063 ceph-mon[29200]: purged_snaps scrub ok 2024-03-22T07:11:20.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:19 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:11:20.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:19 smithi063 ceph-mon[29200]: osd.3 [v2:172.21.15.69:6800/1811566075,v1:172.21.15.69:6801/1811566075] boot 2024-03-22T07:11:20.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:19 smithi063 ceph-mon[29200]: osdmap e27: 4 total, 4 up, 4 in 2024-03-22T07:11:20.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:19 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:11:20.155 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:19 smithi069 ceph-mon[36441]: purged_snaps scrub starts 2024-03-22T07:11:20.155 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:19 smithi069 ceph-mon[36441]: purged_snaps scrub ok 2024-03-22T07:11:20.155 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:19 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:11:20.155 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:19 smithi069 ceph-mon[36441]: osd.3 [v2:172.21.15.69:6800/1811566075,v1:172.21.15.69:6801/1811566075] boot 2024-03-22T07:11:20.155 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:19 smithi069 ceph-mon[36441]: osdmap e27: 4 total, 4 up, 4 in 2024-03-22T07:11:20.155 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:19 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:11:20.938 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi069/config 2024-03-22T07:11:21.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:20 smithi069 ceph-mon[36441]: osdmap e28: 4 total, 4 up, 4 in 2024-03-22T07:11:21.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:20 smithi069 ceph-mon[36441]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:11:21.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:20 smithi063 ceph-mon[29200]: osdmap e28: 4 total, 4 up, 4 in 2024-03-22T07:11:21.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:20 smithi063 ceph-mon[29200]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:11:21.942 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:21 smithi069 ceph-mon[36441]: osdmap e29: 4 total, 4 up, 4 in 2024-03-22T07:11:22.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:21 smithi063 ceph-mon[29200]: osdmap e29: 4 total, 4 up, 4 in 2024-03-22T07:11:23.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.074+0000 7f27e3c14700 1 -- 172.21.15.69:0/1497076137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27dc0fdfe0 msgr2=0x7f27dc0fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:23.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.074+0000 7f27e3c14700 1 --2- 172.21.15.69:0/1497076137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27dc0fdfe0 0x7f27dc0fe450 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f27cc004940 tx=0x7f27cc009e30 comp rx=0 tx=0).stop 2024-03-22T07:11:23.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.074+0000 7f27e3c14700 1 -- 172.21.15.69:0/1497076137 shutdown_connections 2024-03-22T07:11:23.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.074+0000 7f27e3c14700 1 --2- 172.21.15.69:0/1497076137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27dc0fdfe0 0x7f27dc0fe450 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:23.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.074+0000 7f27e3c14700 1 --2- 172.21.15.69:0/1497076137 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27dc0fce70 0x7f27dc0fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:23.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.074+0000 7f27e3c14700 1 -- 172.21.15.69:0/1497076137 >> 172.21.15.69:0/1497076137 conn(0x7f27dc0f8550 msgr2=0x7f27dc0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:11:23.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.074+0000 7f27e3c14700 1 -- 172.21.15.69:0/1497076137 shutdown_connections 2024-03-22T07:11:23.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.074+0000 7f27e3c14700 1 -- 172.21.15.69:0/1497076137 wait complete. 2024-03-22T07:11:23.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.074+0000 7f27e3c14700 1 Processor -- start 2024-03-22T07:11:23.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.074+0000 7f27e3c14700 1 -- start start 2024-03-22T07:11:23.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.074+0000 7f27e3c14700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27dc0fce70 0x7f27dc10d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:23.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.074+0000 7f27e3c14700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27dc0fdfe0 0x7f27dc10de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:23.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.074+0000 7f27e3c14700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc10e460 con 0x7f27dc0fce70 2024-03-22T07:11:23.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.074+0000 7f27e3c14700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc10e5d0 con 0x7f27dc0fdfe0 2024-03-22T07:11:23.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.075+0000 7f27e11af700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27dc0fdfe0 0x7f27dc10de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:23.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.075+0000 7f27e11af700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27dc0fdfe0 0x7f27dc10de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41746/0 (socket says 172.21.15.69:41746) 2024-03-22T07:11:23.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.075+0000 7f27e11af700 1 -- 172.21.15.69:0/529206221 learned_addr learned my addr 172.21.15.69:0/529206221 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:11:23.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.075+0000 7f27e19b0700 1 --2- 172.21.15.69:0/529206221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27dc0fce70 0x7f27dc10d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:23.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.075+0000 7f27e11af700 1 -- 172.21.15.69:0/529206221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27dc0fce70 msgr2=0x7f27dc10d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:23.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.075+0000 7f27e11af700 1 --2- 172.21.15.69:0/529206221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27dc0fce70 0x7f27dc10d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:23.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.075+0000 7f27e11af700 1 -- 172.21.15.69:0/529206221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc005040 con 0x7f27dc0fdfe0 2024-03-22T07:11:23.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.075+0000 7f27e11af700 1 --2- 172.21.15.69:0/529206221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27dc0fdfe0 0x7f27dc10de60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00b3b0 tx=0x7f27cc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:11:23.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.075+0000 7f27d27fc700 1 -- 172.21.15.69:0/529206221 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27cc00e040 con 0x7f27dc0fdfe0 2024-03-22T07:11:23.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.075+0000 7f27e3c14700 1 -- 172.21.15.69:0/529206221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27dc112fd0 con 0x7f27dc0fdfe0 2024-03-22T07:11:23.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.075+0000 7f27d27fc700 1 -- 172.21.15.69:0/529206221 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27cc00bbf0 con 0x7f27dc0fdfe0 2024-03-22T07:11:23.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.075+0000 7f27d27fc700 1 -- 172.21.15.69:0/529206221 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27cc01c630 con 0x7f27dc0fdfe0 2024-03-22T07:11:23.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.075+0000 7f27e3c14700 1 -- 172.21.15.69:0/529206221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27dc113560 con 0x7f27dc0fdfe0 2024-03-22T07:11:23.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.076+0000 7f27e3c14700 1 -- 172.21.15.69:0/529206221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27dc04c710 con 0x7f27dc0fdfe0 2024-03-22T07:11:23.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.077+0000 7f27d27fc700 1 -- 172.21.15.69:0/529206221 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f27cc00bd60 con 0x7f27dc0fdfe0 2024-03-22T07:11:23.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.077+0000 7f27d27fc700 1 --2- 172.21.15.69:0/529206221 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f27c80692a0 0x7f27c806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:23.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.077+0000 7f27d27fc700 1 -- 172.21.15.69:0/529206221 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3770+0+0 (secure 0 0 0) 0x7f27cc0877d0 con 0x7f27dc0fdfe0 2024-03-22T07:11:23.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.077+0000 7f27e19b0700 1 --2- 172.21.15.69:0/529206221 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f27c80692a0 0x7f27c806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:23.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.078+0000 7f27e19b0700 1 --2- 172.21.15.69:0/529206221 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f27c80692a0 0x7f27c806b750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f27d8004f40 tx=0x7f27d800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:11:23.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.081+0000 7f27d27fc700 1 -- 172.21.15.69:0/529206221 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f27cc057c30 con 0x7f27dc0fdfe0 2024-03-22T07:11:23.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:23.272+0000 7f27e3c14700 1 -- 172.21.15.69:0/529206221 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f27dc102370 con 0x7f27c80692a0 2024-03-22T07:11:23.272 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:22 smithi069 ceph-mon[36441]: osdmap e30: 4 total, 4 up, 4 in 2024-03-22T07:11:23.272 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:22 smithi069 ceph-mon[36441]: Detected new or changed devices on smithi069 2024-03-22T07:11:23.273 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:22 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:23.273 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:22 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:23.273 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:22 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:11:23.273 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:22 smithi069 ceph-mon[36441]: Adjusting osd_memory_target on smithi069 to 15936M 2024-03-22T07:11:23.273 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:22 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:23.273 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:22 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:23.273 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:22 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:23.273 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:22 smithi069 ceph-mon[36441]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-22T07:11:23.273 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:22 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:23.273 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:22 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:23.273 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:22 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:23.273 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:22 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:23.273 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:22 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:22 smithi063 ceph-mon[29200]: osdmap e30: 4 total, 4 up, 4 in 2024-03-22T07:11:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:22 smithi063 ceph-mon[29200]: Detected new or changed devices on smithi069 2024-03-22T07:11:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:22 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:22 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:22 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:11:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:22 smithi063 ceph-mon[29200]: Adjusting osd_memory_target on smithi069 to 15936M 2024-03-22T07:11:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:22 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:22 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:23.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:22 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:23.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:22 smithi063 ceph-mon[29200]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-22T07:11:23.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:22 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:23.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:22 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:23.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:22 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:23.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:22 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:23.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:22 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:24.001 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:23 smithi069 ceph-mon[36441]: osdmap e31: 4 total, 4 up, 4 in 2024-03-22T07:11:24.001 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:23 smithi069 ceph-mon[36441]: from='client.24161 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:11:24.001 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:23 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:11:24.001 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:23 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:11:24.001 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:23 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:24.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:23 smithi063 ceph-mon[29200]: osdmap e31: 4 total, 4 up, 4 in 2024-03-22T07:11:24.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:23 smithi063 ceph-mon[29200]: from='client.24161 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:11:24.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:23 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:11:24.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:23 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:11:24.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:23 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:25.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:24 smithi069 ceph-mon[36441]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-03-22T07:11:25.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:24 smithi063 ceph-mon[29200]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-03-22T07:11:26.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:25 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/2274094074' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28bdc8bc-41d5-4c01-9b29-d5833223d1af"}]: dispatch 2024-03-22T07:11:26.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:25 smithi069 ceph-mon[36441]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28bdc8bc-41d5-4c01-9b29-d5833223d1af"}]: dispatch 2024-03-22T07:11:26.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:25 smithi069 ceph-mon[36441]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28bdc8bc-41d5-4c01-9b29-d5833223d1af"}]': finished 2024-03-22T07:11:26.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:25 smithi069 ceph-mon[36441]: osdmap e32: 5 total, 4 up, 5 in 2024-03-22T07:11:26.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:25 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:11:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:25 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/2274094074' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28bdc8bc-41d5-4c01-9b29-d5833223d1af"}]: dispatch 2024-03-22T07:11:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:25 smithi063 ceph-mon[29200]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28bdc8bc-41d5-4c01-9b29-d5833223d1af"}]: dispatch 2024-03-22T07:11:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:25 smithi063 ceph-mon[29200]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28bdc8bc-41d5-4c01-9b29-d5833223d1af"}]': finished 2024-03-22T07:11:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:25 smithi063 ceph-mon[29200]: osdmap e32: 5 total, 4 up, 5 in 2024-03-22T07:11:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:25 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:11:27.274 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:26 smithi063 ceph-mon[29200]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-03-22T07:11:27.274 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:26 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/3962801090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:11:27.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:26 smithi069 ceph-mon[36441]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-03-22T07:11:27.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:26 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/3962801090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:11:29.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:29 smithi069 ceph-mon[36441]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-03-22T07:11:29.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:29 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:11:29.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:29 smithi063 ceph-mon[29200]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-03-22T07:11:29.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:29 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:11:31.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:31 smithi069 ceph-mon[36441]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:11:31.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:31 smithi063 ceph-mon[29200]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:11:33.560 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:33 smithi069 ceph-mon[36441]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:11:33.561 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:33 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T07:11:33.561 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:33 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:33.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:33 smithi063 ceph-mon[29200]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:11:33.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:33 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T07:11:33.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:33 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:34.330 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:34 smithi069 ceph-mon[36441]: Deploying daemon osd.4 on smithi069 2024-03-22T07:11:34.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:34 smithi063 ceph-mon[29200]: Deploying daemon osd.4 on smithi069 2024-03-22T07:11:35.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:35 smithi069 ceph-mon[36441]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:11:35.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:35 smithi063 ceph-mon[29200]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:11:37.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:37 smithi069 ceph-mon[36441]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:11:37.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:37 smithi063 ceph-mon[29200]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:11:39.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:38 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:39.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:38 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:39.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:38 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:39.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:38 smithi069 ceph-mon[36441]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:11:39.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:38 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:39.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:38 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:39.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:38 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:39.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:38 smithi063 ceph-mon[29200]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:11:41.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:41 smithi069 ceph-mon[36441]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:11:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:41 smithi063 ceph-mon[29200]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:11:41.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:41.688+0000 7f27d27fc700 1 -- 172.21.15.69:0/529206221 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f27dc102370 con 0x7f27c80692a0 2024-03-22T07:11:41.689 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 4 on host 'smithi069' 2024-03-22T07:11:41.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:41.690+0000 7f27e3c14700 1 -- 172.21.15.69:0/529206221 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f27c80692a0 msgr2=0x7f27c806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:41.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:41.690+0000 7f27e3c14700 1 --2- 172.21.15.69:0/529206221 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f27c80692a0 0x7f27c806b750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f27d8004f40 tx=0x7f27d800b040 comp rx=0 tx=0).stop 2024-03-22T07:11:41.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:41.690+0000 7f27e3c14700 1 -- 172.21.15.69:0/529206221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27dc0fdfe0 msgr2=0x7f27dc10de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:41.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:41.690+0000 7f27e3c14700 1 --2- 172.21.15.69:0/529206221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27dc0fdfe0 0x7f27dc10de60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00b3b0 tx=0x7f27cc009e30 comp rx=0 tx=0).stop 2024-03-22T07:11:41.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:41.690+0000 7f27e3c14700 1 -- 172.21.15.69:0/529206221 shutdown_connections 2024-03-22T07:11:41.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:41.690+0000 7f27e3c14700 1 --2- 172.21.15.69:0/529206221 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f27c80692a0 0x7f27c806b750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:41.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:41.690+0000 7f27e3c14700 1 --2- 172.21.15.69:0/529206221 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f27dc0fce70 0x7f27dc10d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:41.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:41.690+0000 7f27e3c14700 1 --2- 172.21.15.69:0/529206221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27dc0fdfe0 0x7f27dc10de60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:41.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:41.690+0000 7f27e3c14700 1 -- 172.21.15.69:0/529206221 >> 172.21.15.69:0/529206221 conn(0x7f27dc0f8550 msgr2=0x7f27dc0f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:11:41.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:41.691+0000 7f27e3c14700 1 -- 172.21.15.69:0/529206221 shutdown_connections 2024-03-22T07:11:41.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:41.691+0000 7f27e3c14700 1 -- 172.21.15.69:0/529206221 wait complete. 2024-03-22T07:11:42.370 DEBUG:teuthology.orchestra.run.smithi069:osd.4> sudo journalctl -f -n 0 -u ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.4.service 2024-03-22T07:11:42.373 INFO:tasks.cephadm:Deploying osd.5 on smithi069 with /dev/vg_nvme/lv_2... 2024-03-22T07:11:42.373 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-22T07:11:42.676 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi069/config 2024-03-22T07:11:42.919 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:42 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:42.919 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:42 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:42.919 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:42 smithi069 ceph-mon[36441]: from='osd.4 [v2:172.21.15.69:6808/656541591,v1:172.21.15.69:6809/656541591]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T07:11:42.920 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:42 smithi069 ceph-mon[36441]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T07:11:42.920 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:42 smithi069 ceph-mon[36441]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:11:42.920 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:42 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:42.920 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:42 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:43.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:42 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:43.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:42 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:43.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:42 smithi063 ceph-mon[29200]: from='osd.4 [v2:172.21.15.69:6808/656541591,v1:172.21.15.69:6809/656541591]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T07:11:43.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:42 smithi063 ceph-mon[29200]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T07:11:43.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:42 smithi063 ceph-mon[29200]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:11:43.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:42 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:43.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:42 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:43.777 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:43 smithi069 ceph-mon[36441]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T07:11:43.778 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:43 smithi069 ceph-mon[36441]: osdmap e33: 5 total, 4 up, 5 in 2024-03-22T07:11:43.778 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:43 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:11:43.778 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:43 smithi069 ceph-mon[36441]: from='osd.4 [v2:172.21.15.69:6808/656541591,v1:172.21.15.69:6809/656541591]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:11:43.778 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:43 smithi069 ceph-mon[36441]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:11:43.781 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:11:43 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[48724]: 2024-03-22T07:11:43.667+0000 7f8a92230700 -1 osd.4 0 waiting for initial osdmap 2024-03-22T07:11:43.781 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:11:43 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[48724]: 2024-03-22T07:11:43.674+0000 7f8a8d02b700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:11:44.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:43 smithi063 ceph-mon[29200]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T07:11:44.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:43 smithi063 ceph-mon[29200]: osdmap e33: 5 total, 4 up, 5 in 2024-03-22T07:11:44.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:43 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:11:44.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:43 smithi063 ceph-mon[29200]: from='osd.4 [v2:172.21.15.69:6808/656541591,v1:172.21.15.69:6809/656541591]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:11:44.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:43 smithi063 ceph-mon[29200]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:11:44.577 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-22T07:11:44.578 INFO:teuthology.orchestra.run.smithi069.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-22T07:11:44.578 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10+0 records in 2024-03-22T07:11:44.578 INFO:teuthology.orchestra.run.smithi069.stderr:10+0 records out 2024-03-22T07:11:44.578 INFO:teuthology.orchestra.run.smithi069.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123519 s, 849 MB/s 2024-03-22T07:11:44.578 INFO:teuthology.orchestra.run.smithi069.stderr:--> Zapping successful for: 2024-03-22T07:11:44.932 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:44 smithi069 ceph-mon[36441]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-03-22T07:11:44.933 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:44 smithi069 ceph-mon[36441]: osdmap e34: 5 total, 4 up, 5 in 2024-03-22T07:11:44.933 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:44 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:11:44.933 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:44 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:11:44.933 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:44 smithi069 ceph-mon[36441]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:11:44.933 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:44 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:11:44.933 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:44 smithi069 ceph-mon[36441]: osd.4 [v2:172.21.15.69:6808/656541591,v1:172.21.15.69:6809/656541591] boot 2024-03-22T07:11:44.933 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:44 smithi069 ceph-mon[36441]: osdmap e35: 5 total, 5 up, 5 in 2024-03-22T07:11:44.933 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:44 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:11:45.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:44 smithi063 ceph-mon[29200]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-03-22T07:11:45.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:44 smithi063 ceph-mon[29200]: osdmap e34: 5 total, 4 up, 5 in 2024-03-22T07:11:45.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:11:45.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:11:45.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:44 smithi063 ceph-mon[29200]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-03-22T07:11:45.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:11:45.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:44 smithi063 ceph-mon[29200]: osd.4 [v2:172.21.15.69:6808/656541591,v1:172.21.15.69:6809/656541591] boot 2024-03-22T07:11:45.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:44 smithi063 ceph-mon[29200]: osdmap e35: 5 total, 5 up, 5 in 2024-03-22T07:11:45.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:11:45.860 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi069:vg_nvme/lv_2 2024-03-22T07:11:46.006 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:45 smithi069 ceph-mon[36441]: purged_snaps scrub starts 2024-03-22T07:11:46.006 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:45 smithi069 ceph-mon[36441]: purged_snaps scrub ok 2024-03-22T07:11:46.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:45 smithi063 ceph-mon[29200]: purged_snaps scrub starts 2024-03-22T07:11:46.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:45 smithi063 ceph-mon[29200]: purged_snaps scrub ok 2024-03-22T07:11:46.377 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi069/config 2024-03-22T07:11:46.900 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:46 smithi069 ceph-mon[36441]: osdmap e36: 5 total, 5 up, 5 in 2024-03-22T07:11:46.900 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:46 smithi069 ceph-mon[36441]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:11:47.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:46 smithi063 ceph-mon[29200]: osdmap e36: 5 total, 5 up, 5 in 2024-03-22T07:11:47.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:46 smithi063 ceph-mon[29200]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:11:48.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.524+0000 7fe4ca4a9700 1 -- 172.21.15.69:0/2269345759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4c40fbb50 msgr2=0x7fe4c4102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:48.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.524+0000 7fe4ca4a9700 1 --2- 172.21.15.69:0/2269345759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4c40fbb50 0x7fe4c4102360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8004940 tx=0x7fe4b8009e30 comp rx=0 tx=0).stop 2024-03-22T07:11:48.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.524+0000 7fe4ca4a9700 1 -- 172.21.15.69:0/2269345759 shutdown_connections 2024-03-22T07:11:48.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.524+0000 7fe4ca4a9700 1 --2- 172.21.15.69:0/2269345759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4c40fbb50 0x7fe4c4102360 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:48.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.524+0000 7fe4ca4a9700 1 --2- 172.21.15.69:0/2269345759 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4c40fb1e0 0x7fe4c40fb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:48.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.524+0000 7fe4ca4a9700 1 -- 172.21.15.69:0/2269345759 >> 172.21.15.69:0/2269345759 conn(0x7fe4c40f6ba0 msgr2=0x7fe4c40f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:11:48.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.524+0000 7fe4ca4a9700 1 -- 172.21.15.69:0/2269345759 shutdown_connections 2024-03-22T07:11:48.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.524+0000 7fe4ca4a9700 1 -- 172.21.15.69:0/2269345759 wait complete. 2024-03-22T07:11:48.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.524+0000 7fe4ca4a9700 1 Processor -- start 2024-03-22T07:11:48.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.525+0000 7fe4ca4a9700 1 -- start start 2024-03-22T07:11:48.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.525+0000 7fe4ca4a9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4c40fb1e0 0x7fe4c410da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:48.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.525+0000 7fe4ca4a9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4c40fbb50 0x7fe4c410df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:48.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.525+0000 7fe4ca4a9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c410e620 con 0x7fe4c40fbb50 2024-03-22T07:11:48.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.525+0000 7fe4ca4a9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c410ee10 con 0x7fe4c40fb1e0 2024-03-22T07:11:48.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.525+0000 7fe4c3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4c40fb1e0 0x7fe4c410da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:48.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.525+0000 7fe4c3fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4c40fb1e0 0x7fe4c410da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45006/0 (socket says 172.21.15.69:45006) 2024-03-22T07:11:48.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.525+0000 7fe4c3fff700 1 -- 172.21.15.69:0/2339616191 learned_addr learned my addr 172.21.15.69:0/2339616191 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:11:48.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.525+0000 7fe4c37fe700 1 --2- 172.21.15.69:0/2339616191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4c40fbb50 0x7fe4c410df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:48.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.525+0000 7fe4c3fff700 1 -- 172.21.15.69:0/2339616191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4c40fbb50 msgr2=0x7fe4c410df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:11:48.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.525+0000 7fe4c3fff700 1 --2- 172.21.15.69:0/2339616191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4c40fbb50 0x7fe4c410df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:11:48.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.525+0000 7fe4c3fff700 1 -- 172.21.15.69:0/2339616191 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b8005040 con 0x7fe4c40fb1e0 2024-03-22T07:11:48.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.525+0000 7fe4c3fff700 1 --2- 172.21.15.69:0/2339616191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4c40fb1e0 0x7fe4c410da50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400b790 tx=0x7fe4b4009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:11:48.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.525+0000 7fe4c0ff9700 1 -- 172.21.15.69:0/2339616191 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4b400f6f0 con 0x7fe4c40fb1e0 2024-03-22T07:11:48.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.526+0000 7fe4ca4a9700 1 -- 172.21.15.69:0/2339616191 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c410f0f0 con 0x7fe4c40fb1e0 2024-03-22T07:11:48.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.526+0000 7fe4c0ff9700 1 -- 172.21.15.69:0/2339616191 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4b400fd20 con 0x7fe4c40fb1e0 2024-03-22T07:11:48.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.526+0000 7fe4c0ff9700 1 -- 172.21.15.69:0/2339616191 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4b40103a0 con 0x7fe4c40fb1e0 2024-03-22T07:11:48.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.526+0000 7fe4ca4a9700 1 -- 172.21.15.69:0/2339616191 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c410f6b0 con 0x7fe4c40fb1e0 2024-03-22T07:11:48.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.527+0000 7fe4ca4a9700 1 -- 172.21.15.69:0/2339616191 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c405f5f0 con 0x7fe4c40fb1e0 2024-03-22T07:11:48.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.527+0000 7fe4c0ff9700 1 -- 172.21.15.69:0/2339616191 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe4b4011400 con 0x7fe4c40fb1e0 2024-03-22T07:11:48.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.527+0000 7fe4c0ff9700 1 --2- 172.21.15.69:0/2339616191 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe4ac0692b0 0x7fe4ac06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:11:48.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.528+0000 7fe4c0ff9700 1 -- 172.21.15.69:0/2339616191 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4202+0+0 (secure 0 0 0) 0x7fe4b4086a50 con 0x7fe4c40fb1e0 2024-03-22T07:11:48.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.528+0000 7fe4c37fe700 1 --2- 172.21.15.69:0/2339616191 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe4ac0692b0 0x7fe4ac06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:11:48.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.528+0000 7fe4c37fe700 1 --2- 172.21.15.69:0/2339616191 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe4ac0692b0 0x7fe4ac06b760 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009fd0 tx=0x7fe4b8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:11:48.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.532+0000 7fe4c0ff9700 1 -- 172.21.15.69:0/2339616191 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe4b4056d00 con 0x7fe4c40fb1e0 2024-03-22T07:11:48.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:11:48.723+0000 7fe4ca4a9700 1 -- 172.21.15.69:0/2339616191 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4c410f9f0 con 0x7fe4ac0692b0 2024-03-22T07:11:49.336 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:49 smithi069 ceph-mon[36441]: Detected new or changed devices on smithi069 2024-03-22T07:11:49.336 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:49.336 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:49.336 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:11:49.336 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:11:49.336 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:49 smithi069 ceph-mon[36441]: Adjusting osd_memory_target on smithi069 to 7968M 2024-03-22T07:11:49.336 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:49.336 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:49.336 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:49.336 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:49.336 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:49 smithi069 ceph-mon[36441]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:11:49.337 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:49.337 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:49.337 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:49.337 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:49.337 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:11:49.337 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:11:49.337 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:49 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:49.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:49 smithi063 ceph-mon[29200]: Detected new or changed devices on smithi069 2024-03-22T07:11:49.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:49.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:49.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:11:49.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:11:49.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:49 smithi063 ceph-mon[29200]: Adjusting osd_memory_target on smithi069 to 7968M 2024-03-22T07:11:49.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:49.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:49.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:49.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:49.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:49 smithi063 ceph-mon[29200]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:11:49.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:11:49.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:49.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:11:49.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:11:49.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-22T07:11:49.375 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-22T07:11:49.375 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:49 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:50.259 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:50 smithi069 ceph-mon[36441]: from='client.24179 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:11:50.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:50 smithi063 ceph-mon[29200]: from='client.24179 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:11:51.397 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:51 smithi069 ceph-mon[36441]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:11:51.397 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:51 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/480092945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "566db34e-ce0e-436f-aa90-df2c80d812ae"}]: dispatch 2024-03-22T07:11:51.397 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:51 smithi069 ceph-mon[36441]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "566db34e-ce0e-436f-aa90-df2c80d812ae"}]: dispatch 2024-03-22T07:11:51.397 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:51 smithi069 ceph-mon[36441]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "566db34e-ce0e-436f-aa90-df2c80d812ae"}]': finished 2024-03-22T07:11:51.397 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:51 smithi069 ceph-mon[36441]: osdmap e37: 6 total, 5 up, 6 in 2024-03-22T07:11:51.397 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:11:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:51 smithi063 ceph-mon[29200]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:11:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:51 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/480092945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "566db34e-ce0e-436f-aa90-df2c80d812ae"}]: dispatch 2024-03-22T07:11:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:51 smithi063 ceph-mon[29200]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "566db34e-ce0e-436f-aa90-df2c80d812ae"}]: dispatch 2024-03-22T07:11:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:51 smithi063 ceph-mon[29200]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "566db34e-ce0e-436f-aa90-df2c80d812ae"}]': finished 2024-03-22T07:11:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:51 smithi063 ceph-mon[29200]: osdmap e37: 6 total, 5 up, 6 in 2024-03-22T07:11:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:11:52.477 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:52 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/1567801064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:11:52.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:52 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/1567801064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-22T07:11:53.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:53 smithi069 ceph-mon[36441]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:11:53.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:53 smithi063 ceph-mon[29200]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:11:55.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:55 smithi069 ceph-mon[36441]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:11:55.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:55 smithi063 ceph-mon[29200]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:11:57.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:57 smithi069 ceph-mon[36441]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:11:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:57 smithi063 ceph-mon[29200]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:11:59.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:59 smithi069 ceph-mon[36441]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:11:59.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:59 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T07:11:59.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:59 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:59.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:59 smithi069 ceph-mon[36441]: Deploying daemon osd.5 on smithi069 2024-03-22T07:11:59.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:11:59 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:11:59.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:59 smithi063 ceph-mon[29200]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:11:59.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:59 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T07:11:59.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:59 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:11:59.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:59 smithi063 ceph-mon[29200]: Deploying daemon osd.5 on smithi069 2024-03-22T07:11:59.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:11:59 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:12:01.452 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:01 smithi069 ceph-mon[36441]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:12:01.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:01 smithi063 ceph-mon[29200]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:12:03.494 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:03 smithi069 ceph-mon[36441]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:12:03.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:03 smithi063 ceph-mon[29200]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:12:04.649 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:04 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:04.650 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:04 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:04.650 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:04 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:12:04.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:04 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:04.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:04 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:04.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:04 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:12:05.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:05 smithi063 ceph-mon[29200]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:12:05.899 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:05 smithi069 ceph-mon[36441]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:12:06.634 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 5 on host 'smithi069' 2024-03-22T07:12:06.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:06.634+0000 7fe4c0ff9700 1 -- 172.21.15.69:0/2339616191 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe4c410f9f0 con 0x7fe4ac0692b0 2024-03-22T07:12:06.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:06.636+0000 7fe4ca4a9700 1 -- 172.21.15.69:0/2339616191 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe4ac0692b0 msgr2=0x7fe4ac06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:06.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:06.636+0000 7fe4ca4a9700 1 --2- 172.21.15.69:0/2339616191 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe4ac0692b0 0x7fe4ac06b760 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009fd0 tx=0x7fe4b8009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:06.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:06.636+0000 7fe4ca4a9700 1 -- 172.21.15.69:0/2339616191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4c40fb1e0 msgr2=0x7fe4c410da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:06.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:06.636+0000 7fe4ca4a9700 1 --2- 172.21.15.69:0/2339616191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4c40fb1e0 0x7fe4c410da50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400b790 tx=0x7fe4b4009220 comp rx=0 tx=0).stop 2024-03-22T07:12:06.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:06.636+0000 7fe4ca4a9700 1 -- 172.21.15.69:0/2339616191 shutdown_connections 2024-03-22T07:12:06.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:06.636+0000 7fe4ca4a9700 1 --2- 172.21.15.69:0/2339616191 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe4ac0692b0 0x7fe4ac06b760 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:06.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:06.636+0000 7fe4ca4a9700 1 --2- 172.21.15.69:0/2339616191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4c40fb1e0 0x7fe4c410da50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:06.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:06.636+0000 7fe4ca4a9700 1 --2- 172.21.15.69:0/2339616191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4c40fbb50 0x7fe4c410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:06.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:06.636+0000 7fe4ca4a9700 1 -- 172.21.15.69:0/2339616191 >> 172.21.15.69:0/2339616191 conn(0x7fe4c40f6ba0 msgr2=0x7fe4c4100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:06.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:06.637+0000 7fe4ca4a9700 1 -- 172.21.15.69:0/2339616191 shutdown_connections 2024-03-22T07:12:06.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:06.637+0000 7fe4ca4a9700 1 -- 172.21.15.69:0/2339616191 wait complete. 2024-03-22T07:12:07.320 DEBUG:teuthology.orchestra.run.smithi069:osd.5> sudo journalctl -f -n 0 -u ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.5.service 2024-03-22T07:12:07.322 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-03-22T07:12:07.323 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-22T07:12:07.550 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:12:07 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[55195]: 2024-03-22T07:12:07.457+0000 7f50df2bf540 -1 osd.5 0 log_to_monitors true 2024-03-22T07:12:07.593 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:07.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:07 smithi063 ceph-mon[29200]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:12:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:07 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:07 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:07 smithi063 ceph-mon[29200]: from='osd.5 [v2:172.21.15.69:6816/2157641649,v1:172.21.15.69:6817/2157641649]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T07:12:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:07 smithi063 ceph-mon[29200]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T07:12:07.810 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:07 smithi069 ceph-mon[36441]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:12:07.810 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:07 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:07.810 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:07 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:07.810 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:07 smithi069 ceph-mon[36441]: from='osd.5 [v2:172.21.15.69:6816/2157641649,v1:172.21.15.69:6817/2157641649]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T07:12:07.810 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:07 smithi069 ceph-mon[36441]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T07:12:08.779 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:12:08 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[55195]: 2024-03-22T07:12:08.637+0000 7f50d5a91700 -1 osd.5 0 waiting for initial osdmap 2024-03-22T07:12:08.779 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:12:08 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[55195]: 2024-03-22T07:12:08.645+0000 7f50cf88a700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:12:08.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:08 smithi069 ceph-mon[36441]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T07:12:08.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:08 smithi069 ceph-mon[36441]: osdmap e38: 6 total, 5 up, 6 in 2024-03-22T07:12:08.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:12:08.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:08 smithi069 ceph-mon[36441]: from='osd.5 [v2:172.21.15.69:6816/2157641649,v1:172.21.15.69:6817/2157641649]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:12:08.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:08 smithi069 ceph-mon[36441]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:12:08.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:08 smithi069 ceph-mon[36441]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:12:08.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:08.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:08 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:08.909 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:08 smithi063 ceph-mon[29200]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T07:12:08.910 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:08 smithi063 ceph-mon[29200]: osdmap e38: 6 total, 5 up, 6 in 2024-03-22T07:12:08.910 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:08 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:12:08.910 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:08 smithi063 ceph-mon[29200]: from='osd.5 [v2:172.21.15.69:6816/2157641649,v1:172.21.15.69:6817/2157641649]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:12:08.910 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:08 smithi063 ceph-mon[29200]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:12:08.910 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:08 smithi063 ceph-mon[29200]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:12:08.910 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:08 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:08.910 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:08 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:09.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.212+0000 7fc0df1b9700 1 -- 172.21.15.63:0/851235282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0d80fcf70 msgr2=0x7fc0d80fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:09.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.212+0000 7fc0df1b9700 1 --2- 172.21.15.63:0/851235282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0d80fcf70 0x7fc0d80fd3e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4004940 tx=0x7fc0c4009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:09.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.212+0000 7fc0df1b9700 1 -- 172.21.15.63:0/851235282 shutdown_connections 2024-03-22T07:12:09.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.212+0000 7fc0df1b9700 1 --2- 172.21.15.63:0/851235282 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0d80fcf70 0x7fc0d80fd3e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:09.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.212+0000 7fc0df1b9700 1 --2- 172.21.15.63:0/851235282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d80fbe00 0x7fc0d80fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:09.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.212+0000 7fc0df1b9700 1 -- 172.21.15.63:0/851235282 >> 172.21.15.63:0/851235282 conn(0x7fc0d80f74e0 msgr2=0x7fc0d80f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:09.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.212+0000 7fc0df1b9700 1 -- 172.21.15.63:0/851235282 shutdown_connections 2024-03-22T07:12:09.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.212+0000 7fc0df1b9700 1 -- 172.21.15.63:0/851235282 wait complete. 2024-03-22T07:12:09.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.213+0000 7fc0df1b9700 1 Processor -- start 2024-03-22T07:12:09.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.213+0000 7fc0df1b9700 1 -- start start 2024-03-22T07:12:09.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.213+0000 7fc0df1b9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d80fbe00 0x7fc0d80ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:09.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.213+0000 7fc0df1b9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0d80fcf70 0x7fc0d80fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:09.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.213+0000 7fc0df1b9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d80fe860 con 0x7fc0d80fcf70 2024-03-22T07:12:09.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.213+0000 7fc0df1b9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d80fe9d0 con 0x7fc0d80fbe00 2024-03-22T07:12:09.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.213+0000 7fc0d7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0d80fcf70 0x7fc0d80fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:09.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.213+0000 7fc0d7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0d80fcf70 0x7fc0d80fe2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38124/0 (socket says 172.21.15.63:38124) 2024-03-22T07:12:09.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.213+0000 7fc0d7fff700 1 -- 172.21.15.63:0/2165048617 learned_addr learned my addr 172.21.15.63:0/2165048617 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:09.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.213+0000 7fc0dcf55700 1 --2- 172.21.15.63:0/2165048617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d80fbe00 0x7fc0d80ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:09.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.213+0000 7fc0d7fff700 1 -- 172.21.15.63:0/2165048617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d80fbe00 msgr2=0x7fc0d80ffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:09.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.213+0000 7fc0d7fff700 1 --2- 172.21.15.63:0/2165048617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d80fbe00 0x7fc0d80ffde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:09.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.213+0000 7fc0d7fff700 1 -- 172.21.15.63:0/2165048617 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c4005040 con 0x7fc0d80fcf70 2024-03-22T07:12:09.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.214+0000 7fc0d7fff700 1 --2- 172.21.15.63:0/2165048617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0d80fcf70 0x7fc0d80fe2b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009150 tx=0x7fc0c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:09.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.214+0000 7fc0d57fa700 1 -- 172.21.15.63:0/2165048617 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0c400e040 con 0x7fc0d80fcf70 2024-03-22T07:12:09.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.214+0000 7fc0df1b9700 1 -- 172.21.15.63:0/2165048617 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d80fec50 con 0x7fc0d80fcf70 2024-03-22T07:12:09.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.214+0000 7fc0d57fa700 1 -- 172.21.15.63:0/2165048617 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0c4004290 con 0x7fc0d80fcf70 2024-03-22T07:12:09.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.214+0000 7fc0d57fa700 1 -- 172.21.15.63:0/2165048617 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0c401c630 con 0x7fc0d80fcf70 2024-03-22T07:12:09.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.214+0000 7fc0df1b9700 1 -- 172.21.15.63:0/2165048617 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d8115a50 con 0x7fc0d80fcf70 2024-03-22T07:12:09.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.215+0000 7fc0df1b9700 1 -- 172.21.15.63:0/2165048617 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d8101b00 con 0x7fc0d80fcf70 2024-03-22T07:12:09.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.215+0000 7fc0d57fa700 1 -- 172.21.15.63:0/2165048617 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc0c4026440 con 0x7fc0d80fcf70 2024-03-22T07:12:09.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.216+0000 7fc0d57fa700 1 --2- 172.21.15.63:0/2165048617 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fc0c80692a0 0x7fc0c806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:09.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.216+0000 7fc0d57fa700 1 -- 172.21.15.63:0/2165048617 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4354+0+0 (secure 0 0 0) 0x7fc0c4087a90 con 0x7fc0d80fcf70 2024-03-22T07:12:09.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.216+0000 7fc0dcf55700 1 --2- 172.21.15.63:0/2165048617 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fc0c80692a0 0x7fc0c806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:09.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.216+0000 7fc0dcf55700 1 --2- 172.21.15.63:0/2165048617 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fc0c80692a0 0x7fc0c806b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc0d80ffb70 tx=0x7fc0cc009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:09.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.220+0000 7fc0d57fa700 1 -- 172.21.15.63:0/2165048617 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc0c4054280 con 0x7fc0d80fcf70 2024-03-22T07:12:09.417 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:12:09.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.416+0000 7fc0df1b9700 1 -- 172.21.15.63:0/2165048617 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fc0d804bed0 con 0x7fc0d80fcf70 2024-03-22T07:12:09.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.416+0000 7fc0d57fa700 1 -- 172.21.15.63:0/2165048617 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7fc0c405be60 con 0x7fc0d80fcf70 2024-03-22T07:12:09.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.418+0000 7fc0df1b9700 1 -- 172.21.15.63:0/2165048617 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fc0c80692a0 msgr2=0x7fc0c806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:09.419 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.418+0000 7fc0df1b9700 1 --2- 172.21.15.63:0/2165048617 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fc0c80692a0 0x7fc0c806b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc0d80ffb70 tx=0x7fc0cc009040 comp rx=0 tx=0).stop 2024-03-22T07:12:09.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.418+0000 7fc0df1b9700 1 -- 172.21.15.63:0/2165048617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0d80fcf70 msgr2=0x7fc0d80fe2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:09.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.418+0000 7fc0df1b9700 1 --2- 172.21.15.63:0/2165048617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0d80fcf70 0x7fc0d80fe2b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009150 tx=0x7fc0c4009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:09.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.418+0000 7fc0df1b9700 1 -- 172.21.15.63:0/2165048617 shutdown_connections 2024-03-22T07:12:09.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.418+0000 7fc0df1b9700 1 --2- 172.21.15.63:0/2165048617 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fc0c80692a0 0x7fc0c806b750 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:09.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.419+0000 7fc0df1b9700 1 --2- 172.21.15.63:0/2165048617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d80fbe00 0x7fc0d80ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:09.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.419+0000 7fc0df1b9700 1 --2- 172.21.15.63:0/2165048617 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc0d80fcf70 0x7fc0d80fe2b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:09.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.419+0000 7fc0df1b9700 1 -- 172.21.15.63:0/2165048617 >> 172.21.15.63:0/2165048617 conn(0x7fc0d80f74e0 msgr2=0x7fc0d8066430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:09.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.419+0000 7fc0df1b9700 1 -- 172.21.15.63:0/2165048617 shutdown_connections 2024-03-22T07:12:09.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:09.419+0000 7fc0df1b9700 1 -- 172.21.15.63:0/2165048617 wait complete. 2024-03-22T07:12:09.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:09 smithi069 ceph-mon[36441]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-03-22T07:12:09.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:09 smithi069 ceph-mon[36441]: osdmap e39: 6 total, 5 up, 6 in 2024-03-22T07:12:09.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:09 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:12:09.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:09 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:12:09.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:09 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/2165048617' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T07:12:10.012 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:09 smithi063 ceph-mon[29200]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-03-22T07:12:10.012 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:09 smithi063 ceph-mon[29200]: osdmap e39: 6 total, 5 up, 6 in 2024-03-22T07:12:10.012 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:09 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:12:10.012 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:09 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:12:10.012 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:09 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2165048617' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T07:12:10.044 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":39,"num_osds":6,"num_up_osds":5,"osd_up_since":1711091504,"num_in_osds":6,"osd_in_since":1711091511,"num_remapped_pgs":0} 2024-03-22T07:12:10.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:10 smithi069 ceph-mon[36441]: purged_snaps scrub starts 2024-03-22T07:12:10.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:10 smithi069 ceph-mon[36441]: purged_snaps scrub ok 2024-03-22T07:12:10.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:10 smithi069 ceph-mon[36441]: osd.5 [v2:172.21.15.69:6816/2157641649,v1:172.21.15.69:6817/2157641649] boot 2024-03-22T07:12:10.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:10 smithi069 ceph-mon[36441]: osdmap e40: 6 total, 6 up, 6 in 2024-03-22T07:12:10.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:10 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:12:10.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:10 smithi069 ceph-mon[36441]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:12:11.045 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-22T07:12:11.076 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:10 smithi063 ceph-mon[29200]: purged_snaps scrub starts 2024-03-22T07:12:11.076 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:10 smithi063 ceph-mon[29200]: purged_snaps scrub ok 2024-03-22T07:12:11.076 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:10 smithi063 ceph-mon[29200]: osd.5 [v2:172.21.15.69:6816/2157641649,v1:172.21.15.69:6817/2157641649] boot 2024-03-22T07:12:11.076 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:10 smithi063 ceph-mon[29200]: osdmap e40: 6 total, 6 up, 6 in 2024-03-22T07:12:11.076 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:10 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:12:11.076 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:10 smithi063 ceph-mon[29200]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-03-22T07:12:11.318 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:11.761 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:11 smithi063 ceph-mon[29200]: osdmap e41: 6 total, 6 up, 6 in 2024-03-22T07:12:11.922 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:11 smithi069 ceph-mon[36441]: osdmap e41: 6 total, 6 up, 6 in 2024-03-22T07:12:12.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.975+0000 7f54ef0e8700 1 -- 172.21.15.63:0/163661651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54e80fbce0 msgr2=0x7f54e80fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:12.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.975+0000 7f54ef0e8700 1 --2- 172.21.15.63:0/163661651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54e80fbce0 0x7f54e80fc0f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f54dc004940 tx=0x7f54dc009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:12.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.975+0000 7f54ef0e8700 1 -- 172.21.15.63:0/163661651 shutdown_connections 2024-03-22T07:12:12.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.975+0000 7f54ef0e8700 1 --2- 172.21.15.63:0/163661651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54e80fcf90 0x7f54e80fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:12.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.975+0000 7f54ef0e8700 1 --2- 172.21.15.63:0/163661651 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54e80fbce0 0x7f54e80fc0f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:12.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.975+0000 7f54ef0e8700 1 -- 172.21.15.63:0/163661651 >> 172.21.15.63:0/163661651 conn(0x7f54e80f7440 msgr2=0x7f54e80f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:12.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.976+0000 7f54ef0e8700 1 -- 172.21.15.63:0/163661651 shutdown_connections 2024-03-22T07:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.976+0000 7f54ef0e8700 1 -- 172.21.15.63:0/163661651 wait complete. 2024-03-22T07:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.976+0000 7f54ef0e8700 1 Processor -- start 2024-03-22T07:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.976+0000 7f54ef0e8700 1 -- start start 2024-03-22T07:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.976+0000 7f54ef0e8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54e80fbce0 0x7f54e810c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.976+0000 7f54ef0e8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54e80fcf90 0x7f54e810cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.976+0000 7f54ef0e8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e810d380 con 0x7f54e80fbce0 2024-03-22T07:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.976+0000 7f54ef0e8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e810d4f0 con 0x7f54e80fcf90 2024-03-22T07:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.977+0000 7f54ece84700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54e80fbce0 0x7f54e810c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.977+0000 7f54ece84700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54e80fbce0 0x7f54e810c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:38146/0 (socket says 172.21.15.63:38146) 2024-03-22T07:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.977+0000 7f54ece84700 1 -- 172.21.15.63:0/3772698145 learned_addr learned my addr 172.21.15.63:0/3772698145 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.977+0000 7f54e7fff700 1 --2- 172.21.15.63:0/3772698145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54e80fcf90 0x7f54e810cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.977+0000 7f54ece84700 1 -- 172.21.15.63:0/3772698145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54e80fcf90 msgr2=0x7f54e810cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.977+0000 7f54ece84700 1 --2- 172.21.15.63:0/3772698145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54e80fcf90 0x7f54e810cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.977+0000 7f54ece84700 1 -- 172.21.15.63:0/3772698145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc005040 con 0x7f54e80fbce0 2024-03-22T07:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.977+0000 7f54e7fff700 1 --2- 172.21.15.63:0/3772698145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54e80fcf90 0x7f54e810cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:12:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.977+0000 7f54ece84700 1 --2- 172.21.15.63:0/3772698145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54e80fbce0 0x7f54e810c840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009150 tx=0x7f54dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:12.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.977+0000 7f54e57fa700 1 -- 172.21.15.63:0/3772698145 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54dc00e040 con 0x7f54e80fbce0 2024-03-22T07:12:12.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.977+0000 7f54e57fa700 1 -- 172.21.15.63:0/3772698145 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54dc00bbf0 con 0x7f54e80fbce0 2024-03-22T07:12:12.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.977+0000 7f54ef0e8700 1 -- 172.21.15.63:0/3772698145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e8111ef0 con 0x7f54e80fbce0 2024-03-22T07:12:12.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.977+0000 7f54e57fa700 1 -- 172.21.15.63:0/3772698145 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54dc01c630 con 0x7f54e80fbce0 2024-03-22T07:12:12.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.977+0000 7f54ef0e8700 1 -- 172.21.15.63:0/3772698145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e8112420 con 0x7f54e80fbce0 2024-03-22T07:12:12.981 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.978+0000 7f54ef0e8700 1 -- 172.21.15.63:0/3772698145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e804c710 con 0x7f54e80fbce0 2024-03-22T07:12:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.979+0000 7f54e57fa700 1 -- 172.21.15.63:0/3772698145 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f54dc00bd60 con 0x7f54e80fbce0 2024-03-22T07:12:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.979+0000 7f54e57fa700 1 --2- 172.21.15.63:0/3772698145 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f54d80692a0 0x7f54d806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.979+0000 7f54e57fa700 1 -- 172.21.15.63:0/3772698145 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f54dc088210 con 0x7f54e80fbce0 2024-03-22T07:12:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.979+0000 7f54e7fff700 1 --2- 172.21.15.63:0/3772698145 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f54d80692a0 0x7f54d806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:12.982 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.980+0000 7f54e7fff700 1 --2- 172.21.15.63:0/3772698145 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f54d80692a0 0x7f54d806b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f54d40041d0 tx=0x7f54d4009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:12.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:12.984+0000 7f54e57fa700 1 -- 172.21.15.63:0/3772698145 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f54dc058310 con 0x7f54e80fbce0 2024-03-22T07:12:13.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:12 smithi063 ceph-mon[29200]: Detected new or changed devices on smithi069 2024-03-22T07:12:13.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:12 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:13.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:12 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:13.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:12 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:12:13.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:12 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:12:13.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:12 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:12:13.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:12 smithi063 ceph-mon[29200]: Adjusting osd_memory_target on smithi069 to 5312M 2024-03-22T07:12:13.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:12 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:13.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:12 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:12:13.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:12 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:12:13.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:12 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:13.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:12 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:12:13.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:12 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:12:13.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:12 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:12:13.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:12 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:13.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:12 smithi063 ceph-mon[29200]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:13.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:13.181+0000 7f54ef0e8700 1 -- 172.21.15.63:0/3772698145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f54e81047d0 con 0x7f54e80fbce0 2024-03-22T07:12:13.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:13.182+0000 7f54e57fa700 1 -- 172.21.15.63:0/3772698145 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7f54dc057ea0 con 0x7f54e80fbce0 2024-03-22T07:12:13.183 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:12:13.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:13.183+0000 7f54ef0e8700 1 -- 172.21.15.63:0/3772698145 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f54d80692a0 msgr2=0x7f54d806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:13.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:13.183+0000 7f54ef0e8700 1 --2- 172.21.15.63:0/3772698145 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f54d80692a0 0x7f54d806b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f54d40041d0 tx=0x7f54d4009040 comp rx=0 tx=0).stop 2024-03-22T07:12:13.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:13.183+0000 7f54ef0e8700 1 -- 172.21.15.63:0/3772698145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54e80fbce0 msgr2=0x7f54e810c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:13.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:13.183+0000 7f54ef0e8700 1 --2- 172.21.15.63:0/3772698145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54e80fbce0 0x7f54e810c840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009150 tx=0x7f54dc009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:13.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:13.184+0000 7f54ef0e8700 1 -- 172.21.15.63:0/3772698145 shutdown_connections 2024-03-22T07:12:13.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:13.184+0000 7f54ef0e8700 1 --2- 172.21.15.63:0/3772698145 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f54d80692a0 0x7f54d806b750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:13.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:13.184+0000 7f54ef0e8700 1 --2- 172.21.15.63:0/3772698145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f54e80fbce0 0x7f54e810c840 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:13.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:13.184+0000 7f54ef0e8700 1 --2- 172.21.15.63:0/3772698145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54e80fcf90 0x7f54e810cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:13.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:13.184+0000 7f54ef0e8700 1 -- 172.21.15.63:0/3772698145 >> 172.21.15.63:0/3772698145 conn(0x7f54e80f7440 msgr2=0x7f54e80f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:13.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:13.184+0000 7f54ef0e8700 1 -- 172.21.15.63:0/3772698145 shutdown_connections 2024-03-22T07:12:13.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:13.184+0000 7f54ef0e8700 1 -- 172.21.15.63:0/3772698145 wait complete. 2024-03-22T07:12:13.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:12 smithi069 ceph-mon[36441]: Detected new or changed devices on smithi069 2024-03-22T07:12:13.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:12 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:13.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:12 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:13.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:12 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:12:13.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:12 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:12:13.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:12 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:12:13.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:12 smithi069 ceph-mon[36441]: Adjusting osd_memory_target on smithi069 to 5312M 2024-03-22T07:12:13.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:12 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:13.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:12 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:12:13.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:12 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:12:13.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:12 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:13.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:12 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:12:13.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:12 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:12:13.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:12 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:12:13.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:12 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:12:13.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:12 smithi069 ceph-mon[36441]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:13.913 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1711091529,"num_in_osds":6,"osd_in_since":1711091511,"num_remapped_pgs":0} 2024-03-22T07:12:13.913 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-22T07:12:14.212 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:14.464 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:14 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3772698145' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T07:12:14.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:14 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/3772698145' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-22T07:12:15.466 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:15 smithi063 ceph-mon[29200]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:15.466 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:15 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:12:15.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:15 smithi069 ceph-mon[36441]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:15.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:15 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:12:15.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.813+0000 7fba92419700 1 -- 172.21.15.63:0/1704666705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8c1026c0 msgr2=0x7fba8c0fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:15.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.813+0000 7fba92419700 1 --2- 172.21.15.63:0/1704666705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8c1026c0 0x7fba8c0fbd30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fba80004940 tx=0x7fba80009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:15.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.813+0000 7fba92419700 1 -- 172.21.15.63:0/1704666705 shutdown_connections 2024-03-22T07:12:15.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.813+0000 7fba92419700 1 --2- 172.21.15.63:0/1704666705 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8c1026c0 0x7fba8c0fbd30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:15.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.813+0000 7fba92419700 1 --2- 172.21.15.63:0/1704666705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba8c101d20 0x7fba8c1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:15.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.813+0000 7fba92419700 1 -- 172.21.15.63:0/1704666705 >> 172.21.15.63:0/1704666705 conn(0x7fba8c0f7870 msgr2=0x7fba8c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:15.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.814+0000 7fba92419700 1 -- 172.21.15.63:0/1704666705 shutdown_connections 2024-03-22T07:12:15.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.814+0000 7fba92419700 1 -- 172.21.15.63:0/1704666705 wait complete. 2024-03-22T07:12:15.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.814+0000 7fba92419700 1 Processor -- start 2024-03-22T07:12:15.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.814+0000 7fba92419700 1 -- start start 2024-03-22T07:12:15.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.814+0000 7fba92419700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8c101d20 0x7fba8c1128b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:15.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.814+0000 7fba92419700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba8c1026c0 0x7fba8c112df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:15.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.814+0000 7fba92419700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c1134d0 con 0x7fba8c101d20 2024-03-22T07:12:15.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.814+0000 7fba92419700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c10ca40 con 0x7fba8c1026c0 2024-03-22T07:12:15.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.815+0000 7fba8bfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8c101d20 0x7fba8c1128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:15.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.815+0000 7fba8bfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8c101d20 0x7fba8c1128b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49148/0 (socket says 172.21.15.63:49148) 2024-03-22T07:12:15.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.815+0000 7fba8bfff700 1 -- 172.21.15.63:0/1288178145 learned_addr learned my addr 172.21.15.63:0/1288178145 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:15.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.815+0000 7fba8b7fe700 1 --2- 172.21.15.63:0/1288178145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba8c1026c0 0x7fba8c112df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:15.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.815+0000 7fba8bfff700 1 -- 172.21.15.63:0/1288178145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba8c1026c0 msgr2=0x7fba8c112df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:15.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.815+0000 7fba8bfff700 1 --2- 172.21.15.63:0/1288178145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba8c1026c0 0x7fba8c112df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:15.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.815+0000 7fba8bfff700 1 -- 172.21.15.63:0/1288178145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba80005040 con 0x7fba8c101d20 2024-03-22T07:12:15.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.815+0000 7fba8bfff700 1 --2- 172.21.15.63:0/1288178145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8c101d20 0x7fba8c1128b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00b790 tx=0x7fba7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:15.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.815+0000 7fba88ff9700 1 -- 172.21.15.63:0/1288178145 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba7c00f770 con 0x7fba8c101d20 2024-03-22T07:12:15.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.815+0000 7fba92419700 1 -- 172.21.15.63:0/1288178145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba8c10cd20 con 0x7fba8c101d20 2024-03-22T07:12:15.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.815+0000 7fba88ff9700 1 -- 172.21.15.63:0/1288178145 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba7c00fda0 con 0x7fba8c101d20 2024-03-22T07:12:15.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.815+0000 7fba88ff9700 1 -- 172.21.15.63:0/1288178145 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba7c00d310 con 0x7fba8c101d20 2024-03-22T07:12:15.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.815+0000 7fba92419700 1 -- 172.21.15.63:0/1288178145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba8c10d2e0 con 0x7fba8c101d20 2024-03-22T07:12:15.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.816+0000 7fba92419700 1 -- 172.21.15.63:0/1288178145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8c0fd470 con 0x7fba8c101d20 2024-03-22T07:12:15.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.817+0000 7fba88ff9700 1 -- 172.21.15.63:0/1288178145 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fba7c010420 con 0x7fba8c101d20 2024-03-22T07:12:15.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.817+0000 7fba88ff9700 1 --2- 172.21.15.63:0/1288178145 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fba740692a0 0x7fba7406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:15.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.817+0000 7fba88ff9700 1 -- 172.21.15.63:0/1288178145 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fba7c086e60 con 0x7fba8c101d20 2024-03-22T07:12:15.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.817+0000 7fba8b7fe700 1 --2- 172.21.15.63:0/1288178145 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fba740692a0 0x7fba7406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:15.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.818+0000 7fba8b7fe700 1 --2- 172.21.15.63:0/1288178145 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fba740692a0 0x7fba7406b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fba8c10dc00 tx=0x7fba80009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:15.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:15.821+0000 7fba88ff9700 1 -- 172.21.15.63:0/1288178145 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fba7c056f60 con 0x7fba8c101d20 2024-03-22T07:12:16.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:16.017+0000 7fba92419700 1 -- 172.21.15.63:0/1288178145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba8c108e70 con 0x7fba8c101d20 2024-03-22T07:12:16.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:16.017+0000 7fba88ff9700 1 -- 172.21.15.63:0/1288178145 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11043 (secure 0 0 0) 0x7fba7c056af0 con 0x7fba8c101d20 2024-03-22T07:12:16.019 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:12:16.019 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":41,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","created":"2024-03-22T07:06:38.004887+0000","modified":"2024-03-22T07:12:10.641337+0000","last_up_change":"2024-03-22T07:12:09.636009+0000","last_in_change":"2024-03-22T07:11:51.027495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T07:10:56.524122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"33ade431-7f97-43c7-bf9d-962c12d3ba7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3791141232},{"type":"v1","addr":"172.21.15.63:6803","nonce":3791141232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3791141232},{"type":"v1","addr":"172.21.15.63:6805","nonce":3791141232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":3791141232},{"type":"v1","addr":"172.21.15.63:6809","nonce":3791141232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3791141232},{"type":"v1","addr":"172.21.15.63:6807","nonce":3791141232}]},"public_addr":"172.21.15.63:6803/3791141232","cluster_addr":"172.21.15.63:6805/3791141232","heartbeat_back_addr":"172.21.15.63:6809/3791141232","heartbeat_front_addr":"172.21.15.63:6807/3791141232","state":["exists","up"]},{"osd":1,"uuid":"0643ddb6-db9d-464c-8236-3aea2893de83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1043983969},{"type":"v1","addr":"172.21.15.63:6811","nonce":1043983969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1043983969},{"type":"v1","addr":"172.21.15.63:6813","nonce":1043983969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1043983969},{"type":"v1","addr":"172.21.15.63:6817","nonce":1043983969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1043983969},{"type":"v1","addr":"172.21.15.63:6815","nonce":1043983969}]},"public_addr":"172.21.15.63:6811/1043983969","cluster_addr":"172.21.15.63:6813/1043983969","heartbeat_back_addr":"172.21.15.63:6817/1043983969","heartbeat_front_addr":"172.21.15.63:6815/1043983969","state":["exists","up"]},{"osd":2,"uuid":"140b09d0-1fb6-4a3b-873a-048b797c0180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3603088351},{"type":"v1","addr":"172.21.15.63:6819","nonce":3603088351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3603088351},{"type":"v1","addr":"172.21.15.63:6821","nonce":3603088351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3603088351},{"type":"v1","addr":"172.21.15.63:6825","nonce":3603088351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3603088351},{"type":"v1","addr":"172.21.15.63:6823","nonce":3603088351}]},"public_addr":"172.21.15.63:6819/3603088351","cluster_addr":"172.21.15.63:6821/3603088351","heartbeat_back_addr":"172.21.15.63:6825/3603088351","heartbeat_front_addr":"172.21.15.63:6823/3603088351","state":["exists","up"]},{"osd":3,"uuid":"1d92b77e-6bcc-4585-96f1-72251303cc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1811566075},{"type":"v1","addr":"172.21.15.69:6801","nonce":1811566075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1811566075},{"type":"v1","addr":"172.21.15.69:6803","nonce":1811566075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1811566075},{"type":"v1","addr":"172.21.15.69:6807","nonce":1811566075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1811566075},{"type":"v1","addr":"172.21.15.69:6805","nonce":1811566075}]},"public_addr":"172.21.15.69:6801/1811566075","cluster_addr":"172.21.15.69:6803/1811566075","heartbeat_back_addr":"172.21.15.69:6807/1811566075","heartbeat_front_addr":"172.21.15.69:6805/1811566075","state":["exists","up"]},{"osd":4,"uuid":"28bdc8bc-41d5-4c01-9b29-d5833223d1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":656541591},{"type":"v1","addr":"172.21.15.69:6809","nonce":656541591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":656541591},{"type":"v1","addr":"172.21.15.69:6811","nonce":656541591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":656541591},{"type":"v1","addr":"172.21.15.69:6815","nonce":656541591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":656541591},{"type":"v1","addr":"172.21.15.69:6813","nonce":656541591}]},"public_addr":"172.21.15.69:6809/656541591","cluster_addr":"172.21.15.69:6811/656541591","heartbeat_back_addr":"172.21.15.69:6815/656541591","heartbeat_front_addr":"172.21.15.69:6813/656541591","state":["exists","up"]},{"osd":5,"uuid":"566db34e-ce0e-436f-aa90-df2c80d812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2157641649},{"type":"v1","addr":"172.21.15.69:6817","nonce":2157641649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2157641649},{"type":"v1","addr":"172.21.15.69:6819","nonce":2157641649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2157641649},{"type":"v1","addr":"172.21.15.69:6823","nonce":2157641649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2157641649},{"type":"v1","addr":"172.21.15.69:6821","nonce":2157641649}]},"public_addr":"172.21.15.69:6817/2157641649","cluster_addr":"172.21.15.69:6819/2157641649","heartbeat_back_addr":"172.21.15.69:6823/2157641649","heartbeat_front_addr":"172.21.15.69:6821/2157641649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:09:56.132204+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:10:24.051511+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:10:52.969047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:11:17.576349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:11:42.883846+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:12:08.488698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6801/2296442540":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/47408280":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/466771007":"2024-03-23T07:08:44.186801+0000","172.21.15.63:6801/480818549":"2024-03-23T07:07:41.261927+0000","172.21.15.63:0/1323074353":"2024-03-23T07:07:41.261927+0000","172.21.15.63:0/1818534437":"2024-03-23T07:07:41.261927+0000","172.21.15.63:6801/2616455474":"2024-03-23T07:07:10.417547+0000","172.21.15.63:6800/2616455474":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/1649622442":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/3927392742":"2024-03-23T07:08:44.186801+0000","172.21.15.63:6800/2296442540":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/424866385":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/2739978443":"2024-03-23T07:07:10.417547+0000","172.21.15.63:6800/480818549":"2024-03-23T07:07:41.261927+0000","172.21.15.63:0/2503315479":"2024-03-23T07:07:41.261927+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T07:12:16.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:16.019+0000 7fba92419700 1 -- 172.21.15.63:0/1288178145 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fba740692a0 msgr2=0x7fba7406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:16.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:16.019+0000 7fba92419700 1 --2- 172.21.15.63:0/1288178145 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fba740692a0 0x7fba7406b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fba8c10dc00 tx=0x7fba80009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:16.020 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:16.019+0000 7fba92419700 1 -- 172.21.15.63:0/1288178145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8c101d20 msgr2=0x7fba8c1128b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:16.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:16.019+0000 7fba92419700 1 --2- 172.21.15.63:0/1288178145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8c101d20 0x7fba8c1128b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00b790 tx=0x7fba7c009250 comp rx=0 tx=0).stop 2024-03-22T07:12:16.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:16.019+0000 7fba92419700 1 -- 172.21.15.63:0/1288178145 shutdown_connections 2024-03-22T07:12:16.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:16.019+0000 7fba92419700 1 --2- 172.21.15.63:0/1288178145 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fba740692a0 0x7fba7406b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:16.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:16.019+0000 7fba92419700 1 --2- 172.21.15.63:0/1288178145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fba8c101d20 0x7fba8c1128b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:16.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:16.019+0000 7fba92419700 1 --2- 172.21.15.63:0/1288178145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba8c1026c0 0x7fba8c112df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:16.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:16.019+0000 7fba92419700 1 -- 172.21.15.63:0/1288178145 >> 172.21.15.63:0/1288178145 conn(0x7fba8c0f7870 msgr2=0x7fba8c0f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:16.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:16.020+0000 7fba92419700 1 -- 172.21.15.63:0/1288178145 shutdown_connections 2024-03-22T07:12:16.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:16.020+0000 7fba92419700 1 -- 172.21.15.63:0/1288178145 wait complete. 2024-03-22T07:12:16.350 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:16 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/1288178145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:12:16.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:16 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/1288178145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:12:16.627 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T07:10:56.524122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-03-22T07:12:16.628 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-22T07:12:16.893 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:17.331 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:17 smithi063 ceph-mon[29200]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:17.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:17 smithi069 ceph-mon[36441]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:18.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.422+0000 7f0af989a700 1 -- 172.21.15.63:0/1261583134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0af41026c0 msgr2=0x7f0af40fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:18.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.422+0000 7f0af989a700 1 --2- 172.21.15.63:0/1261583134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0af41026c0 0x7f0af40fbd30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0adc004940 tx=0x7f0adc009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:18.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.422+0000 7f0af989a700 1 -- 172.21.15.63:0/1261583134 shutdown_connections 2024-03-22T07:12:18.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.422+0000 7f0af989a700 1 --2- 172.21.15.63:0/1261583134 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0af41026c0 0x7f0af40fbd30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:18.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.422+0000 7f0af989a700 1 --2- 172.21.15.63:0/1261583134 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0af4101d20 0x7f0af41020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:18.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.422+0000 7f0af989a700 1 -- 172.21.15.63:0/1261583134 >> 172.21.15.63:0/1261583134 conn(0x7f0af40f7870 msgr2=0x7f0af40f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:18.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.422+0000 7f0af989a700 1 -- 172.21.15.63:0/1261583134 shutdown_connections 2024-03-22T07:12:18.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.422+0000 7f0af989a700 1 -- 172.21.15.63:0/1261583134 wait complete. 2024-03-22T07:12:18.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.423+0000 7f0af989a700 1 Processor -- start 2024-03-22T07:12:18.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.423+0000 7f0af989a700 1 -- start start 2024-03-22T07:12:18.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.423+0000 7f0af989a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0af4101d20 0x7f0af4113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:18.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.423+0000 7f0af989a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0af41026c0 0x7f0af4113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:18.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.423+0000 7f0af989a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af4114520 con 0x7f0af41026c0 2024-03-22T07:12:18.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.423+0000 7f0af989a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af410db40 con 0x7f0af4101d20 2024-03-22T07:12:18.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.423+0000 7f0af27fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0af41026c0 0x7f0af4113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:18.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.423+0000 7f0af27fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0af41026c0 0x7f0af4113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49164/0 (socket says 172.21.15.63:49164) 2024-03-22T07:12:18.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.423+0000 7f0af27fc700 1 -- 172.21.15.63:0/4146101871 learned_addr learned my addr 172.21.15.63:0/4146101871 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:18.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.423+0000 7f0af2ffd700 1 --2- 172.21.15.63:0/4146101871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0af4101d20 0x7f0af4113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:18.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.423+0000 7f0af27fc700 1 -- 172.21.15.63:0/4146101871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0af4101d20 msgr2=0x7f0af4113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:18.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.423+0000 7f0af27fc700 1 --2- 172.21.15.63:0/4146101871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0af4101d20 0x7f0af4113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:18.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.423+0000 7f0af27fc700 1 -- 172.21.15.63:0/4146101871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0adc005040 con 0x7f0af41026c0 2024-03-22T07:12:18.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.423+0000 7f0af2ffd700 1 --2- 172.21.15.63:0/4146101871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0af4101d20 0x7f0af4113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:12:18.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.424+0000 7f0af27fc700 1 --2- 172.21.15.63:0/4146101871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0af41026c0 0x7f0af4113ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0adc009100 tx=0x7f0adc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:18.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.424+0000 7f0aeb7fe700 1 -- 172.21.15.63:0/4146101871 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0adc00e040 con 0x7f0af41026c0 2024-03-22T07:12:18.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.424+0000 7f0aeb7fe700 1 -- 172.21.15.63:0/4146101871 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0adc013420 con 0x7f0af41026c0 2024-03-22T07:12:18.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.424+0000 7f0af989a700 1 -- 172.21.15.63:0/4146101871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af410ddc0 con 0x7f0af41026c0 2024-03-22T07:12:18.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.424+0000 7f0aeb7fe700 1 -- 172.21.15.63:0/4146101871 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0adc01c610 con 0x7f0af41026c0 2024-03-22T07:12:18.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.424+0000 7f0af989a700 1 -- 172.21.15.63:0/4146101871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af410e380 con 0x7f0af41026c0 2024-03-22T07:12:18.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.425+0000 7f0af989a700 1 -- 172.21.15.63:0/4146101871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af40fd470 con 0x7f0af41026c0 2024-03-22T07:12:18.427 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.425+0000 7f0aeb7fe700 1 -- 172.21.15.63:0/4146101871 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0adc013590 con 0x7f0af41026c0 2024-03-22T07:12:18.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.425+0000 7f0aeb7fe700 1 --2- 172.21.15.63:0/4146101871 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0ae00692a0 0x7f0ae006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:18.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.426+0000 7f0af2ffd700 1 --2- 172.21.15.63:0/4146101871 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0ae00692a0 0x7f0ae006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:18.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.426+0000 7f0aeb7fe700 1 -- 172.21.15.63:0/4146101871 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0adc0880d0 con 0x7f0af41026c0 2024-03-22T07:12:18.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.426+0000 7f0af2ffd700 1 --2- 172.21.15.63:0/4146101871 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0ae00692a0 0x7f0ae006b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4004c80 tx=0x7f0ae4009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:18.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.430+0000 7f0aeb7fe700 1 -- 172.21.15.63:0/4146101871 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0adc054740 con 0x7f0af41026c0 2024-03-22T07:12:18.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.621+0000 7f0af989a700 1 -- 172.21.15.63:0/4146101871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0af4108e00 con 0x7f0af41026c0 2024-03-22T07:12:18.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.622+0000 7f0aeb7fe700 1 -- 172.21.15.63:0/4146101871 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0adc057d60 con 0x7f0af41026c0 2024-03-22T07:12:18.623 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-03-22T07:12:18.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.624+0000 7f0af989a700 1 -- 172.21.15.63:0/4146101871 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0ae00692a0 msgr2=0x7f0ae006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:18.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.624+0000 7f0af989a700 1 --2- 172.21.15.63:0/4146101871 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0ae00692a0 0x7f0ae006b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4004c80 tx=0x7f0ae4009040 comp rx=0 tx=0).stop 2024-03-22T07:12:18.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.624+0000 7f0af989a700 1 -- 172.21.15.63:0/4146101871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0af41026c0 msgr2=0x7f0af4113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:18.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.624+0000 7f0af989a700 1 --2- 172.21.15.63:0/4146101871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0af41026c0 0x7f0af4113ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0adc009100 tx=0x7f0adc009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.624+0000 7f0af989a700 1 -- 172.21.15.63:0/4146101871 shutdown_connections 2024-03-22T07:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.624+0000 7f0af989a700 1 --2- 172.21.15.63:0/4146101871 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0ae00692a0 0x7f0ae006b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.624+0000 7f0af989a700 1 --2- 172.21.15.63:0/4146101871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0af4101d20 0x7f0af4113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.624+0000 7f0af989a700 1 --2- 172.21.15.63:0/4146101871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0af41026c0 0x7f0af4113ed0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.624+0000 7f0af989a700 1 -- 172.21.15.63:0/4146101871 >> 172.21.15.63:0/4146101871 conn(0x7f0af40f7870 msgr2=0x7f0af40f8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.624+0000 7f0af989a700 1 -- 172.21.15.63:0/4146101871 shutdown_connections 2024-03-22T07:12:18.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:18.624+0000 7f0af989a700 1 -- 172.21.15.63:0/4146101871 wait complete. 2024-03-22T07:12:19.285 INFO:tasks.cephadm:Setting up client nodes... 2024-03-22T07:12:19.285 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-22T07:12:19.549 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:19.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:19 smithi063 ceph-mon[29200]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:19.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:19 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/4146101871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:12:19.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:19 smithi069 ceph-mon[36441]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:19.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:19 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/4146101871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:12:21.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.175+0000 7f465691d700 1 -- 172.21.15.63:0/3128826516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46500fbd20 msgr2=0x7f46500fc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:21.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.175+0000 7f465691d700 1 --2- 172.21.15.63:0/3128826516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46500fbd20 0x7f46500fc130 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4640004940 tx=0x7f4640009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:21.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.175+0000 7f465691d700 1 -- 172.21.15.63:0/3128826516 shutdown_connections 2024-03-22T07:12:21.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.175+0000 7f465691d700 1 --2- 172.21.15.63:0/3128826516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46500fcfd0 0x7f46500fd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:21.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.175+0000 7f465691d700 1 --2- 172.21.15.63:0/3128826516 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46500fbd20 0x7f46500fc130 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:21.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.175+0000 7f465691d700 1 -- 172.21.15.63:0/3128826516 >> 172.21.15.63:0/3128826516 conn(0x7f46500f7460 msgr2=0x7f46500f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:21.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.175+0000 7f465691d700 1 -- 172.21.15.63:0/3128826516 shutdown_connections 2024-03-22T07:12:21.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.175+0000 7f465691d700 1 -- 172.21.15.63:0/3128826516 wait complete. 2024-03-22T07:12:21.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.176+0000 7f465691d700 1 Processor -- start 2024-03-22T07:12:21.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.176+0000 7f465691d700 1 -- start start 2024-03-22T07:12:21.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.176+0000 7f465691d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46500fbd20 0x7f4650066020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:21.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.176+0000 7f465691d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46500fcfd0 0x7f46501126d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:21.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.176+0000 7f465691d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4650112c10 con 0x7f46500fbd20 2024-03-22T07:12:21.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.176+0000 7f465691d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4650112d50 con 0x7f46500fcfd0 2024-03-22T07:12:21.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.176+0000 7f464ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46500fbd20 0x7f4650066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:21.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.177+0000 7f464ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46500fbd20 0x7f4650066020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49180/0 (socket says 172.21.15.63:49180) 2024-03-22T07:12:21.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.177+0000 7f464ffff700 1 -- 172.21.15.63:0/3892359125 learned_addr learned my addr 172.21.15.63:0/3892359125 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:21.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.177+0000 7f464f7fe700 1 --2- 172.21.15.63:0/3892359125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46500fcfd0 0x7f46501126d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:21.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.177+0000 7f464ffff700 1 -- 172.21.15.63:0/3892359125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46500fcfd0 msgr2=0x7f46501126d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:21.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.177+0000 7f464ffff700 1 --2- 172.21.15.63:0/3892359125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46500fcfd0 0x7f46501126d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:21.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.177+0000 7f464ffff700 1 -- 172.21.15.63:0/3892359125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4640005040 con 0x7f46500fbd20 2024-03-22T07:12:21.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.177+0000 7f464ffff700 1 --2- 172.21.15.63:0/3892359125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46500fbd20 0x7f4650066020 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4640009150 tx=0x7f4640009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:21.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.177+0000 7f464cff9700 1 -- 172.21.15.63:0/3892359125 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f464000e040 con 0x7f46500fbd20 2024-03-22T07:12:21.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.177+0000 7f464cff9700 1 -- 172.21.15.63:0/3892359125 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4640004290 con 0x7f46500fbd20 2024-03-22T07:12:21.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.177+0000 7f465691d700 1 -- 172.21.15.63:0/3892359125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4650112f80 con 0x7f46500fbd20 2024-03-22T07:12:21.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.177+0000 7f464cff9700 1 -- 172.21.15.63:0/3892359125 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f464001c630 con 0x7f46500fbd20 2024-03-22T07:12:21.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.177+0000 7f465691d700 1 -- 172.21.15.63:0/3892359125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4650113460 con 0x7f46500fbd20 2024-03-22T07:12:21.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.178+0000 7f465691d700 1 -- 172.21.15.63:0/3892359125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465004c710 con 0x7f46500fbd20 2024-03-22T07:12:21.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.179+0000 7f464cff9700 1 -- 172.21.15.63:0/3892359125 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f464000bbf0 con 0x7f46500fbd20 2024-03-22T07:12:21.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.179+0000 7f464cff9700 1 --2- 172.21.15.63:0/3892359125 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f46380692a0 0x7f463806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:21.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.179+0000 7f464cff9700 1 -- 172.21.15.63:0/3892359125 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4640087bc0 con 0x7f46500fbd20 2024-03-22T07:12:21.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.179+0000 7f464f7fe700 1 --2- 172.21.15.63:0/3892359125 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f46380692a0 0x7f463806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:21.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.179+0000 7f464f7fe700 1 --2- 172.21.15.63:0/3892359125 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f46380692a0 0x7f463806b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4644009fd0 tx=0x7f464400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:21.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.183+0000 7f464cff9700 1 -- 172.21.15.63:0/3892359125 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f4640057cc0 con 0x7f46500fbd20 2024-03-22T07:12:21.457 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.456+0000 7f465691d700 1 -- 172.21.15.63:0/3892359125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f4650104810 con 0x7f46500fbd20 2024-03-22T07:12:21.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.460+0000 7f464cff9700 1 -- 172.21.15.63:0/3892359125 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f4640057850 con 0x7f46500fbd20 2024-03-22T07:12:21.461 INFO:teuthology.orchestra.run.smithi063.stdout:[client.0] 2024-03-22T07:12:21.461 INFO:teuthology.orchestra.run.smithi063.stdout: key = AQBVL/1lS8ZSGxAAaeVkT/QyMGbeTqKH9JcAbQ== 2024-03-22T07:12:21.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.462+0000 7f465691d700 1 -- 172.21.15.63:0/3892359125 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f46380692a0 msgr2=0x7f463806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:21.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.462+0000 7f465691d700 1 --2- 172.21.15.63:0/3892359125 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f46380692a0 0x7f463806b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4644009fd0 tx=0x7f464400b040 comp rx=0 tx=0).stop 2024-03-22T07:12:21.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.462+0000 7f465691d700 1 -- 172.21.15.63:0/3892359125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46500fbd20 msgr2=0x7f4650066020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:21.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.462+0000 7f465691d700 1 --2- 172.21.15.63:0/3892359125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46500fbd20 0x7f4650066020 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4640009150 tx=0x7f4640009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:21.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.462+0000 7f465691d700 1 -- 172.21.15.63:0/3892359125 shutdown_connections 2024-03-22T07:12:21.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.462+0000 7f465691d700 1 --2- 172.21.15.63:0/3892359125 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f46380692a0 0x7f463806b750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:21.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.462+0000 7f465691d700 1 --2- 172.21.15.63:0/3892359125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f46500fbd20 0x7f4650066020 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:21.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.462+0000 7f465691d700 1 --2- 172.21.15.63:0/3892359125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46500fcfd0 0x7f46501126d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:21.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.462+0000 7f465691d700 1 -- 172.21.15.63:0/3892359125 >> 172.21.15.63:0/3892359125 conn(0x7f46500f7460 msgr2=0x7f46500f8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:21.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.462+0000 7f465691d700 1 -- 172.21.15.63:0/3892359125 shutdown_connections 2024-03-22T07:12:21.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:21.462+0000 7f465691d700 1 -- 172.21.15.63:0/3892359125 wait complete. 2024-03-22T07:12:21.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:21 smithi063 ceph-mon[29200]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:21.734 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:21 smithi069 ceph-mon[36441]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:22.087 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T07:12:22.087 DEBUG:teuthology.orchestra.run.smithi063:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-22T07:12:22.087 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-22T07:12:22.129 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-22T07:12:22.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:22 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3892359125' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T07:12:22.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:22 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3892359125' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T07:12:22.386 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi069/config 2024-03-22T07:12:22.509 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:22 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/3892359125' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T07:12:22.509 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:22 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/3892359125' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T07:12:23.718 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:23 smithi069 ceph-mon[36441]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:23.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:23 smithi063 ceph-mon[29200]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:23.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.935+0000 7f7df5bdf700 1 -- 172.21.15.69:0/3819794151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7df00fc750 msgr2=0x7f7df00fcbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:23.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.935+0000 7f7df5bdf700 1 --2- 172.21.15.69:0/3819794151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7df00fc750 0x7f7df00fcbc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7de4004940 tx=0x7f7de4009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:23.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.935+0000 7f7df5bdf700 1 -- 172.21.15.69:0/3819794151 shutdown_connections 2024-03-22T07:12:23.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.935+0000 7f7df5bdf700 1 --2- 172.21.15.69:0/3819794151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7df00fc750 0x7f7df00fcbc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:23.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.935+0000 7f7df5bdf700 1 --2- 172.21.15.69:0/3819794151 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7df00fb4a0 0x7f7df00fb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:23.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.935+0000 7f7df5bdf700 1 -- 172.21.15.69:0/3819794151 >> 172.21.15.69:0/3819794151 conn(0x7f7df00f6c00 msgr2=0x7f7df00f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:23.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.935+0000 7f7df5bdf700 1 -- 172.21.15.69:0/3819794151 shutdown_connections 2024-03-22T07:12:23.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.935+0000 7f7df5bdf700 1 -- 172.21.15.69:0/3819794151 wait complete. 2024-03-22T07:12:23.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.936+0000 7f7df5bdf700 1 Processor -- start 2024-03-22T07:12:23.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.936+0000 7f7df5bdf700 1 -- start start 2024-03-22T07:12:23.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.936+0000 7f7df5bdf700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7df00fb4a0 0x7f7df010c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:23.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.936+0000 7f7df5bdf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7df00fc750 0x7f7df010ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:23.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.936+0000 7f7df5bdf700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df010d430 con 0x7f7df00fb4a0 2024-03-22T07:12:23.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.936+0000 7f7df5bdf700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df010d5a0 con 0x7f7df00fc750 2024-03-22T07:12:23.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.936+0000 7f7deeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7df00fc750 0x7f7df010ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:23.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.936+0000 7f7deeffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7df00fc750 0x7f7df010ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48338/0 (socket says 172.21.15.69:48338) 2024-03-22T07:12:23.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.936+0000 7f7deeffd700 1 -- 172.21.15.69:0/1827462568 learned_addr learned my addr 172.21.15.69:0/1827462568 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-03-22T07:12:23.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.936+0000 7f7def7fe700 1 --2- 172.21.15.69:0/1827462568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7df00fb4a0 0x7f7df010c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:23.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.936+0000 7f7deeffd700 1 -- 172.21.15.69:0/1827462568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7df00fb4a0 msgr2=0x7f7df010c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:23.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.936+0000 7f7deeffd700 1 --2- 172.21.15.69:0/1827462568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7df00fb4a0 0x7f7df010c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:23.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.936+0000 7f7deeffd700 1 -- 172.21.15.69:0/1827462568 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de4005040 con 0x7f7df00fc750 2024-03-22T07:12:23.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.937+0000 7f7deeffd700 1 --2- 172.21.15.69:0/1827462568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7df00fc750 0x7f7df010ce30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7de40090d0 tx=0x7f7de4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:23.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.937+0000 7f7dd7fff700 1 -- 172.21.15.69:0/1827462568 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7de400e040 con 0x7f7df00fc750 2024-03-22T07:12:23.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.937+0000 7f7dd7fff700 1 -- 172.21.15.69:0/1827462568 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7de400bbf0 con 0x7f7df00fc750 2024-03-22T07:12:23.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.937+0000 7f7dd7fff700 1 -- 172.21.15.69:0/1827462568 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7de401c3f0 con 0x7f7df00fc750 2024-03-22T07:12:23.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.937+0000 7f7df5bdf700 1 -- 172.21.15.69:0/1827462568 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df0111fa0 con 0x7f7df00fc750 2024-03-22T07:12:23.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.937+0000 7f7df5bdf700 1 -- 172.21.15.69:0/1827462568 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df0112530 con 0x7f7df00fc750 2024-03-22T07:12:23.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.938+0000 7f7df5bdf700 1 -- 172.21.15.69:0/1827462568 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df004c710 con 0x7f7df00fc750 2024-03-22T07:12:23.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.938+0000 7f7dd7fff700 1 -- 172.21.15.69:0/1827462568 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f7de400bd60 con 0x7f7df00fc750 2024-03-22T07:12:23.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.939+0000 7f7dd7fff700 1 --2- 172.21.15.69:0/1827462568 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f7dd80692b0 0x7f7dd806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:23.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.939+0000 7f7dd7fff700 1 -- 172.21.15.69:0/1827462568 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7de4088170 con 0x7f7df00fc750 2024-03-22T07:12:23.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.939+0000 7f7def7fe700 1 --2- 172.21.15.69:0/1827462568 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f7dd80692b0 0x7f7dd806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:23.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.940+0000 7f7def7fe700 1 --2- 172.21.15.69:0/1827462568 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f7dd80692b0 0x7f7dd806b760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7de0007fd0 tx=0x7f7de0009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:23.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:23.943+0000 7f7dd7fff700 1 -- 172.21.15.69:0/1827462568 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f7de4058310 con 0x7f7df00fc750 2024-03-22T07:12:24.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:24.201+0000 7f7df5bdf700 1 -- 172.21.15.69:0/1827462568 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f7df0103f90 con 0x7f7df00fc750 2024-03-22T07:12:24.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:24.207+0000 7f7dd7fff700 1 -- 172.21.15.69:0/1827462568 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7f7de4057ea0 con 0x7f7df00fc750 2024-03-22T07:12:24.208 INFO:teuthology.orchestra.run.smithi069.stdout:[client.1] 2024-03-22T07:12:24.208 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQBYL/1lT/cyDBAAgtwfh6WEEYS7pZ5rFeKdBg== 2024-03-22T07:12:24.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:24.209+0000 7f7df5bdf700 1 -- 172.21.15.69:0/1827462568 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f7dd80692b0 msgr2=0x7f7dd806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:24.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:24.209+0000 7f7df5bdf700 1 --2- 172.21.15.69:0/1827462568 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f7dd80692b0 0x7f7dd806b760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7de0007fd0 tx=0x7f7de0009040 comp rx=0 tx=0).stop 2024-03-22T07:12:24.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:24.209+0000 7f7df5bdf700 1 -- 172.21.15.69:0/1827462568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7df00fc750 msgr2=0x7f7df010ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:24.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:24.209+0000 7f7df5bdf700 1 --2- 172.21.15.69:0/1827462568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7df00fc750 0x7f7df010ce30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7de40090d0 tx=0x7f7de4009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:24.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:24.209+0000 7f7df5bdf700 1 -- 172.21.15.69:0/1827462568 shutdown_connections 2024-03-22T07:12:24.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:24.209+0000 7f7df5bdf700 1 --2- 172.21.15.69:0/1827462568 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f7dd80692b0 0x7f7dd806b760 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:24.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:24.209+0000 7f7df5bdf700 1 --2- 172.21.15.69:0/1827462568 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7df00fb4a0 0x7f7df010c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:24.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:24.209+0000 7f7df5bdf700 1 --2- 172.21.15.69:0/1827462568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7df00fc750 0x7f7df010ce30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:24.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:24.209+0000 7f7df5bdf700 1 -- 172.21.15.69:0/1827462568 >> 172.21.15.69:0/1827462568 conn(0x7f7df00f6c00 msgr2=0x7f7df00ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:24.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:24.209+0000 7f7df5bdf700 1 -- 172.21.15.69:0/1827462568 shutdown_connections 2024-03-22T07:12:24.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-03-22T07:12:24.209+0000 7f7df5bdf700 1 -- 172.21.15.69:0/1827462568 wait complete. 2024-03-22T07:12:24.769 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:24 smithi069 ceph-mon[36441]: from='client.? 172.21.15.69:0/1827462568' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T07:12:24.769 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:24 smithi069 ceph-mon[36441]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T07:12:24.769 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:24 smithi069 ceph-mon[36441]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T07:12:24.796 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-22T07:12:24.796 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-22T07:12:24.796 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-22T07:12:24.840 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-22T07:12:24.841 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-22T07:12:24.841 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-22T07:12:24.869 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:24 smithi063 ceph-mon[29200]: from='client.? 172.21.15.69:0/1827462568' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T07:12:24.869 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:24 smithi063 ceph-mon[29200]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-22T07:12:24.869 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:24 smithi063 ceph-mon[29200]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-22T07:12:25.111 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:25.524 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:25 smithi063 ceph-mon[29200]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:25.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:25 smithi069 ceph-mon[36441]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:26.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.688+0000 7f0f0df35700 1 -- 172.21.15.63:0/2313182918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f08101d20 msgr2=0x7f0f081020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:26.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.688+0000 7f0f0df35700 1 --2- 172.21.15.63:0/2313182918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f08101d20 0x7f0f081020f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8004940 tx=0x7f0ef8009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:26.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.688+0000 7f0f0df35700 1 -- 172.21.15.63:0/2313182918 shutdown_connections 2024-03-22T07:12:26.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.688+0000 7f0f0df35700 1 --2- 172.21.15.63:0/2313182918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f081026c0 0x7f0f080fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:26.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.688+0000 7f0f0df35700 1 --2- 172.21.15.63:0/2313182918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f08101d20 0x7f0f081020f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:26.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.688+0000 7f0f0df35700 1 -- 172.21.15.63:0/2313182918 >> 172.21.15.63:0/2313182918 conn(0x7f0f080f7870 msgr2=0x7f0f080f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:26.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.688+0000 7f0f0df35700 1 -- 172.21.15.63:0/2313182918 shutdown_connections 2024-03-22T07:12:26.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.688+0000 7f0f0df35700 1 -- 172.21.15.63:0/2313182918 wait complete. 2024-03-22T07:12:26.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.688+0000 7f0f0df35700 1 Processor -- start 2024-03-22T07:12:26.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.689+0000 7f0f0df35700 1 -- start start 2024-03-22T07:12:26.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.689+0000 7f0f0df35700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f08101d20 0x7f0f08102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:26.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.689+0000 7f0f0df35700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f081026c0 0x7f0f081030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:26.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.689+0000 7f0f0df35700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f08104a80 con 0x7f0f081026c0 2024-03-22T07:12:26.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.689+0000 7f0f0df35700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f08103600 con 0x7f0f08101d20 2024-03-22T07:12:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.689+0000 7f0f06ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f081026c0 0x7f0f081030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.689+0000 7f0f077fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f08101d20 0x7f0f08102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.689+0000 7f0f06ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f081026c0 0x7f0f081030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45384/0 (socket says 172.21.15.63:45384) 2024-03-22T07:12:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.689+0000 7f0f06ffd700 1 -- 172.21.15.63:0/3259461832 learned_addr learned my addr 172.21.15.63:0/3259461832 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.689+0000 7f0f077fe700 1 --2- 172.21.15.63:0/3259461832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f08101d20 0x7f0f08102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.63:36298/0 (socket says 172.21.15.63:36298) 2024-03-22T07:12:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.689+0000 7f0f06ffd700 1 -- 172.21.15.63:0/3259461832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f08101d20 msgr2=0x7f0f08102b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.689+0000 7f0f06ffd700 1 --2- 172.21.15.63:0/3259461832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f08101d20 0x7f0f08102b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.689+0000 7f0f06ffd700 1 -- 172.21.15.63:0/3259461832 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef8005040 con 0x7f0f081026c0 2024-03-22T07:12:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.689+0000 7f0f06ffd700 1 --2- 172.21.15.63:0/3259461832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f081026c0 0x7f0f081030c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0efc002a80 tx=0x7f0efc004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.689+0000 7f0eeffff700 1 -- 172.21.15.63:0/3259461832 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0efc004e80 con 0x7f0f081026c0 2024-03-22T07:12:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.689+0000 7f0eeffff700 1 -- 172.21.15.63:0/3259461832 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0efc00f040 con 0x7f0f081026c0 2024-03-22T07:12:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.689+0000 7f0f0df35700 1 -- 172.21.15.63:0/3259461832 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f081038e0 con 0x7f0f081026c0 2024-03-22T07:12:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.689+0000 7f0eeffff700 1 -- 172.21.15.63:0/3259461832 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0efc00daa0 con 0x7f0f081026c0 2024-03-22T07:12:26.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.690+0000 7f0f0df35700 1 -- 172.21.15.63:0/3259461832 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0811a1c0 con 0x7f0f081026c0 2024-03-22T07:12:26.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.690+0000 7f0f0df35700 1 -- 172.21.15.63:0/3259461832 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f080fd470 con 0x7f0f081026c0 2024-03-22T07:12:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.691+0000 7f0eeffff700 1 -- 172.21.15.63:0/3259461832 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0efc007cc0 con 0x7f0f081026c0 2024-03-22T07:12:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.691+0000 7f0eeffff700 1 --2- 172.21.15.63:0/3259461832 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0ef00692a0 0x7f0ef006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.691+0000 7f0eeffff700 1 -- 172.21.15.63:0/3259461832 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0efc086480 con 0x7f0f081026c0 2024-03-22T07:12:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.691+0000 7f0f077fe700 1 --2- 172.21.15.63:0/3259461832 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0ef00692a0 0x7f0ef006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:26.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.692+0000 7f0f077fe700 1 --2- 172.21.15.63:0/3259461832 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0ef00692a0 0x7f0ef006b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8004f90 tx=0x7f0ef8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:26.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.696+0000 7f0eeffff700 1 -- 172.21.15.63:0/3259461832 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0efc04d190 con 0x7f0f081026c0 2024-03-22T07:12:26.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.949+0000 7f0f0df35700 1 -- 172.21.15.63:0/3259461832 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0f0804c710 con 0x7f0f081026c0 2024-03-22T07:12:26.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.953+0000 7f0eeffff700 1 -- 172.21.15.63:0/3259461832 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167163 (secure 0 0 0) 0x7f0efc04ad10 con 0x7f0f081026c0 2024-03-22T07:12:26.956 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:12:26.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.955+0000 7f0f0df35700 1 -- 172.21.15.63:0/3259461832 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0ef00692a0 msgr2=0x7f0ef006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:26.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.955+0000 7f0f0df35700 1 --2- 172.21.15.63:0/3259461832 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0ef00692a0 0x7f0ef006b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8004f90 tx=0x7f0ef8009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:26.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.955+0000 7f0f0df35700 1 -- 172.21.15.63:0/3259461832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f081026c0 msgr2=0x7f0f081030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:26.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.955+0000 7f0f0df35700 1 --2- 172.21.15.63:0/3259461832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f081026c0 0x7f0f081030c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0efc002a80 tx=0x7f0efc004630 comp rx=0 tx=0).stop 2024-03-22T07:12:26.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.955+0000 7f0f0df35700 1 -- 172.21.15.63:0/3259461832 shutdown_connections 2024-03-22T07:12:26.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.955+0000 7f0f0df35700 1 --2- 172.21.15.63:0/3259461832 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0ef00692a0 0x7f0ef006b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:26.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.955+0000 7f0f0df35700 1 --2- 172.21.15.63:0/3259461832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f08101d20 0x7f0f08102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:26.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.955+0000 7f0f0df35700 1 --2- 172.21.15.63:0/3259461832 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0f081026c0 0x7f0f081030c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:26.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.955+0000 7f0f0df35700 1 -- 172.21.15.63:0/3259461832 >> 172.21.15.63:0/3259461832 conn(0x7f0f080f7870 msgr2=0x7f0f080f8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:26.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.955+0000 7f0f0df35700 1 -- 172.21.15.63:0/3259461832 shutdown_connections 2024-03-22T07:12:26.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:26.955+0000 7f0f0df35700 1 -- 172.21.15.63:0/3259461832 wait complete. 2024-03-22T07:12:27.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:27 smithi063 ceph-mon[29200]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:27.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:27 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3259461832' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-22T07:12:27.654 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":19,"active_gid":14208,"active_name":"smithi063.mnxayh","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6800","nonce":2001368263},{"type":"v1","addr":"172.21.15.63:6801","nonce":2001368263}]},"active_addr":"172.21.15.63:6801/2001368263","active_change":"2024-03-22T07:08:44.186912+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14230,"name":"smithi069.gfziwh","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.63:8443/","prometheus":"http://172.21.15.63:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":2835250916}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":1701838}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":41352707}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.63:0","nonce":108295103}]}]} 2024-03-22T07:12:27.657 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-22T07:12:27.657 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-22T07:12:27.657 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-22T07:12:27.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:27 smithi069 ceph-mon[36441]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:27.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:27 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/3259461832' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-22T07:12:27.957 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:29.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:29 smithi063 ceph-mon[29200]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:29.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:29 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:12:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.685+0000 7f3870d89700 1 -- 172.21.15.63:0/2962812189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f386c102710 msgr2=0x7f386c0fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.685+0000 7f3870d89700 1 --2- 172.21.15.63:0/2962812189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f386c102710 0x7f386c0fbd80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f385c004940 tx=0x7f385c009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.685+0000 7f3870d89700 1 -- 172.21.15.63:0/2962812189 shutdown_connections 2024-03-22T07:12:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.685+0000 7f3870d89700 1 --2- 172.21.15.63:0/2962812189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f386c102710 0x7f386c0fbd80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.685+0000 7f3870d89700 1 --2- 172.21.15.63:0/2962812189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f386c101d70 0x7f386c102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.685+0000 7f3870d89700 1 -- 172.21.15.63:0/2962812189 >> 172.21.15.63:0/2962812189 conn(0x7f386c072bc0 msgr2=0x7f386c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:29.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.686+0000 7f3870d89700 1 -- 172.21.15.63:0/2962812189 shutdown_connections 2024-03-22T07:12:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.686+0000 7f3870d89700 1 -- 172.21.15.63:0/2962812189 wait complete. 2024-03-22T07:12:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.686+0000 7f3870d89700 1 Processor -- start 2024-03-22T07:12:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.686+0000 7f3870d89700 1 -- start start 2024-03-22T07:12:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.686+0000 7f3870d89700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f386c101d70 0x7f386c10eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.686+0000 7f3870d89700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f386c102710 0x7f386c10f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.686+0000 7f3870d89700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c10f720 con 0x7f386c102710 2024-03-22T07:12:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.686+0000 7f3870d89700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c10f860 con 0x7f386c101d70 2024-03-22T07:12:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.686+0000 7f3869d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f386c102710 0x7f386c10f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.686+0000 7f3869d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f386c102710 0x7f386c10f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45416/0 (socket says 172.21.15.63:45416) 2024-03-22T07:12:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.686+0000 7f3869d9b700 1 -- 172.21.15.63:0/871665149 learned_addr learned my addr 172.21.15.63:0/871665149 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.687+0000 7f386a59c700 1 --2- 172.21.15.63:0/871665149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f386c101d70 0x7f386c10eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.687+0000 7f3869d9b700 1 -- 172.21.15.63:0/871665149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f386c101d70 msgr2=0x7f386c10eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.687+0000 7f3869d9b700 1 --2- 172.21.15.63:0/871665149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f386c101d70 0x7f386c10eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.687+0000 7f3869d9b700 1 -- 172.21.15.63:0/871665149 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c005040 con 0x7f386c102710 2024-03-22T07:12:29.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.687+0000 7f3869d9b700 1 --2- 172.21.15.63:0/871665149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f386c102710 0x7f386c10f1e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f385c009100 tx=0x7f385c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.687+0000 7f3862ffd700 1 -- 172.21.15.63:0/871665149 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f385c00e040 con 0x7f386c102710 2024-03-22T07:12:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.687+0000 7f3862ffd700 1 -- 172.21.15.63:0/871665149 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f385c013420 con 0x7f386c102710 2024-03-22T07:12:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.687+0000 7f3862ffd700 1 -- 172.21.15.63:0/871665149 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f385c01c610 con 0x7f386c102710 2024-03-22T07:12:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.687+0000 7f3870d89700 1 -- 172.21.15.63:0/871665149 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386c117b50 con 0x7f386c102710 2024-03-22T07:12:29.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.687+0000 7f3870d89700 1 -- 172.21.15.63:0/871665149 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386c1180b0 con 0x7f386c102710 2024-03-22T07:12:29.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.688+0000 7f3870d89700 1 -- 172.21.15.63:0/871665149 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f386c104240 con 0x7f386c102710 2024-03-22T07:12:29.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.688+0000 7f3862ffd700 1 -- 172.21.15.63:0/871665149 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f385c013590 con 0x7f386c102710 2024-03-22T07:12:29.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.689+0000 7f3862ffd700 1 --2- 172.21.15.63:0/871665149 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f38580692a0 0x7f385806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:29.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.689+0000 7f3862ffd700 1 -- 172.21.15.63:0/871665149 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f385c088080 con 0x7f386c102710 2024-03-22T07:12:29.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.689+0000 7f386a59c700 1 --2- 172.21.15.63:0/871665149 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f38580692a0 0x7f385806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:29.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.689+0000 7f386a59c700 1 --2- 172.21.15.63:0/871665149 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f38580692a0 0x7f385806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3854004cc0 tx=0x7f385400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:29.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.694+0000 7f3862ffd700 1 -- 172.21.15.63:0/871665149 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f385c0546f0 con 0x7f386c102710 2024-03-22T07:12:29.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:29 smithi069 ceph-mon[36441]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:29.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:29 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:12:29.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.888+0000 7f3870d89700 1 -- 172.21.15.63:0/871665149 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f386c0fd4c0 con 0x7f386c102710 2024-03-22T07:12:29.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.889+0000 7f3862ffd700 1 -- 172.21.15.63:0/871665149 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11043 (secure 0 0 0) 0x7f385c057d10 con 0x7f386c102710 2024-03-22T07:12:29.890 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:12:29.890 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":41,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","created":"2024-03-22T07:06:38.004887+0000","modified":"2024-03-22T07:12:10.641337+0000","last_up_change":"2024-03-22T07:12:09.636009+0000","last_in_change":"2024-03-22T07:11:51.027495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T07:10:56.524122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"33ade431-7f97-43c7-bf9d-962c12d3ba7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3791141232},{"type":"v1","addr":"172.21.15.63:6803","nonce":3791141232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3791141232},{"type":"v1","addr":"172.21.15.63:6805","nonce":3791141232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":3791141232},{"type":"v1","addr":"172.21.15.63:6809","nonce":3791141232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3791141232},{"type":"v1","addr":"172.21.15.63:6807","nonce":3791141232}]},"public_addr":"172.21.15.63:6803/3791141232","cluster_addr":"172.21.15.63:6805/3791141232","heartbeat_back_addr":"172.21.15.63:6809/3791141232","heartbeat_front_addr":"172.21.15.63:6807/3791141232","state":["exists","up"]},{"osd":1,"uuid":"0643ddb6-db9d-464c-8236-3aea2893de83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1043983969},{"type":"v1","addr":"172.21.15.63:6811","nonce":1043983969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1043983969},{"type":"v1","addr":"172.21.15.63:6813","nonce":1043983969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1043983969},{"type":"v1","addr":"172.21.15.63:6817","nonce":1043983969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1043983969},{"type":"v1","addr":"172.21.15.63:6815","nonce":1043983969}]},"public_addr":"172.21.15.63:6811/1043983969","cluster_addr":"172.21.15.63:6813/1043983969","heartbeat_back_addr":"172.21.15.63:6817/1043983969","heartbeat_front_addr":"172.21.15.63:6815/1043983969","state":["exists","up"]},{"osd":2,"uuid":"140b09d0-1fb6-4a3b-873a-048b797c0180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3603088351},{"type":"v1","addr":"172.21.15.63:6819","nonce":3603088351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3603088351},{"type":"v1","addr":"172.21.15.63:6821","nonce":3603088351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3603088351},{"type":"v1","addr":"172.21.15.63:6825","nonce":3603088351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3603088351},{"type":"v1","addr":"172.21.15.63:6823","nonce":3603088351}]},"public_addr":"172.21.15.63:6819/3603088351","cluster_addr":"172.21.15.63:6821/3603088351","heartbeat_back_addr":"172.21.15.63:6825/3603088351","heartbeat_front_addr":"172.21.15.63:6823/3603088351","state":["exists","up"]},{"osd":3,"uuid":"1d92b77e-6bcc-4585-96f1-72251303cc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1811566075},{"type":"v1","addr":"172.21.15.69:6801","nonce":1811566075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1811566075},{"type":"v1","addr":"172.21.15.69:6803","nonce":1811566075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1811566075},{"type":"v1","addr":"172.21.15.69:6807","nonce":1811566075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1811566075},{"type":"v1","addr":"172.21.15.69:6805","nonce":1811566075}]},"public_addr":"172.21.15.69:6801/1811566075","cluster_addr":"172.21.15.69:6803/1811566075","heartbeat_back_addr":"172.21.15.69:6807/1811566075","heartbeat_front_addr":"172.21.15.69:6805/1811566075","state":["exists","up"]},{"osd":4,"uuid":"28bdc8bc-41d5-4c01-9b29-d5833223d1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":656541591},{"type":"v1","addr":"172.21.15.69:6809","nonce":656541591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":656541591},{"type":"v1","addr":"172.21.15.69:6811","nonce":656541591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":656541591},{"type":"v1","addr":"172.21.15.69:6815","nonce":656541591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":656541591},{"type":"v1","addr":"172.21.15.69:6813","nonce":656541591}]},"public_addr":"172.21.15.69:6809/656541591","cluster_addr":"172.21.15.69:6811/656541591","heartbeat_back_addr":"172.21.15.69:6815/656541591","heartbeat_front_addr":"172.21.15.69:6813/656541591","state":["exists","up"]},{"osd":5,"uuid":"566db34e-ce0e-436f-aa90-df2c80d812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2157641649},{"type":"v1","addr":"172.21.15.69:6817","nonce":2157641649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2157641649},{"type":"v1","addr":"172.21.15.69:6819","nonce":2157641649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2157641649},{"type":"v1","addr":"172.21.15.69:6823","nonce":2157641649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2157641649},{"type":"v1","addr":"172.21.15.69:6821","nonce":2157641649}]},"public_addr":"172.21.15.69:6817/2157641649","cluster_addr":"172.21.15.69:6819/2157641649","heartbeat_back_addr":"172.21.15.69:6823/2157641649","heartbeat_front_addr":"172.21.15.69:6821/2157641649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:09:56.132204+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:10:24.051511+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:10:52.969047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:11:17.576349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:11:42.883846+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:12:08.488698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6801/2296442540":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/47408280":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/466771007":"2024-03-23T07:08:44.186801+0000","172.21.15.63:6801/480818549":"2024-03-23T07:07:41.261927+0000","172.21.15.63:0/1323074353":"2024-03-23T07:07:41.261927+0000","172.21.15.63:0/1818534437":"2024-03-23T07:07:41.261927+0000","172.21.15.63:6801/2616455474":"2024-03-23T07:07:10.417547+0000","172.21.15.63:6800/2616455474":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/1649622442":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/3927392742":"2024-03-23T07:08:44.186801+0000","172.21.15.63:6800/2296442540":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/424866385":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/2739978443":"2024-03-23T07:07:10.417547+0000","172.21.15.63:6800/480818549":"2024-03-23T07:07:41.261927+0000","172.21.15.63:0/2503315479":"2024-03-23T07:07:41.261927+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T07:12:29.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.891+0000 7f3870d89700 1 -- 172.21.15.63:0/871665149 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f38580692a0 msgr2=0x7f385806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:29.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.891+0000 7f3870d89700 1 --2- 172.21.15.63:0/871665149 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f38580692a0 0x7f385806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3854004cc0 tx=0x7f385400b040 comp rx=0 tx=0).stop 2024-03-22T07:12:29.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.891+0000 7f3870d89700 1 -- 172.21.15.63:0/871665149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f386c102710 msgr2=0x7f386c10f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:29.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.891+0000 7f3870d89700 1 --2- 172.21.15.63:0/871665149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f386c102710 0x7f386c10f1e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f385c009100 tx=0x7f385c009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:29.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.891+0000 7f3870d89700 1 -- 172.21.15.63:0/871665149 shutdown_connections 2024-03-22T07:12:29.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.891+0000 7f3870d89700 1 --2- 172.21.15.63:0/871665149 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f38580692a0 0x7f385806b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:29.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.891+0000 7f3870d89700 1 --2- 172.21.15.63:0/871665149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f386c101d70 0x7f386c10eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:29.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.891+0000 7f3870d89700 1 --2- 172.21.15.63:0/871665149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f386c102710 0x7f386c10f1e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:29.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.891+0000 7f3870d89700 1 -- 172.21.15.63:0/871665149 >> 172.21.15.63:0/871665149 conn(0x7f386c072bc0 msgr2=0x7f386c0fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:29.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.891+0000 7f3870d89700 1 -- 172.21.15.63:0/871665149 shutdown_connections 2024-03-22T07:12:29.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:29.891+0000 7f3870d89700 1 -- 172.21.15.63:0/871665149 wait complete. 2024-03-22T07:12:30.617 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-22T07:12:30.617 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-22T07:12:30.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:30 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/871665149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:12:30.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:30 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/871665149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:12:30.879 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:31.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:31 smithi069 ceph-mon[36441]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:31.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:31 smithi063 ceph-mon[29200]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:32.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.502+0000 7f817ea2a700 1 -- 172.21.15.63:0/3468045371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81781037d0 msgr2=0x7f81780fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:32.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.502+0000 7f817ea2a700 1 --2- 172.21.15.63:0/3468045371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81781037d0 0x7f81780fce40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f816c004940 tx=0x7f816c009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.502+0000 7f817ea2a700 1 -- 172.21.15.63:0/3468045371 shutdown_connections 2024-03-22T07:12:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.502+0000 7f817ea2a700 1 --2- 172.21.15.63:0/3468045371 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f81781037d0 0x7f81780fce40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.502+0000 7f817ea2a700 1 --2- 172.21.15.63:0/3468045371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8178102e30 0x7f8178103200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.502+0000 7f817ea2a700 1 -- 172.21.15.63:0/3468045371 >> 172.21.15.63:0/3468045371 conn(0x7f81780f8a20 msgr2=0x7f81780fae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.502+0000 7f817ea2a700 1 -- 172.21.15.63:0/3468045371 shutdown_connections 2024-03-22T07:12:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.502+0000 7f817ea2a700 1 -- 172.21.15.63:0/3468045371 wait complete. 2024-03-22T07:12:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.503+0000 7f817ea2a700 1 Processor -- start 2024-03-22T07:12:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.503+0000 7f817ea2a700 1 -- start start 2024-03-22T07:12:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.503+0000 7f817ea2a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8178102e30 0x7f81780702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.503+0000 7f817ea2a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81781037d0 0x7f8178070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.503+0000 7f817ea2a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8178072210 con 0x7f8178102e30 2024-03-22T07:12:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.503+0000 7f817ea2a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8178070d40 con 0x7f81781037d0 2024-03-22T07:12:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.503+0000 7f8177fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8178102e30 0x7f81780702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:32.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.503+0000 7f8177fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8178102e30 0x7f81780702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45436/0 (socket says 172.21.15.63:45436) 2024-03-22T07:12:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.503+0000 7f8177fff700 1 -- 172.21.15.63:0/2717841535 learned_addr learned my addr 172.21.15.63:0/2717841535 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.504+0000 7f81777fe700 1 --2- 172.21.15.63:0/2717841535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81781037d0 0x7f8178070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.504+0000 7f8177fff700 1 -- 172.21.15.63:0/2717841535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81781037d0 msgr2=0x7f8178070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.504+0000 7f8177fff700 1 --2- 172.21.15.63:0/2717841535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81781037d0 0x7f8178070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.504+0000 7f8177fff700 1 -- 172.21.15.63:0/2717841535 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816c005040 con 0x7f8178102e30 2024-03-22T07:12:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.504+0000 7f81777fe700 1 --2- 172.21.15.63:0/2717841535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81781037d0 0x7f8178070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:12:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.504+0000 7f8177fff700 1 --2- 172.21.15.63:0/2717841535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8178102e30 0x7f81780702c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f816800c830 tx=0x7f8168004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.504+0000 7f8174ff9700 1 -- 172.21.15.63:0/2717841535 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f816800a760 con 0x7f8178102e30 2024-03-22T07:12:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.504+0000 7f817ea2a700 1 -- 172.21.15.63:0/2717841535 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8178071020 con 0x7f8178102e30 2024-03-22T07:12:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.504+0000 7f8174ff9700 1 -- 172.21.15.63:0/2717841535 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f816800ad90 con 0x7f8178102e30 2024-03-22T07:12:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.504+0000 7f8174ff9700 1 -- 172.21.15.63:0/2717841535 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81680113a0 con 0x7f8178102e30 2024-03-22T07:12:32.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.504+0000 7f817ea2a700 1 -- 172.21.15.63:0/2717841535 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817811a140 con 0x7f8178102e30 2024-03-22T07:12:32.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.505+0000 7f817ea2a700 1 -- 172.21.15.63:0/2717841535 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8178105300 con 0x7f8178102e30 2024-03-22T07:12:32.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.506+0000 7f8174ff9700 1 -- 172.21.15.63:0/2717841535 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f816800a8c0 con 0x7f8178102e30 2024-03-22T07:12:32.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.506+0000 7f8174ff9700 1 --2- 172.21.15.63:0/2717841535 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f81600692a0 0x7f816006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:32.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.506+0000 7f8174ff9700 1 -- 172.21.15.63:0/2717841535 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8168086e90 con 0x7f8178102e30 2024-03-22T07:12:32.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.506+0000 7f81777fe700 1 --2- 172.21.15.63:0/2717841535 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f81600692a0 0x7f816006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:32.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.506+0000 7f81777fe700 1 --2- 172.21.15.63:0/2717841535 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f81600692a0 0x7f816006b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f816c009fd0 tx=0x7f816c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:32.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.510+0000 7f8174ff9700 1 -- 172.21.15.63:0/2717841535 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8168053500 con 0x7f8178102e30 2024-03-22T07:12:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.708+0000 7f817ea2a700 1 -- 172.21.15.63:0/2717841535 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81780fe580 con 0x7f8178102e30 2024-03-22T07:12:32.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.709+0000 7f8174ff9700 1 -- 172.21.15.63:0/2717841535 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11043 (secure 0 0 0) 0x7f8168056b20 con 0x7f8178102e30 2024-03-22T07:12:32.710 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:12:32.710 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":41,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","created":"2024-03-22T07:06:38.004887+0000","modified":"2024-03-22T07:12:10.641337+0000","last_up_change":"2024-03-22T07:12:09.636009+0000","last_in_change":"2024-03-22T07:11:51.027495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T07:10:56.524122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"33ade431-7f97-43c7-bf9d-962c12d3ba7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3791141232},{"type":"v1","addr":"172.21.15.63:6803","nonce":3791141232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3791141232},{"type":"v1","addr":"172.21.15.63:6805","nonce":3791141232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":3791141232},{"type":"v1","addr":"172.21.15.63:6809","nonce":3791141232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3791141232},{"type":"v1","addr":"172.21.15.63:6807","nonce":3791141232}]},"public_addr":"172.21.15.63:6803/3791141232","cluster_addr":"172.21.15.63:6805/3791141232","heartbeat_back_addr":"172.21.15.63:6809/3791141232","heartbeat_front_addr":"172.21.15.63:6807/3791141232","state":["exists","up"]},{"osd":1,"uuid":"0643ddb6-db9d-464c-8236-3aea2893de83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1043983969},{"type":"v1","addr":"172.21.15.63:6811","nonce":1043983969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1043983969},{"type":"v1","addr":"172.21.15.63:6813","nonce":1043983969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1043983969},{"type":"v1","addr":"172.21.15.63:6817","nonce":1043983969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1043983969},{"type":"v1","addr":"172.21.15.63:6815","nonce":1043983969}]},"public_addr":"172.21.15.63:6811/1043983969","cluster_addr":"172.21.15.63:6813/1043983969","heartbeat_back_addr":"172.21.15.63:6817/1043983969","heartbeat_front_addr":"172.21.15.63:6815/1043983969","state":["exists","up"]},{"osd":2,"uuid":"140b09d0-1fb6-4a3b-873a-048b797c0180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3603088351},{"type":"v1","addr":"172.21.15.63:6819","nonce":3603088351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3603088351},{"type":"v1","addr":"172.21.15.63:6821","nonce":3603088351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3603088351},{"type":"v1","addr":"172.21.15.63:6825","nonce":3603088351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3603088351},{"type":"v1","addr":"172.21.15.63:6823","nonce":3603088351}]},"public_addr":"172.21.15.63:6819/3603088351","cluster_addr":"172.21.15.63:6821/3603088351","heartbeat_back_addr":"172.21.15.63:6825/3603088351","heartbeat_front_addr":"172.21.15.63:6823/3603088351","state":["exists","up"]},{"osd":3,"uuid":"1d92b77e-6bcc-4585-96f1-72251303cc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1811566075},{"type":"v1","addr":"172.21.15.69:6801","nonce":1811566075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1811566075},{"type":"v1","addr":"172.21.15.69:6803","nonce":1811566075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1811566075},{"type":"v1","addr":"172.21.15.69:6807","nonce":1811566075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1811566075},{"type":"v1","addr":"172.21.15.69:6805","nonce":1811566075}]},"public_addr":"172.21.15.69:6801/1811566075","cluster_addr":"172.21.15.69:6803/1811566075","heartbeat_back_addr":"172.21.15.69:6807/1811566075","heartbeat_front_addr":"172.21.15.69:6805/1811566075","state":["exists","up"]},{"osd":4,"uuid":"28bdc8bc-41d5-4c01-9b29-d5833223d1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":656541591},{"type":"v1","addr":"172.21.15.69:6809","nonce":656541591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":656541591},{"type":"v1","addr":"172.21.15.69:6811","nonce":656541591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":656541591},{"type":"v1","addr":"172.21.15.69:6815","nonce":656541591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":656541591},{"type":"v1","addr":"172.21.15.69:6813","nonce":656541591}]},"public_addr":"172.21.15.69:6809/656541591","cluster_addr":"172.21.15.69:6811/656541591","heartbeat_back_addr":"172.21.15.69:6815/656541591","heartbeat_front_addr":"172.21.15.69:6813/656541591","state":["exists","up"]},{"osd":5,"uuid":"566db34e-ce0e-436f-aa90-df2c80d812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2157641649},{"type":"v1","addr":"172.21.15.69:6817","nonce":2157641649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2157641649},{"type":"v1","addr":"172.21.15.69:6819","nonce":2157641649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2157641649},{"type":"v1","addr":"172.21.15.69:6823","nonce":2157641649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2157641649},{"type":"v1","addr":"172.21.15.69:6821","nonce":2157641649}]},"public_addr":"172.21.15.69:6817/2157641649","cluster_addr":"172.21.15.69:6819/2157641649","heartbeat_back_addr":"172.21.15.69:6823/2157641649","heartbeat_front_addr":"172.21.15.69:6821/2157641649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:09:56.132204+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:10:24.051511+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:10:52.969047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:11:17.576349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:11:42.883846+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:12:08.488698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6801/2296442540":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/47408280":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/466771007":"2024-03-23T07:08:44.186801+0000","172.21.15.63:6801/480818549":"2024-03-23T07:07:41.261927+0000","172.21.15.63:0/1323074353":"2024-03-23T07:07:41.261927+0000","172.21.15.63:0/1818534437":"2024-03-23T07:07:41.261927+0000","172.21.15.63:6801/2616455474":"2024-03-23T07:07:10.417547+0000","172.21.15.63:6800/2616455474":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/1649622442":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/3927392742":"2024-03-23T07:08:44.186801+0000","172.21.15.63:6800/2296442540":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/424866385":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/2739978443":"2024-03-23T07:07:10.417547+0000","172.21.15.63:6800/480818549":"2024-03-23T07:07:41.261927+0000","172.21.15.63:0/2503315479":"2024-03-23T07:07:41.261927+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T07:12:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.711+0000 7f817ea2a700 1 -- 172.21.15.63:0/2717841535 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f81600692a0 msgr2=0x7f816006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.711+0000 7f817ea2a700 1 --2- 172.21.15.63:0/2717841535 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f81600692a0 0x7f816006b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f816c009fd0 tx=0x7f816c009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.711+0000 7f817ea2a700 1 -- 172.21.15.63:0/2717841535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8178102e30 msgr2=0x7f81780702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.711+0000 7f817ea2a700 1 --2- 172.21.15.63:0/2717841535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8178102e30 0x7f81780702c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f816800c830 tx=0x7f8168004470 comp rx=0 tx=0).stop 2024-03-22T07:12:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.711+0000 7f817ea2a700 1 -- 172.21.15.63:0/2717841535 shutdown_connections 2024-03-22T07:12:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.711+0000 7f817ea2a700 1 --2- 172.21.15.63:0/2717841535 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f81600692a0 0x7f816006b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.711+0000 7f817ea2a700 1 --2- 172.21.15.63:0/2717841535 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8178102e30 0x7f81780702c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.711+0000 7f817ea2a700 1 --2- 172.21.15.63:0/2717841535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f81781037d0 0x7f8178070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:32.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.711+0000 7f817ea2a700 1 -- 172.21.15.63:0/2717841535 >> 172.21.15.63:0/2717841535 conn(0x7f81780f8a20 msgr2=0x7f81781021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:32.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.711+0000 7f817ea2a700 1 -- 172.21.15.63:0/2717841535 shutdown_connections 2024-03-22T07:12:32.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:32.711+0000 7f817ea2a700 1 -- 172.21.15.63:0/2717841535 wait complete. 2024-03-22T07:12:33.416 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-22T07:12:33.416 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-22T07:12:33.417 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-22T07:12:33.417 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-22T07:12:33.417 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-22T07:12:33.418 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-22T07:12:33.726 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:33.728 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:33 smithi069 ceph-mon[36441]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:33 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/2717841535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:12:33.863 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:33 smithi063 ceph-mon[29200]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:33.863 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:33 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2717841535' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:12:35.254 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:35.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:35 smithi063 ceph-mon[29200]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:35.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:35 smithi069 ceph-mon[36441]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:36.159 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:36.160 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:36.166 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:37.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.562+0000 7f0c46bc0700 1 -- 172.21.15.63:0/2417591468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c401037b0 msgr2=0x7f0c400fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:37.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.562+0000 7f0c46bc0700 1 --2- 172.21.15.63:0/2417591468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c401037b0 0x7f0c400fce20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0c34004940 tx=0x7f0c34009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:37.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.562+0000 7f0c46bc0700 1 -- 172.21.15.63:0/2417591468 shutdown_connections 2024-03-22T07:12:37.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.562+0000 7f0c46bc0700 1 --2- 172.21.15.63:0/2417591468 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c401037b0 0x7f0c400fce20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:37.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.562+0000 7f0c46bc0700 1 --2- 172.21.15.63:0/2417591468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c40102e10 0x7f0c401031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:37.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.562+0000 7f0c46bc0700 1 -- 172.21.15.63:0/2417591468 >> 172.21.15.63:0/2417591468 conn(0x7f0c400f89a0 msgr2=0x7f0c400fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:37.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.562+0000 7f0c46bc0700 1 -- 172.21.15.63:0/2417591468 shutdown_connections 2024-03-22T07:12:37.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.562+0000 7f0c46bc0700 1 -- 172.21.15.63:0/2417591468 wait complete. 2024-03-22T07:12:37.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.563+0000 7f0c46bc0700 1 Processor -- start 2024-03-22T07:12:37.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.563+0000 7f0c46bc0700 1 -- start start 2024-03-22T07:12:37.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.563+0000 7f0c46bc0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c40102e10 0x7f0c400702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:37.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.563+0000 7f0c46bc0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c401037b0 0x7f0c40070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:37.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.563+0000 7f0c46bc0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c40070d90 con 0x7f0c401037b0 2024-03-22T07:12:37.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.563+0000 7f0c46bc0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c40070ed0 con 0x7f0c40102e10 2024-03-22T07:12:37.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.563+0000 7f0c3ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c401037b0 0x7f0c40070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:37.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.563+0000 7f0c3ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c401037b0 0x7f0c40070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59048/0 (socket says 172.21.15.63:59048) 2024-03-22T07:12:37.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.563+0000 7f0c3ffff700 1 -- 172.21.15.63:0/3394036554 learned_addr learned my addr 172.21.15.63:0/3394036554 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:37.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.564+0000 7f0c3ffff700 1 -- 172.21.15.63:0/3394036554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c40102e10 msgr2=0x7f0c400702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:37.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.564+0000 7f0c3ffff700 1 --2- 172.21.15.63:0/3394036554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c40102e10 0x7f0c400702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:37.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.564+0000 7f0c3ffff700 1 -- 172.21.15.63:0/3394036554 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c34005040 con 0x7f0c401037b0 2024-03-22T07:12:37.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.564+0000 7f0c3ffff700 1 --2- 172.21.15.63:0/3394036554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c401037b0 0x7f0c40070800 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009180 tx=0x7f0c34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:37.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.564+0000 7f0c3d7fa700 1 -- 172.21.15.63:0/3394036554 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c3400e040 con 0x7f0c401037b0 2024-03-22T07:12:37.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.564+0000 7f0c3d7fa700 1 -- 172.21.15.63:0/3394036554 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c3400bbc0 con 0x7f0c401037b0 2024-03-22T07:12:37.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.564+0000 7f0c46bc0700 1 -- 172.21.15.63:0/3394036554 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c40071120 con 0x7f0c401037b0 2024-03-22T07:12:37.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.564+0000 7f0c3d7fa700 1 -- 172.21.15.63:0/3394036554 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c3401c3f0 con 0x7f0c401037b0 2024-03-22T07:12:37.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.564+0000 7f0c46bc0700 1 -- 172.21.15.63:0/3394036554 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4011a2d0 con 0x7f0c401037b0 2024-03-22T07:12:37.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.565+0000 7f0c46bc0700 1 -- 172.21.15.63:0/3394036554 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0c4005e6c0 con 0x7f0c401037b0 2024-03-22T07:12:37.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.565+0000 7f0c3d7fa700 1 -- 172.21.15.63:0/3394036554 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0c3400bd30 con 0x7f0c401037b0 2024-03-22T07:12:37.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.566+0000 7f0c3d7fa700 1 --2- 172.21.15.63:0/3394036554 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0c280692a0 0x7f0c2806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:37.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.566+0000 7f0c3d7fa700 1 -- 172.21.15.63:0/3394036554 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0c34088c20 con 0x7f0c401037b0 2024-03-22T07:12:37.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.566+0000 7f0c4495c700 1 --2- 172.21.15.63:0/3394036554 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0c280692a0 0x7f0c2806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:37.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.566+0000 7f0c3d7fa700 1 --2- 172.21.15.63:0/3394036554 >> [v2:172.21.15.69:6800/1811566075,v1:172.21.15.69:6801/1811566075] conn(0x7f0c2806ef70 0x7f0c28071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:37.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.566+0000 7f0c3d7fa700 1 -- 172.21.15.63:0/3394036554 --> [v2:172.21.15.69:6800/1811566075,v1:172.21.15.69:6801/1811566075] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c28071a30 con 0x7f0c2806ef70 2024-03-22T07:12:37.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.566+0000 7f0c3d7fa700 1 -- 172.21.15.63:0/3394036554 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c34089000 con 0x7f0c401037b0 2024-03-22T07:12:37.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.566+0000 7f0c4495c700 1 --2- 172.21.15.63:0/3394036554 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0c280692a0 0x7f0c2806b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0c30004b70 tx=0x7f0c30007cc0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:37.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.566+0000 7f0c4515d700 1 --2- 172.21.15.63:0/3394036554 >> [v2:172.21.15.69:6800/1811566075,v1:172.21.15.69:6801/1811566075] conn(0x7f0c2806ef70 0x7f0c28071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:37.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.567+0000 7f0c4515d700 1 --2- 172.21.15.63:0/3394036554 >> [v2:172.21.15.69:6800/1811566075,v1:172.21.15.69:6801/1811566075] conn(0x7f0c2806ef70 0x7f0c28071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:37.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.568+0000 7f0c3d7fa700 1 -- 172.21.15.63:0/3394036554 <== osd.3 v2:172.21.15.69:6800/1811566075 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f0c28071a30 con 0x7f0c2806ef70 2024-03-22T07:12:37.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.590+0000 7f0c46bc0700 1 -- 172.21.15.63:0/3394036554 --> [v2:172.21.15.69:6800/1811566075,v1:172.21.15.69:6801/1811566075] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0c4005eb40 con 0x7f0c2806ef70 2024-03-22T07:12:37.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.591+0000 7f0c3d7fa700 1 -- 172.21.15.63:0/3394036554 <== osd.3 v2:172.21.15.69:6800/1811566075 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0c4005eb40 con 0x7f0c2806ef70 2024-03-22T07:12:37.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.591+0000 7f0c46bc0700 1 -- 172.21.15.63:0/3394036554 >> [v2:172.21.15.69:6800/1811566075,v1:172.21.15.69:6801/1811566075] conn(0x7f0c2806ef70 msgr2=0x7f0c28071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:37.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.591+0000 7f0c46bc0700 1 --2- 172.21.15.63:0/3394036554 >> [v2:172.21.15.69:6800/1811566075,v1:172.21.15.69:6801/1811566075] conn(0x7f0c2806ef70 0x7f0c28071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:37.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.591+0000 7f0c46bc0700 1 -- 172.21.15.63:0/3394036554 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0c280692a0 msgr2=0x7f0c2806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:37.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.591+0000 7f0c46bc0700 1 --2- 172.21.15.63:0/3394036554 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0c280692a0 0x7f0c2806b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0c30004b70 tx=0x7f0c30007cc0 comp rx=0 tx=0).stop 2024-03-22T07:12:37.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.591+0000 7f0c46bc0700 1 -- 172.21.15.63:0/3394036554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c401037b0 msgr2=0x7f0c40070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:37.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.591+0000 7f0c46bc0700 1 --2- 172.21.15.63:0/3394036554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c401037b0 0x7f0c40070800 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009180 tx=0x7f0c34009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:37.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.591+0000 7f0c46bc0700 1 -- 172.21.15.63:0/3394036554 shutdown_connections 2024-03-22T07:12:37.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.591+0000 7f0c46bc0700 1 --2- 172.21.15.63:0/3394036554 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0c280692a0 0x7f0c2806b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:37.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.591+0000 7f0c46bc0700 1 --2- 172.21.15.63:0/3394036554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0c40102e10 0x7f0c400702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:37.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.591+0000 7f0c46bc0700 1 --2- 172.21.15.63:0/3394036554 >> [v2:172.21.15.69:6800/1811566075,v1:172.21.15.69:6801/1811566075] conn(0x7f0c2806ef70 0x7f0c28071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:37.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.591+0000 7f0c46bc0700 1 --2- 172.21.15.63:0/3394036554 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0c401037b0 0x7f0c40070800 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:37.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.591+0000 7f0c46bc0700 1 -- 172.21.15.63:0/3394036554 >> 172.21.15.63:0/3394036554 conn(0x7f0c400f89a0 msgr2=0x7f0c40102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:37.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.591+0000 7f0c46bc0700 1 -- 172.21.15.63:0/3394036554 shutdown_connections 2024-03-22T07:12:37.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:37.591+0000 7f0c46bc0700 1 -- 172.21.15.63:0/3394036554 wait complete. 2024-03-22T07:12:37.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:37 smithi063 ceph-mon[29200]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:37.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:37 smithi069 ceph-mon[36441]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:38.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.207+0000 7fde5a63c700 1 -- 172.21.15.63:0/3409820883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde541037a0 msgr2=0x7fde540fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:38.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.207+0000 7fde5a63c700 1 --2- 172.21.15.63:0/3409820883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde541037a0 0x7fde540fce10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fde48004940 tx=0x7fde48009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:38.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.207+0000 7fde5a63c700 1 -- 172.21.15.63:0/3409820883 shutdown_connections 2024-03-22T07:12:38.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.207+0000 7fde5a63c700 1 --2- 172.21.15.63:0/3409820883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde541037a0 0x7fde540fce10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:38.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.207+0000 7fde5a63c700 1 --2- 172.21.15.63:0/3409820883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde54102e00 0x7fde541031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:38.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.207+0000 7fde5a63c700 1 -- 172.21.15.63:0/3409820883 >> 172.21.15.63:0/3409820883 conn(0x7fde540f89b0 msgr2=0x7fde540fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:38.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.207+0000 7fde5a63c700 1 -- 172.21.15.63:0/3409820883 shutdown_connections 2024-03-22T07:12:38.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.207+0000 7fde5a63c700 1 -- 172.21.15.63:0/3409820883 wait complete. 2024-03-22T07:12:38.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.208+0000 7fde5a63c700 1 Processor -- start 2024-03-22T07:12:38.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.208+0000 7fde5a63c700 1 -- start start 2024-03-22T07:12:38.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.208+0000 7fde5a63c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde54102e00 0x7fde54115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:38.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.208+0000 7fde5a63c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde541037a0 0x7fde541160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:38.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.208+0000 7fde5a63c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde54116700 con 0x7fde54102e00 2024-03-22T07:12:38.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.208+0000 7fde5a63c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5410fce0 con 0x7fde541037a0 2024-03-22T07:12:38.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.208+0000 7fde53fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde54102e00 0x7fde54115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:38.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.208+0000 7fde53fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde54102e00 0x7fde54115b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59078/0 (socket says 172.21.15.63:59078) 2024-03-22T07:12:38.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.208+0000 7fde53fff700 1 -- 172.21.15.63:0/820247904 learned_addr learned my addr 172.21.15.63:0/820247904 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:38.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.209+0000 7fde537fe700 1 --2- 172.21.15.63:0/820247904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde541037a0 0x7fde541160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:38.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.209+0000 7fde53fff700 1 -- 172.21.15.63:0/820247904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde541037a0 msgr2=0x7fde541160b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:38.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.209+0000 7fde53fff700 1 --2- 172.21.15.63:0/820247904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde541037a0 0x7fde541160b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:38.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.209+0000 7fde53fff700 1 -- 172.21.15.63:0/820247904 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde48005040 con 0x7fde54102e00 2024-03-22T07:12:38.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.209+0000 7fde53fff700 1 --2- 172.21.15.63:0/820247904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde54102e00 0x7fde54115b70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fde44002a50 tx=0x7fde440046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:38.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.209+0000 7fde50ff9700 1 -- 172.21.15.63:0/820247904 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde44007cc0 con 0x7fde54102e00 2024-03-22T07:12:38.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.209+0000 7fde5a63c700 1 -- 172.21.15.63:0/820247904 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde5410ffc0 con 0x7fde54102e00 2024-03-22T07:12:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.209+0000 7fde50ff9700 1 -- 172.21.15.63:0/820247904 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde4400f040 con 0x7fde54102e00 2024-03-22T07:12:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.209+0000 7fde50ff9700 1 -- 172.21.15.63:0/820247904 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde4400da30 con 0x7fde54102e00 2024-03-22T07:12:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.209+0000 7fde5a63c700 1 -- 172.21.15.63:0/820247904 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde54110550 con 0x7fde54102e00 2024-03-22T07:12:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.210+0000 7fde5a63c700 1 -- 172.21.15.63:0/820247904 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fde54110af0 con 0x7fde54102e00 2024-03-22T07:12:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.211+0000 7fde50ff9700 1 -- 172.21.15.63:0/820247904 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fde4400dbb0 con 0x7fde54102e00 2024-03-22T07:12:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.211+0000 7fde50ff9700 1 --2- 172.21.15.63:0/820247904 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fde3c0692a0 0x7fde3c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.211+0000 7fde50ff9700 1 -- 172.21.15.63:0/820247904 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fde44086be0 con 0x7fde54102e00 2024-03-22T07:12:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.211+0000 7fde537fe700 1 --2- 172.21.15.63:0/820247904 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fde3c0692a0 0x7fde3c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.211+0000 7fde50ff9700 1 --2- 172.21.15.63:0/820247904 >> [v2:172.21.15.69:6808/656541591,v1:172.21.15.69:6809/656541591] conn(0x7fde3c06ef70 0x7fde3c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.211+0000 7fde50ff9700 1 -- 172.21.15.63:0/820247904 --> [v2:172.21.15.69:6808/656541591,v1:172.21.15.69:6809/656541591] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde3c071a30 con 0x7fde3c06ef70 2024-03-22T07:12:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.211+0000 7fde50ff9700 1 -- 172.21.15.63:0/820247904 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde4408c1e0 con 0x7fde54102e00 2024-03-22T07:12:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.211+0000 7fde58bd9700 1 --2- 172.21.15.63:0/820247904 >> [v2:172.21.15.69:6808/656541591,v1:172.21.15.69:6809/656541591] conn(0x7fde3c06ef70 0x7fde3c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.211+0000 7fde537fe700 1 --2- 172.21.15.63:0/820247904 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fde3c0692a0 0x7fde3c06b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fde48004890 tx=0x7fde48009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:38.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.212+0000 7fde58bd9700 1 --2- 172.21.15.63:0/820247904 >> [v2:172.21.15.69:6808/656541591,v1:172.21.15.69:6809/656541591] conn(0x7fde3c06ef70 0x7fde3c071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:38.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.212+0000 7fde50ff9700 1 -- 172.21.15.63:0/820247904 <== osd.4 v2:172.21.15.69:6808/656541591 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fde3c071a30 con 0x7fde3c06ef70 2024-03-22T07:12:38.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.227+0000 7fde5a63c700 1 -- 172.21.15.63:0/820247904 --> [v2:172.21.15.69:6808/656541591,v1:172.21.15.69:6809/656541591] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fde54110150 con 0x7fde3c06ef70 2024-03-22T07:12:38.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.228+0000 7fde50ff9700 1 -- 172.21.15.63:0/820247904 <== osd.4 v2:172.21.15.69:6808/656541591 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fde54110150 con 0x7fde3c06ef70 2024-03-22T07:12:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.228+0000 7fde5a63c700 1 -- 172.21.15.63:0/820247904 >> [v2:172.21.15.69:6808/656541591,v1:172.21.15.69:6809/656541591] conn(0x7fde3c06ef70 msgr2=0x7fde3c071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.228+0000 7fde5a63c700 1 --2- 172.21.15.63:0/820247904 >> [v2:172.21.15.69:6808/656541591,v1:172.21.15.69:6809/656541591] conn(0x7fde3c06ef70 0x7fde3c071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.228+0000 7fde5a63c700 1 -- 172.21.15.63:0/820247904 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fde3c0692a0 msgr2=0x7fde3c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.228+0000 7fde5a63c700 1 --2- 172.21.15.63:0/820247904 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fde3c0692a0 0x7fde3c06b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fde48004890 tx=0x7fde48009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.228+0000 7fde5a63c700 1 -- 172.21.15.63:0/820247904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde54102e00 msgr2=0x7fde54115b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.228+0000 7fde5a63c700 1 --2- 172.21.15.63:0/820247904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde54102e00 0x7fde54115b70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fde44002a50 tx=0x7fde440046b0 comp rx=0 tx=0).stop 2024-03-22T07:12:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.228+0000 7fde5a63c700 1 -- 172.21.15.63:0/820247904 shutdown_connections 2024-03-22T07:12:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.228+0000 7fde5a63c700 1 --2- 172.21.15.63:0/820247904 >> [v2:172.21.15.69:6808/656541591,v1:172.21.15.69:6809/656541591] conn(0x7fde3c06ef70 0x7fde3c071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.228+0000 7fde5a63c700 1 --2- 172.21.15.63:0/820247904 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fde3c0692a0 0x7fde3c06b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.228+0000 7fde5a63c700 1 --2- 172.21.15.63:0/820247904 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde54102e00 0x7fde54115b70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.228+0000 7fde5a63c700 1 --2- 172.21.15.63:0/820247904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde541037a0 0x7fde541160b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.228+0000 7fde5a63c700 1 -- 172.21.15.63:0/820247904 >> 172.21.15.63:0/820247904 conn(0x7fde540f89b0 msgr2=0x7fde541021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:38.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.228+0000 7fde5a63c700 1 -- 172.21.15.63:0/820247904 shutdown_connections 2024-03-22T07:12:38.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:38.228+0000 7fde5a63c700 1 -- 172.21.15.63:0/820247904 wait complete. 2024-03-22T07:12:38.946 INFO:teuthology.orchestra.run.smithi063.stdout:150323855372 2024-03-22T07:12:38.946 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-22T07:12:39.697 INFO:teuthology.orchestra.run.smithi063.stdout:115964117011 2024-03-22T07:12:39.697 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-22T07:12:39.721 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:39 smithi063 ceph-mon[29200]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:39.828 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:39 smithi069 ceph-mon[36441]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:41.599 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:41 smithi063 ceph-mon[29200]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:41.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:41 smithi069 ceph-mon[36441]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:42.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.233+0000 7f548aaed700 1 -- 172.21.15.63:0/1044425891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5484102710 msgr2=0x7f54840fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:42.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.233+0000 7f548aaed700 1 --2- 172.21.15.63:0/1044425891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5484102710 0x7f54840fbd80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5470004940 tx=0x7f5470009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:42.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.233+0000 7f548aaed700 1 -- 172.21.15.63:0/1044425891 shutdown_connections 2024-03-22T07:12:42.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.233+0000 7f548aaed700 1 --2- 172.21.15.63:0/1044425891 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5484102710 0x7f54840fbd80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:42.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.233+0000 7f548aaed700 1 --2- 172.21.15.63:0/1044425891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5484101d70 0x7f5484102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:42.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.233+0000 7f548aaed700 1 -- 172.21.15.63:0/1044425891 >> 172.21.15.63:0/1044425891 conn(0x7f5484072bc0 msgr2=0x7f5484072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:42.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.233+0000 7f548aaed700 1 -- 172.21.15.63:0/1044425891 shutdown_connections 2024-03-22T07:12:42.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.233+0000 7f548aaed700 1 -- 172.21.15.63:0/1044425891 wait complete. 2024-03-22T07:12:42.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.234+0000 7f548aaed700 1 Processor -- start 2024-03-22T07:12:42.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.234+0000 7f548aaed700 1 -- start start 2024-03-22T07:12:42.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.234+0000 7f548aaed700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5484101d70 0x7f5484112a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:42.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.234+0000 7f548aaed700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5484102710 0x7f5484112fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:42.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.234+0000 7f548aaed700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54841136b0 con 0x7f5484101d70 2024-03-22T07:12:42.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.234+0000 7f548aaed700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548410ca10 con 0x7f5484102710 2024-03-22T07:12:42.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.235+0000 7f5488889700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5484101d70 0x7f5484112a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:42.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.235+0000 7f5488889700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5484101d70 0x7f5484112a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59098/0 (socket says 172.21.15.63:59098) 2024-03-22T07:12:42.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.235+0000 7f5488889700 1 -- 172.21.15.63:0/1792951798 learned_addr learned my addr 172.21.15.63:0/1792951798 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:42.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.235+0000 7f5483fff700 1 --2- 172.21.15.63:0/1792951798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5484102710 0x7f5484112fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:42.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.235+0000 7f5488889700 1 -- 172.21.15.63:0/1792951798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5484102710 msgr2=0x7f5484112fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:42.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.235+0000 7f5488889700 1 --2- 172.21.15.63:0/1792951798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5484102710 0x7f5484112fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:42.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.235+0000 7f5488889700 1 -- 172.21.15.63:0/1792951798 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5470005040 con 0x7f5484101d70 2024-03-22T07:12:42.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.235+0000 7f5488889700 1 --2- 172.21.15.63:0/1792951798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5484101d70 0x7f5484112a90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f547800b790 tx=0x7f5478009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:42.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.235+0000 7f54817fa700 1 -- 172.21.15.63:0/1792951798 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f547800f6f0 con 0x7f5484101d70 2024-03-22T07:12:42.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.235+0000 7f54817fa700 1 -- 172.21.15.63:0/1792951798 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f547800fd20 con 0x7f5484101d70 2024-03-22T07:12:42.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.235+0000 7f548aaed700 1 -- 172.21.15.63:0/1792951798 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548410ccf0 con 0x7f5484101d70 2024-03-22T07:12:42.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.235+0000 7f54817fa700 1 -- 172.21.15.63:0/1792951798 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54780103a0 con 0x7f5484101d70 2024-03-22T07:12:42.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.235+0000 7f548aaed700 1 -- 172.21.15.63:0/1792951798 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548410d280 con 0x7f5484101d70 2024-03-22T07:12:42.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.236+0000 7f548aaed700 1 -- 172.21.15.63:0/1792951798 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f548410d830 con 0x7f5484101d70 2024-03-22T07:12:42.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.237+0000 7f54817fa700 1 -- 172.21.15.63:0/1792951798 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5478011400 con 0x7f5484101d70 2024-03-22T07:12:42.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.237+0000 7f54817fa700 1 --2- 172.21.15.63:0/1792951798 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f54740692a0 0x7f547406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:42.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.237+0000 7f54817fa700 1 -- 172.21.15.63:0/1792951798 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5478087690 con 0x7f5484101d70 2024-03-22T07:12:42.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.237+0000 7f5483fff700 1 --2- 172.21.15.63:0/1792951798 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f54740692a0 0x7f547406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:42.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.237+0000 7f54817fa700 1 --2- 172.21.15.63:0/1792951798 >> [v2:172.21.15.63:6802/3791141232,v1:172.21.15.63:6803/3791141232] conn(0x7f547406ef70 0x7f5474071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:42.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.237+0000 7f54817fa700 1 -- 172.21.15.63:0/1792951798 --> [v2:172.21.15.63:6802/3791141232,v1:172.21.15.63:6803/3791141232] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5474071a30 con 0x7f547406ef70 2024-03-22T07:12:42.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.237+0000 7f54817fa700 1 -- 172.21.15.63:0/1792951798 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5478087a70 con 0x7f5484101d70 2024-03-22T07:12:42.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.237+0000 7f548908a700 1 --2- 172.21.15.63:0/1792951798 >> [v2:172.21.15.63:6802/3791141232,v1:172.21.15.63:6803/3791141232] conn(0x7f547406ef70 0x7f5474071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:42.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.238+0000 7f5483fff700 1 --2- 172.21.15.63:0/1792951798 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f54740692a0 0x7f547406b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5470004890 tx=0x7f5470009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:42.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.238+0000 7f548908a700 1 --2- 172.21.15.63:0/1792951798 >> [v2:172.21.15.63:6802/3791141232,v1:172.21.15.63:6803/3791141232] conn(0x7f547406ef70 0x7f5474071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:42.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.238+0000 7f54817fa700 1 -- 172.21.15.63:0/1792951798 <== osd.0 v2:172.21.15.63:6802/3791141232 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f5474071a30 con 0x7f547406ef70 2024-03-22T07:12:42.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.255+0000 7f548aaed700 1 -- 172.21.15.63:0/1792951798 --> [v2:172.21.15.63:6802/3791141232,v1:172.21.15.63:6803/3791141232] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f548410ce80 con 0x7f547406ef70 2024-03-22T07:12:42.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.255+0000 7f54817fa700 1 -- 172.21.15.63:0/1792951798 <== osd.0 v2:172.21.15.63:6802/3791141232 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f548410ce80 con 0x7f547406ef70 2024-03-22T07:12:42.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.255+0000 7f548aaed700 1 -- 172.21.15.63:0/1792951798 >> [v2:172.21.15.63:6802/3791141232,v1:172.21.15.63:6803/3791141232] conn(0x7f547406ef70 msgr2=0x7f5474071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:42.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.255+0000 7f548aaed700 1 --2- 172.21.15.63:0/1792951798 >> [v2:172.21.15.63:6802/3791141232,v1:172.21.15.63:6803/3791141232] conn(0x7f547406ef70 0x7f5474071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:42.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.255+0000 7f548aaed700 1 -- 172.21.15.63:0/1792951798 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f54740692a0 msgr2=0x7f547406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:42.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.255+0000 7f548aaed700 1 --2- 172.21.15.63:0/1792951798 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f54740692a0 0x7f547406b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5470004890 tx=0x7f5470009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:42.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.255+0000 7f548aaed700 1 -- 172.21.15.63:0/1792951798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5484101d70 msgr2=0x7f5484112a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:42.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.255+0000 7f548aaed700 1 --2- 172.21.15.63:0/1792951798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5484101d70 0x7f5484112a90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f547800b790 tx=0x7f5478009250 comp rx=0 tx=0).stop 2024-03-22T07:12:42.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.256+0000 7f548aaed700 1 -- 172.21.15.63:0/1792951798 shutdown_connections 2024-03-22T07:12:42.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.256+0000 7f548aaed700 1 --2- 172.21.15.63:0/1792951798 >> [v2:172.21.15.63:6802/3791141232,v1:172.21.15.63:6803/3791141232] conn(0x7f547406ef70 0x7f5474071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:42.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.256+0000 7f548aaed700 1 --2- 172.21.15.63:0/1792951798 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f54740692a0 0x7f547406b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:42.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.256+0000 7f548aaed700 1 --2- 172.21.15.63:0/1792951798 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5484101d70 0x7f5484112a90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:42.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.256+0000 7f548aaed700 1 --2- 172.21.15.63:0/1792951798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5484102710 0x7f5484112fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:42.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.256+0000 7f548aaed700 1 -- 172.21.15.63:0/1792951798 >> 172.21.15.63:0/1792951798 conn(0x7f5484072bc0 msgr2=0x7f54840faf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:42.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.256+0000 7f548aaed700 1 -- 172.21.15.63:0/1792951798 shutdown_connections 2024-03-22T07:12:42.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:40.256+0000 7f548aaed700 1 -- 172.21.15.63:0/1792951798 wait complete. 2024-03-22T07:12:43.123 INFO:teuthology.orchestra.run.smithi063.stdout:38654705700 2024-03-22T07:12:43.123 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-22T07:12:43.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:43 smithi063 ceph-mon[29200]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:43.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.632+0000 7f6e51377700 1 -- 172.21.15.63:0/3469770645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e4c101d10 msgr2=0x7f6e4c1020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:43.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.632+0000 7f6e51377700 1 --2- 172.21.15.63:0/3469770645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e4c101d10 0x7f6e4c1020e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c004940 tx=0x7f6e3c009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:43.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.632+0000 7f6e51377700 1 -- 172.21.15.63:0/3469770645 shutdown_connections 2024-03-22T07:12:43.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.632+0000 7f6e51377700 1 --2- 172.21.15.63:0/3469770645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e4c1026b0 0x7f6e4c0fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:43.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.632+0000 7f6e51377700 1 --2- 172.21.15.63:0/3469770645 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e4c101d10 0x7f6e4c1020e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:43.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.632+0000 7f6e51377700 1 -- 172.21.15.63:0/3469770645 >> 172.21.15.63:0/3469770645 conn(0x7f6e4c0f7880 msgr2=0x7f6e4c0f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:43.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.633+0000 7f6e51377700 1 -- 172.21.15.63:0/3469770645 shutdown_connections 2024-03-22T07:12:43.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.633+0000 7f6e51377700 1 -- 172.21.15.63:0/3469770645 wait complete. 2024-03-22T07:12:43.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.633+0000 7f6e51377700 1 Processor -- start 2024-03-22T07:12:43.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.633+0000 7f6e51377700 1 -- start start 2024-03-22T07:12:43.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.634+0000 7f6e51377700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e4c101d10 0x7f6e4c10efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:43.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.634+0000 7f6e51377700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e4c1026b0 0x7f6e4c10f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:43.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.634+0000 7f6e51377700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c10fa50 con 0x7f6e4c101d10 2024-03-22T07:12:43.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.634+0000 7f6e51377700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c10fb90 con 0x7f6e4c1026b0 2024-03-22T07:12:43.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.634+0000 7f6e4affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e4c101d10 0x7f6e4c10efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:43.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.634+0000 7f6e4affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e4c101d10 0x7f6e4c10efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59118/0 (socket says 172.21.15.63:59118) 2024-03-22T07:12:43.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.634+0000 7f6e4affd700 1 -- 172.21.15.63:0/1001188517 learned_addr learned my addr 172.21.15.63:0/1001188517 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:43.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.634+0000 7f6e4a7fc700 1 --2- 172.21.15.63:0/1001188517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e4c1026b0 0x7f6e4c10f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:43.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.634+0000 7f6e4affd700 1 -- 172.21.15.63:0/1001188517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e4c1026b0 msgr2=0x7f6e4c10f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:43.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.634+0000 7f6e4affd700 1 --2- 172.21.15.63:0/1001188517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e4c1026b0 0x7f6e4c10f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:43.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.634+0000 7f6e4affd700 1 -- 172.21.15.63:0/1001188517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3c005040 con 0x7f6e4c101d10 2024-03-22T07:12:43.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.634+0000 7f6e4affd700 1 --2- 172.21.15.63:0/1001188517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e4c101d10 0x7f6e4c10efd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009150 tx=0x7f6e3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:43.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.634+0000 7f6e337fe700 1 -- 172.21.15.63:0/1001188517 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e3c00e040 con 0x7f6e4c101d10 2024-03-22T07:12:43.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.634+0000 7f6e337fe700 1 -- 172.21.15.63:0/1001188517 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e3c013420 con 0x7f6e4c101d10 2024-03-22T07:12:43.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.634+0000 7f6e51377700 1 -- 172.21.15.63:0/1001188517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4c117ee0 con 0x7f6e4c101d10 2024-03-22T07:12:43.645 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.634+0000 7f6e337fe700 1 -- 172.21.15.63:0/1001188517 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e3c01c630 con 0x7f6e4c101d10 2024-03-22T07:12:43.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.634+0000 7f6e51377700 1 -- 172.21.15.63:0/1001188517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4c118410 con 0x7f6e4c101d10 2024-03-22T07:12:43.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.635+0000 7f6e51377700 1 -- 172.21.15.63:0/1001188517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6e4c0fd460 con 0x7f6e4c101d10 2024-03-22T07:12:43.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.636+0000 7f6e337fe700 1 -- 172.21.15.63:0/1001188517 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6e3c013590 con 0x7f6e4c101d10 2024-03-22T07:12:43.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.636+0000 7f6e337fe700 1 --2- 172.21.15.63:0/1001188517 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f6e340692b0 0x7f6e3406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:43.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.636+0000 7f6e337fe700 1 -- 172.21.15.63:0/1001188517 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6e3c088aa0 con 0x7f6e4c101d10 2024-03-22T07:12:43.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.636+0000 7f6e4a7fc700 1 --2- 172.21.15.63:0/1001188517 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f6e340692b0 0x7f6e3406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:43.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.636+0000 7f6e337fe700 1 --2- 172.21.15.63:0/1001188517 >> [v2:172.21.15.63:6810/1043983969,v1:172.21.15.63:6811/1043983969] conn(0x7f6e3406ef80 0x7f6e34071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:43.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.636+0000 7f6e337fe700 1 -- 172.21.15.63:0/1001188517 --> [v2:172.21.15.63:6810/1043983969,v1:172.21.15.63:6811/1043983969] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e34071a40 con 0x7f6e3406ef80 2024-03-22T07:12:43.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.636+0000 7f6e337fe700 1 -- 172.21.15.63:0/1001188517 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e3c088e80 con 0x7f6e4c101d10 2024-03-22T07:12:43.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.636+0000 7f6e4b7fe700 1 --2- 172.21.15.63:0/1001188517 >> [v2:172.21.15.63:6810/1043983969,v1:172.21.15.63:6811/1043983969] conn(0x7f6e3406ef80 0x7f6e34071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:43.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.637+0000 7f6e4a7fc700 1 --2- 172.21.15.63:0/1001188517 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f6e340692b0 0x7f6e3406b760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6e40006fd0 tx=0x7f6e40008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:43.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.637+0000 7f6e4b7fe700 1 --2- 172.21.15.63:0/1001188517 >> [v2:172.21.15.63:6810/1043983969,v1:172.21.15.63:6811/1043983969] conn(0x7f6e3406ef80 0x7f6e34071390 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:43.646 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.637+0000 7f6e337fe700 1 -- 172.21.15.63:0/1001188517 <== osd.1 v2:172.21.15.63:6810/1043983969 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f6e34071a40 con 0x7f6e3406ef80 2024-03-22T07:12:43.650 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:43.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.656+0000 7f6e51377700 1 -- 172.21.15.63:0/1001188517 --> [v2:172.21.15.63:6810/1043983969,v1:172.21.15.63:6811/1043983969] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6e4c05f5f0 con 0x7f6e3406ef80 2024-03-22T07:12:43.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.656+0000 7f6e337fe700 1 -- 172.21.15.63:0/1001188517 <== osd.1 v2:172.21.15.63:6810/1043983969 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6e4c05f5f0 con 0x7f6e3406ef80 2024-03-22T07:12:43.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.657+0000 7f6e51377700 1 -- 172.21.15.63:0/1001188517 >> [v2:172.21.15.63:6810/1043983969,v1:172.21.15.63:6811/1043983969] conn(0x7f6e3406ef80 msgr2=0x7f6e34071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:43.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.657+0000 7f6e51377700 1 --2- 172.21.15.63:0/1001188517 >> [v2:172.21.15.63:6810/1043983969,v1:172.21.15.63:6811/1043983969] conn(0x7f6e3406ef80 0x7f6e34071390 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:43.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.657+0000 7f6e51377700 1 -- 172.21.15.63:0/1001188517 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f6e340692b0 msgr2=0x7f6e3406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:43.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.657+0000 7f6e51377700 1 --2- 172.21.15.63:0/1001188517 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f6e340692b0 0x7f6e3406b760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6e40006fd0 tx=0x7f6e40008040 comp rx=0 tx=0).stop 2024-03-22T07:12:43.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.657+0000 7f6e51377700 1 -- 172.21.15.63:0/1001188517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e4c101d10 msgr2=0x7f6e4c10efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:43.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.657+0000 7f6e51377700 1 --2- 172.21.15.63:0/1001188517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e4c101d10 0x7f6e4c10efd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009150 tx=0x7f6e3c009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:43.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.657+0000 7f6e51377700 1 -- 172.21.15.63:0/1001188517 shutdown_connections 2024-03-22T07:12:43.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.657+0000 7f6e51377700 1 --2- 172.21.15.63:0/1001188517 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f6e340692b0 0x7f6e3406b760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:43.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.657+0000 7f6e51377700 1 --2- 172.21.15.63:0/1001188517 >> [v2:172.21.15.63:6810/1043983969,v1:172.21.15.63:6811/1043983969] conn(0x7f6e3406ef80 0x7f6e34071390 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:43.657 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.657+0000 7f6e51377700 1 --2- 172.21.15.63:0/1001188517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6e4c101d10 0x7f6e4c10efd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:43.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.657+0000 7f6e51377700 1 --2- 172.21.15.63:0/1001188517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e4c1026b0 0x7f6e4c10f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:43.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.657+0000 7f6e51377700 1 -- 172.21.15.63:0/1001188517 >> 172.21.15.63:0/1001188517 conn(0x7f6e4c0f7880 msgr2=0x7f6e4c0f8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:43.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.657+0000 7f6e51377700 1 -- 172.21.15.63:0/1001188517 shutdown_connections 2024-03-22T07:12:43.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:43.657+0000 7f6e51377700 1 -- 172.21.15.63:0/1001188517 wait complete. 2024-03-22T07:12:43.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:43 smithi069 ceph-mon[36441]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:44.086 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:44.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:44 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:12:44.791 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:12:45.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.207+0000 7f60565e5700 1 -- 172.21.15.63:0/4253576780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60500fe830 msgr2=0x7f6050106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:45.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.207+0000 7f60565e5700 1 --2- 172.21.15.63:0/4253576780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60500fe830 0x7f6050106f00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6044004940 tx=0x7f6044009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:45.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.208+0000 7f60565e5700 1 -- 172.21.15.63:0/4253576780 shutdown_connections 2024-03-22T07:12:45.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.208+0000 7f60565e5700 1 --2- 172.21.15.63:0/4253576780 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60500fe830 0x7f6050106f00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:45.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.208+0000 7f60565e5700 1 --2- 172.21.15.63:0/4253576780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60500fde90 0x7f60500fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:45.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.208+0000 7f60565e5700 1 -- 172.21.15.63:0/4253576780 >> 172.21.15.63:0/4253576780 conn(0x7f6050072bc0 msgr2=0x7f6050072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:45.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.208+0000 7f60565e5700 1 -- 172.21.15.63:0/4253576780 shutdown_connections 2024-03-22T07:12:45.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.208+0000 7f60565e5700 1 -- 172.21.15.63:0/4253576780 wait complete. 2024-03-22T07:12:45.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.208+0000 7f60565e5700 1 Processor -- start 2024-03-22T07:12:45.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.208+0000 7f60565e5700 1 -- start start 2024-03-22T07:12:45.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.208+0000 7f60565e5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60500fde90 0x7f6050113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:45.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.208+0000 7f60565e5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60500fe830 0x7f60501140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:45.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.208+0000 7f60565e5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60501147b0 con 0x7f60500fde90 2024-03-22T07:12:45.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.208+0000 7f60565e5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605010db10 con 0x7f60500fe830 2024-03-22T07:12:45.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.209+0000 7f604ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60500fde90 0x7f6050113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:45.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.209+0000 7f604ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60500fde90 0x7f6050113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41178/0 (socket says 172.21.15.63:41178) 2024-03-22T07:12:45.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.209+0000 7f604ffff700 1 -- 172.21.15.63:0/4293103572 learned_addr learned my addr 172.21.15.63:0/4293103572 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:45.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.209+0000 7f604f7fe700 1 --2- 172.21.15.63:0/4293103572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60500fe830 0x7f60501140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:45.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.209+0000 7f604ffff700 1 -- 172.21.15.63:0/4293103572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60500fe830 msgr2=0x7f60501140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:45.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.209+0000 7f604ffff700 1 --2- 172.21.15.63:0/4293103572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60500fe830 0x7f60501140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:45.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.209+0000 7f604ffff700 1 -- 172.21.15.63:0/4293103572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6044005040 con 0x7f60500fde90 2024-03-22T07:12:45.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.209+0000 7f604ffff700 1 --2- 172.21.15.63:0/4293103572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60500fde90 0x7f6050113b90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f604000b790 tx=0x7f6040009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:45.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.209+0000 7f604cff9700 1 -- 172.21.15.63:0/4293103572 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f604000f770 con 0x7f60500fde90 2024-03-22T07:12:45.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.209+0000 7f604cff9700 1 -- 172.21.15.63:0/4293103572 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f604000fda0 con 0x7f60500fde90 2024-03-22T07:12:45.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.209+0000 7f60565e5700 1 -- 172.21.15.63:0/4293103572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605010ddf0 con 0x7f60500fde90 2024-03-22T07:12:45.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.209+0000 7f604cff9700 1 -- 172.21.15.63:0/4293103572 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f604000d310 con 0x7f60500fde90 2024-03-22T07:12:45.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.209+0000 7f60565e5700 1 -- 172.21.15.63:0/4293103572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605010e330 con 0x7f60500fde90 2024-03-22T07:12:45.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.210+0000 7f60565e5700 1 -- 172.21.15.63:0/4293103572 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f605010e8b0 con 0x7f60500fde90 2024-03-22T07:12:45.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.211+0000 7f604cff9700 1 -- 172.21.15.63:0/4293103572 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f6040010420 con 0x7f60500fde90 2024-03-22T07:12:45.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.211+0000 7f604cff9700 1 --2- 172.21.15.63:0/4293103572 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f60380692a0 0x7f603806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:45.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.211+0000 7f604cff9700 1 -- 172.21.15.63:0/4293103572 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6040086e60 con 0x7f60500fde90 2024-03-22T07:12:45.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.211+0000 7f604f7fe700 1 --2- 172.21.15.63:0/4293103572 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f60380692a0 0x7f603806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:45.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.211+0000 7f604cff9700 1 --2- 172.21.15.63:0/4293103572 >> [v2:172.21.15.63:6818/3603088351,v1:172.21.15.63:6819/3603088351] conn(0x7f603806ef70 0x7f6038071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:45.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.211+0000 7f604cff9700 1 -- 172.21.15.63:0/4293103572 --> [v2:172.21.15.63:6818/3603088351,v1:172.21.15.63:6819/3603088351] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6038071a30 con 0x7f603806ef70 2024-03-22T07:12:45.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.211+0000 7f604cff9700 1 -- 172.21.15.63:0/4293103572 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f604008c460 con 0x7f60500fde90 2024-03-22T07:12:45.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.212+0000 7f6054b82700 1 --2- 172.21.15.63:0/4293103572 >> [v2:172.21.15.63:6818/3603088351,v1:172.21.15.63:6819/3603088351] conn(0x7f603806ef70 0x7f6038071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:45.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.212+0000 7f604f7fe700 1 --2- 172.21.15.63:0/4293103572 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f60380692a0 0x7f603806b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f605010ecd0 tx=0x7f6044009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:45.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.212+0000 7f6054b82700 1 --2- 172.21.15.63:0/4293103572 >> [v2:172.21.15.63:6818/3603088351,v1:172.21.15.63:6819/3603088351] conn(0x7f603806ef70 0x7f6038071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:45.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.212+0000 7f604cff9700 1 -- 172.21.15.63:0/4293103572 <== osd.2 v2:172.21.15.63:6818/3603088351 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f6038071a30 con 0x7f603806ef70 2024-03-22T07:12:45.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.227+0000 7f60565e5700 1 -- 172.21.15.63:0/4293103572 --> [v2:172.21.15.63:6818/3603088351,v1:172.21.15.63:6819/3603088351] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6050102410 con 0x7f603806ef70 2024-03-22T07:12:45.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.228+0000 7f604cff9700 1 -- 172.21.15.63:0/4293103572 <== osd.2 v2:172.21.15.63:6818/3603088351 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6050102410 con 0x7f603806ef70 2024-03-22T07:12:45.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.228+0000 7f60565e5700 1 -- 172.21.15.63:0/4293103572 >> [v2:172.21.15.63:6818/3603088351,v1:172.21.15.63:6819/3603088351] conn(0x7f603806ef70 msgr2=0x7f6038071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:45.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.228+0000 7f60565e5700 1 --2- 172.21.15.63:0/4293103572 >> [v2:172.21.15.63:6818/3603088351,v1:172.21.15.63:6819/3603088351] conn(0x7f603806ef70 0x7f6038071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:45.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.228+0000 7f60565e5700 1 -- 172.21.15.63:0/4293103572 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f60380692a0 msgr2=0x7f603806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:45.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.228+0000 7f60565e5700 1 --2- 172.21.15.63:0/4293103572 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f60380692a0 0x7f603806b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f605010ecd0 tx=0x7f6044009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:45.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.228+0000 7f60565e5700 1 -- 172.21.15.63:0/4293103572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60500fde90 msgr2=0x7f6050113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:45.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.228+0000 7f60565e5700 1 --2- 172.21.15.63:0/4293103572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60500fde90 0x7f6050113b90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f604000b790 tx=0x7f6040009250 comp rx=0 tx=0).stop 2024-03-22T07:12:45.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.228+0000 7f60565e5700 1 -- 172.21.15.63:0/4293103572 shutdown_connections 2024-03-22T07:12:45.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.228+0000 7f60565e5700 1 --2- 172.21.15.63:0/4293103572 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f60380692a0 0x7f603806b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:45.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.228+0000 7f60565e5700 1 --2- 172.21.15.63:0/4293103572 >> [v2:172.21.15.63:6818/3603088351,v1:172.21.15.63:6819/3603088351] conn(0x7f603806ef70 0x7f6038071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:45.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.228+0000 7f60565e5700 1 --2- 172.21.15.63:0/4293103572 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60500fde90 0x7f6050113b90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:45.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.228+0000 7f60565e5700 1 --2- 172.21.15.63:0/4293103572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60500fe830 0x7f60501140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:45.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.228+0000 7f60565e5700 1 -- 172.21.15.63:0/4293103572 >> 172.21.15.63:0/4293103572 conn(0x7f6050072bc0 msgr2=0x7f6050103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:45.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.228+0000 7f60565e5700 1 -- 172.21.15.63:0/4293103572 shutdown_connections 2024-03-22T07:12:45.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.228+0000 7f60565e5700 1 -- 172.21.15.63:0/4293103572 wait complete. 2024-03-22T07:12:45.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.400+0000 7f53cf515700 1 -- 172.21.15.63:0/2660897735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53c8102710 msgr2=0x7f53c80fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:45.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.400+0000 7f53cf515700 1 --2- 172.21.15.63:0/2660897735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53c8102710 0x7f53c80fbd80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f53b8004940 tx=0x7f53b8009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:45.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.400+0000 7f53cf515700 1 -- 172.21.15.63:0/2660897735 shutdown_connections 2024-03-22T07:12:45.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.400+0000 7f53cf515700 1 --2- 172.21.15.63:0/2660897735 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53c8102710 0x7f53c80fbd80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:45.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.400+0000 7f53cf515700 1 --2- 172.21.15.63:0/2660897735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53c8101d70 0x7f53c8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:45.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.400+0000 7f53cf515700 1 -- 172.21.15.63:0/2660897735 >> 172.21.15.63:0/2660897735 conn(0x7f53c8072bc0 msgr2=0x7f53c8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:45.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.400+0000 7f53cf515700 1 -- 172.21.15.63:0/2660897735 shutdown_connections 2024-03-22T07:12:45.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.400+0000 7f53cf515700 1 -- 172.21.15.63:0/2660897735 wait complete. 2024-03-22T07:12:45.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.400+0000 7f53cf515700 1 Processor -- start 2024-03-22T07:12:45.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.400+0000 7f53cf515700 1 -- start start 2024-03-22T07:12:45.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.401+0000 7f53cf515700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53c8101d70 0x7f53c81129e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:45.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.401+0000 7f53cf515700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53c8102710 0x7f53c8112f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:45.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.401+0000 7f53cf515700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c8113570 con 0x7f53c8101d70 2024-03-22T07:12:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.401+0000 7f53cf515700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c810c960 con 0x7f53c8102710 2024-03-22T07:12:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.401+0000 7f53cd2b1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53c8101d70 0x7f53c81129e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.401+0000 7f53cd2b1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53c8101d70 0x7f53c81129e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41198/0 (socket says 172.21.15.63:41198) 2024-03-22T07:12:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.401+0000 7f53cd2b1700 1 -- 172.21.15.63:0/104430652 learned_addr learned my addr 172.21.15.63:0/104430652 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.401+0000 7f53ccab0700 1 --2- 172.21.15.63:0/104430652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53c8102710 0x7f53c8112f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.401+0000 7f53cd2b1700 1 -- 172.21.15.63:0/104430652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53c8102710 msgr2=0x7f53c8112f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.401+0000 7f53cd2b1700 1 --2- 172.21.15.63:0/104430652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53c8102710 0x7f53c8112f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.401+0000 7f53cd2b1700 1 -- 172.21.15.63:0/104430652 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b8005040 con 0x7f53c8101d70 2024-03-22T07:12:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.401+0000 7f53cd2b1700 1 --2- 172.21.15.63:0/104430652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53c8101d70 0x7f53c81129e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f53c400fd10 tx=0x7f53c40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.401+0000 7f53bdffb700 1 -- 172.21.15.63:0/104430652 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53c400dcf0 con 0x7f53c8101d70 2024-03-22T07:12:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.401+0000 7f53bdffb700 1 -- 172.21.15.63:0/104430652 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53c400de50 con 0x7f53c8101d70 2024-03-22T07:12:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.401+0000 7f53cf515700 1 -- 172.21.15.63:0/104430652 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c810cbe0 con 0x7f53c8101d70 2024-03-22T07:12:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.401+0000 7f53bdffb700 1 -- 172.21.15.63:0/104430652 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53c40126a0 con 0x7f53c8101d70 2024-03-22T07:12:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.402+0000 7f53cf515700 1 -- 172.21.15.63:0/104430652 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c810d170 con 0x7f53c8101d70 2024-03-22T07:12:45.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.402+0000 7f53cf515700 1 -- 172.21.15.63:0/104430652 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f53c810d650 con 0x7f53c8101d70 2024-03-22T07:12:45.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.403+0000 7f53bdffb700 1 -- 172.21.15.63:0/104430652 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f53c4007ab0 con 0x7f53c8101d70 2024-03-22T07:12:45.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.403+0000 7f53bdffb700 1 --2- 172.21.15.63:0/104430652 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f53b40692f0 0x7f53b406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:45.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.404+0000 7f53bdffb700 1 -- 172.21.15.63:0/104430652 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f53c4089460 con 0x7f53c8101d70 2024-03-22T07:12:45.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.404+0000 7f53ccab0700 1 --2- 172.21.15.63:0/104430652 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f53b40692f0 0x7f53b406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:45.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.404+0000 7f53bdffb700 1 --2- 172.21.15.63:0/104430652 >> [v2:172.21.15.69:6816/2157641649,v1:172.21.15.69:6817/2157641649] conn(0x7f53b406efc0 0x7f53b40713d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:45.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.404+0000 7f53bdffb700 1 -- 172.21.15.63:0/104430652 --> [v2:172.21.15.69:6816/2157641649,v1:172.21.15.69:6817/2157641649] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f53b4071a80 con 0x7f53b406efc0 2024-03-22T07:12:45.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.404+0000 7f53bdffb700 1 -- 172.21.15.63:0/104430652 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53c408b9a0 con 0x7f53c8101d70 2024-03-22T07:12:45.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.404+0000 7f53ccab0700 1 --2- 172.21.15.63:0/104430652 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f53b40692f0 0x7f53b406b7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53c810dc40 tx=0x7f53b8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:45.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.404+0000 7f53cdab2700 1 --2- 172.21.15.63:0/104430652 >> [v2:172.21.15.69:6816/2157641649,v1:172.21.15.69:6817/2157641649] conn(0x7f53b406efc0 0x7f53b40713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:45.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.405+0000 7f53cdab2700 1 --2- 172.21.15.63:0/104430652 >> [v2:172.21.15.69:6816/2157641649,v1:172.21.15.69:6817/2157641649] conn(0x7f53b406efc0 0x7f53b40713d0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:45.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.405+0000 7f53bdffb700 1 -- 172.21.15.63:0/104430652 <== osd.5 v2:172.21.15.69:6816/2157641649 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f53b4071a80 con 0x7f53b406efc0 2024-03-22T07:12:45.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.420+0000 7f53cf515700 1 -- 172.21.15.63:0/104430652 --> [v2:172.21.15.69:6816/2157641649,v1:172.21.15.69:6817/2157641649] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f53c805e6c0 con 0x7f53b406efc0 2024-03-22T07:12:45.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.421+0000 7f53bdffb700 1 -- 172.21.15.63:0/104430652 <== osd.5 v2:172.21.15.69:6816/2157641649 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f53c805e6c0 con 0x7f53b406efc0 2024-03-22T07:12:45.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.421+0000 7f53cf515700 1 -- 172.21.15.63:0/104430652 >> [v2:172.21.15.69:6816/2157641649,v1:172.21.15.69:6817/2157641649] conn(0x7f53b406efc0 msgr2=0x7f53b40713d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:45.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.421+0000 7f53cf515700 1 --2- 172.21.15.63:0/104430652 >> [v2:172.21.15.69:6816/2157641649,v1:172.21.15.69:6817/2157641649] conn(0x7f53b406efc0 0x7f53b40713d0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:45.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.421+0000 7f53cf515700 1 -- 172.21.15.63:0/104430652 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f53b40692f0 msgr2=0x7f53b406b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:45.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.421+0000 7f53cf515700 1 --2- 172.21.15.63:0/104430652 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f53b40692f0 0x7f53b406b7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53c810dc40 tx=0x7f53b8009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:45.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.421+0000 7f53cf515700 1 -- 172.21.15.63:0/104430652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53c8101d70 msgr2=0x7f53c81129e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:45.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.421+0000 7f53cf515700 1 --2- 172.21.15.63:0/104430652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53c8101d70 0x7f53c81129e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f53c400fd10 tx=0x7f53c40046b0 comp rx=0 tx=0).stop 2024-03-22T07:12:45.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.421+0000 7f53cf515700 1 -- 172.21.15.63:0/104430652 shutdown_connections 2024-03-22T07:12:45.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.421+0000 7f53cf515700 1 --2- 172.21.15.63:0/104430652 >> [v2:172.21.15.69:6816/2157641649,v1:172.21.15.69:6817/2157641649] conn(0x7f53b406efc0 0x7f53b40713d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:45.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.421+0000 7f53cf515700 1 --2- 172.21.15.63:0/104430652 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f53b40692f0 0x7f53b406b7a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:45.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.421+0000 7f53cf515700 1 --2- 172.21.15.63:0/104430652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f53c8101d70 0x7f53c81129e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:45.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.421+0000 7f53cf515700 1 --2- 172.21.15.63:0/104430652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f53c8102710 0x7f53c8112f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:45.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.421+0000 7f53cf515700 1 -- 172.21.15.63:0/104430652 >> 172.21.15.63:0/104430652 conn(0x7f53c8072bc0 msgr2=0x7f53c80faf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:45.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.421+0000 7f53cf515700 1 -- 172.21.15.63:0/104430652 shutdown_connections 2024-03-22T07:12:45.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:45.421+0000 7f53cf515700 1 -- 172.21.15.63:0/104430652 wait complete. 2024-03-22T07:12:45.745 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:45 smithi063 ceph-mon[29200]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:45.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:45 smithi069 ceph-mon[36441]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:46.770 INFO:teuthology.orchestra.run.smithi063.stdout:60129542174 2024-03-22T07:12:46.770 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-22T07:12:47.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:47 smithi069 ceph-mon[36441]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:47.782 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:47 smithi063 ceph-mon[29200]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:47.813 INFO:teuthology.orchestra.run.smithi063.stdout:171798691850 2024-03-22T07:12:47.813 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-22T07:12:48.235 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:48.753 INFO:teuthology.orchestra.run.smithi063.stdout:81604378649 2024-03-22T07:12:48.754 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-22T07:12:49.135 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:49.661 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:49 smithi063 ceph-mon[29200]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:49.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:49 smithi069 ceph-mon[36441]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:50.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.812+0000 7f1cbe763700 1 -- 172.21.15.63:0/480687236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb8102e20 msgr2=0x7f1cb81031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:50.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.812+0000 7f1cbe763700 1 --2- 172.21.15.63:0/480687236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb8102e20 0x7f1cb81031f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8004940 tx=0x7f1ca8009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:50.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.812+0000 7f1cbe763700 1 -- 172.21.15.63:0/480687236 shutdown_connections 2024-03-22T07:12:50.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.812+0000 7f1cbe763700 1 --2- 172.21.15.63:0/480687236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cb81037c0 0x7f1cb80fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:50.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.812+0000 7f1cbe763700 1 --2- 172.21.15.63:0/480687236 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb8102e20 0x7f1cb81031f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:50.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.812+0000 7f1cbe763700 1 -- 172.21.15.63:0/480687236 >> 172.21.15.63:0/480687236 conn(0x7f1cb80f8970 msgr2=0x7f1cb80fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:50.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.812+0000 7f1cbe763700 1 -- 172.21.15.63:0/480687236 shutdown_connections 2024-03-22T07:12:50.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.812+0000 7f1cbe763700 1 -- 172.21.15.63:0/480687236 wait complete. 2024-03-22T07:12:50.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.813+0000 7f1cbe763700 1 Processor -- start 2024-03-22T07:12:50.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.813+0000 7f1cbe763700 1 -- start start 2024-03-22T07:12:50.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.813+0000 7f1cbe763700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb8102e20 0x7f1cb8113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:50.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.813+0000 7f1cbe763700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cb81037c0 0x7f1cb8114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:50.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.813+0000 7f1cbe763700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb81146e0 con 0x7f1cb8102e20 2024-03-22T07:12:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.813+0000 7f1cbe763700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb810dad0 con 0x7f1cb81037c0 2024-03-22T07:12:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.814+0000 7f1cb7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb8102e20 0x7f1cb8113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.814+0000 7f1cb7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb8102e20 0x7f1cb8113b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41210/0 (socket says 172.21.15.63:41210) 2024-03-22T07:12:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.814+0000 7f1cb7fff700 1 -- 172.21.15.63:0/117667862 learned_addr learned my addr 172.21.15.63:0/117667862 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.814+0000 7f1cb77fe700 1 --2- 172.21.15.63:0/117667862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cb81037c0 0x7f1cb8114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.814+0000 7f1cb7fff700 1 -- 172.21.15.63:0/117667862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cb81037c0 msgr2=0x7f1cb8114090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.814+0000 7f1cb7fff700 1 --2- 172.21.15.63:0/117667862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cb81037c0 0x7f1cb8114090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.814+0000 7f1cb7fff700 1 -- 172.21.15.63:0/117667862 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca8005040 con 0x7f1cb8102e20 2024-03-22T07:12:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.814+0000 7f1cb7fff700 1 --2- 172.21.15.63:0/117667862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb8102e20 0x7f1cb8113b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8007f50 tx=0x7f1ca8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.814+0000 7f1cb4ff9700 1 -- 172.21.15.63:0/117667862 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ca8021070 con 0x7f1cb8102e20 2024-03-22T07:12:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.814+0000 7f1cbe763700 1 -- 172.21.15.63:0/117667862 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb810dd50 con 0x7f1cb8102e20 2024-03-22T07:12:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.814+0000 7f1cb4ff9700 1 -- 172.21.15.63:0/117667862 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ca8013cd0 con 0x7f1cb8102e20 2024-03-22T07:12:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.814+0000 7f1cb4ff9700 1 -- 172.21.15.63:0/117667862 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ca800e070 con 0x7f1cb8102e20 2024-03-22T07:12:50.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.814+0000 7f1cbe763700 1 -- 172.21.15.63:0/117667862 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb810e2b0 con 0x7f1cb8102e20 2024-03-22T07:12:50.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.815+0000 7f1cbe763700 1 -- 172.21.15.63:0/117667862 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb8100130 con 0x7f1cb8102e20 2024-03-22T07:12:50.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.816+0000 7f1cb4ff9700 1 -- 172.21.15.63:0/117667862 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1ca801c960 con 0x7f1cb8102e20 2024-03-22T07:12:50.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.816+0000 7f1cb4ff9700 1 --2- 172.21.15.63:0/117667862 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f1ca00692a0 0x7f1ca006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:50.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.816+0000 7f1cb77fe700 1 --2- 172.21.15.63:0/117667862 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f1ca00692a0 0x7f1ca006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:50.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.816+0000 7f1cb4ff9700 1 -- 172.21.15.63:0/117667862 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1ca8088190 con 0x7f1cb8102e20 2024-03-22T07:12:50.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.817+0000 7f1cb77fe700 1 --2- 172.21.15.63:0/117667862 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f1ca00692a0 0x7f1ca006b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1cb810ec90 tx=0x7f1cac00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:50.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:50.821+0000 7f1cb4ff9700 1 -- 172.21.15.63:0/117667862 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f1ca8058330 con 0x7f1cb8102e20 2024-03-22T07:12:51.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.028+0000 7f1cbe763700 1 -- 172.21.15.63:0/117667862 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1cb810f1a0 con 0x7f1cb8102e20 2024-03-22T07:12:51.029 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.029+0000 7f1cb4ff9700 1 -- 172.21.15.63:0/117667862 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1ca8054020 con 0x7f1cb8102e20 2024-03-22T07:12:51.029 INFO:teuthology.orchestra.run.smithi063.stdout:38654705702 2024-03-22T07:12:51.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.031+0000 7f1cbe763700 1 -- 172.21.15.63:0/117667862 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f1ca00692a0 msgr2=0x7f1ca006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:51.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.031+0000 7f1cbe763700 1 --2- 172.21.15.63:0/117667862 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f1ca00692a0 0x7f1ca006b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1cb810ec90 tx=0x7f1cac00b040 comp rx=0 tx=0).stop 2024-03-22T07:12:51.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.031+0000 7f1cbe763700 1 -- 172.21.15.63:0/117667862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb8102e20 msgr2=0x7f1cb8113b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:51.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.031+0000 7f1cbe763700 1 --2- 172.21.15.63:0/117667862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb8102e20 0x7f1cb8113b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8007f50 tx=0x7f1ca8009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:51.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.031+0000 7f1cbe763700 1 -- 172.21.15.63:0/117667862 shutdown_connections 2024-03-22T07:12:51.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.031+0000 7f1cbe763700 1 --2- 172.21.15.63:0/117667862 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f1ca00692a0 0x7f1ca006b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:51.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.031+0000 7f1cbe763700 1 --2- 172.21.15.63:0/117667862 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cb8102e20 0x7f1cb8113b50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:51.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.031+0000 7f1cbe763700 1 --2- 172.21.15.63:0/117667862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cb81037c0 0x7f1cb8114090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:51.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.031+0000 7f1cbe763700 1 -- 172.21.15.63:0/117667862 >> 172.21.15.63:0/117667862 conn(0x7f1cb80f8970 msgr2=0x7f1cb81073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:51.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.031+0000 7f1cbe763700 1 -- 172.21.15.63:0/117667862 shutdown_connections 2024-03-22T07:12:51.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.031+0000 7f1cbe763700 1 -- 172.21.15.63:0/117667862 wait complete. 2024-03-22T07:12:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:51 smithi063 ceph-mon[29200]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:51 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/117667862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-22T07:12:51.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:51 smithi069 ceph-mon[36441]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:51.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:51 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/117667862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-22T07:12:51.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.859+0000 7f01d710f700 1 -- 172.21.15.63:0/3042594529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01d00fb4a0 msgr2=0x7f01d00fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:51.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.859+0000 7f01d710f700 1 --2- 172.21.15.63:0/3042594529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01d00fb4a0 0x7f01d00fb910 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f01bc004940 tx=0x7f01bc009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:51.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.859+0000 7f01d710f700 1 -- 172.21.15.63:0/3042594529 shutdown_connections 2024-03-22T07:12:51.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.859+0000 7f01d710f700 1 --2- 172.21.15.63:0/3042594529 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01d00fb4a0 0x7f01d00fb910 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:51.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.859+0000 7f01d710f700 1 --2- 172.21.15.63:0/3042594529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01d0102e70 0x7f01d0103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:51.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.859+0000 7f01d710f700 1 -- 172.21.15.63:0/3042594529 >> 172.21.15.63:0/3042594529 conn(0x7f01d00714c0 msgr2=0x7f01d00718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:51.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.859+0000 7f01d710f700 1 -- 172.21.15.63:0/3042594529 shutdown_connections 2024-03-22T07:12:51.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.859+0000 7f01d710f700 1 -- 172.21.15.63:0/3042594529 wait complete. 2024-03-22T07:12:51.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.860+0000 7f01d710f700 1 Processor -- start 2024-03-22T07:12:51.860 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.860+0000 7f01d710f700 1 -- start start 2024-03-22T07:12:51.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.860+0000 7f01d710f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01d00fb4a0 0x7f01d0113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:51.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.860+0000 7f01d710f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01d0102e70 0x7f01d01140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:51.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.860+0000 7f01d710f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d0114720 con 0x7f01d00fb4a0 2024-03-22T07:12:51.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.860+0000 7f01d710f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d010db10 con 0x7f01d0102e70 2024-03-22T07:12:51.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.860+0000 7f01d4eab700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01d00fb4a0 0x7f01d0113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:51.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.860+0000 7f01d4eab700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01d00fb4a0 0x7f01d0113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41228/0 (socket says 172.21.15.63:41228) 2024-03-22T07:12:51.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.860+0000 7f01d4eab700 1 -- 172.21.15.63:0/3120374961 learned_addr learned my addr 172.21.15.63:0/3120374961 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:51.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.860+0000 7f01cffff700 1 --2- 172.21.15.63:0/3120374961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01d0102e70 0x7f01d01140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:51.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.861+0000 7f01d4eab700 1 -- 172.21.15.63:0/3120374961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01d0102e70 msgr2=0x7f01d01140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:51.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.861+0000 7f01d4eab700 1 --2- 172.21.15.63:0/3120374961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01d0102e70 0x7f01d01140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:51.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.861+0000 7f01d4eab700 1 -- 172.21.15.63:0/3120374961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc005040 con 0x7f01d00fb4a0 2024-03-22T07:12:51.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.861+0000 7f01d4eab700 1 --2- 172.21.15.63:0/3120374961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01d00fb4a0 0x7f01d0113b90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f01c4002a50 tx=0x7f01c40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:51.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.861+0000 7f01cd7fa700 1 -- 172.21.15.63:0/3120374961 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01c4007cc0 con 0x7f01d00fb4a0 2024-03-22T07:12:51.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.861+0000 7f01d710f700 1 -- 172.21.15.63:0/3120374961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d010ddf0 con 0x7f01d00fb4a0 2024-03-22T07:12:51.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.861+0000 7f01cd7fa700 1 -- 172.21.15.63:0/3120374961 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01c400f040 con 0x7f01d00fb4a0 2024-03-22T07:12:51.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.861+0000 7f01cd7fa700 1 -- 172.21.15.63:0/3120374961 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01c400da30 con 0x7f01d00fb4a0 2024-03-22T07:12:51.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.861+0000 7f01d710f700 1 -- 172.21.15.63:0/3120374961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d010e300 con 0x7f01d00fb4a0 2024-03-22T07:12:51.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.862+0000 7f01d710f700 1 -- 172.21.15.63:0/3120374961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d0100170 con 0x7f01d00fb4a0 2024-03-22T07:12:51.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.862+0000 7f01cd7fa700 1 -- 172.21.15.63:0/3120374961 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f01c400db90 con 0x7f01d00fb4a0 2024-03-22T07:12:51.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.863+0000 7f01cd7fa700 1 --2- 172.21.15.63:0/3120374961 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f01c00692a0 0x7f01c006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:51.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.863+0000 7f01cffff700 1 --2- 172.21.15.63:0/3120374961 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f01c00692a0 0x7f01c006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:51.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.863+0000 7f01cd7fa700 1 -- 172.21.15.63:0/3120374961 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f01c4086be0 con 0x7f01d00fb4a0 2024-03-22T07:12:51.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.863+0000 7f01cffff700 1 --2- 172.21.15.63:0/3120374961 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f01c00692a0 0x7f01c006b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f01d010edf0 tx=0x7f01bc009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:51.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:51.867+0000 7f01cd7fa700 1 -- 172.21.15.63:0/3120374961 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f01c4056c60 con 0x7f01d00fb4a0 2024-03-22T07:12:52.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:52.069+0000 7f01d710f700 1 -- 172.21.15.63:0/3120374961 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f01d010f1e0 con 0x7f01d00fb4a0 2024-03-22T07:12:52.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:52.069+0000 7f01cd7fa700 1 -- 172.21.15.63:0/3120374961 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f01c40567f0 con 0x7f01d00fb4a0 2024-03-22T07:12:52.069 INFO:teuthology.orchestra.run.smithi063.stdout:150323855375 2024-03-22T07:12:52.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:52.071+0000 7f01d710f700 1 -- 172.21.15.63:0/3120374961 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f01c00692a0 msgr2=0x7f01c006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:52.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:52.071+0000 7f01d710f700 1 --2- 172.21.15.63:0/3120374961 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f01c00692a0 0x7f01c006b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f01d010edf0 tx=0x7f01bc009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:52.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:52.071+0000 7f01d710f700 1 -- 172.21.15.63:0/3120374961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01d00fb4a0 msgr2=0x7f01d0113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:52.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:52.071+0000 7f01d710f700 1 --2- 172.21.15.63:0/3120374961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01d00fb4a0 0x7f01d0113b90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f01c4002a50 tx=0x7f01c40046b0 comp rx=0 tx=0).stop 2024-03-22T07:12:52.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:52.071+0000 7f01d710f700 1 -- 172.21.15.63:0/3120374961 shutdown_connections 2024-03-22T07:12:52.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:52.071+0000 7f01d710f700 1 --2- 172.21.15.63:0/3120374961 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f01c00692a0 0x7f01c006b750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:52.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:52.071+0000 7f01d710f700 1 --2- 172.21.15.63:0/3120374961 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f01d00fb4a0 0x7f01d0113b90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:52.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:52.071+0000 7f01d710f700 1 --2- 172.21.15.63:0/3120374961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01d0102e70 0x7f01d01140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:52.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:52.071+0000 7f01d710f700 1 -- 172.21.15.63:0/3120374961 >> 172.21.15.63:0/3120374961 conn(0x7f01d00714c0 msgr2=0x7f01d00fa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:52.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:52.071+0000 7f01d710f700 1 -- 172.21.15.63:0/3120374961 shutdown_connections 2024-03-22T07:12:52.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:52.071+0000 7f01d710f700 1 -- 172.21.15.63:0/3120374961 wait complete. 2024-03-22T07:12:52.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:52 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/3120374961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-22T07:12:52.809 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:52 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3120374961' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-22T07:12:52.989 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705700 got 38654705702 for osd.0 2024-03-22T07:12:52.990 DEBUG:teuthology.parallel:result is None 2024-03-22T07:12:53.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:53 smithi063 ceph-mon[29200]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:53.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:53 smithi069 ceph-mon[36441]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:53.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.786+0000 7fc6b713b700 1 -- 172.21.15.63:0/3521782010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6b01037b0 msgr2=0x7fc6b00fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:53.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.786+0000 7fc6b713b700 1 --2- 172.21.15.63:0/3521782010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6b01037b0 0x7fc6b00fce20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc69c004940 tx=0x7fc69c009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:53.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.786+0000 7fc6b713b700 1 -- 172.21.15.63:0/3521782010 shutdown_connections 2024-03-22T07:12:53.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.786+0000 7fc6b713b700 1 --2- 172.21.15.63:0/3521782010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6b01037b0 0x7fc6b00fce20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:53.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.786+0000 7fc6b713b700 1 --2- 172.21.15.63:0/3521782010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6b0102e10 0x7fc6b01031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:53.786 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.786+0000 7fc6b713b700 1 -- 172.21.15.63:0/3521782010 >> 172.21.15.63:0/3521782010 conn(0x7fc6b00f8980 msgr2=0x7fc6b00fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:53.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.786+0000 7fc6b713b700 1 -- 172.21.15.63:0/3521782010 shutdown_connections 2024-03-22T07:12:53.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.786+0000 7fc6b713b700 1 -- 172.21.15.63:0/3521782010 wait complete. 2024-03-22T07:12:53.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.787+0000 7fc6b713b700 1 Processor -- start 2024-03-22T07:12:53.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.787+0000 7fc6b713b700 1 -- start start 2024-03-22T07:12:53.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.788+0000 7fc6b713b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6b0102e10 0x7fc6b0113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:53.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.788+0000 7fc6b713b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6b01037b0 0x7fc6b0114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:53.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.788+0000 7fc6b713b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b0114670 con 0x7fc6b0102e10 2024-03-22T07:12:53.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.788+0000 7fc6b713b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b010da60 con 0x7fc6b01037b0 2024-03-22T07:12:53.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.788+0000 7fc6b4ed7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6b0102e10 0x7fc6b0113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:53.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.788+0000 7fc6b4ed7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6b0102e10 0x7fc6b0113ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41250/0 (socket says 172.21.15.63:41250) 2024-03-22T07:12:53.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.788+0000 7fc6b4ed7700 1 -- 172.21.15.63:0/166396654 learned_addr learned my addr 172.21.15.63:0/166396654 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:53.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.788+0000 7fc6affff700 1 --2- 172.21.15.63:0/166396654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6b01037b0 0x7fc6b0114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:53.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.788+0000 7fc6b4ed7700 1 -- 172.21.15.63:0/166396654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6b01037b0 msgr2=0x7fc6b0114020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:53.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.788+0000 7fc6b4ed7700 1 --2- 172.21.15.63:0/166396654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6b01037b0 0x7fc6b0114020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:53.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.788+0000 7fc6b4ed7700 1 -- 172.21.15.63:0/166396654 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c005040 con 0x7fc6b0102e10 2024-03-22T07:12:53.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.788+0000 7fc6affff700 1 --2- 172.21.15.63:0/166396654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6b01037b0 0x7fc6b0114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:12:53.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.788+0000 7fc6b4ed7700 1 --2- 172.21.15.63:0/166396654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6b0102e10 0x7fc6b0113ae0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400eb40 tx=0x7fc6a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:53.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.788+0000 7fc6ad7fa700 1 -- 172.21.15.63:0/166396654 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6a4007cc0 con 0x7fc6b0102e10 2024-03-22T07:12:53.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.788+0000 7fc6ad7fa700 1 -- 172.21.15.63:0/166396654 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6a400f040 con 0x7fc6b0102e10 2024-03-22T07:12:53.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.788+0000 7fc6ad7fa700 1 -- 172.21.15.63:0/166396654 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6a4009620 con 0x7fc6b0102e10 2024-03-22T07:12:53.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.788+0000 7fc6b713b700 1 -- 172.21.15.63:0/166396654 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b010dce0 con 0x7fc6b0102e10 2024-03-22T07:12:53.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.789+0000 7fc6b713b700 1 -- 172.21.15.63:0/166396654 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b010e1a0 con 0x7fc6b0102e10 2024-03-22T07:12:53.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.789+0000 7fc6b713b700 1 -- 172.21.15.63:0/166396654 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b01052e0 con 0x7fc6b0102e10 2024-03-22T07:12:53.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.790+0000 7fc6ad7fa700 1 -- 172.21.15.63:0/166396654 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fc6a4009780 con 0x7fc6b0102e10 2024-03-22T07:12:53.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.791+0000 7fc6ad7fa700 1 --2- 172.21.15.63:0/166396654 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fc6a00692a0 0x7fc6a006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:53.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.791+0000 7fc6ad7fa700 1 -- 172.21.15.63:0/166396654 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc6a4086af0 con 0x7fc6b0102e10 2024-03-22T07:12:53.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.791+0000 7fc6affff700 1 --2- 172.21.15.63:0/166396654 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fc6a00692a0 0x7fc6a006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:53.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.791+0000 7fc6affff700 1 --2- 172.21.15.63:0/166396654 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fc6a00692a0 0x7fc6a006b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc6b010ed40 tx=0x7fc69c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:53.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.796+0000 7fc6ad7fa700 1 -- 172.21.15.63:0/166396654 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fc6a404e190 con 0x7fc6b0102e10 2024-03-22T07:12:53.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.993+0000 7fc6b713b700 1 -- 172.21.15.63:0/166396654 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc6b010f130 con 0x7fc6b0102e10 2024-03-22T07:12:53.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.993+0000 7fc6ad7fa700 1 -- 172.21.15.63:0/166396654 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc6a4056c20 con 0x7fc6b0102e10 2024-03-22T07:12:53.993 INFO:teuthology.orchestra.run.smithi063.stdout:115964117014 2024-03-22T07:12:53.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.995+0000 7fc6b713b700 1 -- 172.21.15.63:0/166396654 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fc6a00692a0 msgr2=0x7fc6a006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:53.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.995+0000 7fc6b713b700 1 --2- 172.21.15.63:0/166396654 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fc6a00692a0 0x7fc6a006b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc6b010ed40 tx=0x7fc69c009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:53.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.995+0000 7fc6b713b700 1 -- 172.21.15.63:0/166396654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6b0102e10 msgr2=0x7fc6b0113ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:53.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.995+0000 7fc6b713b700 1 --2- 172.21.15.63:0/166396654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6b0102e10 0x7fc6b0113ae0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400eb40 tx=0x7fc6a400b040 comp rx=0 tx=0).stop 2024-03-22T07:12:53.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.995+0000 7fc6b713b700 1 -- 172.21.15.63:0/166396654 shutdown_connections 2024-03-22T07:12:53.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.996+0000 7fc6b713b700 1 --2- 172.21.15.63:0/166396654 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fc6a00692a0 0x7fc6a006b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:53.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.996+0000 7fc6b713b700 1 --2- 172.21.15.63:0/166396654 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc6b0102e10 0x7fc6b0113ae0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:53.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.996+0000 7fc6b713b700 1 --2- 172.21.15.63:0/166396654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6b01037b0 0x7fc6b0114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:53.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.996+0000 7fc6b713b700 1 -- 172.21.15.63:0/166396654 >> 172.21.15.63:0/166396654 conn(0x7fc6b00f8980 msgr2=0x7fc6b01021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:53.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.996+0000 7fc6b713b700 1 -- 172.21.15.63:0/166396654 shutdown_connections 2024-03-22T07:12:53.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:53.996+0000 7fc6b713b700 1 -- 172.21.15.63:0/166396654 wait complete. 2024-03-22T07:12:54.773 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:54 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/166396654' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-22T07:12:54.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:54 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/166396654' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-22T07:12:54.798 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117011 got 115964117014 for osd.3 2024-03-22T07:12:54.798 DEBUG:teuthology.parallel:result is None 2024-03-22T07:12:54.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.985+0000 7fdda8705700 1 -- 172.21.15.63:0/2340348055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdda0070300 msgr2=0x7fdda00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:54.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.985+0000 7fdda8705700 1 --2- 172.21.15.63:0/2340348055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdda0070300 0x7fdda00706d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c004970 tx=0x7fdd9c009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:54.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.986+0000 7fdda8705700 1 -- 172.21.15.63:0/2340348055 shutdown_connections 2024-03-22T07:12:54.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.986+0000 7fdda8705700 1 --2- 172.21.15.63:0/2340348055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdda0070c10 0x7fdda0109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:54.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.986+0000 7fdda8705700 1 --2- 172.21.15.63:0/2340348055 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdda0070300 0x7fdda00706d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:54.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.986+0000 7fdda8705700 1 -- 172.21.15.63:0/2340348055 >> 172.21.15.63:0/2340348055 conn(0x7fdda00f89b0 msgr2=0x7fdda00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:54.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.986+0000 7fdda8705700 1 -- 172.21.15.63:0/2340348055 shutdown_connections 2024-03-22T07:12:54.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.986+0000 7fdda8705700 1 -- 172.21.15.63:0/2340348055 wait complete. 2024-03-22T07:12:54.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.986+0000 7fdda8705700 1 Processor -- start 2024-03-22T07:12:54.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.986+0000 7fdda8705700 1 -- start start 2024-03-22T07:12:54.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.986+0000 7fdda8705700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdda0070300 0x7fdda0115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:54.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.986+0000 7fdda8705700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdda0070c10 0x7fdda0116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:54.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.986+0000 7fdda8705700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda01166e0 con 0x7fdda0070300 2024-03-22T07:12:54.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.986+0000 7fdda8705700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda010fcc0 con 0x7fdda0070c10 2024-03-22T07:12:54.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.986+0000 7fdda64a1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdda0070300 0x7fdda0115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:54.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.986+0000 7fdda64a1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdda0070300 0x7fdda0115b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53292/0 (socket says 172.21.15.63:53292) 2024-03-22T07:12:54.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.987+0000 7fdda64a1700 1 -- 172.21.15.63:0/1841589827 learned_addr learned my addr 172.21.15.63:0/1841589827 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:54.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.987+0000 7fdda5ca0700 1 --2- 172.21.15.63:0/1841589827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdda0070c10 0x7fdda0116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:54.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.987+0000 7fdda64a1700 1 -- 172.21.15.63:0/1841589827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdda0070c10 msgr2=0x7fdda0116090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:54.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.987+0000 7fdda64a1700 1 --2- 172.21.15.63:0/1841589827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdda0070c10 0x7fdda0116090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:54.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.987+0000 7fdda64a1700 1 -- 172.21.15.63:0/1841589827 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9c005040 con 0x7fdda0070300 2024-03-22T07:12:54.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.987+0000 7fdda64a1700 1 --2- 172.21.15.63:0/1841589827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdda0070300 0x7fdda0115b50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c000c00 tx=0x7fdd9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:54.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.987+0000 7fdd96ffd700 1 -- 172.21.15.63:0/1841589827 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd9c00e040 con 0x7fdda0070300 2024-03-22T07:12:54.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.987+0000 7fdd96ffd700 1 -- 172.21.15.63:0/1841589827 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd9c00bc10 con 0x7fdda0070300 2024-03-22T07:12:54.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.987+0000 7fdda8705700 1 -- 172.21.15.63:0/1841589827 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda010ff40 con 0x7fdda0070300 2024-03-22T07:12:54.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.987+0000 7fdd96ffd700 1 -- 172.21.15.63:0/1841589827 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd9c01c3f0 con 0x7fdda0070300 2024-03-22T07:12:54.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.987+0000 7fdda8705700 1 -- 172.21.15.63:0/1841589827 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda0110480 con 0x7fdda0070300 2024-03-22T07:12:54.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.988+0000 7fdda8705700 1 -- 172.21.15.63:0/1841589827 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda0071a40 con 0x7fdda0070300 2024-03-22T07:12:54.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.989+0000 7fdd96ffd700 1 -- 172.21.15.63:0/1841589827 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fdd9c013420 con 0x7fdda0070300 2024-03-22T07:12:54.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.989+0000 7fdd96ffd700 1 --2- 172.21.15.63:0/1841589827 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fdd8c0692a0 0x7fdd8c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:54.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.989+0000 7fdd96ffd700 1 -- 172.21.15.63:0/1841589827 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fdd9c088060 con 0x7fdda0070300 2024-03-22T07:12:54.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.989+0000 7fdda5ca0700 1 --2- 172.21.15.63:0/1841589827 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fdd8c0692a0 0x7fdd8c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:54.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.989+0000 7fdda5ca0700 1 --2- 172.21.15.63:0/1841589827 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fdd8c0692a0 0x7fdd8c06b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdda01113f0 tx=0x7fdd9000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:54.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:54.993+0000 7fdd96ffd700 1 -- 172.21.15.63:0/1841589827 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdd9c058160 con 0x7fdda0070300 2024-03-22T07:12:55.073 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:55.073 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:12:55.098 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855372 got 150323855375 for osd.4 2024-03-22T07:12:55.181 DEBUG:teuthology.parallel:result is None 2024-03-22T07:12:55.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:55.202+0000 7fdda8705700 1 -- 172.21.15.63:0/1841589827 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fdda01111a0 con 0x7fdda0070300 2024-03-22T07:12:55.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:55.203+0000 7fdd96ffd700 1 -- 172.21.15.63:0/1841589827 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdd9c004de0 con 0x7fdda0070300 2024-03-22T07:12:55.203 INFO:teuthology.orchestra.run.smithi063.stdout:171798691852 2024-03-22T07:12:55.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:55.204+0000 7fdda8705700 1 -- 172.21.15.63:0/1841589827 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fdd8c0692a0 msgr2=0x7fdd8c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:55.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:55.204+0000 7fdda8705700 1 --2- 172.21.15.63:0/1841589827 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fdd8c0692a0 0x7fdd8c06b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdda01113f0 tx=0x7fdd9000b040 comp rx=0 tx=0).stop 2024-03-22T07:12:55.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:55.204+0000 7fdda8705700 1 -- 172.21.15.63:0/1841589827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdda0070300 msgr2=0x7fdda0115b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:55.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:55.204+0000 7fdda8705700 1 --2- 172.21.15.63:0/1841589827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdda0070300 0x7fdda0115b50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c000c00 tx=0x7fdd9c009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:55.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:55.205+0000 7fdda8705700 1 -- 172.21.15.63:0/1841589827 shutdown_connections 2024-03-22T07:12:55.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:55.205+0000 7fdda8705700 1 --2- 172.21.15.63:0/1841589827 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fdd8c0692a0 0x7fdd8c06b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:55.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:55.205+0000 7fdda8705700 1 --2- 172.21.15.63:0/1841589827 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdda0070300 0x7fdda0115b50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:55.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:55.205+0000 7fdda8705700 1 --2- 172.21.15.63:0/1841589827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdda0070c10 0x7fdda0116090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:55.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:55.205+0000 7fdda8705700 1 -- 172.21.15.63:0/1841589827 >> 172.21.15.63:0/1841589827 conn(0x7fdda00f89b0 msgr2=0x7fdda00f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:55.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:55.205+0000 7fdda8705700 1 -- 172.21.15.63:0/1841589827 shutdown_connections 2024-03-22T07:12:55.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:55.205+0000 7fdda8705700 1 -- 172.21.15.63:0/1841589827 wait complete. 2024-03-22T07:12:55.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:55 smithi069 ceph-mon[36441]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:55.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:55 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/1841589827' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-22T07:12:55.839 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:55 smithi063 ceph-mon[29200]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:55.839 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:55 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/1841589827' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-22T07:12:56.638 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691850 got 171798691852 for osd.5 2024-03-22T07:12:56.638 DEBUG:teuthology.parallel:result is None 2024-03-22T07:12:57.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:57 smithi069 ceph-mon[36441]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:57.801 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:57 smithi063 ceph-mon[29200]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:58.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.579+0000 7f326e386700 1 -- 172.21.15.63:0/3052621579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3268102e10 msgr2=0x7f32681031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:58.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.579+0000 7f326e386700 1 --2- 172.21.15.63:0/3052621579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3268102e10 0x7f32681031e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3258004940 tx=0x7f3258009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:58.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.579+0000 7f326e386700 1 -- 172.21.15.63:0/3052621579 shutdown_connections 2024-03-22T07:12:58.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.579+0000 7f326e386700 1 --2- 172.21.15.63:0/3052621579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32681037b0 0x7f32680fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:58.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.579+0000 7f326e386700 1 --2- 172.21.15.63:0/3052621579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3268102e10 0x7f32681031e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:58.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.579+0000 7f326e386700 1 -- 172.21.15.63:0/3052621579 >> 172.21.15.63:0/3052621579 conn(0x7f32680f8980 msgr2=0x7f32680fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:58.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.579+0000 7f326e386700 1 -- 172.21.15.63:0/3052621579 shutdown_connections 2024-03-22T07:12:58.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.579+0000 7f326e386700 1 -- 172.21.15.63:0/3052621579 wait complete. 2024-03-22T07:12:58.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.580+0000 7f326e386700 1 Processor -- start 2024-03-22T07:12:58.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.580+0000 7f326e386700 1 -- start start 2024-03-22T07:12:58.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.580+0000 7f326e386700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3268102e10 0x7f32680702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:58.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.580+0000 7f326e386700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32681037b0 0x7f3268070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:58.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.580+0000 7f326e386700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3268070d90 con 0x7f3268102e10 2024-03-22T07:12:58.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.580+0000 7f326e386700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3268070ed0 con 0x7f32681037b0 2024-03-22T07:12:58.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.581+0000 7f3267fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3268102e10 0x7f32680702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:58.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.581+0000 7f3267fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3268102e10 0x7f32680702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53298/0 (socket says 172.21.15.63:53298) 2024-03-22T07:12:58.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.581+0000 7f3267fff700 1 -- 172.21.15.63:0/3018774126 learned_addr learned my addr 172.21.15.63:0/3018774126 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:58.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.581+0000 7f32677fe700 1 --2- 172.21.15.63:0/3018774126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32681037b0 0x7f3268070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:58.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.581+0000 7f3267fff700 1 -- 172.21.15.63:0/3018774126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32681037b0 msgr2=0x7f3268070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:58.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.581+0000 7f3267fff700 1 --2- 172.21.15.63:0/3018774126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32681037b0 0x7f3268070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:58.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.581+0000 7f3267fff700 1 -- 172.21.15.63:0/3018774126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3258005040 con 0x7f3268102e10 2024-03-22T07:12:58.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.581+0000 7f3267fff700 1 --2- 172.21.15.63:0/3018774126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3268102e10 0x7f32680702c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f325800b3b0 tx=0x7f3258009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:58.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.581+0000 7f3264ff9700 1 -- 172.21.15.63:0/3018774126 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f325800e040 con 0x7f3268102e10 2024-03-22T07:12:58.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.581+0000 7f3264ff9700 1 -- 172.21.15.63:0/3018774126 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f325800bbc0 con 0x7f3268102e10 2024-03-22T07:12:58.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.581+0000 7f326e386700 1 -- 172.21.15.63:0/3018774126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3268071120 con 0x7f3268102e10 2024-03-22T07:12:58.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.581+0000 7f3264ff9700 1 -- 172.21.15.63:0/3018774126 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f325801c3f0 con 0x7f3268102e10 2024-03-22T07:12:58.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.581+0000 7f326e386700 1 -- 172.21.15.63:0/3018774126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326811a310 con 0x7f3268102e10 2024-03-22T07:12:58.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.582+0000 7f326e386700 1 -- 172.21.15.63:0/3018774126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3268100120 con 0x7f3268102e10 2024-03-22T07:12:58.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.583+0000 7f3264ff9700 1 -- 172.21.15.63:0/3018774126 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f325800bd30 con 0x7f3268102e10 2024-03-22T07:12:58.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.583+0000 7f3264ff9700 1 --2- 172.21.15.63:0/3018774126 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f32500692a0 0x7f325006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:58.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.583+0000 7f3264ff9700 1 -- 172.21.15.63:0/3018774126 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3258088030 con 0x7f3268102e10 2024-03-22T07:12:58.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.583+0000 7f32677fe700 1 --2- 172.21.15.63:0/3018774126 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f32500692a0 0x7f325006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:58.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.584+0000 7f32677fe700 1 --2- 172.21.15.63:0/3018774126 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f32500692a0 0x7f325006b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f325c007fd0 tx=0x7f325c009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:58.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.588+0000 7f3264ff9700 1 -- 172.21.15.63:0/3018774126 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f32580546a0 con 0x7f3268102e10 2024-03-22T07:12:58.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.775+0000 7ffbccb93700 1 -- 172.21.15.63:0/2546877469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbc80fded0 msgr2=0x7ffbc80fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:58.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.775+0000 7ffbccb93700 1 --2- 172.21.15.63:0/2546877469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbc80fded0 0x7ffbc80fe2a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8004940 tx=0x7ffbb8009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:58.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.775+0000 7ffbccb93700 1 -- 172.21.15.63:0/2546877469 shutdown_connections 2024-03-22T07:12:58.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.775+0000 7ffbccb93700 1 --2- 172.21.15.63:0/2546877469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbc80fe870 0x7ffbc8106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:58.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.775+0000 7ffbccb93700 1 --2- 172.21.15.63:0/2546877469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbc80fded0 0x7ffbc80fe2a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:58.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.775+0000 7ffbccb93700 1 -- 172.21.15.63:0/2546877469 >> 172.21.15.63:0/2546877469 conn(0x7ffbc8072bc0 msgr2=0x7ffbc8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:58.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.775+0000 7ffbccb93700 1 -- 172.21.15.63:0/2546877469 shutdown_connections 2024-03-22T07:12:58.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.775+0000 7ffbccb93700 1 -- 172.21.15.63:0/2546877469 wait complete. 2024-03-22T07:12:58.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.775+0000 7ffbccb93700 1 Processor -- start 2024-03-22T07:12:58.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.776+0000 7ffbccb93700 1 -- start start 2024-03-22T07:12:58.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.776+0000 7ffbccb93700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbc80fded0 0x7ffbc8113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:58.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.776+0000 7ffbccb93700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbc80fe870 0x7ffbc8113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:58.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.776+0000 7ffbccb93700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc81145e0 con 0x7ffbc80fe870 2024-03-22T07:12:58.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.776+0000 7ffbccb93700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc810da20 con 0x7ffbc80fded0 2024-03-22T07:12:58.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.776+0000 7ffbc659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbc80fded0 0x7ffbc8113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:58.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.776+0000 7ffbc5d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbc80fe870 0x7ffbc8113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:58.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.776+0000 7ffbc659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbc80fded0 0x7ffbc8113a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.63:51416/0 (socket says 172.21.15.63:51416) 2024-03-22T07:12:58.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.776+0000 7ffbc659c700 1 -- 172.21.15.63:0/45921411 learned_addr learned my addr 172.21.15.63:0/45921411 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:12:58.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.776+0000 7ffbc5d9b700 1 -- 172.21.15.63:0/45921411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbc80fded0 msgr2=0x7ffbc8113a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:58.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.776+0000 7ffbc5d9b700 1 --2- 172.21.15.63:0/45921411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbc80fded0 0x7ffbc8113a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:58.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.776+0000 7ffbc5d9b700 1 -- 172.21.15.63:0/45921411 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb8005040 con 0x7ffbc80fe870 2024-03-22T07:12:58.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.777+0000 7ffbc5d9b700 1 --2- 172.21.15.63:0/45921411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbc80fe870 0x7ffbc8113f90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc002a50 tx=0x7ffbbc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:58.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.777+0000 7ffbb6ffd700 1 -- 172.21.15.63:0/45921411 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbbc007cc0 con 0x7ffbc80fe870 2024-03-22T07:12:58.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.777+0000 7ffbccb93700 1 -- 172.21.15.63:0/45921411 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc810dd00 con 0x7ffbc80fe870 2024-03-22T07:12:58.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.777+0000 7ffbb6ffd700 1 -- 172.21.15.63:0/45921411 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffbbc00f040 con 0x7ffbc80fe870 2024-03-22T07:12:58.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.777+0000 7ffbb6ffd700 1 -- 172.21.15.63:0/45921411 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbbc009530 con 0x7ffbc80fe870 2024-03-22T07:12:58.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.777+0000 7ffbccb93700 1 -- 172.21.15.63:0/45921411 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc810e240 con 0x7ffbc80fe870 2024-03-22T07:12:58.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.778+0000 7ffbccb93700 1 -- 172.21.15.63:0/45921411 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc8065df0 con 0x7ffbc80fe870 2024-03-22T07:12:58.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.779+0000 7ffbb6ffd700 1 -- 172.21.15.63:0/45921411 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ffbbc0097a0 con 0x7ffbc80fe870 2024-03-22T07:12:58.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.779+0000 7ffbb6ffd700 1 --2- 172.21.15.63:0/45921411 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ffbb00692a0 0x7ffbb006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:12:58.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.780+0000 7ffbc659c700 1 --2- 172.21.15.63:0/45921411 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ffbb00692a0 0x7ffbb006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:12:58.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.780+0000 7ffbb6ffd700 1 -- 172.21.15.63:0/45921411 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ffbbc086aa0 con 0x7ffbc80fe870 2024-03-22T07:12:58.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.780+0000 7ffbc659c700 1 --2- 172.21.15.63:0/45921411 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ffbb00692a0 0x7ffbb006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009fd0 tx=0x7ffbb8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:12:58.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.788+0000 7ffbb6ffd700 1 -- 172.21.15.63:0/45921411 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ffbbc056ba0 con 0x7ffbc80fe870 2024-03-22T07:12:58.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.810+0000 7f326e386700 1 -- 172.21.15.63:0/3018774126 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f326804bed0 con 0x7f3268102e10 2024-03-22T07:12:58.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.811+0000 7f3264ff9700 1 -- 172.21.15.63:0/3018774126 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3258057cc0 con 0x7f3268102e10 2024-03-22T07:12:58.811 INFO:teuthology.orchestra.run.smithi063.stdout:60129542177 2024-03-22T07:12:58.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.813+0000 7f326e386700 1 -- 172.21.15.63:0/3018774126 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f32500692a0 msgr2=0x7f325006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:58.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.813+0000 7f326e386700 1 --2- 172.21.15.63:0/3018774126 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f32500692a0 0x7f325006b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f325c007fd0 tx=0x7f325c009040 comp rx=0 tx=0).stop 2024-03-22T07:12:58.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.813+0000 7f326e386700 1 -- 172.21.15.63:0/3018774126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3268102e10 msgr2=0x7f32680702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:58.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.813+0000 7f326e386700 1 --2- 172.21.15.63:0/3018774126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3268102e10 0x7f32680702c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f325800b3b0 tx=0x7f3258009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:58.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.813+0000 7f326e386700 1 -- 172.21.15.63:0/3018774126 shutdown_connections 2024-03-22T07:12:58.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.813+0000 7f326e386700 1 --2- 172.21.15.63:0/3018774126 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f32500692a0 0x7f325006b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:58.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.813+0000 7f326e386700 1 --2- 172.21.15.63:0/3018774126 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3268102e10 0x7f32680702c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:58.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.813+0000 7f326e386700 1 --2- 172.21.15.63:0/3018774126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32681037b0 0x7f3268070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:58.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.813+0000 7f326e386700 1 -- 172.21.15.63:0/3018774126 >> 172.21.15.63:0/3018774126 conn(0x7f32680f8980 msgr2=0x7f3268107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:58.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.813+0000 7f326e386700 1 -- 172.21.15.63:0/3018774126 shutdown_connections 2024-03-22T07:12:58.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.813+0000 7f326e386700 1 -- 172.21.15.63:0/3018774126 wait complete. 2024-03-22T07:12:58.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.995+0000 7ffbccb93700 1 -- 172.21.15.63:0/45921411 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ffbc810f0d0 con 0x7ffbc80fe870 2024-03-22T07:12:58.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.995+0000 7ffbb6ffd700 1 -- 172.21.15.63:0/45921411 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffbbc056730 con 0x7ffbc80fe870 2024-03-22T07:12:58.995 INFO:teuthology.orchestra.run.smithi063.stdout:81604378652 2024-03-22T07:12:58.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.997+0000 7ffbccb93700 1 -- 172.21.15.63:0/45921411 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ffbb00692a0 msgr2=0x7ffbb006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:58.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.997+0000 7ffbccb93700 1 --2- 172.21.15.63:0/45921411 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ffbb00692a0 0x7ffbb006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009fd0 tx=0x7ffbb8009e30 comp rx=0 tx=0).stop 2024-03-22T07:12:58.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.997+0000 7ffbccb93700 1 -- 172.21.15.63:0/45921411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbc80fe870 msgr2=0x7ffbc8113f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:12:58.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.997+0000 7ffbccb93700 1 --2- 172.21.15.63:0/45921411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbc80fe870 0x7ffbc8113f90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc002a50 tx=0x7ffbbc004600 comp rx=0 tx=0).stop 2024-03-22T07:12:58.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.997+0000 7ffbccb93700 1 -- 172.21.15.63:0/45921411 shutdown_connections 2024-03-22T07:12:58.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.997+0000 7ffbccb93700 1 --2- 172.21.15.63:0/45921411 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ffbb00692a0 0x7ffbb006b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:58.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.997+0000 7ffbccb93700 1 --2- 172.21.15.63:0/45921411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffbc80fded0 0x7ffbc8113a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:58.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.997+0000 7ffbccb93700 1 --2- 172.21.15.63:0/45921411 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ffbc80fe870 0x7ffbc8113f90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:12:58.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.997+0000 7ffbccb93700 1 -- 172.21.15.63:0/45921411 >> 172.21.15.63:0/45921411 conn(0x7ffbc8072bc0 msgr2=0x7ffbc81037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:12:58.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.997+0000 7ffbccb93700 1 -- 172.21.15.63:0/45921411 shutdown_connections 2024-03-22T07:12:58.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:12:58.997+0000 7ffbccb93700 1 -- 172.21.15.63:0/45921411 wait complete. 2024-03-22T07:12:59.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:59 smithi063 ceph-mon[29200]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:59.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:59 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3018774126' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-22T07:12:59.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:59 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/45921411' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-22T07:12:59.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:12:59 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:12:59.763 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378649 got 81604378652 for osd.2 2024-03-22T07:12:59.763 DEBUG:teuthology.parallel:result is None 2024-03-22T07:12:59.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:59 smithi069 ceph-mon[36441]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:12:59.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:59 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/3018774126' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-22T07:12:59.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:59 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/45921411' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-22T07:12:59.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:12:59 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:13:00.192 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542174 got 60129542177 for osd.1 2024-03-22T07:13:00.192 DEBUG:teuthology.parallel:result is None 2024-03-22T07:13:00.192 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-22T07:13:00.193 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-22T07:13:00.459 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:13:01.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:01 smithi069 ceph-mon[36441]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:01.855 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:01 smithi063 ceph-mon[29200]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:02.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.238+0000 7fa3090fc700 1 -- 172.21.15.63:0/2230414648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa304102710 msgr2=0x7fa3040fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:02.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.238+0000 7fa3090fc700 1 --2- 172.21.15.63:0/2230414648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa304102710 0x7fa3040fbd80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8004940 tx=0x7fa2f8009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:02.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.238+0000 7fa3090fc700 1 -- 172.21.15.63:0/2230414648 shutdown_connections 2024-03-22T07:13:02.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.238+0000 7fa3090fc700 1 --2- 172.21.15.63:0/2230414648 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa304102710 0x7fa3040fbd80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:02.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.238+0000 7fa3090fc700 1 --2- 172.21.15.63:0/2230414648 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa304101d70 0x7fa304102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:02.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.238+0000 7fa3090fc700 1 -- 172.21.15.63:0/2230414648 >> 172.21.15.63:0/2230414648 conn(0x7fa304072bc0 msgr2=0x7fa304072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:02.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.238+0000 7fa3090fc700 1 -- 172.21.15.63:0/2230414648 shutdown_connections 2024-03-22T07:13:02.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.238+0000 7fa3090fc700 1 -- 172.21.15.63:0/2230414648 wait complete. 2024-03-22T07:13:02.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.239+0000 7fa3090fc700 1 Processor -- start 2024-03-22T07:13:02.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.239+0000 7fa3090fc700 1 -- start start 2024-03-22T07:13:02.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.239+0000 7fa3090fc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa304101d70 0x7fa30410ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:02.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.239+0000 7fa3090fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa304102710 0x7fa30410f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:02.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.239+0000 7fa3090fc700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30410f750 con 0x7fa304102710 2024-03-22T07:13:02.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.240+0000 7fa3090fc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30410f890 con 0x7fa304101d70 2024-03-22T07:13:02.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.240+0000 7fa30259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa304102710 0x7fa30410f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:02.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.240+0000 7fa30259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa304102710 0x7fa30410f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53336/0 (socket says 172.21.15.63:53336) 2024-03-22T07:13:02.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.240+0000 7fa30259c700 1 -- 172.21.15.63:0/2998921815 learned_addr learned my addr 172.21.15.63:0/2998921815 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:02.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.240+0000 7fa302d9d700 1 --2- 172.21.15.63:0/2998921815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa304101d70 0x7fa30410ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:02.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.240+0000 7fa30259c700 1 -- 172.21.15.63:0/2998921815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa304101d70 msgr2=0x7fa30410ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:02.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.240+0000 7fa30259c700 1 --2- 172.21.15.63:0/2998921815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa304101d70 0x7fa30410ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:02.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.240+0000 7fa30259c700 1 -- 172.21.15.63:0/2998921815 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f8005040 con 0x7fa304102710 2024-03-22T07:13:02.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.240+0000 7fa30259c700 1 --2- 172.21.15.63:0/2998921815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa304102710 0x7fa30410f210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009100 tx=0x7fa2f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:02.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.240+0000 7fa2eb7fe700 1 -- 172.21.15.63:0/2998921815 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2f800e040 con 0x7fa304102710 2024-03-22T07:13:02.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.240+0000 7fa3090fc700 1 -- 172.21.15.63:0/2998921815 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa304117be0 con 0x7fa304102710 2024-03-22T07:13:02.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.240+0000 7fa2eb7fe700 1 -- 172.21.15.63:0/2998921815 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2f8004290 con 0x7fa304102710 2024-03-22T07:13:02.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.240+0000 7fa2eb7fe700 1 -- 172.21.15.63:0/2998921815 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2f801c3f0 con 0x7fa304102710 2024-03-22T07:13:02.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.240+0000 7fa3090fc700 1 -- 172.21.15.63:0/2998921815 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3041181a0 con 0x7fa304102710 2024-03-22T07:13:02.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.241+0000 7fa3090fc700 1 -- 172.21.15.63:0/2998921815 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3040ff080 con 0x7fa304102710 2024-03-22T07:13:02.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.242+0000 7fa2eb7fe700 1 -- 172.21.15.63:0/2998921815 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa2f8013690 con 0x7fa304102710 2024-03-22T07:13:02.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.242+0000 7fa2eb7fe700 1 --2- 172.21.15.63:0/2998921815 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa2ec0692b0 0x7fa2ec06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:02.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.242+0000 7fa2eb7fe700 1 -- 172.21.15.63:0/2998921815 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa2f8087fe0 con 0x7fa304102710 2024-03-22T07:13:02.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.242+0000 7fa302d9d700 1 --2- 172.21.15.63:0/2998921815 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa2ec0692b0 0x7fa2ec06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:02.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.243+0000 7fa302d9d700 1 --2- 172.21.15.63:0/2998921815 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa2ec0692b0 0x7fa2ec06b760 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009bf0 tx=0x7fa2f40091a0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:02.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.247+0000 7fa2eb7fe700 1 -- 172.21.15.63:0/2998921815 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa2f8054630 con 0x7fa304102710 2024-03-22T07:13:02.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.436+0000 7fa3090fc700 1 -- 172.21.15.63:0/2998921815 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa304065e30 con 0x7fa2ec0692b0 2024-03-22T07:13:02.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.437+0000 7fa2eb7fe700 1 -- 172.21.15.63:0/2998921815 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27924 (secure 0 0 0) 0x7fa304065e30 con 0x7fa2ec0692b0 2024-03-22T07:13:02.438 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:13:02.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.440+0000 7fa3090fc700 1 -- 172.21.15.63:0/2998921815 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa2ec0692b0 msgr2=0x7fa2ec06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:02.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.440+0000 7fa3090fc700 1 --2- 172.21.15.63:0/2998921815 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa2ec0692b0 0x7fa2ec06b760 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009bf0 tx=0x7fa2f40091a0 comp rx=0 tx=0).stop 2024-03-22T07:13:02.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.440+0000 7fa3090fc700 1 -- 172.21.15.63:0/2998921815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa304102710 msgr2=0x7fa30410f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:02.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.440+0000 7fa3090fc700 1 --2- 172.21.15.63:0/2998921815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa304102710 0x7fa30410f210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009100 tx=0x7fa2f8009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:02.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.440+0000 7fa3090fc700 1 -- 172.21.15.63:0/2998921815 shutdown_connections 2024-03-22T07:13:02.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.440+0000 7fa3090fc700 1 --2- 172.21.15.63:0/2998921815 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa2ec0692b0 0x7fa2ec06b760 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:02.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.440+0000 7fa3090fc700 1 --2- 172.21.15.63:0/2998921815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa304101d70 0x7fa30410ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:02.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.440+0000 7fa3090fc700 1 --2- 172.21.15.63:0/2998921815 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa304102710 0x7fa30410f210 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:02.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.440+0000 7fa3090fc700 1 -- 172.21.15.63:0/2998921815 >> 172.21.15.63:0/2998921815 conn(0x7fa304072bc0 msgr2=0x7fa3040faf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:02.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.440+0000 7fa3090fc700 1 -- 172.21.15.63:0/2998921815 shutdown_connections 2024-03-22T07:13:02.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:02.440+0000 7fa3090fc700 1 -- 172.21.15.63:0/2998921815 wait complete. 2024-03-22T07:13:02.441 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-03-22T07:13:03.124 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-03-22T07:13:02.236350+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787652,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683260,"statfs":{"total":575970213888,"available":574139658240,"internally_reserved":0,"allocated":4018176,"data_stored":2624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002209"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T07:12:09.639136+0000","last_change":"2024-03-22T07:11:44.661197+0000","last_active":"2024-03-22T07:12:09.639136+0000","last_peered":"2024-03-22T07:12:09.639136+0000","last_clean":"2024-03-22T07:12:09.639136+0000","last_became_active":"2024-03-22T07:11:44.660930+0000","last_became_peered":"2024-03-22T07:11:44.660930+0000","last_unstale":"2024-03-22T07:12:09.639136+0000","last_undegraded":"2024-03-22T07:12:09.639136+0000","last_fullsized":"2024-03-22T07:12:09.639136+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T07:10:56.836018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T07:10:56.836018+0000","last_clean_scrub_stamp":"2024-03-22T07:10:56.836018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:49:34.164300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]}]},{"osd":4,"up_from":35,"seq":150323855377,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 07:12:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.495}]},{"osd":1,"last update":"Fri Mar 22 07:12:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.625},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.66900000000000004}]},{"osd":2,"last update":"Fri Mar 22 07:12:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.46600000000000003}]},{"osd":3,"last update":"Fri Mar 22 07:12:45 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.39800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 07:12:19 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.48699999999999999}]},{"osd":1,"last update":"Fri Mar 22 07:12:19 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.45700000000000002}]},{"osd":2,"last update":"Fri Mar 22 07:12:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.503}]},{"osd":4,"last update":"Fri Mar 22 07:12:47 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.38300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.378,"15min":0.38},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.46200000000000002}]},{"osd":1,"last update":"Fri Mar 22 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.38100000000000001,"15min":0.38400000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.39400000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.42999999999999999}]},{"osd":3,"last update":"Fri Mar 22 07:12:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.504}]},{"osd":4,"last update":"Fri Mar 22 07:12:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 22 07:12:33 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.33900000000000002,"15min":0.33200000000000002},"min":{"1min":0.26700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.35199999999999998,"15min":0.34000000000000002},"min":{"1min":0.247,"5min":0.218,"15min":0.218},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Mar 22 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38600000000000001,"15min":0.38300000000000001},"min":{"1min":0.27400000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.46200000000000002}]},{"osd":3,"last update":"Fri Mar 22 07:12:22 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.753},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":8.0239999999999991,"5min":8.0239999999999991,"15min":8.0239999999999991},"last":0.77200000000000002}]},{"osd":4,"last update":"Fri Mar 22 07:12:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.80600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 07:12:29 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.30299999999999999,"15min":0.29499999999999998},"min":{"1min":0.19700000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.316,"15min":0.30599999999999999},"min":{"1min":0.23300000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.33900000000000002}]},{"osd":2,"last update":"Fri Mar 22 07:13:01 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.34999999999999998,"15min":0.34799999999999998},"min":{"1min":0.26500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.499,"5min":0.501,"15min":0.501},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53000000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.36499999999999999}]},{"osd":3,"last update":"Fri Mar 22 07:12:22 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.752}]},{"osd":4,"last update":"Fri Mar 22 07:12:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.67000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T07:13:03.126 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-22T07:13:03.389 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:13:03.843 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:03 smithi063 ceph-mon[29200]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:03.843 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:03 smithi063 ceph-mon[29200]: from='client.14420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T07:13:04.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:03 smithi069 ceph-mon[36441]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:04.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:03 smithi069 ceph-mon[36441]: from='client.14420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T07:13:05.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.168+0000 7fbcbca60700 1 -- 172.21.15.63:0/967692442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcb8102e00 msgr2=0x7fbcb81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:05.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.168+0000 7fbcbca60700 1 --2- 172.21.15.63:0/967692442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcb8102e00 0x7fbcb81031d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbca8004940 tx=0x7fbca8009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:05.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.168+0000 7fbcbca60700 1 -- 172.21.15.63:0/967692442 shutdown_connections 2024-03-22T07:13:05.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.168+0000 7fbcbca60700 1 --2- 172.21.15.63:0/967692442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcb81037a0 0x7fbcb80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:05.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.168+0000 7fbcbca60700 1 --2- 172.21.15.63:0/967692442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcb8102e00 0x7fbcb81031d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:05.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.168+0000 7fbcbca60700 1 -- 172.21.15.63:0/967692442 >> 172.21.15.63:0/967692442 conn(0x7fbcb80f89b0 msgr2=0x7fbcb80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:05.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.168+0000 7fbcbca60700 1 -- 172.21.15.63:0/967692442 shutdown_connections 2024-03-22T07:13:05.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.168+0000 7fbcbca60700 1 -- 172.21.15.63:0/967692442 wait complete. 2024-03-22T07:13:05.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.169+0000 7fbcbca60700 1 Processor -- start 2024-03-22T07:13:05.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.169+0000 7fbcbca60700 1 -- start start 2024-03-22T07:13:05.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.169+0000 7fbcbca60700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcb8102e00 0x7fbcb80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:05.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.169+0000 7fbcbca60700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcb81037a0 0x7fbcb8070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:05.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.169+0000 7fbcbca60700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb80721c0 con 0x7fbcb8102e00 2024-03-22T07:13:05.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.169+0000 7fbcbca60700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb8070d40 con 0x7fbcb81037a0 2024-03-22T07:13:05.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.169+0000 7fbcb659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcb8102e00 0x7fbcb80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:05.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.169+0000 7fbcb659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcb8102e00 0x7fbcb80702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47994/0 (socket says 172.21.15.63:47994) 2024-03-22T07:13:05.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.169+0000 7fbcb659c700 1 -- 172.21.15.63:0/1910136401 learned_addr learned my addr 172.21.15.63:0/1910136401 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:05.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.169+0000 7fbcb5d9b700 1 --2- 172.21.15.63:0/1910136401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcb81037a0 0x7fbcb8070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:05.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.170+0000 7fbcb659c700 1 -- 172.21.15.63:0/1910136401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcb81037a0 msgr2=0x7fbcb8070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:05.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.170+0000 7fbcb659c700 1 --2- 172.21.15.63:0/1910136401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcb81037a0 0x7fbcb8070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:05.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.170+0000 7fbcb659c700 1 -- 172.21.15.63:0/1910136401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca8005040 con 0x7fbcb8102e00 2024-03-22T07:13:05.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.170+0000 7fbcb659c700 1 --2- 172.21.15.63:0/1910136401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcb8102e00 0x7fbcb80702c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbca800b3b0 tx=0x7fbca8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:05.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.170+0000 7fbca6ffd700 1 -- 172.21.15.63:0/1910136401 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbca800e040 con 0x7fbcb8102e00 2024-03-22T07:13:05.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.170+0000 7fbcbca60700 1 -- 172.21.15.63:0/1910136401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb8070fc0 con 0x7fbcb8102e00 2024-03-22T07:13:05.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.170+0000 7fbca6ffd700 1 -- 172.21.15.63:0/1910136401 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbca8013420 con 0x7fbcb8102e00 2024-03-22T07:13:05.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.170+0000 7fbca6ffd700 1 -- 172.21.15.63:0/1910136401 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbca801c630 con 0x7fbcb8102e00 2024-03-22T07:13:05.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.170+0000 7fbcbca60700 1 -- 172.21.15.63:0/1910136401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb811a130 con 0x7fbcb8102e00 2024-03-22T07:13:05.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.171+0000 7fbcbca60700 1 -- 172.21.15.63:0/1910136401 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb8100110 con 0x7fbcb8102e00 2024-03-22T07:13:05.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.171+0000 7fbca6ffd700 1 -- 172.21.15.63:0/1910136401 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fbca8013590 con 0x7fbcb8102e00 2024-03-22T07:13:05.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.172+0000 7fbca6ffd700 1 --2- 172.21.15.63:0/1910136401 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fbca00692a0 0x7fbca006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:05.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.172+0000 7fbca6ffd700 1 -- 172.21.15.63:0/1910136401 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fbca8088010 con 0x7fbcb8102e00 2024-03-22T07:13:05.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.172+0000 7fbcb5d9b700 1 --2- 172.21.15.63:0/1910136401 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fbca00692a0 0x7fbca006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:05.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.172+0000 7fbcb5d9b700 1 --2- 172.21.15.63:0/1910136401 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fbca00692a0 0x7fbca006b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80719a0 tx=0x7fbcac00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:05.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.176+0000 7fbca6ffd700 1 -- 172.21.15.63:0/1910136401 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbca8054680 con 0x7fbcb8102e00 2024-03-22T07:13:05.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.366+0000 7fbcbca60700 1 -- 172.21.15.63:0/1910136401 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbcb8071750 con 0x7fbca00692a0 2024-03-22T07:13:05.368 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.368+0000 7fbca6ffd700 1 -- 172.21.15.63:0/1910136401 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27924 (secure 0 0 0) 0x7fbcb8071750 con 0x7fbca00692a0 2024-03-22T07:13:05.368 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:13:05.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.370+0000 7fbcbca60700 1 -- 172.21.15.63:0/1910136401 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fbca00692a0 msgr2=0x7fbca006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:05.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.370+0000 7fbcbca60700 1 --2- 172.21.15.63:0/1910136401 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fbca00692a0 0x7fbca006b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80719a0 tx=0x7fbcac00b040 comp rx=0 tx=0).stop 2024-03-22T07:13:05.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.370+0000 7fbcbca60700 1 -- 172.21.15.63:0/1910136401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcb8102e00 msgr2=0x7fbcb80702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:05.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.370+0000 7fbcbca60700 1 --2- 172.21.15.63:0/1910136401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcb8102e00 0x7fbcb80702c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbca800b3b0 tx=0x7fbca8009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:05.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.370+0000 7fbcbca60700 1 -- 172.21.15.63:0/1910136401 shutdown_connections 2024-03-22T07:13:05.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.370+0000 7fbcbca60700 1 --2- 172.21.15.63:0/1910136401 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fbca00692a0 0x7fbca006b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:05.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.370+0000 7fbcbca60700 1 --2- 172.21.15.63:0/1910136401 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbcb8102e00 0x7fbcb80702c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:05.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.370+0000 7fbcbca60700 1 --2- 172.21.15.63:0/1910136401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbcb81037a0 0x7fbcb8070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:05.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.370+0000 7fbcbca60700 1 -- 172.21.15.63:0/1910136401 >> 172.21.15.63:0/1910136401 conn(0x7fbcb80f89b0 msgr2=0x7fbcb81073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:05.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.370+0000 7fbcbca60700 1 -- 172.21.15.63:0/1910136401 shutdown_connections 2024-03-22T07:13:05.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:05.370+0000 7fbcbca60700 1 -- 172.21.15.63:0/1910136401 wait complete. 2024-03-22T07:13:05.371 INFO:teuthology.orchestra.run.smithi063.stderr:dumped all 2024-03-22T07:13:05.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:05 smithi063 ceph-mon[29200]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:05.978 INFO:teuthology.orchestra.run.smithi063.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-03-22T07:13:04.236680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1787652,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":1783680,"kb_avail":560683260,"statfs":{"total":575970213888,"available":574139658240,"internally_reserved":0,"allocated":4018176,"data_stored":2624466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1826488320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001999"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":188,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-22T07:12:09.639136+0000","last_change":"2024-03-22T07:11:44.661197+0000","last_active":"2024-03-22T07:12:09.639136+0000","last_peered":"2024-03-22T07:12:09.639136+0000","last_clean":"2024-03-22T07:12:09.639136+0000","last_became_active":"2024-03-22T07:11:44.660930+0000","last_became_peered":"2024-03-22T07:11:44.660930+0000","last_unstale":"2024-03-22T07:12:09.639136+0000","last_undegraded":"2024-03-22T07:12:09.639136+0000","last_fullsized":"2024-03-22T07:12:09.639136+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-22T07:10:56.836018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-22T07:10:56.836018+0000","last_clean_scrub_stamp":"2024-03-22T07:10:56.836018+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T11:49:34.164300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]}]},{"osd":4,"up_from":35,"seq":150323855377,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 07:12:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.495}]},{"osd":1,"last update":"Fri Mar 22 07:12:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.625},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.66900000000000004}]},{"osd":2,"last update":"Fri Mar 22 07:12:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.46600000000000003}]},{"osd":3,"last update":"Fri Mar 22 07:12:45 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.39800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 07:12:19 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.48699999999999999}]},{"osd":1,"last update":"Fri Mar 22 07:12:19 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.45700000000000002}]},{"osd":2,"last update":"Fri Mar 22 07:12:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.503}]},{"osd":4,"last update":"Fri Mar 22 07:12:47 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.38300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297652,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93447500,"statfs":{"total":95995035648,"available":95690240000,"internally_reserved":0,"allocated":372736,"data_stored":142227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.378,"15min":0.38},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.46200000000000002}]},{"osd":1,"last update":"Fri Mar 22 07:12:58 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.38100000000000001,"15min":0.38400000000000001},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.39200000000000002,"15min":0.39400000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.42999999999999999}]},{"osd":3,"last update":"Fri Mar 22 07:12:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.504}]},{"osd":4,"last update":"Fri Mar 22 07:12:46 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]}]},{"osd":0,"up_from":9,"seq":38654705705,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 22 07:12:33 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.33900000000000002,"15min":0.33200000000000002},"min":{"1min":0.26700000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.35199999999999998,"15min":0.34000000000000002},"min":{"1min":0.247,"5min":0.218,"15min":0.218},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.28299999999999997}]},{"osd":2,"last update":"Fri Mar 22 07:12:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38600000000000001,"15min":0.38300000000000001},"min":{"1min":0.27400000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.33500000000000002}]},{"osd":3,"last update":"Fri Mar 22 07:12:22 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":8.0239999999999991,"5min":8.0239999999999991,"15min":8.0239999999999991},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Mar 22 07:12:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.439},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.45400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":93446920,"statfs":{"total":95995035648,"available":95689646080,"internally_reserved":0,"allocated":966656,"data_stored":732595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 22 07:12:29 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.30299999999999999,"15min":0.29499999999999998},"min":{"1min":0.19700000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.316,"15min":0.30599999999999999},"min":{"1min":0.23300000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.33900000000000002}]},{"osd":2,"last update":"Fri Mar 22 07:13:01 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.34999999999999998,"15min":0.34799999999999998},"min":{"1min":0.26500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.499,"5min":0.501,"15min":0.501},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53000000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.36499999999999999}]},{"osd":3,"last update":"Fri Mar 22 07:12:22 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.752}]},{"osd":4,"last update":"Fri Mar 22 07:12:48 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.67000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-22T07:13:05.979 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-22T07:13:05.979 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-22T07:13:05.979 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-22T07:13:05.979 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-22T07:13:06.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:05 smithi069 ceph-mon[36441]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:06.239 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:13:06.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:06 smithi063 ceph-mon[29200]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T07:13:07.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:06 smithi069 ceph-mon[36441]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-22T07:13:07.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:07 smithi063 ceph-mon[29200]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.805+0000 7fd6b4316700 1 -- 172.21.15.63:0/1739028897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac1037a0 msgr2=0x7fd6ac0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.805+0000 7fd6b4316700 1 --2- 172.21.15.63:0/1739028897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac1037a0 0x7fd6ac0fce10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd69c004940 tx=0x7fd69c009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.805+0000 7fd6b4316700 1 -- 172.21.15.63:0/1739028897 shutdown_connections 2024-03-22T07:13:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.805+0000 7fd6b4316700 1 --2- 172.21.15.63:0/1739028897 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac1037a0 0x7fd6ac0fce10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.805+0000 7fd6b4316700 1 --2- 172.21.15.63:0/1739028897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6ac102e00 0x7fd6ac1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.805+0000 7fd6b4316700 1 -- 172.21.15.63:0/1739028897 >> 172.21.15.63:0/1739028897 conn(0x7fd6ac0f89b0 msgr2=0x7fd6ac0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.806+0000 7fd6b4316700 1 -- 172.21.15.63:0/1739028897 shutdown_connections 2024-03-22T07:13:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.806+0000 7fd6b4316700 1 -- 172.21.15.63:0/1739028897 wait complete. 2024-03-22T07:13:07.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.806+0000 7fd6b4316700 1 Processor -- start 2024-03-22T07:13:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.806+0000 7fd6b4316700 1 -- start start 2024-03-22T07:13:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.806+0000 7fd6b4316700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac102e00 0x7fd6ac0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.806+0000 7fd6b4316700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6ac1037a0 0x7fd6ac070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.806+0000 7fd6b4316700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac072210 con 0x7fd6ac102e00 2024-03-22T07:13:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.806+0000 7fd6b4316700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac070d40 con 0x7fd6ac1037a0 2024-03-22T07:13:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.806+0000 7fd6b20b2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac102e00 0x7fd6ac0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.806+0000 7fd6b20b2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac102e00 0x7fd6ac0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48008/0 (socket says 172.21.15.63:48008) 2024-03-22T07:13:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.806+0000 7fd6b20b2700 1 -- 172.21.15.63:0/1947659020 learned_addr learned my addr 172.21.15.63:0/1947659020 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.806+0000 7fd6b18b1700 1 --2- 172.21.15.63:0/1947659020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6ac1037a0 0x7fd6ac070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.807+0000 7fd6b20b2700 1 -- 172.21.15.63:0/1947659020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6ac1037a0 msgr2=0x7fd6ac070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.807+0000 7fd6b20b2700 1 --2- 172.21.15.63:0/1947659020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6ac1037a0 0x7fd6ac070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.807+0000 7fd6b20b2700 1 -- 172.21.15.63:0/1947659020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c005040 con 0x7fd6ac102e00 2024-03-22T07:13:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.807+0000 7fd6b18b1700 1 --2- 172.21.15.63:0/1947659020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6ac1037a0 0x7fd6ac070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:13:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.807+0000 7fd6b20b2700 1 --2- 172.21.15.63:0/1947659020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac102e00 0x7fd6ac0702c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8002a50 tx=0x7fd6a80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:07.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.807+0000 7fd6a2ffd700 1 -- 172.21.15.63:0/1947659020 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6a8007cc0 con 0x7fd6ac102e00 2024-03-22T07:13:07.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.807+0000 7fd6a2ffd700 1 -- 172.21.15.63:0/1947659020 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6a800f040 con 0x7fd6ac102e00 2024-03-22T07:13:07.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.807+0000 7fd6b4316700 1 -- 172.21.15.63:0/1947659020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac071020 con 0x7fd6ac102e00 2024-03-22T07:13:07.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.807+0000 7fd6a2ffd700 1 -- 172.21.15.63:0/1947659020 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6a800db70 con 0x7fd6ac102e00 2024-03-22T07:13:07.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.807+0000 7fd6b4316700 1 -- 172.21.15.63:0/1947659020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac11a130 con 0x7fd6ac102e00 2024-03-22T07:13:07.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.808+0000 7fd6b4316700 1 -- 172.21.15.63:0/1947659020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ac1052d0 con 0x7fd6ac102e00 2024-03-22T07:13:07.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.808+0000 7fd6a2ffd700 1 -- 172.21.15.63:0/1947659020 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fd6a800dd10 con 0x7fd6ac102e00 2024-03-22T07:13:07.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.809+0000 7fd6a2ffd700 1 --2- 172.21.15.63:0/1947659020 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fd6980692a0 0x7fd69806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:07.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.809+0000 7fd6a2ffd700 1 -- 172.21.15.63:0/1947659020 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd6a8086580 con 0x7fd6ac102e00 2024-03-22T07:13:07.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.809+0000 7fd6b18b1700 1 --2- 172.21.15.63:0/1947659020 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fd6980692a0 0x7fd69806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:07.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.809+0000 7fd6b18b1700 1 --2- 172.21.15.63:0/1947659020 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fd6980692a0 0x7fd69806b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00b3b0 tx=0x7fd69c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:07.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:07.813+0000 7fd6a2ffd700 1 -- 172.21.15.63:0/1947659020 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fd6a8052b70 con 0x7fd6ac102e00 2024-03-22T07:13:08.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:07 smithi069 ceph-mon[36441]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:08.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:08.047+0000 7fd6b4316700 1 -- 172.21.15.63:0/1947659020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd6ac0717a0 con 0x7fd6ac102e00 2024-03-22T07:13:08.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:08.048+0000 7fd6a2ffd700 1 -- 172.21.15.63:0/1947659020 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd6a8056190 con 0x7fd6ac102e00 2024-03-22T07:13:08.048 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:13:08.048 INFO:teuthology.orchestra.run.smithi063.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-22T07:13:08.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:08.050+0000 7fd6b4316700 1 -- 172.21.15.63:0/1947659020 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fd6980692a0 msgr2=0x7fd69806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:08.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:08.050+0000 7fd6b4316700 1 --2- 172.21.15.63:0/1947659020 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fd6980692a0 0x7fd69806b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00b3b0 tx=0x7fd69c009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:08.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:08.050+0000 7fd6b4316700 1 -- 172.21.15.63:0/1947659020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac102e00 msgr2=0x7fd6ac0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:08.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:08.050+0000 7fd6b4316700 1 --2- 172.21.15.63:0/1947659020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac102e00 0x7fd6ac0702c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8002a50 tx=0x7fd6a80046b0 comp rx=0 tx=0).stop 2024-03-22T07:13:08.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:08.050+0000 7fd6b4316700 1 -- 172.21.15.63:0/1947659020 shutdown_connections 2024-03-22T07:13:08.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:08.050+0000 7fd6b4316700 1 --2- 172.21.15.63:0/1947659020 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fd6980692a0 0x7fd69806b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:08.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:08.050+0000 7fd6b4316700 1 --2- 172.21.15.63:0/1947659020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd6ac102e00 0x7fd6ac0702c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:08.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:08.050+0000 7fd6b4316700 1 --2- 172.21.15.63:0/1947659020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6ac1037a0 0x7fd6ac070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:08.050 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:08.050+0000 7fd6b4316700 1 -- 172.21.15.63:0/1947659020 >> 172.21.15.63:0/1947659020 conn(0x7fd6ac0f89b0 msgr2=0x7fd6ac102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:08.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:08.050+0000 7fd6b4316700 1 -- 172.21.15.63:0/1947659020 shutdown_connections 2024-03-22T07:13:08.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:08.050+0000 7fd6b4316700 1 -- 172.21.15.63:0/1947659020 wait complete. 2024-03-22T07:13:08.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:08 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/1947659020' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-22T07:13:08.627 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-22T07:13:08.627 INFO:tasks.cephadm:Setup complete, yielding 2024-03-22T07:13:08.628 INFO:teuthology.run_tasks:Running task print... 2024-03-22T07:13:08.636 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-03-22T07:13:08.636 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T07:13:08.644 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-03-22T07:13:08.645 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-03-22T07:13:08.907 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:13:09.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:08 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/1947659020' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-22T07:13:09.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:09 smithi063 ceph-mon[29200]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:10.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:09 smithi069 ceph-mon[36441]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:10.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.660+0000 7fcf4c795700 1 -- 172.21.15.63:0/3539332024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf440fcf80 msgr2=0x7fcf440fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:10.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.660+0000 7fcf4c795700 1 --2- 172.21.15.63:0/3539332024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf440fcf80 0x7fcf440fd3f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcf38004940 tx=0x7fcf38009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:10.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.660+0000 7fcf4c795700 1 -- 172.21.15.63:0/3539332024 shutdown_connections 2024-03-22T07:13:10.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.660+0000 7fcf4c795700 1 --2- 172.21.15.63:0/3539332024 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf440fcf80 0x7fcf440fd3f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:10.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.660+0000 7fcf4c795700 1 --2- 172.21.15.63:0/3539332024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf440fbcd0 0x7fcf440fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:10.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.660+0000 7fcf4c795700 1 -- 172.21.15.63:0/3539332024 >> 172.21.15.63:0/3539332024 conn(0x7fcf440f7430 msgr2=0x7fcf440f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:10.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.660+0000 7fcf4c795700 1 -- 172.21.15.63:0/3539332024 shutdown_connections 2024-03-22T07:13:10.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.660+0000 7fcf4c795700 1 -- 172.21.15.63:0/3539332024 wait complete. 2024-03-22T07:13:10.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.660+0000 7fcf4c795700 1 Processor -- start 2024-03-22T07:13:10.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.660+0000 7fcf4c795700 1 -- start start 2024-03-22T07:13:10.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.661+0000 7fcf4c795700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf440fbcd0 0x7fcf440ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:10.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.661+0000 7fcf4c795700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf440fcf80 0x7fcf440fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:10.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.661+0000 7fcf4c795700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440fe870 con 0x7fcf440fcf80 2024-03-22T07:13:10.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.661+0000 7fcf4c795700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440fe9e0 con 0x7fcf440fbcd0 2024-03-22T07:13:10.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.661+0000 7fcf49d30700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf440fcf80 0x7fcf440fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:10.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.661+0000 7fcf49d30700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf440fcf80 0x7fcf440fe2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48026/0 (socket says 172.21.15.63:48026) 2024-03-22T07:13:10.662 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.661+0000 7fcf49d30700 1 -- 172.21.15.63:0/2389289671 learned_addr learned my addr 172.21.15.63:0/2389289671 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.661+0000 7fcf49d30700 1 -- 172.21.15.63:0/2389289671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf440fbcd0 msgr2=0x7fcf440ffdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.661+0000 7fcf49d30700 1 --2- 172.21.15.63:0/2389289671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf440fbcd0 0x7fcf440ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.661+0000 7fcf49d30700 1 -- 172.21.15.63:0/2389289671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf38005040 con 0x7fcf440fcf80 2024-03-22T07:13:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.661+0000 7fcf49d30700 1 --2- 172.21.15.63:0/2389289671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf440fcf80 0x7fcf440fe2c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800b3b0 tx=0x7fcf38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.661+0000 7fcf42ffd700 1 -- 172.21.15.63:0/2389289671 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf3800e040 con 0x7fcf440fcf80 2024-03-22T07:13:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.661+0000 7fcf4c795700 1 -- 172.21.15.63:0/2389289671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf440fec60 con 0x7fcf440fcf80 2024-03-22T07:13:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.661+0000 7fcf42ffd700 1 -- 172.21.15.63:0/2389289671 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf3800bcc0 con 0x7fcf440fcf80 2024-03-22T07:13:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.661+0000 7fcf42ffd700 1 -- 172.21.15.63:0/2389289671 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf3801c630 con 0x7fcf440fcf80 2024-03-22T07:13:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.661+0000 7fcf4c795700 1 -- 172.21.15.63:0/2389289671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf44115a90 con 0x7fcf440fcf80 2024-03-22T07:13:10.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.662+0000 7fcf4c795700 1 -- 172.21.15.63:0/2389289671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf44101b10 con 0x7fcf440fcf80 2024-03-22T07:13:10.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.663+0000 7fcf42ffd700 1 -- 172.21.15.63:0/2389289671 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fcf38013980 con 0x7fcf440fcf80 2024-03-22T07:13:10.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.663+0000 7fcf42ffd700 1 --2- 172.21.15.63:0/2389289671 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fcf2c0692a0 0x7fcf2c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:10.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.663+0000 7fcf42ffd700 1 -- 172.21.15.63:0/2389289671 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fcf38087c80 con 0x7fcf440fcf80 2024-03-22T07:13:10.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.663+0000 7fcf4a531700 1 --2- 172.21.15.63:0/2389289671 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fcf2c0692a0 0x7fcf2c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:10.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.664+0000 7fcf4a531700 1 --2- 172.21.15.63:0/2389289671 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fcf2c0692a0 0x7fcf2c06b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf440ffb80 tx=0x7fcf3c009b50 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:10.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.668+0000 7fcf42ffd700 1 -- 172.21.15.63:0/2389289671 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fcf38054270 con 0x7fcf440fcf80 2024-03-22T07:13:10.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.862+0000 7fcf4c795700 1 -- 172.21.15.63:0/2389289671 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fcf441048e0 con 0x7fcf440fcf80 2024-03-22T07:13:10.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.866+0000 7fcf42ffd700 1 -- 172.21.15.63:0/2389289671 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7fcf38057890 con 0x7fcf440fcf80 2024-03-22T07:13:10.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.868+0000 7fcf4c795700 1 -- 172.21.15.63:0/2389289671 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fcf2c0692a0 msgr2=0x7fcf2c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:10.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.868+0000 7fcf4c795700 1 --2- 172.21.15.63:0/2389289671 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fcf2c0692a0 0x7fcf2c06b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf440ffb80 tx=0x7fcf3c009b50 comp rx=0 tx=0).stop 2024-03-22T07:13:10.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.868+0000 7fcf4c795700 1 -- 172.21.15.63:0/2389289671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf440fcf80 msgr2=0x7fcf440fe2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:10.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.868+0000 7fcf4c795700 1 --2- 172.21.15.63:0/2389289671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf440fcf80 0x7fcf440fe2c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800b3b0 tx=0x7fcf38009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:10.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.868+0000 7fcf4c795700 1 -- 172.21.15.63:0/2389289671 shutdown_connections 2024-03-22T07:13:10.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.868+0000 7fcf4c795700 1 --2- 172.21.15.63:0/2389289671 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fcf2c0692a0 0x7fcf2c06b750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:10.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.868+0000 7fcf4c795700 1 --2- 172.21.15.63:0/2389289671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf440fbcd0 0x7fcf440ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:10.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.868+0000 7fcf4c795700 1 --2- 172.21.15.63:0/2389289671 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcf440fcf80 0x7fcf440fe2c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:10.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.868+0000 7fcf4c795700 1 -- 172.21.15.63:0/2389289671 >> 172.21.15.63:0/2389289671 conn(0x7fcf440f7430 msgr2=0x7fcf44065c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:10.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.868+0000 7fcf4c795700 1 -- 172.21.15.63:0/2389289671 shutdown_connections 2024-03-22T07:13:10.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:10.869+0000 7fcf4c795700 1 -- 172.21.15.63:0/2389289671 wait complete. 2024-03-22T07:13:11.627 INFO:teuthology.run_tasks:Running task print... 2024-03-22T07:13:11.636 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-03-22T07:13:11.636 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T07:13:11.643 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-03-22T07:13:11.644 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-22T07:13:11.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:11 smithi063 ceph-mon[29200]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:11.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:11 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2389289671' entity='client.admin' 2024-03-22T07:13:11.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:11 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:13:11.921 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:13:12.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:11 smithi069 ceph-mon[36441]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:12.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:11 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/2389289671' entity='client.admin' 2024-03-22T07:13:12.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:11 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:13:12.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:12 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:12.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:12 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:13:12.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:12 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:13.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:12 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:13.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:12 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:13:13.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:12 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:13.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.614+0000 7efec4d06700 1 -- 172.21.15.63:0/3047845447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efec0070870 msgr2=0x7efec0070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:13.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.614+0000 7efec4d06700 1 --2- 172.21.15.63:0/3047845447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efec0070870 0x7efec0070ce0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7efeb4004940 tx=0x7efeb4009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:13.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.614+0000 7efec4d06700 1 -- 172.21.15.63:0/3047845447 shutdown_connections 2024-03-22T07:13:13.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.614+0000 7efec4d06700 1 --2- 172.21.15.63:0/3047845447 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efec0070870 0x7efec0070ce0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:13.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.614+0000 7efec4d06700 1 --2- 172.21.15.63:0/3047845447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efec0071e20 0x7efec0070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:13.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.614+0000 7efec4d06700 1 -- 172.21.15.63:0/3047845447 >> 172.21.15.63:0/3047845447 conn(0x7efec00f85a0 msgr2=0x7efec00fa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:13.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.614+0000 7efec4d06700 1 -- 172.21.15.63:0/3047845447 shutdown_connections 2024-03-22T07:13:13.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.614+0000 7efec4d06700 1 -- 172.21.15.63:0/3047845447 wait complete. 2024-03-22T07:13:13.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.615+0000 7efec4d06700 1 Processor -- start 2024-03-22T07:13:13.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.615+0000 7efec4d06700 1 -- start start 2024-03-22T07:13:13.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.615+0000 7efec4d06700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efec0070870 0x7efec010fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:13.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.615+0000 7efec4d06700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efec0071e20 0x7efec01101a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:13.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.615+0000 7efec4d06700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec0110830 con 0x7efec0071e20 2024-03-22T07:13:13.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.615+0000 7efec4d06700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec0111020 con 0x7efec0070870 2024-03-22T07:13:13.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.615+0000 7efebdd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efec0071e20 0x7efec01101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:13.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.615+0000 7efebdd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efec0071e20 0x7efec01101a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48044/0 (socket says 172.21.15.63:48044) 2024-03-22T07:13:13.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.615+0000 7efebdd9b700 1 -- 172.21.15.63:0/4109931953 learned_addr learned my addr 172.21.15.63:0/4109931953 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:13.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.615+0000 7efebe59c700 1 --2- 172.21.15.63:0/4109931953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efec0070870 0x7efec010fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:13.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.615+0000 7efebdd9b700 1 -- 172.21.15.63:0/4109931953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efec0070870 msgr2=0x7efec010fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.615+0000 7efebdd9b700 1 --2- 172.21.15.63:0/4109931953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efec0070870 0x7efec010fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.615+0000 7efebdd9b700 1 -- 172.21.15.63:0/4109931953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb4005040 con 0x7efec0071e20 2024-03-22T07:13:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.615+0000 7efebe59c700 1 --2- 172.21.15.63:0/4109931953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efec0070870 0x7efec010fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:13:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.616+0000 7efebdd9b700 1 --2- 172.21.15.63:0/4109931953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efec0071e20 0x7efec01101a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efeb400b3b0 tx=0x7efeb4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.616+0000 7efeaeffd700 1 -- 172.21.15.63:0/4109931953 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efeb400e040 con 0x7efec0071e20 2024-03-22T07:13:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.616+0000 7efeaeffd700 1 -- 172.21.15.63:0/4109931953 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efeb400bcc0 con 0x7efec0071e20 2024-03-22T07:13:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.616+0000 7efec4d06700 1 -- 172.21.15.63:0/4109931953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec01112a0 con 0x7efec0071e20 2024-03-22T07:13:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.616+0000 7efeaeffd700 1 -- 172.21.15.63:0/4109931953 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efeb401c630 con 0x7efec0071e20 2024-03-22T07:13:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.616+0000 7efec4d06700 1 -- 172.21.15.63:0/4109931953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec0111860 con 0x7efec0071e20 2024-03-22T07:13:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.617+0000 7efec4d06700 1 -- 172.21.15.63:0/4109931953 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec005f5f0 con 0x7efec0071e20 2024-03-22T07:13:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.617+0000 7efeaeffd700 1 -- 172.21.15.63:0/4109931953 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7efeb4013980 con 0x7efec0071e20 2024-03-22T07:13:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.618+0000 7efeaeffd700 1 --2- 172.21.15.63:0/4109931953 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7efea80692a0 0x7efea806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.618+0000 7efebe59c700 1 --2- 172.21.15.63:0/4109931953 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7efea80692a0 0x7efea806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.618+0000 7efeaeffd700 1 -- 172.21.15.63:0/4109931953 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7efeb4087bc0 con 0x7efec0071e20 2024-03-22T07:13:13.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.618+0000 7efebe59c700 1 --2- 172.21.15.63:0/4109931953 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7efea80692a0 0x7efea806b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efeb0006fd0 tx=0x7efeb0008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:13.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.622+0000 7efeaeffd700 1 -- 172.21.15.63:0/4109931953 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7efeb4057cc0 con 0x7efec0071e20 2024-03-22T07:13:13.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.834+0000 7efec4d06700 1 -- 172.21.15.63:0/4109931953 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7efec0111b40 con 0x7efea80692a0 2024-03-22T07:13:13.835 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:13 smithi063 ceph-mon[29200]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:13.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.835+0000 7efeaeffd700 1 -- 172.21.15.63:0/4109931953 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7efec0111b40 con 0x7efea80692a0 2024-03-22T07:13:13.835 INFO:teuthology.orchestra.run.smithi063.stdout:Backend: cephadm 2024-03-22T07:13:13.836 INFO:teuthology.orchestra.run.smithi063.stdout:Available: Yes 2024-03-22T07:13:13.836 INFO:teuthology.orchestra.run.smithi063.stdout:Paused: No 2024-03-22T07:13:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.837+0000 7efec4d06700 1 -- 172.21.15.63:0/4109931953 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7efea80692a0 msgr2=0x7efea806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.837+0000 7efec4d06700 1 --2- 172.21.15.63:0/4109931953 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7efea80692a0 0x7efea806b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efeb0006fd0 tx=0x7efeb0008040 comp rx=0 tx=0).stop 2024-03-22T07:13:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.837+0000 7efec4d06700 1 -- 172.21.15.63:0/4109931953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efec0071e20 msgr2=0x7efec01101a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.837+0000 7efec4d06700 1 --2- 172.21.15.63:0/4109931953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efec0071e20 0x7efec01101a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efeb400b3b0 tx=0x7efeb4009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.838+0000 7efec4d06700 1 -- 172.21.15.63:0/4109931953 shutdown_connections 2024-03-22T07:13:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.838+0000 7efec4d06700 1 --2- 172.21.15.63:0/4109931953 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7efea80692a0 0x7efea806b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.838+0000 7efec4d06700 1 --2- 172.21.15.63:0/4109931953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efec0070870 0x7efec010fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.838+0000 7efec4d06700 1 --2- 172.21.15.63:0/4109931953 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efec0071e20 0x7efec01101a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.838+0000 7efec4d06700 1 -- 172.21.15.63:0/4109931953 >> 172.21.15.63:0/4109931953 conn(0x7efec00f85a0 msgr2=0x7efec00f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.838+0000 7efec4d06700 1 -- 172.21.15.63:0/4109931953 shutdown_connections 2024-03-22T07:13:13.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:13.838+0000 7efec4d06700 1 -- 172.21.15.63:0/4109931953 wait complete. 2024-03-22T07:13:14.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:13 smithi069 ceph-mon[36441]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:14.483 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-22T07:13:14.743 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:13:14.865 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:14 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:13:15.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:14 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:13:15.824 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:15 smithi063 ceph-mon[29200]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:15.824 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:15 smithi063 ceph-mon[29200]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:16.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:15 smithi069 ceph-mon[36441]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:16.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:15 smithi069 ceph-mon[36441]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:16.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.335+0000 7f48b8f4d700 1 -- 172.21.15.63:0/3491242668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48b40fce70 msgr2=0x7f48b40fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:16.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.335+0000 7f48b8f4d700 1 --2- 172.21.15.63:0/3491242668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48b40fce70 0x7f48b40fd280 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f48a4004940 tx=0x7f48a4009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:16.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.335+0000 7f48b8f4d700 1 -- 172.21.15.63:0/3491242668 shutdown_connections 2024-03-22T07:13:16.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.336+0000 7f48b8f4d700 1 --2- 172.21.15.63:0/3491242668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48b40fdfe0 0x7f48b40fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:16.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.336+0000 7f48b8f4d700 1 --2- 172.21.15.63:0/3491242668 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48b40fce70 0x7f48b40fd280 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:16.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.336+0000 7f48b8f4d700 1 -- 172.21.15.63:0/3491242668 >> 172.21.15.63:0/3491242668 conn(0x7f48b40f8550 msgr2=0x7f48b40fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:16.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.336+0000 7f48b8f4d700 1 -- 172.21.15.63:0/3491242668 shutdown_connections 2024-03-22T07:13:16.337 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.336+0000 7f48b8f4d700 1 -- 172.21.15.63:0/3491242668 wait complete. 2024-03-22T07:13:16.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.336+0000 7f48b8f4d700 1 Processor -- start 2024-03-22T07:13:16.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.336+0000 7f48b8f4d700 1 -- start start 2024-03-22T07:13:16.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.336+0000 7f48b8f4d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48b40fce70 0x7f48b410d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:16.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.336+0000 7f48b8f4d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48b40fdfe0 0x7f48b410de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:16.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.336+0000 7f48b8f4d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b410e490 con 0x7f48b40fdfe0 2024-03-22T07:13:16.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.336+0000 7f48b8f4d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b410e600 con 0x7f48b40fce70 2024-03-22T07:13:16.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.336+0000 7f48b1d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48b40fdfe0 0x7f48b410de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:16.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.336+0000 7f48b1d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48b40fdfe0 0x7f48b410de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40566/0 (socket says 172.21.15.63:40566) 2024-03-22T07:13:16.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.336+0000 7f48b1d9b700 1 -- 172.21.15.63:0/3658300419 learned_addr learned my addr 172.21.15.63:0/3658300419 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:16.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.336+0000 7f48b259c700 1 --2- 172.21.15.63:0/3658300419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48b40fce70 0x7f48b410d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:16.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.337+0000 7f48b1d9b700 1 -- 172.21.15.63:0/3658300419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48b40fce70 msgr2=0x7f48b410d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:16.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.337+0000 7f48b1d9b700 1 --2- 172.21.15.63:0/3658300419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48b40fce70 0x7f48b410d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:16.338 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.337+0000 7f48b1d9b700 1 -- 172.21.15.63:0/3658300419 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a4005040 con 0x7f48b40fdfe0 2024-03-22T07:13:16.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.337+0000 7f48b1d9b700 1 --2- 172.21.15.63:0/3658300419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48b40fdfe0 0x7f48b410de90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f48a8002a50 tx=0x7f48a80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:16.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.337+0000 7f48a2ffd700 1 -- 172.21.15.63:0/3658300419 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48a8007cc0 con 0x7f48b40fdfe0 2024-03-22T07:13:16.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.337+0000 7f48b8f4d700 1 -- 172.21.15.63:0/3658300419 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b4113060 con 0x7f48b40fdfe0 2024-03-22T07:13:16.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.337+0000 7f48a2ffd700 1 -- 172.21.15.63:0/3658300419 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48a800f040 con 0x7f48b40fdfe0 2024-03-22T07:13:16.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.337+0000 7f48a2ffd700 1 -- 172.21.15.63:0/3658300419 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48a800da30 con 0x7f48b40fdfe0 2024-03-22T07:13:16.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.337+0000 7f48b8f4d700 1 -- 172.21.15.63:0/3658300419 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b4113620 con 0x7f48b40fdfe0 2024-03-22T07:13:16.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.338+0000 7f48b8f4d700 1 -- 172.21.15.63:0/3658300419 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b404c710 con 0x7f48b40fdfe0 2024-03-22T07:13:16.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.339+0000 7f48a2ffd700 1 -- 172.21.15.63:0/3658300419 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f48a800db90 con 0x7f48b40fdfe0 2024-03-22T07:13:16.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.339+0000 7f48a2ffd700 1 --2- 172.21.15.63:0/3658300419 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f489c0692a0 0x7f489c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:16.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.339+0000 7f48a2ffd700 1 -- 172.21.15.63:0/3658300419 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f48a8086c00 con 0x7f48b40fdfe0 2024-03-22T07:13:16.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.339+0000 7f48b259c700 1 --2- 172.21.15.63:0/3658300419 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f489c0692a0 0x7f489c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:16.340 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.339+0000 7f48b259c700 1 --2- 172.21.15.63:0/3658300419 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f489c0692a0 0x7f489c06b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f48a400b970 tx=0x7f48a4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:16.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.343+0000 7f48a2ffd700 1 -- 172.21.15.63:0/3658300419 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f48a808c020 con 0x7f48b40fdfe0 2024-03-22T07:13:16.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.549+0000 7f48b8f4d700 1 -- 172.21.15.63:0/3658300419 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f48b4102340 con 0x7f489c0692a0 2024-03-22T07:13:16.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.558+0000 7f48a2ffd700 1 -- 172.21.15.63:0/3658300419 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2682 (secure 0 0 0) 0x7f48b4102340 con 0x7f489c0692a0 2024-03-22T07:13:16.558 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:13:16.558 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (3m) 2m ago 5m 25.7M - 0.25.0 c8568f914cd2 2f8544438b7c 2024-03-22T07:13:16.558 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (5m) 2m ago 5m 6660k - 17.2.7-726-g9c4e0c34 837124da804f d6c6bbeb6904 2024-03-22T07:13:16.558 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (4m) 68s ago 4m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 30992f8aa88c 2024-03-22T07:13:16.558 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (3m) 2m ago 4m 80.6M - 9.4.7 954c08fa6188 f0b1d2a54439 2024-03-22T07:13:16.558 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:9283 running (6m) 2m ago 6m 471M - 17.2.7-726-g9c4e0c34 837124da804f 422b5e836153 2024-03-22T07:13:16.558 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283 running (4m) 68s ago 4m 434M - 17.2.7-726-g9c4e0c34 837124da804f b48492cc30bb 2024-03-22T07:13:16.558 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (6m) 2m ago 6m 44.6M 2048M 17.2.7-726-g9c4e0c34 837124da804f f92a5bb6caee 2024-03-22T07:13:16.558 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (4m) 68s ago 4m 38.6M 2048M 17.2.7-726-g9c4e0c34 837124da804f 95f6103f7603 2024-03-22T07:13:16.558 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (5m) 2m ago 5m 16.0M - 1.5.0 0da6a335fe13 47501603d3e8 2024-03-22T07:13:16.558 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (4m) 68s ago 4m 16.1M - 1.5.0 0da6a335fe13 49f4ca105fb9 2024-03-22T07:13:16.558 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (3m) 2m ago 3m 39.2M 4288M 17.2.7-726-g9c4e0c34 837124da804f 580f22188a10 2024-03-22T07:13:16.559 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (2m) 2m ago 2m 41.3M 4288M 17.2.7-726-g9c4e0c34 837124da804f fe1ace35c88a 2024-03-22T07:13:16.559 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (2m) 2m ago 2m 37.4M 4288M 17.2.7-726-g9c4e0c34 837124da804f 4cd995265dcf 2024-03-22T07:13:16.559 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (2m) 68s ago 2m 41.4M 5312M 17.2.7-726-g9c4e0c34 837124da804f 45c9a759b76d 2024-03-22T07:13:16.559 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (98s) 68s ago 98s 36.1M 5312M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:13:16.559 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (73s) 68s ago 73s 11.0M 5312M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:13:16.559 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (3m) 2m ago 4m 35.5M - 2.43.0 a07b618ecd1d 827638a5870a 2024-03-22T07:13:16.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.560+0000 7f48b8f4d700 1 -- 172.21.15.63:0/3658300419 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f489c0692a0 msgr2=0x7f489c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:16.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.560+0000 7f48b8f4d700 1 --2- 172.21.15.63:0/3658300419 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f489c0692a0 0x7f489c06b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f48a400b970 tx=0x7f48a4009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:16.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.560+0000 7f48b8f4d700 1 -- 172.21.15.63:0/3658300419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48b40fdfe0 msgr2=0x7f48b410de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:16.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.560+0000 7f48b8f4d700 1 --2- 172.21.15.63:0/3658300419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48b40fdfe0 0x7f48b410de90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f48a8002a50 tx=0x7f48a80046b0 comp rx=0 tx=0).stop 2024-03-22T07:13:16.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.560+0000 7f48b8f4d700 1 -- 172.21.15.63:0/3658300419 shutdown_connections 2024-03-22T07:13:16.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.560+0000 7f48b8f4d700 1 --2- 172.21.15.63:0/3658300419 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f489c0692a0 0x7f489c06b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:16.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.560+0000 7f48b8f4d700 1 --2- 172.21.15.63:0/3658300419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f48b40fce70 0x7f48b410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:16.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.560+0000 7f48b8f4d700 1 --2- 172.21.15.63:0/3658300419 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f48b40fdfe0 0x7f48b410de90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:16.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.560+0000 7f48b8f4d700 1 -- 172.21.15.63:0/3658300419 >> 172.21.15.63:0/3658300419 conn(0x7f48b40f8550 msgr2=0x7f48b40f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:16.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.560+0000 7f48b8f4d700 1 -- 172.21.15.63:0/3658300419 shutdown_connections 2024-03-22T07:13:16.561 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:16.560+0000 7f48b8f4d700 1 -- 172.21.15.63:0/3658300419 wait complete. 2024-03-22T07:13:17.164 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-22T07:13:17.422 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:13:17.852 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:17 smithi063 ceph-mon[29200]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:17.852 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:17 smithi063 ceph-mon[29200]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:18.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:17 smithi069 ceph-mon[36441]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:18.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:17 smithi069 ceph-mon[36441]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:18.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.989+0000 7fb58a253700 1 -- 172.21.15.63:0/3263246185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb584066d90 msgr2=0x7fb5840ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:18.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.989+0000 7fb58a253700 1 --2- 172.21.15.63:0/3263246185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb584066d90 0x7fb5840ff9f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb574004940 tx=0x7fb574009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:18.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.989+0000 7fb58a253700 1 -- 172.21.15.63:0/3263246185 shutdown_connections 2024-03-22T07:13:18.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.989+0000 7fb58a253700 1 --2- 172.21.15.63:0/3263246185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5840fff30 0x7fb584102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:18.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.989+0000 7fb58a253700 1 --2- 172.21.15.63:0/3263246185 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb584066d90 0x7fb5840ff9f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:18.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.989+0000 7fb58a253700 1 -- 172.21.15.63:0/3263246185 >> 172.21.15.63:0/3263246185 conn(0x7fb5840f73e0 msgr2=0x7fb5840f97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:18.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.989+0000 7fb58a253700 1 -- 172.21.15.63:0/3263246185 shutdown_connections 2024-03-22T07:13:18.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.989+0000 7fb58a253700 1 -- 172.21.15.63:0/3263246185 wait complete. 2024-03-22T07:13:18.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.989+0000 7fb58a253700 1 Processor -- start 2024-03-22T07:13:18.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.989+0000 7fb58a253700 1 -- start start 2024-03-22T07:13:18.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.990+0000 7fb58a253700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb584066d90 0x7fb58410d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:18.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.990+0000 7fb58a253700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5840fff30 0x7fb58410df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:18.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.990+0000 7fb58a253700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58410e510 con 0x7fb5840fff30 2024-03-22T07:13:18.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.990+0000 7fb58a253700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58410ed60 con 0x7fb584066d90 2024-03-22T07:13:18.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.990+0000 7fb582ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5840fff30 0x7fb58410df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.990+0000 7fb582ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5840fff30 0x7fb58410df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40588/0 (socket says 172.21.15.63:40588) 2024-03-22T07:13:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.990+0000 7fb582ffd700 1 -- 172.21.15.63:0/527372902 learned_addr learned my addr 172.21.15.63:0/527372902 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.990+0000 7fb5837fe700 1 --2- 172.21.15.63:0/527372902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb584066d90 0x7fb58410d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.990+0000 7fb582ffd700 1 -- 172.21.15.63:0/527372902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb584066d90 msgr2=0x7fb58410d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.990+0000 7fb582ffd700 1 --2- 172.21.15.63:0/527372902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb584066d90 0x7fb58410d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.990+0000 7fb582ffd700 1 -- 172.21.15.63:0/527372902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb574005040 con 0x7fb5840fff30 2024-03-22T07:13:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.990+0000 7fb582ffd700 1 --2- 172.21.15.63:0/527372902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5840fff30 0x7fb58410df10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb578002a50 tx=0x7fb5780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.990+0000 7fb56bfff700 1 -- 172.21.15.63:0/527372902 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb578007cc0 con 0x7fb5840fff30 2024-03-22T07:13:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.990+0000 7fb56bfff700 1 -- 172.21.15.63:0/527372902 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb57800f040 con 0x7fb5840fff30 2024-03-22T07:13:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.990+0000 7fb56bfff700 1 -- 172.21.15.63:0/527372902 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb57800da30 con 0x7fb5840fff30 2024-03-22T07:13:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.990+0000 7fb58a253700 1 -- 172.21.15.63:0/527372902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58410f040 con 0x7fb5840fff30 2024-03-22T07:13:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.990+0000 7fb58a253700 1 -- 172.21.15.63:0/527372902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58410f580 con 0x7fb5840fff30 2024-03-22T07:13:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.991+0000 7fb58a253700 1 -- 172.21.15.63:0/527372902 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5840fb060 con 0x7fb5840fff30 2024-03-22T07:13:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.992+0000 7fb56bfff700 1 -- 172.21.15.63:0/527372902 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb57800db90 con 0x7fb5840fff30 2024-03-22T07:13:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.992+0000 7fb56bfff700 1 --2- 172.21.15.63:0/527372902 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fb56c0692a0 0x7fb56c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:18.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.992+0000 7fb56bfff700 1 -- 172.21.15.63:0/527372902 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb578086b20 con 0x7fb5840fff30 2024-03-22T07:13:18.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.992+0000 7fb5837fe700 1 --2- 172.21.15.63:0/527372902 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fb56c0692a0 0x7fb56c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:18.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.993+0000 7fb5837fe700 1 --2- 172.21.15.63:0/527372902 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fb56c0692a0 0x7fb56c06b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb574004890 tx=0x7fb574009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:18.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:18.997+0000 7fb56bfff700 1 -- 172.21.15.63:0/527372902 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb578056c20 con 0x7fb5840fff30 2024-03-22T07:13:19.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:19.200+0000 7fb58a253700 1 -- 172.21.15.63:0/527372902 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb58410f970 con 0x7fb56c0692a0 2024-03-22T07:13:19.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:19.204+0000 7fb56bfff700 1 -- 172.21.15.63:0/527372902 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1161 (secure 0 0 0) 0x7fb58410f970 con 0x7fb56c0692a0 2024-03-22T07:13:19.204 INFO:teuthology.orchestra.run.smithi063.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-22T07:13:19.204 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-03-22T07:13:19.204 INFO:teuthology.orchestra.run.smithi063.stdout:crash 2/2 2m ago 5m * 2024-03-22T07:13:19.205 INFO:teuthology.orchestra.run.smithi063.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-03-22T07:13:19.205 INFO:teuthology.orchestra.run.smithi063.stdout:mgr 2/2 2m ago 5m count:2 2024-03-22T07:13:19.205 INFO:teuthology.orchestra.run.smithi063.stdout:mon 2/2 2m ago 5m smithi063:172.21.15.63=smithi063;smithi069:172.21.15.69=smithi069;count:2 2024-03-22T07:13:19.205 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-03-22T07:13:19.205 INFO:teuthology.orchestra.run.smithi063.stdout:osd 6 2m ago - 2024-03-22T07:13:19.205 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-03-22T07:13:19.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:19.206+0000 7fb58a253700 1 -- 172.21.15.63:0/527372902 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fb56c0692a0 msgr2=0x7fb56c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:19.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:19.206+0000 7fb58a253700 1 --2- 172.21.15.63:0/527372902 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fb56c0692a0 0x7fb56c06b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb574004890 tx=0x7fb574009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:19.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:19.206+0000 7fb58a253700 1 -- 172.21.15.63:0/527372902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5840fff30 msgr2=0x7fb58410df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:19.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:19.206+0000 7fb58a253700 1 --2- 172.21.15.63:0/527372902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5840fff30 0x7fb58410df10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb578002a50 tx=0x7fb5780046b0 comp rx=0 tx=0).stop 2024-03-22T07:13:19.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:19.206+0000 7fb58a253700 1 -- 172.21.15.63:0/527372902 shutdown_connections 2024-03-22T07:13:19.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:19.206+0000 7fb58a253700 1 --2- 172.21.15.63:0/527372902 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fb56c0692a0 0x7fb56c06b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:19.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:19.206+0000 7fb58a253700 1 --2- 172.21.15.63:0/527372902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb584066d90 0x7fb58410d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:19.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:19.206+0000 7fb58a253700 1 --2- 172.21.15.63:0/527372902 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5840fff30 0x7fb58410df10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:19.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:19.206+0000 7fb58a253700 1 -- 172.21.15.63:0/527372902 >> 172.21.15.63:0/527372902 conn(0x7fb5840f73e0 msgr2=0x7fb5840f77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:19.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:19.207+0000 7fb58a253700 1 -- 172.21.15.63:0/527372902 shutdown_connections 2024-03-22T07:13:19.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:19.207+0000 7fb58a253700 1 -- 172.21.15.63:0/527372902 wait complete. 2024-03-22T07:13:19.788 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:19 smithi063 ceph-mon[29200]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:19.814 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-22T07:13:20.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:19 smithi069 ceph-mon[36441]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:20.076 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:13:20.800 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:20 smithi069 ceph-mon[36441]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:20.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:20 smithi063 ceph-mon[29200]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:21.689 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.689+0000 7f87e1c77700 1 -- 172.21.15.63:0/2862577917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87dc0fdfe0 msgr2=0x7f87dc0fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:21.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.689+0000 7f87e1c77700 1 --2- 172.21.15.63:0/2862577917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87dc0fdfe0 0x7f87dc0fe450 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f87d0004940 tx=0x7f87d0009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:21.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.689+0000 7f87e1c77700 1 -- 172.21.15.63:0/2862577917 shutdown_connections 2024-03-22T07:13:21.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.689+0000 7f87e1c77700 1 --2- 172.21.15.63:0/2862577917 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87dc0fdfe0 0x7f87dc0fe450 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:21.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.689+0000 7f87e1c77700 1 --2- 172.21.15.63:0/2862577917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87dc0fce70 0x7f87dc0fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:21.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.689+0000 7f87e1c77700 1 -- 172.21.15.63:0/2862577917 >> 172.21.15.63:0/2862577917 conn(0x7f87dc0f8550 msgr2=0x7f87dc0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:21.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.689+0000 7f87e1c77700 1 -- 172.21.15.63:0/2862577917 shutdown_connections 2024-03-22T07:13:21.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.689+0000 7f87e1c77700 1 -- 172.21.15.63:0/2862577917 wait complete. 2024-03-22T07:13:21.690 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:21 smithi063 ceph-mon[29200]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:21.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.689+0000 7f87e1c77700 1 Processor -- start 2024-03-22T07:13:21.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.690+0000 7f87e1c77700 1 -- start start 2024-03-22T07:13:21.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.690+0000 7f87e1c77700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87dc0fce70 0x7f87dc10d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:21.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.690+0000 7f87e1c77700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87dc0fdfe0 0x7f87dc10ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:21.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.690+0000 7f87e1c77700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc10e3d0 con 0x7f87dc0fce70 2024-03-22T07:13:21.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.690+0000 7f87e1c77700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc10e540 con 0x7f87dc0fdfe0 2024-03-22T07:13:21.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.690+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87dc0fce70 0x7f87dc10d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:21.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.690+0000 7f87db7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87dc0fce70 0x7f87dc10d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40602/0 (socket says 172.21.15.63:40602) 2024-03-22T07:13:21.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.690+0000 7f87db7fe700 1 -- 172.21.15.63:0/559958069 learned_addr learned my addr 172.21.15.63:0/559958069 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:21.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.690+0000 7f87daffd700 1 --2- 172.21.15.63:0/559958069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87dc0fdfe0 0x7f87dc10ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:21.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.690+0000 7f87db7fe700 1 -- 172.21.15.63:0/559958069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87dc0fdfe0 msgr2=0x7f87dc10ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:21.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.690+0000 7f87db7fe700 1 --2- 172.21.15.63:0/559958069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87dc0fdfe0 0x7f87dc10ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:21.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.690+0000 7f87db7fe700 1 -- 172.21.15.63:0/559958069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d0005040 con 0x7f87dc0fce70 2024-03-22T07:13:21.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.691+0000 7f87db7fe700 1 --2- 172.21.15.63:0/559958069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87dc0fce70 0x7f87dc10d890 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00eb40 tx=0x7f87cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:21.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.691+0000 7f87c3fff700 1 -- 172.21.15.63:0/559958069 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87cc007cc0 con 0x7f87dc0fce70 2024-03-22T07:13:21.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.691+0000 7f87c3fff700 1 -- 172.21.15.63:0/559958069 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87cc00f040 con 0x7f87dc0fce70 2024-03-22T07:13:21.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.691+0000 7f87e1c77700 1 -- 172.21.15.63:0/559958069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87dc112f40 con 0x7f87dc0fce70 2024-03-22T07:13:21.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.691+0000 7f87c3fff700 1 -- 172.21.15.63:0/559958069 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87cc00ca10 con 0x7f87dc0fce70 2024-03-22T07:13:21.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.691+0000 7f87e1c77700 1 -- 172.21.15.63:0/559958069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc1133f0 con 0x7f87dc0fce70 2024-03-22T07:13:21.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.692+0000 7f87e1c77700 1 -- 172.21.15.63:0/559958069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87dc04c710 con 0x7f87dc0fce70 2024-03-22T07:13:21.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.692+0000 7f87c3fff700 1 -- 172.21.15.63:0/559958069 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f87cc00cc80 con 0x7f87dc0fce70 2024-03-22T07:13:21.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.693+0000 7f87c3fff700 1 --2- 172.21.15.63:0/559958069 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f87c40692a0 0x7f87c406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:21.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.693+0000 7f87c3fff700 1 -- 172.21.15.63:0/559958069 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f87cc086a00 con 0x7f87dc0fce70 2024-03-22T07:13:21.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.693+0000 7f87daffd700 1 --2- 172.21.15.63:0/559958069 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f87c40692a0 0x7f87c406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:21.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.693+0000 7f87daffd700 1 --2- 172.21.15.63:0/559958069 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f87c40692a0 0x7f87c406b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009b30 tx=0x7f87d0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:21.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.697+0000 7f87c3fff700 1 -- 172.21.15.63:0/559958069 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f87cc04e190 con 0x7f87dc0fce70 2024-03-22T07:13:21.901 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.902+0000 7f87e1c77700 1 -- 172.21.15.63:0/559958069 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f87dc1135e0 con 0x7f87c40692a0 2024-03-22T07:13:21.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.903+0000 7f87c3fff700 1 -- 172.21.15.63:0/559958069 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+145 (secure 0 0 0) 0x7f87dc1135e0 con 0x7f87c40692a0 2024-03-22T07:13:21.903 INFO:teuthology.orchestra.run.smithi063.stdout:HOST ADDR LABELS STATUS 2024-03-22T07:13:21.903 INFO:teuthology.orchestra.run.smithi063.stdout:smithi063 172.21.15.63 2024-03-22T07:13:21.903 INFO:teuthology.orchestra.run.smithi063.stdout:smithi069 172.21.15.69 2024-03-22T07:13:21.903 INFO:teuthology.orchestra.run.smithi063.stdout:2 hosts in cluster 2024-03-22T07:13:21.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.905+0000 7f87e1c77700 1 -- 172.21.15.63:0/559958069 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f87c40692a0 msgr2=0x7f87c406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:21.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.905+0000 7f87e1c77700 1 --2- 172.21.15.63:0/559958069 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f87c40692a0 0x7f87c406b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009b30 tx=0x7f87d0009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:21.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.905+0000 7f87e1c77700 1 -- 172.21.15.63:0/559958069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87dc0fce70 msgr2=0x7f87dc10d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:21.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.905+0000 7f87e1c77700 1 --2- 172.21.15.63:0/559958069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87dc0fce70 0x7f87dc10d890 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00eb40 tx=0x7f87cc00b040 comp rx=0 tx=0).stop 2024-03-22T07:13:21.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.905+0000 7f87e1c77700 1 -- 172.21.15.63:0/559958069 shutdown_connections 2024-03-22T07:13:21.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.905+0000 7f87e1c77700 1 --2- 172.21.15.63:0/559958069 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f87c40692a0 0x7f87c406b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:21.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.905+0000 7f87e1c77700 1 --2- 172.21.15.63:0/559958069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f87dc0fce70 0x7f87dc10d890 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:21.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.905+0000 7f87e1c77700 1 --2- 172.21.15.63:0/559958069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87dc0fdfe0 0x7f87dc10ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:21.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.905+0000 7f87e1c77700 1 -- 172.21.15.63:0/559958069 >> 172.21.15.63:0/559958069 conn(0x7f87dc0f8550 msgr2=0x7f87dc0f9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:21.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.905+0000 7f87e1c77700 1 -- 172.21.15.63:0/559958069 shutdown_connections 2024-03-22T07:13:21.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:21.905+0000 7f87e1c77700 1 -- 172.21.15.63:0/559958069 wait complete. 2024-03-22T07:13:22.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:21 smithi069 ceph-mon[36441]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:22.559 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-22T07:13:22.817 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:13:23.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:23 smithi063 ceph-mon[29200]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:23.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:23 smithi063 ceph-mon[29200]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:24.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:23 smithi069 ceph-mon[36441]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:24.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:23 smithi069 ceph-mon[36441]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:24.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.556+0000 7fe57fa1c700 1 -- 172.21.15.63:0/1714590515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5780feda0 msgr2=0x7fe5780faee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:24.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.556+0000 7fe57fa1c700 1 --2- 172.21.15.63:0/1714590515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5780feda0 0x7fe5780faee0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe568004940 tx=0x7fe568009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:24.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.556+0000 7fe57fa1c700 1 -- 172.21.15.63:0/1714590515 shutdown_connections 2024-03-22T07:13:24.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.556+0000 7fe57fa1c700 1 --2- 172.21.15.63:0/1714590515 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5780feda0 0x7fe5780faee0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:24.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.556+0000 7fe57fa1c700 1 --2- 172.21.15.63:0/1714590515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5780fe450 0x7fe5780fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:24.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.556+0000 7fe57fa1c700 1 -- 172.21.15.63:0/1714590515 >> 172.21.15.63:0/1714590515 conn(0x7fe5780f6c10 msgr2=0x7fe5780f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:24.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.556+0000 7fe57fa1c700 1 -- 172.21.15.63:0/1714590515 shutdown_connections 2024-03-22T07:13:24.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.556+0000 7fe57fa1c700 1 -- 172.21.15.63:0/1714590515 wait complete. 2024-03-22T07:13:24.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.557+0000 7fe57fa1c700 1 Processor -- start 2024-03-22T07:13:24.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.557+0000 7fe57fa1c700 1 -- start start 2024-03-22T07:13:24.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.557+0000 7fe57fa1c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5780fe450 0x7fe57810d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:24.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.557+0000 7fe57fa1c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5780feda0 0x7fe57810de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:24.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.557+0000 7fe57fa1c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57810e4e0 con 0x7fe5780fe450 2024-03-22T07:13:24.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.557+0000 7fe57fa1c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe578112e80 con 0x7fe5780feda0 2024-03-22T07:13:24.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.557+0000 7fe57d7b8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5780fe450 0x7fe57810d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:24.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.557+0000 7fe57d7b8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5780fe450 0x7fe57810d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40802/0 (socket says 172.21.15.63:40802) 2024-03-22T07:13:24.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.557+0000 7fe57d7b8700 1 -- 172.21.15.63:0/2164475208 learned_addr learned my addr 172.21.15.63:0/2164475208 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:24.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.557+0000 7fe57d7b8700 1 -- 172.21.15.63:0/2164475208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5780feda0 msgr2=0x7fe57810de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:24.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.557+0000 7fe57d7b8700 1 --2- 172.21.15.63:0/2164475208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5780feda0 0x7fe57810de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:24.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.557+0000 7fe57d7b8700 1 -- 172.21.15.63:0/2164475208 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe568005040 con 0x7fe5780fe450 2024-03-22T07:13:24.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.557+0000 7fe57d7b8700 1 --2- 172.21.15.63:0/2164475208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5780fe450 0x7fe57810d910 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe57400b790 tx=0x7fe574009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:24.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.557+0000 7fe56dffb700 1 -- 172.21.15.63:0/2164475208 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe57400f770 con 0x7fe5780fe450 2024-03-22T07:13:24.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.557+0000 7fe56dffb700 1 -- 172.21.15.63:0/2164475208 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe57400fda0 con 0x7fe5780fe450 2024-03-22T07:13:24.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.557+0000 7fe57fa1c700 1 -- 172.21.15.63:0/2164475208 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5781130e0 con 0x7fe5780fe450 2024-03-22T07:13:24.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.558+0000 7fe56dffb700 1 -- 172.21.15.63:0/2164475208 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe57400d310 con 0x7fe5780fe450 2024-03-22T07:13:24.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.558+0000 7fe57fa1c700 1 -- 172.21.15.63:0/2164475208 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe578113650 con 0x7fe5780fe450 2024-03-22T07:13:24.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.559+0000 7fe57fa1c700 1 -- 172.21.15.63:0/2164475208 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57804c710 con 0x7fe5780fe450 2024-03-22T07:13:24.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.559+0000 7fe56dffb700 1 -- 172.21.15.63:0/2164475208 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe574010420 con 0x7fe5780fe450 2024-03-22T07:13:24.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.559+0000 7fe56dffb700 1 --2- 172.21.15.63:0/2164475208 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe5640692a0 0x7fe56406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:24.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.559+0000 7fe56dffb700 1 -- 172.21.15.63:0/2164475208 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe574086d80 con 0x7fe5780fe450 2024-03-22T07:13:24.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.559+0000 7fe57cfb7700 1 --2- 172.21.15.63:0/2164475208 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe5640692a0 0x7fe56406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:24.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.560+0000 7fe57cfb7700 1 --2- 172.21.15.63:0/2164475208 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe5640692a0 0x7fe56406b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe568004f90 tx=0x7fe568009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:24.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.564+0000 7fe56dffb700 1 -- 172.21.15.63:0/2164475208 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe574056f20 con 0x7fe5780fe450 2024-03-22T07:13:24.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.769+0000 7fe57fa1c700 1 -- 172.21.15.63:0/2164475208 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5781024d0 con 0x7fe5640692a0 2024-03-22T07:13:24.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.771+0000 7fe56dffb700 1 -- 172.21.15.63:0/2164475208 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7fe5781024d0 con 0x7fe5640692a0 2024-03-22T07:13:24.771 INFO:teuthology.orchestra.run.smithi063.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-22T07:13:24.771 INFO:teuthology.orchestra.run.smithi063.stdout:smithi063 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5344005B400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-22T07:13:24.771 INFO:teuthology.orchestra.run.smithi063.stdout:smithi069 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y0400BGN 372G No 72s ago Has a FileSystem, LVM detected 2024-03-22T07:13:24.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.773+0000 7fe57fa1c700 1 -- 172.21.15.63:0/2164475208 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe5640692a0 msgr2=0x7fe56406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:24.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.773+0000 7fe57fa1c700 1 --2- 172.21.15.63:0/2164475208 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe5640692a0 0x7fe56406b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe568004f90 tx=0x7fe568009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:24.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.773+0000 7fe57fa1c700 1 -- 172.21.15.63:0/2164475208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5780fe450 msgr2=0x7fe57810d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:24.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.773+0000 7fe57fa1c700 1 --2- 172.21.15.63:0/2164475208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5780fe450 0x7fe57810d910 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe57400b790 tx=0x7fe574009250 comp rx=0 tx=0).stop 2024-03-22T07:13:24.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.773+0000 7fe57fa1c700 1 -- 172.21.15.63:0/2164475208 shutdown_connections 2024-03-22T07:13:24.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.773+0000 7fe57fa1c700 1 --2- 172.21.15.63:0/2164475208 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe5640692a0 0x7fe56406b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:24.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.773+0000 7fe57fa1c700 1 --2- 172.21.15.63:0/2164475208 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5780fe450 0x7fe57810d910 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:24.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.773+0000 7fe57fa1c700 1 --2- 172.21.15.63:0/2164475208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5780feda0 0x7fe57810de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:24.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.773+0000 7fe57fa1c700 1 -- 172.21.15.63:0/2164475208 >> 172.21.15.63:0/2164475208 conn(0x7fe5780f6c10 msgr2=0x7fe5780f81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:24.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.773+0000 7fe57fa1c700 1 -- 172.21.15.63:0/2164475208 shutdown_connections 2024-03-22T07:13:24.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:24.773+0000 7fe57fa1c700 1 -- 172.21.15.63:0/2164475208 wait complete. 2024-03-22T07:13:25.442 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T07:13:25.459 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-03-22T07:13:25.459 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-03-22T07:13:25.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:25 smithi063 ceph-mon[29200]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:25.721 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:13:26.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:25 smithi069 ceph-mon[36441]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:26.837 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:26 smithi063 ceph-mon[29200]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:27.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:26 smithi069 ceph-mon[36441]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:27.359 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.355+0000 7fb6a698b700 1 -- 172.21.15.63:0/1193164469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6a00fcf80 msgr2=0x7fb6a00fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.355+0000 7fb6a698b700 1 --2- 172.21.15.63:0/1193164469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6a00fcf80 0x7fb6a00fd3f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb694004940 tx=0x7fb694009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.355+0000 7fb6a698b700 1 -- 172.21.15.63:0/1193164469 shutdown_connections 2024-03-22T07:13:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.355+0000 7fb6a698b700 1 --2- 172.21.15.63:0/1193164469 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6a00fcf80 0x7fb6a00fd3f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.355+0000 7fb6a698b700 1 --2- 172.21.15.63:0/1193164469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6a00fbcd0 0x7fb6a00fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.356+0000 7fb6a698b700 1 -- 172.21.15.63:0/1193164469 >> 172.21.15.63:0/1193164469 conn(0x7fb6a00f7430 msgr2=0x7fb6a00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.356+0000 7fb6a698b700 1 -- 172.21.15.63:0/1193164469 shutdown_connections 2024-03-22T07:13:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.356+0000 7fb6a698b700 1 -- 172.21.15.63:0/1193164469 wait complete. 2024-03-22T07:13:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.356+0000 7fb6a698b700 1 Processor -- start 2024-03-22T07:13:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.356+0000 7fb6a698b700 1 -- start start 2024-03-22T07:13:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.356+0000 7fb6a698b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6a00fbcd0 0x7fb6a010fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.356+0000 7fb6a698b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6a00fcf80 0x7fb6a01100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.356+0000 7fb6a698b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a01106b0 con 0x7fb6a00fbcd0 2024-03-22T07:13:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.356+0000 7fb6a698b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a0110820 con 0x7fb6a00fcf80 2024-03-22T07:13:27.360 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.356+0000 7fb69ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6a00fbcd0 0x7fb6a010fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.357+0000 7fb69ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6a00fbcd0 0x7fb6a010fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40814/0 (socket says 172.21.15.63:40814) 2024-03-22T07:13:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.357+0000 7fb69ffff700 1 -- 172.21.15.63:0/3739141589 learned_addr learned my addr 172.21.15.63:0/3739141589 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.357+0000 7fb69f7fe700 1 --2- 172.21.15.63:0/3739141589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6a00fcf80 0x7fb6a01100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.357+0000 7fb69ffff700 1 -- 172.21.15.63:0/3739141589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6a00fcf80 msgr2=0x7fb6a01100b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.357+0000 7fb69ffff700 1 --2- 172.21.15.63:0/3739141589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6a00fcf80 0x7fb6a01100b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.357+0000 7fb69ffff700 1 -- 172.21.15.63:0/3739141589 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb694005040 con 0x7fb6a00fbcd0 2024-03-22T07:13:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.357+0000 7fb69ffff700 1 --2- 172.21.15.63:0/3739141589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6a00fbcd0 0x7fb6a010fb70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb69000c830 tx=0x7fb690004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.357+0000 7fb69cff9700 1 -- 172.21.15.63:0/3739141589 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb69000a760 con 0x7fb6a00fbcd0 2024-03-22T07:13:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.357+0000 7fb69cff9700 1 -- 172.21.15.63:0/3739141589 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb69000ad90 con 0x7fb6a00fbcd0 2024-03-22T07:13:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.357+0000 7fb6a698b700 1 -- 172.21.15.63:0/3739141589 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a0115230 con 0x7fb6a00fbcd0 2024-03-22T07:13:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.357+0000 7fb69cff9700 1 -- 172.21.15.63:0/3739141589 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6900113a0 con 0x7fb6a00fbcd0 2024-03-22T07:13:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.357+0000 7fb6a698b700 1 -- 172.21.15.63:0/3739141589 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a01157f0 con 0x7fb6a00fbcd0 2024-03-22T07:13:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.358+0000 7fb6a698b700 1 -- 172.21.15.63:0/3739141589 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a0101910 con 0x7fb6a00fbcd0 2024-03-22T07:13:27.361 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.359+0000 7fb69cff9700 1 -- 172.21.15.63:0/3739141589 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb690012400 con 0x7fb6a00fbcd0 2024-03-22T07:13:27.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.359+0000 7fb69cff9700 1 --2- 172.21.15.63:0/3739141589 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fb6880692a0 0x7fb68806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:27.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.359+0000 7fb69cff9700 1 -- 172.21.15.63:0/3739141589 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb690086ef0 con 0x7fb6a00fbcd0 2024-03-22T07:13:27.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.359+0000 7fb69f7fe700 1 --2- 172.21.15.63:0/3739141589 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fb6880692a0 0x7fb68806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:27.362 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.360+0000 7fb69f7fe700 1 --2- 172.21.15.63:0/3739141589 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fb6880692a0 0x7fb68806b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb69400b3b0 tx=0x7fb694009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:27.363 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.363+0000 7fb69cff9700 1 -- 172.21.15.63:0/3739141589 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb690056ff0 con 0x7fb6a00fbcd0 2024-03-22T07:13:27.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:27.587+0000 7fb6a698b700 1 -- 172.21.15.63:0/3739141589 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6a002b340 con 0x7fb6880692a0 2024-03-22T07:13:27.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:27 smithi063 ceph-mon[29200]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:28.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:27 smithi069 ceph-mon[36441]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:28.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:28 smithi063 ceph-mon[29200]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:28.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:28 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-22T07:13:29.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:28 smithi069 ceph-mon[36441]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:13:29.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:28 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-22T07:13:29.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:29.619+0000 7fb69cff9700 1 -- 172.21.15.63:0/3739141589 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb6a002b340 con 0x7fb6880692a0 2024-03-22T07:13:29.622 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:29.622+0000 7fb6a698b700 1 -- 172.21.15.63:0/3739141589 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fb6880692a0 msgr2=0x7fb68806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:29.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:29.622+0000 7fb6a698b700 1 --2- 172.21.15.63:0/3739141589 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fb6880692a0 0x7fb68806b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb69400b3b0 tx=0x7fb694009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:29.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:29.622+0000 7fb6a698b700 1 -- 172.21.15.63:0/3739141589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6a00fbcd0 msgr2=0x7fb6a010fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:29.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:29.622+0000 7fb6a698b700 1 --2- 172.21.15.63:0/3739141589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6a00fbcd0 0x7fb6a010fb70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb69000c830 tx=0x7fb690004470 comp rx=0 tx=0).stop 2024-03-22T07:13:29.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:29.622+0000 7fb6a698b700 1 -- 172.21.15.63:0/3739141589 shutdown_connections 2024-03-22T07:13:29.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:29.622+0000 7fb6a698b700 1 --2- 172.21.15.63:0/3739141589 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fb6880692a0 0x7fb68806b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:29.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:29.622+0000 7fb6a698b700 1 --2- 172.21.15.63:0/3739141589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb6a00fbcd0 0x7fb6a010fb70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:29.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:29.622+0000 7fb6a698b700 1 --2- 172.21.15.63:0/3739141589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6a00fcf80 0x7fb6a01100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:29.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:29.622+0000 7fb6a698b700 1 -- 172.21.15.63:0/3739141589 >> 172.21.15.63:0/3739141589 conn(0x7fb6a00f7430 msgr2=0x7fb6a00667a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:29.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:29.623+0000 7fb6a698b700 1 -- 172.21.15.63:0/3739141589 shutdown_connections 2024-03-22T07:13:29.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:29.623+0000 7fb6a698b700 1 -- 172.21.15.63:0/3739141589 wait complete. 2024-03-22T07:13:29.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:29 smithi063 ceph-mon[29200]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:29.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:29 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-22T07:13:29.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:29 smithi063 ceph-mon[29200]: osdmap e42: 6 total, 6 up, 6 in 2024-03-22T07:13:29.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:29 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-22T07:13:29.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:29 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:13:29.635 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:29 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063[29196]: 2024-03-22T07:13:29.602+0000 7fa8aa4db700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T07:13:30.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:29 smithi069 ceph-mon[36441]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:30.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:29 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-22T07:13:30.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:29 smithi069 ceph-mon[36441]: osdmap e42: 6 total, 6 up, 6 in 2024-03-22T07:13:30.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:29 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-22T07:13:30.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:29 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:13:30.251 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-22T07:13:30.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:30 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-22T07:13:30.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:30 smithi063 ceph-mon[29200]: osdmap e43: 6 total, 6 up, 6 in 2024-03-22T07:13:30.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:30 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-22T07:13:30.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:30 smithi063 ceph-mon[29200]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T07:13:30.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:30 smithi063 ceph-mon[29200]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-22T07:13:30.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:30 smithi063 ceph-mon[29200]: osdmap e44: 6 total, 6 up, 6 in 2024-03-22T07:13:30.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:30 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-22T07:13:30.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:30 smithi063 ceph-mon[29200]: fsmap cephfs:0 2024-03-22T07:13:30.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:30 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:30.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:30 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:13:30.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:30 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:30.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:30 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:13:30.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:30 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:30.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:30 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.oytapz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:13:30.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:30 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.oytapz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T07:13:30.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:30 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:31.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:30 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-22T07:13:31.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:30 smithi069 ceph-mon[36441]: osdmap e43: 6 total, 6 up, 6 in 2024-03-22T07:13:31.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:30 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-22T07:13:31.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:30 smithi069 ceph-mon[36441]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T07:13:31.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:30 smithi069 ceph-mon[36441]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-22T07:13:31.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:30 smithi069 ceph-mon[36441]: osdmap e44: 6 total, 6 up, 6 in 2024-03-22T07:13:31.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:30 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-22T07:13:31.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:30 smithi069 ceph-mon[36441]: fsmap cephfs:0 2024-03-22T07:13:31.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:30 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:31.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:30 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:13:31.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:30 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:31.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:30 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:13:31.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:30 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:31.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:30 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.oytapz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:13:31.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:30 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.oytapz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T07:13:31.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:30 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:31.458 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:13:31.631 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:31 smithi063 ceph-mon[29200]: Saving service mds.cephfs spec with placement count:4 2024-03-22T07:13:31.631 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:31 smithi063 ceph-mon[29200]: Deploying daemon mds.cephfs.smithi063.oytapz on smithi063 2024-03-22T07:13:31.631 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:31 smithi063 ceph-mon[29200]: pgmap v175: 65 pgs: 15 active+clean, 1 creating+activating, 9 creating+peering, 40 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:31.631 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:31 smithi063 ceph-mon[29200]: osdmap e45: 6 total, 6 up, 6 in 2024-03-22T07:13:32.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:31 smithi069 ceph-mon[36441]: Saving service mds.cephfs spec with placement count:4 2024-03-22T07:13:32.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:31 smithi069 ceph-mon[36441]: Deploying daemon mds.cephfs.smithi063.oytapz on smithi063 2024-03-22T07:13:32.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:31 smithi069 ceph-mon[36441]: pgmap v175: 65 pgs: 15 active+clean, 1 creating+activating, 9 creating+peering, 40 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:32.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:31 smithi069 ceph-mon[36441]: osdmap e45: 6 total, 6 up, 6 in 2024-03-22T07:13:32.687 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:32 smithi063 ceph-mon[29200]: osdmap e46: 6 total, 6 up, 6 in 2024-03-22T07:13:32.687 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:32 smithi063 ceph-mon[29200]: pgmap v178: 65 pgs: 49 active+clean, 1 creating+activating, 9 creating+peering, 6 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:33.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:32 smithi069 ceph-mon[36441]: osdmap e46: 6 total, 6 up, 6 in 2024-03-22T07:13:33.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:32 smithi069 ceph-mon[36441]: pgmap v178: 65 pgs: 49 active+clean, 1 creating+activating, 9 creating+peering, 6 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:33.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.156+0000 7f2b0d211700 1 -- 172.21.15.63:0/336248048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b080fed90 msgr2=0x7f2b080faed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:33.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.156+0000 7f2b0d211700 1 --2- 172.21.15.63:0/336248048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b080fed90 0x7f2b080faed0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2afc004940 tx=0x7f2afc009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:33.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.156+0000 7f2b0d211700 1 -- 172.21.15.63:0/336248048 shutdown_connections 2024-03-22T07:13:33.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.156+0000 7f2b0d211700 1 --2- 172.21.15.63:0/336248048 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b080fed90 0x7f2b080faed0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:33.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.156+0000 7f2b0d211700 1 --2- 172.21.15.63:0/336248048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b080fe440 0x7f2b080fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:33.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.156+0000 7f2b0d211700 1 -- 172.21.15.63:0/336248048 >> 172.21.15.63:0/336248048 conn(0x7f2b080f6be0 msgr2=0x7f2b080f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:33.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.156+0000 7f2b0d211700 1 -- 172.21.15.63:0/336248048 shutdown_connections 2024-03-22T07:13:33.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.157+0000 7f2b0d211700 1 -- 172.21.15.63:0/336248048 wait complete. 2024-03-22T07:13:33.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.157+0000 7f2b0d211700 1 Processor -- start 2024-03-22T07:13:33.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.157+0000 7f2b0d211700 1 -- start start 2024-03-22T07:13:33.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.157+0000 7f2b0d211700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b080fe440 0x7f2b080fda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:33.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.157+0000 7f2b0d211700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b080fed90 0x7f2b080fc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:33.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.157+0000 7f2b0d211700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b080fc640 con 0x7f2b080fe440 2024-03-22T07:13:33.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.157+0000 7f2b0d211700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b080fc7b0 con 0x7f2b080fed90 2024-03-22T07:13:33.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.157+0000 7f2b06d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b080fe440 0x7f2b080fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:33.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.157+0000 7f2b06d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b080fe440 0x7f2b080fda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40828/0 (socket says 172.21.15.63:40828) 2024-03-22T07:13:33.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.157+0000 7f2b06d9d700 1 -- 172.21.15.63:0/2742209662 learned_addr learned my addr 172.21.15.63:0/2742209662 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:33.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.157+0000 7f2b0659c700 1 --2- 172.21.15.63:0/2742209662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b080fed90 0x7f2b080fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:33.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.158+0000 7f2b06d9d700 1 -- 172.21.15.63:0/2742209662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b080fed90 msgr2=0x7f2b080fc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:33.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.158+0000 7f2b06d9d700 1 --2- 172.21.15.63:0/2742209662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b080fed90 0x7f2b080fc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:33.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.158+0000 7f2b06d9d700 1 -- 172.21.15.63:0/2742209662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc005040 con 0x7f2b080fe440 2024-03-22T07:13:33.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.158+0000 7f2b0659c700 1 --2- 172.21.15.63:0/2742209662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b080fed90 0x7f2b080fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:13:33.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.158+0000 7f2b06d9d700 1 --2- 172.21.15.63:0/2742209662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b080fe440 0x7f2b080fda50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2af800b790 tx=0x7f2af8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:33.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.158+0000 7f2aef7fe700 1 -- 172.21.15.63:0/2742209662 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2af800f770 con 0x7f2b080fe440 2024-03-22T07:13:33.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.158+0000 7f2aef7fe700 1 -- 172.21.15.63:0/2742209662 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2af800fda0 con 0x7f2b080fe440 2024-03-22T07:13:33.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.158+0000 7f2b0d211700 1 -- 172.21.15.63:0/2742209662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b080fca90 con 0x7f2b080fe440 2024-03-22T07:13:33.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.158+0000 7f2aef7fe700 1 -- 172.21.15.63:0/2742209662 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2af800d310 con 0x7f2b080fe440 2024-03-22T07:13:33.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.158+0000 7f2b0d211700 1 -- 172.21.15.63:0/2742209662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b08115a50 con 0x7f2b080fe440 2024-03-22T07:13:33.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.159+0000 7f2b0d211700 1 -- 172.21.15.63:0/2742209662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b0804c710 con 0x7f2b080fe440 2024-03-22T07:13:33.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.159+0000 7f2aef7fe700 1 -- 172.21.15.63:0/2742209662 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f2af8010420 con 0x7f2b080fe440 2024-03-22T07:13:33.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.160+0000 7f2aef7fe700 1 --2- 172.21.15.63:0/2742209662 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f2af00692a0 0x7f2af006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:33.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.160+0000 7f2b0659c700 1 --2- 172.21.15.63:0/2742209662 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f2af00692a0 0x7f2af006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:33.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.160+0000 7f2aef7fe700 1 -- 172.21.15.63:0/2742209662 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f2af80870a0 con 0x7f2b080fe440 2024-03-22T07:13:33.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.160+0000 7f2b0659c700 1 --2- 172.21.15.63:0/2742209662 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f2af00692a0 0x7f2af006b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00b3b0 tx=0x7f2afc009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:33.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.165+0000 7f2aef7fe700 1 -- 172.21.15.63:0/2742209662 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2af8056f00 con 0x7f2b080fe440 2024-03-22T07:13:33.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.409+0000 7f2b0d211700 1 -- 172.21.15.63:0/2742209662 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2b08115d40 con 0x7f2b080fe440 2024-03-22T07:13:33.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.410+0000 7f2aef7fe700 1 -- 172.21.15.63:0/2742209662 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7f2af8056a90 con 0x7f2b080fe440 2024-03-22T07:13:33.410 INFO:teuthology.orchestra.run.smithi063.stdout:e2 2024-03-22T07:13:33.410 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:13:33.410 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:13:33.410 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-03-22T07:13:33.410 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:13:33.411 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:13:33.411 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-03-22T07:13:33.411 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 2 2024-03-22T07:13:33.411 INFO:teuthology.orchestra.run.smithi063.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-22T07:13:33.411 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-03-22T07:13:29.602905+0000 2024-03-22T07:13:33.411 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-03-22T07:13:29.602965+0000 2024-03-22T07:13:33.411 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-03-22T07:13:33.411 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-03-22T07:13:33.411 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-03-22T07:13:33.411 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-03-22T07:13:33.411 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-03-22T07:13:33.411 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-03-22T07:13:33.411 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-03-22T07:13:33.411 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 0 2024-03-22T07:13:33.412 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:13:33.412 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 1 2024-03-22T07:13:33.412 INFO:teuthology.orchestra.run.smithi063.stdout:in 2024-03-22T07:13:33.412 INFO:teuthology.orchestra.run.smithi063.stdout:up {} 2024-03-22T07:13:33.412 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-03-22T07:13:33.412 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-03-22T07:13:33.412 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-03-22T07:13:33.412 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-03-22T07:13:33.412 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-03-22T07:13:33.412 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-03-22T07:13:33.412 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-03-22T07:13:33.412 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 0 2024-03-22T07:13:33.412 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:13:33.412 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:13:33.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.412+0000 7f2b0d211700 1 -- 172.21.15.63:0/2742209662 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f2af00692a0 msgr2=0x7f2af006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:33.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.412+0000 7f2b0d211700 1 --2- 172.21.15.63:0/2742209662 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f2af00692a0 0x7f2af006b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00b3b0 tx=0x7f2afc009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:33.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.412+0000 7f2b0d211700 1 -- 172.21.15.63:0/2742209662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b080fe440 msgr2=0x7f2b080fda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:33.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.412+0000 7f2b0d211700 1 --2- 172.21.15.63:0/2742209662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b080fe440 0x7f2b080fda50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2af800b790 tx=0x7f2af8009250 comp rx=0 tx=0).stop 2024-03-22T07:13:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.412+0000 7f2b0d211700 1 -- 172.21.15.63:0/2742209662 shutdown_connections 2024-03-22T07:13:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.412+0000 7f2b0d211700 1 --2- 172.21.15.63:0/2742209662 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f2af00692a0 0x7f2af006b750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.412+0000 7f2b0d211700 1 --2- 172.21.15.63:0/2742209662 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b080fe440 0x7f2b080fda50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.412+0000 7f2b0d211700 1 --2- 172.21.15.63:0/2742209662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b080fed90 0x7f2b080fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.412+0000 7f2b0d211700 1 -- 172.21.15.63:0/2742209662 >> 172.21.15.63:0/2742209662 conn(0x7f2b080f6be0 msgr2=0x7f2b080f8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.412+0000 7f2b0d211700 1 -- 172.21.15.63:0/2742209662 shutdown_connections 2024-03-22T07:13:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:33.412+0000 7f2b0d211700 1 -- 172.21.15.63:0/2742209662 wait complete. 2024-03-22T07:13:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 2 2024-03-22T07:13:33.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:33 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2742209662' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:13:34.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:33 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/2742209662' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:13:34.147 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T07:13:34.156 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-03-22T07:13:34.156 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-03-22T07:13:34.506 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:13:35.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:35 smithi063 ceph-mon[29200]: pgmap v179: 65 pgs: 58 active+clean, 1 creating+activating, 6 creating+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:35.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:35 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:35.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:35 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:35.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:35 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:35.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:35 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi069.wwtsjn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:13:35.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:35 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi069.wwtsjn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T07:13:35.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:35 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:35.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:35 smithi063 ceph-mon[29200]: Deploying daemon mds.cephfs.smithi069.wwtsjn on smithi069 2024-03-22T07:13:35.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:35 smithi063 ceph-mon[29200]: mds.? [v2:172.21.15.63:6826/1314414011,v1:172.21.15.63:6827/1314414011] up:boot 2024-03-22T07:13:35.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:35 smithi063 ceph-mon[29200]: daemon mds.cephfs.smithi063.oytapz assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-22T07:13:35.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:35 smithi063 ceph-mon[29200]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T07:13:35.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:35 smithi063 ceph-mon[29200]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-22T07:13:35.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:35 smithi063 ceph-mon[29200]: Cluster is now healthy 2024-03-22T07:13:35.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:35 smithi063 ceph-mon[29200]: fsmap cephfs:0 1 up:standby 2024-03-22T07:13:35.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:35 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.oytapz"}]: dispatch 2024-03-22T07:13:35.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:35 smithi063 ceph-mon[29200]: fsmap cephfs:1 {0=cephfs.smithi063.oytapz=up:creating} 2024-03-22T07:13:35.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:35 smithi063 ceph-mon[29200]: daemon mds.cephfs.smithi063.oytapz is now active in filesystem cephfs as rank 0 2024-03-22T07:13:35.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:35 smithi069 ceph-mon[36441]: pgmap v179: 65 pgs: 58 active+clean, 1 creating+activating, 6 creating+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:13:35.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:35 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:35.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:35 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:35.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:35 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:35.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:35 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi069.wwtsjn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:13:35.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:35 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi069.wwtsjn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T07:13:35.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:35 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:35.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:35 smithi069 ceph-mon[36441]: Deploying daemon mds.cephfs.smithi069.wwtsjn on smithi069 2024-03-22T07:13:35.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:35 smithi069 ceph-mon[36441]: mds.? [v2:172.21.15.63:6826/1314414011,v1:172.21.15.63:6827/1314414011] up:boot 2024-03-22T07:13:35.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:35 smithi069 ceph-mon[36441]: daemon mds.cephfs.smithi063.oytapz assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-22T07:13:35.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:35 smithi069 ceph-mon[36441]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T07:13:35.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:35 smithi069 ceph-mon[36441]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-22T07:13:35.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:35 smithi069 ceph-mon[36441]: Cluster is now healthy 2024-03-22T07:13:35.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:35 smithi069 ceph-mon[36441]: fsmap cephfs:0 1 up:standby 2024-03-22T07:13:35.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:35 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.oytapz"}]: dispatch 2024-03-22T07:13:35.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:35 smithi069 ceph-mon[36441]: fsmap cephfs:1 {0=cephfs.smithi063.oytapz=up:creating} 2024-03-22T07:13:35.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:35 smithi069 ceph-mon[36441]: daemon mds.cephfs.smithi063.oytapz is now active in filesystem cephfs as rank 0 2024-03-22T07:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.198+0000 7f3e4a11e700 1 -- 172.21.15.63:0/1898046660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e440fcf80 msgr2=0x7f3e440fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.198+0000 7f3e4a11e700 1 --2- 172.21.15.63:0/1898046660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e440fcf80 0x7f3e440fd3f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c004940 tx=0x7f3e2c009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.198+0000 7f3e4a11e700 1 -- 172.21.15.63:0/1898046660 shutdown_connections 2024-03-22T07:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.198+0000 7f3e4a11e700 1 --2- 172.21.15.63:0/1898046660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e440fcf80 0x7f3e440fd3f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.198+0000 7f3e4a11e700 1 --2- 172.21.15.63:0/1898046660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e440fbcd0 0x7f3e440fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.198+0000 7f3e4a11e700 1 -- 172.21.15.63:0/1898046660 >> 172.21.15.63:0/1898046660 conn(0x7f3e440f7430 msgr2=0x7f3e440f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.198+0000 7f3e4a11e700 1 -- 172.21.15.63:0/1898046660 shutdown_connections 2024-03-22T07:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.198+0000 7f3e4a11e700 1 -- 172.21.15.63:0/1898046660 wait complete. 2024-03-22T07:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.198+0000 7f3e4a11e700 1 Processor -- start 2024-03-22T07:13:36.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.199+0000 7f3e4a11e700 1 -- start start 2024-03-22T07:13:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.199+0000 7f3e4a11e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e440fbcd0 0x7f3e4410c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.199+0000 7f3e4a11e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e440fcf80 0x7f3e4410ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.199+0000 7f3e4a11e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4410d480 con 0x7f3e440fbcd0 2024-03-22T07:13:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.199+0000 7f3e4a11e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4410dcd0 con 0x7f3e440fcf80 2024-03-22T07:13:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.199+0000 7f3e437fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e440fbcd0 0x7f3e4410c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.199+0000 7f3e437fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e440fbcd0 0x7f3e4410c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35234/0 (socket says 172.21.15.63:35234) 2024-03-22T07:13:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.199+0000 7f3e437fe700 1 -- 172.21.15.63:0/3728948278 learned_addr learned my addr 172.21.15.63:0/3728948278 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.199+0000 7f3e42ffd700 1 --2- 172.21.15.63:0/3728948278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e440fcf80 0x7f3e4410ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.199+0000 7f3e437fe700 1 -- 172.21.15.63:0/3728948278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e440fcf80 msgr2=0x7f3e4410ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.199+0000 7f3e437fe700 1 --2- 172.21.15.63:0/3728948278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e440fcf80 0x7f3e4410ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.199+0000 7f3e437fe700 1 -- 172.21.15.63:0/3728948278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2c005040 con 0x7f3e440fbcd0 2024-03-22T07:13:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.199+0000 7f3e437fe700 1 --2- 172.21.15.63:0/3728948278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e440fbcd0 0x7f3e4410c940 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c002a50 tx=0x7f3e3c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:36.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.200+0000 7f3e3bfff700 1 -- 172.21.15.63:0/3728948278 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e3c007cc0 con 0x7f3e440fbcd0 2024-03-22T07:13:36.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.200+0000 7f3e4a11e700 1 -- 172.21.15.63:0/3728948278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e4410dfb0 con 0x7f3e440fbcd0 2024-03-22T07:13:36.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.200+0000 7f3e3bfff700 1 -- 172.21.15.63:0/3728948278 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e3c00f040 con 0x7f3e440fbcd0 2024-03-22T07:13:36.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.200+0000 7f3e3bfff700 1 -- 172.21.15.63:0/3728948278 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e3c00da30 con 0x7f3e440fbcd0 2024-03-22T07:13:36.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.200+0000 7f3e4a11e700 1 -- 172.21.15.63:0/3728948278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e4410e4f0 con 0x7f3e440fbcd0 2024-03-22T07:13:36.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.201+0000 7f3e4a11e700 1 -- 172.21.15.63:0/3728948278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e4405f5f0 con 0x7f3e440fbcd0 2024-03-22T07:13:36.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.201+0000 7f3e3bfff700 1 -- 172.21.15.63:0/3728948278 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f3e3c00db90 con 0x7f3e440fbcd0 2024-03-22T07:13:36.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.201+0000 7f3e3bfff700 1 --2- 172.21.15.63:0/3728948278 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f3e300692a0 0x7f3e3006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:36.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.202+0000 7f3e42ffd700 1 --2- 172.21.15.63:0/3728948278 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f3e300692a0 0x7f3e3006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:36.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.202+0000 7f3e3bfff700 1 -- 172.21.15.63:0/3728948278 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3e3c086f40 con 0x7f3e440fbcd0 2024-03-22T07:13:36.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.202+0000 7f3e42ffd700 1 --2- 172.21.15.63:0/3728948278 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f3e300692a0 0x7f3e3006b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00b970 tx=0x7f3e2c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:36.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.206+0000 7f3e3bfff700 1 -- 172.21.15.63:0/3728948278 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3e3c056c80 con 0x7f3e440fbcd0 2024-03-22T07:13:36.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.446+0000 7f3e4a11e700 1 -- 172.21.15.63:0/3728948278 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f3e4410e790 con 0x7f3e440fbcd0 2024-03-22T07:13:36.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.634+0000 7f3e3bfff700 1 -- 172.21.15.63:0/3728948278 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f3e3c056810 con 0x7f3e440fbcd0 2024-03-22T07:13:36.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.636+0000 7f3e4a11e700 1 -- 172.21.15.63:0/3728948278 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f3e300692a0 msgr2=0x7f3e3006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:36.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.636+0000 7f3e4a11e700 1 --2- 172.21.15.63:0/3728948278 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f3e300692a0 0x7f3e3006b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00b970 tx=0x7f3e2c009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:36.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.636+0000 7f3e4a11e700 1 -- 172.21.15.63:0/3728948278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e440fbcd0 msgr2=0x7f3e4410c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:36.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.636+0000 7f3e4a11e700 1 --2- 172.21.15.63:0/3728948278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e440fbcd0 0x7f3e4410c940 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c002a50 tx=0x7f3e3c0046b0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.637+0000 7f3e4a11e700 1 -- 172.21.15.63:0/3728948278 shutdown_connections 2024-03-22T07:13:36.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.637+0000 7f3e4a11e700 1 --2- 172.21.15.63:0/3728948278 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f3e300692a0 0x7f3e3006b750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.637+0000 7f3e4a11e700 1 --2- 172.21.15.63:0/3728948278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3e440fbcd0 0x7f3e4410c940 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.637+0000 7f3e4a11e700 1 --2- 172.21.15.63:0/3728948278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e440fcf80 0x7f3e4410ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:36.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.637+0000 7f3e4a11e700 1 -- 172.21.15.63:0/3728948278 >> 172.21.15.63:0/3728948278 conn(0x7f3e440f7430 msgr2=0x7f3e440f8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:36.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.637+0000 7f3e4a11e700 1 -- 172.21.15.63:0/3728948278 shutdown_connections 2024-03-22T07:13:36.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:36.637+0000 7f3e4a11e700 1 -- 172.21.15.63:0/3728948278 wait complete. 2024-03-22T07:13:36.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:36 smithi063 ceph-mon[29200]: mds.? [v2:172.21.15.63:6826/1314414011,v1:172.21.15.63:6827/1314414011] up:active 2024-03-22T07:13:36.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:36 smithi063 ceph-mon[29200]: fsmap cephfs:1 {0=cephfs.smithi063.oytapz=up:active} 2024-03-22T07:13:36.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:36 smithi063 ceph-mon[29200]: pgmap v180: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 617 B/s wr, 4 op/s 2024-03-22T07:13:36.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:36 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3728948278' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-03-22T07:13:36.924 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:36 smithi069 ceph-mon[36441]: mds.? [v2:172.21.15.63:6826/1314414011,v1:172.21.15.63:6827/1314414011] up:active 2024-03-22T07:13:36.924 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:36 smithi069 ceph-mon[36441]: fsmap cephfs:1 {0=cephfs.smithi063.oytapz=up:active} 2024-03-22T07:13:36.924 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:36 smithi069 ceph-mon[36441]: pgmap v180: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 617 B/s wr, 4 op/s 2024-03-22T07:13:36.924 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:36 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/3728948278' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-03-22T07:13:37.275 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T07:13:37.286 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-03-22T07:13:37.286 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-03-22T07:13:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:37 smithi063 ceph-mon[29200]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-22T07:13:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:37 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3728948278' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-03-22T07:13:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:37 smithi063 ceph-mon[29200]: fsmap cephfs:1 {0=cephfs.smithi063.oytapz=up:active} 2024-03-22T07:13:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:37 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:37 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:37 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:37 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.xeelpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:13:37.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:37 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.xeelpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T07:13:37.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:37 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:37.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:37 smithi063 ceph-mon[29200]: Deploying daemon mds.cephfs.smithi063.xeelpk on smithi063 2024-03-22T07:13:38.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:37 smithi069 ceph-mon[36441]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-22T07:13:38.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:37 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/3728948278' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-03-22T07:13:38.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:37 smithi069 ceph-mon[36441]: fsmap cephfs:1 {0=cephfs.smithi063.oytapz=up:active} 2024-03-22T07:13:38.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:37 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:38.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:37 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:38.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:37 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:38.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:37 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.xeelpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:13:38.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:37 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.xeelpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T07:13:38.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:37 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:38.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:37 smithi069 ceph-mon[36441]: Deploying daemon mds.cephfs.smithi063.xeelpk on smithi063 2024-03-22T07:13:38.503 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:13:38.656 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:38 smithi063 ceph-mon[29200]: mds.? [v2:172.21.15.69:6824/3798247699,v1:172.21.15.69:6825/3798247699] up:boot 2024-03-22T07:13:38.657 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:38 smithi063 ceph-mon[29200]: daemon mds.cephfs.smithi069.wwtsjn assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-22T07:13:38.657 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:38 smithi063 ceph-mon[29200]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-22T07:13:38.657 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:38 smithi063 ceph-mon[29200]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-22T07:13:38.657 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:38 smithi063 ceph-mon[29200]: fsmap cephfs:1 {0=cephfs.smithi063.oytapz=up:active} 1 up:standby 2024-03-22T07:13:38.657 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:38 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.wwtsjn"}]: dispatch 2024-03-22T07:13:38.657 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:38 smithi063 ceph-mon[29200]: fsmap cephfs:2 {0=cephfs.smithi063.oytapz=up:active,1=cephfs.smithi069.wwtsjn=up:creating} 2024-03-22T07:13:38.657 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:38 smithi063 ceph-mon[29200]: daemon mds.cephfs.smithi069.wwtsjn is now active in filesystem cephfs as rank 1 2024-03-22T07:13:38.657 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:38 smithi063 ceph-mon[29200]: pgmap v181: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 4 op/s 2024-03-22T07:13:38.979 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:38 smithi069 ceph-mon[36441]: mds.? [v2:172.21.15.69:6824/3798247699,v1:172.21.15.69:6825/3798247699] up:boot 2024-03-22T07:13:38.979 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:38 smithi069 ceph-mon[36441]: daemon mds.cephfs.smithi069.wwtsjn assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-22T07:13:38.979 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:38 smithi069 ceph-mon[36441]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-22T07:13:38.979 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:38 smithi069 ceph-mon[36441]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-22T07:13:38.979 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:38 smithi069 ceph-mon[36441]: fsmap cephfs:1 {0=cephfs.smithi063.oytapz=up:active} 1 up:standby 2024-03-22T07:13:38.979 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:38 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.wwtsjn"}]: dispatch 2024-03-22T07:13:38.979 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:38 smithi069 ceph-mon[36441]: fsmap cephfs:2 {0=cephfs.smithi063.oytapz=up:active,1=cephfs.smithi069.wwtsjn=up:creating} 2024-03-22T07:13:38.980 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:38 smithi069 ceph-mon[36441]: daemon mds.cephfs.smithi069.wwtsjn is now active in filesystem cephfs as rank 1 2024-03-22T07:13:38.980 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:38 smithi069 ceph-mon[36441]: pgmap v181: 65 pgs: 65 active+clean; 578 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 4 op/s 2024-03-22T07:13:39.679 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:39 smithi063 ceph-mon[29200]: mds.? [v2:172.21.15.69:6824/3798247699,v1:172.21.15.69:6825/3798247699] up:active 2024-03-22T07:13:39.679 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:39 smithi063 ceph-mon[29200]: fsmap cephfs:2 {0=cephfs.smithi063.oytapz=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2024-03-22T07:13:40.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:39 smithi069 ceph-mon[36441]: mds.? [v2:172.21.15.69:6824/3798247699,v1:172.21.15.69:6825/3798247699] up:active 2024-03-22T07:13:40.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:39 smithi069 ceph-mon[36441]: fsmap cephfs:2 {0=cephfs.smithi063.oytapz=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2024-03-22T07:13:40.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.230+0000 7f5261484700 1 -- 172.21.15.63:0/920994886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f525c0fff00 msgr2=0x7f525c102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:40.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.230+0000 7f5261484700 1 --2- 172.21.15.63:0/920994886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f525c0fff00 0x7f525c102320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5250004940 tx=0x7f5250009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:40.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.230+0000 7f5261484700 1 -- 172.21.15.63:0/920994886 shutdown_connections 2024-03-22T07:13:40.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.230+0000 7f5261484700 1 --2- 172.21.15.63:0/920994886 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f525c0fff00 0x7f525c102320 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:40.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.230+0000 7f5261484700 1 --2- 172.21.15.63:0/920994886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f525c0662e0 0x7f525c0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:40.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.230+0000 7f5261484700 1 -- 172.21.15.63:0/920994886 >> 172.21.15.63:0/920994886 conn(0x7f525c0f7430 msgr2=0x7f525c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:40.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.230+0000 7f5261484700 1 -- 172.21.15.63:0/920994886 shutdown_connections 2024-03-22T07:13:40.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.230+0000 7f5261484700 1 -- 172.21.15.63:0/920994886 wait complete. 2024-03-22T07:13:40.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.231+0000 7f5261484700 1 Processor -- start 2024-03-22T07:13:40.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.231+0000 7f5261484700 1 -- start start 2024-03-22T07:13:40.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.231+0000 7f5261484700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f525c0662e0 0x7f525c0fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:40.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.231+0000 7f5261484700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f525c0fff00 0x7f525c0fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:40.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.231+0000 7f5261484700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c0fbe70 con 0x7f525c0662e0 2024-03-22T07:13:40.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.231+0000 7f5261484700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c0fbfe0 con 0x7f525c0fff00 2024-03-22T07:13:40.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.231+0000 7f525affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f525c0662e0 0x7f525c0fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:40.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.231+0000 7f525affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f525c0662e0 0x7f525c0fd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35254/0 (socket says 172.21.15.63:35254) 2024-03-22T07:13:40.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.231+0000 7f525affd700 1 -- 172.21.15.63:0/4055205704 learned_addr learned my addr 172.21.15.63:0/4055205704 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:40.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.231+0000 7f525a7fc700 1 --2- 172.21.15.63:0/4055205704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f525c0fff00 0x7f525c0fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:40.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.231+0000 7f525affd700 1 -- 172.21.15.63:0/4055205704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f525c0fff00 msgr2=0x7f525c0fb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:40.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.231+0000 7f525affd700 1 --2- 172.21.15.63:0/4055205704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f525c0fff00 0x7f525c0fb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:40.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.231+0000 7f525affd700 1 -- 172.21.15.63:0/4055205704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5250005040 con 0x7f525c0662e0 2024-03-22T07:13:40.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.231+0000 7f525affd700 1 --2- 172.21.15.63:0/4055205704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f525c0662e0 0x7f525c0fd3f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f524c002a80 tx=0x7f524c004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:40.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.232+0000 7f52437fe700 1 -- 172.21.15.63:0/4055205704 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f524c004e80 con 0x7f525c0662e0 2024-03-22T07:13:40.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.232+0000 7f52437fe700 1 -- 172.21.15.63:0/4055205704 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f524c00f040 con 0x7f525c0662e0 2024-03-22T07:13:40.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.232+0000 7f52437fe700 1 -- 172.21.15.63:0/4055205704 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f524c00daa0 con 0x7f525c0662e0 2024-03-22T07:13:40.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.232+0000 7f5261484700 1 -- 172.21.15.63:0/4055205704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525c0fc2c0 con 0x7f525c0662e0 2024-03-22T07:13:40.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.232+0000 7f5261484700 1 -- 172.21.15.63:0/4055205704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525c115aa0 con 0x7f525c0662e0 2024-03-22T07:13:40.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.233+0000 7f5261484700 1 -- 172.21.15.63:0/4055205704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525c05f5f0 con 0x7f525c0662e0 2024-03-22T07:13:40.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.233+0000 7f52437fe700 1 -- 172.21.15.63:0/4055205704 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f524c007cc0 con 0x7f525c0662e0 2024-03-22T07:13:40.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.234+0000 7f52437fe700 1 --2- 172.21.15.63:0/4055205704 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f52440692a0 0x7f524406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:40.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.234+0000 7f525a7fc700 1 --2- 172.21.15.63:0/4055205704 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f52440692a0 0x7f524406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:40.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.234+0000 7f52437fe700 1 -- 172.21.15.63:0/4055205704 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f524c086810 con 0x7f525c0662e0 2024-03-22T07:13:40.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.234+0000 7f525a7fc700 1 --2- 172.21.15.63:0/4055205704 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f52440692a0 0x7f524406b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f525000b970 tx=0x7f5250009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:40.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.238+0000 7f52437fe700 1 -- 172.21.15.63:0/4055205704 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f524c04d190 con 0x7f525c0662e0 2024-03-22T07:13:40.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.482+0000 7f5261484700 1 -- 172.21.15.63:0/4055205704 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f525c04bed0 con 0x7f525c0662e0 2024-03-22T07:13:40.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.652+0000 7f52437fe700 1 -- 172.21.15.63:0/4055205704 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v10) v1 ==== 121+0+0 (secure 0 0 0) 0x7f524c052ac0 con 0x7f525c0662e0 2024-03-22T07:13:40.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.654+0000 7f5261484700 1 -- 172.21.15.63:0/4055205704 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f52440692a0 msgr2=0x7f524406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:40.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.654+0000 7f5261484700 1 --2- 172.21.15.63:0/4055205704 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f52440692a0 0x7f524406b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f525000b970 tx=0x7f5250009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:40.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.654+0000 7f5261484700 1 -- 172.21.15.63:0/4055205704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f525c0662e0 msgr2=0x7f525c0fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:40.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.654+0000 7f5261484700 1 --2- 172.21.15.63:0/4055205704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f525c0662e0 0x7f525c0fd3f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f524c002a80 tx=0x7f524c004630 comp rx=0 tx=0).stop 2024-03-22T07:13:40.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.655+0000 7f5261484700 1 -- 172.21.15.63:0/4055205704 shutdown_connections 2024-03-22T07:13:40.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.655+0000 7f5261484700 1 --2- 172.21.15.63:0/4055205704 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f52440692a0 0x7f524406b750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:40.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.655+0000 7f5261484700 1 --2- 172.21.15.63:0/4055205704 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f525c0662e0 0x7f525c0fd3f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:40.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.655+0000 7f5261484700 1 --2- 172.21.15.63:0/4055205704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f525c0fff00 0x7f525c0fb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:40.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.655+0000 7f5261484700 1 -- 172.21.15.63:0/4055205704 >> 172.21.15.63:0/4055205704 conn(0x7f525c0f7430 msgr2=0x7f525c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:40.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.655+0000 7f5261484700 1 -- 172.21.15.63:0/4055205704 shutdown_connections 2024-03-22T07:13:40.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:40.655+0000 7f5261484700 1 -- 172.21.15.63:0/4055205704 wait complete. 2024-03-22T07:13:40.739 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:40 smithi063 ceph-mon[29200]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 6 op/s 2024-03-22T07:13:40.739 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:40 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/4055205704' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-22T07:13:41.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:40 smithi069 ceph-mon[36441]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s wr, 6 op/s 2024-03-22T07:13:41.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:40 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/4055205704' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-22T07:13:41.325 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T07:13:41.336 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-03-22T07:13:41.336 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data false' 2024-03-22T07:13:41.604 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:13:41.880 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:41 smithi069 ceph-mon[36441]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-22T07:13:41.880 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:41 smithi069 ceph-mon[36441]: Cluster is now healthy 2024-03-22T07:13:41.880 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:41 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/4055205704' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-22T07:13:41.880 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:41 smithi069 ceph-mon[36441]: mds.? [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] up:boot 2024-03-22T07:13:41.880 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:41 smithi069 ceph-mon[36441]: fsmap cephfs:2 {0=cephfs.smithi063.oytapz=up:active,1=cephfs.smithi069.wwtsjn=up:active} 1 up:standby 2024-03-22T07:13:41.880 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:41 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.xeelpk"}]: dispatch 2024-03-22T07:13:41.881 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:41 smithi069 ceph-mon[36441]: fsmap cephfs:2 {0=cephfs.smithi063.oytapz=up:active,1=cephfs.smithi069.wwtsjn=up:active} 1 up:standby-replay 2024-03-22T07:13:41.881 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:41 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:41.881 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:41 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:41.881 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:41 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:41.881 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:41 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi069.cijxah", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:13:41.881 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:41 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi069.cijxah", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T07:13:41.881 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:41 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:41.881 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:41 smithi069 ceph-mon[36441]: Deploying daemon mds.cephfs.smithi069.cijxah on smithi069 2024-03-22T07:13:41.945 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:41 smithi063 ceph-mon[29200]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-22T07:13:41.945 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:41 smithi063 ceph-mon[29200]: Cluster is now healthy 2024-03-22T07:13:41.945 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:41 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/4055205704' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-22T07:13:41.945 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:41 smithi063 ceph-mon[29200]: mds.? [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] up:boot 2024-03-22T07:13:41.946 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:41 smithi063 ceph-mon[29200]: fsmap cephfs:2 {0=cephfs.smithi063.oytapz=up:active,1=cephfs.smithi069.wwtsjn=up:active} 1 up:standby 2024-03-22T07:13:41.946 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:41 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.xeelpk"}]: dispatch 2024-03-22T07:13:41.946 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:41 smithi063 ceph-mon[29200]: fsmap cephfs:2 {0=cephfs.smithi063.oytapz=up:active,1=cephfs.smithi069.wwtsjn=up:active} 1 up:standby-replay 2024-03-22T07:13:41.946 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:41 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:41.946 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:41 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:41.946 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:41 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:41.946 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:41 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi069.cijxah", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:13:41.946 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:41 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi069.cijxah", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-22T07:13:41.946 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:41 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:41.946 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:41 smithi063 ceph-mon[29200]: Deploying daemon mds.cephfs.smithi069.cijxah on smithi069 2024-03-22T07:13:42.867 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:42 smithi063 ceph-mon[29200]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 96 B/s rd, 2.3 KiB/s wr, 8 op/s 2024-03-22T07:13:42.995 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:42 smithi069 ceph-mon[36441]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 96 B/s rd, 2.3 KiB/s wr, 8 op/s 2024-03-22T07:13:43.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.089+0000 7f5db6df9700 1 -- 172.21.15.63:0/1333692243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5db00fcf80 msgr2=0x7f5db00fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:43.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.089+0000 7f5db6df9700 1 --2- 172.21.15.63:0/1333692243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5db00fcf80 0x7f5db00fd3f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c004940 tx=0x7f5d9c009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:43.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.090+0000 7f5db6df9700 1 -- 172.21.15.63:0/1333692243 shutdown_connections 2024-03-22T07:13:43.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.090+0000 7f5db6df9700 1 --2- 172.21.15.63:0/1333692243 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5db00fcf80 0x7f5db00fd3f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:43.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.090+0000 7f5db6df9700 1 --2- 172.21.15.63:0/1333692243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5db00fbcd0 0x7f5db00fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:43.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.090+0000 7f5db6df9700 1 -- 172.21.15.63:0/1333692243 >> 172.21.15.63:0/1333692243 conn(0x7f5db00f7430 msgr2=0x7f5db00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:43.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.090+0000 7f5db6df9700 1 -- 172.21.15.63:0/1333692243 shutdown_connections 2024-03-22T07:13:43.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.090+0000 7f5db6df9700 1 -- 172.21.15.63:0/1333692243 wait complete. 2024-03-22T07:13:43.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.090+0000 7f5db6df9700 1 Processor -- start 2024-03-22T07:13:43.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.090+0000 7f5db6df9700 1 -- start start 2024-03-22T07:13:43.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.090+0000 7f5db6df9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5db00fbcd0 0x7f5db010c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:43.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.090+0000 7f5db6df9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5db00fcf80 0x7f5db010cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:43.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.090+0000 7f5db6df9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db010d390 con 0x7f5db00fbcd0 2024-03-22T07:13:43.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.090+0000 7f5db6df9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db010d500 con 0x7f5db00fcf80 2024-03-22T07:13:43.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.091+0000 7f5db4b95700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5db00fbcd0 0x7f5db010c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:43.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.091+0000 7f5daffff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5db00fcf80 0x7f5db010cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:43.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.091+0000 7f5db4b95700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5db00fbcd0 0x7f5db010c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35278/0 (socket says 172.21.15.63:35278) 2024-03-22T07:13:43.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.091+0000 7f5db4b95700 1 -- 172.21.15.63:0/960783360 learned_addr learned my addr 172.21.15.63:0/960783360 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:43.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.091+0000 7f5db4b95700 1 -- 172.21.15.63:0/960783360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5db00fcf80 msgr2=0x7f5db010cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:43.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.091+0000 7f5db4b95700 1 --2- 172.21.15.63:0/960783360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5db00fcf80 0x7f5db010cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:43.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.091+0000 7f5db4b95700 1 -- 172.21.15.63:0/960783360 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c005040 con 0x7f5db00fbcd0 2024-03-22T07:13:43.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.091+0000 7f5daffff700 1 --2- 172.21.15.63:0/960783360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5db00fcf80 0x7f5db010cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:13:43.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.091+0000 7f5db4b95700 1 --2- 172.21.15.63:0/960783360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5db00fbcd0 0x7f5db010c850 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5da4002a50 tx=0x7f5da40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:43.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.091+0000 7f5dad7fa700 1 -- 172.21.15.63:0/960783360 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5da4007cc0 con 0x7f5db00fbcd0 2024-03-22T07:13:43.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.091+0000 7f5dad7fa700 1 -- 172.21.15.63:0/960783360 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5da400f040 con 0x7f5db00fbcd0 2024-03-22T07:13:43.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.091+0000 7f5db6df9700 1 -- 172.21.15.63:0/960783360 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db0111f60 con 0x7f5db00fbcd0 2024-03-22T07:13:43.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.091+0000 7f5dad7fa700 1 -- 172.21.15.63:0/960783360 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5da400da30 con 0x7f5db00fbcd0 2024-03-22T07:13:43.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.091+0000 7f5db6df9700 1 -- 172.21.15.63:0/960783360 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db0112490 con 0x7f5db00fbcd0 2024-03-22T07:13:43.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.092+0000 7f5db6df9700 1 -- 172.21.15.63:0/960783360 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db004c710 con 0x7f5db00fbcd0 2024-03-22T07:13:43.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.093+0000 7f5dad7fa700 1 -- 172.21.15.63:0/960783360 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f5da400db90 con 0x7f5db00fbcd0 2024-03-22T07:13:43.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.093+0000 7f5dad7fa700 1 --2- 172.21.15.63:0/960783360 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f5da00692a0 0x7f5da006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:43.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.093+0000 7f5dad7fa700 1 -- 172.21.15.63:0/960783360 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5da4086e60 con 0x7f5db00fbcd0 2024-03-22T07:13:43.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.093+0000 7f5daffff700 1 --2- 172.21.15.63:0/960783360 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f5da00692a0 0x7f5da006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:43.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.094+0000 7f5daffff700 1 --2- 172.21.15.63:0/960783360 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f5da00692a0 0x7f5da006b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009fd0 tx=0x7f5d9c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:43.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.098+0000 7f5dad7fa700 1 -- 172.21.15.63:0/960783360 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f5da4056c20 con 0x7f5db00fbcd0 2024-03-22T07:13:43.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.339+0000 7f5db6df9700 1 -- 172.21.15.63:0/960783360 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f5db01047c0 con 0x7f5db00fbcd0 2024-03-22T07:13:43.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.656+0000 7f5dad7fa700 1 -- 172.21.15.63:0/960783360 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7f5da40567b0 con 0x7f5db00fbcd0 2024-03-22T07:13:43.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.659+0000 7f5db6df9700 1 -- 172.21.15.63:0/960783360 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f5da00692a0 msgr2=0x7f5da006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:43.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.659+0000 7f5db6df9700 1 --2- 172.21.15.63:0/960783360 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f5da00692a0 0x7f5da006b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009fd0 tx=0x7f5d9c009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:43.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.659+0000 7f5db6df9700 1 -- 172.21.15.63:0/960783360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5db00fbcd0 msgr2=0x7f5db010c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:43.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.659+0000 7f5db6df9700 1 --2- 172.21.15.63:0/960783360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5db00fbcd0 0x7f5db010c850 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5da4002a50 tx=0x7f5da40046b0 comp rx=0 tx=0).stop 2024-03-22T07:13:43.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.659+0000 7f5db6df9700 1 -- 172.21.15.63:0/960783360 shutdown_connections 2024-03-22T07:13:43.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.659+0000 7f5db6df9700 1 --2- 172.21.15.63:0/960783360 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f5da00692a0 0x7f5da006b750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:43.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.659+0000 7f5db6df9700 1 --2- 172.21.15.63:0/960783360 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5db00fbcd0 0x7f5db010c850 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:43.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.659+0000 7f5db6df9700 1 --2- 172.21.15.63:0/960783360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5db00fcf80 0x7f5db010cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:43.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.659+0000 7f5db6df9700 1 -- 172.21.15.63:0/960783360 >> 172.21.15.63:0/960783360 conn(0x7f5db00f7430 msgr2=0x7f5db00f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:43.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.659+0000 7f5db6df9700 1 -- 172.21.15.63:0/960783360 shutdown_connections 2024-03-22T07:13:43.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:43.659+0000 7f5db6df9700 1 -- 172.21.15.63:0/960783360 wait complete. 2024-03-22T07:13:43.660 INFO:teuthology.orchestra.run.smithi063.stderr:inline data disabled 2024-03-22T07:13:44.266 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T07:13:44.277 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-03-22T07:13:44.278 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-22T07:13:44.573 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:44 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:44.573 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:44 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:44.574 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:44 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/960783360' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-22T07:13:44.574 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:44 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:44.574 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:44 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:44.574 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:44 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:44.574 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:44 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:13:44.574 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:44 smithi069 ceph-mon[36441]: mds.? [v2:172.21.15.69:6826/3380719311,v1:172.21.15.69:6827/3380719311] up:boot 2024-03-22T07:13:44.574 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:44 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/960783360' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-22T07:13:44.574 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:44 smithi069 ceph-mon[36441]: fsmap cephfs:2 {0=cephfs.smithi063.oytapz=up:active,1=cephfs.smithi069.wwtsjn=up:active} 1 up:standby-replay 1 up:standby 2024-03-22T07:13:44.574 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:44 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.cijxah"}]: dispatch 2024-03-22T07:13:44.574 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:44 smithi069 ceph-mon[36441]: fsmap cephfs:2 {0=cephfs.smithi063.oytapz=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2 up:standby-replay 2024-03-22T07:13:44.574 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:44 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:13:44.574 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:44 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:44 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/960783360' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-22T07:13:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:13:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:44 smithi063 ceph-mon[29200]: mds.? [v2:172.21.15.69:6826/3380719311,v1:172.21.15.69:6827/3380719311] up:boot 2024-03-22T07:13:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:44 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/960783360' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-22T07:13:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:44 smithi063 ceph-mon[29200]: fsmap cephfs:2 {0=cephfs.smithi063.oytapz=up:active,1=cephfs.smithi069.wwtsjn=up:active} 1 up:standby-replay 1 up:standby 2024-03-22T07:13:44.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.cijxah"}]: dispatch 2024-03-22T07:13:44.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:44 smithi063 ceph-mon[29200]: fsmap cephfs:2 {0=cephfs.smithi063.oytapz=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2 up:standby-replay 2024-03-22T07:13:44.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:13:44.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:45.195 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:13:45.595 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:45 smithi063 ceph-mon[29200]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-03-22T07:13:45.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:45 smithi069 ceph-mon[36441]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-03-22T07:13:46.610 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:46 smithi063 ceph-mon[29200]: mds.? [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] up:standby-replay 2024-03-22T07:13:46.610 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:46 smithi063 ceph-mon[29200]: Dropping low affinity active daemon mds.cephfs.smithi063.oytapz in favor of higher affinity standby. 2024-03-22T07:13:46.610 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:46 smithi063 ceph-mon[29200]: Replacing daemon mds.cephfs.smithi063.oytapz as rank 0 with standby daemon mds.cephfs.smithi063.xeelpk 2024-03-22T07:13:46.610 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:46 smithi063 ceph-mon[29200]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T07:13:46.610 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:46 smithi063 ceph-mon[29200]: fsmap cephfs:2 {0=cephfs.smithi063.oytapz=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2 up:standby-replay 2024-03-22T07:13:46.610 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:46 smithi063 ceph-mon[29200]: osdmap e47: 6 total, 6 up, 6 in 2024-03-22T07:13:46.610 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:46 smithi063 ceph-mon[29200]: fsmap cephfs:2/2 {0=cephfs.smithi063.xeelpk=up:replay,1=cephfs.smithi069.wwtsjn=up:active} 1 up:standby-replay 2024-03-22T07:13:46.610 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:46 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:46.610 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:46 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:46.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:46 smithi069 ceph-mon[36441]: mds.? [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] up:standby-replay 2024-03-22T07:13:46.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:46 smithi069 ceph-mon[36441]: Dropping low affinity active daemon mds.cephfs.smithi063.oytapz in favor of higher affinity standby. 2024-03-22T07:13:46.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:46 smithi069 ceph-mon[36441]: Replacing daemon mds.cephfs.smithi063.oytapz as rank 0 with standby daemon mds.cephfs.smithi063.xeelpk 2024-03-22T07:13:46.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:46 smithi069 ceph-mon[36441]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T07:13:46.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:46 smithi069 ceph-mon[36441]: fsmap cephfs:2 {0=cephfs.smithi063.oytapz=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2 up:standby-replay 2024-03-22T07:13:46.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:46 smithi069 ceph-mon[36441]: osdmap e47: 6 total, 6 up, 6 in 2024-03-22T07:13:46.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:46 smithi069 ceph-mon[36441]: fsmap cephfs:2/2 {0=cephfs.smithi063.xeelpk=up:replay,1=cephfs.smithi069.wwtsjn=up:active} 1 up:standby-replay 2024-03-22T07:13:46.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:46 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:46.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:46 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:47 smithi063 ceph-mon[29200]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-03-22T07:13:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:47 smithi063 ceph-mon[29200]: mds.? [v2:172.21.15.63:6826/3211595802,v1:172.21.15.63:6827/3211595802] up:boot 2024-03-22T07:13:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:47 smithi063 ceph-mon[29200]: mds.? [v2:172.21.15.69:6824/3798247699,v1:172.21.15.69:6825/3798247699] up:active 2024-03-22T07:13:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:47 smithi063 ceph-mon[29200]: mds.? [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] up:resolve 2024-03-22T07:13:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:47 smithi063 ceph-mon[29200]: fsmap cephfs:2/2 {0=cephfs.smithi063.xeelpk=up:resolve,1=cephfs.smithi069.wwtsjn=up:active} 1 up:standby-replay 1 up:standby 2024-03-22T07:13:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:47 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.oytapz"}]: dispatch 2024-03-22T07:13:47.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:47 smithi069 ceph-mon[36441]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-03-22T07:13:47.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:47 smithi069 ceph-mon[36441]: mds.? [v2:172.21.15.63:6826/3211595802,v1:172.21.15.63:6827/3211595802] up:boot 2024-03-22T07:13:47.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:47 smithi069 ceph-mon[36441]: mds.? [v2:172.21.15.69:6824/3798247699,v1:172.21.15.69:6825/3798247699] up:active 2024-03-22T07:13:47.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:47 smithi069 ceph-mon[36441]: mds.? [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] up:resolve 2024-03-22T07:13:47.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:47 smithi069 ceph-mon[36441]: fsmap cephfs:2/2 {0=cephfs.smithi063.xeelpk=up:resolve,1=cephfs.smithi069.wwtsjn=up:active} 1 up:standby-replay 1 up:standby 2024-03-22T07:13:47.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:47 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.oytapz"}]: dispatch 2024-03-22T07:13:48.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:48 smithi063 ceph-mon[29200]: mds.? [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] up:reconnect 2024-03-22T07:13:48.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:48 smithi063 ceph-mon[29200]: mds.? [v2:172.21.15.69:6826/3380719311,v1:172.21.15.69:6827/3380719311] up:standby-replay 2024-03-22T07:13:48.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:48 smithi063 ceph-mon[29200]: fsmap cephfs:2/2 {0=cephfs.smithi063.xeelpk=up:reconnect,1=cephfs.smithi069.wwtsjn=up:active} 1 up:standby-replay 1 up:standby 2024-03-22T07:13:48.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.702+0000 7fcd34d30700 1 -- 172.21.15.63:0/316240766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd300fdfe0 msgr2=0x7fcd300fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.702+0000 7fcd34d30700 1 --2- 172.21.15.63:0/316240766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd300fdfe0 0x7fcd300fe450 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcd20004940 tx=0x7fcd20009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.702+0000 7fcd34d30700 1 -- 172.21.15.63:0/316240766 shutdown_connections 2024-03-22T07:13:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.702+0000 7fcd34d30700 1 --2- 172.21.15.63:0/316240766 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd300fdfe0 0x7fcd300fe450 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.702+0000 7fcd34d30700 1 --2- 172.21.15.63:0/316240766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd300fce70 0x7fcd300fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.702+0000 7fcd34d30700 1 -- 172.21.15.63:0/316240766 >> 172.21.15.63:0/316240766 conn(0x7fcd300f8550 msgr2=0x7fcd300fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.702+0000 7fcd34d30700 1 -- 172.21.15.63:0/316240766 shutdown_connections 2024-03-22T07:13:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.702+0000 7fcd34d30700 1 -- 172.21.15.63:0/316240766 wait complete. 2024-03-22T07:13:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.703+0000 7fcd34d30700 1 Processor -- start 2024-03-22T07:13:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.703+0000 7fcd34d30700 1 -- start start 2024-03-22T07:13:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.703+0000 7fcd34d30700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd300fce70 0x7fcd30071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.703+0000 7fcd34d30700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd300fdfe0 0x7fcd30070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.703+0000 7fcd34d30700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd30072260 con 0x7fcd300fdfe0 2024-03-22T07:13:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.703+0000 7fcd34d30700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd30070850 con 0x7fcd300fce70 2024-03-22T07:13:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.703+0000 7fcd2dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd300fdfe0 0x7fcd30070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:48.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.703+0000 7fcd2dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd300fdfe0 0x7fcd30070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36876/0 (socket says 172.21.15.63:36876) 2024-03-22T07:13:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.703+0000 7fcd2dd9b700 1 -- 172.21.15.63:0/2804956392 learned_addr learned my addr 172.21.15.63:0/2804956392 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.703+0000 7fcd2e59c700 1 --2- 172.21.15.63:0/2804956392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd300fce70 0x7fcd30071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.703+0000 7fcd2dd9b700 1 -- 172.21.15.63:0/2804956392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd300fce70 msgr2=0x7fcd30071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.703+0000 7fcd2dd9b700 1 --2- 172.21.15.63:0/2804956392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd300fce70 0x7fcd30071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.703+0000 7fcd2dd9b700 1 -- 172.21.15.63:0/2804956392 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd20005040 con 0x7fcd300fdfe0 2024-03-22T07:13:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.704+0000 7fcd2dd9b700 1 --2- 172.21.15.63:0/2804956392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd300fdfe0 0x7fcd30070310 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000b3b0 tx=0x7fcd20009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.704+0000 7fcd1effd700 1 -- 172.21.15.63:0/2804956392 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd2000e040 con 0x7fcd300fdfe0 2024-03-22T07:13:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.704+0000 7fcd34d30700 1 -- 172.21.15.63:0/2804956392 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd30070aa0 con 0x7fcd300fdfe0 2024-03-22T07:13:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.704+0000 7fcd1effd700 1 -- 172.21.15.63:0/2804956392 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd2000bcc0 con 0x7fcd300fdfe0 2024-03-22T07:13:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.704+0000 7fcd1effd700 1 -- 172.21.15.63:0/2804956392 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd2001c630 con 0x7fcd300fdfe0 2024-03-22T07:13:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.704+0000 7fcd34d30700 1 -- 172.21.15.63:0/2804956392 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd30071060 con 0x7fcd300fdfe0 2024-03-22T07:13:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.705+0000 7fcd34d30700 1 -- 172.21.15.63:0/2804956392 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd3004c710 con 0x7fcd300fdfe0 2024-03-22T07:13:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.705+0000 7fcd1effd700 1 -- 172.21.15.63:0/2804956392 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fcd20013980 con 0x7fcd300fdfe0 2024-03-22T07:13:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.706+0000 7fcd1effd700 1 --2- 172.21.15.63:0/2804956392 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fcd180692a0 0x7fcd1806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.706+0000 7fcd1effd700 1 -- 172.21.15.63:0/2804956392 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fcd20088000 con 0x7fcd300fdfe0 2024-03-22T07:13:48.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.706+0000 7fcd2e59c700 1 --2- 172.21.15.63:0/2804956392 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fcd180692a0 0x7fcd1806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:48.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.706+0000 7fcd2e59c700 1 --2- 172.21.15.63:0/2804956392 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fcd180692a0 0x7fcd1806b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcd24007fd0 tx=0x7fcd24009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:48.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.710+0000 7fcd1effd700 1 -- 172.21.15.63:0/2804956392 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fcd20057cf0 con 0x7fcd300fdfe0 2024-03-22T07:13:48.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:48 smithi069 ceph-mon[36441]: mds.? [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] up:reconnect 2024-03-22T07:13:48.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:48 smithi069 ceph-mon[36441]: mds.? [v2:172.21.15.69:6826/3380719311,v1:172.21.15.69:6827/3380719311] up:standby-replay 2024-03-22T07:13:48.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:48 smithi069 ceph-mon[36441]: fsmap cephfs:2/2 {0=cephfs.smithi063.xeelpk=up:reconnect,1=cephfs.smithi069.wwtsjn=up:active} 1 up:standby-replay 1 up:standby 2024-03-22T07:13:48.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.955+0000 7fcd34d30700 1 -- 172.21.15.63:0/2804956392 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcd301058d0 con 0x7fcd300fdfe0 2024-03-22T07:13:48.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.956+0000 7fcd1effd700 1 -- 172.21.15.63:0/2804956392 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1843 (secure 0 0 0) 0x7fcd20057880 con 0x7fcd300fdfe0 2024-03-22T07:13:48.957 INFO:teuthology.orchestra.run.smithi063.stdout:e18 2024-03-22T07:13:48.957 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:13:48.957 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:13:48.957 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-03-22T07:13:48.957 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:13:48.957 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:13:48.958 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-03-22T07:13:48.958 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 18 2024-03-22T07:13:48.958 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-22T07:13:48.958 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-03-22T07:13:29.602905+0000 2024-03-22T07:13:48.958 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-03-22T07:13:48.359115+0000 2024-03-22T07:13:48.958 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-03-22T07:13:48.958 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-03-22T07:13:48.958 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-03-22T07:13:48.958 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-03-22T07:13:48.958 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-03-22T07:13:48.958 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-03-22T07:13:48.958 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-03-22T07:13:48.958 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-03-22T07:13:48.958 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:13:48.958 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 2 2024-03-22T07:13:48.959 INFO:teuthology.orchestra.run.smithi063.stdout:in 0,1 2024-03-22T07:13:48.959 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=14480,1=24285} 2024-03-22T07:13:48.959 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-03-22T07:13:48.959 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-03-22T07:13:48.959 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-03-22T07:13:48.959 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-03-22T07:13:48.959 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-03-22T07:13:48.959 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-03-22T07:13:48.959 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-03-22T07:13:48.959 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-03-22T07:13:48.959 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.xeelpk{0:14480} state up:rejoin seq 5 join_fscid=1 addr [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:13:48.959 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.wwtsjn{1:24285} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.69:6824/3798247699,v1:172.21.15.69:6825/3798247699] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:13:48.959 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.cijxah{1:24301} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.69:6826/3380719311,v1:172.21.15.69:6827/3380719311] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:13:48.959 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:13:48.959 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:13:48.960 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-03-22T07:13:48.960 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:13:48.960 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.oytapz{-1:14492} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3211595802,v1:172.21.15.63:6827/3211595802] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:13:48.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.959+0000 7fcd34d30700 1 -- 172.21.15.63:0/2804956392 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fcd180692a0 msgr2=0x7fcd1806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:48.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.959+0000 7fcd34d30700 1 --2- 172.21.15.63:0/2804956392 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fcd180692a0 0x7fcd1806b750 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcd24007fd0 tx=0x7fcd24009040 comp rx=0 tx=0).stop 2024-03-22T07:13:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.959+0000 7fcd34d30700 1 -- 172.21.15.63:0/2804956392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd300fdfe0 msgr2=0x7fcd30070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.959+0000 7fcd34d30700 1 --2- 172.21.15.63:0/2804956392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd300fdfe0 0x7fcd30070310 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000b3b0 tx=0x7fcd20009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.959+0000 7fcd34d30700 1 -- 172.21.15.63:0/2804956392 shutdown_connections 2024-03-22T07:13:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.959+0000 7fcd34d30700 1 --2- 172.21.15.63:0/2804956392 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fcd180692a0 0x7fcd1806b750 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.959+0000 7fcd34d30700 1 --2- 172.21.15.63:0/2804956392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd300fce70 0x7fcd30071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.959+0000 7fcd34d30700 1 --2- 172.21.15.63:0/2804956392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd300fdfe0 0x7fcd30070310 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.959+0000 7fcd34d30700 1 -- 172.21.15.63:0/2804956392 >> 172.21.15.63:0/2804956392 conn(0x7fcd300f8550 msgr2=0x7fcd300f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.959+0000 7fcd34d30700 1 -- 172.21.15.63:0/2804956392 shutdown_connections 2024-03-22T07:13:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:48.960+0000 7fcd34d30700 1 -- 172.21.15.63:0/2804956392 wait complete. 2024-03-22T07:13:48.961 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 18 2024-03-22T07:13:49.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:49 smithi063 ceph-mon[29200]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-03-22T07:13:49.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:49 smithi063 ceph-mon[29200]: mds.? [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] up:rejoin 2024-03-22T07:13:49.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:49 smithi063 ceph-mon[29200]: fsmap cephfs:2/2 {0=cephfs.smithi063.xeelpk=up:rejoin,1=cephfs.smithi069.wwtsjn=up:active} 1 up:standby-replay 1 up:standby 2024-03-22T07:13:49.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:49 smithi063 ceph-mon[29200]: daemon mds.cephfs.smithi063.xeelpk is now active in filesystem cephfs as rank 0 2024-03-22T07:13:49.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:49 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2804956392' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:13:49.656 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:49 smithi069 ceph-mon[36441]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-03-22T07:13:49.656 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:49 smithi069 ceph-mon[36441]: mds.? [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] up:rejoin 2024-03-22T07:13:49.656 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:49 smithi069 ceph-mon[36441]: fsmap cephfs:2/2 {0=cephfs.smithi063.xeelpk=up:rejoin,1=cephfs.smithi069.wwtsjn=up:active} 1 up:standby-replay 1 up:standby 2024-03-22T07:13:49.656 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:49 smithi069 ceph-mon[36441]: daemon mds.cephfs.smithi063.xeelpk is now active in filesystem cephfs as rank 0 2024-03-22T07:13:49.656 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:49 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/2804956392' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:13:49.670 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-03-22T07:13:50.121 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:13:50.464 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:50 smithi063 ceph-mon[29200]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T07:13:50.464 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:50 smithi063 ceph-mon[29200]: Cluster is now healthy 2024-03-22T07:13:50.464 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:50 smithi063 ceph-mon[29200]: mds.? [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] up:active 2024-03-22T07:13:50.464 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:50 smithi063 ceph-mon[29200]: fsmap cephfs:2 {0=cephfs.smithi063.xeelpk=up:active,1=cephfs.smithi069.wwtsjn=up:active} 1 up:standby-replay 1 up:standby 2024-03-22T07:13:50.464 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:50 smithi063 ceph-mon[29200]: fsmap cephfs:2 {0=cephfs.smithi063.xeelpk=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2 up:standby-replay 2024-03-22T07:13:50.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:50 smithi069 ceph-mon[36441]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T07:13:50.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:50 smithi069 ceph-mon[36441]: Cluster is now healthy 2024-03-22T07:13:50.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:50 smithi069 ceph-mon[36441]: mds.? [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] up:active 2024-03-22T07:13:50.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:50 smithi069 ceph-mon[36441]: fsmap cephfs:2 {0=cephfs.smithi063.xeelpk=up:active,1=cephfs.smithi069.wwtsjn=up:active} 1 up:standby-replay 1 up:standby 2024-03-22T07:13:50.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:50 smithi069 ceph-mon[36441]: fsmap cephfs:2 {0=cephfs.smithi063.xeelpk=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2 up:standby-replay 2024-03-22T07:13:51.452 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:51 smithi063 ceph-mon[29200]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.4 KiB/s rd, 1023 B/s wr, 7 op/s 2024-03-22T07:13:51.452 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:51.452 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:51.452 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:51.452 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:13:51.452 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:51.453 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:51 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:13:51.686 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:51 smithi069 ceph-mon[36441]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.4 KiB/s rd, 1023 B/s wr, 7 op/s 2024-03-22T07:13:51.686 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:51.686 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:51.686 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:51.686 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:13:51.686 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:51.686 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:51 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:13:51.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.816+0000 7f643044f700 1 -- 172.21.15.63:0/2442053566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64280fffc0 msgr2=0x7f64281023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:51.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.816+0000 7f643044f700 1 --2- 172.21.15.63:0/2442053566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64280fffc0 0x7f64281023a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6418004940 tx=0x7f6418009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:51.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.816+0000 7f643044f700 1 -- 172.21.15.63:0/2442053566 shutdown_connections 2024-03-22T07:13:51.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.816+0000 7f643044f700 1 --2- 172.21.15.63:0/2442053566 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64280fffc0 0x7f64281023a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:51.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.816+0000 7f643044f700 1 --2- 172.21.15.63:0/2442053566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6428066e90 0x7f64280ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:51.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.816+0000 7f643044f700 1 -- 172.21.15.63:0/2442053566 >> 172.21.15.63:0/2442053566 conn(0x7f64280f7450 msgr2=0x7f64280f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:51.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.816+0000 7f643044f700 1 -- 172.21.15.63:0/2442053566 shutdown_connections 2024-03-22T07:13:51.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.816+0000 7f643044f700 1 -- 172.21.15.63:0/2442053566 wait complete. 2024-03-22T07:13:51.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.817+0000 7f643044f700 1 Processor -- start 2024-03-22T07:13:51.819 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.817+0000 7f643044f700 1 -- start start 2024-03-22T07:13:51.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.817+0000 7f643044f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6428066e90 0x7f642810d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:51.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.817+0000 7f643044f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64280fffc0 0x7f642810de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:51.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.817+0000 7f643044f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642810e410 con 0x7f6428066e90 2024-03-22T07:13:51.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.817+0000 7f643044f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642810e580 con 0x7f64280fffc0 2024-03-22T07:13:51.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.817+0000 7f642e1eb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6428066e90 0x7f642810d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:51.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.817+0000 7f642e1eb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6428066e90 0x7f642810d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:36894/0 (socket says 172.21.15.63:36894) 2024-03-22T07:13:51.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.817+0000 7f642e1eb700 1 -- 172.21.15.63:0/4292619277 learned_addr learned my addr 172.21.15.63:0/4292619277 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:51.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.817+0000 7f642d9ea700 1 --2- 172.21.15.63:0/4292619277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64280fffc0 0x7f642810de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:51.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.817+0000 7f642e1eb700 1 -- 172.21.15.63:0/4292619277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64280fffc0 msgr2=0x7f642810de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:51.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.817+0000 7f642e1eb700 1 --2- 172.21.15.63:0/4292619277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64280fffc0 0x7f642810de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:51.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.817+0000 7f642e1eb700 1 -- 172.21.15.63:0/4292619277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6418005040 con 0x7f6428066e90 2024-03-22T07:13:51.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.818+0000 7f642e1eb700 1 --2- 172.21.15.63:0/4292619277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6428066e90 0x7f642810d8d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f642400eb70 tx=0x7f642400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:51.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.818+0000 7f641effd700 1 -- 172.21.15.63:0/4292619277 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6424007cc0 con 0x7f6428066e90 2024-03-22T07:13:51.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.818+0000 7f643044f700 1 -- 172.21.15.63:0/4292619277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6428112f80 con 0x7f6428066e90 2024-03-22T07:13:51.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.818+0000 7f643044f700 1 -- 172.21.15.63:0/4292619277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6428113430 con 0x7f6428066e90 2024-03-22T07:13:51.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.818+0000 7f641effd700 1 -- 172.21.15.63:0/4292619277 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f642400f040 con 0x7f6428066e90 2024-03-22T07:13:51.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.818+0000 7f641effd700 1 -- 172.21.15.63:0/4292619277 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f642400cb20 con 0x7f6428066e90 2024-03-22T07:13:51.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.819+0000 7f643044f700 1 -- 172.21.15.63:0/4292619277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f640c0052e0 con 0x7f6428066e90 2024-03-22T07:13:51.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.819+0000 7f641effd700 1 -- 172.21.15.63:0/4292619277 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f642400cc80 con 0x7f6428066e90 2024-03-22T07:13:51.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.820+0000 7f641effd700 1 --2- 172.21.15.63:0/4292619277 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f64140692a0 0x7f641406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:51.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.820+0000 7f641effd700 1 -- 172.21.15.63:0/4292619277 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f6424086a40 con 0x7f6428066e90 2024-03-22T07:13:51.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.820+0000 7f642d9ea700 1 --2- 172.21.15.63:0/4292619277 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f64140692a0 0x7f641406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:51.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.820+0000 7f642d9ea700 1 --2- 172.21.15.63:0/4292619277 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f64140692a0 0x7f641406b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6418004890 tx=0x7f6418009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:51.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:51.824+0000 7f641effd700 1 -- 172.21.15.63:0/4292619277 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f642404e190 con 0x7f6428066e90 2024-03-22T07:13:52.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:52.067+0000 7f643044f700 1 -- 172.21.15.63:0/4292619277 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f640c005b10 con 0x7f6428066e90 2024-03-22T07:13:52.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:52.068+0000 7f641effd700 1 -- 172.21.15.63:0/4292619277 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4719 (secure 0 0 0) 0x7f6424019090 con 0x7f6428066e90 2024-03-22T07:13:52.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:52.070+0000 7f643044f700 1 -- 172.21.15.63:0/4292619277 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f64140692a0 msgr2=0x7f641406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:52.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:52.070+0000 7f643044f700 1 --2- 172.21.15.63:0/4292619277 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f64140692a0 0x7f641406b750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6418004890 tx=0x7f6418009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:52.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:52.070+0000 7f643044f700 1 -- 172.21.15.63:0/4292619277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6428066e90 msgr2=0x7f642810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:52.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:52.070+0000 7f643044f700 1 --2- 172.21.15.63:0/4292619277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6428066e90 0x7f642810d8d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f642400eb70 tx=0x7f642400b040 comp rx=0 tx=0).stop 2024-03-22T07:13:52.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:52.070+0000 7f643044f700 1 -- 172.21.15.63:0/4292619277 shutdown_connections 2024-03-22T07:13:52.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:52.070+0000 7f643044f700 1 --2- 172.21.15.63:0/4292619277 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f64140692a0 0x7f641406b750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:52.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:52.070+0000 7f643044f700 1 --2- 172.21.15.63:0/4292619277 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f6428066e90 0x7f642810d8d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:52.071 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:52.070+0000 7f643044f700 1 --2- 172.21.15.63:0/4292619277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64280fffc0 0x7f642810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:52.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:52.070+0000 7f643044f700 1 -- 172.21.15.63:0/4292619277 >> 172.21.15.63:0/4292619277 conn(0x7f64280f7450 msgr2=0x7f64280f7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:52.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:52.070+0000 7f643044f700 1 -- 172.21.15.63:0/4292619277 shutdown_connections 2024-03-22T07:13:52.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:52.070+0000 7f643044f700 1 -- 172.21.15.63:0/4292619277 wait complete. 2024-03-22T07:13:52.072 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 20 2024-03-22T07:13:52.081 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:13:52.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:52 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/4292619277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T07:13:52.760 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:52 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/4292619277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T07:13:52.785 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-03-22T07:13:53.160 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:13:53.590 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:53 smithi063 ceph-mon[29200]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-22T07:13:53.590 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:53 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:53.590 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:53 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:53.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:53 smithi069 ceph-mon[36441]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.7 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-22T07:13:53.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:53 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:53.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:53 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:55.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.389+0000 7f44245c5700 1 -- 172.21.15.63:0/121977389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f441c066b10 msgr2=0x7f441c0ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:55.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.389+0000 7f44245c5700 1 --2- 172.21.15.63:0/121977389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f441c066b10 0x7f441c0ffa30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4418004940 tx=0x7f4418009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:55.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.389+0000 7f44245c5700 1 -- 172.21.15.63:0/121977389 shutdown_connections 2024-03-22T07:13:55.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.389+0000 7f44245c5700 1 --2- 172.21.15.63:0/121977389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f441c0fff70 0x7f441c102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:55.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.389+0000 7f44245c5700 1 --2- 172.21.15.63:0/121977389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f441c066b10 0x7f441c0ffa30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:55.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.389+0000 7f44245c5700 1 -- 172.21.15.63:0/121977389 >> 172.21.15.63:0/121977389 conn(0x7f441c0f7440 msgr2=0x7f441c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:55.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.389+0000 7f44245c5700 1 -- 172.21.15.63:0/121977389 shutdown_connections 2024-03-22T07:13:55.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.389+0000 7f44245c5700 1 -- 172.21.15.63:0/121977389 wait complete. 2024-03-22T07:13:55.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.390+0000 7f44245c5700 1 Processor -- start 2024-03-22T07:13:55.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.390+0000 7f44245c5700 1 -- start start 2024-03-22T07:13:55.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.390+0000 7f44245c5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f441c066b10 0x7f441c0fd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:55.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.390+0000 7f44245c5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f441c0fff70 0x7f441c0fb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:55.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.390+0000 7f44245c5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c0fbec0 con 0x7f441c0fff70 2024-03-22T07:13:55.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.390+0000 7f44245c5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c0fc000 con 0x7f441c066b10 2024-03-22T07:13:55.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.390+0000 7f4421b60700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f441c0fff70 0x7f441c0fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:55.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.390+0000 7f4421b60700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f441c0fff70 0x7f441c0fb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37232/0 (socket says 172.21.15.63:37232) 2024-03-22T07:13:55.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.390+0000 7f4421b60700 1 -- 172.21.15.63:0/3302458962 learned_addr learned my addr 172.21.15.63:0/3302458962 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:55.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.390+0000 7f4422361700 1 --2- 172.21.15.63:0/3302458962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f441c066b10 0x7f441c0fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:55.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.391+0000 7f4421b60700 1 -- 172.21.15.63:0/3302458962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f441c066b10 msgr2=0x7f441c0fd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:55.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.391+0000 7f4421b60700 1 --2- 172.21.15.63:0/3302458962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f441c066b10 0x7f441c0fd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:55.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.391+0000 7f4421b60700 1 -- 172.21.15.63:0/3302458962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4418005040 con 0x7f441c0fff70 2024-03-22T07:13:55.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.391+0000 7f4421b60700 1 --2- 172.21.15.63:0/3302458962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f441c0fff70 0x7f441c0fb930 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f440c00eb40 tx=0x7f440c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:55.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.393+0000 7f4412ffd700 1 -- 172.21.15.63:0/3302458962 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f440c007cc0 con 0x7f441c0fff70 2024-03-22T07:13:55.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.393+0000 7f4412ffd700 1 -- 172.21.15.63:0/3302458962 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f440c00f040 con 0x7f441c0fff70 2024-03-22T07:13:55.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.393+0000 7f44245c5700 1 -- 172.21.15.63:0/3302458962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441c0fc280 con 0x7f441c0fff70 2024-03-22T07:13:55.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.394+0000 7f4412ffd700 1 -- 172.21.15.63:0/3302458962 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f440c00ca10 con 0x7f441c0fff70 2024-03-22T07:13:55.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.394+0000 7f44245c5700 1 -- 172.21.15.63:0/3302458962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441c115a40 con 0x7f441c0fff70 2024-03-22T07:13:55.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.395+0000 7f44245c5700 1 -- 172.21.15.63:0/3302458962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f441c05f5f0 con 0x7f441c0fff70 2024-03-22T07:13:55.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.395+0000 7f4412ffd700 1 -- 172.21.15.63:0/3302458962 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f440c00cc80 con 0x7f441c0fff70 2024-03-22T07:13:55.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.396+0000 7f4412ffd700 1 --2- 172.21.15.63:0/3302458962 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f44080692f0 0x7f440806b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:55.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.396+0000 7f4412ffd700 1 -- 172.21.15.63:0/3302458962 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f440c086e70 con 0x7f441c0fff70 2024-03-22T07:13:55.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.396+0000 7f4422361700 1 --2- 172.21.15.63:0/3302458962 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f44080692f0 0x7f440806b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:55.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.396+0000 7f4422361700 1 --2- 172.21.15.63:0/3302458962 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f44080692f0 0x7f440806b7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4418013740 tx=0x7f4418009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:55.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.401+0000 7f4412ffd700 1 -- 172.21.15.63:0/3302458962 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f440c04e190 con 0x7f441c0fff70 2024-03-22T07:13:55.604 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:55 smithi063 ceph-mon[29200]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-22T07:13:55.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.687+0000 7f44245c5700 1 -- 172.21.15.63:0/3302458962 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f441c04bed0 con 0x7f441c0fff70 2024-03-22T07:13:55.688 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.687+0000 7f4412ffd700 1 -- 172.21.15.63:0/3302458962 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v20) v1 ==== 78+0+99 (secure 0 0 0) 0x7f440c0530e0 con 0x7f441c0fff70 2024-03-22T07:13:55.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.689+0000 7f44245c5700 1 -- 172.21.15.63:0/3302458962 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f44080692f0 msgr2=0x7f440806b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:55.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.689+0000 7f44245c5700 1 --2- 172.21.15.63:0/3302458962 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f44080692f0 0x7f440806b7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4418013740 tx=0x7f4418009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:55.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.689+0000 7f44245c5700 1 -- 172.21.15.63:0/3302458962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f441c0fff70 msgr2=0x7f441c0fb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:55.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.689+0000 7f44245c5700 1 --2- 172.21.15.63:0/3302458962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f441c0fff70 0x7f441c0fb930 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f440c00eb40 tx=0x7f440c00b040 comp rx=0 tx=0).stop 2024-03-22T07:13:55.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.689+0000 7f44245c5700 1 -- 172.21.15.63:0/3302458962 shutdown_connections 2024-03-22T07:13:55.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.689+0000 7f44245c5700 1 --2- 172.21.15.63:0/3302458962 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f44080692f0 0x7f440806b7a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:55.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.689+0000 7f44245c5700 1 --2- 172.21.15.63:0/3302458962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f441c066b10 0x7f441c0fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:55.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.689+0000 7f44245c5700 1 --2- 172.21.15.63:0/3302458962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f441c0fff70 0x7f441c0fb930 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:55.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.689+0000 7f44245c5700 1 -- 172.21.15.63:0/3302458962 >> 172.21.15.63:0/3302458962 conn(0x7f441c0f7440 msgr2=0x7f441c0f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:55.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.689+0000 7f44245c5700 1 -- 172.21.15.63:0/3302458962 shutdown_connections 2024-03-22T07:13:55.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:55.689+0000 7f44245c5700 1 -- 172.21.15.63:0/3302458962 wait complete. 2024-03-22T07:13:55.700 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:13:55.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:55 smithi069 ceph-mon[36441]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-22T07:13:56.315 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-03-22T07:13:56.327 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T07:13:56.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.456+0000 7fe433a07640 1 -- 172.21.15.63:0/3369956844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe42c0fd470 msgr2=0x7fe42c0fd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:56.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.456+0000 7fe433a07640 1 --2- 172.21.15.63:0/3369956844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe42c0fd470 0x7fe42c0fd8e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0048d0 tx=0x7fe41c009da0 comp rx=0 tx=0).stop 2024-03-22T07:13:56.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.456+0000 7fe433a07640 1 -- 172.21.15.63:0/3369956844 shutdown_connections 2024-03-22T07:13:56.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.456+0000 7fe433a07640 1 --2- 172.21.15.63:0/3369956844 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe42c0fd470 0x7fe42c0fd8e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:56.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.456+0000 7fe433a07640 1 --2- 172.21.15.63:0/3369956844 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe42c0fc1a0 0x7fe42c0fc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:56.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.456+0000 7fe433a07640 1 -- 172.21.15.63:0/3369956844 >> 172.21.15.63:0/3369956844 conn(0x7fe42c0f7960 msgr2=0x7fe42c0f9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:56.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.457+0000 7fe433a07640 1 -- 172.21.15.63:0/3369956844 shutdown_connections 2024-03-22T07:13:56.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.457+0000 7fe433a07640 1 -- 172.21.15.63:0/3369956844 wait complete. 2024-03-22T07:13:56.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.457+0000 7fe433a07640 1 Processor -- start 2024-03-22T07:13:56.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.457+0000 7fe433a07640 1 -- start start 2024-03-22T07:13:56.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.457+0000 7fe433a07640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe42c0fc1a0 0x7fe42c10ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:56.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.457+0000 7fe433a07640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe42c0fd470 0x7fe42c10d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:56.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.457+0000 7fe433a07640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c10d870 con 0x7fe42c0fc1a0 2024-03-22T07:13:56.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.457+0000 7fe433a07640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c112230 con 0x7fe42c0fd470 2024-03-22T07:13:56.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.457+0000 7fe4317a3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe42c0fc1a0 0x7fe42c10ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:56.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.457+0000 7fe4317a3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe42c0fc1a0 0x7fe42c10ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37254/0 (socket says 172.21.15.63:37254) 2024-03-22T07:13:56.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.457+0000 7fe4317a3640 1 -- 172.21.15.63:0/1871652353 learned_addr learned my addr 172.21.15.63:0/1871652353 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:56.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.458+0000 7fe430fa2640 1 --2- 172.21.15.63:0/1871652353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe42c0fd470 0x7fe42c10d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:56.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.458+0000 7fe4317a3640 1 -- 172.21.15.63:0/1871652353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe42c0fd470 msgr2=0x7fe42c10d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:56.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.458+0000 7fe4317a3640 1 --2- 172.21.15.63:0/1871652353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe42c0fd470 0x7fe42c10d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:56.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.458+0000 7fe4317a3640 1 -- 172.21.15.63:0/1871652353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe41c005040 con 0x7fe42c0fc1a0 2024-03-22T07:13:56.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.458+0000 7fe430fa2640 1 --2- 172.21.15.63:0/1871652353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe42c0fd470 0x7fe42c10d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:13:56.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.458+0000 7fe4317a3640 1 --2- 172.21.15.63:0/1871652353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe42c0fc1a0 0x7fe42c10ccb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe42800c7c0 tx=0x7fe4280043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:56.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.458+0000 7fe421ffb640 1 -- 172.21.15.63:0/1871652353 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe42800a820 con 0x7fe42c0fc1a0 2024-03-22T07:13:56.461 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.458+0000 7fe421ffb640 1 -- 172.21.15.63:0/1871652353 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe42800ae50 con 0x7fe42c0fc1a0 2024-03-22T07:13:56.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.458+0000 7fe433a07640 1 -- 172.21.15.63:0/1871652353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe42c1123d0 con 0x7fe42c0fc1a0 2024-03-22T07:13:56.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.458+0000 7fe421ffb640 1 -- 172.21.15.63:0/1871652353 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4280115e0 con 0x7fe42c0fc1a0 2024-03-22T07:13:56.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.458+0000 7fe433a07640 1 -- 172.21.15.63:0/1871652353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe42c1128c0 con 0x7fe42c0fc1a0 2024-03-22T07:13:56.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.459+0000 7fe433a07640 1 -- 172.21.15.63:0/1871652353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe42c04bcc0 con 0x7fe42c0fc1a0 2024-03-22T07:13:56.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.459+0000 7fe421ffb640 1 -- 172.21.15.63:0/1871652353 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe428012400 con 0x7fe42c0fc1a0 2024-03-22T07:13:56.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.460+0000 7fe421ffb640 1 --2- 172.21.15.63:0/1871652353 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe418069210 0x7fe41806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:56.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.460+0000 7fe421ffb640 1 -- 172.21.15.63:0/1871652353 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe4280873b0 con 0x7fe42c0fc1a0 2024-03-22T07:13:56.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.460+0000 7fe430fa2640 1 --2- 172.21.15.63:0/1871652353 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe418069210 0x7fe41806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:56.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.460+0000 7fe430fa2640 1 --2- 172.21.15.63:0/1871652353 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe418069210 0x7fe41806b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe41c003080 tx=0x7fe41c009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:56.464 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.464+0000 7fe421ffb640 1 -- 172.21.15.63:0/1871652353 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe4280537f0 con 0x7fe42c0fc1a0 2024-03-22T07:13:56.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:56 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3302458962' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-22T07:13:56.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.659+0000 7fe433a07640 1 -- 172.21.15.63:0/1871652353 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe42c112b90 con 0x7fe42c0fc1a0 2024-03-22T07:13:56.661 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.660+0000 7fe421ffb640 1 -- 172.21.15.63:0/1871652353 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+14358 (secure 0 0 0) 0x7fe428056e10 con 0x7fe42c0fc1a0 2024-03-22T07:13:56.661 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:13:56.661 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":47,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","created":"2024-03-22T07:06:38.004887+0000","modified":"2024-03-22T07:13:45.306424+0000","last_up_change":"2024-03-22T07:12:09.636009+0000","last_in_change":"2024-03-22T07:11:51.027495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T07:10:56.524122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-22T07:13:27.835268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-22T07:13:28.836648+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33ade431-7f97-43c7-bf9d-962c12d3ba7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":3791141232},{"type":"v1","addr":"172.21.15.63:6803","nonce":3791141232}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":3791141232},{"type":"v1","addr":"172.21.15.63:6805","nonce":3791141232}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":3791141232},{"type":"v1","addr":"172.21.15.63:6809","nonce":3791141232}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":3791141232},{"type":"v1","addr":"172.21.15.63:6807","nonce":3791141232}]},"public_addr":"172.21.15.63:6803/3791141232","cluster_addr":"172.21.15.63:6805/3791141232","heartbeat_back_addr":"172.21.15.63:6809/3791141232","heartbeat_front_addr":"172.21.15.63:6807/3791141232","state":["exists","up"]},{"osd":1,"uuid":"0643ddb6-db9d-464c-8236-3aea2893de83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":1043983969},{"type":"v1","addr":"172.21.15.63:6811","nonce":1043983969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":1043983969},{"type":"v1","addr":"172.21.15.63:6813","nonce":1043983969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":1043983969},{"type":"v1","addr":"172.21.15.63:6817","nonce":1043983969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":1043983969},{"type":"v1","addr":"172.21.15.63:6815","nonce":1043983969}]},"public_addr":"172.21.15.63:6811/1043983969","cluster_addr":"172.21.15.63:6813/1043983969","heartbeat_back_addr":"172.21.15.63:6817/1043983969","heartbeat_front_addr":"172.21.15.63:6815/1043983969","state":["exists","up"]},{"osd":2,"uuid":"140b09d0-1fb6-4a3b-873a-048b797c0180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":3603088351},{"type":"v1","addr":"172.21.15.63:6819","nonce":3603088351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":3603088351},{"type":"v1","addr":"172.21.15.63:6821","nonce":3603088351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":3603088351},{"type":"v1","addr":"172.21.15.63:6825","nonce":3603088351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":3603088351},{"type":"v1","addr":"172.21.15.63:6823","nonce":3603088351}]},"public_addr":"172.21.15.63:6819/3603088351","cluster_addr":"172.21.15.63:6821/3603088351","heartbeat_back_addr":"172.21.15.63:6825/3603088351","heartbeat_front_addr":"172.21.15.63:6823/3603088351","state":["exists","up"]},{"osd":3,"uuid":"1d92b77e-6bcc-4585-96f1-72251303cc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":1811566075},{"type":"v1","addr":"172.21.15.69:6801","nonce":1811566075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":1811566075},{"type":"v1","addr":"172.21.15.69:6803","nonce":1811566075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":1811566075},{"type":"v1","addr":"172.21.15.69:6807","nonce":1811566075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":1811566075},{"type":"v1","addr":"172.21.15.69:6805","nonce":1811566075}]},"public_addr":"172.21.15.69:6801/1811566075","cluster_addr":"172.21.15.69:6803/1811566075","heartbeat_back_addr":"172.21.15.69:6807/1811566075","heartbeat_front_addr":"172.21.15.69:6805/1811566075","state":["exists","up"]},{"osd":4,"uuid":"28bdc8bc-41d5-4c01-9b29-d5833223d1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":656541591},{"type":"v1","addr":"172.21.15.69:6809","nonce":656541591}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":656541591},{"type":"v1","addr":"172.21.15.69:6811","nonce":656541591}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":656541591},{"type":"v1","addr":"172.21.15.69:6815","nonce":656541591}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":656541591},{"type":"v1","addr":"172.21.15.69:6813","nonce":656541591}]},"public_addr":"172.21.15.69:6809/656541591","cluster_addr":"172.21.15.69:6811/656541591","heartbeat_back_addr":"172.21.15.69:6815/656541591","heartbeat_front_addr":"172.21.15.69:6813/656541591","state":["exists","up"]},{"osd":5,"uuid":"566db34e-ce0e-436f-aa90-df2c80d812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":2157641649},{"type":"v1","addr":"172.21.15.69:6817","nonce":2157641649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2157641649},{"type":"v1","addr":"172.21.15.69:6819","nonce":2157641649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2157641649},{"type":"v1","addr":"172.21.15.69:6823","nonce":2157641649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2157641649},{"type":"v1","addr":"172.21.15.69:6821","nonce":2157641649}]},"public_addr":"172.21.15.69:6817/2157641649","cluster_addr":"172.21.15.69:6819/2157641649","heartbeat_back_addr":"172.21.15.69:6823/2157641649","heartbeat_front_addr":"172.21.15.69:6821/2157641649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:09:56.132204+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:10:24.051511+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:10:52.969047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:11:17.576349+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:11:42.883846+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:12:08.488698+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6827/1314414011":"2024-03-23T07:13:45.306403+0000","172.21.15.63:0/2503315479":"2024-03-23T07:07:41.261927+0000","172.21.15.63:6800/480818549":"2024-03-23T07:07:41.261927+0000","172.21.15.63:0/2739978443":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/424866385":"2024-03-23T07:07:10.417547+0000","172.21.15.63:6826/1314414011":"2024-03-23T07:13:45.306403+0000","172.21.15.63:6800/2616455474":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/3927392742":"2024-03-23T07:08:44.186801+0000","172.21.15.63:6801/2616455474":"2024-03-23T07:07:10.417547+0000","172.21.15.63:6800/2296442540":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/1818534437":"2024-03-23T07:07:41.261927+0000","172.21.15.63:0/1649622442":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/1323074353":"2024-03-23T07:07:41.261927+0000","172.21.15.63:6801/480818549":"2024-03-23T07:07:41.261927+0000","172.21.15.63:0/466771007":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/47408280":"2024-03-23T07:08:44.186801+0000","172.21.15.63:6801/2296442540":"2024-03-23T07:08:44.186801+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T07:13:56.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.662+0000 7fe433a07640 1 -- 172.21.15.63:0/1871652353 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe418069210 msgr2=0x7fe41806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:56.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.662+0000 7fe433a07640 1 --2- 172.21.15.63:0/1871652353 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe418069210 0x7fe41806b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe41c003080 tx=0x7fe41c009da0 comp rx=0 tx=0).stop 2024-03-22T07:13:56.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.662+0000 7fe433a07640 1 -- 172.21.15.63:0/1871652353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe42c0fc1a0 msgr2=0x7fe42c10ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:56.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.662+0000 7fe433a07640 1 --2- 172.21.15.63:0/1871652353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe42c0fc1a0 0x7fe42c10ccb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe42800c7c0 tx=0x7fe4280043c0 comp rx=0 tx=0).stop 2024-03-22T07:13:56.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.662+0000 7fe433a07640 1 -- 172.21.15.63:0/1871652353 shutdown_connections 2024-03-22T07:13:56.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.662+0000 7fe433a07640 1 --2- 172.21.15.63:0/1871652353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe42c0fd470 0x7fe42c10d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:56.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.662+0000 7fe433a07640 1 --2- 172.21.15.63:0/1871652353 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe418069210 0x7fe41806b6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:56.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.662+0000 7fe433a07640 1 --2- 172.21.15.63:0/1871652353 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe42c0fc1a0 0x7fe42c10ccb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:56.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.662+0000 7fe433a07640 1 -- 172.21.15.63:0/1871652353 >> 172.21.15.63:0/1871652353 conn(0x7fe42c0f7960 msgr2=0x7fe42c0f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:56.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.662+0000 7fe433a07640 1 -- 172.21.15.63:0/1871652353 shutdown_connections 2024-03-22T07:13:56.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.662+0000 7fe433a07640 1 -- 172.21.15.63:0/1871652353 wait complete. 2024-03-22T07:13:56.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T07:10:56.524122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-22T07:13:27.835268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-22T07:13:28.836648+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-22T07:13:56.675 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T07:13:56.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:56 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/3302458962' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-22T07:13:56.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.779+0000 7fed5791d640 1 -- 172.21.15.63:0/3090475095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed500fb090 msgr2=0x7fed500fb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:56.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.779+0000 7fed5791d640 1 --2- 172.21.15.63:0/3090475095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed500fb090 0x7fed500fb4a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0048d0 tx=0x7fed4c009da0 comp rx=0 tx=0).stop 2024-03-22T07:13:56.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.779+0000 7fed5791d640 1 -- 172.21.15.63:0/3090475095 shutdown_connections 2024-03-22T07:13:56.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.779+0000 7fed5791d640 1 --2- 172.21.15.63:0/3090475095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed500fc360 0x7fed500fc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:56.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.779+0000 7fed5791d640 1 --2- 172.21.15.63:0/3090475095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed500fb090 0x7fed500fb4a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:56.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.779+0000 7fed5791d640 1 -- 172.21.15.63:0/3090475095 >> 172.21.15.63:0/3090475095 conn(0x7fed500f6870 msgr2=0x7fed500f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:56.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.779+0000 7fed5791d640 1 -- 172.21.15.63:0/3090475095 shutdown_connections 2024-03-22T07:13:56.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.779+0000 7fed5791d640 1 -- 172.21.15.63:0/3090475095 wait complete. 2024-03-22T07:13:56.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.779+0000 7fed5791d640 1 Processor -- start 2024-03-22T07:13:56.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.780+0000 7fed5791d640 1 -- start start 2024-03-22T07:13:56.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.780+0000 7fed5791d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed500fb090 0x7fed5010bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:56.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.780+0000 7fed5791d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed500fc360 0x7fed5010c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:56.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.780+0000 7fed5791d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5010c740 con 0x7fed500fc360 2024-03-22T07:13:56.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.780+0000 7fed5791d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed50111140 con 0x7fed500fb090 2024-03-22T07:13:56.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.780+0000 7fed54eb8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed500fc360 0x7fed5010c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:56.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.780+0000 7fed54eb8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed500fc360 0x7fed5010c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37280/0 (socket says 172.21.15.63:37280) 2024-03-22T07:13:56.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.780+0000 7fed54eb8640 1 -- 172.21.15.63:0/1248852756 learned_addr learned my addr 172.21.15.63:0/1248852756 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:56.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.780+0000 7fed556b9640 1 --2- 172.21.15.63:0/1248852756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed500fb090 0x7fed5010bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:56.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.780+0000 7fed54eb8640 1 -- 172.21.15.63:0/1248852756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed500fb090 msgr2=0x7fed5010bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:56.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.780+0000 7fed54eb8640 1 --2- 172.21.15.63:0/1248852756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed500fb090 0x7fed5010bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:56.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.780+0000 7fed54eb8640 1 -- 172.21.15.63:0/1248852756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4c005040 con 0x7fed500fc360 2024-03-22T07:13:56.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.780+0000 7fed54eb8640 1 --2- 172.21.15.63:0/1248852756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed500fc360 0x7fed5010c0f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fed4000c810 tx=0x7fed400043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:56.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.780+0000 7fed45ffb640 1 -- 172.21.15.63:0/1248852756 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed4000a820 con 0x7fed500fc360 2024-03-22T07:13:56.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.780+0000 7fed45ffb640 1 -- 172.21.15.63:0/1248852756 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed4000ae50 con 0x7fed500fc360 2024-03-22T07:13:56.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.780+0000 7fed5791d640 1 -- 172.21.15.63:0/1248852756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed501112e0 con 0x7fed500fc360 2024-03-22T07:13:56.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.780+0000 7fed45ffb640 1 -- 172.21.15.63:0/1248852756 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed400115c0 con 0x7fed500fc360 2024-03-22T07:13:56.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.781+0000 7fed5791d640 1 -- 172.21.15.63:0/1248852756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed501117d0 con 0x7fed500fc360 2024-03-22T07:13:56.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.782+0000 7fed5791d640 1 -- 172.21.15.63:0/1248852756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed50105570 con 0x7fed500fc360 2024-03-22T07:13:56.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.782+0000 7fed45ffb640 1 -- 172.21.15.63:0/1248852756 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fed4000a980 con 0x7fed500fc360 2024-03-22T07:13:56.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.782+0000 7fed45ffb640 1 --2- 172.21.15.63:0/1248852756 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fed3c069210 0x7fed3c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:56.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.782+0000 7fed556b9640 1 --2- 172.21.15.63:0/1248852756 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fed3c069210 0x7fed3c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:56.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.782+0000 7fed45ffb640 1 -- 172.21.15.63:0/1248852756 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fed400877b0 con 0x7fed500fc360 2024-03-22T07:13:56.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.783+0000 7fed556b9640 1 --2- 172.21.15.63:0/1248852756 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fed3c069210 0x7fed3c06b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0317e0 tx=0x7fed4c009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:56.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.787+0000 7fed45ffb640 1 -- 172.21.15.63:0/1248852756 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fed40053a10 con 0x7fed500fc360 2024-03-22T07:13:56.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.963+0000 7fed5791d640 1 -- 172.21.15.63:0/1248852756 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fed50111a80 con 0x7fed500fc360 2024-03-22T07:13:56.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.963+0000 7fed45ffb640 1 -- 172.21.15.63:0/1248852756 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fed400160a0 con 0x7fed500fc360 2024-03-22T07:13:56.965 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-03-22T07:13:56.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.966+0000 7fed5791d640 1 -- 172.21.15.63:0/1248852756 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fed3c069210 msgr2=0x7fed3c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:56.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.966+0000 7fed5791d640 1 --2- 172.21.15.63:0/1248852756 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fed3c069210 0x7fed3c06b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0317e0 tx=0x7fed4c009da0 comp rx=0 tx=0).stop 2024-03-22T07:13:56.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.966+0000 7fed5791d640 1 -- 172.21.15.63:0/1248852756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed500fc360 msgr2=0x7fed5010c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:56.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.966+0000 7fed5791d640 1 --2- 172.21.15.63:0/1248852756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed500fc360 0x7fed5010c0f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fed4000c810 tx=0x7fed400043c0 comp rx=0 tx=0).stop 2024-03-22T07:13:56.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.966+0000 7fed5791d640 1 -- 172.21.15.63:0/1248852756 shutdown_connections 2024-03-22T07:13:56.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.966+0000 7fed5791d640 1 --2- 172.21.15.63:0/1248852756 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fed3c069210 0x7fed3c06b6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:56.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.966+0000 7fed5791d640 1 --2- 172.21.15.63:0/1248852756 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fed500fc360 0x7fed5010c0f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:56.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.966+0000 7fed5791d640 1 --2- 172.21.15.63:0/1248852756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fed500fb090 0x7fed5010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:56.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.966+0000 7fed5791d640 1 -- 172.21.15.63:0/1248852756 >> 172.21.15.63:0/1248852756 conn(0x7fed500f6870 msgr2=0x7fed500ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:56.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.966+0000 7fed5791d640 1 -- 172.21.15.63:0/1248852756 shutdown_connections 2024-03-22T07:13:56.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:56.966+0000 7fed5791d640 1 -- 172.21.15.63:0/1248852756 wait complete. 2024-03-22T07:13:56.978 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-22T07:13:57.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.084+0000 7fae042ae640 1 -- 172.21.15.63:0/3248415077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadfc06fda0 msgr2=0x7fadfc070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:57.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.084+0000 7fae042ae640 1 --2- 172.21.15.63:0/3248415077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadfc06fda0 0x7fadfc070210 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fadec0048d0 tx=0x7fadec009da0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.084+0000 7fae042ae640 1 -- 172.21.15.63:0/3248415077 shutdown_connections 2024-03-22T07:13:57.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.084+0000 7fae042ae640 1 --2- 172.21.15.63:0/3248415077 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadfc06fda0 0x7fadfc070210 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.084+0000 7fae042ae640 1 --2- 172.21.15.63:0/3248415077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadfc071330 0x7fadfc06f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.084+0000 7fae042ae640 1 -- 172.21.15.63:0/3248415077 >> 172.21.15.63:0/3248415077 conn(0x7fadfc0f7960 msgr2=0x7fadfc0f9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:57.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.084+0000 7fae042ae640 1 -- 172.21.15.63:0/3248415077 shutdown_connections 2024-03-22T07:13:57.087 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.084+0000 7fae042ae640 1 -- 172.21.15.63:0/3248415077 wait complete. 2024-03-22T07:13:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.085+0000 7fae042ae640 1 Processor -- start 2024-03-22T07:13:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.085+0000 7fae042ae640 1 -- start start 2024-03-22T07:13:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.085+0000 7fae042ae640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadfc06fda0 0x7fadfc10eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.085+0000 7fae042ae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadfc071330 0x7fadfc10f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.085+0000 7fae042ae640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc10fa60 con 0x7fadfc06fda0 2024-03-22T07:13:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.085+0000 7fae042ae640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc114460 con 0x7fadfc071330 2024-03-22T07:13:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.085+0000 7fae0204a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadfc06fda0 0x7fadfc10eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.085+0000 7fae0204a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadfc06fda0 0x7fadfc10eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37302/0 (socket says 172.21.15.63:37302) 2024-03-22T07:13:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.085+0000 7fae0204a640 1 -- 172.21.15.63:0/3463832505 learned_addr learned my addr 172.21.15.63:0/3463832505 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.085+0000 7fae01849640 1 --2- 172.21.15.63:0/3463832505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadfc071330 0x7fadfc10f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.085+0000 7fae0204a640 1 -- 172.21.15.63:0/3463832505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadfc071330 msgr2=0x7fadfc10f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.085+0000 7fae0204a640 1 --2- 172.21.15.63:0/3463832505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadfc071330 0x7fadfc10f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.085+0000 7fae0204a640 1 -- 172.21.15.63:0/3463832505 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec005040 con 0x7fadfc06fda0 2024-03-22T07:13:57.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.085+0000 7fae0204a640 1 --2- 172.21.15.63:0/3463832505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadfc06fda0 0x7fadfc10eed0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fadf800eb70 tx=0x7fadf800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:57.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.086+0000 7fadf2ffd640 1 -- 172.21.15.63:0/3463832505 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadf8007c30 con 0x7fadfc06fda0 2024-03-22T07:13:57.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.086+0000 7fadf2ffd640 1 -- 172.21.15.63:0/3463832505 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadf800f040 con 0x7fadfc06fda0 2024-03-22T07:13:57.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.086+0000 7fadf2ffd640 1 -- 172.21.15.63:0/3463832505 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadf800cc30 con 0x7fadfc06fda0 2024-03-22T07:13:57.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.086+0000 7fae042ae640 1 -- 172.21.15.63:0/3463832505 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadfc114600 con 0x7fadfc06fda0 2024-03-22T07:13:57.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.086+0000 7fae042ae640 1 -- 172.21.15.63:0/3463832505 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc114af0 con 0x7fadfc06fda0 2024-03-22T07:13:57.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.087+0000 7fae042ae640 1 -- 172.21.15.63:0/3463832505 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadfc04bcc0 con 0x7fadfc06fda0 2024-03-22T07:13:57.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.087+0000 7fadf2ffd640 1 -- 172.21.15.63:0/3463832505 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fadf8009110 con 0x7fadfc06fda0 2024-03-22T07:13:57.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.087+0000 7fadf2ffd640 1 --2- 172.21.15.63:0/3463832505 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fade8069210 0x7fade806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:57.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.087+0000 7fadf2ffd640 1 -- 172.21.15.63:0/3463832505 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fadf8087470 con 0x7fadfc06fda0 2024-03-22T07:13:57.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.087+0000 7fae01849640 1 --2- 172.21.15.63:0/3463832505 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fade8069210 0x7fade806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:57.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.088+0000 7fae01849640 1 --2- 172.21.15.63:0/3463832505 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fade8069210 0x7fade806b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fadec0317e0 tx=0x7fadec009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:57.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.092+0000 7fadf2ffd640 1 -- 172.21.15.63:0/3463832505 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fadf80536d0 con 0x7fadfc06fda0 2024-03-22T07:13:57.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.265+0000 7fae042ae640 1 -- 172.21.15.63:0/3463832505 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fadfc106f20 con 0x7fadfc06fda0 2024-03-22T07:13:57.266 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.265+0000 7fadf2ffd640 1 -- 172.21.15.63:0/3463832505 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7fadf8056cf0 con 0x7fadfc06fda0 2024-03-22T07:13:57.266 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-03-22T07:13:57.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.267+0000 7fae042ae640 1 -- 172.21.15.63:0/3463832505 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fade8069210 msgr2=0x7fade806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:57.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.267+0000 7fae042ae640 1 --2- 172.21.15.63:0/3463832505 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fade8069210 0x7fade806b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fadec0317e0 tx=0x7fadec009da0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.267+0000 7fae042ae640 1 -- 172.21.15.63:0/3463832505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadfc06fda0 msgr2=0x7fadfc10eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:57.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.267+0000 7fae042ae640 1 --2- 172.21.15.63:0/3463832505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadfc06fda0 0x7fadfc10eed0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fadf800eb70 tx=0x7fadf800b040 comp rx=0 tx=0).stop 2024-03-22T07:13:57.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.267+0000 7fae042ae640 1 -- 172.21.15.63:0/3463832505 shutdown_connections 2024-03-22T07:13:57.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.267+0000 7fae042ae640 1 --2- 172.21.15.63:0/3463832505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadfc071330 0x7fadfc10f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.267+0000 7fae042ae640 1 --2- 172.21.15.63:0/3463832505 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fade8069210 0x7fade806b6c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.267+0000 7fae042ae640 1 --2- 172.21.15.63:0/3463832505 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadfc06fda0 0x7fadfc10eed0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.267+0000 7fae042ae640 1 -- 172.21.15.63:0/3463832505 >> 172.21.15.63:0/3463832505 conn(0x7fadfc0f7960 msgr2=0x7fadfc0f8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:57.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.267+0000 7fae042ae640 1 -- 172.21.15.63:0/3463832505 shutdown_connections 2024-03-22T07:13:57.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.267+0000 7fae042ae640 1 -- 172.21.15.63:0/3463832505 wait complete. 2024-03-22T07:13:57.279 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-22T07:13:57.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.402+0000 7fdb414ae640 1 -- 172.21.15.63:0/969959835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c0fc690 msgr2=0x7fdb3c0fcaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:57.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.402+0000 7fdb414ae640 1 --2- 172.21.15.63:0/969959835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c0fc690 0x7fdb3c0fcaa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0048d0 tx=0x7fdb2c009da0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.402+0000 7fdb414ae640 1 -- 172.21.15.63:0/969959835 shutdown_connections 2024-03-22T07:13:57.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.402+0000 7fdb414ae640 1 --2- 172.21.15.63:0/969959835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb3c0fcfe0 0x7fdb3c0fb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.402+0000 7fdb414ae640 1 --2- 172.21.15.63:0/969959835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c0fc690 0x7fdb3c0fcaa0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.402+0000 7fdb414ae640 1 -- 172.21.15.63:0/969959835 >> 172.21.15.63:0/969959835 conn(0x7fdb3c0f5ff0 msgr2=0x7fdb3c0f8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:57.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.402+0000 7fdb414ae640 1 -- 172.21.15.63:0/969959835 shutdown_connections 2024-03-22T07:13:57.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.402+0000 7fdb414ae640 1 -- 172.21.15.63:0/969959835 wait complete. 2024-03-22T07:13:57.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.403+0000 7fdb414ae640 1 Processor -- start 2024-03-22T07:13:57.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.403+0000 7fdb414ae640 1 -- start start 2024-03-22T07:13:57.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.403+0000 7fdb414ae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb3c0fc690 0x7fdb3c0ff090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:57.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.403+0000 7fdb414ae640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c0fcfe0 0x7fdb3c0fd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:57.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.403+0000 7fdb414ae640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c0ff6e0 con 0x7fdb3c0fcfe0 2024-03-22T07:13:57.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.403+0000 7fdb414ae640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c0fdca0 con 0x7fdb3c0fc690 2024-03-22T07:13:57.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.403+0000 7fdb3a7fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c0fcfe0 0x7fdb3c0fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:57.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.403+0000 7fdb3a7fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c0fcfe0 0x7fdb3c0fd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37324/0 (socket says 172.21.15.63:37324) 2024-03-22T07:13:57.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.403+0000 7fdb3a7fc640 1 -- 172.21.15.63:0/345629443 learned_addr learned my addr 172.21.15.63:0/345629443 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:57.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.403+0000 7fdb3affd640 1 --2- 172.21.15.63:0/345629443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb3c0fc690 0x7fdb3c0ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:57.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.404+0000 7fdb3a7fc640 1 -- 172.21.15.63:0/345629443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb3c0fc690 msgr2=0x7fdb3c0ff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:57.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.404+0000 7fdb3a7fc640 1 --2- 172.21.15.63:0/345629443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb3c0fc690 0x7fdb3c0ff090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.404+0000 7fdb3a7fc640 1 -- 172.21.15.63:0/345629443 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c005040 con 0x7fdb3c0fcfe0 2024-03-22T07:13:57.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.404+0000 7fdb3affd640 1 --2- 172.21.15.63:0/345629443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb3c0fc690 0x7fdb3c0ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:13:57.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.404+0000 7fdb3a7fc640 1 --2- 172.21.15.63:0/345629443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c0fcfe0 0x7fdb3c0fd760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000eb70 tx=0x7fdb3000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:57.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.404+0000 7fdb237fe640 1 -- 172.21.15.63:0/345629443 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb30007c30 con 0x7fdb3c0fcfe0 2024-03-22T07:13:57.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.404+0000 7fdb237fe640 1 -- 172.21.15.63:0/345629443 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb3000f040 con 0x7fdb3c0fcfe0 2024-03-22T07:13:57.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.404+0000 7fdb414ae640 1 -- 172.21.15.63:0/345629443 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3c0fdf20 con 0x7fdb3c0fcfe0 2024-03-22T07:13:57.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.404+0000 7fdb237fe640 1 -- 172.21.15.63:0/345629443 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb300095b0 con 0x7fdb3c0fcfe0 2024-03-22T07:13:57.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.404+0000 7fdb414ae640 1 -- 172.21.15.63:0/345629443 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3c0fe440 con 0x7fdb3c0fcfe0 2024-03-22T07:13:57.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.405+0000 7fdb414ae640 1 -- 172.21.15.63:0/345629443 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3c04bcc0 con 0x7fdb3c0fcfe0 2024-03-22T07:13:57.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.406+0000 7fdb237fe640 1 -- 172.21.15.63:0/345629443 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fdb30009730 con 0x7fdb3c0fcfe0 2024-03-22T07:13:57.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.406+0000 7fdb237fe640 1 --2- 172.21.15.63:0/345629443 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fdb24069210 0x7fdb2406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:57.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.407+0000 7fdb3affd640 1 --2- 172.21.15.63:0/345629443 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fdb24069210 0x7fdb2406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:57.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.407+0000 7fdb237fe640 1 -- 172.21.15.63:0/345629443 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fdb30086f40 con 0x7fdb3c0fcfe0 2024-03-22T07:13:57.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.407+0000 7fdb3affd640 1 --2- 172.21.15.63:0/345629443 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fdb24069210 0x7fdb2406b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0317e0 tx=0x7fdb2c009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:57.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.413+0000 7fdb237fe640 1 -- 172.21.15.63:0/345629443 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fdb30053270 con 0x7fdb3c0fcfe0 2024-03-22T07:13:57.569 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:57 smithi063 ceph-mon[29200]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.9 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-22T07:13:57.569 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:57 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/1871652353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:13:57.569 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:57 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/1248852756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:13:57.569 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:57 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3463832505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T07:13:57.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.590+0000 7fdb414ae640 1 -- 172.21.15.63:0/345629443 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fdb3c103c00 con 0x7fdb3c0fcfe0 2024-03-22T07:13:57.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.590+0000 7fdb237fe640 1 -- 172.21.15.63:0/345629443 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7fdb30056890 con 0x7fdb3c0fcfe0 2024-03-22T07:13:57.591 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-03-22T07:13:57.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.592+0000 7fdb414ae640 1 -- 172.21.15.63:0/345629443 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fdb24069210 msgr2=0x7fdb2406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:57.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.592+0000 7fdb414ae640 1 --2- 172.21.15.63:0/345629443 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fdb24069210 0x7fdb2406b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0317e0 tx=0x7fdb2c009da0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.592+0000 7fdb414ae640 1 -- 172.21.15.63:0/345629443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c0fcfe0 msgr2=0x7fdb3c0fd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:57.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.592+0000 7fdb414ae640 1 --2- 172.21.15.63:0/345629443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c0fcfe0 0x7fdb3c0fd760 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000eb70 tx=0x7fdb3000b040 comp rx=0 tx=0).stop 2024-03-22T07:13:57.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.592+0000 7fdb414ae640 1 -- 172.21.15.63:0/345629443 shutdown_connections 2024-03-22T07:13:57.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.592+0000 7fdb414ae640 1 --2- 172.21.15.63:0/345629443 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fdb24069210 0x7fdb2406b6c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.592+0000 7fdb414ae640 1 --2- 172.21.15.63:0/345629443 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdb3c0fcfe0 0x7fdb3c0fd760 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.592+0000 7fdb414ae640 1 --2- 172.21.15.63:0/345629443 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb3c0fc690 0x7fdb3c0ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.592+0000 7fdb414ae640 1 -- 172.21.15.63:0/345629443 >> 172.21.15.63:0/345629443 conn(0x7fdb3c0f5ff0 msgr2=0x7fdb3c0fa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:57.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.592+0000 7fdb414ae640 1 -- 172.21.15.63:0/345629443 shutdown_connections 2024-03-22T07:13:57.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.592+0000 7fdb414ae640 1 -- 172.21.15.63:0/345629443 wait complete. 2024-03-22T07:13:57.604 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T07:13:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.712+0000 7ff7b5a6e640 1 -- 172.21.15.63:0/3110378144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7b0101770 msgr2=0x7ff7b0103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.712+0000 7ff7b5a6e640 1 --2- 172.21.15.63:0/3110378144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7b0101770 0x7ff7b0103b50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40048d0 tx=0x7ff7a4009da0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.712+0000 7ff7b5a6e640 1 -- 172.21.15.63:0/3110378144 shutdown_connections 2024-03-22T07:13:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.712+0000 7ff7b5a6e640 1 --2- 172.21.15.63:0/3110378144 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7b0101770 0x7ff7b0103b50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.712+0000 7ff7b5a6e640 1 --2- 172.21.15.63:0/3110378144 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7b0064fb0 0x7ff7b0101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.712+0000 7ff7b5a6e640 1 -- 172.21.15.63:0/3110378144 >> 172.21.15.63:0/3110378144 conn(0x7ff7b00f6840 msgr2=0x7ff7b00f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.712+0000 7ff7b5a6e640 1 -- 172.21.15.63:0/3110378144 shutdown_connections 2024-03-22T07:13:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.712+0000 7ff7b5a6e640 1 -- 172.21.15.63:0/3110378144 wait complete. 2024-03-22T07:13:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.712+0000 7ff7b5a6e640 1 Processor -- start 2024-03-22T07:13:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.713+0000 7ff7b5a6e640 1 -- start start 2024-03-22T07:13:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.713+0000 7ff7b5a6e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7b0064fb0 0x7ff7b010ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.713+0000 7ff7b5a6e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7b0101770 0x7ff7b010f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.713+0000 7ff7b5a6e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b010faf0 con 0x7ff7b0101770 2024-03-22T07:13:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.713+0000 7ff7b5a6e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b01144a0 con 0x7ff7b0064fb0 2024-03-22T07:13:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.713+0000 7ff7aeffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7b0101770 0x7ff7b010f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.713+0000 7ff7aeffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7b0101770 0x7ff7b010f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37354/0 (socket says 172.21.15.63:37354) 2024-03-22T07:13:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.713+0000 7ff7aeffd640 1 -- 172.21.15.63:0/2912572373 learned_addr learned my addr 172.21.15.63:0/2912572373 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.713+0000 7ff7af7fe640 1 --2- 172.21.15.63:0/2912572373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7b0064fb0 0x7ff7b010ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.713+0000 7ff7aeffd640 1 -- 172.21.15.63:0/2912572373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7b0064fb0 msgr2=0x7ff7b010ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.713+0000 7ff7aeffd640 1 --2- 172.21.15.63:0/2912572373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7b0064fb0 0x7ff7b010ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.713+0000 7ff7aeffd640 1 -- 172.21.15.63:0/2912572373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a4005040 con 0x7ff7b0101770 2024-03-22T07:13:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.713+0000 7ff7aeffd640 1 --2- 172.21.15.63:0/2912572373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7b0101770 0x7ff7b010f4a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4000c00 tx=0x7ff7a4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.714+0000 7ff797fff640 1 -- 172.21.15.63:0/2912572373 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7a40094c0 con 0x7ff7b0101770 2024-03-22T07:13:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.714+0000 7ff7b5a6e640 1 -- 172.21.15.63:0/2912572373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b01145e0 con 0x7ff7b0101770 2024-03-22T07:13:57.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.714+0000 7ff797fff640 1 -- 172.21.15.63:0/2912572373 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7a4032e90 con 0x7ff7b0101770 2024-03-22T07:13:57.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.714+0000 7ff797fff640 1 -- 172.21.15.63:0/2912572373 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7a40435f0 con 0x7ff7b0101770 2024-03-22T07:13:57.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.714+0000 7ff7b5a6e640 1 -- 172.21.15.63:0/2912572373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b0114a70 con 0x7ff7b0101770 2024-03-22T07:13:57.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.715+0000 7ff7b5a6e640 1 -- 172.21.15.63:0/2912572373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b004bcc0 con 0x7ff7b0101770 2024-03-22T07:13:57.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.715+0000 7ff797fff640 1 -- 172.21.15.63:0/2912572373 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff7a402f070 con 0x7ff7b0101770 2024-03-22T07:13:57.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.716+0000 7ff797fff640 1 --2- 172.21.15.63:0/2912572373 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ff798069210 0x7ff79806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:57.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.716+0000 7ff7af7fe640 1 --2- 172.21.15.63:0/2912572373 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ff798069210 0x7ff79806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:57.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.716+0000 7ff797fff640 1 -- 172.21.15.63:0/2912572373 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff7a40a90c0 con 0x7ff7b0101770 2024-03-22T07:13:57.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.716+0000 7ff7af7fe640 1 --2- 172.21.15.63:0/2912572373 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ff798069210 0x7ff79806b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0064d10 tx=0x7ff7a000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:57.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.720+0000 7ff797fff640 1 -- 172.21.15.63:0/2912572373 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff7a40753d0 con 0x7ff7b0101770 2024-03-22T07:13:57.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:57 smithi069 ceph-mon[36441]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 6.9 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-22T07:13:57.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:57 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/1871652353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:13:57.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:57 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/1248852756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:13:57.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:57 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/3463832505' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T07:13:57.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.937+0000 7ff7b5a6e640 1 -- 172.21.15.63:0/2912572373 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7b0114cc0 con 0x7ff7b0101770 2024-03-22T07:13:57.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.937+0000 7ff797fff640 1 -- 172.21.15.63:0/2912572373 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4719 (secure 0 0 0) 0x7ff7a40789f0 con 0x7ff7b0101770 2024-03-22T07:13:57.938 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:13:57.939 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:13:49.376571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi063.xeelpk","rank":0,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.63:6829/1373266298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1373266298},{"type":"v1","addr":"172.21.15.63:6829","nonce":1373266298}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14492":{"gid":14492,"name":"cephfs.smithi063.oytapz","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6827/3211595802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":3211595802},{"type":"v1","addr":"172.21.15.63:6827","nonce":3211595802}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24285":{"gid":24285,"name":"cephfs.smithi069.wwtsjn","rank":1,"incarnation":8,"state":"up:active","state_seq":4,"addr":"172.21.15.69:6825/3798247699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3798247699},{"type":"v1","addr":"172.21.15.69:6825","nonce":3798247699}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24301":{"gid":24301,"name":"cephfs.smithi069.cijxah","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.69:6827/3380719311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3380719311},{"type":"v1","addr":"172.21.15.69:6827","nonce":3380719311}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-03-22T07:13:57.939 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 20 2024-03-22T07:13:57.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.939+0000 7ff7b5a6e640 1 -- 172.21.15.63:0/2912572373 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ff798069210 msgr2=0x7ff79806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:57.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.939+0000 7ff7b5a6e640 1 --2- 172.21.15.63:0/2912572373 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ff798069210 0x7ff79806b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0064d10 tx=0x7ff7a000b040 comp rx=0 tx=0).stop 2024-03-22T07:13:57.940 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.939+0000 7ff7b5a6e640 1 -- 172.21.15.63:0/2912572373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7b0101770 msgr2=0x7ff7b010f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:57.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.939+0000 7ff7b5a6e640 1 --2- 172.21.15.63:0/2912572373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7b0101770 0x7ff7b010f4a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4000c00 tx=0x7ff7a4009da0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.939+0000 7ff7b5a6e640 1 -- 172.21.15.63:0/2912572373 shutdown_connections 2024-03-22T07:13:57.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.939+0000 7ff7b5a6e640 1 --2- 172.21.15.63:0/2912572373 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ff798069210 0x7ff79806b6c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.939+0000 7ff7b5a6e640 1 --2- 172.21.15.63:0/2912572373 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff7b0101770 0x7ff7b010f4a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.939+0000 7ff7b5a6e640 1 --2- 172.21.15.63:0/2912572373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7b0064fb0 0x7ff7b010ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:57.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.939+0000 7ff7b5a6e640 1 -- 172.21.15.63:0/2912572373 >> 172.21.15.63:0/2912572373 conn(0x7ff7b00f6840 msgr2=0x7ff7b00f6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:57.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.939+0000 7ff7b5a6e640 1 -- 172.21.15.63:0/2912572373 shutdown_connections 2024-03-22T07:13:57.941 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:57.939+0000 7ff7b5a6e640 1 -- 172.21.15.63:0/2912572373 wait complete. 2024-03-22T07:13:57.952 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 20, 'max_mds': 2, 'flags': 50} 2024-03-22T07:13:57.952 INFO:teuthology.run_tasks:Running task kclient... 2024-03-22T07:13:57.976 INFO:tasks.kclient:Mounting kernel clients... 2024-03-22T07:13:57.977 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-03-22T07:13:57.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:13:57.977 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-03-22T07:13:57.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:13:57.992 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link delete ceph-brx 2024-03-22T07:13:58.061 INFO:teuthology.orchestra.run.smithi063.stderr:Cannot find device "ceph-brx" 2024-03-22T07:13:58.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T07:13:58.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:13:58.063 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-03-22T07:13:58.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:13:58.083 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2024-03-22T07:13:58.159 INFO:teuthology.orchestra.run.smithi069.stderr:Cannot find device "ceph-brx" 2024-03-22T07:13:58.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T07:13:58.161 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-03-22T07:13:58.161 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-22T07:13:58.161 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T07:13:58.161 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-22T07:13:58.444 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:13:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:58 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/345629443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T07:13:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:58 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/2912572373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T07:13:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:13:58.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:58 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:58.566 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:58 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/345629443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T07:13:58.566 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:58 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2912572373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T07:13:58.566 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:58.566 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:58.566 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:13:58.567 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:13:58.567 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:58 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:13:59.681 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:59 smithi063 ceph-mon[29200]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.9 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-22T07:13:59.681 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:13:59 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:13:59.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:59 smithi069 ceph-mon[36441]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.9 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-22T07:13:59.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:13:59 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:13:59.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.990+0000 7fe218fe7700 1 -- 172.21.15.63:0/620933384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2141026c0 msgr2=0x7fe2140fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:59.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.990+0000 7fe218fe7700 1 --2- 172.21.15.63:0/620933384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2141026c0 0x7fe2140fbd30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe208004940 tx=0x7fe208009e30 comp rx=0 tx=0).stop 2024-03-22T07:13:59.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.990+0000 7fe218fe7700 1 -- 172.21.15.63:0/620933384 shutdown_connections 2024-03-22T07:13:59.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.990+0000 7fe218fe7700 1 --2- 172.21.15.63:0/620933384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2141026c0 0x7fe2140fbd30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:59.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.990+0000 7fe218fe7700 1 --2- 172.21.15.63:0/620933384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe214101d20 0x7fe2141020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:59.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.990+0000 7fe218fe7700 1 -- 172.21.15.63:0/620933384 >> 172.21.15.63:0/620933384 conn(0x7fe2140f7870 msgr2=0x7fe2140f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:13:59.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.991+0000 7fe218fe7700 1 -- 172.21.15.63:0/620933384 shutdown_connections 2024-03-22T07:13:59.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.991+0000 7fe218fe7700 1 -- 172.21.15.63:0/620933384 wait complete. 2024-03-22T07:13:59.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.991+0000 7fe218fe7700 1 Processor -- start 2024-03-22T07:13:59.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.991+0000 7fe218fe7700 1 -- start start 2024-03-22T07:13:59.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.991+0000 7fe218fe7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe214101d20 0x7fe2140fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:59.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.991+0000 7fe218fe7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2141026c0 0x7fe2140fdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:59.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.991+0000 7fe218fe7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2140ff8f0 con 0x7fe2141026c0 2024-03-22T07:13:59.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.991+0000 7fe218fe7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2140fe420 con 0x7fe214101d20 2024-03-22T07:13:59.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.992+0000 7fe211d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2141026c0 0x7fe2140fdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:59.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.992+0000 7fe211d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2141026c0 0x7fe2140fdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37380/0 (socket says 172.21.15.63:37380) 2024-03-22T07:13:59.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.992+0000 7fe211d9b700 1 -- 172.21.15.63:0/263966690 learned_addr learned my addr 172.21.15.63:0/263966690 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:13:59.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.992+0000 7fe21259c700 1 --2- 172.21.15.63:0/263966690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe214101d20 0x7fe2140fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:59.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.992+0000 7fe211d9b700 1 -- 172.21.15.63:0/263966690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe214101d20 msgr2=0x7fe2140fd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:13:59.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.992+0000 7fe211d9b700 1 --2- 172.21.15.63:0/263966690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe214101d20 0x7fe2140fd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:13:59.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.992+0000 7fe211d9b700 1 -- 172.21.15.63:0/263966690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe208005040 con 0x7fe2141026c0 2024-03-22T07:13:59.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.992+0000 7fe211d9b700 1 --2- 172.21.15.63:0/263966690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2141026c0 0x7fe2140fdee0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe208009100 tx=0x7fe208009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:59.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.992+0000 7fe202ffd700 1 -- 172.21.15.63:0/263966690 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe20800e040 con 0x7fe2141026c0 2024-03-22T07:13:59.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.992+0000 7fe202ffd700 1 -- 172.21.15.63:0/263966690 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe208009af0 con 0x7fe2141026c0 2024-03-22T07:13:59.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.992+0000 7fe218fe7700 1 -- 172.21.15.63:0/263966690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2140fe6a0 con 0x7fe2141026c0 2024-03-22T07:13:59.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.992+0000 7fe202ffd700 1 -- 172.21.15.63:0/263966690 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe20801c3f0 con 0x7fe2141026c0 2024-03-22T07:13:59.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.992+0000 7fe218fe7700 1 -- 172.21.15.63:0/263966690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21411a120 con 0x7fe2141026c0 2024-03-22T07:13:59.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.993+0000 7fe218fe7700 1 -- 172.21.15.63:0/263966690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2140fd470 con 0x7fe2141026c0 2024-03-22T07:13:59.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.994+0000 7fe202ffd700 1 -- 172.21.15.63:0/263966690 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fe20800baf0 con 0x7fe2141026c0 2024-03-22T07:13:59.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.994+0000 7fe202ffd700 1 --2- 172.21.15.63:0/263966690 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe1fc0692a0 0x7fe1fc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:13:59.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.994+0000 7fe21259c700 1 --2- 172.21.15.63:0/263966690 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe1fc0692a0 0x7fe1fc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:13:59.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.994+0000 7fe202ffd700 1 -- 172.21.15.63:0/263966690 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe208088480 con 0x7fe2141026c0 2024-03-22T07:13:59.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.995+0000 7fe21259c700 1 --2- 172.21.15.63:0/263966690 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe1fc0692a0 0x7fe1fc06b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe2040041d0 tx=0x7fe2040043c0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:13:59.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:13:59.999+0000 7fe202ffd700 1 -- 172.21.15.63:0/263966690 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fe208054760 con 0x7fe2141026c0 2024-03-22T07:14:00.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:00.240+0000 7fe218fe7700 1 -- 172.21.15.63:0/263966690 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe21411a310 con 0x7fe2141026c0 2024-03-22T07:14:00.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:00.241+0000 7fe202ffd700 1 -- 172.21.15.63:0/263966690 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+83 (secure 0 0 0) 0x7fe208086e60 con 0x7fe2141026c0 2024-03-22T07:14:00.242 INFO:teuthology.orchestra.run.smithi063.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-22T07:14:00.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:00.243+0000 7fe218fe7700 1 -- 172.21.15.63:0/263966690 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe1fc0692a0 msgr2=0x7fe1fc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:00.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:00.243+0000 7fe218fe7700 1 --2- 172.21.15.63:0/263966690 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe1fc0692a0 0x7fe1fc06b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe2040041d0 tx=0x7fe2040043c0 comp rx=0 tx=0).stop 2024-03-22T07:14:00.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:00.243+0000 7fe218fe7700 1 -- 172.21.15.63:0/263966690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2141026c0 msgr2=0x7fe2140fdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:00.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:00.243+0000 7fe218fe7700 1 --2- 172.21.15.63:0/263966690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2141026c0 0x7fe2140fdee0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe208009100 tx=0x7fe208009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:00.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:00.244+0000 7fe218fe7700 1 -- 172.21.15.63:0/263966690 shutdown_connections 2024-03-22T07:14:00.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:00.244+0000 7fe218fe7700 1 --2- 172.21.15.63:0/263966690 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fe1fc0692a0 0x7fe1fc06b750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:00.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:00.244+0000 7fe218fe7700 1 --2- 172.21.15.63:0/263966690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe214101d20 0x7fe2140fd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:00.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:00.244+0000 7fe218fe7700 1 --2- 172.21.15.63:0/263966690 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe2141026c0 0x7fe2140fdee0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:00.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:00.244+0000 7fe218fe7700 1 -- 172.21.15.63:0/263966690 >> 172.21.15.63:0/263966690 conn(0x7fe2140f7870 msgr2=0x7fe2140662a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:00.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:00.244+0000 7fe218fe7700 1 -- 172.21.15.63:0/263966690 shutdown_connections 2024-03-22T07:14:00.245 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:00.244+0000 7fe218fe7700 1 -- 172.21.15.63:0/263966690 wait complete. 2024-03-22T07:14:00.606 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:00 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/263966690' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T07:14:00.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:00 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/263966690' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T07:14:00.911 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-22T07:14:00.911 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T07:14:00.911 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi063.front.sepia.ceph.com 2024-03-22T07:14:00.911 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-22T07:14:00.911 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-22T07:14:00.912 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-22T07:14:00.912 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-22T07:14:00.912 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T07:14:00.912 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-22T07:14:00.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:00.913 DEBUG:teuthology.orchestra.run.smithi063:> ip addr 2024-03-22T07:14:00.928 INFO:teuthology.orchestra.run.smithi063.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T07:14:00.929 INFO:teuthology.orchestra.run.smithi063.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T07:14:00.929 INFO:teuthology.orchestra.run.smithi063.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T07:14:00.929 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-03-22T07:14:00.929 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 ::1/128 scope host 2024-03-22T07:14:00.929 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-03-22T07:14:00.929 INFO:teuthology.orchestra.run.smithi063.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T07:14:00.929 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:76 brd ff:ff:ff:ff:ff:ff 2024-03-22T07:14:00.929 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f0 2024-03-22T07:14:00.929 INFO:teuthology.orchestra.run.smithi063.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T07:14:00.929 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:a8:b4:77 brd ff:ff:ff:ff:ff:ff 2024-03-22T07:14:00.929 INFO:teuthology.orchestra.run.smithi063.stdout: altname enp7s0f1 2024-03-22T07:14:00.929 INFO:teuthology.orchestra.run.smithi063.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T07:14:00.929 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d6 brd ff:ff:ff:ff:ff:ff 2024-03-22T07:14:00.930 INFO:teuthology.orchestra.run.smithi063.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T07:14:00.930 INFO:teuthology.orchestra.run.smithi063.stdout: link/ether 0c:c4:7a:b6:f8:d7 brd ff:ff:ff:ff:ff:ff 2024-03-22T07:14:00.930 INFO:teuthology.orchestra.run.smithi063.stdout: inet 172.21.15.63/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-22T07:14:00.930 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft 42290sec preferred_lft 42290sec 2024-03-22T07:14:00.930 INFO:teuthology.orchestra.run.smithi063.stdout: inet6 fe80::ec4:7aff:feb6:f8d7/64 scope link 2024-03-22T07:14:00.930 INFO:teuthology.orchestra.run.smithi063.stdout: valid_lft forever preferred_lft forever 2024-03-22T07:14:00.930 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-22T07:14:00.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:00.931 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec bash -c ' 2024-03-22T07:14:00.931 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-03-22T07:14:00.931 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add name ceph-brx type bridge 2024-03-22T07:14:00.931 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr flush dev ceph-brx 2024-03-22T07:14:00.931 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set ceph-brx up 2024-03-22T07:14:00.931 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-22T07:14:00.931 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-03-22T07:14:01.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:01.091 DEBUG:teuthology.orchestra.run.smithi063:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T07:14:01.116 INFO:teuthology.orchestra.run.smithi063.stdout:1 2024-03-22T07:14:01.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:01.118 DEBUG:teuthology.orchestra.run.smithi063:> route 2024-03-22T07:14:01.190 INFO:teuthology.orchestra.run.smithi063.stdout:Kernel IP routing table 2024-03-22T07:14:01.190 INFO:teuthology.orchestra.run.smithi063.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T07:14:01.190 INFO:teuthology.orchestra.run.smithi063.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T07:14:01.190 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T07:14:01.190 INFO:teuthology.orchestra.run.smithi063.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T07:14:01.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:01.191 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec bash -c ' 2024-03-22T07:14:01.191 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-03-22T07:14:01.191 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T07:14:01.191 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T07:14:01.191 DEBUG:teuthology.orchestra.run.smithi063:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T07:14:01.191 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-03-22T07:14:01.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:01.484 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-03-22T07:14:01.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:01.499 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list-id 2024-03-22T07:14:01.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:01.553 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec bash -c ' 2024-03-22T07:14:01.553 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-03-22T07:14:01.553 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T07:14:01.553 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-22T07:14:01.553 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-03-22T07:14:01.618 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:01 smithi063 ceph-mon[29200]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-22T07:14:01.637 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-22T07:14:01.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:01.637 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec bash -c ' 2024-03-22T07:14:01.637 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-03-22T07:14:01.637 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-22T07:14:01.637 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-22T07:14:01.637 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-22T07:14:01.637 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-22T07:14:01.637 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-22T07:14:01.638 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-03-22T07:14:01.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:01.771 DEBUG:teuthology.orchestra.run.smithi063:> (cd / && exec bash -c ' 2024-03-22T07:14:01.771 DEBUG:teuthology.orchestra.run.smithi063:> set -e 2024-03-22T07:14:01.771 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set brx.0 up 2024-03-22T07:14:01.771 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link set dev brx.0 master ceph-brx 2024-03-22T07:14:01.771 DEBUG:teuthology.orchestra.run.smithi063:> ') 2024-03-22T07:14:01.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:01 smithi069 ceph-mon[36441]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-22T07:14:01.813 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T07:14:01.813 DEBUG:teuthology.orchestra.run.smithi063:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-22T07:14:01.868 INFO:teuthology.orchestra.run.smithi063.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-22T07:14:01.868 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T07:14:01.869 DEBUG:teuthology.orchestra.run.smithi063:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-22T07:14:01.923 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T07:14:01.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:01.924 DEBUG:teuthology.orchestra.run.smithi063:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T07:14:02.013 INFO:teuthology.orchestra.run.smithi063.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T07:14:02.013 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T07:14:02.013 INFO:teuthology.orchestra.run.smithi063.stdout:invalid new device string format 2024-03-22T07:14:02.126 INFO:teuthology.orchestra.run.smithi063.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T07:14:02.126 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T07:14:02.127 INFO:teuthology.orchestra.run.smithi063.stdout:invalid new device string format 2024-03-22T07:14:02.127 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: resolved to: "172.21.15.63:3300,172.21.15.69:3300" 2024-03-22T07:14:02.127 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T07:14:02.131 INFO:teuthology.orchestra.run.smithi063.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-22T07:14:02.131 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-22T07:14:02.131 INFO:teuthology.orchestra.run.smithi063.stdout:invalid new device string format 2024-03-22T07:14:02.131 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: resolved to: "172.21.15.63:3300,172.21.15.69:3300" 2024-03-22T07:14:02.131 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T07:14:02.131 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: resolved to: "172.21.15.63:6789,172.21.15.69:6789" 2024-03-22T07:14:02.131 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.63:6789,172.21.15.69:6789:/ 2024-03-22T07:14:02.132 INFO:teuthology.orchestra.run.smithi063.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9a0ba9d6-e81a-11ee-95cd-87774f69a715" will pass to kernel 2024-03-22T07:14:02.132 INFO:teuthology.orchestra.run.smithi063.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-22T07:14:02.132 INFO:teuthology.orchestra.run.smithi063.stdout: You just mounted a file system that supports labels which does not 2024-03-22T07:14:02.132 INFO:teuthology.orchestra.run.smithi063.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-22T07:14:02.132 INFO:teuthology.orchestra.run.smithi063.stdout: applications will generate AVC messages and not be allowed access to 2024-03-22T07:14:02.132 INFO:teuthology.orchestra.run.smithi063.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-22T07:14:02.134 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T07:14:02.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:02.134 DEBUG:teuthology.orchestra.run.smithi063:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-22T07:14:02.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:02.161 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T07:14:02.222 INFO:teuthology.orchestra.run.smithi063.stdout:client24333 2024-03-22T07:14:02.222 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T07:14:02.222 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T07:14:02.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:02.279 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T07:14:02.340 INFO:teuthology.orchestra.run.smithi063.stdout:client24333 2024-03-22T07:14:02.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:02.341 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T07:14:02.401 INFO:teuthology.orchestra.run.smithi063.stdout:client24333 2024-03-22T07:14:02.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:02.402 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715.client24333/status') 2024-03-22T07:14:02.471 INFO:teuthology.orchestra.run.smithi063.stdout:instance: client.24333 (3)172.21.15.63:0/2280312002 2024-03-22T07:14:02.472 INFO:teuthology.orchestra.run.smithi063.stdout:blocklisted: false 2024-03-22T07:14:02.472 INFO:teuthology.orchestra.run.smithi063.stderr:0+1 records in 2024-03-22T07:14:02.472 INFO:teuthology.orchestra.run.smithi063.stderr:0+1 records out 2024-03-22T07:14:02.472 INFO:teuthology.orchestra.run.smithi063.stderr:71 bytes copied, 2.0764e-05 s, 3.4 MB/s 2024-03-22T07:14:02.473 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-03-22T07:14:02.473 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-22T07:14:02.473 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-22T07:14:02.473 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-22T07:14:02.776 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:14:03.562 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:03 smithi063 ceph-mon[29200]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.6 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-22T07:14:03.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:03 smithi069 ceph-mon[36441]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.6 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-22T07:14:04.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.371+0000 7f954ac1e700 1 -- 172.21.15.63:0/4153165644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95440fe830 msgr2=0x7f9544106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:04.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.371+0000 7f954ac1e700 1 --2- 172.21.15.63:0/4153165644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95440fe830 0x7f9544106f00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9530004940 tx=0x7f9530009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:04.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.371+0000 7f954ac1e700 1 -- 172.21.15.63:0/4153165644 shutdown_connections 2024-03-22T07:14:04.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.371+0000 7f954ac1e700 1 --2- 172.21.15.63:0/4153165644 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95440fe830 0x7f9544106f00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.371+0000 7f954ac1e700 1 --2- 172.21.15.63:0/4153165644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95440fde90 0x7f95440fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.371+0000 7f954ac1e700 1 -- 172.21.15.63:0/4153165644 >> 172.21.15.63:0/4153165644 conn(0x7f9544072bc0 msgr2=0x7f9544072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.371+0000 7f954ac1e700 1 -- 172.21.15.63:0/4153165644 shutdown_connections 2024-03-22T07:14:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.371+0000 7f954ac1e700 1 -- 172.21.15.63:0/4153165644 wait complete. 2024-03-22T07:14:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.371+0000 7f954ac1e700 1 Processor -- start 2024-03-22T07:14:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.371+0000 7f954ac1e700 1 -- start start 2024-03-22T07:14:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.372+0000 7f954ac1e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95440fde90 0x7f95440fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.372+0000 7f954ac1e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95440fe830 0x7f95440fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.372+0000 7f954ac1e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95440fc820 con 0x7f95440fde90 2024-03-22T07:14:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.372+0000 7f954ac1e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95440fc960 con 0x7f95440fe830 2024-03-22T07:14:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.372+0000 7f95489ba700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95440fde90 0x7f95440fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.372+0000 7f95489ba700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95440fde90 0x7f95440fbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37402/0 (socket says 172.21.15.63:37402) 2024-03-22T07:14:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.372+0000 7f95489ba700 1 -- 172.21.15.63:0/1066652565 learned_addr learned my addr 172.21.15.63:0/1066652565 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:14:04.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.372+0000 7f9543fff700 1 --2- 172.21.15.63:0/1066652565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95440fe830 0x7f95440fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:04.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.372+0000 7f95489ba700 1 -- 172.21.15.63:0/1066652565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95440fe830 msgr2=0x7f95440fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:04.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.372+0000 7f95489ba700 1 --2- 172.21.15.63:0/1066652565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95440fe830 0x7f95440fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:04.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.372+0000 7f95489ba700 1 -- 172.21.15.63:0/1066652565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9530005040 con 0x7f95440fde90 2024-03-22T07:14:04.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.372+0000 7f95489ba700 1 --2- 172.21.15.63:0/1066652565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95440fde90 0x7f95440fbd50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f953800c8c0 tx=0x7f95380044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:04.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.372+0000 7f95417fa700 1 -- 172.21.15.63:0/1066652565 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f953800a760 con 0x7f95440fde90 2024-03-22T07:14:04.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.372+0000 7f954ac1e700 1 -- 172.21.15.63:0/1066652565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9544119ea0 con 0x7f95440fde90 2024-03-22T07:14:04.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.372+0000 7f95417fa700 1 -- 172.21.15.63:0/1066652565 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f953800ad90 con 0x7f95440fde90 2024-03-22T07:14:04.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.372+0000 7f95417fa700 1 -- 172.21.15.63:0/1066652565 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95380113a0 con 0x7f95440fde90 2024-03-22T07:14:04.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.372+0000 7f954ac1e700 1 -- 172.21.15.63:0/1066652565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954411a2a0 con 0x7f95440fde90 2024-03-22T07:14:04.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.374+0000 7f954ac1e700 1 -- 172.21.15.63:0/1066652565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9544065e30 con 0x7f95440fde90 2024-03-22T07:14:04.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.374+0000 7f95417fa700 1 -- 172.21.15.63:0/1066652565 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f9538012400 con 0x7f95440fde90 2024-03-22T07:14:04.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.374+0000 7f95417fa700 1 --2- 172.21.15.63:0/1066652565 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f95340692a0 0x7f953406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:04.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.375+0000 7f95417fa700 1 -- 172.21.15.63:0/1066652565 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f9538087240 con 0x7f95440fde90 2024-03-22T07:14:04.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.375+0000 7f9543fff700 1 --2- 172.21.15.63:0/1066652565 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f95340692a0 0x7f953406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:04.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.375+0000 7f9543fff700 1 --2- 172.21.15.63:0/1066652565 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f95340692a0 0x7f953406b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f95440fd690 tx=0x7f9530009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:04.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.379+0000 7f95417fa700 1 -- 172.21.15.63:0/1066652565 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f9538053520 con 0x7f95440fde90 2024-03-22T07:14:04.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.617+0000 7f954ac1e700 1 -- 172.21.15.63:0/1066652565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f954404bf00 con 0x7f95440fde90 2024-03-22T07:14:04.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.618+0000 7f95417fa700 1 -- 172.21.15.63:0/1066652565 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+83 (secure 0 0 0) 0x7f953805c070 con 0x7f95440fde90 2024-03-22T07:14:04.619 INFO:teuthology.orchestra.run.smithi063.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-22T07:14:04.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.620+0000 7f954ac1e700 1 -- 172.21.15.63:0/1066652565 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f95340692a0 msgr2=0x7f953406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:04.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.620+0000 7f954ac1e700 1 --2- 172.21.15.63:0/1066652565 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f95340692a0 0x7f953406b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f95440fd690 tx=0x7f9530009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:04.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.620+0000 7f954ac1e700 1 -- 172.21.15.63:0/1066652565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95440fde90 msgr2=0x7f95440fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:04.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.620+0000 7f954ac1e700 1 --2- 172.21.15.63:0/1066652565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95440fde90 0x7f95440fbd50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f953800c8c0 tx=0x7f95380044c0 comp rx=0 tx=0).stop 2024-03-22T07:14:04.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.620+0000 7f954ac1e700 1 -- 172.21.15.63:0/1066652565 shutdown_connections 2024-03-22T07:14:04.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.620+0000 7f954ac1e700 1 --2- 172.21.15.63:0/1066652565 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f95340692a0 0x7f953406b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:04.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.620+0000 7f954ac1e700 1 --2- 172.21.15.63:0/1066652565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f95440fde90 0x7f95440fbd50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:04.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.620+0000 7f954ac1e700 1 --2- 172.21.15.63:0/1066652565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f95440fe830 0x7f95440fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:04.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.620+0000 7f954ac1e700 1 -- 172.21.15.63:0/1066652565 >> 172.21.15.63:0/1066652565 conn(0x7f9544072bc0 msgr2=0x7f95441037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:04.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.620+0000 7f954ac1e700 1 -- 172.21.15.63:0/1066652565 shutdown_connections 2024-03-22T07:14:04.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:04.620+0000 7f954ac1e700 1 -- 172.21.15.63:0/1066652565 wait complete. 2024-03-22T07:14:05.278 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-22T07:14:05.278 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-22T07:14:05.278 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi069.front.sepia.ceph.com 2024-03-22T07:14:05.279 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-22T07:14:05.279 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-22T07:14:05.279 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-22T07:14:05.279 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-22T07:14:05.279 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-22T07:14:05.279 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-22T07:14:05.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:05.279 DEBUG:teuthology.orchestra.run.smithi069:> ip addr 2024-03-22T07:14:05.298 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-22T07:14:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-22T07:14:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2024-03-22T07:14:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-03-22T07:14:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2024-03-22T07:14:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-03-22T07:14:05.299 INFO:teuthology.orchestra.run.smithi069.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T07:14:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c2 brd ff:ff:ff:ff:ff:ff 2024-03-22T07:14:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f0 2024-03-22T07:14:05.299 INFO:teuthology.orchestra.run.smithi069.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T07:14:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c3 brd ff:ff:ff:ff:ff:ff 2024-03-22T07:14:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f1 2024-03-22T07:14:05.299 INFO:teuthology.orchestra.run.smithi069.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-22T07:14:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:36 brd ff:ff:ff:ff:ff:ff 2024-03-22T07:14:05.299 INFO:teuthology.orchestra.run.smithi069.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-22T07:14:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:37 brd ff:ff:ff:ff:ff:ff 2024-03-22T07:14:05.299 INFO:teuthology.orchestra.run.smithi069.stdout: inet 172.21.15.69/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-22T07:14:05.300 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft 42252sec preferred_lft 42252sec 2024-03-22T07:14:05.300 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::ec4:7aff:fe8f:d737/64 scope link 2024-03-22T07:14:05.300 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-03-22T07:14:05.300 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-22T07:14:05.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:05.300 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2024-03-22T07:14:05.300 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-03-22T07:14:05.300 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add name ceph-brx type bridge 2024-03-22T07:14:05.300 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr flush dev ceph-brx 2024-03-22T07:14:05.300 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set ceph-brx up 2024-03-22T07:14:05.300 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-22T07:14:05.300 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-03-22T07:14:05.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:05.479 DEBUG:teuthology.orchestra.run.smithi069:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-22T07:14:05.507 INFO:teuthology.orchestra.run.smithi069.stdout:1 2024-03-22T07:14:05.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:05.509 DEBUG:teuthology.orchestra.run.smithi069:> route 2024-03-22T07:14:05.584 INFO:teuthology.orchestra.run.smithi069.stdout:Kernel IP routing table 2024-03-22T07:14:05.584 INFO:teuthology.orchestra.run.smithi069.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-22T07:14:05.584 INFO:teuthology.orchestra.run.smithi069.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-22T07:14:05.584 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-22T07:14:05.585 INFO:teuthology.orchestra.run.smithi069.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-22T07:14:05.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:05.586 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2024-03-22T07:14:05.586 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-03-22T07:14:05.586 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-22T07:14:05.586 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-22T07:14:05.586 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-22T07:14:05.587 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-03-22T07:14:05.650 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:05 smithi069 ceph-mon[36441]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 3 op/s 2024-03-22T07:14:05.650 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:05 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/1066652565' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T07:14:05.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:05.837 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-03-22T07:14:05.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:05 smithi063 ceph-mon[29200]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 3 op/s 2024-03-22T07:14:05.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:05 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/1066652565' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-22T07:14:05.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:05.893 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list-id 2024-03-22T07:14:05.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:05.947 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2024-03-22T07:14:05.947 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-03-22T07:14:05.947 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-22T07:14:05.947 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-22T07:14:05.947 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-03-22T07:14:06.031 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-22T07:14:06.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:06.032 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2024-03-22T07:14:06.032 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-03-22T07:14:06.032 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-22T07:14:06.032 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-22T07:14:06.032 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-22T07:14:06.032 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-22T07:14:06.032 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-22T07:14:06.032 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-03-22T07:14:06.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:06.171 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec bash -c ' 2024-03-22T07:14:06.171 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-03-22T07:14:06.171 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set brx.0 up 2024-03-22T07:14:06.171 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set dev brx.0 master ceph-brx 2024-03-22T07:14:06.171 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-03-22T07:14:06.213 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T07:14:06.213 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-22T07:14:06.268 INFO:teuthology.orchestra.run.smithi069.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-22T07:14:06.268 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-22T07:14:06.269 DEBUG:teuthology.orchestra.run.smithi069:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-22T07:14:06.323 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-22T07:14:06.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:06.324 DEBUG:teuthology.orchestra.run.smithi069:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-22T07:14:06.425 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-22T07:14:06.425 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-22T07:14:06.425 INFO:teuthology.orchestra.run.smithi069.stdout:invalid new device string format 2024-03-22T07:14:06.541 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-22T07:14:06.541 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-22T07:14:06.541 INFO:teuthology.orchestra.run.smithi069.stdout:invalid new device string format 2024-03-22T07:14:06.541 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: resolved to: "172.21.15.63:3300,172.21.15.69:3300" 2024-03-22T07:14:06.541 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T07:14:06.546 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-22T07:14:06.546 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-22T07:14:06.546 INFO:teuthology.orchestra.run.smithi069.stdout:invalid new device string format 2024-03-22T07:14:06.546 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: resolved to: "172.21.15.63:3300,172.21.15.69:3300" 2024-03-22T07:14:06.546 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-22T07:14:06.546 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: resolved to: "172.21.15.63:6789,172.21.15.69:6789" 2024-03-22T07:14:06.547 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.63:6789,172.21.15.69:6789:/ 2024-03-22T07:14:06.547 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9a0ba9d6-e81a-11ee-95cd-87774f69a715" will pass to kernel 2024-03-22T07:14:06.547 INFO:teuthology.orchestra.run.smithi069.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-22T07:14:06.547 INFO:teuthology.orchestra.run.smithi069.stdout: You just mounted a file system that supports labels which does not 2024-03-22T07:14:06.547 INFO:teuthology.orchestra.run.smithi069.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-22T07:14:06.547 INFO:teuthology.orchestra.run.smithi069.stdout: applications will generate AVC messages and not be allowed access to 2024-03-22T07:14:06.547 INFO:teuthology.orchestra.run.smithi069.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-22T07:14:06.549 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-22T07:14:06.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:06.549 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-22T07:14:06.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:06.576 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T07:14:06.633 INFO:teuthology.orchestra.run.smithi069.stdout:client24337 2024-03-22T07:14:06.634 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-22T07:14:06.634 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T07:14:06.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:06.691 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T07:14:06.748 INFO:teuthology.orchestra.run.smithi069.stdout:client24337 2024-03-22T07:14:06.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:06.748 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-22T07:14:06.804 INFO:teuthology.orchestra.run.smithi069.stdout:client24337 2024-03-22T07:14:06.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:14:06.805 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715.client24337/status') 2024-03-22T07:14:06.872 INFO:teuthology.orchestra.run.smithi069.stdout:instance: client.24337 (3)192.168.0.1:0/596998459 2024-03-22T07:14:06.872 INFO:teuthology.orchestra.run.smithi069.stdout:blocklisted: false 2024-03-22T07:14:06.873 INFO:teuthology.orchestra.run.smithi069.stderr:0+1 records in 2024-03-22T07:14:06.873 INFO:teuthology.orchestra.run.smithi069.stderr:0+1 records out 2024-03-22T07:14:06.873 INFO:teuthology.orchestra.run.smithi069.stderr:69 bytes copied, 2.1386e-05 s, 3.2 MB/s 2024-03-22T07:14:06.874 INFO:teuthology.run_tasks:Running task print... 2024-03-22T07:14:06.884 INFO:teuthology.task.print:**** done client 2024-03-22T07:14:06.884 INFO:teuthology.run_tasks:Running task parallel... 2024-03-22T07:14:06.894 INFO:teuthology.task.parallel:starting parallel... 2024-03-22T07:14:06.895 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-22T07:14:06.895 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-22T07:14:06.896 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-03-22T07:14:06.896 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-22T07:14:06.896 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-22T07:14:06.896 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-03-22T07:14:06.900 INFO:tasks.workunit:Pulling workunits from ref 768f6cd6e95d0c7a626042719c230df59bdba9d3 2024-03-22T07:14:06.901 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-22T07:14:06.901 INFO:tasks.workunit:timeout=3h 2024-03-22T07:14:06.901 INFO:tasks.workunit:cleanup=True 2024-03-22T07:14:06.901 DEBUG:teuthology.orchestra.run.smithi063:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-22T07:14:06.922 INFO:teuthology.orchestra.run.smithi063.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-22T07:14:06.922 INFO:teuthology.orchestra.run.smithi063.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-22T07:14:06.922 INFO:teuthology.orchestra.run.smithi063.stdout:Device: 55h/85d Inode: 1 Links: 2 2024-03-22T07:14:06.922 INFO:teuthology.orchestra.run.smithi063.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:14:06.922 INFO:teuthology.orchestra.run.smithi063.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-22T07:14:06.922 INFO:teuthology.orchestra.run.smithi063.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-22T07:14:06.922 INFO:teuthology.orchestra.run.smithi063.stdout:Modify: 2024-03-22 07:13:34.626319428 +0000 2024-03-22T07:14:06.922 INFO:teuthology.orchestra.run.smithi063.stdout:Change: 2024-03-22 07:14:02.158817179 +0000 2024-03-22T07:14:06.922 INFO:teuthology.orchestra.run.smithi063.stdout: Birth: 2024-03-22 07:13:34.626319428 +0000 2024-03-22T07:14:06.923 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-22T07:14:06.923 DEBUG:teuthology.orchestra.run.smithi063:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-22T07:14:06.995 DEBUG:teuthology.orchestra.run.smithi069:> stat -- /home/ubuntu/cephtest/mnt.1 2024-03-22T07:14:07.010 INFO:teuthology.orchestra.run.smithi069.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-03-22T07:14:07.010 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2024-03-22T07:14:07.011 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 73h/115d Inode: 1 Links: 3 2024-03-22T07:14:07.011 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-22T07:14:07.011 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-22T07:14:07.011 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-22T07:14:07.011 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-03-22 07:14:06.989766970 +0000 2024-03-22T07:14:07.011 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-03-22 07:14:06.989766970 +0000 2024-03-22T07:14:07.011 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-03-22 07:13:34.626319428 +0000 2024-03-22T07:14:07.011 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-03-22T07:14:07.011 DEBUG:teuthology.orchestra.run.smithi069:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-03-22T07:14:07.081 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 768f6cd6e95d0c7a626042719c230df59bdba9d3 2024-03-22T07:14:07.082 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 768f6cd6e95d0c7a626042719c230df59bdba9d3 2024-03-22T07:14:07.097 INFO:tasks.workunit.client.0.smithi063.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-22T07:14:07.137 INFO:tasks.workunit.client.1.smithi069.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-03-22T07:14:07.178 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:14:07.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:07 smithi069 ceph-mon[36441]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 170 B/s wr, 4 op/s 2024-03-22T07:14:07.598 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:07 smithi063 ceph-mon[29200]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 170 B/s wr, 4 op/s 2024-03-22T07:14:08.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.728+0000 7f8470ad9700 1 -- 172.21.15.63:0/1409652384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f846c0fce80 msgr2=0x7f846c0fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:08.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.728+0000 7f8470ad9700 1 --2- 172.21.15.63:0/1409652384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f846c0fce80 0x7f846c0fd290 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f845c004940 tx=0x7f845c009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:08.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.728+0000 7f8470ad9700 1 -- 172.21.15.63:0/1409652384 shutdown_connections 2024-03-22T07:14:08.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.728+0000 7f8470ad9700 1 --2- 172.21.15.63:0/1409652384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f846c0fdff0 0x7f846c0fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.728+0000 7f8470ad9700 1 --2- 172.21.15.63:0/1409652384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f846c0fce80 0x7f846c0fd290 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.728+0000 7f8470ad9700 1 -- 172.21.15.63:0/1409652384 >> 172.21.15.63:0/1409652384 conn(0x7f846c0f8540 msgr2=0x7f846c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:08.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.728+0000 7f8470ad9700 1 -- 172.21.15.63:0/1409652384 shutdown_connections 2024-03-22T07:14:08.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.728+0000 7f8470ad9700 1 -- 172.21.15.63:0/1409652384 wait complete. 2024-03-22T07:14:08.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.729+0000 7f8470ad9700 1 Processor -- start 2024-03-22T07:14:08.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.729+0000 7f8470ad9700 1 -- start start 2024-03-22T07:14:08.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.729+0000 7f8470ad9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f846c0fce80 0x7f846c10d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:08.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.729+0000 7f8470ad9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f846c0fdff0 0x7f846c10def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:08.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.729+0000 7f8470ad9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c10e4f0 con 0x7f846c0fdff0 2024-03-22T07:14:08.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.729+0000 7f8470ad9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c10ed40 con 0x7f846c0fce80 2024-03-22T07:14:08.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.729+0000 7f8469d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f846c0fdff0 0x7f846c10def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:08.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.729+0000 7f8469d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f846c0fdff0 0x7f846c10def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51124/0 (socket says 172.21.15.63:51124) 2024-03-22T07:14:08.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.729+0000 7f8469d9b700 1 -- 172.21.15.63:0/2661298063 learned_addr learned my addr 172.21.15.63:0/2661298063 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:14:08.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.729+0000 7f846a59c700 1 --2- 172.21.15.63:0/2661298063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f846c0fce80 0x7f846c10d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:08.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.729+0000 7f8469d9b700 1 -- 172.21.15.63:0/2661298063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f846c0fce80 msgr2=0x7f846c10d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:08.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.729+0000 7f8469d9b700 1 --2- 172.21.15.63:0/2661298063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f846c0fce80 0x7f846c10d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.729+0000 7f8469d9b700 1 -- 172.21.15.63:0/2661298063 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c005040 con 0x7f846c0fdff0 2024-03-22T07:14:08.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.730+0000 7f8469d9b700 1 --2- 172.21.15.63:0/2661298063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f846c0fdff0 0x7f846c10def0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8460002a80 tx=0x7f84600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:08.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.730+0000 7f845affd700 1 -- 172.21.15.63:0/2661298063 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8460007cc0 con 0x7f846c0fdff0 2024-03-22T07:14:08.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.730+0000 7f845affd700 1 -- 172.21.15.63:0/2661298063 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f846000f040 con 0x7f846c0fdff0 2024-03-22T07:14:08.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.730+0000 7f8470ad9700 1 -- 172.21.15.63:0/2661298063 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846c10f020 con 0x7f846c0fdff0 2024-03-22T07:14:08.731 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.730+0000 7f845affd700 1 -- 172.21.15.63:0/2661298063 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f846000dac0 con 0x7f846c0fdff0 2024-03-22T07:14:08.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.730+0000 7f8470ad9700 1 -- 172.21.15.63:0/2661298063 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846c10f560 con 0x7f846c0fdff0 2024-03-22T07:14:08.732 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.731+0000 7f8470ad9700 1 -- 172.21.15.63:0/2661298063 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f846c05f5f0 con 0x7f846c0fdff0 2024-03-22T07:14:08.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.731+0000 7f845affd700 1 -- 172.21.15.63:0/2661298063 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f846000dc20 con 0x7f846c0fdff0 2024-03-22T07:14:08.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.732+0000 7f845affd700 1 --2- 172.21.15.63:0/2661298063 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f84540692b0 0x7f845406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:08.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.732+0000 7f845affd700 1 -- 172.21.15.63:0/2661298063 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8460086f60 con 0x7f846c0fdff0 2024-03-22T07:14:08.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.732+0000 7f846a59c700 1 --2- 172.21.15.63:0/2661298063 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f84540692b0 0x7f845406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:08.733 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.732+0000 7f846a59c700 1 --2- 172.21.15.63:0/2661298063 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f84540692b0 0x7f845406b760 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f845c00b3b0 tx=0x7f845c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:08.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.736+0000 7f845affd700 1 -- 172.21.15.63:0/2661298063 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8460056c50 con 0x7f846c0fdff0 2024-03-22T07:14:08.936 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.936+0000 7f8470ad9700 1 -- 172.21.15.63:0/2661298063 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f846c04bed0 con 0x7f846c0fdff0 2024-03-22T07:14:08.937 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.937+0000 7f845affd700 1 -- 172.21.15.63:0/2661298063 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f84600567e0 con 0x7f846c0fdff0 2024-03-22T07:14:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.939+0000 7f8470ad9700 1 -- 172.21.15.63:0/2661298063 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f84540692b0 msgr2=0x7f845406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.939+0000 7f8470ad9700 1 --2- 172.21.15.63:0/2661298063 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f84540692b0 0x7f845406b760 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f845c00b3b0 tx=0x7f845c009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.939+0000 7f8470ad9700 1 -- 172.21.15.63:0/2661298063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f846c0fdff0 msgr2=0x7f846c10def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.939+0000 7f8470ad9700 1 --2- 172.21.15.63:0/2661298063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f846c0fdff0 0x7f846c10def0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8460002a80 tx=0x7f84600046b0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.939+0000 7f8470ad9700 1 -- 172.21.15.63:0/2661298063 shutdown_connections 2024-03-22T07:14:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.939+0000 7f8470ad9700 1 --2- 172.21.15.63:0/2661298063 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f84540692b0 0x7f845406b760 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.939+0000 7f8470ad9700 1 --2- 172.21.15.63:0/2661298063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f846c0fce80 0x7f846c10d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.939+0000 7f8470ad9700 1 --2- 172.21.15.63:0/2661298063 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f846c0fdff0 0x7f846c10def0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.939+0000 7f8470ad9700 1 -- 172.21.15.63:0/2661298063 >> 172.21.15.63:0/2661298063 conn(0x7f846c0f8540 msgr2=0x7f846c0f9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.939+0000 7f8470ad9700 1 -- 172.21.15.63:0/2661298063 shutdown_connections 2024-03-22T07:14:08.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:08.939+0000 7f8470ad9700 1 -- 172.21.15.63:0/2661298063 wait complete. 2024-03-22T07:14:09.633 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-22T07:14:09.697 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:09 smithi063 ceph-mon[29200]: pgmap v197: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 170 B/s wr, 3 op/s 2024-03-22T07:14:09.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:09 smithi069 ceph-mon[36441]: pgmap v197: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 170 B/s wr, 3 op/s 2024-03-22T07:14:09.933 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:14:11.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.555+0000 7f2615efe700 1 -- 172.21.15.63:0/3033609703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26100fdfe0 msgr2=0x7f26100fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:11.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.555+0000 7f2615efe700 1 --2- 172.21.15.63:0/3033609703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26100fdfe0 0x7f26100fe450 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2604004940 tx=0x7f2604009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:11.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.555+0000 7f2615efe700 1 -- 172.21.15.63:0/3033609703 shutdown_connections 2024-03-22T07:14:11.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.555+0000 7f2615efe700 1 --2- 172.21.15.63:0/3033609703 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26100fdfe0 0x7f26100fe450 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:11.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.555+0000 7f2615efe700 1 --2- 172.21.15.63:0/3033609703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26100fce70 0x7f26100fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:11.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.555+0000 7f2615efe700 1 -- 172.21.15.63:0/3033609703 >> 172.21.15.63:0/3033609703 conn(0x7f26100f8550 msgr2=0x7f26100fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:11.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.555+0000 7f2615efe700 1 -- 172.21.15.63:0/3033609703 shutdown_connections 2024-03-22T07:14:11.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.555+0000 7f2615efe700 1 -- 172.21.15.63:0/3033609703 wait complete. 2024-03-22T07:14:11.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.556+0000 7f2615efe700 1 Processor -- start 2024-03-22T07:14:11.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.556+0000 7f2615efe700 1 -- start start 2024-03-22T07:14:11.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.556+0000 7f2615efe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26100fce70 0x7f261010d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:11.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.556+0000 7f2615efe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26100fdfe0 0x7f261010de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:11.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.556+0000 7f2615efe700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261010e510 con 0x7f26100fce70 2024-03-22T07:14:11.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.556+0000 7f2615efe700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2610112eb0 con 0x7f26100fdfe0 2024-03-22T07:14:11.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.556+0000 7f260f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26100fce70 0x7f261010d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:11.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.556+0000 7f260f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26100fce70 0x7f261010d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51146/0 (socket says 172.21.15.63:51146) 2024-03-22T07:14:11.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.556+0000 7f260f7fe700 1 -- 172.21.15.63:0/1535910864 learned_addr learned my addr 172.21.15.63:0/1535910864 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:14:11.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.557+0000 7f260effd700 1 --2- 172.21.15.63:0/1535910864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26100fdfe0 0x7f261010de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:11.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.557+0000 7f260f7fe700 1 -- 172.21.15.63:0/1535910864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26100fdfe0 msgr2=0x7f261010de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:11.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.557+0000 7f260f7fe700 1 --2- 172.21.15.63:0/1535910864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26100fdfe0 0x7f261010de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:11.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.557+0000 7f260f7fe700 1 -- 172.21.15.63:0/1535910864 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2604005040 con 0x7f26100fce70 2024-03-22T07:14:11.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.557+0000 7f260effd700 1 --2- 172.21.15.63:0/1535910864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26100fdfe0 0x7f261010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:14:11.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.557+0000 7f260f7fe700 1 --2- 172.21.15.63:0/1535910864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26100fce70 0x7f261010d940 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f260000b790 tx=0x7f2600009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:11.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.557+0000 7f25f7fff700 1 -- 172.21.15.63:0/1535910864 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f260000f770 con 0x7f26100fce70 2024-03-22T07:14:11.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.557+0000 7f25f7fff700 1 -- 172.21.15.63:0/1535910864 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f260000fda0 con 0x7f26100fce70 2024-03-22T07:14:11.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.557+0000 7f2615efe700 1 -- 172.21.15.63:0/1535910864 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2610113110 con 0x7f26100fce70 2024-03-22T07:14:11.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.557+0000 7f25f7fff700 1 -- 172.21.15.63:0/1535910864 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f260000d310 con 0x7f26100fce70 2024-03-22T07:14:11.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.557+0000 7f2615efe700 1 -- 172.21.15.63:0/1535910864 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2610113680 con 0x7f26100fce70 2024-03-22T07:14:11.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.558+0000 7f2615efe700 1 -- 172.21.15.63:0/1535910864 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261004c710 con 0x7f26100fce70 2024-03-22T07:14:11.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.558+0000 7f25f7fff700 1 -- 172.21.15.63:0/1535910864 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f260000f8d0 con 0x7f26100fce70 2024-03-22T07:14:11.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.559+0000 7f25f7fff700 1 --2- 172.21.15.63:0/1535910864 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f25f80692a0 0x7f25f806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:11.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.559+0000 7f25f7fff700 1 -- 172.21.15.63:0/1535910864 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f2600087260 con 0x7f26100fce70 2024-03-22T07:14:11.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.559+0000 7f260effd700 1 --2- 172.21.15.63:0/1535910864 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f25f80692a0 0x7f25f806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:11.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.559+0000 7f260effd700 1 --2- 172.21.15.63:0/1535910864 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f25f80692a0 0x7f25f806b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f260400b020 tx=0x7f2604009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:11.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.563+0000 7f25f7fff700 1 -- 172.21.15.63:0/1535910864 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f2600056fd0 con 0x7f26100fce70 2024-03-22T07:14:11.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:11 smithi063 ceph-mon[29200]: pgmap v198: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 682 B/s wr, 4 op/s 2024-03-22T07:14:11.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.758+0000 7f2615efe700 1 -- 172.21.15.63:0/1535910864 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f2610105950 con 0x7f26100fce70 2024-03-22T07:14:11.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.759+0000 7f25f7fff700 1 -- 172.21.15.63:0/1535910864 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f2600056b60 con 0x7f26100fce70 2024-03-22T07:14:11.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.761+0000 7f2615efe700 1 -- 172.21.15.63:0/1535910864 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f25f80692a0 msgr2=0x7f25f806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:11.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.761+0000 7f2615efe700 1 --2- 172.21.15.63:0/1535910864 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f25f80692a0 0x7f25f806b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f260400b020 tx=0x7f2604009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:11.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.761+0000 7f2615efe700 1 -- 172.21.15.63:0/1535910864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26100fce70 msgr2=0x7f261010d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:11.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.761+0000 7f2615efe700 1 --2- 172.21.15.63:0/1535910864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26100fce70 0x7f261010d940 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f260000b790 tx=0x7f2600009250 comp rx=0 tx=0).stop 2024-03-22T07:14:11.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.761+0000 7f2615efe700 1 -- 172.21.15.63:0/1535910864 shutdown_connections 2024-03-22T07:14:11.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.761+0000 7f2615efe700 1 --2- 172.21.15.63:0/1535910864 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f25f80692a0 0x7f25f806b750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:11.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.761+0000 7f2615efe700 1 --2- 172.21.15.63:0/1535910864 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26100fce70 0x7f261010d940 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:11.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.761+0000 7f2615efe700 1 --2- 172.21.15.63:0/1535910864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26100fdfe0 0x7f261010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:11.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.761+0000 7f2615efe700 1 -- 172.21.15.63:0/1535910864 >> 172.21.15.63:0/1535910864 conn(0x7f26100f8550 msgr2=0x7f26100f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:11.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.761+0000 7f2615efe700 1 -- 172.21.15.63:0/1535910864 shutdown_connections 2024-03-22T07:14:11.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:11.761+0000 7f2615efe700 1 -- 172.21.15.63:0/1535910864 wait complete. 2024-03-22T07:14:11.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:11 smithi069 ceph-mon[36441]: pgmap v198: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 682 B/s wr, 4 op/s 2024-03-22T07:14:12.366 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-22T07:14:12.678 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:14:13.761 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:13 smithi063 ceph-mon[29200]: pgmap v199: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-22T07:14:13.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:13 smithi069 ceph-mon[36441]: pgmap v199: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-22T07:14:14.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.251+0000 7fbd60180700 1 -- 172.21.15.63:0/383421311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd580fbcd0 msgr2=0x7fbd580fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:14.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.251+0000 7fbd60180700 1 --2- 172.21.15.63:0/383421311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd580fbcd0 0x7fbd580fc0e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c004940 tx=0x7fbd4c009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:14.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.251+0000 7fbd60180700 1 -- 172.21.15.63:0/383421311 shutdown_connections 2024-03-22T07:14:14.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.251+0000 7fbd60180700 1 --2- 172.21.15.63:0/383421311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd580fcf80 0x7fbd580fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:14.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.251+0000 7fbd60180700 1 --2- 172.21.15.63:0/383421311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd580fbcd0 0x7fbd580fc0e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:14.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.251+0000 7fbd60180700 1 -- 172.21.15.63:0/383421311 >> 172.21.15.63:0/383421311 conn(0x7fbd580f7430 msgr2=0x7fbd580f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:14.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.252+0000 7fbd60180700 1 -- 172.21.15.63:0/383421311 shutdown_connections 2024-03-22T07:14:14.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.252+0000 7fbd60180700 1 -- 172.21.15.63:0/383421311 wait complete. 2024-03-22T07:14:14.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.252+0000 7fbd60180700 1 Processor -- start 2024-03-22T07:14:14.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.252+0000 7fbd60180700 1 -- start start 2024-03-22T07:14:14.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.252+0000 7fbd60180700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd580fbcd0 0x7fbd58066d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:14.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.252+0000 7fbd60180700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd580fcf80 0x7fbd58112700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:14.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.252+0000 7fbd60180700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd58112c40 con 0x7fbd580fcf80 2024-03-22T07:14:14.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.252+0000 7fbd60180700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd58112d80 con 0x7fbd580fbcd0 2024-03-22T07:14:14.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.252+0000 7fbd5d71b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd580fcf80 0x7fbd58112700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:14.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.252+0000 7fbd5d71b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd580fcf80 0x7fbd58112700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51156/0 (socket says 172.21.15.63:51156) 2024-03-22T07:14:14.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.252+0000 7fbd5d71b700 1 -- 172.21.15.63:0/3176110530 learned_addr learned my addr 172.21.15.63:0/3176110530 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:14:14.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.253+0000 7fbd5df1c700 1 --2- 172.21.15.63:0/3176110530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd580fbcd0 0x7fbd58066d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:14.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.253+0000 7fbd5d71b700 1 -- 172.21.15.63:0/3176110530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd580fbcd0 msgr2=0x7fbd58066d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:14.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.253+0000 7fbd5d71b700 1 --2- 172.21.15.63:0/3176110530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd580fbcd0 0x7fbd58066d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:14.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.253+0000 7fbd5d71b700 1 -- 172.21.15.63:0/3176110530 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4c005040 con 0x7fbd580fcf80 2024-03-22T07:14:14.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.253+0000 7fbd5d71b700 1 --2- 172.21.15.63:0/3176110530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd580fcf80 0x7fbd58112700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400c830 tx=0x7fbd54004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:14.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.253+0000 7fbd4a7fc700 1 -- 172.21.15.63:0/3176110530 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd5400a700 con 0x7fbd580fcf80 2024-03-22T07:14:14.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.253+0000 7fbd4a7fc700 1 -- 172.21.15.63:0/3176110530 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd5400ad30 con 0x7fbd580fcf80 2024-03-22T07:14:14.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.253+0000 7fbd60180700 1 -- 172.21.15.63:0/3176110530 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd58113010 con 0x7fbd580fcf80 2024-03-22T07:14:14.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.253+0000 7fbd4a7fc700 1 -- 172.21.15.63:0/3176110530 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd540113a0 con 0x7fbd580fcf80 2024-03-22T07:14:14.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.253+0000 7fbd60180700 1 -- 172.21.15.63:0/3176110530 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd581135a0 con 0x7fbd580fcf80 2024-03-22T07:14:14.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.254+0000 7fbd60180700 1 -- 172.21.15.63:0/3176110530 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5804c710 con 0x7fbd580fcf80 2024-03-22T07:14:14.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.255+0000 7fbd4a7fc700 1 -- 172.21.15.63:0/3176110530 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fbd54012400 con 0x7fbd580fcf80 2024-03-22T07:14:14.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.255+0000 7fbd4a7fc700 1 --2- 172.21.15.63:0/3176110530 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fbd440692a0 0x7fbd4406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:14.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.255+0000 7fbd4a7fc700 1 -- 172.21.15.63:0/3176110530 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fbd540871c0 con 0x7fbd580fcf80 2024-03-22T07:14:14.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.255+0000 7fbd5df1c700 1 --2- 172.21.15.63:0/3176110530 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fbd440692a0 0x7fbd4406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:14.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.256+0000 7fbd5df1c700 1 --2- 172.21.15.63:0/3176110530 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fbd440692a0 0x7fbd4406b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00b3b0 tx=0x7fbd4c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:14.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.260+0000 7fbd4a7fc700 1 -- 172.21.15.63:0/3176110530 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fbd54056fd0 con 0x7fbd580fcf80 2024-03-22T07:14:14.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.456+0000 7fbd60180700 1 -- 172.21.15.63:0/3176110530 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fbd58113840 con 0x7fbd580fcf80 2024-03-22T07:14:14.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.456+0000 7fbd4a7fc700 1 -- 172.21.15.63:0/3176110530 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7fbd54056b60 con 0x7fbd580fcf80 2024-03-22T07:14:14.458 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.458+0000 7fbd60180700 1 -- 172.21.15.63:0/3176110530 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fbd440692a0 msgr2=0x7fbd4406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.458+0000 7fbd60180700 1 --2- 172.21.15.63:0/3176110530 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fbd440692a0 0x7fbd4406b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00b3b0 tx=0x7fbd4c009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.458+0000 7fbd60180700 1 -- 172.21.15.63:0/3176110530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd580fcf80 msgr2=0x7fbd58112700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.458+0000 7fbd60180700 1 --2- 172.21.15.63:0/3176110530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd580fcf80 0x7fbd58112700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400c830 tx=0x7fbd54004470 comp rx=0 tx=0).stop 2024-03-22T07:14:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.458+0000 7fbd60180700 1 -- 172.21.15.63:0/3176110530 shutdown_connections 2024-03-22T07:14:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.458+0000 7fbd60180700 1 --2- 172.21.15.63:0/3176110530 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fbd440692a0 0x7fbd4406b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.458+0000 7fbd60180700 1 --2- 172.21.15.63:0/3176110530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd580fbcd0 0x7fbd58066d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.458+0000 7fbd60180700 1 --2- 172.21.15.63:0/3176110530 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbd580fcf80 0x7fbd58112700 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.458+0000 7fbd60180700 1 -- 172.21.15.63:0/3176110530 >> 172.21.15.63:0/3176110530 conn(0x7fbd580f7430 msgr2=0x7fbd580f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.459+0000 7fbd60180700 1 -- 172.21.15.63:0/3176110530 shutdown_connections 2024-03-22T07:14:14.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:14.459+0000 7fbd60180700 1 -- 172.21.15.63:0/3176110530 wait complete. 2024-03-22T07:14:14.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:14 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:14:14.825 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:14 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:14:15.093 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-03-22T07:14:15.355 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:14:15.679 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:15 smithi063 ceph-mon[29200]: pgmap v200: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-03-22T07:14:15.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:15 smithi069 ceph-mon[36441]: pgmap v200: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-03-22T07:14:16.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.927+0000 7ff6fa2a0700 1 -- 172.21.15.63:0/718387232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6f40fe450 msgr2=0x7ff6f40fe860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:16.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.927+0000 7ff6fa2a0700 1 --2- 172.21.15.63:0/718387232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6f40fe450 0x7ff6f40fe860 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4004940 tx=0x7ff6e4009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:16.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.927+0000 7ff6fa2a0700 1 -- 172.21.15.63:0/718387232 shutdown_connections 2024-03-22T07:14:16.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.927+0000 7ff6fa2a0700 1 --2- 172.21.15.63:0/718387232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f40feda0 0x7ff6f40faee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:16.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.927+0000 7ff6fa2a0700 1 --2- 172.21.15.63:0/718387232 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6f40fe450 0x7ff6f40fe860 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:16.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.927+0000 7ff6fa2a0700 1 -- 172.21.15.63:0/718387232 >> 172.21.15.63:0/718387232 conn(0x7ff6f40f6c10 msgr2=0x7ff6f40f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:16.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.928+0000 7ff6fa2a0700 1 -- 172.21.15.63:0/718387232 shutdown_connections 2024-03-22T07:14:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.928+0000 7ff6fa2a0700 1 -- 172.21.15.63:0/718387232 wait complete. 2024-03-22T07:14:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.928+0000 7ff6fa2a0700 1 Processor -- start 2024-03-22T07:14:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.928+0000 7ff6fa2a0700 1 -- start start 2024-03-22T07:14:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.928+0000 7ff6fa2a0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6f40fe450 0x7ff6f410d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.928+0000 7ff6fa2a0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f40feda0 0x7ff6f410dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.928+0000 7ff6fa2a0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f410e4a0 con 0x7ff6f40fe450 2024-03-22T07:14:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.928+0000 7ff6fa2a0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f410e610 con 0x7ff6f40feda0 2024-03-22T07:14:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.928+0000 7ff6f3fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6f40fe450 0x7ff6f410d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.928+0000 7ff6f3fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6f40fe450 0x7ff6f410d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:48870/0 (socket says 172.21.15.63:48870) 2024-03-22T07:14:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.929+0000 7ff6f3fff700 1 -- 172.21.15.63:0/157386710 learned_addr learned my addr 172.21.15.63:0/157386710 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:14:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.929+0000 7ff6f37fe700 1 --2- 172.21.15.63:0/157386710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f40feda0 0x7ff6f410dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.929+0000 7ff6f3fff700 1 -- 172.21.15.63:0/157386710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f40feda0 msgr2=0x7ff6f410dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.929+0000 7ff6f3fff700 1 --2- 172.21.15.63:0/157386710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f40feda0 0x7ff6f410dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:16.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.929+0000 7ff6f3fff700 1 -- 172.21.15.63:0/157386710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e4005040 con 0x7ff6f40fe450 2024-03-22T07:14:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.929+0000 7ff6f3fff700 1 --2- 172.21.15.63:0/157386710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6f40fe450 0x7ff6f410d960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400b3b0 tx=0x7ff6e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.929+0000 7ff6f0ff9700 1 -- 172.21.15.63:0/157386710 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6e400e040 con 0x7ff6f40fe450 2024-03-22T07:14:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.929+0000 7ff6fa2a0700 1 -- 172.21.15.63:0/157386710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f4113010 con 0x7ff6f40fe450 2024-03-22T07:14:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.929+0000 7ff6f0ff9700 1 -- 172.21.15.63:0/157386710 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6e400bbf0 con 0x7ff6f40fe450 2024-03-22T07:14:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.929+0000 7ff6f0ff9700 1 -- 172.21.15.63:0/157386710 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6e401c630 con 0x7ff6f40fe450 2024-03-22T07:14:16.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.929+0000 7ff6fa2a0700 1 -- 172.21.15.63:0/157386710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f41135d0 con 0x7ff6f40fe450 2024-03-22T07:14:16.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.930+0000 7ff6fa2a0700 1 -- 172.21.15.63:0/157386710 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f404c710 con 0x7ff6f40fe450 2024-03-22T07:14:16.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.931+0000 7ff6f0ff9700 1 -- 172.21.15.63:0/157386710 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff6e400bd60 con 0x7ff6f40fe450 2024-03-22T07:14:16.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.931+0000 7ff6f0ff9700 1 --2- 172.21.15.63:0/157386710 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ff6dc0692a0 0x7ff6dc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:16.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.931+0000 7ff6f0ff9700 1 -- 172.21.15.63:0/157386710 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff6e4087e90 con 0x7ff6f40fe450 2024-03-22T07:14:16.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.931+0000 7ff6f37fe700 1 --2- 172.21.15.63:0/157386710 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ff6dc0692a0 0x7ff6dc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:16.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.931+0000 7ff6f37fe700 1 --2- 172.21.15.63:0/157386710 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ff6dc0692a0 0x7ff6dc06b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8004f40 tx=0x7ff6e800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:16.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:16.935+0000 7ff6f0ff9700 1 -- 172.21.15.63:0/157386710 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff6e4057c00 con 0x7ff6f40fe450 2024-03-22T07:14:17.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:17.148+0000 7ff6fa2a0700 1 -- 172.21.15.63:0/157386710 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6f41025c0 con 0x7ff6dc0692a0 2024-03-22T07:14:17.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:17 smithi069 ceph-mon[36441]: pgmap v201: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-22T07:14:17.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:17 smithi063 ceph-mon[29200]: pgmap v201: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-03-22T07:14:18.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:18 smithi069 ceph-mon[36441]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:18.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:18 smithi063 ceph-mon[29200]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:19.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:19 smithi069 ceph-mon[36441]: pgmap v202: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 597 B/s wr, 4 op/s 2024-03-22T07:14:19.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:19 smithi063 ceph-mon[29200]: pgmap v202: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 597 B/s wr, 4 op/s 2024-03-22T07:14:21.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:21 smithi069 ceph-mon[36441]: pgmap v203: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 597 B/s wr, 4 op/s 2024-03-22T07:14:21.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:21 smithi063 ceph-mon[29200]: pgmap v203: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 597 B/s wr, 4 op/s 2024-03-22T07:14:23.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:23 smithi069 ceph-mon[36441]: pgmap v204: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 4 op/s 2024-03-22T07:14:23.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:23 smithi063 ceph-mon[29200]: pgmap v204: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 4 op/s 2024-03-22T07:14:25.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:25 smithi069 ceph-mon[36441]: pgmap v205: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:14:25.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:25 smithi063 ceph-mon[29200]: pgmap v205: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:14:27.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:27 smithi069 ceph-mon[36441]: pgmap v206: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:14:27.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:27 smithi063 ceph-mon[29200]: pgmap v206: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:14:29.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:29 smithi069 ceph-mon[36441]: pgmap v207: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:14:29.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:29 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:14:29.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:29 smithi063 ceph-mon[29200]: pgmap v207: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:14:29.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:29 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:14:31.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:31 smithi069 ceph-mon[36441]: pgmap v208: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-22T07:14:31.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:31 smithi063 ceph-mon[29200]: pgmap v208: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-22T07:14:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:33 smithi069 ceph-mon[36441]: pgmap v209: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-22T07:14:33.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:33 smithi063 ceph-mon[29200]: pgmap v209: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-22T07:14:35.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:35 smithi069 ceph-mon[36441]: pgmap v210: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:14:35.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:35 smithi063 ceph-mon[29200]: pgmap v210: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:14:37.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:37 smithi069 ceph-mon[36441]: pgmap v211: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:14:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:37 smithi063 ceph-mon[29200]: pgmap v211: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:14:39.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:39 smithi069 ceph-mon[36441]: pgmap v212: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:14:39.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:39 smithi063 ceph-mon[29200]: pgmap v212: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:14:41.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:41 smithi069 ceph-mon[36441]: pgmap v213: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 255 B/s wr, 4 op/s 2024-03-22T07:14:41.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:41 smithi063 ceph-mon[29200]: pgmap v213: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 255 B/s wr, 4 op/s 2024-03-22T07:14:43.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:43 smithi069 ceph-mon[36441]: pgmap v214: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-03-22T07:14:43.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:43 smithi063 ceph-mon[29200]: pgmap v214: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-03-22T07:14:44.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:44 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:14:44.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:14:45.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:45 smithi069 ceph-mon[36441]: pgmap v215: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-03-22T07:14:45.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:45 smithi063 ceph-mon[29200]: pgmap v215: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-03-22T07:14:47.614 INFO:teuthology.orchestra.run.smithi063.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:14:47.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:47.613+0000 7ff6f0ff9700 1 -- 172.21.15.63:0/157386710 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7ff6f41025c0 con 0x7ff6dc0692a0 2024-03-22T07:14:47.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:47.616+0000 7ff6fa2a0700 1 -- 172.21.15.63:0/157386710 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ff6dc0692a0 msgr2=0x7ff6dc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:47.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:47.616+0000 7ff6fa2a0700 1 --2- 172.21.15.63:0/157386710 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ff6dc0692a0 0x7ff6dc06b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8004f40 tx=0x7ff6e800b040 comp rx=0 tx=0).stop 2024-03-22T07:14:47.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:47.616+0000 7ff6fa2a0700 1 -- 172.21.15.63:0/157386710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6f40fe450 msgr2=0x7ff6f410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:47.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:47.616+0000 7ff6fa2a0700 1 --2- 172.21.15.63:0/157386710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6f40fe450 0x7ff6f410d960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400b3b0 tx=0x7ff6e4009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:47.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:47.616+0000 7ff6fa2a0700 1 -- 172.21.15.63:0/157386710 shutdown_connections 2024-03-22T07:14:47.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:47.616+0000 7ff6fa2a0700 1 --2- 172.21.15.63:0/157386710 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ff6dc0692a0 0x7ff6dc06b750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:47.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:47.616+0000 7ff6fa2a0700 1 --2- 172.21.15.63:0/157386710 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6f40fe450 0x7ff6f410d960 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:47.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:47.616+0000 7ff6fa2a0700 1 --2- 172.21.15.63:0/157386710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f40feda0 0x7ff6f410dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:47.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:47.616+0000 7ff6fa2a0700 1 -- 172.21.15.63:0/157386710 >> 172.21.15.63:0/157386710 conn(0x7ff6f40f6c10 msgr2=0x7ff6f40f8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:47.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:47.616+0000 7ff6fa2a0700 1 -- 172.21.15.63:0/157386710 shutdown_connections 2024-03-22T07:14:47.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:47.616+0000 7ff6fa2a0700 1 -- 172.21.15.63:0/157386710 wait complete. 2024-03-22T07:14:47.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:47 smithi069 ceph-mon[36441]: pgmap v216: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-22T07:14:47.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:47 smithi063 ceph-mon[29200]: pgmap v216: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-22T07:14:48.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:48 smithi063 ceph-mon[29200]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:14:48.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:48 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:14:48.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:48 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:14:48.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:48 smithi063 ceph-mon[29200]: pgmap v217: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-03-22T07:14:49.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:48 smithi069 ceph-mon[36441]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:14:49.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:48 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:14:49.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:48 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:14:49.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:48 smithi069 ceph-mon[36441]: pgmap v217: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-03-22T07:14:49.970 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-03-22T07:14:50.334 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:14:50.495 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:50 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:14:50.495 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:50 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:14:50.496 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:50 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:14:50.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:50 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:14:50.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:50 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:14:50.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:50 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:14:51.610 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:51 smithi063 ceph-mon[29200]: pgmap v218: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-03-22T07:14:51.610 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:51 smithi063 ceph-mon[29200]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:14:51.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:51 smithi069 ceph-mon[36441]: pgmap v218: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-03-22T07:14:51.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:51 smithi069 ceph-mon[36441]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:14:52.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.241+0000 7f658eb27700 1 -- 172.21.15.63:0/1732835073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fcef0 msgr2=0x7f65880fd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:52.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.241+0000 7f658eb27700 1 --2- 172.21.15.63:0/1732835073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fcef0 0x7f65880fd300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f657c004940 tx=0x7f657c009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:52.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.241+0000 7f658eb27700 1 -- 172.21.15.63:0/1732835073 shutdown_connections 2024-03-22T07:14:52.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.241+0000 7f658eb27700 1 --2- 172.21.15.63:0/1732835073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65880fe060 0x7f65880fe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:52.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.241+0000 7f658eb27700 1 --2- 172.21.15.63:0/1732835073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fcef0 0x7f65880fd300 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:52.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.241+0000 7f658eb27700 1 -- 172.21.15.63:0/1732835073 >> 172.21.15.63:0/1732835073 conn(0x7f65880f85b0 msgr2=0x7f65880fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:52.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.241+0000 7f658eb27700 1 -- 172.21.15.63:0/1732835073 shutdown_connections 2024-03-22T07:14:52.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.241+0000 7f658eb27700 1 -- 172.21.15.63:0/1732835073 wait complete. 2024-03-22T07:14:52.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.242+0000 7f658eb27700 1 Processor -- start 2024-03-22T07:14:52.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.242+0000 7f658eb27700 1 -- start start 2024-03-22T07:14:52.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.242+0000 7f658eb27700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65880fcef0 0x7f658810d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:52.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.242+0000 7f658eb27700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fe060 0x7f658810def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:52.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.242+0000 7f658eb27700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658810e4f0 con 0x7f65880fe060 2024-03-22T07:14:52.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.242+0000 7f658eb27700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658810ed40 con 0x7f65880fcef0 2024-03-22T07:14:52.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.242+0000 7f6587fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fe060 0x7f658810def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:52.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.242+0000 7f658c8c3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65880fcef0 0x7f658810d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:52.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.242+0000 7f658c8c3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65880fcef0 0x7f658810d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.63:37018/0 (socket says 172.21.15.63:37018) 2024-03-22T07:14:52.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.242+0000 7f6587fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fe060 0x7f658810def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39940/0 (socket says 172.21.15.63:39940) 2024-03-22T07:14:52.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.242+0000 7f658c8c3700 1 -- 172.21.15.63:0/2863369611 learned_addr learned my addr 172.21.15.63:0/2863369611 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:14:52.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.243+0000 7f658c8c3700 1 -- 172.21.15.63:0/2863369611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fe060 msgr2=0x7f658810def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:52.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.243+0000 7f658c8c3700 1 --2- 172.21.15.63:0/2863369611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fe060 0x7f658810def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:52.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.243+0000 7f658c8c3700 1 -- 172.21.15.63:0/2863369611 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c005040 con 0x7f65880fcef0 2024-03-22T07:14:52.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.243+0000 7f6587fff700 1 --2- 172.21.15.63:0/2863369611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fe060 0x7f658810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:14:52.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.243+0000 7f658c8c3700 1 --2- 172.21.15.63:0/2863369611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65880fcef0 0x7f658810d9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f657c013870 tx=0x7f657c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:52.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.243+0000 7f65857fa700 1 -- 172.21.15.63:0/2863369611 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f657c021070 con 0x7f65880fcef0 2024-03-22T07:14:52.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.243+0000 7f658eb27700 1 -- 172.21.15.63:0/2863369611 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658810efc0 con 0x7f65880fcef0 2024-03-22T07:14:52.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.243+0000 7f65857fa700 1 -- 172.21.15.63:0/2863369611 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f657c009430 con 0x7f65880fcef0 2024-03-22T07:14:52.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.243+0000 7f65857fa700 1 -- 172.21.15.63:0/2863369611 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f657c00e070 con 0x7f65880fcef0 2024-03-22T07:14:52.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.243+0000 7f658eb27700 1 -- 172.21.15.63:0/2863369611 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f658810f500 con 0x7f65880fcef0 2024-03-22T07:14:52.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.244+0000 7f658eb27700 1 -- 172.21.15.63:0/2863369611 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658805f5f0 con 0x7f65880fcef0 2024-03-22T07:14:52.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.245+0000 7f65857fa700 1 -- 172.21.15.63:0/2863369611 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f657c01c750 con 0x7f65880fcef0 2024-03-22T07:14:52.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.246+0000 7f65857fa700 1 --2- 172.21.15.63:0/2863369611 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f65780692a0 0x7f657806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:52.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.246+0000 7f65857fa700 1 -- 172.21.15.63:0/2863369611 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f657c0883e0 con 0x7f65880fcef0 2024-03-22T07:14:52.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.246+0000 7f6587fff700 1 --2- 172.21.15.63:0/2863369611 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f65780692a0 0x7f657806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:52.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.247+0000 7f6587fff700 1 --2- 172.21.15.63:0/2863369611 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f65780692a0 0x7f657806b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6574009b80 tx=0x7f657400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:52.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.254+0000 7f65857fa700 1 -- 172.21.15.63:0/2863369611 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f657c0581f0 con 0x7f65880fcef0 2024-03-22T07:14:52.615 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.615+0000 7f658eb27700 1 -- 172.21.15.63:0/2863369611 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f658810f830 con 0x7f65780692a0 2024-03-22T07:14:52.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.617+0000 7f65857fa700 1 -- 172.21.15.63:0/2863369611 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f658810f830 con 0x7f65780692a0 2024-03-22T07:14:52.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.620+0000 7f658eb27700 1 -- 172.21.15.63:0/2863369611 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f65780692a0 msgr2=0x7f657806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:52.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.620+0000 7f658eb27700 1 --2- 172.21.15.63:0/2863369611 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f65780692a0 0x7f657806b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6574009b80 tx=0x7f657400b040 comp rx=0 tx=0).stop 2024-03-22T07:14:52.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.620+0000 7f658eb27700 1 -- 172.21.15.63:0/2863369611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65880fcef0 msgr2=0x7f658810d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:52.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.620+0000 7f658eb27700 1 --2- 172.21.15.63:0/2863369611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65880fcef0 0x7f658810d9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f657c013870 tx=0x7f657c009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:52.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.620+0000 7f658eb27700 1 -- 172.21.15.63:0/2863369611 shutdown_connections 2024-03-22T07:14:52.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.620+0000 7f658eb27700 1 --2- 172.21.15.63:0/2863369611 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f65780692a0 0x7f657806b750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:52.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.620+0000 7f658eb27700 1 --2- 172.21.15.63:0/2863369611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65880fcef0 0x7f658810d9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:52.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.620+0000 7f658eb27700 1 --2- 172.21.15.63:0/2863369611 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f65880fe060 0x7f658810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:52.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.621+0000 7f658eb27700 1 -- 172.21.15.63:0/2863369611 >> 172.21.15.63:0/2863369611 conn(0x7f65880f85b0 msgr2=0x7f65880f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:52.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.621+0000 7f658eb27700 1 -- 172.21.15.63:0/2863369611 shutdown_connections 2024-03-22T07:14:52.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.621+0000 7f658eb27700 1 -- 172.21.15.63:0/2863369611 wait complete. 2024-03-22T07:14:52.634 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:14:52.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.742+0000 7fa6040ea700 1 -- 172.21.15.63:0/3114673387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5fc0fc2b0 msgr2=0x7fa5fc0fc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:52.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.742+0000 7fa6040ea700 1 --2- 172.21.15.63:0/3114673387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5fc0fc2b0 0x7fa5fc0fc6c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8004940 tx=0x7fa5f8009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:52.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.742+0000 7fa6040ea700 1 -- 172.21.15.63:0/3114673387 shutdown_connections 2024-03-22T07:14:52.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.742+0000 7fa6040ea700 1 --2- 172.21.15.63:0/3114673387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0fcc00 0x7fa5fc0fd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:52.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.742+0000 7fa6040ea700 1 --2- 172.21.15.63:0/3114673387 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5fc0fc2b0 0x7fa5fc0fc6c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:52.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.742+0000 7fa6040ea700 1 -- 172.21.15.63:0/3114673387 >> 172.21.15.63:0/3114673387 conn(0x7fa5fc0f6ba0 msgr2=0x7fa5fc0f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:52.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.743+0000 7fa6040ea700 1 -- 172.21.15.63:0/3114673387 shutdown_connections 2024-03-22T07:14:52.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.743+0000 7fa6040ea700 1 -- 172.21.15.63:0/3114673387 wait complete. 2024-03-22T07:14:52.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.743+0000 7fa6040ea700 1 Processor -- start 2024-03-22T07:14:52.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.744+0000 7fa6040ea700 1 -- start start 2024-03-22T07:14:52.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.744+0000 7fa6040ea700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0fc2b0 0x7fa5fc113ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:52.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.744+0000 7fa6040ea700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5fc0fcc00 0x7fa5fc114530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:52.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.744+0000 7fa6040ea700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc114b30 con 0x7fa5fc0fcc00 2024-03-22T07:14:52.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.744+0000 7fa601e86700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0fc2b0 0x7fa5fc113ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:52.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.744+0000 7fa601e86700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0fc2b0 0x7fa5fc113ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.63:37030/0 (socket says 172.21.15.63:37030) 2024-03-22T07:14:52.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.744+0000 7fa601e86700 1 -- 172.21.15.63:0/1680085093 learned_addr learned my addr 172.21.15.63:0/1680085093 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:14:52.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.744+0000 7fa6040ea700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc115380 con 0x7fa5fc0fc2b0 2024-03-22T07:14:52.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.744+0000 7fa601685700 1 --2- 172.21.15.63:0/1680085093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5fc0fcc00 0x7fa5fc114530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:52.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.745+0000 7fa601e86700 1 -- 172.21.15.63:0/1680085093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5fc0fcc00 msgr2=0x7fa5fc114530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:52.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.745+0000 7fa601e86700 1 --2- 172.21.15.63:0/1680085093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5fc0fcc00 0x7fa5fc114530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:52.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.745+0000 7fa601e86700 1 -- 172.21.15.63:0/1680085093 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f8005040 con 0x7fa5fc0fc2b0 2024-03-22T07:14:52.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.745+0000 7fa601e86700 1 --2- 172.21.15.63:0/1680085093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0fc2b0 0x7fa5fc113ff0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80043d0 tx=0x7fa5f8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:52.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.745+0000 7fa5f27fc700 1 -- 172.21.15.63:0/1680085093 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5f800e070 con 0x7fa5fc0fc2b0 2024-03-22T07:14:52.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.745+0000 7fa6040ea700 1 -- 172.21.15.63:0/1680085093 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5fc115600 con 0x7fa5fc0fc2b0 2024-03-22T07:14:52.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.745+0000 7fa5f27fc700 1 -- 172.21.15.63:0/1680085093 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5f801c400 con 0x7fa5fc0fc2b0 2024-03-22T07:14:52.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.745+0000 7fa5f27fc700 1 -- 172.21.15.63:0/1680085093 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5f800b650 con 0x7fa5fc0fc2b0 2024-03-22T07:14:52.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.746+0000 7fa6040ea700 1 -- 172.21.15.63:0/1680085093 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5fc115ae0 con 0x7fa5fc0fc2b0 2024-03-22T07:14:52.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.747+0000 7fa6040ea700 1 -- 172.21.15.63:0/1680085093 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5fc05f5f0 con 0x7fa5fc0fc2b0 2024-03-22T07:14:52.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.747+0000 7fa5f27fc700 1 -- 172.21.15.63:0/1680085093 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa5f801ca90 con 0x7fa5fc0fc2b0 2024-03-22T07:14:52.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.748+0000 7fa5f27fc700 1 --2- 172.21.15.63:0/1680085093 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa5e8069300 0x7fa5e806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:52.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.748+0000 7fa5f27fc700 1 -- 172.21.15.63:0/1680085093 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa5f8088710 con 0x7fa5fc0fc2b0 2024-03-22T07:14:52.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.748+0000 7fa601685700 1 --2- 172.21.15.63:0/1680085093 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa5e8069300 0x7fa5e806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:52.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.749+0000 7fa601685700 1 --2- 172.21.15.63:0/1680085093 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa5e8069300 0x7fa5e806b7b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec009b80 tx=0x7fa5ec00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:52.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:52.757+0000 7fa5f27fc700 1 -- 172.21.15.63:0/1680085093 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa5f8058520 con 0x7fa5fc0fc2b0 2024-03-22T07:14:53.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.144+0000 7fa6040ea700 1 -- 172.21.15.63:0/1680085093 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5fc115dc0 con 0x7fa5e8069300 2024-03-22T07:14:53.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.146+0000 7fa5f27fc700 1 -- 172.21.15.63:0/1680085093 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7fa5fc115dc0 con 0x7fa5e8069300 2024-03-22T07:14:53.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.148+0000 7fa6040ea700 1 -- 172.21.15.63:0/1680085093 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa5e8069300 msgr2=0x7fa5e806b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:53.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.148+0000 7fa6040ea700 1 --2- 172.21.15.63:0/1680085093 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa5e8069300 0x7fa5e806b7b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec009b80 tx=0x7fa5ec00b040 comp rx=0 tx=0).stop 2024-03-22T07:14:53.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.148+0000 7fa6040ea700 1 -- 172.21.15.63:0/1680085093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0fc2b0 msgr2=0x7fa5fc113ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:53.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.148+0000 7fa6040ea700 1 --2- 172.21.15.63:0/1680085093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0fc2b0 0x7fa5fc113ff0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80043d0 tx=0x7fa5f8009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:53.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.148+0000 7fa6040ea700 1 -- 172.21.15.63:0/1680085093 shutdown_connections 2024-03-22T07:14:53.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.148+0000 7fa6040ea700 1 --2- 172.21.15.63:0/1680085093 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa5e8069300 0x7fa5e806b7b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:53.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.148+0000 7fa6040ea700 1 --2- 172.21.15.63:0/1680085093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5fc0fc2b0 0x7fa5fc113ff0 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80043d0 tx=0x7fa5f8009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:53.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.148+0000 7fa6040ea700 1 --2- 172.21.15.63:0/1680085093 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa5fc0fcc00 0x7fa5fc114530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:53.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.148+0000 7fa6040ea700 1 -- 172.21.15.63:0/1680085093 >> 172.21.15.63:0/1680085093 conn(0x7fa5fc0f6ba0 msgr2=0x7fa5fc0f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:53.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.149+0000 7fa6040ea700 1 -- 172.21.15.63:0/1680085093 shutdown_connections 2024-03-22T07:14:53.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.149+0000 7fa6040ea700 1 -- 172.21.15.63:0/1680085093 wait complete. 2024-03-22T07:14:53.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.293+0000 7ff966b9e700 1 -- 172.21.15.63:0/2657907422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9600fdbe0 msgr2=0x7ff9600fc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:53.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.293+0000 7ff966b9e700 1 --2- 172.21.15.63:0/2657907422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9600fdbe0 0x7ff9600fc000 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff94c004940 tx=0x7ff94c009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:53.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.293+0000 7ff966b9e700 1 -- 172.21.15.63:0/2657907422 shutdown_connections 2024-03-22T07:14:53.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.293+0000 7ff966b9e700 1 --2- 172.21.15.63:0/2657907422 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff9600fdbe0 0x7ff9600fc000 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:53.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.293+0000 7ff966b9e700 1 --2- 172.21.15.63:0/2657907422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9600fd290 0x7ff9600fd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:53.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.293+0000 7ff966b9e700 1 -- 172.21.15.63:0/2657907422 >> 172.21.15.63:0/2657907422 conn(0x7ff9600f6ba0 msgr2=0x7ff9600f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:53.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.293+0000 7ff966b9e700 1 -- 172.21.15.63:0/2657907422 shutdown_connections 2024-03-22T07:14:53.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.294+0000 7ff966b9e700 1 -- 172.21.15.63:0/2657907422 wait complete. 2024-03-22T07:14:53.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.294+0000 7ff966b9e700 1 Processor -- start 2024-03-22T07:14:53.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.294+0000 7ff966b9e700 1 -- start start 2024-03-22T07:14:53.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.295+0000 7ff966b9e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9600fd290 0x7ff96010fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.295+0000 7ff966b9e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff960110170 0x7ff960110f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.295+0000 7ff966b9e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff960110630 con 0x7ff960110170 2024-03-22T07:14:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.295+0000 7ff966b9e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff960110770 con 0x7ff9600fd290 2024-03-22T07:14:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.295+0000 7ff95ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff960110170 0x7ff960110f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.295+0000 7ff95ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff960110170 0x7ff960110f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40000/0 (socket says 172.21.15.63:40000) 2024-03-22T07:14:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.295+0000 7ff95ffff700 1 -- 172.21.15.63:0/3025432793 learned_addr learned my addr 172.21.15.63:0/3025432793 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:14:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.295+0000 7ff96493a700 1 --2- 172.21.15.63:0/3025432793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9600fd290 0x7ff96010fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.295+0000 7ff95ffff700 1 -- 172.21.15.63:0/3025432793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9600fd290 msgr2=0x7ff96010fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.295+0000 7ff95ffff700 1 --2- 172.21.15.63:0/3025432793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9600fd290 0x7ff96010fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.295+0000 7ff95ffff700 1 -- 172.21.15.63:0/3025432793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c005040 con 0x7ff960110170 2024-03-22T07:14:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.296+0000 7ff95ffff700 1 --2- 172.21.15.63:0/3025432793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff960110170 0x7ff960110f70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff94c013870 tx=0x7ff94c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.296+0000 7ff95d7fa700 1 -- 172.21.15.63:0/3025432793 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff94c021070 con 0x7ff960110170 2024-03-22T07:14:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.296+0000 7ff966b9e700 1 -- 172.21.15.63:0/3025432793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff960111570 con 0x7ff960110170 2024-03-22T07:14:53.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.296+0000 7ff95d7fa700 1 -- 172.21.15.63:0/3025432793 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff94c009430 con 0x7ff960110170 2024-03-22T07:14:53.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.296+0000 7ff95d7fa700 1 -- 172.21.15.63:0/3025432793 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff94c00e070 con 0x7ff960110170 2024-03-22T07:14:53.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.297+0000 7ff966b9e700 1 -- 172.21.15.63:0/3025432793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff960111a80 con 0x7ff960110170 2024-03-22T07:14:53.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.298+0000 7ff95d7fa700 1 -- 172.21.15.63:0/3025432793 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7ff94c01c750 con 0x7ff960110170 2024-03-22T07:14:53.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.298+0000 7ff966b9e700 1 -- 172.21.15.63:0/3025432793 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff960103eb0 con 0x7ff960110170 2024-03-22T07:14:53.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.298+0000 7ff95d7fa700 1 --2- 172.21.15.63:0/3025432793 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ff9500692a0 0x7ff95006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:53.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.299+0000 7ff95d7fa700 1 -- 172.21.15.63:0/3025432793 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff94c088160 con 0x7ff960110170 2024-03-22T07:14:53.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.299+0000 7ff96493a700 1 --2- 172.21.15.63:0/3025432793 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ff9500692a0 0x7ff95006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:53.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.299+0000 7ff96493a700 1 --2- 172.21.15.63:0/3025432793 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ff9500692a0 0x7ff95006b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff954009b80 tx=0x7ff95400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:53.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.306+0000 7ff95d7fa700 1 -- 172.21.15.63:0/3025432793 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7ff94c057eb0 con 0x7ff960110170 2024-03-22T07:14:53.501 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:53 smithi063 ceph-mon[29200]: pgmap v219: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 426 B/s wr, 4 op/s 2024-03-22T07:14:53.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.653+0000 7ff966b9e700 1 -- 172.21.15.63:0/3025432793 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff960103850 con 0x7ff9500692a0 2024-03-22T07:14:53.673 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:14:53.673 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (5m) 55s ago 6m 24.4M - 0.25.0 c8568f914cd2 2f8544438b7c 2024-03-22T07:14:53.673 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (6m) 55s ago 6m 6660k - 17.2.7-726-g9c4e0c34 837124da804f d6c6bbeb6904 2024-03-22T07:14:53.673 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (5m) 60s ago 5m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 30992f8aa88c 2024-03-22T07:14:53.673 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (5m) 55s ago 6m 85.4M - 9.4.7 954c08fa6188 f0b1d2a54439 2024-03-22T07:14:53.673 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (80s) 55s ago 79s 11.2M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:14:53.673 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (73s) 55s ago 72s 13.8M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:14:53.673 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (70s) 60s ago 70s 14.5M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:14:53.673 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (77s) 60s ago 76s 16.1M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:14:53.673 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:9283 running (8m) 55s ago 8m 480M - 17.2.7-726-g9c4e0c34 837124da804f 422b5e836153 2024-03-22T07:14:53.673 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283 running (5m) 60s ago 5m 435M - 17.2.7-726-g9c4e0c34 837124da804f b48492cc30bb 2024-03-22T07:14:53.673 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (8m) 55s ago 8m 49.2M 2048M 17.2.7-726-g9c4e0c34 837124da804f f92a5bb6caee 2024-03-22T07:14:53.674 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (5m) 60s ago 5m 32.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f 95f6103f7603 2024-03-22T07:14:53.674 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (6m) 55s ago 6m 16.3M - 1.5.0 0da6a335fe13 47501603d3e8 2024-03-22T07:14:53.674 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (5m) 60s ago 5m 16.5M - 1.5.0 0da6a335fe13 49f4ca105fb9 2024-03-22T07:14:53.674 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (5m) 55s ago 5m 46.2M 4288M 17.2.7-726-g9c4e0c34 837124da804f 580f22188a10 2024-03-22T07:14:53.674 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (4m) 55s ago 4m 49.1M 4288M 17.2.7-726-g9c4e0c34 837124da804f fe1ace35c88a 2024-03-22T07:14:53.674 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (4m) 55s ago 4m 45.9M 4288M 17.2.7-726-g9c4e0c34 837124da804f 4cd995265dcf 2024-03-22T07:14:53.674 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (3m) 60s ago 3m 47.0M 5312M 17.2.7-726-g9c4e0c34 837124da804f 45c9a759b76d 2024-03-22T07:14:53.674 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (3m) 60s ago 3m 41.9M 5312M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:14:53.674 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (2m) 60s ago 2m 42.5M 5312M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:14:53.674 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (5m) 55s ago 6m 53.7M - 2.43.0 a07b618ecd1d 827638a5870a 2024-03-22T07:14:53.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.671+0000 7ff95d7fa700 1 -- 172.21.15.63:0/3025432793 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7ff960103850 con 0x7ff9500692a0 2024-03-22T07:14:53.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.674+0000 7ff966b9e700 1 -- 172.21.15.63:0/3025432793 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ff9500692a0 msgr2=0x7ff95006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:53.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.674+0000 7ff966b9e700 1 --2- 172.21.15.63:0/3025432793 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ff9500692a0 0x7ff95006b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff954009b80 tx=0x7ff95400b040 comp rx=0 tx=0).stop 2024-03-22T07:14:53.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.674+0000 7ff966b9e700 1 -- 172.21.15.63:0/3025432793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff960110170 msgr2=0x7ff960110f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:53.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.674+0000 7ff966b9e700 1 --2- 172.21.15.63:0/3025432793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff960110170 0x7ff960110f70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff94c013870 tx=0x7ff94c009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:53.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.675+0000 7ff966b9e700 1 -- 172.21.15.63:0/3025432793 shutdown_connections 2024-03-22T07:14:53.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.675+0000 7ff966b9e700 1 --2- 172.21.15.63:0/3025432793 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7ff9500692a0 0x7ff95006b750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:53.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.675+0000 7ff966b9e700 1 --2- 172.21.15.63:0/3025432793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9600fd290 0x7ff96010fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:53.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.675+0000 7ff966b9e700 1 --2- 172.21.15.63:0/3025432793 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff960110170 0x7ff960110f70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:53.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.675+0000 7ff966b9e700 1 -- 172.21.15.63:0/3025432793 >> 172.21.15.63:0/3025432793 conn(0x7ff9600f6ba0 msgr2=0x7ff9600f86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:53.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.675+0000 7ff966b9e700 1 -- 172.21.15.63:0/3025432793 shutdown_connections 2024-03-22T07:14:53.676 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.675+0000 7ff966b9e700 1 -- 172.21.15.63:0/3025432793 wait complete. 2024-03-22T07:14:53.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:53 smithi069 ceph-mon[36441]: pgmap v219: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 426 B/s wr, 4 op/s 2024-03-22T07:14:53.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.836+0000 7f69ba843700 1 -- 172.21.15.63:0/2665671860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69b40fc6f0 msgr2=0x7f69b40fcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:53.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.836+0000 7f69ba843700 1 --2- 172.21.15.63:0/2665671860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69b40fc6f0 0x7f69b40fcb60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f69a8004940 tx=0x7f69a8009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:53.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.836+0000 7f69ba843700 1 -- 172.21.15.63:0/2665671860 shutdown_connections 2024-03-22T07:14:53.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.836+0000 7f69ba843700 1 --2- 172.21.15.63:0/2665671860 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69b40fc6f0 0x7f69b40fcb60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:53.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.836+0000 7f69ba843700 1 --2- 172.21.15.63:0/2665671860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69b40fb440 0x7f69b40fb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:53.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.836+0000 7f69ba843700 1 -- 172.21.15.63:0/2665671860 >> 172.21.15.63:0/2665671860 conn(0x7f69b40f6ba0 msgr2=0x7f69b40f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:53.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.836+0000 7f69ba843700 1 -- 172.21.15.63:0/2665671860 shutdown_connections 2024-03-22T07:14:53.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.837+0000 7f69ba843700 1 -- 172.21.15.63:0/2665671860 wait complete. 2024-03-22T07:14:53.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.837+0000 7f69ba843700 1 Processor -- start 2024-03-22T07:14:53.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.837+0000 7f69ba843700 1 -- start start 2024-03-22T07:14:53.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.838+0000 7f69ba843700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69b40fb440 0x7f69b410d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:53.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.838+0000 7f69ba843700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69b40fc6f0 0x7f69b410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:53.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.838+0000 7f69ba843700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b410e4e0 con 0x7f69b40fb440 2024-03-22T07:14:53.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.838+0000 7f69ba843700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b410ed30 con 0x7f69b40fc6f0 2024-03-22T07:14:53.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.838+0000 7f69b37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69b40fc6f0 0x7f69b410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:53.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.838+0000 7f69b37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69b40fc6f0 0x7f69b410dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.63:37066/0 (socket says 172.21.15.63:37066) 2024-03-22T07:14:53.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.838+0000 7f69b37fe700 1 -- 172.21.15.63:0/2807735397 learned_addr learned my addr 172.21.15.63:0/2807735397 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:14:53.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.838+0000 7f69b37fe700 1 -- 172.21.15.63:0/2807735397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69b40fb440 msgr2=0x7f69b410d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:53.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.839+0000 7f69b37fe700 1 --2- 172.21.15.63:0/2807735397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69b40fb440 0x7f69b410d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:53.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.839+0000 7f69b37fe700 1 -- 172.21.15.63:0/2807735397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a8005040 con 0x7f69b40fc6f0 2024-03-22T07:14:53.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.839+0000 7f69b37fe700 1 --2- 172.21.15.63:0/2807735397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69b40fc6f0 0x7f69b410dee0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f69a800b0d0 tx=0x7f69a8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:53.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.840+0000 7f69b0ff9700 1 -- 172.21.15.63:0/2807735397 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69a800e030 con 0x7f69b40fc6f0 2024-03-22T07:14:53.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.840+0000 7f69b0ff9700 1 -- 172.21.15.63:0/2807735397 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69a801c400 con 0x7f69b40fc6f0 2024-03-22T07:14:53.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.840+0000 7f69b0ff9700 1 -- 172.21.15.63:0/2807735397 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69a800b650 con 0x7f69b40fc6f0 2024-03-22T07:14:53.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.841+0000 7f69ba843700 1 -- 172.21.15.63:0/2807735397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b410efb0 con 0x7f69b40fc6f0 2024-03-22T07:14:53.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.841+0000 7f69ba843700 1 -- 172.21.15.63:0/2807735397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b410f430 con 0x7f69b40fc6f0 2024-03-22T07:14:53.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.842+0000 7f69ba843700 1 -- 172.21.15.63:0/2807735397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b405f5f0 con 0x7f69b40fc6f0 2024-03-22T07:14:53.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.843+0000 7f69b0ff9700 1 -- 172.21.15.63:0/2807735397 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f69a801c570 con 0x7f69b40fc6f0 2024-03-22T07:14:53.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.843+0000 7f69b0ff9700 1 --2- 172.21.15.63:0/2807735397 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f699c0692a0 0x7f699c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:53.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.843+0000 7f69b0ff9700 1 -- 172.21.15.63:0/2807735397 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f69a80889b0 con 0x7f69b40fc6f0 2024-03-22T07:14:53.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.844+0000 7f69b3fff700 1 --2- 172.21.15.63:0/2807735397 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f699c0692a0 0x7f699c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:53.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.844+0000 7f69b3fff700 1 --2- 172.21.15.63:0/2807735397 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f699c0692a0 0x7f699c06b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f69a4009fd0 tx=0x7f69a400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:53.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:53.852+0000 7f69b0ff9700 1 -- 172.21.15.63:0/2807735397 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f69a8058740 con 0x7f69b40fc6f0 2024-03-22T07:14:54.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.262+0000 7f69ba843700 1 -- 172.21.15.63:0/2807735397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f69b410f800 con 0x7f69b40fc6f0 2024-03-22T07:14:54.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.264+0000 7f69b0ff9700 1 -- 172.21.15.63:0/2807735397 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f69a8004490 con 0x7f69b40fc6f0 2024-03-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:14:54.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:14:54.265 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:14:54.265 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 14 2024-03-22T07:14:54.265 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:14:54.265 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:14:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.267+0000 7f69ba843700 1 -- 172.21.15.63:0/2807735397 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f699c0692a0 msgr2=0x7f699c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.267+0000 7f69ba843700 1 --2- 172.21.15.63:0/2807735397 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f699c0692a0 0x7f699c06b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f69a4009fd0 tx=0x7f69a400b040 comp rx=0 tx=0).stop 2024-03-22T07:14:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.267+0000 7f69ba843700 1 -- 172.21.15.63:0/2807735397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69b40fc6f0 msgr2=0x7f69b410dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.267+0000 7f69ba843700 1 --2- 172.21.15.63:0/2807735397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69b40fc6f0 0x7f69b410dee0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f69a800b0d0 tx=0x7f69a8009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.267+0000 7f69ba843700 1 -- 172.21.15.63:0/2807735397 shutdown_connections 2024-03-22T07:14:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.267+0000 7f69ba843700 1 --2- 172.21.15.63:0/2807735397 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f699c0692a0 0x7f699c06b750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:54.268 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.267+0000 7f69ba843700 1 --2- 172.21.15.63:0/2807735397 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f69b40fb440 0x7f69b410d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.267+0000 7f69ba843700 1 --2- 172.21.15.63:0/2807735397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f69b40fc6f0 0x7f69b410dee0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.267+0000 7f69ba843700 1 -- 172.21.15.63:0/2807735397 >> 172.21.15.63:0/2807735397 conn(0x7f69b40f6ba0 msgr2=0x7f69b40f8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.267+0000 7f69ba843700 1 -- 172.21.15.63:0/2807735397 shutdown_connections 2024-03-22T07:14:54.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.268+0000 7f69ba843700 1 -- 172.21.15.63:0/2807735397 wait complete. 2024-03-22T07:14:54.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.401+0000 7f44d2587700 1 -- 172.21.15.63:0/1368768551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44cc0ffff0 msgr2=0x7f44cc1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:54.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.401+0000 7f44d2587700 1 --2- 172.21.15.63:0/1368768551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44cc0ffff0 0x7f44cc1023d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f44c0004940 tx=0x7f44c0009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:54.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.401+0000 7f44d2587700 1 -- 172.21.15.63:0/1368768551 shutdown_connections 2024-03-22T07:14:54.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.401+0000 7f44d2587700 1 --2- 172.21.15.63:0/1368768551 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44cc0ffff0 0x7f44cc1023d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:54.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.401+0000 7f44d2587700 1 --2- 172.21.15.63:0/1368768551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44cc066b10 0x7f44cc0ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:54.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.401+0000 7f44d2587700 1 -- 172.21.15.63:0/1368768551 >> 172.21.15.63:0/1368768551 conn(0x7f44cc0f74e0 msgr2=0x7f44cc0f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:54.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.401+0000 7f44d2587700 1 -- 172.21.15.63:0/1368768551 shutdown_connections 2024-03-22T07:14:54.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.401+0000 7f44d2587700 1 -- 172.21.15.63:0/1368768551 wait complete. 2024-03-22T07:14:54.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.402+0000 7f44d2587700 1 Processor -- start 2024-03-22T07:14:54.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.402+0000 7f44d2587700 1 -- start start 2024-03-22T07:14:54.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.402+0000 7f44d2587700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44cc066b10 0x7f44cc10da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:54.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.402+0000 7f44d2587700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44cc0ffff0 0x7f44cc10df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:54.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.402+0000 7f44d2587700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc10e570 con 0x7f44cc066b10 2024-03-22T07:14:54.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.402+0000 7f44d2587700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc10ed70 con 0x7f44cc0ffff0 2024-03-22T07:14:54.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.402+0000 7f44cbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44cc066b10 0x7f44cc10da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:54.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.402+0000 7f44cbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44cc066b10 0x7f44cc10da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:40038/0 (socket says 172.21.15.63:40038) 2024-03-22T07:14:54.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.402+0000 7f44cbfff700 1 -- 172.21.15.63:0/3649748354 learned_addr learned my addr 172.21.15.63:0/3649748354 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:14:54.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.402+0000 7f44cb7fe700 1 --2- 172.21.15.63:0/3649748354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44cc0ffff0 0x7f44cc10df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:54.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.403+0000 7f44cb7fe700 1 -- 172.21.15.63:0/3649748354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44cc066b10 msgr2=0x7f44cc10da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:54.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.403+0000 7f44cb7fe700 1 --2- 172.21.15.63:0/3649748354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44cc066b10 0x7f44cc10da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:54.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.403+0000 7f44cb7fe700 1 -- 172.21.15.63:0/3649748354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c0005040 con 0x7f44cc0ffff0 2024-03-22T07:14:54.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.403+0000 7f44cb7fe700 1 --2- 172.21.15.63:0/3649748354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44cc0ffff0 0x7f44cc10df70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f44c0000c00 tx=0x7f44c0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:54.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.403+0000 7f44c8ff9700 1 -- 172.21.15.63:0/3649748354 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44c000e040 con 0x7f44cc0ffff0 2024-03-22T07:14:54.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.403+0000 7f44c8ff9700 1 -- 172.21.15.63:0/3649748354 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44c000bc20 con 0x7f44cc0ffff0 2024-03-22T07:14:54.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.403+0000 7f44c8ff9700 1 -- 172.21.15.63:0/3649748354 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44c0013680 con 0x7f44cc0ffff0 2024-03-22T07:14:54.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.403+0000 7f44d2587700 1 -- 172.21.15.63:0/3649748354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44cc10eff0 con 0x7f44cc0ffff0 2024-03-22T07:14:54.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.403+0000 7f44d2587700 1 -- 172.21.15.63:0/3649748354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44cc10f4e0 con 0x7f44cc0ffff0 2024-03-22T07:14:54.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.404+0000 7f44d2587700 1 -- 172.21.15.63:0/3649748354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44cc05f5f0 con 0x7f44cc0ffff0 2024-03-22T07:14:54.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.406+0000 7f44c8ff9700 1 -- 172.21.15.63:0/3649748354 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f44c000bd90 con 0x7f44cc0ffff0 2024-03-22T07:14:54.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.407+0000 7f44c8ff9700 1 --2- 172.21.15.63:0/3649748354 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f44b4069370 0x7f44b406b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:14:54.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.407+0000 7f44c8ff9700 1 -- 172.21.15.63:0/3649748354 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f44c008ca20 con 0x7f44cc0ffff0 2024-03-22T07:14:54.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.407+0000 7f44cbfff700 1 --2- 172.21.15.63:0/3649748354 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f44b4069370 0x7f44b406b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:14:54.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.407+0000 7f44cbfff700 1 --2- 172.21.15.63:0/3649748354 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f44b4069370 0x7f44b406b820 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f44bc010210 tx=0x7f44bc00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:14:54.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.411+0000 7f44c8ff9700 1 -- 172.21.15.63:0/3649748354 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f44c005c770 con 0x7f44cc0ffff0 2024-03-22T07:14:54.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:54 smithi063 ceph-mon[29200]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:54.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:54 smithi063 ceph-mon[29200]: from='client.24357 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:54.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:54 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:14:54.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:54 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:14:54.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:54 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:14:54.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:54 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2807735397' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:14:54.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.630+0000 7f44d2587700 1 -- 172.21.15.63:0/3649748354 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f44cc10f8d0 con 0x7f44b4069370 2024-03-22T07:14:54.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.631+0000 7f44c8ff9700 1 -- 172.21.15.63:0/3649748354 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f44cc10f8d0 con 0x7f44b4069370 2024-03-22T07:14:54.631 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:14:54.631 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:14:54.631 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:14:54.631 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-22T07:14:54.631 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-03-22T07:14:54.631 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "0/2 daemons upgraded", 2024-03-22T07:14:54.631 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 image on host smithi069", 2024-03-22T07:14:54.631 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:14:54.631 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:14:54.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.633+0000 7f44d2587700 1 -- 172.21.15.63:0/3649748354 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f44b4069370 msgr2=0x7f44b406b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:54.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.633+0000 7f44d2587700 1 --2- 172.21.15.63:0/3649748354 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f44b4069370 0x7f44b406b820 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f44bc010210 tx=0x7f44bc00b040 comp rx=0 tx=0).stop 2024-03-22T07:14:54.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.633+0000 7f44d2587700 1 -- 172.21.15.63:0/3649748354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44cc0ffff0 msgr2=0x7f44cc10df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:14:54.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.633+0000 7f44d2587700 1 --2- 172.21.15.63:0/3649748354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44cc0ffff0 0x7f44cc10df70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f44c0000c00 tx=0x7f44c0009e30 comp rx=0 tx=0).stop 2024-03-22T07:14:54.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.633+0000 7f44d2587700 1 -- 172.21.15.63:0/3649748354 shutdown_connections 2024-03-22T07:14:54.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.633+0000 7f44d2587700 1 --2- 172.21.15.63:0/3649748354 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f44b4069370 0x7f44b406b820 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:54.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.633+0000 7f44d2587700 1 --2- 172.21.15.63:0/3649748354 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f44cc066b10 0x7f44cc10da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:54.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.633+0000 7f44d2587700 1 --2- 172.21.15.63:0/3649748354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44cc0ffff0 0x7f44cc10df70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:14:54.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.633+0000 7f44d2587700 1 -- 172.21.15.63:0/3649748354 >> 172.21.15.63:0/3649748354 conn(0x7f44cc0f74e0 msgr2=0x7f44cc0f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:14:54.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.634+0000 7f44d2587700 1 -- 172.21.15.63:0/3649748354 shutdown_connections 2024-03-22T07:14:54.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:14:54.634+0000 7f44d2587700 1 -- 172.21.15.63:0/3649748354 wait complete. 2024-03-22T07:14:54.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:54 smithi069 ceph-mon[36441]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:54.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:54 smithi069 ceph-mon[36441]: from='client.24357 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:54.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:54 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:14:54.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:54 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:14:54.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:54 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:14:54.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:54 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/2807735397' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:14:55.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:55 smithi063 ceph-mon[29200]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:55.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:55 smithi063 ceph-mon[29200]: Upgrade: Target is version 19.0.0-1667-gdb0330b1 (unknown) 2024-03-22T07:14:55.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:55 smithi063 ceph-mon[29200]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10'] 2024-03-22T07:14:55.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:55 smithi063 ceph-mon[29200]: Upgrade: Need to upgrade myself (mgr.smithi063.mnxayh) 2024-03-22T07:14:55.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:55 smithi063 ceph-mon[29200]: pgmap v220: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-03-22T07:14:55.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:55 smithi063 ceph-mon[29200]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 on smithi069 2024-03-22T07:14:55.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:55 smithi069 ceph-mon[36441]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:55.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:55 smithi069 ceph-mon[36441]: Upgrade: Target is version 19.0.0-1667-gdb0330b1 (unknown) 2024-03-22T07:14:55.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:55 smithi069 ceph-mon[36441]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10'] 2024-03-22T07:14:55.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:55 smithi069 ceph-mon[36441]: Upgrade: Need to upgrade myself (mgr.smithi063.mnxayh) 2024-03-22T07:14:55.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:55 smithi069 ceph-mon[36441]: pgmap v220: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-03-22T07:14:55.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:55 smithi069 ceph-mon[36441]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 on smithi069 2024-03-22T07:14:56.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:56 smithi063 ceph-mon[29200]: from='client.24369 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:56.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:56 smithi069 ceph-mon[36441]: from='client.24369 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:14:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:57 smithi063 ceph-mon[29200]: pgmap v221: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-22T07:14:57.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:57 smithi069 ceph-mon[36441]: pgmap v221: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-03-22T07:14:59.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:59 smithi063 ceph-mon[29200]: pgmap v222: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 170 B/s wr, 4 op/s 2024-03-22T07:14:59.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:14:59 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:14:59.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:59 smithi069 ceph-mon[36441]: pgmap v222: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 170 B/s wr, 4 op/s 2024-03-22T07:14:59.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:14:59 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:15:01.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:01 smithi063 ceph-mon[29200]: pgmap v223: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 4 op/s 2024-03-22T07:15:01.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:01 smithi069 ceph-mon[36441]: pgmap v223: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 4 op/s 2024-03-22T07:15:03.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:03 smithi063 ceph-mon[29200]: pgmap v224: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 4 op/s 2024-03-22T07:15:03.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:03 smithi069 ceph-mon[36441]: pgmap v224: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 85 B/s wr, 4 op/s 2024-03-22T07:15:05.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:05 smithi063 ceph-mon[29200]: pgmap v225: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:15:05.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:05 smithi069 ceph-mon[36441]: pgmap v225: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:15:07.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:07 smithi063 ceph-mon[29200]: pgmap v226: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:15:07.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:07 smithi069 ceph-mon[36441]: pgmap v226: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:15:09.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:09 smithi063 ceph-mon[29200]: pgmap v227: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:15:09.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:09 smithi069 ceph-mon[36441]: pgmap v227: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:15:11.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:11 smithi063 ceph-mon[29200]: pgmap v228: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-22T07:15:11.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:11 smithi069 ceph-mon[36441]: pgmap v228: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-22T07:15:13.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:13 smithi063 ceph-mon[29200]: pgmap v229: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-22T07:15:13.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:13 smithi069 ceph-mon[36441]: pgmap v229: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-22T07:15:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:14 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:15:14.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:14 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:15:15.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:15 smithi063 ceph-mon[29200]: pgmap v230: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:15:15.756 INFO:tasks.workunit.client.0.smithi063.stderr:Note: switching to '768f6cd6e95d0c7a626042719c230df59bdba9d3'. 2024-03-22T07:15:15.756 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-03-22T07:15:15.756 INFO:tasks.workunit.client.0.smithi063.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-22T07:15:15.756 INFO:tasks.workunit.client.0.smithi063.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-22T07:15:15.756 INFO:tasks.workunit.client.0.smithi063.stderr:state without impacting any branches by switching back to a branch. 2024-03-22T07:15:15.756 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-03-22T07:15:15.756 INFO:tasks.workunit.client.0.smithi063.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-22T07:15:15.756 INFO:tasks.workunit.client.0.smithi063.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-22T07:15:15.756 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-03-22T07:15:15.756 INFO:tasks.workunit.client.0.smithi063.stderr: git switch -c 2024-03-22T07:15:15.757 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-03-22T07:15:15.757 INFO:tasks.workunit.client.0.smithi063.stderr:Or undo this operation with: 2024-03-22T07:15:15.757 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-03-22T07:15:15.757 INFO:tasks.workunit.client.0.smithi063.stderr: git switch - 2024-03-22T07:15:15.757 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-03-22T07:15:15.757 INFO:tasks.workunit.client.0.smithi063.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-22T07:15:15.757 INFO:tasks.workunit.client.0.smithi063.stderr: 2024-03-22T07:15:15.757 INFO:tasks.workunit.client.0.smithi063.stderr:HEAD is now at 768f6cd6e95 Merge pull request #56339 from zdover23/wip-doc-2024-03-20-backport-56333-to-squid 2024-03-22T07:15:15.759 DEBUG:teuthology.orchestra.run.smithi063:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-22T07:15:15.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:15 smithi069 ceph-mon[36441]: pgmap v230: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:15:15.791 INFO:tasks.workunit.client.0.smithi063.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-22T07:15:15.793 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-22T07:15:15.793 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-22T07:15:16.655 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-22T07:15:16.732 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-22T07:15:16.775 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-22T07:15:16.776 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-22T07:15:16.776 INFO:tasks.workunit.client.0.smithi063.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-22T07:15:16.816 INFO:tasks.workunit.client.0.smithi063.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-22T07:15:16.819 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T07:15:16.820 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-22T07:15:16.875 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-03-22T07:15:16.876 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-22T07:15:16.877 DEBUG:teuthology.orchestra.run.smithi063:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=768f6cd6e95d0c7a626042719c230df59bdba9d3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-03-22T07:15:16.936 INFO:tasks.workunit.client.0.smithi063.stderr:+ mkdir -p fsstress 2024-03-22T07:15:16.937 INFO:tasks.workunit.client.0.smithi063.stderr:+ pushd fsstress 2024-03-22T07:15:16.938 INFO:tasks.workunit.client.0.smithi063.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-22T07:15:16.938 INFO:tasks.workunit.client.0.smithi063.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-22T07:15:17.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:17 smithi063 ceph-mon[29200]: pgmap v231: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:15:17.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:17 smithi069 ceph-mon[36441]: pgmap v231: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:15:18.028 INFO:tasks.workunit.client.0.smithi063.stderr:+ tar xzf ltp-full.tgz 2024-03-22T07:15:18.094 INFO:tasks.workunit.client.1.smithi069.stderr:Updating files: 92% (11522/12393) Updating files: 93% (11526/12393) Updating files: 94% (11650/12393) Updating files: 95% (11774/12393) Updating files: 96% (11898/12393) Updating files: 97% (12022/12393) Updating files: 98% (12146/12393) Updating files: 99% (12270/12393) Updating files: 100% (12393/12393) Updating files: 100% (12393/12393), done. 2024-03-22T07:15:18.390 INFO:tasks.workunit.client.1.smithi069.stderr:Note: switching to '768f6cd6e95d0c7a626042719c230df59bdba9d3'. 2024-03-22T07:15:18.390 INFO:tasks.workunit.client.1.smithi069.stderr: 2024-03-22T07:15:18.390 INFO:tasks.workunit.client.1.smithi069.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-22T07:15:18.390 INFO:tasks.workunit.client.1.smithi069.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-22T07:15:18.390 INFO:tasks.workunit.client.1.smithi069.stderr:state without impacting any branches by switching back to a branch. 2024-03-22T07:15:18.390 INFO:tasks.workunit.client.1.smithi069.stderr: 2024-03-22T07:15:18.390 INFO:tasks.workunit.client.1.smithi069.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-22T07:15:18.391 INFO:tasks.workunit.client.1.smithi069.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-22T07:15:18.391 INFO:tasks.workunit.client.1.smithi069.stderr: 2024-03-22T07:15:18.391 INFO:tasks.workunit.client.1.smithi069.stderr: git switch -c 2024-03-22T07:15:18.391 INFO:tasks.workunit.client.1.smithi069.stderr: 2024-03-22T07:15:18.391 INFO:tasks.workunit.client.1.smithi069.stderr:Or undo this operation with: 2024-03-22T07:15:18.391 INFO:tasks.workunit.client.1.smithi069.stderr: 2024-03-22T07:15:18.391 INFO:tasks.workunit.client.1.smithi069.stderr: git switch - 2024-03-22T07:15:18.391 INFO:tasks.workunit.client.1.smithi069.stderr: 2024-03-22T07:15:18.391 INFO:tasks.workunit.client.1.smithi069.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-22T07:15:18.391 INFO:tasks.workunit.client.1.smithi069.stderr: 2024-03-22T07:15:18.391 INFO:tasks.workunit.client.1.smithi069.stderr:HEAD is now at 768f6cd6e95 Merge pull request #56339 from zdover23/wip-doc-2024-03-20-backport-56333-to-squid 2024-03-22T07:15:18.393 DEBUG:teuthology.orchestra.run.smithi069:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-03-22T07:15:18.459 INFO:tasks.workunit.client.1.smithi069.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-22T07:15:18.460 INFO:tasks.workunit.client.1.smithi069.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-22T07:15:18.461 INFO:tasks.workunit.client.1.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-22T07:15:19.216 INFO:tasks.workunit.client.1.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-22T07:15:19.281 INFO:tasks.workunit.client.1.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-22T07:15:19.325 INFO:tasks.workunit.client.1.smithi069.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-22T07:15:19.326 INFO:tasks.workunit.client.1.smithi069.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-22T07:15:19.326 INFO:tasks.workunit.client.1.smithi069.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-22T07:15:19.369 INFO:tasks.workunit.client.1.smithi069.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-22T07:15:19.372 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-22T07:15:19.372 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-03-22T07:15:19.428 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-03-22T07:15:19.429 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-22T07:15:19.429 DEBUG:teuthology.orchestra.run.smithi069:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=768f6cd6e95d0c7a626042719c230df59bdba9d3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-03-22T07:15:19.489 INFO:tasks.workunit.client.1.smithi069.stderr:+ mkdir -p fsstress 2024-03-22T07:15:19.491 INFO:tasks.workunit.client.1.smithi069.stderr:+ pushd fsstress 2024-03-22T07:15:19.492 INFO:tasks.workunit.client.1.smithi069.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-22T07:15:19.492 INFO:tasks.workunit.client.1.smithi069.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-22T07:15:19.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:19 smithi063 ceph-mon[29200]: pgmap v232: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:15:19.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:19 smithi069 ceph-mon[36441]: pgmap v232: 65 pgs: 65 active+clean; 589 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:15:20.694 INFO:tasks.workunit.client.1.smithi069.stderr:+ tar xzf ltp-full.tgz 2024-03-22T07:15:21.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:21 smithi069 ceph-mon[36441]: pgmap v233: 65 pgs: 65 active+clean; 4.6 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 279 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-03-22T07:15:21.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:21 smithi063 ceph-mon[29200]: pgmap v233: 65 pgs: 65 active+clean; 4.6 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 279 KiB/s rd, 341 KiB/s wr, 5 op/s 2024-03-22T07:15:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:23 smithi063 ceph-mon[29200]: Upgrade: Updating mgr.smithi069.gfziwh 2024-03-22T07:15:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:23 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:15:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:23 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi069.gfziwh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:15:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:23 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:15:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:23 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:15:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:23 smithi063 ceph-mon[29200]: Deploying daemon mgr.smithi069.gfziwh on smithi069 2024-03-22T07:15:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:23 smithi063 ceph-mon[29200]: pgmap v234: 65 pgs: 65 active+clean; 8.6 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 620 KiB/s rd, 701 KiB/s wr, 5 op/s 2024-03-22T07:15:23.499 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:23 smithi069 ceph-mon[36441]: Upgrade: Updating mgr.smithi069.gfziwh 2024-03-22T07:15:23.500 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:23 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:15:23.500 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:23 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi069.gfziwh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:15:23.500 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:23 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:15:23.500 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:23 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:15:23.500 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:23 smithi069 ceph-mon[36441]: Deploying daemon mgr.smithi069.gfziwh on smithi069 2024-03-22T07:15:23.500 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:23 smithi069 ceph-mon[36441]: pgmap v234: 65 pgs: 65 active+clean; 8.6 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 620 KiB/s rd, 701 KiB/s wr, 5 op/s 2024-03-22T07:15:24.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.771+0000 7fa074228700 1 -- 172.21.15.63:0/1668798173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa06c0fed50 msgr2=0x7fa06c0fae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:24.773 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.771+0000 7fa074228700 1 --2- 172.21.15.63:0/1668798173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa06c0fed50 0x7fa06c0fae90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa05c004940 tx=0x7fa05c009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:24.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.771+0000 7fa074228700 1 -- 172.21.15.63:0/1668798173 shutdown_connections 2024-03-22T07:15:24.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.771+0000 7fa074228700 1 --2- 172.21.15.63:0/1668798173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa06c0fed50 0x7fa06c0fae90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:24.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.771+0000 7fa074228700 1 --2- 172.21.15.63:0/1668798173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa06c0fe400 0x7fa06c0fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:24.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.771+0000 7fa074228700 1 -- 172.21.15.63:0/1668798173 >> 172.21.15.63:0/1668798173 conn(0x7fa06c0f6ba0 msgr2=0x7fa06c0f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:24.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.772+0000 7fa074228700 1 -- 172.21.15.63:0/1668798173 shutdown_connections 2024-03-22T07:15:24.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.772+0000 7fa074228700 1 -- 172.21.15.63:0/1668798173 wait complete. 2024-03-22T07:15:24.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.772+0000 7fa074228700 1 Processor -- start 2024-03-22T07:15:24.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.772+0000 7fa074228700 1 -- start start 2024-03-22T07:15:24.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.772+0000 7fa074228700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa06c0fe400 0x7fa06c0fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:24.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.772+0000 7fa074228700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa06c0fed50 0x7fa06c0fc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:24.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.772+0000 7fa074228700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c0fc650 con 0x7fa06c0fe400 2024-03-22T07:15:24.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.772+0000 7fa074228700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c0fc790 con 0x7fa06c0fed50 2024-03-22T07:15:24.774 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.772+0000 7fa071fc4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa06c0fe400 0x7fa06c0fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:24.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.773+0000 7fa071fc4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa06c0fe400 0x7fa06c0fdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37712/0 (socket says 172.21.15.63:37712) 2024-03-22T07:15:24.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.773+0000 7fa071fc4700 1 -- 172.21.15.63:0/1309263420 learned_addr learned my addr 172.21.15.63:0/1309263420 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:15:24.775 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.773+0000 7fa0717c3700 1 --2- 172.21.15.63:0/1309263420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa06c0fed50 0x7fa06c0fc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:24.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.773+0000 7fa071fc4700 1 -- 172.21.15.63:0/1309263420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa06c0fed50 msgr2=0x7fa06c0fc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:24.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.773+0000 7fa071fc4700 1 --2- 172.21.15.63:0/1309263420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa06c0fed50 0x7fa06c0fc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:24.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.773+0000 7fa071fc4700 1 -- 172.21.15.63:0/1309263420 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c005040 con 0x7fa06c0fe400 2024-03-22T07:15:24.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.773+0000 7fa071fc4700 1 --2- 172.21.15.63:0/1309263420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa06c0fe400 0x7fa06c0fdb80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa06800eb40 tx=0x7fa06800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:24.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.773+0000 7fa0627fc700 1 -- 172.21.15.63:0/1309263420 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa068007cc0 con 0x7fa06c0fe400 2024-03-22T07:15:24.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.773+0000 7fa074228700 1 -- 172.21.15.63:0/1309263420 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06c0fc9e0 con 0x7fa06c0fe400 2024-03-22T07:15:24.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.773+0000 7fa0627fc700 1 -- 172.21.15.63:0/1309263420 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa06800f040 con 0x7fa06c0fe400 2024-03-22T07:15:24.776 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.773+0000 7fa0627fc700 1 -- 172.21.15.63:0/1309263420 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa06800ca10 con 0x7fa06c0fe400 2024-03-22T07:15:24.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.773+0000 7fa074228700 1 -- 172.21.15.63:0/1309263420 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06c0fcf20 con 0x7fa06c0fe400 2024-03-22T07:15:24.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.775+0000 7fa074228700 1 -- 172.21.15.63:0/1309263420 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa06c05f5f0 con 0x7fa06c0fe400 2024-03-22T07:15:24.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.775+0000 7fa0627fc700 1 -- 172.21.15.63:0/1309263420 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fa06800cb70 con 0x7fa06c0fe400 2024-03-22T07:15:24.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.775+0000 7fa0627fc700 1 --2- 172.21.15.63:0/1309263420 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa0580692a0 0x7fa05806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:24.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.776+0000 7fa0627fc700 1 -- 172.21.15.63:0/1309263420 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa068086cf0 con 0x7fa06c0fe400 2024-03-22T07:15:24.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.776+0000 7fa0717c3700 1 --2- 172.21.15.63:0/1309263420 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa0580692a0 0x7fa05806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:24.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.776+0000 7fa0717c3700 1 --2- 172.21.15.63:0/1309263420 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa0580692a0 0x7fa05806b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009b30 tx=0x7fa05c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:24.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:24.780+0000 7fa0627fc700 1 -- 172.21.15.63:0/1309263420 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa06804e190 con 0x7fa06c0fe400 2024-03-22T07:15:25.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.191+0000 7fa074228700 1 -- 172.21.15.63:0/1309263420 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa06c102450 con 0x7fa0580692a0 2024-03-22T07:15:25.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.192+0000 7fa0627fc700 1 -- 172.21.15.63:0/1309263420 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fa06c102450 con 0x7fa0580692a0 2024-03-22T07:15:25.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.194+0000 7fa074228700 1 -- 172.21.15.63:0/1309263420 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa0580692a0 msgr2=0x7fa05806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:25.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.194+0000 7fa074228700 1 --2- 172.21.15.63:0/1309263420 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa0580692a0 0x7fa05806b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009b30 tx=0x7fa05c009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:25.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.194+0000 7fa074228700 1 -- 172.21.15.63:0/1309263420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa06c0fe400 msgr2=0x7fa06c0fdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:25.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.194+0000 7fa074228700 1 --2- 172.21.15.63:0/1309263420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa06c0fe400 0x7fa06c0fdb80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa06800eb40 tx=0x7fa06800b040 comp rx=0 tx=0).stop 2024-03-22T07:15:25.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.194+0000 7fa074228700 1 -- 172.21.15.63:0/1309263420 shutdown_connections 2024-03-22T07:15:25.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.194+0000 7fa074228700 1 --2- 172.21.15.63:0/1309263420 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa0580692a0 0x7fa05806b750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:25.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.194+0000 7fa074228700 1 --2- 172.21.15.63:0/1309263420 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa06c0fe400 0x7fa06c0fdb80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:25.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.194+0000 7fa074228700 1 --2- 172.21.15.63:0/1309263420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa06c0fed50 0x7fa06c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:25.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.194+0000 7fa074228700 1 -- 172.21.15.63:0/1309263420 >> 172.21.15.63:0/1309263420 conn(0x7fa06c0f6ba0 msgr2=0x7fa06c0f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:25.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.195+0000 7fa074228700 1 -- 172.21.15.63:0/1309263420 shutdown_connections 2024-03-22T07:15:25.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.195+0000 7fa074228700 1 -- 172.21.15.63:0/1309263420 wait complete. 2024-03-22T07:15:25.208 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:15:25.306 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.305+0000 7f8b95fdb700 1 -- 172.21.15.63:0/3497395794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b900fee10 msgr2=0x7f8b900faf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.305+0000 7f8b95fdb700 1 --2- 172.21.15.63:0/3497395794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b900fee10 0x7f8b900faf50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8b84004940 tx=0x7f8b84009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.305+0000 7f8b95fdb700 1 -- 172.21.15.63:0/3497395794 shutdown_connections 2024-03-22T07:15:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.305+0000 7f8b95fdb700 1 --2- 172.21.15.63:0/3497395794 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b900fee10 0x7f8b900faf50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.305+0000 7f8b95fdb700 1 --2- 172.21.15.63:0/3497395794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b900fe4c0 0x7f8b900fe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.305+0000 7f8b95fdb700 1 -- 172.21.15.63:0/3497395794 >> 172.21.15.63:0/3497395794 conn(0x7f8b900f6ba0 msgr2=0x7f8b900f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.305+0000 7f8b95fdb700 1 -- 172.21.15.63:0/3497395794 shutdown_connections 2024-03-22T07:15:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.305+0000 7f8b95fdb700 1 -- 172.21.15.63:0/3497395794 wait complete. 2024-03-22T07:15:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.306+0000 7f8b95fdb700 1 Processor -- start 2024-03-22T07:15:25.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.306+0000 7f8b95fdb700 1 -- start start 2024-03-22T07:15:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.306+0000 7f8b95fdb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b900fe4c0 0x7f8b9010d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.306+0000 7f8b95fdb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b900fee10 0x7f8b9010df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.306+0000 7f8b95fdb700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9010e530 con 0x7f8b900fe4c0 2024-03-22T07:15:25.308 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.306+0000 7f8b95fdb700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9010ed30 con 0x7f8b900fee10 2024-03-22T07:15:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.306+0000 7f8b8f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b900fe4c0 0x7f8b9010d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.306+0000 7f8b8f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b900fe4c0 0x7f8b9010d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37730/0 (socket says 172.21.15.63:37730) 2024-03-22T07:15:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.306+0000 7f8b8f7fe700 1 -- 172.21.15.63:0/933820872 learned_addr learned my addr 172.21.15.63:0/933820872 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:15:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.306+0000 7f8b8effd700 1 --2- 172.21.15.63:0/933820872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b900fee10 0x7f8b9010df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.307+0000 7f8b8f7fe700 1 -- 172.21.15.63:0/933820872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b900fee10 msgr2=0x7f8b9010df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.307+0000 7f8b8f7fe700 1 --2- 172.21.15.63:0/933820872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b900fee10 0x7f8b9010df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.307+0000 7f8b8f7fe700 1 -- 172.21.15.63:0/933820872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b84005040 con 0x7f8b900fe4c0 2024-03-22T07:15:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.307+0000 7f8b8f7fe700 1 --2- 172.21.15.63:0/933820872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b900fe4c0 0x7f8b9010d9f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000eb40 tx=0x7f8b8000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.307+0000 7f8b77fff700 1 -- 172.21.15.63:0/933820872 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b80007cc0 con 0x7f8b900fe4c0 2024-03-22T07:15:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.307+0000 7f8b77fff700 1 -- 172.21.15.63:0/933820872 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b8000f040 con 0x7f8b900fe4c0 2024-03-22T07:15:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.307+0000 7f8b95fdb700 1 -- 172.21.15.63:0/933820872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b9010efb0 con 0x7f8b900fe4c0 2024-03-22T07:15:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.307+0000 7f8b77fff700 1 -- 172.21.15.63:0/933820872 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b8000ca10 con 0x7f8b900fe4c0 2024-03-22T07:15:25.309 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.307+0000 7f8b95fdb700 1 -- 172.21.15.63:0/933820872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9010f4f0 con 0x7f8b900fe4c0 2024-03-22T07:15:25.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.308+0000 7f8b95fdb700 1 -- 172.21.15.63:0/933820872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b9005f5f0 con 0x7f8b900fe4c0 2024-03-22T07:15:25.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.309+0000 7f8b77fff700 1 -- 172.21.15.63:0/933820872 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f8b8000cb70 con 0x7f8b900fe4c0 2024-03-22T07:15:25.310 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.310+0000 7f8b77fff700 1 --2- 172.21.15.63:0/933820872 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f8b780692a0 0x7f8b7806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:25.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.310+0000 7f8b77fff700 1 -- 172.21.15.63:0/933820872 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8b80086d20 con 0x7f8b900fe4c0 2024-03-22T07:15:25.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.310+0000 7f8b8effd700 1 --2- 172.21.15.63:0/933820872 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f8b780692a0 0x7f8b7806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:25.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.310+0000 7f8b8effd700 1 --2- 172.21.15.63:0/933820872 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f8b780692a0 0x7f8b7806b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8b84009b30 tx=0x7f8b84009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:25.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.317+0000 7f8b77fff700 1 -- 172.21.15.63:0/933820872 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f8b8004e190 con 0x7f8b900fe4c0 2024-03-22T07:15:25.505 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:25 smithi069 ceph-mon[36441]: pgmap v235: 65 pgs: 65 active+clean; 19 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 620 KiB/s rd, 1.6 MiB/s wr, 73 op/s 2024-03-22T07:15:25.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.583+0000 7f8b95fdb700 1 -- 172.21.15.63:0/933820872 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b9010f8b0 con 0x7f8b780692a0 2024-03-22T07:15:25.585 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:25 smithi063 ceph-mon[29200]: pgmap v235: 65 pgs: 65 active+clean; 19 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 620 KiB/s rd, 1.6 MiB/s wr, 73 op/s 2024-03-22T07:15:25.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.585+0000 7f8b77fff700 1 -- 172.21.15.63:0/933820872 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f8b9010f8b0 con 0x7f8b780692a0 2024-03-22T07:15:25.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.588+0000 7f8b95fdb700 1 -- 172.21.15.63:0/933820872 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f8b780692a0 msgr2=0x7f8b7806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:25.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.588+0000 7f8b95fdb700 1 --2- 172.21.15.63:0/933820872 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f8b780692a0 0x7f8b7806b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8b84009b30 tx=0x7f8b84009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:25.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.588+0000 7f8b95fdb700 1 -- 172.21.15.63:0/933820872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b900fe4c0 msgr2=0x7f8b9010d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:25.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.588+0000 7f8b95fdb700 1 --2- 172.21.15.63:0/933820872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b900fe4c0 0x7f8b9010d9f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000eb40 tx=0x7f8b8000b040 comp rx=0 tx=0).stop 2024-03-22T07:15:25.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.588+0000 7f8b95fdb700 1 -- 172.21.15.63:0/933820872 shutdown_connections 2024-03-22T07:15:25.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.588+0000 7f8b95fdb700 1 --2- 172.21.15.63:0/933820872 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f8b780692a0 0x7f8b7806b750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:25.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.588+0000 7f8b95fdb700 1 --2- 172.21.15.63:0/933820872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8b900fe4c0 0x7f8b9010d9f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:25.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.588+0000 7f8b95fdb700 1 --2- 172.21.15.63:0/933820872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8b900fee10 0x7f8b9010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:25.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.588+0000 7f8b95fdb700 1 -- 172.21.15.63:0/933820872 >> 172.21.15.63:0/933820872 conn(0x7f8b900f6ba0 msgr2=0x7f8b900f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:25.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.588+0000 7f8b95fdb700 1 -- 172.21.15.63:0/933820872 shutdown_connections 2024-03-22T07:15:25.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.588+0000 7f8b95fdb700 1 -- 172.21.15.63:0/933820872 wait complete. 2024-03-22T07:15:25.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.708+0000 7f0e0729c700 1 -- 172.21.15.63:0/2851162579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e000fcf80 msgr2=0x7f0e000fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:25.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.708+0000 7f0e0729c700 1 --2- 172.21.15.63:0/2851162579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e000fcf80 0x7f0e000fd3f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0df0004940 tx=0x7f0df0009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:25.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.709+0000 7f0e0729c700 1 -- 172.21.15.63:0/2851162579 shutdown_connections 2024-03-22T07:15:25.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.709+0000 7f0e0729c700 1 --2- 172.21.15.63:0/2851162579 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e000fcf80 0x7f0e000fd3f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:25.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.709+0000 7f0e0729c700 1 --2- 172.21.15.63:0/2851162579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e000fbcd0 0x7f0e000fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:25.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.709+0000 7f0e0729c700 1 -- 172.21.15.63:0/2851162579 >> 172.21.15.63:0/2851162579 conn(0x7f0e000f7430 msgr2=0x7f0e000f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:25.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.709+0000 7f0e0729c700 1 -- 172.21.15.63:0/2851162579 shutdown_connections 2024-03-22T07:15:25.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.709+0000 7f0e0729c700 1 -- 172.21.15.63:0/2851162579 wait complete. 2024-03-22T07:15:25.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.709+0000 7f0e0729c700 1 Processor -- start 2024-03-22T07:15:25.711 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.710+0000 7f0e0729c700 1 -- start start 2024-03-22T07:15:25.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.710+0000 7f0e0729c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e000fbcd0 0x7f0e000ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:25.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.710+0000 7f0e0729c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e000fcf80 0x7f0e000fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:25.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.710+0000 7f0e0729c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000fe850 con 0x7f0e000fbcd0 2024-03-22T07:15:25.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.710+0000 7f0e0729c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000fe990 con 0x7f0e000fcf80 2024-03-22T07:15:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.710+0000 7f0e05038700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e000fbcd0 0x7f0e000ffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.710+0000 7f0e05038700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e000fbcd0 0x7f0e000ffda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37738/0 (socket says 172.21.15.63:37738) 2024-03-22T07:15:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.710+0000 7f0e05038700 1 -- 172.21.15.63:0/3440451234 learned_addr learned my addr 172.21.15.63:0/3440451234 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:15:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.710+0000 7f0e04837700 1 --2- 172.21.15.63:0/3440451234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e000fcf80 0x7f0e000fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.710+0000 7f0e05038700 1 -- 172.21.15.63:0/3440451234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e000fcf80 msgr2=0x7f0e000fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.710+0000 7f0e05038700 1 --2- 172.21.15.63:0/3440451234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e000fcf80 0x7f0e000fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.710+0000 7f0e05038700 1 -- 172.21.15.63:0/3440451234 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc005040 con 0x7f0e000fbcd0 2024-03-22T07:15:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.711+0000 7f0e05038700 1 --2- 172.21.15.63:0/3440451234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e000fbcd0 0x7f0e000ffda0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc0096d0 tx=0x7f0dfc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.711+0000 7f0df5ffb700 1 -- 172.21.15.63:0/3440451234 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dfc00e840 con 0x7f0e000fbcd0 2024-03-22T07:15:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.711+0000 7f0df5ffb700 1 -- 172.21.15.63:0/3440451234 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0dfc00f040 con 0x7f0e000fbcd0 2024-03-22T07:15:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.711+0000 7f0df5ffb700 1 -- 172.21.15.63:0/3440451234 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dfc013e50 con 0x7f0e000fbcd0 2024-03-22T07:15:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.711+0000 7f0e0729c700 1 -- 172.21.15.63:0/3440451234 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df0005040 con 0x7f0e000fbcd0 2024-03-22T07:15:25.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.712+0000 7f0e0729c700 1 -- 172.21.15.63:0/3440451234 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e000ff050 con 0x7f0e000fbcd0 2024-03-22T07:15:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.713+0000 7f0e0729c700 1 -- 172.21.15.63:0/3440451234 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e00101b10 con 0x7f0e000fbcd0 2024-03-22T07:15:25.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.713+0000 7f0df5ffb700 1 -- 172.21.15.63:0/3440451234 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f0dfc00e9a0 con 0x7f0e000fbcd0 2024-03-22T07:15:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.713+0000 7f0df5ffb700 1 --2- 172.21.15.63:0/3440451234 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0dec0692b0 0x7f0dec06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.714+0000 7f0df5ffb700 1 -- 172.21.15.63:0/3440451234 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0dfc086fd0 con 0x7f0e000fbcd0 2024-03-22T07:15:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.714+0000 7f0e04837700 1 --2- 172.21.15.63:0/3440451234 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0dec0692b0 0x7f0dec06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:25.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.714+0000 7f0e04837700 1 --2- 172.21.15.63:0/3440451234 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0dec0692b0 0x7f0dec06b760 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0df000b0d0 tx=0x7f0df0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:25.719 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:25.718+0000 7f0df5ffb700 1 -- 172.21.15.63:0/3440451234 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f0dfc0532b0 con 0x7f0e000fbcd0 2024-03-22T07:15:26.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.047+0000 7f0e0729c700 1 -- 172.21.15.63:0/3440451234 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e0005e400 con 0x7f0dec0692b0 2024-03-22T07:15:26.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.058+0000 7f0df5ffb700 1 -- 172.21.15.63:0/3440451234 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7f0e0005e400 con 0x7f0dec0692b0 2024-03-22T07:15:26.059 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:15:26.059 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (5m) 87s ago 7m 24.4M - 0.25.0 c8568f914cd2 2f8544438b7c 2024-03-22T07:15:26.059 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (7m) 87s ago 7m 6660k - 17.2.7-726-g9c4e0c34 837124da804f d6c6bbeb6904 2024-03-22T07:15:26.059 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (6m) 93s ago 6m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 30992f8aa88c 2024-03-22T07:15:26.059 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (5m) 87s ago 7m 85.4M - 9.4.7 954c08fa6188 f0b1d2a54439 2024-03-22T07:15:26.059 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (112s) 87s ago 111s 11.2M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:15:26.059 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (106s) 87s ago 105s 13.8M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:15:26.059 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (103s) 93s ago 102s 14.5M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:15:26.060 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (109s) 93s ago 109s 16.1M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:15:26.060 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:9283 running (8m) 87s ago 8m 480M - 17.2.7-726-g9c4e0c34 837124da804f 422b5e836153 2024-03-22T07:15:26.060 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283 starting - - - - 2024-03-22T07:15:26.060 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (8m) 87s ago 8m 49.2M 2048M 17.2.7-726-g9c4e0c34 837124da804f f92a5bb6caee 2024-03-22T07:15:26.060 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (6m) 93s ago 6m 32.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f 95f6103f7603 2024-03-22T07:15:26.060 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (7m) 87s ago 7m 16.3M - 1.5.0 0da6a335fe13 47501603d3e8 2024-03-22T07:15:26.060 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (6m) 93s ago 6m 16.5M - 1.5.0 0da6a335fe13 49f4ca105fb9 2024-03-22T07:15:26.060 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (5m) 87s ago 5m 46.2M 4288M 17.2.7-726-g9c4e0c34 837124da804f 580f22188a10 2024-03-22T07:15:26.060 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (5m) 87s ago 5m 49.1M 4288M 17.2.7-726-g9c4e0c34 837124da804f fe1ace35c88a 2024-03-22T07:15:26.060 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (4m) 87s ago 4m 45.9M 4288M 17.2.7-726-g9c4e0c34 837124da804f 4cd995265dcf 2024-03-22T07:15:26.060 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (4m) 93s ago 4m 47.0M 5312M 17.2.7-726-g9c4e0c34 837124da804f 45c9a759b76d 2024-03-22T07:15:26.060 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (3m) 93s ago 3m 41.9M 5312M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:15:26.060 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (3m) 93s ago 3m 42.5M 5312M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:15:26.060 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (5m) 87s ago 6m 53.7M - 2.43.0 a07b618ecd1d 827638a5870a 2024-03-22T07:15:26.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.061+0000 7f0e0729c700 1 -- 172.21.15.63:0/3440451234 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0dec0692b0 msgr2=0x7f0dec06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:26.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.061+0000 7f0e0729c700 1 --2- 172.21.15.63:0/3440451234 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0dec0692b0 0x7f0dec06b760 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0df000b0d0 tx=0x7f0df0009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:26.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.061+0000 7f0e0729c700 1 -- 172.21.15.63:0/3440451234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e000fbcd0 msgr2=0x7f0e000ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:26.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.061+0000 7f0e0729c700 1 --2- 172.21.15.63:0/3440451234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e000fbcd0 0x7f0e000ffda0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc0096d0 tx=0x7f0dfc009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:26.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.061+0000 7f0e0729c700 1 -- 172.21.15.63:0/3440451234 shutdown_connections 2024-03-22T07:15:26.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.061+0000 7f0e0729c700 1 --2- 172.21.15.63:0/3440451234 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f0dec0692b0 0x7f0dec06b760 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:26.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.061+0000 7f0e0729c700 1 --2- 172.21.15.63:0/3440451234 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0e000fbcd0 0x7f0e000ffda0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:26.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.061+0000 7f0e0729c700 1 --2- 172.21.15.63:0/3440451234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e000fcf80 0x7f0e000fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:26.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.061+0000 7f0e0729c700 1 -- 172.21.15.63:0/3440451234 >> 172.21.15.63:0/3440451234 conn(0x7f0e000f7430 msgr2=0x7f0e00065ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:26.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.061+0000 7f0e0729c700 1 -- 172.21.15.63:0/3440451234 shutdown_connections 2024-03-22T07:15:26.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.062+0000 7f0e0729c700 1 -- 172.21.15.63:0/3440451234 wait complete. 2024-03-22T07:15:26.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.170+0000 7f19ab0fa700 1 -- 172.21.15.63:0/1440784510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a40662e0 msgr2=0x7f19a40ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:26.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.170+0000 7f19ab0fa700 1 --2- 172.21.15.63:0/1440784510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a40662e0 0x7f19a40ff9c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1998004940 tx=0x7f1998009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:26.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.170+0000 7f19ab0fa700 1 -- 172.21.15.63:0/1440784510 shutdown_connections 2024-03-22T07:15:26.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.170+0000 7f19ab0fa700 1 --2- 172.21.15.63:0/1440784510 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19a40fff00 0x7f19a4102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:26.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.170+0000 7f19ab0fa700 1 --2- 172.21.15.63:0/1440784510 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a40662e0 0x7f19a40ff9c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:26.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.170+0000 7f19ab0fa700 1 -- 172.21.15.63:0/1440784510 >> 172.21.15.63:0/1440784510 conn(0x7f19a40f7430 msgr2=0x7f19a40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:26.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.170+0000 7f19ab0fa700 1 -- 172.21.15.63:0/1440784510 shutdown_connections 2024-03-22T07:15:26.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.170+0000 7f19ab0fa700 1 -- 172.21.15.63:0/1440784510 wait complete. 2024-03-22T07:15:26.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.171+0000 7f19ab0fa700 1 Processor -- start 2024-03-22T07:15:26.173 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.171+0000 7f19ab0fa700 1 -- start start 2024-03-22T07:15:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.171+0000 7f19ab0fa700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19a40662e0 0x7f19a410da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.171+0000 7f19ab0fa700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a40fff00 0x7f19a410df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.171+0000 7f19ab0fa700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a410e540 con 0x7f19a40fff00 2024-03-22T07:15:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.171+0000 7f19ab0fa700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a410ed40 con 0x7f19a40662e0 2024-03-22T07:15:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.172+0000 7f19a8e96700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19a40662e0 0x7f19a410da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.172+0000 7f19a3fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a40fff00 0x7f19a410df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.172+0000 7f19a8e96700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19a40662e0 0x7f19a410da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.63:55412/0 (socket says 172.21.15.63:55412) 2024-03-22T07:15:26.174 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.172+0000 7f19a3fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a40fff00 0x7f19a410df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37756/0 (socket says 172.21.15.63:37756) 2024-03-22T07:15:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.172+0000 7f19a8e96700 1 -- 172.21.15.63:0/673641070 learned_addr learned my addr 172.21.15.63:0/673641070 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:15:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.172+0000 7f19a3fff700 1 -- 172.21.15.63:0/673641070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19a40662e0 msgr2=0x7f19a410da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.172+0000 7f19a3fff700 1 --2- 172.21.15.63:0/673641070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19a40662e0 0x7f19a410da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.172+0000 7f19a3fff700 1 -- 172.21.15.63:0/673641070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1998005040 con 0x7f19a40fff00 2024-03-22T07:15:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.172+0000 7f19a3fff700 1 --2- 172.21.15.63:0/673641070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a40fff00 0x7f19a410df40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1990009fd0 tx=0x7f199000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.172+0000 7f19a17fa700 1 -- 172.21.15.63:0/673641070 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1990004e90 con 0x7f19a40fff00 2024-03-22T07:15:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.172+0000 7f19a17fa700 1 -- 172.21.15.63:0/673641070 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f199000f040 con 0x7f19a40fff00 2024-03-22T07:15:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.172+0000 7f19ab0fa700 1 -- 172.21.15.63:0/673641070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a410efc0 con 0x7f19a40fff00 2024-03-22T07:15:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.172+0000 7f19a17fa700 1 -- 172.21.15.63:0/673641070 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f199000ca50 con 0x7f19a40fff00 2024-03-22T07:15:26.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.172+0000 7f19ab0fa700 1 -- 172.21.15.63:0/673641070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a410f500 con 0x7f19a40fff00 2024-03-22T07:15:26.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.174+0000 7f19ab0fa700 1 -- 172.21.15.63:0/673641070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a405f5f0 con 0x7f19a40fff00 2024-03-22T07:15:26.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.174+0000 7f19a17fa700 1 -- 172.21.15.63:0/673641070 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7f1990007cc0 con 0x7f19a40fff00 2024-03-22T07:15:26.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.175+0000 7f19a17fa700 1 --2- 172.21.15.63:0/673641070 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f19940692a0 0x7f199406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:26.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.175+0000 7f19a17fa700 1 -- 172.21.15.63:0/673641070 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f1990086ba0 con 0x7f19a40fff00 2024-03-22T07:15:26.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.175+0000 7f19a8e96700 1 --2- 172.21.15.63:0/673641070 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f19940692a0 0x7f199406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:26.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.175+0000 7f19a8e96700 1 --2- 172.21.15.63:0/673641070 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f19940692a0 0x7f199406b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f19980136a0 tx=0x7f1998009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:26.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.183+0000 7f19a17fa700 1 -- 172.21.15.63:0/673641070 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f199004e190 con 0x7f19a40fff00 2024-03-22T07:15:26.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.519+0000 7f19ab0fa700 1 -- 172.21.15.63:0/673641070 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f19a404bed0 con 0x7f19a40fff00 2024-03-22T07:15:26.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.519+0000 7f19a17fa700 1 -- 172.21.15.63:0/673641070 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f19900569c0 con 0x7f19a40fff00 2024-03-22T07:15:26.520 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:15:26.520 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:15:26.520 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:15:26.520 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:15:26.520 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:15:26.520 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:15:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:15:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:15:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:15:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:15:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:15:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:15:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:15:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:15:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 14 2024-03-22T07:15:26.521 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:15:26.521 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:15:26.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.522+0000 7f19ab0fa700 1 -- 172.21.15.63:0/673641070 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f19940692a0 msgr2=0x7f199406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:26.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.522+0000 7f19ab0fa700 1 --2- 172.21.15.63:0/673641070 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f19940692a0 0x7f199406b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f19980136a0 tx=0x7f1998009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:26.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.522+0000 7f19ab0fa700 1 -- 172.21.15.63:0/673641070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a40fff00 msgr2=0x7f19a410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:26.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.522+0000 7f19ab0fa700 1 --2- 172.21.15.63:0/673641070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a40fff00 0x7f19a410df40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1990009fd0 tx=0x7f199000b040 comp rx=0 tx=0).stop 2024-03-22T07:15:26.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.522+0000 7f19ab0fa700 1 -- 172.21.15.63:0/673641070 shutdown_connections 2024-03-22T07:15:26.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.522+0000 7f19ab0fa700 1 --2- 172.21.15.63:0/673641070 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f19940692a0 0x7f199406b750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:26.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.522+0000 7f19ab0fa700 1 --2- 172.21.15.63:0/673641070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19a40662e0 0x7f19a410da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:26.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.522+0000 7f19ab0fa700 1 --2- 172.21.15.63:0/673641070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19a40fff00 0x7f19a410df40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:26.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.522+0000 7f19ab0fa700 1 -- 172.21.15.63:0/673641070 >> 172.21.15.63:0/673641070 conn(0x7f19a40f7430 msgr2=0x7f19a40f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:26.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.522+0000 7f19ab0fa700 1 -- 172.21.15.63:0/673641070 shutdown_connections 2024-03-22T07:15:26.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.522+0000 7f19ab0fa700 1 -- 172.21.15.63:0/673641070 wait complete. 2024-03-22T07:15:26.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:26 smithi063 ceph-mon[29200]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:26.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:26 smithi063 ceph-mon[29200]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:26.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:26 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:15:26.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:26 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:15:26.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:26 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:15:26.648 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.644+0000 7fb4a1886700 1 -- 172.21.15.63:0/2821546247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49c0fe440 msgr2=0x7fb49c0fe850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:26.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.644+0000 7fb4a1886700 1 --2- 172.21.15.63:0/2821546247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49c0fe440 0x7fb49c0fe850 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb48c004940 tx=0x7fb48c009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:26.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.644+0000 7fb4a1886700 1 -- 172.21.15.63:0/2821546247 shutdown_connections 2024-03-22T07:15:26.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.644+0000 7fb4a1886700 1 --2- 172.21.15.63:0/2821546247 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb49c0fed90 0x7fb49c0faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:26.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.644+0000 7fb4a1886700 1 --2- 172.21.15.63:0/2821546247 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49c0fe440 0x7fb49c0fe850 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:26.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.644+0000 7fb4a1886700 1 -- 172.21.15.63:0/2821546247 >> 172.21.15.63:0/2821546247 conn(0x7fb49c0f6be0 msgr2=0x7fb49c0f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:26.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.644+0000 7fb4a1886700 1 -- 172.21.15.63:0/2821546247 shutdown_connections 2024-03-22T07:15:26.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.645+0000 7fb4a1886700 1 -- 172.21.15.63:0/2821546247 wait complete. 2024-03-22T07:15:26.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.645+0000 7fb4a1886700 1 Processor -- start 2024-03-22T07:15:26.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.645+0000 7fb4a1886700 1 -- start start 2024-03-22T07:15:26.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.645+0000 7fb4a1886700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49c0fe440 0x7fb49c10fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:26.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.645+0000 7fb4a1886700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb49c0fed90 0x7fb49c1100a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:26.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.645+0000 7fb4a1886700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c1106a0 con 0x7fb49c0fe440 2024-03-22T07:15:26.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.645+0000 7fb4a1886700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c110810 con 0x7fb49c0fed90 2024-03-22T07:15:26.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.645+0000 7fb49affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49c0fe440 0x7fb49c10fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:26.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.646+0000 7fb49affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49c0fe440 0x7fb49c10fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37778/0 (socket says 172.21.15.63:37778) 2024-03-22T07:15:26.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.646+0000 7fb49affd700 1 -- 172.21.15.63:0/1591206494 learned_addr learned my addr 172.21.15.63:0/1591206494 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:15:26.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.646+0000 7fb49a7fc700 1 --2- 172.21.15.63:0/1591206494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb49c0fed90 0x7fb49c1100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:26.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.646+0000 7fb49affd700 1 -- 172.21.15.63:0/1591206494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb49c0fed90 msgr2=0x7fb49c1100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:26.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.646+0000 7fb49affd700 1 --2- 172.21.15.63:0/1591206494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb49c0fed90 0x7fb49c1100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:26.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.646+0000 7fb49affd700 1 -- 172.21.15.63:0/1591206494 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c005040 con 0x7fb49c0fe440 2024-03-22T07:15:26.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.646+0000 7fb49affd700 1 --2- 172.21.15.63:0/1591206494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49c0fe440 0x7fb49c10fb60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00b3b0 tx=0x7fb48c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:26.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.646+0000 7fb4837fe700 1 -- 172.21.15.63:0/1591206494 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb48c00e040 con 0x7fb49c0fe440 2024-03-22T07:15:26.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.646+0000 7fb4837fe700 1 -- 172.21.15.63:0/1591206494 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb48c00bbf0 con 0x7fb49c0fe440 2024-03-22T07:15:26.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.646+0000 7fb4837fe700 1 -- 172.21.15.63:0/1591206494 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb48c01c630 con 0x7fb49c0fe440 2024-03-22T07:15:26.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.646+0000 7fb4a1886700 1 -- 172.21.15.63:0/1591206494 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb49c115210 con 0x7fb49c0fe440 2024-03-22T07:15:26.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.646+0000 7fb4a1886700 1 -- 172.21.15.63:0/1591206494 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49c115780 con 0x7fb49c0fe440 2024-03-22T07:15:26.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.647+0000 7fb4a1886700 1 -- 172.21.15.63:0/1591206494 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49c101b80 con 0x7fb49c0fe440 2024-03-22T07:15:26.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.648+0000 7fb4837fe700 1 -- 172.21.15.63:0/1591206494 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 19) v1 ==== 86621+0+0 (secure 0 0 0) 0x7fb48c00bd60 con 0x7fb49c0fe440 2024-03-22T07:15:26.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.649+0000 7fb4837fe700 1 --2- 172.21.15.63:0/1591206494 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fb4840692a0 0x7fb48406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:26.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.649+0000 7fb4837fe700 1 -- 172.21.15.63:0/1591206494 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb48c087e90 con 0x7fb49c0fe440 2024-03-22T07:15:26.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.649+0000 7fb49a7fc700 1 --2- 172.21.15.63:0/1591206494 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fb4840692a0 0x7fb48406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:26.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.649+0000 7fb49a7fc700 1 --2- 172.21.15.63:0/1591206494 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fb4840692a0 0x7fb48406b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb490004f40 tx=0x7fb49000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:26.658 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.657+0000 7fb4837fe700 1 -- 172.21.15.63:0/1591206494 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fb48c057c00 con 0x7fb49c0fe440 2024-03-22T07:15:26.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:26 smithi069 ceph-mon[36441]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:26.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:26 smithi069 ceph-mon[36441]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:26.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:26 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:15:26.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:26 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:15:26.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:26 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:15:26.946 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.945+0000 7fb4a1886700 1 -- 172.21.15.63:0/1591206494 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb49c02b340 con 0x7fb4840692a0 2024-03-22T07:15:26.947 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.947+0000 7fb4837fe700 1 -- 172.21.15.63:0/1591206494 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fb49c02b340 con 0x7fb4840692a0 2024-03-22T07:15:26.948 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:15:26.948 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:15:26.948 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:15:26.948 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-22T07:15:26.948 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-03-22T07:15:26.948 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "", 2024-03-22T07:15:26.948 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mgr daemons", 2024-03-22T07:15:26.948 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:15:26.948 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:15:26.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.949+0000 7fb4a1886700 1 -- 172.21.15.63:0/1591206494 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fb4840692a0 msgr2=0x7fb48406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:26.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.949+0000 7fb4a1886700 1 --2- 172.21.15.63:0/1591206494 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fb4840692a0 0x7fb48406b750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb490004f40 tx=0x7fb49000b040 comp rx=0 tx=0).stop 2024-03-22T07:15:26.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.949+0000 7fb4a1886700 1 -- 172.21.15.63:0/1591206494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49c0fe440 msgr2=0x7fb49c10fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:26.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.949+0000 7fb4a1886700 1 --2- 172.21.15.63:0/1591206494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49c0fe440 0x7fb49c10fb60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00b3b0 tx=0x7fb48c009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:26.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.949+0000 7fb4a1886700 1 -- 172.21.15.63:0/1591206494 shutdown_connections 2024-03-22T07:15:26.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.949+0000 7fb4a1886700 1 --2- 172.21.15.63:0/1591206494 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fb4840692a0 0x7fb48406b750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:26.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.949+0000 7fb4a1886700 1 --2- 172.21.15.63:0/1591206494 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb49c0fe440 0x7fb49c10fb60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:26.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.949+0000 7fb4a1886700 1 --2- 172.21.15.63:0/1591206494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb49c0fed90 0x7fb49c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:26.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.949+0000 7fb4a1886700 1 -- 172.21.15.63:0/1591206494 >> 172.21.15.63:0/1591206494 conn(0x7fb49c0f6be0 msgr2=0x7fb49c0f81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:26.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.949+0000 7fb4a1886700 1 -- 172.21.15.63:0/1591206494 shutdown_connections 2024-03-22T07:15:26.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:26.949+0000 7fb4a1886700 1 -- 172.21.15.63:0/1591206494 wait complete. 2024-03-22T07:15:27.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:27 smithi063 ceph-mon[29200]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:27.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:27 smithi063 ceph-mon[29200]: pgmap v236: 65 pgs: 65 active+clean; 50 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 362 op/s 2024-03-22T07:15:27.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:27 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/673641070' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:15:27.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:27 smithi069 ceph-mon[36441]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:27.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:27 smithi069 ceph-mon[36441]: pgmap v236: 65 pgs: 65 active+clean; 50 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 362 op/s 2024-03-22T07:15:27.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:27 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/673641070' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:15:28.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:28 smithi069 ceph-mon[36441]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:28.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:28 smithi063 ceph-mon[29200]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:29.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:29 smithi069 ceph-mon[36441]: pgmap v237: 65 pgs: 65 active+clean; 62 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 441 op/s 2024-03-22T07:15:29.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:29 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:15:29.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:29 smithi063 ceph-mon[29200]: pgmap v237: 65 pgs: 65 active+clean; 62 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 441 op/s 2024-03-22T07:15:29.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:29 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:15:31.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:31 smithi069 ceph-mon[36441]: pgmap v238: 65 pgs: 65 active+clean; 77 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 516 op/s 2024-03-22T07:15:31.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:31 smithi063 ceph-mon[29200]: pgmap v238: 65 pgs: 65 active+clean; 77 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 516 op/s 2024-03-22T07:15:32.622 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:32 smithi069 ceph-mon[36441]: fsmap cephfs:2 {0=cephfs.smithi063.xeelpk=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2 up:standby-replay 2024-03-22T07:15:32.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:32 smithi063 ceph-mon[29200]: fsmap cephfs:2 {0=cephfs.smithi063.xeelpk=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2 up:standby-replay 2024-03-22T07:15:33.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:33 smithi063 ceph-mon[29200]: pgmap v239: 65 pgs: 65 active+clean; 81 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 517 op/s 2024-03-22T07:15:33.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:33 smithi063 ceph-mon[29200]: Standby manager daemon smithi069.gfziwh restarted 2024-03-22T07:15:33.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:33 smithi063 ceph-mon[29200]: Standby manager daemon smithi069.gfziwh started 2024-03-22T07:15:33.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:33 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/crt"}]: dispatch 2024-03-22T07:15:33.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:33 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:15:33.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:33 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/key"}]: dispatch 2024-03-22T07:15:33.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:33 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:15:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:33 smithi069 ceph-mon[36441]: pgmap v239: 65 pgs: 65 active+clean; 81 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 517 op/s 2024-03-22T07:15:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:33 smithi069 ceph-mon[36441]: Standby manager daemon smithi069.gfziwh restarted 2024-03-22T07:15:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:33 smithi069 ceph-mon[36441]: Standby manager daemon smithi069.gfziwh started 2024-03-22T07:15:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:33 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/crt"}]: dispatch 2024-03-22T07:15:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:33 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:15:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:33 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/key"}]: dispatch 2024-03-22T07:15:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:33 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:15:34.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:34 smithi069 ceph-mon[36441]: mgrmap e20: smithi063.mnxayh(active, since 6m), standbys: smithi069.gfziwh 2024-03-22T07:15:34.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:34 smithi063 ceph-mon[29200]: mgrmap e20: smithi063.mnxayh(active, since 6m), standbys: smithi069.gfziwh 2024-03-22T07:15:35.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:35 smithi069 ceph-mon[36441]: pgmap v240: 65 pgs: 65 active+clean; 85 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 516 op/s 2024-03-22T07:15:35.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:35 smithi063 ceph-mon[29200]: pgmap v240: 65 pgs: 65 active+clean; 85 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 516 op/s 2024-03-22T07:15:37.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:37 smithi063 ceph-mon[29200]: pgmap v241: 65 pgs: 65 active+clean; 97 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 6.8 MiB/s wr, 534 op/s 2024-03-22T07:15:37.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:37 smithi069 ceph-mon[36441]: pgmap v241: 65 pgs: 65 active+clean; 97 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.8 MiB/s rd, 6.8 MiB/s wr, 534 op/s 2024-03-22T07:15:39.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:39 smithi063 ceph-mon[29200]: pgmap v242: 65 pgs: 65 active+clean; 108 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 5.0 MiB/s wr, 297 op/s 2024-03-22T07:15:39.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:39 smithi069 ceph-mon[36441]: pgmap v242: 65 pgs: 65 active+clean; 108 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s rd, 5.0 MiB/s wr, 297 op/s 2024-03-22T07:15:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:41 smithi063 ceph-mon[29200]: pgmap v243: 65 pgs: 65 active+clean; 127 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 5.8 MiB/s wr, 537 op/s 2024-03-22T07:15:41.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:41 smithi069 ceph-mon[36441]: pgmap v243: 65 pgs: 65 active+clean; 127 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 5.8 MiB/s wr, 537 op/s 2024-03-22T07:15:43.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:43 smithi063 ceph-mon[29200]: pgmap v244: 65 pgs: 65 active+clean; 146 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 6.2 MiB/s wr, 713 op/s 2024-03-22T07:15:43.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:43 smithi069 ceph-mon[36441]: pgmap v244: 65 pgs: 65 active+clean; 146 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 6.2 MiB/s wr, 713 op/s 2024-03-22T07:15:44.351 INFO:tasks.workunit.client.0.smithi063.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-22T07:15:44.352 INFO:tasks.workunit.client.0.smithi063.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-22T07:15:44.352 INFO:tasks.workunit.client.0.smithi063.stderr:+ make 2024-03-22T07:15:44.532 INFO:tasks.workunit.client.0.smithi063.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-22T07:15:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:44 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:15:44.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:44 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:15:44.789 INFO:tasks.workunit.client.0.smithi063.stderr:++ readlink -f fsstress 2024-03-22T07:15:44.790 INFO:tasks.workunit.client.0.smithi063.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-22T07:15:44.790 INFO:tasks.workunit.client.0.smithi063.stderr:+ popd 2024-03-22T07:15:44.790 INFO:tasks.workunit.client.0.smithi063.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-22T07:15:44.791 INFO:tasks.workunit.client.0.smithi063.stderr:+ popd 2024-03-22T07:15:44.791 INFO:tasks.workunit.client.0.smithi063.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-22T07:15:44.791 INFO:tasks.workunit.client.0.smithi063.stderr:++ mktemp -d -p . 2024-03-22T07:15:44.792 INFO:tasks.workunit.client.0.smithi063.stderr:+ T=./tmp.4YiehetO4A 2024-03-22T07:15:44.792 INFO:tasks.workunit.client.0.smithi063.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.4YiehetO4A -l 1 -n 1000 -p 10 -v 2024-03-22T07:15:44.792 INFO:tasks.workunit.client.0.smithi063.stdout:seed = 1710304336 2024-03-22T07:15:44.794 INFO:tasks.workunit.client.0.smithi063.stdout:0/0: readlink - no filename 2024-03-22T07:15:44.794 INFO:tasks.workunit.client.0.smithi063.stdout:0/1: write - no filename 2024-03-22T07:15:44.794 INFO:tasks.workunit.client.0.smithi063.stdout:0/2: write - no filename 2024-03-22T07:15:44.794 INFO:tasks.workunit.client.0.smithi063.stdout:1/0: fdatasync - no filename 2024-03-22T07:15:44.794 INFO:tasks.workunit.client.0.smithi063.stdout:1/1: truncate - no filename 2024-03-22T07:15:44.794 INFO:tasks.workunit.client.0.smithi063.stdout:1/2: truncate - no filename 2024-03-22T07:15:44.794 INFO:tasks.workunit.client.0.smithi063.stdout:1/3: link - no file 2024-03-22T07:15:44.795 INFO:tasks.workunit.client.0.smithi063.stdout:0/3: creat f0 x:0 0 0 2024-03-22T07:15:44.795 INFO:tasks.workunit.client.0.smithi063.stdout:0/4: write f0 [930391,40834] 0 2024-03-22T07:15:44.795 INFO:tasks.workunit.client.0.smithi063.stdout:2/0: write - no filename 2024-03-22T07:15:44.795 INFO:tasks.workunit.client.0.smithi063.stdout:2/1: dread - no filename 2024-03-22T07:15:44.795 INFO:tasks.workunit.client.0.smithi063.stdout:2/2: dwrite - no filename 2024-03-22T07:15:44.796 INFO:tasks.workunit.client.0.smithi063.stdout:1/4: creat f0 x:0 0 0 2024-03-22T07:15:44.798 INFO:tasks.workunit.client.0.smithi063.stdout:2/3: mkdir d0 0 2024-03-22T07:15:44.798 INFO:tasks.workunit.client.0.smithi063.stdout:2/4: rename d0 to d0/d1 22 2024-03-22T07:15:44.798 INFO:tasks.workunit.client.0.smithi063.stdout:2/5: stat d0 0 2024-03-22T07:15:44.798 INFO:tasks.workunit.client.0.smithi063.stdout:1/5: symlink l1 0 2024-03-22T07:15:44.798 INFO:tasks.workunit.client.0.smithi063.stdout:1/6: chown l1 107 1 2024-03-22T07:15:44.798 INFO:tasks.workunit.client.0.smithi063.stdout:4/0: chown . 137 1 2024-03-22T07:15:44.799 INFO:tasks.workunit.client.0.smithi063.stdout:3/0: symlink l0 0 2024-03-22T07:15:44.799 INFO:tasks.workunit.client.0.smithi063.stdout:3/1: dread - no filename 2024-03-22T07:15:44.801 INFO:tasks.workunit.client.0.smithi063.stdout:2/6: mknod d0/c2 0 2024-03-22T07:15:44.804 INFO:tasks.workunit.client.0.smithi063.stdout:3/2: mknod c1 0 2024-03-22T07:15:44.806 INFO:tasks.workunit.client.0.smithi063.stdout:2/7: rmdir d0 39 2024-03-22T07:15:44.807 INFO:tasks.workunit.client.0.smithi063.stdout:2/8: write - no filename 2024-03-22T07:15:44.807 INFO:tasks.workunit.client.0.smithi063.stdout:2/9: readlink - no filename 2024-03-22T07:15:44.807 INFO:tasks.workunit.client.0.smithi063.stdout:6/0: dread - no filename 2024-03-22T07:15:44.807 INFO:tasks.workunit.client.0.smithi063.stdout:6/1: dread - no filename 2024-03-22T07:15:44.808 INFO:tasks.workunit.client.0.smithi063.stdout:5/0: mknod c0 0 2024-03-22T07:15:44.810 INFO:tasks.workunit.client.0.smithi063.stdout:3/3: unlink l0 0 2024-03-22T07:15:44.810 INFO:tasks.workunit.client.0.smithi063.stdout:3/4: truncate - no filename 2024-03-22T07:15:44.811 INFO:tasks.workunit.client.0.smithi063.stdout:0/5: fsync f0 0 2024-03-22T07:15:44.814 INFO:tasks.workunit.client.0.smithi063.stdout:6/2: symlink l0 0 2024-03-22T07:15:44.814 INFO:tasks.workunit.client.0.smithi063.stdout:6/3: chown l0 1 1 2024-03-22T07:15:44.824 INFO:tasks.workunit.client.0.smithi063.stdout:3/5: rename c1 to c2 0 2024-03-22T07:15:44.824 INFO:tasks.workunit.client.0.smithi063.stdout:3/6: truncate - no filename 2024-03-22T07:15:44.825 INFO:tasks.workunit.client.0.smithi063.stdout:2/10: rmdir d0 39 2024-03-22T07:15:44.831 INFO:tasks.workunit.client.0.smithi063.stdout:7/0: mknod c0 0 2024-03-22T07:15:44.832 INFO:tasks.workunit.client.0.smithi063.stdout:6/4: mknod c1 0 2024-03-22T07:15:44.832 INFO:tasks.workunit.client.0.smithi063.stdout:6/5: write - no filename 2024-03-22T07:15:44.835 INFO:tasks.workunit.client.0.smithi063.stdout:3/7: mknod c3 0 2024-03-22T07:15:44.838 INFO:tasks.workunit.client.0.smithi063.stdout:2/11: rename d0/c2 to d0/c3 0 2024-03-22T07:15:44.840 INFO:tasks.workunit.client.0.smithi063.stdout:6/6: unlink l0 0 2024-03-22T07:15:44.841 INFO:tasks.workunit.client.0.smithi063.stdout:6/7: fsync - no filename 2024-03-22T07:15:44.841 INFO:tasks.workunit.client.0.smithi063.stdout:6/8: dwrite - no filename 2024-03-22T07:15:44.841 INFO:tasks.workunit.client.0.smithi063.stdout:6/9: write - no filename 2024-03-22T07:15:44.841 INFO:tasks.workunit.client.0.smithi063.stdout:3/8: mknod c4 0 2024-03-22T07:15:44.841 INFO:tasks.workunit.client.0.smithi063.stdout:3/9: chown c4 9 1 2024-03-22T07:15:44.841 INFO:tasks.workunit.client.0.smithi063.stdout:3/10: rmdir - no directory 2024-03-22T07:15:44.841 INFO:tasks.workunit.client.0.smithi063.stdout:3/11: chown c3 26 1 2024-03-22T07:15:44.844 INFO:tasks.workunit.client.0.smithi063.stdout:2/12: creat d0/f4 x:0 0 0 2024-03-22T07:15:44.844 INFO:tasks.workunit.client.0.smithi063.stdout:2/13: dread - d0/f4 zero size 2024-03-22T07:15:44.844 INFO:tasks.workunit.client.0.smithi063.stdout:2/14: creat d0/f5 x:0 0 0 2024-03-22T07:15:44.844 INFO:tasks.workunit.client.0.smithi063.stdout:6/10: creat f2 x:0 0 0 2024-03-22T07:15:44.844 INFO:tasks.workunit.client.0.smithi063.stdout:6/11: read - f2 zero size 2024-03-22T07:15:44.845 INFO:tasks.workunit.client.0.smithi063.stdout:6/12: dread - f2 zero size 2024-03-22T07:15:44.845 INFO:tasks.workunit.client.0.smithi063.stdout:6/13: read - f2 zero size 2024-03-22T07:15:44.845 INFO:tasks.workunit.client.0.smithi063.stdout:6/14: dread - f2 zero size 2024-03-22T07:15:44.845 INFO:tasks.workunit.client.0.smithi063.stdout:6/15: write f2 [927088,5945] 0 2024-03-22T07:15:44.847 INFO:tasks.workunit.client.0.smithi063.stdout:3/12: creat f5 x:0 0 0 2024-03-22T07:15:44.906 INFO:tasks.workunit.client.0.smithi063.stdout:5/1: getdents . 0 2024-03-22T07:15:45.031 INFO:tasks.workunit.client.0.smithi063.stdout:1/7: dwrite f0 [0,4194304] 0 2024-03-22T07:15:45.125 INFO:tasks.workunit.client.0.smithi063.stdout:0/6: dread f0 [0,4194304] 0 2024-03-22T07:15:45.152 INFO:tasks.workunit.client.0.smithi063.stdout:0/7: dread f0 [0,4194304] 0 2024-03-22T07:15:45.152 INFO:tasks.workunit.client.0.smithi063.stdout:0/8: chown f0 25 1 2024-03-22T07:15:45.157 INFO:tasks.workunit.client.0.smithi063.stdout:0/9: dread f0 [0,4194304] 0 2024-03-22T07:15:45.159 INFO:tasks.workunit.client.0.smithi063.stdout:0/10: dread f0 [0,4194304] 0 2024-03-22T07:15:45.162 INFO:tasks.workunit.client.0.smithi063.stdout:0/11: dread f0 [0,4194304] 0 2024-03-22T07:15:45.162 INFO:tasks.workunit.client.0.smithi063.stdout:0/12: creat f1 x:0 0 0 2024-03-22T07:15:45.162 INFO:tasks.workunit.client.0.smithi063.stdout:0/13: chown f1 24584 1 2024-03-22T07:15:45.240 INFO:tasks.workunit.client.0.smithi063.stdout:2/15: dwrite d0/f4 [0,4194304] 0 2024-03-22T07:15:45.283 INFO:tasks.workunit.client.0.smithi063.stdout:1/8: link f0 f2 0 2024-03-22T07:15:45.290 INFO:tasks.workunit.client.0.smithi063.stdout:1/9: write f2 [689294,125693] 0 2024-03-22T07:15:45.290 INFO:tasks.workunit.client.0.smithi063.stdout:7/1: write - no filename 2024-03-22T07:15:45.290 INFO:tasks.workunit.client.0.smithi063.stdout:9/0: link - no file 2024-03-22T07:15:45.291 INFO:tasks.workunit.client.0.smithi063.stdout:9/1: stat - no entries 2024-03-22T07:15:45.291 INFO:tasks.workunit.client.0.smithi063.stdout:8/0: dwrite - no filename 2024-03-22T07:15:45.570 INFO:tasks.workunit.client.0.smithi063.stdout:1/10: dwrite f2 [0,4194304] 0 2024-03-22T07:15:45.570 INFO:tasks.workunit.client.0.smithi063.stdout:1/11: rmdir - no directory 2024-03-22T07:15:45.728 INFO:tasks.workunit.client.0.smithi063.stdout:3/13: dwrite f5 [0,4194304] 0 2024-03-22T07:15:45.728 INFO:tasks.workunit.client.0.smithi063.stdout:3/14: chown c4 16235319 1 2024-03-22T07:15:45.728 INFO:tasks.workunit.client.0.smithi063.stdout:3/15: creat f6 x:0 0 0 2024-03-22T07:15:45.728 INFO:tasks.workunit.client.0.smithi063.stdout:3/16: rmdir - no directory 2024-03-22T07:15:45.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:45 smithi069 ceph-mon[36441]: pgmap v245: 65 pgs: 65 active+clean; 150 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 6.2 MiB/s wr, 711 op/s 2024-03-22T07:15:45.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:45 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:15:45.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:45 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:15:45.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:45 smithi063 ceph-mon[29200]: pgmap v245: 65 pgs: 65 active+clean; 150 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 6.2 MiB/s wr, 711 op/s 2024-03-22T07:15:45.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:45 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:15:45.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:45 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:15:46.236 INFO:tasks.workunit.client.0.smithi063.stdout:3/17: dread f5 [0,4194304] 0 2024-03-22T07:15:46.383 INFO:tasks.workunit.client.0.smithi063.stdout:3/18: dwrite f5 [0,4194304] 0 2024-03-22T07:15:46.609 INFO:tasks.workunit.client.0.smithi063.stdout:0/14: rename f0 to f2 0 2024-03-22T07:15:46.609 INFO:tasks.workunit.client.0.smithi063.stdout:0/15: creat f3 x:0 0 0 2024-03-22T07:15:46.609 INFO:tasks.workunit.client.0.smithi063.stdout:0/16: creat f4 x:0 0 0 2024-03-22T07:15:47.300 INFO:tasks.workunit.client.0.smithi063.stdout:5/2: creat f1 x:0 0 0 2024-03-22T07:15:47.300 INFO:tasks.workunit.client.0.smithi063.stdout:5/3: rmdir - no directory 2024-03-22T07:15:47.301 INFO:tasks.workunit.client.0.smithi063.stdout:7/2: symlink l1 0 2024-03-22T07:15:47.301 INFO:tasks.workunit.client.0.smithi063.stdout:7/3: dread - no filename 2024-03-22T07:15:47.301 INFO:tasks.workunit.client.0.smithi063.stdout:7/4: dread - no filename 2024-03-22T07:15:47.302 INFO:tasks.workunit.client.0.smithi063.stdout:9/2: mknod c0 0 2024-03-22T07:15:47.302 INFO:tasks.workunit.client.0.smithi063.stdout:9/3: write - no filename 2024-03-22T07:15:47.303 INFO:tasks.workunit.client.0.smithi063.stdout:8/1: mkdir d0 0 2024-03-22T07:15:47.303 INFO:tasks.workunit.client.0.smithi063.stdout:8/2: chown d0 4 1 2024-03-22T07:15:47.303 INFO:tasks.workunit.client.0.smithi063.stdout:8/3: dread - no filename 2024-03-22T07:15:47.303 INFO:tasks.workunit.client.0.smithi063.stdout:8/4: dwrite - no filename 2024-03-22T07:15:47.303 INFO:tasks.workunit.client.0.smithi063.stdout:8/5: dread - no filename 2024-03-22T07:15:47.443 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:47 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:15:47.443 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:47 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:15:47.443 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:47 smithi069 ceph-mon[36441]: pgmap v246: 65 pgs: 65 active+clean; 173 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.8 MiB/s rd, 7.8 MiB/s wr, 814 op/s 2024-03-22T07:15:47.557 INFO:tasks.workunit.client.0.smithi063.stdout:5/4: rename f1 to f2 0 2024-03-22T07:15:47.558 INFO:tasks.workunit.client.0.smithi063.stdout:7/5: rename c0 to c2 0 2024-03-22T07:15:47.558 INFO:tasks.workunit.client.0.smithi063.stdout:7/6: dwrite - no filename 2024-03-22T07:15:47.559 INFO:tasks.workunit.client.0.smithi063.stdout:9/4: link c0 c1 0 2024-03-22T07:15:47.559 INFO:tasks.workunit.client.0.smithi063.stdout:9/5: dread - no filename 2024-03-22T07:15:47.560 INFO:tasks.workunit.client.0.smithi063.stdout:8/6: creat d0/f1 x:0 0 0 2024-03-22T07:15:47.560 INFO:tasks.workunit.client.0.smithi063.stdout:8/7: readlink - no filename 2024-03-22T07:15:47.560 INFO:tasks.workunit.client.0.smithi063.stdout:8/8: stat d0/f1 0 2024-03-22T07:15:47.562 INFO:tasks.workunit.client.0.smithi063.stdout:0/17: rename f4 to f5 0 2024-03-22T07:15:47.563 INFO:tasks.workunit.client.0.smithi063.stdout:5/5: rename c0 to c3 0 2024-03-22T07:15:47.564 INFO:tasks.workunit.client.0.smithi063.stdout:7/7: mknod c3 0 2024-03-22T07:15:47.565 INFO:tasks.workunit.client.0.smithi063.stdout:9/6: creat f2 x:0 0 0 2024-03-22T07:15:47.565 INFO:tasks.workunit.client.0.smithi063.stdout:9/7: chown c0 338395 1 2024-03-22T07:15:47.566 INFO:tasks.workunit.client.0.smithi063.stdout:0/18: mknod c6 0 2024-03-22T07:15:47.567 INFO:tasks.workunit.client.0.smithi063.stdout:0/19: chown f2 1712010 1 2024-03-22T07:15:47.569 INFO:tasks.workunit.client.0.smithi063.stdout:0/20: symlink l7 0 2024-03-22T07:15:47.571 INFO:tasks.workunit.client.0.smithi063.stdout:0/21: unlink c6 0 2024-03-22T07:15:47.571 INFO:tasks.workunit.client.0.smithi063.stdout:0/22: fsync f2 0 2024-03-22T07:15:47.572 INFO:tasks.workunit.client.0.smithi063.stdout:0/23: chown l7 7809344 1 2024-03-22T07:15:47.572 INFO:tasks.workunit.client.0.smithi063.stdout:0/24: write f1 [167081,105512] 0 2024-03-22T07:15:47.573 INFO:tasks.workunit.client.0.smithi063.stdout:0/25: dread f2 [0,4194304] 0 2024-03-22T07:15:47.575 INFO:tasks.workunit.client.0.smithi063.stdout:0/26: link f3 f8 0 2024-03-22T07:15:47.575 INFO:tasks.workunit.client.0.smithi063.stdout:0/27: write f3 [680742,24765] 0 2024-03-22T07:15:47.575 INFO:tasks.workunit.client.0.smithi063.stdout:0/28: write f8 [1661496,48648] 0 2024-03-22T07:15:47.620 INFO:tasks.workunit.client.0.smithi063.stdout:2/16: truncate d0/f4 1670386 0 2024-03-22T07:15:47.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:47 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:15:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:47 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:15:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:47 smithi063 ceph-mon[29200]: pgmap v246: 65 pgs: 65 active+clean; 173 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 3.8 MiB/s rd, 7.8 MiB/s wr, 814 op/s 2024-03-22T07:15:47.734 INFO:tasks.workunit.client.0.smithi063.stdout:3/19: write f5 [4846653,22720] 0 2024-03-22T07:15:47.735 INFO:tasks.workunit.client.0.smithi063.stdout:3/20: symlink l7 0 2024-03-22T07:15:47.736 INFO:tasks.workunit.client.0.smithi063.stdout:3/21: symlink l8 0 2024-03-22T07:15:47.737 INFO:tasks.workunit.client.0.smithi063.stdout:3/22: rename l7 to l9 0 2024-03-22T07:15:47.739 INFO:tasks.workunit.client.0.smithi063.stdout:3/23: fdatasync f5 0 2024-03-22T07:15:47.739 INFO:tasks.workunit.client.0.smithi063.stdout:3/24: readlink l9 0 2024-03-22T07:15:47.739 INFO:tasks.workunit.client.0.smithi063.stdout:3/25: chown c2 336199952 1 2024-03-22T07:15:47.740 INFO:tasks.workunit.client.0.smithi063.stdout:3/26: dread - f6 zero size 2024-03-22T07:15:47.740 INFO:tasks.workunit.client.0.smithi063.stdout:7/8: link c2 c4 0 2024-03-22T07:15:47.740 INFO:tasks.workunit.client.0.smithi063.stdout:7/9: chown c2 335 1 2024-03-22T07:15:47.740 INFO:tasks.workunit.client.0.smithi063.stdout:7/10: write - no filename 2024-03-22T07:15:47.741 INFO:tasks.workunit.client.0.smithi063.stdout:7/11: mkdir d5 0 2024-03-22T07:15:47.741 INFO:tasks.workunit.client.0.smithi063.stdout:7/12: write - no filename 2024-03-22T07:15:47.743 INFO:tasks.workunit.client.0.smithi063.stdout:8/9: fsync d0/f1 0 2024-03-22T07:15:47.743 INFO:tasks.workunit.client.0.smithi063.stdout:8/10: chown d0 104435078 1 2024-03-22T07:15:47.743 INFO:tasks.workunit.client.0.smithi063.stdout:1/12: truncate f2 43907 0 2024-03-22T07:15:47.744 INFO:tasks.workunit.client.0.smithi063.stdout:8/11: mknod d0/c2 0 2024-03-22T07:15:47.745 INFO:tasks.workunit.client.0.smithi063.stdout:1/13: symlink l3 0 2024-03-22T07:15:47.746 INFO:tasks.workunit.client.0.smithi063.stdout:5/6: getdents . 0 2024-03-22T07:15:47.747 INFO:tasks.workunit.client.0.smithi063.stdout:8/12: symlink d0/l3 0 2024-03-22T07:15:47.748 INFO:tasks.workunit.client.0.smithi063.stdout:8/13: chown d0 615 1 2024-03-22T07:15:47.748 INFO:tasks.workunit.client.0.smithi063.stdout:8/14: truncate d0/f1 838227 0 2024-03-22T07:15:47.749 INFO:tasks.workunit.client.0.smithi063.stdout:1/14: mkdir d4 0 2024-03-22T07:15:47.749 INFO:tasks.workunit.client.0.smithi063.stdout:9/8: fsync f2 0 2024-03-22T07:15:47.750 INFO:tasks.workunit.client.0.smithi063.stdout:9/9: rmdir - no directory 2024-03-22T07:15:47.750 INFO:tasks.workunit.client.0.smithi063.stdout:9/10: chown c0 3363 1 2024-03-22T07:15:47.750 INFO:tasks.workunit.client.0.smithi063.stdout:9/11: read - f2 zero size 2024-03-22T07:15:47.750 INFO:tasks.workunit.client.0.smithi063.stdout:9/12: write f2 [256212,99704] 0 2024-03-22T07:15:47.750 INFO:tasks.workunit.client.0.smithi063.stdout:5/7: symlink l4 0 2024-03-22T07:15:47.751 INFO:tasks.workunit.client.0.smithi063.stdout:1/15: rename l1 to d4/l5 0 2024-03-22T07:15:47.752 INFO:tasks.workunit.client.0.smithi063.stdout:5/8: fsync f2 0 2024-03-22T07:15:47.752 INFO:tasks.workunit.client.0.smithi063.stdout:5/9: truncate f2 458081 0 2024-03-22T07:15:47.758 INFO:tasks.workunit.client.0.smithi063.stdout:0/29: truncate f3 892756 0 2024-03-22T07:15:47.758 INFO:tasks.workunit.client.0.smithi063.stdout:0/30: write f1 [1134792,14988] 0 2024-03-22T07:15:47.760 INFO:tasks.workunit.client.0.smithi063.stdout:5/10: rename c3 to c5 0 2024-03-22T07:15:47.762 INFO:tasks.workunit.client.0.smithi063.stdout:5/11: read f2 [76585,102015] 0 2024-03-22T07:15:47.764 INFO:tasks.workunit.client.0.smithi063.stdout:5/12: mkdir d6 0 2024-03-22T07:15:47.766 INFO:tasks.workunit.client.0.smithi063.stdout:5/13: rmdir d6 0 2024-03-22T07:15:47.771 INFO:tasks.workunit.client.0.smithi063.stdout:8/15: dwrite d0/f1 [0,4194304] 0 2024-03-22T07:15:47.771 INFO:tasks.workunit.client.0.smithi063.stdout:8/16: creat d0/f4 x:0 0 0 2024-03-22T07:15:47.773 INFO:tasks.workunit.client.0.smithi063.stdout:9/13: dwrite f2 [0,4194304] 0 2024-03-22T07:15:47.773 INFO:tasks.workunit.client.0.smithi063.stdout:9/14: creat f3 x:0 0 0 2024-03-22T07:15:47.773 INFO:tasks.workunit.client.0.smithi063.stdout:9/15: rmdir - no directory 2024-03-22T07:15:47.781 INFO:tasks.workunit.client.0.smithi063.stdout:8/17: rename d0/f1 to d0/f5 0 2024-03-22T07:15:47.781 INFO:tasks.workunit.client.0.smithi063.stdout:8/18: fdatasync d0/f4 0 2024-03-22T07:15:47.781 INFO:tasks.workunit.client.0.smithi063.stdout:8/19: write d0/f4 [515239,15039] 0 2024-03-22T07:15:47.786 INFO:tasks.workunit.client.0.smithi063.stdout:0/31: rename f8 to f9 0 2024-03-22T07:15:47.787 INFO:tasks.workunit.client.0.smithi063.stdout:0/32: mkdir da 0 2024-03-22T07:15:47.789 INFO:tasks.workunit.client.0.smithi063.stdout:0/33: rename f2 to da/fb 0 2024-03-22T07:15:47.791 INFO:tasks.workunit.client.0.smithi063.stdout:5/14: dwrite f2 [0,4194304] 0 2024-03-22T07:15:47.791 INFO:tasks.workunit.client.0.smithi063.stdout:1/16: dwrite f2 [0,4194304] 0 2024-03-22T07:15:47.796 INFO:tasks.workunit.client.0.smithi063.stdout:1/17: getdents d4 0 2024-03-22T07:15:47.796 INFO:tasks.workunit.client.0.smithi063.stdout:1/18: stat d4 0 2024-03-22T07:15:47.796 INFO:tasks.workunit.client.0.smithi063.stdout:1/19: truncate f2 4926790 0 2024-03-22T07:15:47.797 INFO:tasks.workunit.client.0.smithi063.stdout:8/20: getdents d0 0 2024-03-22T07:15:47.798 INFO:tasks.workunit.client.0.smithi063.stdout:5/15: symlink l7 0 2024-03-22T07:15:47.799 INFO:tasks.workunit.client.0.smithi063.stdout:1/20: symlink d4/l6 0 2024-03-22T07:15:47.800 INFO:tasks.workunit.client.0.smithi063.stdout:8/21: mknod d0/c6 0 2024-03-22T07:15:47.801 INFO:tasks.workunit.client.0.smithi063.stdout:1/21: creat d4/f7 x:0 0 0 2024-03-22T07:15:47.803 INFO:tasks.workunit.client.0.smithi063.stdout:1/22: chown d4/l5 51 1 2024-03-22T07:15:47.805 INFO:tasks.workunit.client.0.smithi063.stdout:8/22: mknod d0/c7 0 2024-03-22T07:15:47.806 INFO:tasks.workunit.client.0.smithi063.stdout:1/23: creat d4/f8 x:0 0 0 2024-03-22T07:15:47.806 INFO:tasks.workunit.client.0.smithi063.stdout:1/24: stat f0 0 2024-03-22T07:15:47.806 INFO:tasks.workunit.client.0.smithi063.stdout:8/23: write d0/f4 [858204,13750] 0 2024-03-22T07:15:47.807 INFO:tasks.workunit.client.0.smithi063.stdout:8/24: mknod d0/c8 0 2024-03-22T07:15:47.808 INFO:tasks.workunit.client.0.smithi063.stdout:0/34: dwrite da/fb [0,4194304] 0 2024-03-22T07:15:47.809 INFO:tasks.workunit.client.0.smithi063.stdout:5/16: dread f2 [0,4194304] 0 2024-03-22T07:15:47.811 INFO:tasks.workunit.client.0.smithi063.stdout:0/35: creat da/fc x:0 0 0 2024-03-22T07:15:47.814 INFO:tasks.workunit.client.0.smithi063.stdout:0/36: rename da/fb to da/fd 0 2024-03-22T07:15:47.817 INFO:tasks.workunit.client.0.smithi063.stdout:1/25: write f2 [1283926,87967] 0 2024-03-22T07:15:47.820 INFO:tasks.workunit.client.0.smithi063.stdout:1/26: creat d4/f9 x:0 0 0 2024-03-22T07:15:47.822 INFO:tasks.workunit.client.0.smithi063.stdout:1/27: link d4/f8 d4/fa 0 2024-03-22T07:15:47.822 INFO:tasks.workunit.client.0.smithi063.stdout:1/28: readlink d4/l5 0 2024-03-22T07:15:47.823 INFO:tasks.workunit.client.0.smithi063.stdout:1/29: creat d4/fb x:0 0 0 2024-03-22T07:15:47.823 INFO:tasks.workunit.client.0.smithi063.stdout:1/30: dread - d4/f8 zero size 2024-03-22T07:15:47.825 INFO:tasks.workunit.client.0.smithi063.stdout:1/31: rename d4/l5 to d4/lc 0 2024-03-22T07:15:47.828 INFO:tasks.workunit.client.0.smithi063.stdout:8/25: dwrite d0/f4 [0,4194304] 0 2024-03-22T07:15:47.837 INFO:tasks.workunit.client.0.smithi063.stdout:1/32: dread f0 [0,4194304] 0 2024-03-22T07:15:47.838 INFO:tasks.workunit.client.0.smithi063.stdout:1/33: mknod d4/cd 0 2024-03-22T07:15:47.839 INFO:tasks.workunit.client.0.smithi063.stdout:1/34: symlink d4/le 0 2024-03-22T07:15:47.840 INFO:tasks.workunit.client.0.smithi063.stdout:8/26: read d0/f4 [665353,68062] 0 2024-03-22T07:15:47.840 INFO:tasks.workunit.client.0.smithi063.stdout:0/37: dwrite da/fc [0,4194304] 0 2024-03-22T07:15:47.841 INFO:tasks.workunit.client.0.smithi063.stdout:1/35: mkdir d4/df 0 2024-03-22T07:15:47.841 INFO:tasks.workunit.client.0.smithi063.stdout:8/27: rmdir d0 39 2024-03-22T07:15:47.843 INFO:tasks.workunit.client.0.smithi063.stdout:1/36: unlink d4/fa 0 2024-03-22T07:15:47.845 INFO:tasks.workunit.client.0.smithi063.stdout:1/37: mknod d4/df/c10 0 2024-03-22T07:15:47.846 INFO:tasks.workunit.client.0.smithi063.stdout:8/28: mknod d0/c9 0 2024-03-22T07:15:47.848 INFO:tasks.workunit.client.0.smithi063.stdout:1/38: rename d4/f7 to d4/df/f11 0 2024-03-22T07:15:47.852 INFO:tasks.workunit.client.0.smithi063.stdout:1/39: truncate f2 1984217 0 2024-03-22T07:15:47.853 INFO:tasks.workunit.client.0.smithi063.stdout:1/40: dread - d4/f8 zero size 2024-03-22T07:15:47.854 INFO:tasks.workunit.client.0.smithi063.stdout:0/38: dwrite f9 [0,4194304] 0 2024-03-22T07:15:47.858 INFO:tasks.workunit.client.0.smithi063.stdout:0/39: mkdir da/de 0 2024-03-22T07:15:47.859 INFO:tasks.workunit.client.0.smithi063.stdout:0/40: write f5 [794314,63054] 0 2024-03-22T07:15:47.860 INFO:tasks.workunit.client.0.smithi063.stdout:0/41: link f5 da/ff 0 2024-03-22T07:15:47.861 INFO:tasks.workunit.client.0.smithi063.stdout:0/42: stat da/fc 0 2024-03-22T07:15:47.861 INFO:tasks.workunit.client.0.smithi063.stdout:0/43: mknod da/c10 0 2024-03-22T07:15:47.863 INFO:tasks.workunit.client.0.smithi063.stdout:1/41: getdents d4 0 2024-03-22T07:15:47.864 INFO:tasks.workunit.client.0.smithi063.stdout:1/42: chown f2 276 1 2024-03-22T07:15:47.868 INFO:tasks.workunit.client.0.smithi063.stdout:8/29: dwrite d0/f4 [0,4194304] 0 2024-03-22T07:15:47.868 INFO:tasks.workunit.client.0.smithi063.stdout:8/30: fdatasync d0/f4 0 2024-03-22T07:15:47.870 INFO:tasks.workunit.client.0.smithi063.stdout:1/43: dread f0 [0,4194304] 0 2024-03-22T07:15:47.870 INFO:tasks.workunit.client.0.smithi063.stdout:1/44: chown f2 19656 1 2024-03-22T07:15:47.870 INFO:tasks.workunit.client.0.smithi063.stdout:8/31: rmdir d0 39 2024-03-22T07:15:47.870 INFO:tasks.workunit.client.0.smithi063.stdout:8/32: truncate d0/f4 4471658 0 2024-03-22T07:15:47.871 INFO:tasks.workunit.client.0.smithi063.stdout:1/45: creat d4/f12 x:0 0 0 2024-03-22T07:15:47.871 INFO:tasks.workunit.client.0.smithi063.stdout:1/46: write d4/df/f11 [633977,59856] 0 2024-03-22T07:15:47.872 INFO:tasks.workunit.client.0.smithi063.stdout:8/33: mknod d0/ca 0 2024-03-22T07:15:47.885 INFO:tasks.workunit.client.0.smithi063.stdout:8/34: write d0/f4 [2549962,56029] 0 2024-03-22T07:15:47.885 INFO:tasks.workunit.client.0.smithi063.stdout:0/44: dwrite da/fc [0,4194304] 0 2024-03-22T07:15:47.886 INFO:tasks.workunit.client.0.smithi063.stdout:8/35: mknod d0/cb 0 2024-03-22T07:15:47.887 INFO:tasks.workunit.client.0.smithi063.stdout:1/47: dwrite d4/f12 [0,4194304] 0 2024-03-22T07:15:47.887 INFO:tasks.workunit.client.0.smithi063.stdout:1/48: read f2 [1723376,38155] 0 2024-03-22T07:15:47.888 INFO:tasks.workunit.client.0.smithi063.stdout:1/49: chown d4/lc 3 1 2024-03-22T07:15:47.888 INFO:tasks.workunit.client.0.smithi063.stdout:8/36: creat d0/fc x:0 0 0 2024-03-22T07:15:47.890 INFO:tasks.workunit.client.0.smithi063.stdout:1/50: mkdir d4/df/d13 0 2024-03-22T07:15:47.890 INFO:tasks.workunit.client.0.smithi063.stdout:1/51: stat d4/le 0 2024-03-22T07:15:47.892 INFO:tasks.workunit.client.0.smithi063.stdout:8/37: creat d0/fd x:0 0 0 2024-03-22T07:15:47.893 INFO:tasks.workunit.client.0.smithi063.stdout:1/52: mkdir d4/d14 0 2024-03-22T07:15:47.895 INFO:tasks.workunit.client.0.smithi063.stdout:1/53: rename f0 to d4/df/f15 0 2024-03-22T07:15:47.895 INFO:tasks.workunit.client.0.smithi063.stdout:1/54: truncate d4/f9 164277 0 2024-03-22T07:15:47.896 INFO:tasks.workunit.client.0.smithi063.stdout:1/55: write d4/df/f11 [48397,54838] 0 2024-03-22T07:15:47.896 INFO:tasks.workunit.client.0.smithi063.stdout:1/56: write d4/fb [426173,38349] 0 2024-03-22T07:15:47.896 INFO:tasks.workunit.client.0.smithi063.stdout:1/57: read - d4/f8 zero size 2024-03-22T07:15:47.896 INFO:tasks.workunit.client.0.smithi063.stdout:1/58: truncate d4/fb 1338035 0 2024-03-22T07:15:47.896 INFO:tasks.workunit.client.0.smithi063.stdout:1/59: read f2 [525712,95807] 0 2024-03-22T07:15:47.896 INFO:tasks.workunit.client.0.smithi063.stdout:1/60: fdatasync d4/f9 0 2024-03-22T07:15:47.897 INFO:tasks.workunit.client.0.smithi063.stdout:1/61: chown d4/le 210 1 2024-03-22T07:15:47.897 INFO:tasks.workunit.client.0.smithi063.stdout:1/62: write d4/df/f15 [809574,9210] 0 2024-03-22T07:15:47.897 INFO:tasks.workunit.client.0.smithi063.stdout:1/63: symlink d4/l16 0 2024-03-22T07:15:47.897 INFO:tasks.workunit.client.0.smithi063.stdout:1/64: mknod d4/c17 0 2024-03-22T07:15:47.897 INFO:tasks.workunit.client.0.smithi063.stdout:1/65: dread - d4/f8 zero size 2024-03-22T07:15:47.903 INFO:tasks.workunit.client.0.smithi063.stdout:1/66: fsync d4/df/f15 0 2024-03-22T07:15:47.903 INFO:tasks.workunit.client.0.smithi063.stdout:1/67: readlink d4/le 0 2024-03-22T07:15:47.905 INFO:tasks.workunit.client.0.smithi063.stdout:8/38: dread d0/f5 [0,4194304] 0 2024-03-22T07:15:47.906 INFO:tasks.workunit.client.0.smithi063.stdout:8/39: mkdir d0/de 0 2024-03-22T07:15:47.907 INFO:tasks.workunit.client.0.smithi063.stdout:8/40: dread - d0/fc zero size 2024-03-22T07:15:47.907 INFO:tasks.workunit.client.0.smithi063.stdout:8/41: rmdir d0/de 0 2024-03-22T07:15:47.920 INFO:tasks.workunit.client.0.smithi063.stdout:1/68: dwrite d4/df/f11 [0,4194304] 0 2024-03-22T07:15:47.922 INFO:tasks.workunit.client.0.smithi063.stdout:1/69: creat d4/d14/f18 x:0 0 0 2024-03-22T07:15:48.806 INFO:tasks.workunit.client.1.smithi069.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-22T07:15:48.812 INFO:tasks.workunit.client.1.smithi069.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-22T07:15:48.812 INFO:tasks.workunit.client.1.smithi069.stderr:+ make 2024-03-22T07:15:49.052 INFO:tasks.workunit.client.1.smithi069.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-22T07:15:49.232 INFO:tasks.workunit.client.0.smithi063.stdout:3/27: sync 2024-03-22T07:15:49.232 INFO:tasks.workunit.client.0.smithi063.stdout:2/17: sync 2024-03-22T07:15:49.232 INFO:tasks.workunit.client.0.smithi063.stdout:6/16: sync 2024-03-22T07:15:49.232 INFO:tasks.workunit.client.0.smithi063.stdout:6/17: fdatasync f2 0 2024-03-22T07:15:49.232 INFO:tasks.workunit.client.0.smithi063.stdout:4/1: sync 2024-03-22T07:15:49.233 INFO:tasks.workunit.client.0.smithi063.stdout:9/16: sync 2024-03-22T07:15:49.233 INFO:tasks.workunit.client.0.smithi063.stdout:1/70: sync 2024-03-22T07:15:49.233 INFO:tasks.workunit.client.0.smithi063.stdout:8/42: sync 2024-03-22T07:15:49.233 INFO:tasks.workunit.client.0.smithi063.stdout:7/13: sync 2024-03-22T07:15:49.233 INFO:tasks.workunit.client.0.smithi063.stdout:5/17: sync 2024-03-22T07:15:49.233 INFO:tasks.workunit.client.0.smithi063.stdout:9/17: creat f4 x:0 0 0 2024-03-22T07:15:49.233 INFO:tasks.workunit.client.0.smithi063.stdout:0/45: sync 2024-03-22T07:15:49.233 INFO:tasks.workunit.client.0.smithi063.stdout:0/46: creat da/f11 x:0 0 0 2024-03-22T07:15:49.233 INFO:tasks.workunit.client.0.smithi063.stdout:3/28: mkdir da 0 2024-03-22T07:15:49.234 INFO:tasks.workunit.client.0.smithi063.stdout:6/18: mkdir d3 0 2024-03-22T07:15:49.234 INFO:tasks.workunit.client.0.smithi063.stdout:6/19: chown d3 600 1 2024-03-22T07:15:49.236 INFO:tasks.workunit.client.0.smithi063.stdout:1/71: rename d4/le to d4/df/l19 0 2024-03-22T07:15:49.236 INFO:tasks.workunit.client.0.smithi063.stdout:7/14: symlink d5/l6 0 2024-03-22T07:15:49.237 INFO:tasks.workunit.client.0.smithi063.stdout:7/15: dwrite - no filename 2024-03-22T07:15:49.237 INFO:tasks.workunit.client.0.smithi063.stdout:7/16: dread - no filename 2024-03-22T07:15:49.237 INFO:tasks.workunit.client.0.smithi063.stdout:7/17: dwrite - no filename 2024-03-22T07:15:49.237 INFO:tasks.workunit.client.0.smithi063.stdout:9/18: symlink l5 0 2024-03-22T07:15:49.237 INFO:tasks.workunit.client.0.smithi063.stdout:9/19: dread - f4 zero size 2024-03-22T07:15:49.239 INFO:tasks.workunit.client.0.smithi063.stdout:0/47: mkdir da/d12 0 2024-03-22T07:15:49.239 INFO:tasks.workunit.client.0.smithi063.stdout:0/48: truncate f5 1469169 0 2024-03-22T07:15:49.239 INFO:tasks.workunit.client.0.smithi063.stdout:0/49: truncate da/ff 2495378 0 2024-03-22T07:15:49.239 INFO:tasks.workunit.client.0.smithi063.stdout:0/50: fdatasync da/fc 0 2024-03-22T07:15:49.239 INFO:tasks.workunit.client.0.smithi063.stdout:0/51: creat da/f13 x:0 0 0 2024-03-22T07:15:49.240 INFO:tasks.workunit.client.0.smithi063.stdout:2/18: getdents d0 0 2024-03-22T07:15:49.240 INFO:tasks.workunit.client.0.smithi063.stdout:6/20: creat d3/f4 x:0 0 0 2024-03-22T07:15:49.240 INFO:tasks.workunit.client.0.smithi063.stdout:6/21: creat d3/f5 x:0 0 0 2024-03-22T07:15:49.240 INFO:tasks.workunit.client.0.smithi063.stdout:8/43: mknod d0/cf 0 2024-03-22T07:15:49.241 INFO:tasks.workunit.client.0.smithi063.stdout:3/29: rename f5 to da/fb 0 2024-03-22T07:15:49.243 INFO:tasks.workunit.client.0.smithi063.stdout:1/72: link d4/df/f11 d4/df/f1a 0 2024-03-22T07:15:49.245 INFO:tasks.workunit.client.0.smithi063.stdout:7/18: creat d5/f7 x:0 0 0 2024-03-22T07:15:49.245 INFO:tasks.workunit.client.0.smithi063.stdout:5/18: mkdir d8 0 2024-03-22T07:15:49.246 INFO:tasks.workunit.client.0.smithi063.stdout:9/20: read f2 [1260263,124582] 0 2024-03-22T07:15:49.246 INFO:tasks.workunit.client.0.smithi063.stdout:9/21: stat l5 0 2024-03-22T07:15:49.247 INFO:tasks.workunit.client.0.smithi063.stdout:0/52: mknod da/c14 0 2024-03-22T07:15:49.247 INFO:tasks.workunit.client.0.smithi063.stdout:0/53: dread - da/f13 zero size 2024-03-22T07:15:49.247 INFO:tasks.workunit.client.0.smithi063.stdout:0/54: chown da/ff 1878264 1 2024-03-22T07:15:49.248 INFO:tasks.workunit.client.0.smithi063.stdout:2/19: mknod d0/c6 0 2024-03-22T07:15:49.251 INFO:tasks.workunit.client.0.smithi063.stdout:6/22: unlink c1 0 2024-03-22T07:15:49.251 INFO:tasks.workunit.client.0.smithi063.stdout:6/23: write f2 [1963213,67607] 0 2024-03-22T07:15:49.253 INFO:tasks.workunit.client.0.smithi063.stdout:3/30: mknod da/cc 0 2024-03-22T07:15:49.253 INFO:tasks.workunit.client.0.smithi063.stdout:3/31: write da/fb [4926881,32172] 0 2024-03-22T07:15:49.253 INFO:tasks.workunit.client.0.smithi063.stdout:3/32: truncate f6 111705 0 2024-03-22T07:15:49.254 INFO:tasks.workunit.client.0.smithi063.stdout:5/19: unlink f2 0 2024-03-22T07:15:49.255 INFO:tasks.workunit.client.0.smithi063.stdout:5/20: read - no filename 2024-03-22T07:15:49.255 INFO:tasks.workunit.client.0.smithi063.stdout:5/21: fsync - no filename 2024-03-22T07:15:49.255 INFO:tasks.workunit.client.0.smithi063.stdout:5/22: chown l4 193668 1 2024-03-22T07:15:49.255 INFO:tasks.workunit.client.0.smithi063.stdout:5/23: dwrite - no filename 2024-03-22T07:15:49.255 INFO:tasks.workunit.client.0.smithi063.stdout:5/24: dwrite - no filename 2024-03-22T07:15:49.255 INFO:tasks.workunit.client.0.smithi063.stdout:5/25: dwrite - no filename 2024-03-22T07:15:49.256 INFO:tasks.workunit.client.0.smithi063.stdout:9/22: mknod c6 0 2024-03-22T07:15:49.256 INFO:tasks.workunit.client.0.smithi063.stdout:9/23: creat f7 x:0 0 0 2024-03-22T07:15:49.256 INFO:tasks.workunit.client.0.smithi063.stdout:9/24: dread - f7 zero size 2024-03-22T07:15:49.258 INFO:tasks.workunit.client.0.smithi063.stdout:8/44: symlink d0/l10 0 2024-03-22T07:15:49.258 INFO:tasks.workunit.client.0.smithi063.stdout:8/45: chown d0/cb 154329 1 2024-03-22T07:15:49.258 INFO:tasks.workunit.client.0.smithi063.stdout:8/46: write d0/fd [932353,44578] 0 2024-03-22T07:15:49.260 INFO:tasks.workunit.client.0.smithi063.stdout:6/24: dread f2 [0,4194304] 0 2024-03-22T07:15:49.260 INFO:tasks.workunit.client.0.smithi063.stdout:6/25: dread - d3/f5 zero size 2024-03-22T07:15:49.261 INFO:tasks.workunit.client.0.smithi063.stdout:5/26: link l4 d8/l9 0 2024-03-22T07:15:49.261 INFO:tasks.workunit.client.0.smithi063.stdout:5/27: dread - no filename 2024-03-22T07:15:49.261 INFO:tasks.workunit.client.0.smithi063.stdout:5/28: dwrite - no filename 2024-03-22T07:15:49.261 INFO:tasks.workunit.client.0.smithi063.stdout:5/29: stat d8 0 2024-03-22T07:15:49.263 INFO:tasks.workunit.client.0.smithi063.stdout:9/25: mkdir d8 0 2024-03-22T07:15:49.264 INFO:tasks.workunit.client.0.smithi063.stdout:2/20: symlink d0/l7 0 2024-03-22T07:15:49.266 INFO:tasks.workunit.client.0.smithi063.stdout:6/26: mkdir d3/d6 0 2024-03-22T07:15:49.266 INFO:tasks.workunit.client.0.smithi063.stdout:6/27: chown d3/f5 0 1 2024-03-22T07:15:49.266 INFO:tasks.workunit.client.0.smithi063.stdout:5/30: mknod d8/ca 0 2024-03-22T07:15:49.267 INFO:tasks.workunit.client.0.smithi063.stdout:9/26: creat d8/f9 x:0 0 0 2024-03-22T07:15:49.267 INFO:tasks.workunit.client.0.smithi063.stdout:9/27: dread - f4 zero size 2024-03-22T07:15:49.269 INFO:tasks.workunit.client.0.smithi063.stdout:2/21: creat d0/f8 x:0 0 0 2024-03-22T07:15:49.270 INFO:tasks.workunit.client.0.smithi063.stdout:8/47: rename d0/c8 to d0/c11 0 2024-03-22T07:15:49.272 INFO:tasks.workunit.client.0.smithi063.stdout:0/55: dread f9 [0,4194304] 0 2024-03-22T07:15:49.272 INFO:tasks.workunit.client.0.smithi063.stdout:0/56: chown f1 7897026 1 2024-03-22T07:15:49.273 INFO:tasks.workunit.client.0.smithi063.stdout:1/73: dwrite d4/df/f15 [0,4194304] 0 2024-03-22T07:15:49.273 INFO:tasks.workunit.client.0.smithi063.stdout:7/19: dwrite d5/f7 [0,4194304] 0 2024-03-22T07:15:49.273 INFO:tasks.workunit.client.0.smithi063.stdout:7/20: creat d5/f8 x:0 0 0 2024-03-22T07:15:49.273 INFO:tasks.workunit.client.0.smithi063.stdout:7/21: chown c3 16519885 1 2024-03-22T07:15:49.273 INFO:tasks.workunit.client.0.smithi063.stdout:7/22: creat d5/f9 x:0 0 0 2024-03-22T07:15:49.274 INFO:tasks.workunit.client.0.smithi063.stdout:5/31: link l4 d8/lb 0 2024-03-22T07:15:49.274 INFO:tasks.workunit.client.0.smithi063.stdout:2/22: write d0/f4 [2431950,34303] 0 2024-03-22T07:15:49.274 INFO:tasks.workunit.client.0.smithi063.stdout:2/23: dread - d0/f8 zero size 2024-03-22T07:15:49.278 INFO:tasks.workunit.client.0.smithi063.stdout:0/57: rename da/fd to da/de/f15 0 2024-03-22T07:15:49.278 INFO:tasks.workunit.client.0.smithi063.stdout:0/58: dread - da/f11 zero size 2024-03-22T07:15:49.278 INFO:tasks.workunit.client.0.smithi063.stdout:0/59: fdatasync da/f13 0 2024-03-22T07:15:49.281 INFO:tasks.workunit.client.0.smithi063.stdout:2/24: write d0/f4 [2427602,112134] 0 2024-03-22T07:15:49.281 INFO:tasks.workunit.client.0.smithi063.stdout:2/25: write d0/f4 [1880730,54967] 0 2024-03-22T07:15:49.281 INFO:tasks.workunit.client.0.smithi063.stdout:8/48: link d0/f5 d0/f12 0 2024-03-22T07:15:49.283 INFO:tasks.workunit.client.0.smithi063.stdout:1/74: rename d4/f8 to d4/d14/f1b 0 2024-03-22T07:15:49.284 INFO:tasks.workunit.client.0.smithi063.stdout:1/75: dread d4/f9 [0,4194304] 0 2024-03-22T07:15:49.284 INFO:tasks.workunit.client.0.smithi063.stdout:0/60: rename da/c14 to da/c16 0 2024-03-22T07:15:49.284 INFO:tasks.workunit.client.0.smithi063.stdout:0/61: chown da/d12 3641 1 2024-03-22T07:15:49.285 INFO:tasks.workunit.client.0.smithi063.stdout:0/62: write da/ff [1543285,19493] 0 2024-03-22T07:15:49.286 INFO:tasks.workunit.client.0.smithi063.stdout:5/32: rename l7 to d8/lc 0 2024-03-22T07:15:49.286 INFO:tasks.workunit.client.0.smithi063.stdout:5/33: dread - no filename 2024-03-22T07:15:49.286 INFO:tasks.workunit.client.0.smithi063.stdout:5/34: dread - no filename 2024-03-22T07:15:49.287 INFO:tasks.workunit.client.0.smithi063.stdout:7/23: write d5/f7 [1188129,51819] 0 2024-03-22T07:15:49.287 INFO:tasks.workunit.client.0.smithi063.stdout:1/76: symlink d4/df/l1c 0 2024-03-22T07:15:49.287 INFO:tasks.workunit.client.0.smithi063.stdout:1/77: creat d4/f1d x:0 0 0 2024-03-22T07:15:49.288 INFO:tasks.workunit.client.0.smithi063.stdout:6/28: dwrite d3/f4 [0,4194304] 0 2024-03-22T07:15:49.289 INFO:tasks.workunit.client.0.smithi063.stdout:8/49: rename d0/c6 to d0/c13 0 2024-03-22T07:15:49.289 INFO:tasks.workunit.client.0.smithi063.stdout:8/50: truncate d0/f12 4399234 0 2024-03-22T07:15:49.291 INFO:tasks.workunit.client.0.smithi063.stdout:0/63: rename da/d12 to da/de/d17 0 2024-03-22T07:15:49.293 INFO:tasks.workunit.client.0.smithi063.stdout:5/35: rename d8/ca to d8/cd 0 2024-03-22T07:15:49.293 INFO:tasks.workunit.client.0.smithi063.stdout:5/36: dwrite - no filename 2024-03-22T07:15:49.293 INFO:tasks.workunit.client.0.smithi063.stdout:5/37: write - no filename 2024-03-22T07:15:49.293 INFO:tasks.workunit.client.0.smithi063.stdout:5/38: chown c5 1149489 1 2024-03-22T07:15:49.293 INFO:tasks.workunit.client.0.smithi063.stdout:9/28: dwrite f4 [0,4194304] 0 2024-03-22T07:15:49.298 INFO:tasks.workunit.client.0.smithi063.stdout:2/26: link d0/f5 d0/f9 0 2024-03-22T07:15:49.298 INFO:tasks.workunit.client.0.smithi063.stdout:2/27: dread - d0/f5 zero size 2024-03-22T07:15:49.298 INFO:tasks.workunit.client.0.smithi063.stdout:2/28: chown d0/f8 429898049 1 2024-03-22T07:15:49.302 INFO:tasks.workunit.client.0.smithi063.stdout:9/29: dread f4 [0,4194304] 0 2024-03-22T07:15:49.302 INFO:tasks.workunit.client.0.smithi063.stdout:9/30: truncate f3 516323 0 2024-03-22T07:15:49.305 INFO:tasks.workunit.client.0.smithi063.stdout:5/39: creat d8/fe x:0 0 0 2024-03-22T07:15:49.306 INFO:tasks.workunit.client.0.smithi063.stdout:1/78: dwrite d4/fb [0,4194304] 0 2024-03-22T07:15:49.306 INFO:tasks.workunit.client.0.smithi063.stdout:1/79: getdents d4/df/d13 0 2024-03-22T07:15:49.306 INFO:tasks.workunit.client.0.smithi063.stdout:1/80: chown d4/df/c10 0 1 2024-03-22T07:15:49.308 INFO:tasks.workunit.client.0.smithi063.stdout:2/29: creat d0/fa x:0 0 0 2024-03-22T07:15:49.314 INFO:tasks.workunit.client.0.smithi063.stdout:1/81: dread d4/df/f11 [0,4194304] 0 2024-03-22T07:15:49.314 INFO:tasks.workunit.client.0.smithi063.stdout:1/82: chown d4/fb 15040540 1 2024-03-22T07:15:49.315 INFO:tasks.workunit.client.0.smithi063.stdout:2/30: creat d0/fb x:0 0 0 2024-03-22T07:15:49.315 INFO:tasks.workunit.client.0.smithi063.stdout:2/31: write d0/f5 [196579,104058] 0 2024-03-22T07:15:49.317 INFO:tasks.workunit.client.0.smithi063.stdout:2/32: creat d0/fc x:0 0 0 2024-03-22T07:15:49.317 INFO:tasks.workunit.client.0.smithi063.stdout:2/33: readlink d0/l7 0 2024-03-22T07:15:49.317 INFO:tasks.workunit.client.0.smithi063.stdout:2/34: dread - d0/fa zero size 2024-03-22T07:15:49.318 INFO:tasks.workunit.client.0.smithi063.stdout:6/29: truncate f2 250700 0 2024-03-22T07:15:49.321 INFO:tasks.workunit.client.0.smithi063.stdout:2/35: fsync d0/f8 0 2024-03-22T07:15:49.321 INFO:tasks.workunit.client.0.smithi063.stdout:2/36: write d0/fb [876565,81479] 0 2024-03-22T07:15:49.324 INFO:tasks.workunit.client.0.smithi063.stdout:0/64: rename da/de/f15 to da/de/f18 0 2024-03-22T07:15:49.324 INFO:tasks.workunit.client.0.smithi063.stdout:0/65: chown da/ff 3105 1 2024-03-22T07:15:49.325 INFO:tasks.workunit.client.0.smithi063.stdout:9/31: dwrite f2 [0,4194304] 0 2024-03-22T07:15:49.326 INFO:tasks.workunit.client.0.smithi063.stdout:7/24: getdents d5 0 2024-03-22T07:15:49.327 INFO:tasks.workunit.client.0.smithi063.stdout:7/25: truncate d5/f7 4771747 0 2024-03-22T07:15:49.327 INFO:tasks.workunit.client.0.smithi063.stdout:7/26: write d5/f7 [1281194,110441] 0 2024-03-22T07:15:49.327 INFO:tasks.workunit.client.0.smithi063.stdout:1/83: dread d4/df/f15 [0,4194304] 0 2024-03-22T07:15:49.327 INFO:tasks.workunit.client.0.smithi063.stdout:1/84: creat d4/f1e x:0 0 0 2024-03-22T07:15:49.329 INFO:tasks.workunit.client.0.smithi063.stdout:6/30: read d3/f4 [2309610,95681] 0 2024-03-22T07:15:49.332 INFO:tasks.workunit.client.0.smithi063.stdout:7/27: mkdir d5/da 0 2024-03-22T07:15:49.332 INFO:tasks.workunit.client.0.smithi063.stdout:7/28: stat d5/f7 0 2024-03-22T07:15:49.332 INFO:tasks.workunit.client.0.smithi063.stdout:7/29: rename d5 to d5/da/db 22 2024-03-22T07:15:49.332 INFO:tasks.workunit.client.0.smithi063.stdout:0/66: dread da/ff [0,4194304] 0 2024-03-22T07:15:49.333 INFO:tasks.workunit.client.0.smithi063.stdout:1/85: write f2 [2467053,71275] 0 2024-03-22T07:15:49.333 INFO:tasks.workunit.client.0.smithi063.stdout:1/86: creat d4/f1f x:0 0 0 2024-03-22T07:15:49.333 INFO:tasks.workunit.client.0.smithi063.stdout:1/87: chown d4/f1d 999 1 2024-03-22T07:15:49.334 INFO:tasks.workunit.client.0.smithi063.stdout:6/31: mknod d3/d6/c7 0 2024-03-22T07:15:49.335 INFO:tasks.workunit.client.0.smithi063.stdout:7/30: symlink d5/lc 0 2024-03-22T07:15:49.336 INFO:tasks.workunit.client.0.smithi063.stdout:7/31: rename d5 to d5/da/dd 22 2024-03-22T07:15:49.336 INFO:tasks.workunit.client.0.smithi063.stdout:0/67: mkdir da/de/d19 0 2024-03-22T07:15:49.337 INFO:tasks.workunit.client.0.smithi063.stdout:0/68: readlink l7 0 2024-03-22T07:15:49.338 INFO:tasks.workunit.client.0.smithi063.stdout:2/37: dwrite d0/fa [0,4194304] 0 2024-03-22T07:15:49.338 INFO:tasks.workunit.client.0.smithi063.stdout:2/38: write d0/f4 [1187107,38786] 0 2024-03-22T07:15:49.338 INFO:tasks.workunit.client.0.smithi063.stdout:0/69: dread f5 [0,4194304] 0 2024-03-22T07:15:49.339 INFO:tasks.workunit.client.0.smithi063.stdout:0/70: write f1 [1153146,93223] 0 2024-03-22T07:15:49.339 INFO:tasks.workunit.client.0.smithi063.stdout:0/71: write f5 [3302910,43301] 0 2024-03-22T07:15:49.339 INFO:tasks.workunit.client.0.smithi063.stdout:0/72: chown da/f11 0 1 2024-03-22T07:15:49.339 INFO:tasks.workunit.client.0.smithi063.stdout:1/88: symlink d4/df/d13/l20 0 2024-03-22T07:15:49.339 INFO:tasks.workunit.client.0.smithi063.stdout:1/89: write f2 [3453840,98611] 0 2024-03-22T07:15:49.340 INFO:tasks.workunit.client.0.smithi063.stdout:1/90: creat d4/d14/f21 x:0 0 0 2024-03-22T07:15:49.340 INFO:tasks.workunit.client.0.smithi063.stdout:2/39: fdatasync d0/f5 0 2024-03-22T07:15:49.341 INFO:tasks.workunit.client.0.smithi063.stdout:6/32: symlink d3/d6/l8 0 2024-03-22T07:15:49.341 INFO:tasks.workunit.client.0.smithi063.stdout:6/33: creat d3/f9 x:0 0 0 2024-03-22T07:15:49.341 INFO:tasks.workunit.client.0.smithi063.stdout:6/34: write d3/f5 [783485,18886] 0 2024-03-22T07:15:49.341 INFO:tasks.workunit.client.0.smithi063.stdout:7/32: symlink d5/le 0 2024-03-22T07:15:49.344 INFO:tasks.workunit.client.0.smithi063.stdout:0/73: creat da/de/d17/f1a x:0 0 0 2024-03-22T07:15:49.345 INFO:tasks.workunit.client.0.smithi063.stdout:5/40: getdents d8 0 2024-03-22T07:15:49.346 INFO:tasks.workunit.client.0.smithi063.stdout:1/91: truncate d4/f12 1965937 0 2024-03-22T07:15:49.348 INFO:tasks.workunit.client.0.smithi063.stdout:8/51: truncate d0/f12 1470881 0 2024-03-22T07:15:49.349 INFO:tasks.workunit.client.0.smithi063.stdout:8/52: write d0/fd [1085081,110942] 0 2024-03-22T07:15:49.350 INFO:tasks.workunit.client.0.smithi063.stdout:6/35: creat d3/d6/fa x:0 0 0 2024-03-22T07:15:49.350 INFO:tasks.workunit.client.0.smithi063.stdout:6/36: creat d3/d6/fb x:0 0 0 2024-03-22T07:15:49.350 INFO:tasks.workunit.client.0.smithi063.stdout:6/37: creat d3/d6/fc x:0 0 0 2024-03-22T07:15:49.350 INFO:tasks.workunit.client.0.smithi063.stdout:6/38: truncate d3/d6/fb 203987 0 2024-03-22T07:15:49.350 INFO:tasks.workunit.client.0.smithi063.stdout:7/33: creat d5/da/ff x:0 0 0 2024-03-22T07:15:49.352 INFO:tasks.workunit.client.0.smithi063.stdout:0/74: symlink da/de/d17/l1b 0 2024-03-22T07:15:49.352 INFO:tasks.workunit.client.0.smithi063.stdout:0/75: chown da/f11 446707 1 2024-03-22T07:15:49.352 INFO:tasks.workunit.client.0.smithi063.stdout:0/76: stat f1 0 2024-03-22T07:15:49.353 INFO:tasks.workunit.client.0.smithi063.stdout:5/41: symlink d8/lf 0 2024-03-22T07:15:49.355 INFO:tasks.workunit.client.0.smithi063.stdout:8/53: dread d0/fd [0,4194304] 0 2024-03-22T07:15:49.355 INFO:tasks.workunit.client.0.smithi063.stdout:1/92: symlink d4/df/d13/l22 0 2024-03-22T07:15:49.356 INFO:tasks.workunit.client.0.smithi063.stdout:1/93: creat d4/d14/f23 x:0 0 0 2024-03-22T07:15:49.356 INFO:tasks.workunit.client.0.smithi063.stdout:1/94: readlink d4/lc 0 2024-03-22T07:15:49.356 INFO:tasks.workunit.client.0.smithi063.stdout:1/95: rename d4 to d4/d14/d24 22 2024-03-22T07:15:49.358 INFO:tasks.workunit.client.0.smithi063.stdout:2/40: mknod d0/cd 0 2024-03-22T07:15:49.361 INFO:tasks.workunit.client.0.smithi063.stdout:9/32: truncate f2 1485588 0 2024-03-22T07:15:49.361 INFO:tasks.workunit.client.0.smithi063.stdout:9/33: chown c0 5325 1 2024-03-22T07:15:49.361 INFO:tasks.workunit.client.0.smithi063.stdout:9/34: truncate f3 969601 0 2024-03-22T07:15:49.361 INFO:tasks.workunit.client.0.smithi063.stdout:9/35: creat d8/fa x:0 0 0 2024-03-22T07:15:49.361 INFO:tasks.workunit.client.0.smithi063.stdout:9/36: dread - d8/fa zero size 2024-03-22T07:15:49.361 INFO:tasks.workunit.client.0.smithi063.stdout:9/37: creat d8/fb x:0 0 0 2024-03-22T07:15:49.361 INFO:tasks.workunit.client.0.smithi063.stdout:2/41: dread d0/f9 [0,4194304] 0 2024-03-22T07:15:49.362 INFO:tasks.workunit.client.0.smithi063.stdout:0/77: read da/fc [1486384,54392] 0 2024-03-22T07:15:49.371 INFO:tasks.workunit.client.0.smithi063.stdout:7/34: dwrite d5/da/ff [0,4194304] 0 2024-03-22T07:15:49.371 INFO:tasks.workunit.client.0.smithi063.stdout:7/35: stat d5/l6 0 2024-03-22T07:15:49.371 INFO:tasks.workunit.client.0.smithi063.stdout:9/38: rename f4 to d8/fc 0 2024-03-22T07:15:49.372 INFO:tasks.workunit.client.0.smithi063.stdout:7/36: rename d5 to d5/d10 22 2024-03-22T07:15:49.372 INFO:tasks.workunit.client.0.smithi063.stdout:7/37: chown c4 79522604 1 2024-03-22T07:15:49.372 INFO:tasks.workunit.client.0.smithi063.stdout:7/38: creat d5/da/f11 x:0 0 0 2024-03-22T07:15:49.372 INFO:tasks.workunit.client.0.smithi063.stdout:7/39: chown l1 179 1 2024-03-22T07:15:49.372 INFO:tasks.workunit.client.0.smithi063.stdout:9/39: write f7 [14944,116624] 0 2024-03-22T07:15:49.372 INFO:tasks.workunit.client.0.smithi063.stdout:9/40: dread - d8/fb zero size 2024-03-22T07:15:49.373 INFO:tasks.workunit.client.0.smithi063.stdout:9/41: creat d8/fd x:0 0 0 2024-03-22T07:15:49.373 INFO:tasks.workunit.client.0.smithi063.stdout:0/78: rmdir da/de/d19 0 2024-03-22T07:15:49.374 INFO:tasks.workunit.client.0.smithi063.stdout:8/54: mknod d0/c14 0 2024-03-22T07:15:49.374 INFO:tasks.workunit.client.0.smithi063.stdout:8/55: write d0/f4 [3908394,93756] 0 2024-03-22T07:15:49.380 INFO:tasks.workunit.client.0.smithi063.stdout:7/40: dread d5/da/ff [0,4194304] 0 2024-03-22T07:15:49.380 INFO:tasks.workunit.client.0.smithi063.stdout:1/96: dwrite d4/f1f [0,4194304] 0 2024-03-22T07:15:49.382 INFO:tasks.workunit.client.0.smithi063.stdout:0/79: dread f9 [0,4194304] 0 2024-03-22T07:15:49.383 INFO:tasks.workunit.client.0.smithi063.stdout:2/42: mkdir d0/de 0 2024-03-22T07:15:49.384 INFO:tasks.workunit.client.0.smithi063.stdout:7/41: dread d5/da/ff [0,4194304] 0 2024-03-22T07:15:49.384 INFO:tasks.workunit.client.0.smithi063.stdout:7/42: chown d5/l6 56706 1 2024-03-22T07:15:49.384 INFO:tasks.workunit.client.0.smithi063.stdout:7/43: write d5/f8 [693412,63691] 0 2024-03-22T07:15:49.388 INFO:tasks.workunit.client.0.smithi063.stdout:5/42: getdents d8 0 2024-03-22T07:15:49.388 INFO:tasks.workunit.client.0.smithi063.stdout:5/43: dread - d8/fe zero size 2024-03-22T07:15:49.391 INFO:tasks.workunit.client.0.smithi063.stdout:0/80: mknod da/de/c1c 0 2024-03-22T07:15:49.393 INFO:tasks.workunit.client.0.smithi063.stdout:7/44: mkdir d5/d12 0 2024-03-22T07:15:49.395 INFO:tasks.workunit.client.0.smithi063.stdout:5/44: creat d8/f10 x:0 0 0 2024-03-22T07:15:49.395 INFO:tasks.workunit.client.0.smithi063.stdout:0/81: dread da/fc [0,4194304] 0 2024-03-22T07:15:49.395 INFO:tasks.workunit.client.0.smithi063.stdout:0/82: dread - da/f11 zero size 2024-03-22T07:15:49.396 INFO:tasks.workunit.client.0.smithi063.stdout:7/45: fdatasync d5/f7 0 2024-03-22T07:15:49.398 INFO:tasks.workunit.client.0.smithi063.stdout:6/39: rmdir d3 39 2024-03-22T07:15:49.399 INFO:tasks.workunit.client.0.smithi063.stdout:1/97: dwrite d4/d14/f1b [0,4194304] 0 2024-03-22T07:15:49.401 INFO:tasks.workunit.client.0.smithi063.stdout:8/56: dwrite d0/fd [0,4194304] 0 2024-03-22T07:15:49.403 INFO:tasks.workunit.client.0.smithi063.stdout:0/83: rmdir da/de/d17 39 2024-03-22T07:15:49.403 INFO:tasks.workunit.client.0.smithi063.stdout:0/84: readlink da/de/d17/l1b 0 2024-03-22T07:15:49.408 INFO:tasks.workunit.client.0.smithi063.stdout:7/46: rmdir d5/d12 0 2024-03-22T07:15:49.409 INFO:tasks.workunit.client.0.smithi063.stdout:7/47: write d5/f7 [2109550,90953] 0 2024-03-22T07:15:49.410 INFO:tasks.workunit.client.0.smithi063.stdout:6/40: symlink d3/d6/ld 0 2024-03-22T07:15:49.411 INFO:tasks.workunit.client.0.smithi063.stdout:5/45: dwrite d8/f10 [0,4194304] 0 2024-03-22T07:15:49.413 INFO:tasks.workunit.client.0.smithi063.stdout:7/48: dread d5/f8 [0,4194304] 0 2024-03-22T07:15:49.413 INFO:tasks.workunit.client.0.smithi063.stdout:7/49: dread - d5/da/f11 zero size 2024-03-22T07:15:49.413 INFO:tasks.workunit.client.0.smithi063.stdout:1/98: creat d4/df/d13/f25 x:0 0 0 2024-03-22T07:15:49.414 INFO:tasks.workunit.client.0.smithi063.stdout:9/42: getdents d8 0 2024-03-22T07:15:49.417 INFO:tasks.workunit.client.0.smithi063.stdout:0/85: symlink da/de/l1d 0 2024-03-22T07:15:49.418 INFO:tasks.workunit.client.0.smithi063.stdout:0/86: chown da/fc 159073 1 2024-03-22T07:15:49.419 INFO:tasks.workunit.client.0.smithi063.stdout:6/41: creat d3/d6/fe x:0 0 0 2024-03-22T07:15:49.422 INFO:tasks.workunit.client.0.smithi063.stdout:0/87: dread f9 [0,4194304] 0 2024-03-22T07:15:49.422 INFO:tasks.workunit.client.0.smithi063.stdout:0/88: fdatasync da/de/d17/f1a 0 2024-03-22T07:15:49.422 INFO:tasks.workunit.client.0.smithi063.stdout:5/46: stat d8/lb 0 2024-03-22T07:15:49.425 INFO:tasks.workunit.client.0.smithi063.stdout:1/99: symlink d4/l26 0 2024-03-22T07:15:49.425 INFO:tasks.workunit.client.0.smithi063.stdout:1/100: readlink l3 0 2024-03-22T07:15:49.428 INFO:tasks.workunit.client.0.smithi063.stdout:8/57: symlink d0/l15 0 2024-03-22T07:15:49.429 INFO:tasks.workunit.client.0.smithi063.stdout:0/89: mknod da/c1e 0 2024-03-22T07:15:49.429 INFO:tasks.workunit.client.0.smithi063.stdout:0/90: truncate f9 4765756 0 2024-03-22T07:15:49.429 INFO:tasks.workunit.client.0.smithi063.stdout:0/91: fsync da/de/f18 0 2024-03-22T07:15:49.429 INFO:tasks.workunit.client.0.smithi063.stdout:5/47: stat d8/fe 0 2024-03-22T07:15:49.429 INFO:tasks.workunit.client.0.smithi063.stdout:5/48: fdatasync d8/fe 0 2024-03-22T07:15:49.432 INFO:tasks.workunit.client.0.smithi063.stdout:0/92: mknod da/de/d17/c1f 0 2024-03-22T07:15:49.434 INFO:tasks.workunit.client.0.smithi063.stdout:7/50: getdents d5 0 2024-03-22T07:15:49.434 INFO:tasks.workunit.client.0.smithi063.stdout:6/42: dwrite d3/f4 [0,4194304] 0 2024-03-22T07:15:49.436 INFO:tasks.workunit.client.0.smithi063.stdout:1/101: rmdir d4/d14 39 2024-03-22T07:15:49.436 INFO:tasks.workunit.client.0.smithi063.stdout:1/102: chown d4/f9 510 1 2024-03-22T07:15:49.436 INFO:tasks.workunit.client.0.smithi063.stdout:1/103: fdatasync d4/f1e 0 2024-03-22T07:15:49.436 INFO:tasks.workunit.client.0.smithi063.stdout:1/104: creat d4/df/d13/f27 x:0 0 0 2024-03-22T07:15:49.437 INFO:tasks.workunit.client.0.smithi063.stdout:8/58: symlink d0/l16 0 2024-03-22T07:15:49.439 INFO:tasks.workunit.client.0.smithi063.stdout:6/43: symlink d3/lf 0 2024-03-22T07:15:49.439 INFO:tasks.workunit.client.0.smithi063.stdout:6/44: chown f2 6521 1 2024-03-22T07:15:49.439 INFO:tasks.workunit.client.0.smithi063.stdout:6/45: stat d3/d6/l8 0 2024-03-22T07:15:49.442 INFO:tasks.workunit.client.0.smithi063.stdout:0/93: truncate f5 2652667 0 2024-03-22T07:15:49.444 INFO:tasks.workunit.client.0.smithi063.stdout:0/94: mkdir da/d20 0 2024-03-22T07:15:49.450 INFO:tasks.workunit.client.0.smithi063.stdout:5/49: dwrite d8/fe [0,4194304] 0 2024-03-22T07:15:49.454 INFO:tasks.workunit.client.0.smithi063.stdout:5/50: creat d8/f11 x:0 0 0 2024-03-22T07:15:49.456 INFO:tasks.workunit.client.0.smithi063.stdout:8/59: getdents d0 0 2024-03-22T07:15:49.457 INFO:tasks.workunit.client.0.smithi063.stdout:8/60: write d0/f4 [4730177,46614] 0 2024-03-22T07:15:49.458 INFO:tasks.workunit.client.0.smithi063.stdout:6/46: getdents d3 0 2024-03-22T07:15:49.459 INFO:tasks.workunit.client.0.smithi063.stdout:6/47: rename d3/d6/fe to d3/d6/f10 0 2024-03-22T07:15:49.462 INFO:tasks.workunit.client.0.smithi063.stdout:6/48: dread d3/f5 [0,4194304] 0 2024-03-22T07:15:49.468 INFO:tasks.workunit.client.0.smithi063.stdout:5/51: dread d8/f10 [0,4194304] 0 2024-03-22T07:15:49.468 INFO:tasks.workunit.client.0.smithi063.stdout:5/52: readlink l4 0 2024-03-22T07:15:49.468 INFO:tasks.workunit.client.0.smithi063.stdout:5/53: read - d8/f11 zero size 2024-03-22T07:15:49.468 INFO:tasks.workunit.client.0.smithi063.stdout:5/54: creat d8/f12 x:0 0 0 2024-03-22T07:15:49.469 INFO:tasks.workunit.client.0.smithi063.stdout:5/55: fdatasync d8/f10 0 2024-03-22T07:15:49.470 INFO:tasks.workunit.client.0.smithi063.stdout:5/56: mkdir d8/d13 0 2024-03-22T07:15:49.473 INFO:tasks.workunit.client.0.smithi063.stdout:0/95: dwrite da/de/f18 [0,4194304] 0 2024-03-22T07:15:49.473 INFO:tasks.workunit.client.0.smithi063.stdout:0/96: stat da/de/f18 0 2024-03-22T07:15:49.477 INFO:tasks.workunit.client.0.smithi063.stdout:0/97: rmdir da/d20 0 2024-03-22T07:15:49.477 INFO:tasks.workunit.client.0.smithi063.stdout:7/51: dwrite d5/da/ff [4194304,4194304] 0 2024-03-22T07:15:49.477 INFO:tasks.workunit.client.0.smithi063.stdout:7/52: chown d5/da/f11 6 1 2024-03-22T07:15:49.477 INFO:tasks.workunit.client.0.smithi063.stdout:7/53: getdents d5 0 2024-03-22T07:15:49.477 INFO:tasks.workunit.client.0.smithi063.stdout:1/105: dwrite d4/f1e [0,4194304] 0 2024-03-22T07:15:49.477 INFO:tasks.workunit.client.0.smithi063.stdout:1/106: chown d4/cd 729681662 1 2024-03-22T07:15:49.479 INFO:tasks.workunit.client.0.smithi063.stdout:7/54: rename d5/f8 to d5/f13 0 2024-03-22T07:15:49.483 INFO:tasks.workunit.client.0.smithi063.stdout:1/107: rename d4/c17 to d4/df/d13/c28 0 2024-03-22T07:15:49.484 INFO:tasks.workunit.client.0.smithi063.stdout:8/61: dread d0/f4 [0,4194304] 0 2024-03-22T07:15:49.484 INFO:tasks.workunit.client.0.smithi063.stdout:7/55: mkdir d5/d14 0 2024-03-22T07:15:49.484 INFO:tasks.workunit.client.0.smithi063.stdout:7/56: chown l1 15 1 2024-03-22T07:15:49.485 INFO:tasks.workunit.client.0.smithi063.stdout:8/62: creat d0/f17 x:0 0 0 2024-03-22T07:15:49.485 INFO:tasks.workunit.client.0.smithi063.stdout:8/63: dread - d0/fc zero size 2024-03-22T07:15:49.487 INFO:tasks.workunit.client.0.smithi063.stdout:0/98: dread da/fc [0,4194304] 0 2024-03-22T07:15:49.487 INFO:tasks.workunit.client.0.smithi063.stdout:6/49: dwrite d3/f9 [0,4194304] 0 2024-03-22T07:15:49.487 INFO:tasks.workunit.client.0.smithi063.stdout:0/99: fsync da/de/f18 0 2024-03-22T07:15:49.487 INFO:tasks.workunit.client.0.smithi063.stdout:0/100: write da/fc [4837091,110727] 0 2024-03-22T07:15:49.487 INFO:tasks.workunit.client.0.smithi063.stdout:0/101: stat da/c10 0 2024-03-22T07:15:49.487 INFO:tasks.workunit.client.0.smithi063.stdout:0/102: write da/f11 [839718,48790] 0 2024-03-22T07:15:49.488 INFO:tasks.workunit.client.0.smithi063.stdout:7/57: rmdir d5/da 39 2024-03-22T07:15:49.488 INFO:tasks.workunit.client.0.smithi063.stdout:7/58: readlink l1 0 2024-03-22T07:15:49.490 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:49 smithi069 ceph-mon[36441]: pgmap v247: 65 pgs: 65 active+clean; 185 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 3.8 MiB/s rd, 8.0 MiB/s wr, 804 op/s 2024-03-22T07:15:49.491 INFO:tasks.workunit.client.0.smithi063.stdout:6/50: link d3/d6/fc d3/d6/f11 0 2024-03-22T07:15:49.493 INFO:tasks.workunit.client.0.smithi063.stdout:0/103: getdents da/de/d17 0 2024-03-22T07:15:49.494 INFO:tasks.workunit.client.0.smithi063.stdout:7/59: link d5/le d5/l15 0 2024-03-22T07:15:49.496 INFO:tasks.workunit.client.0.smithi063.stdout:6/51: dread f2 [0,4194304] 0 2024-03-22T07:15:49.496 INFO:tasks.workunit.client.0.smithi063.stdout:6/52: dread - d3/d6/fc zero size 2024-03-22T07:15:49.496 INFO:tasks.workunit.client.0.smithi063.stdout:7/60: rmdir d5 39 2024-03-22T07:15:49.498 INFO:tasks.workunit.client.0.smithi063.stdout:6/53: mknod d3/c12 0 2024-03-22T07:15:49.498 INFO:tasks.workunit.client.0.smithi063.stdout:1/108: write d4/f1e [1848228,63683] 0 2024-03-22T07:15:49.499 INFO:tasks.workunit.client.0.smithi063.stdout:5/57: dwrite d8/f10 [0,4194304] 0 2024-03-22T07:15:49.499 INFO:tasks.workunit.client.0.smithi063.stdout:1/109: chown d4/df 161413897 1 2024-03-22T07:15:49.500 INFO:tasks.workunit.client.0.smithi063.stdout:6/54: symlink d3/d6/l13 0 2024-03-22T07:15:49.503 INFO:tasks.workunit.client.0.smithi063.stdout:5/58: unlink d8/f11 0 2024-03-22T07:15:49.503 INFO:tasks.workunit.client.0.smithi063.stdout:5/59: readlink d8/lb 0 2024-03-22T07:15:49.505 INFO:tasks.workunit.client.0.smithi063.stdout:1/110: link f2 d4/df/f29 0 2024-03-22T07:15:49.514 INFO:tasks.workunit.client.0.smithi063.stdout:6/55: read d3/f9 [3954571,68865] 0 2024-03-22T07:15:49.514 INFO:tasks.workunit.client.0.smithi063.stdout:6/56: readlink d3/d6/ld 0 2024-03-22T07:15:49.514 INFO:tasks.workunit.client.0.smithi063.stdout:1/111: write d4/f12 [374000,48292] 0 2024-03-22T07:15:49.515 INFO:tasks.workunit.client.0.smithi063.stdout:6/57: rename d3/d6/fb to d3/f14 0 2024-03-22T07:15:49.515 INFO:tasks.workunit.client.0.smithi063.stdout:8/64: dwrite d0/f5 [0,4194304] 0 2024-03-22T07:15:49.516 INFO:tasks.workunit.client.0.smithi063.stdout:6/58: mkdir d3/d15 0 2024-03-22T07:15:49.516 INFO:tasks.workunit.client.0.smithi063.stdout:5/60: read d8/f10 [1420703,23555] 0 2024-03-22T07:15:49.517 INFO:tasks.workunit.client.0.smithi063.stdout:8/65: mkdir d0/d18 0 2024-03-22T07:15:49.517 INFO:tasks.workunit.client.0.smithi063.stdout:8/66: readlink d0/l15 0 2024-03-22T07:15:49.517 INFO:tasks.workunit.client.0.smithi063.stdout:8/67: write d0/f17 [637122,16403] 0 2024-03-22T07:15:49.517 INFO:tasks.workunit.client.0.smithi063.stdout:8/68: fsync d0/fc 0 2024-03-22T07:15:49.517 INFO:tasks.workunit.client.0.smithi063.stdout:6/59: mkdir d3/d15/d16 0 2024-03-22T07:15:49.518 INFO:tasks.workunit.client.0.smithi063.stdout:6/60: creat d3/d15/f17 x:0 0 0 2024-03-22T07:15:49.518 INFO:tasks.workunit.client.0.smithi063.stdout:6/61: read d3/f9 [3864257,31444] 0 2024-03-22T07:15:49.522 INFO:tasks.workunit.client.0.smithi063.stdout:6/62: dread f2 [0,4194304] 0 2024-03-22T07:15:49.522 INFO:tasks.workunit.client.0.smithi063.stdout:6/63: fdatasync d3/f4 0 2024-03-22T07:15:49.522 INFO:tasks.workunit.client.0.smithi063.stdout:6/64: dread - d3/d6/f10 zero size 2024-03-22T07:15:49.523 INFO:tasks.workunit.client.0.smithi063.stdout:7/61: dwrite d5/f13 [0,4194304] 0 2024-03-22T07:15:49.523 INFO:tasks.workunit.client.0.smithi063.stdout:6/65: link d3/d6/f11 d3/f18 0 2024-03-22T07:15:49.525 INFO:tasks.workunit.client.0.smithi063.stdout:7/62: mkdir d5/d16 0 2024-03-22T07:15:49.536 INFO:tasks.workunit.client.0.smithi063.stdout:0/104: dwrite da/f11 [0,4194304] 0 2024-03-22T07:15:49.536 INFO:tasks.workunit.client.0.smithi063.stdout:8/69: write d0/f12 [2923944,39040] 0 2024-03-22T07:15:49.536 INFO:tasks.workunit.client.0.smithi063.stdout:5/61: dwrite d8/f12 [0,4194304] 0 2024-03-22T07:15:49.536 INFO:tasks.workunit.client.0.smithi063.stdout:8/70: creat d0/f19 x:0 0 0 2024-03-22T07:15:49.536 INFO:tasks.workunit.client.0.smithi063.stdout:8/71: creat d0/f1a x:0 0 0 2024-03-22T07:15:49.539 INFO:tasks.workunit.client.0.smithi063.stdout:0/105: dread da/ff [0,4194304] 0 2024-03-22T07:15:49.539 INFO:tasks.workunit.client.0.smithi063.stdout:1/112: fsync d4/df/f29 0 2024-03-22T07:15:49.539 INFO:tasks.workunit.client.0.smithi063.stdout:1/113: dread - d4/f1d zero size 2024-03-22T07:15:49.540 INFO:tasks.workunit.client.0.smithi063.stdout:5/62: creat d8/f14 x:0 0 0 2024-03-22T07:15:49.540 INFO:tasks.workunit.client.0.smithi063.stdout:5/63: write d8/f10 [1192383,59239] 0 2024-03-22T07:15:49.541 INFO:tasks.workunit.client.0.smithi063.stdout:6/66: dwrite f2 [0,4194304] 0 2024-03-22T07:15:49.544 INFO:tasks.workunit.client.0.smithi063.stdout:8/72: mkdir d0/d1b 0 2024-03-22T07:15:49.546 INFO:tasks.workunit.client.0.smithi063.stdout:1/114: creat d4/d14/f2a x:0 0 0 2024-03-22T07:15:49.550 INFO:tasks.workunit.client.0.smithi063.stdout:0/106: dread da/f11 [0,4194304] 0 2024-03-22T07:15:49.551 INFO:tasks.workunit.client.0.smithi063.stdout:5/64: write d8/fe [824475,42325] 0 2024-03-22T07:15:49.552 INFO:tasks.workunit.client.0.smithi063.stdout:7/63: write d5/f13 [4926537,116486] 0 2024-03-22T07:15:49.553 INFO:tasks.workunit.client.0.smithi063.stdout:1/115: truncate f2 2222385 0 2024-03-22T07:15:49.556 INFO:tasks.workunit.client.0.smithi063.stdout:0/107: creat da/de/d17/f21 x:0 0 0 2024-03-22T07:15:49.559 INFO:tasks.workunit.client.0.smithi063.stdout:5/65: stat d8/cd 0 2024-03-22T07:15:49.566 INFO:tasks.workunit.client.0.smithi063.stdout:7/64: creat d5/f17 x:0 0 0 2024-03-22T07:15:49.567 INFO:tasks.workunit.client.0.smithi063.stdout:6/67: rmdir d3/d6 39 2024-03-22T07:15:49.574 INFO:tasks.workunit.client.0.smithi063.stdout:5/66: dread d8/f12 [0,4194304] 0 2024-03-22T07:15:49.575 INFO:tasks.workunit.client.0.smithi063.stdout:5/67: mknod d8/c15 0 2024-03-22T07:15:49.576 INFO:tasks.workunit.client.0.smithi063.stdout:5/68: creat d8/f16 x:0 0 0 2024-03-22T07:15:49.576 INFO:tasks.workunit.client.0.smithi063.stdout:5/69: dread - d8/f16 zero size 2024-03-22T07:15:49.576 INFO:tasks.workunit.client.0.smithi063.stdout:5/70: mknod d8/d13/c17 0 2024-03-22T07:15:49.580 INFO:tasks.workunit.client.0.smithi063.stdout:8/73: dwrite d0/f5 [0,4194304] 0 2024-03-22T07:15:49.582 INFO:tasks.workunit.client.0.smithi063.stdout:1/116: dwrite d4/f1d [0,4194304] 0 2024-03-22T07:15:49.583 INFO:tasks.workunit.client.0.smithi063.stdout:8/74: mknod d0/d18/c1c 0 2024-03-22T07:15:49.583 INFO:tasks.workunit.client.0.smithi063.stdout:8/75: fsync d0/f17 0 2024-03-22T07:15:49.583 INFO:tasks.workunit.client.0.smithi063.stdout:8/76: chown d0/c7 1 1 2024-03-22T07:15:49.585 INFO:tasks.workunit.client.0.smithi063.stdout:0/108: dwrite f3 [0,4194304] 0 2024-03-22T07:15:49.587 INFO:tasks.workunit.client.0.smithi063.stdout:8/77: mknod d0/d18/c1d 0 2024-03-22T07:15:49.587 INFO:tasks.workunit.client.0.smithi063.stdout:8/78: write d0/fc [148086,52619] 0 2024-03-22T07:15:49.589 INFO:tasks.workunit.client.0.smithi063.stdout:6/68: dwrite f2 [4194304,4194304] 0 2024-03-22T07:15:49.590 INFO:tasks.workunit.client.0.smithi063.stdout:8/79: fdatasync d0/fc 0 2024-03-22T07:15:49.593 INFO:tasks.workunit.client.0.smithi063.stdout:6/69: link d3/f5 d3/f19 0 2024-03-22T07:15:49.599 INFO:tasks.workunit.client.0.smithi063.stdout:8/80: rmdir d0 39 2024-03-22T07:15:49.599 INFO:tasks.workunit.client.0.smithi063.stdout:8/81: write d0/f1a [167575,130998] 0 2024-03-22T07:15:49.599 INFO:tasks.workunit.client.0.smithi063.stdout:8/82: write d0/f4 [2375829,78679] 0 2024-03-22T07:15:49.599 INFO:tasks.workunit.client.0.smithi063.stdout:8/83: truncate d0/f1a 556944 0 2024-03-22T07:15:49.601 INFO:tasks.workunit.client.0.smithi063.stdout:8/84: creat d0/d1b/f1e x:0 0 0 2024-03-22T07:15:49.602 INFO:tasks.workunit.client.0.smithi063.stdout:8/85: dread - d0/f19 zero size 2024-03-22T07:15:49.602 INFO:tasks.workunit.client.0.smithi063.stdout:8/86: rmdir d0 39 2024-03-22T07:15:49.604 INFO:tasks.workunit.client.0.smithi063.stdout:7/65: dwrite d5/f13 [4194304,4194304] 0 2024-03-22T07:15:49.604 INFO:tasks.workunit.client.0.smithi063.stdout:7/66: creat d5/f18 x:0 0 0 2024-03-22T07:15:49.612 INFO:tasks.workunit.client.0.smithi063.stdout:8/87: dread d0/fd [0,4194304] 0 2024-03-22T07:15:49.612 INFO:tasks.workunit.client.0.smithi063.stdout:8/88: mkdir d0/d1f 0 2024-03-22T07:15:49.614 INFO:tasks.workunit.client.0.smithi063.stdout:8/89: dread d0/f17 [0,4194304] 0 2024-03-22T07:15:49.614 INFO:tasks.workunit.client.0.smithi063.stdout:8/90: creat d0/d1b/f20 x:0 0 0 2024-03-22T07:15:49.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:49 smithi063 ceph-mon[29200]: pgmap v247: 65 pgs: 65 active+clean; 185 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 3.8 MiB/s rd, 8.0 MiB/s wr, 804 op/s 2024-03-22T07:15:49.629 INFO:tasks.workunit.client.1.smithi069.stderr:++ readlink -f fsstress 2024-03-22T07:15:49.629 INFO:tasks.workunit.client.1.smithi069.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-22T07:15:49.629 INFO:tasks.workunit.client.1.smithi069.stderr:+ popd 2024-03-22T07:15:49.630 INFO:tasks.workunit.client.1.smithi069.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-22T07:15:49.630 INFO:tasks.workunit.client.1.smithi069.stderr:+ popd 2024-03-22T07:15:49.631 INFO:tasks.workunit.client.1.smithi069.stdout:~/cephtest/mnt.1/client.1/tmp 2024-03-22T07:15:49.631 INFO:tasks.workunit.client.1.smithi069.stderr:++ mktemp -d -p . 2024-03-22T07:15:49.632 INFO:tasks.workunit.client.1.smithi069.stderr:+ T=./tmp.XQHUhCuUpu 2024-03-22T07:15:49.632 INFO:tasks.workunit.client.1.smithi069.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.XQHUhCuUpu -l 1 -n 1000 -p 10 -v 2024-03-22T07:15:49.633 INFO:tasks.workunit.client.1.smithi069.stdout:seed = 1710530355 2024-03-22T07:15:49.635 INFO:tasks.workunit.client.1.smithi069.stdout:0/0: link - no file 2024-03-22T07:15:49.635 INFO:tasks.workunit.client.1.smithi069.stdout:0/1: stat - no entries 2024-03-22T07:15:49.635 INFO:tasks.workunit.client.1.smithi069.stdout:0/2: truncate - no filename 2024-03-22T07:15:49.635 INFO:tasks.workunit.client.1.smithi069.stdout:0/3: chown . 2811 1 2024-03-22T07:15:49.635 INFO:tasks.workunit.client.1.smithi069.stdout:1/0: chown . 224108 1 2024-03-22T07:15:49.635 INFO:tasks.workunit.client.1.smithi069.stdout:1/1: unlink - no file 2024-03-22T07:15:49.635 INFO:tasks.workunit.client.1.smithi069.stdout:1/2: truncate - no filename 2024-03-22T07:15:49.635 INFO:tasks.workunit.client.1.smithi069.stdout:1/3: dread - no filename 2024-03-22T07:15:49.636 INFO:tasks.workunit.client.1.smithi069.stdout:0/4: symlink l0 0 2024-03-22T07:15:49.636 INFO:tasks.workunit.client.1.smithi069.stdout:2/0: stat - no entries 2024-03-22T07:15:49.636 INFO:tasks.workunit.client.1.smithi069.stdout:2/1: dwrite - no filename 2024-03-22T07:15:49.637 INFO:tasks.workunit.client.1.smithi069.stdout:2/2: dread - no filename 2024-03-22T07:15:49.637 INFO:tasks.workunit.client.1.smithi069.stdout:2/3: readlink - no filename 2024-03-22T07:15:49.637 INFO:tasks.workunit.client.1.smithi069.stdout:1/4: mkdir d0 0 2024-03-22T07:15:49.637 INFO:tasks.workunit.client.1.smithi069.stdout:1/5: write - no filename 2024-03-22T07:15:49.638 INFO:tasks.workunit.client.1.smithi069.stdout:1/6: dwrite - no filename 2024-03-22T07:15:49.638 INFO:tasks.workunit.client.1.smithi069.stdout:1/7: truncate - no filename 2024-03-22T07:15:49.638 INFO:tasks.workunit.client.1.smithi069.stdout:1/8: write - no filename 2024-03-22T07:15:49.638 INFO:tasks.workunit.client.1.smithi069.stdout:1/9: fdatasync - no filename 2024-03-22T07:15:49.638 INFO:tasks.workunit.client.1.smithi069.stdout:0/5: mkdir d1 0 2024-03-22T07:15:49.638 INFO:tasks.workunit.client.1.smithi069.stdout:3/0: fsync - no filename 2024-03-22T07:15:49.639 INFO:tasks.workunit.client.1.smithi069.stdout:3/1: dwrite - no filename 2024-03-22T07:15:49.639 INFO:tasks.workunit.client.1.smithi069.stdout:3/2: dread - no filename 2024-03-22T07:15:49.639 INFO:tasks.workunit.client.1.smithi069.stdout:2/4: symlink l0 0 2024-03-22T07:15:49.639 INFO:tasks.workunit.client.1.smithi069.stdout:2/5: write - no filename 2024-03-22T07:15:49.639 INFO:tasks.workunit.client.1.smithi069.stdout:2/6: chown l0 3 1 2024-03-22T07:15:49.639 INFO:tasks.workunit.client.1.smithi069.stdout:2/7: dwrite - no filename 2024-03-22T07:15:49.639 INFO:tasks.workunit.client.1.smithi069.stdout:1/10: rmdir d0 0 2024-03-22T07:15:49.639 INFO:tasks.workunit.client.1.smithi069.stdout:1/11: dread - no filename 2024-03-22T07:15:49.639 INFO:tasks.workunit.client.1.smithi069.stdout:1/12: fdatasync - no filename 2024-03-22T07:15:49.640 INFO:tasks.workunit.client.1.smithi069.stdout:0/6: unlink l0 0 2024-03-22T07:15:49.640 INFO:tasks.workunit.client.1.smithi069.stdout:0/7: link - no file 2024-03-22T07:15:49.641 INFO:tasks.workunit.client.1.smithi069.stdout:0/8: write - no filename 2024-03-22T07:15:49.641 INFO:tasks.workunit.client.1.smithi069.stdout:0/9: dread - no filename 2024-03-22T07:15:49.641 INFO:tasks.workunit.client.1.smithi069.stdout:4/0: fsync - no filename 2024-03-22T07:15:49.641 INFO:tasks.workunit.client.1.smithi069.stdout:4/1: fdatasync - no filename 2024-03-22T07:15:49.641 INFO:tasks.workunit.client.1.smithi069.stdout:4/2: dread - no filename 2024-03-22T07:15:49.642 INFO:tasks.workunit.client.1.smithi069.stdout:3/3: symlink l0 0 2024-03-22T07:15:49.642 INFO:tasks.workunit.client.1.smithi069.stdout:3/4: stat l0 0 2024-03-22T07:15:49.642 INFO:tasks.workunit.client.1.smithi069.stdout:3/5: truncate - no filename 2024-03-22T07:15:49.642 INFO:tasks.workunit.client.1.smithi069.stdout:3/6: dwrite - no filename 2024-03-22T07:15:49.643 INFO:tasks.workunit.client.1.smithi069.stdout:2/8: creat f1 x:0 0 0 2024-03-22T07:15:49.644 INFO:tasks.workunit.client.1.smithi069.stdout:2/9: creat f2 x:0 0 0 2024-03-22T07:15:49.644 INFO:tasks.workunit.client.1.smithi069.stdout:2/10: creat f3 x:0 0 0 2024-03-22T07:15:49.645 INFO:tasks.workunit.client.1.smithi069.stdout:0/10: rmdir d1 0 2024-03-22T07:15:49.645 INFO:tasks.workunit.client.1.smithi069.stdout:0/11: read - no filename 2024-03-22T07:15:49.645 INFO:tasks.workunit.client.1.smithi069.stdout:0/12: chown . 4612036 1 2024-03-22T07:15:49.646 INFO:tasks.workunit.client.1.smithi069.stdout:5/0: read - no filename 2024-03-22T07:15:49.646 INFO:tasks.workunit.client.1.smithi069.stdout:5/1: dread - no filename 2024-03-22T07:15:49.647 INFO:tasks.workunit.client.1.smithi069.stdout:4/3: mknod c0 0 2024-03-22T07:15:49.647 INFO:tasks.workunit.client.1.smithi069.stdout:4/4: dwrite - no filename 2024-03-22T07:15:49.648 INFO:tasks.workunit.client.1.smithi069.stdout:3/7: unlink l0 0 2024-03-22T07:15:49.648 INFO:tasks.workunit.client.1.smithi069.stdout:3/8: dwrite - no filename 2024-03-22T07:15:49.652 INFO:tasks.workunit.client.1.smithi069.stdout:4/5: rename c0 to c1 0 2024-03-22T07:15:49.654 INFO:tasks.workunit.client.1.smithi069.stdout:3/9: mknod c1 0 2024-03-22T07:15:49.655 INFO:tasks.workunit.client.1.smithi069.stdout:4/6: symlink l2 0 2024-03-22T07:15:49.655 INFO:tasks.workunit.client.1.smithi069.stdout:4/7: truncate - no filename 2024-03-22T07:15:49.659 INFO:tasks.workunit.client.1.smithi069.stdout:3/10: creat f2 x:0 0 0 2024-03-22T07:15:49.666 INFO:tasks.workunit.client.1.smithi069.stdout:3/11: symlink l3 0 2024-03-22T07:15:49.668 INFO:tasks.workunit.client.1.smithi069.stdout:3/12: symlink l4 0 2024-03-22T07:15:49.668 INFO:tasks.workunit.client.1.smithi069.stdout:3/13: dread - f2 zero size 2024-03-22T07:15:49.669 INFO:tasks.workunit.client.1.smithi069.stdout:3/14: mkdir d5 0 2024-03-22T07:15:49.672 INFO:tasks.workunit.client.1.smithi069.stdout:3/15: creat d5/f6 x:0 0 0 2024-03-22T07:15:49.672 INFO:tasks.workunit.client.1.smithi069.stdout:3/16: write d5/f6 [951970,77096] 0 2024-03-22T07:15:49.804 INFO:tasks.workunit.client.1.smithi069.stdout:4/8: getdents . 0 2024-03-22T07:15:49.804 INFO:tasks.workunit.client.1.smithi069.stdout:4/9: readlink l2 0 2024-03-22T07:15:49.804 INFO:tasks.workunit.client.1.smithi069.stdout:4/10: write - no filename 2024-03-22T07:15:49.806 INFO:tasks.workunit.client.1.smithi069.stdout:3/17: fsync f2 0 2024-03-22T07:15:49.842 INFO:tasks.workunit.client.1.smithi069.stdout:2/11: dwrite f3 [0,4194304] 0 2024-03-22T07:15:49.931 INFO:tasks.workunit.client.1.smithi069.stdout:4/11: creat f3 x:0 0 0 2024-03-22T07:15:49.931 INFO:tasks.workunit.client.1.smithi069.stdout:4/12: rmdir - no directory 2024-03-22T07:15:49.954 INFO:tasks.workunit.client.1.smithi069.stdout:6/0: dread - no filename 2024-03-22T07:15:49.954 INFO:tasks.workunit.client.1.smithi069.stdout:7/0: read - no filename 2024-03-22T07:15:49.954 INFO:tasks.workunit.client.1.smithi069.stdout:7/1: unlink - no file 2024-03-22T07:15:49.954 INFO:tasks.workunit.client.1.smithi069.stdout:7/2: unlink - no file 2024-03-22T07:15:49.954 INFO:tasks.workunit.client.1.smithi069.stdout:6/1: write - no filename 2024-03-22T07:15:49.954 INFO:tasks.workunit.client.1.smithi069.stdout:7/3: rename - no filename 2024-03-22T07:15:49.954 INFO:tasks.workunit.client.1.smithi069.stdout:6/2: truncate - no filename 2024-03-22T07:15:49.955 INFO:tasks.workunit.client.1.smithi069.stdout:7/4: dwrite - no filename 2024-03-22T07:15:49.955 INFO:tasks.workunit.client.1.smithi069.stdout:7/5: truncate - no filename 2024-03-22T07:15:49.955 INFO:tasks.workunit.client.1.smithi069.stdout:7/6: dwrite - no filename 2024-03-22T07:15:49.955 INFO:tasks.workunit.client.1.smithi069.stdout:7/7: dwrite - no filename 2024-03-22T07:15:49.955 INFO:tasks.workunit.client.1.smithi069.stdout:8/0: dread - no filename 2024-03-22T07:15:49.955 INFO:tasks.workunit.client.1.smithi069.stdout:8/1: link - no file 2024-03-22T07:15:49.955 INFO:tasks.workunit.client.1.smithi069.stdout:8/2: write - no filename 2024-03-22T07:15:49.955 INFO:tasks.workunit.client.1.smithi069.stdout:8/3: write - no filename 2024-03-22T07:15:49.955 INFO:tasks.workunit.client.1.smithi069.stdout:8/4: fsync - no filename 2024-03-22T07:15:49.955 INFO:tasks.workunit.client.1.smithi069.stdout:8/5: chown . 1 1 2024-03-22T07:15:49.955 INFO:tasks.workunit.client.1.smithi069.stdout:8/6: truncate - no filename 2024-03-22T07:15:49.956 INFO:tasks.workunit.client.1.smithi069.stdout:8/7: write - no filename 2024-03-22T07:15:49.956 INFO:tasks.workunit.client.1.smithi069.stdout:8/8: dread - no filename 2024-03-22T07:15:49.956 INFO:tasks.workunit.client.1.smithi069.stdout:8/9: dread - no filename 2024-03-22T07:15:49.956 INFO:tasks.workunit.client.1.smithi069.stdout:8/10: rename - no filename 2024-03-22T07:15:49.956 INFO:tasks.workunit.client.1.smithi069.stdout:8/11: getdents . 0 2024-03-22T07:15:49.956 INFO:tasks.workunit.client.1.smithi069.stdout:8/12: fdatasync - no filename 2024-03-22T07:15:49.956 INFO:tasks.workunit.client.1.smithi069.stdout:8/13: chown . 172 1 2024-03-22T07:15:50.063 INFO:tasks.workunit.client.0.smithi063.stdout:4/2: sync 2024-03-22T07:15:50.064 INFO:tasks.workunit.client.0.smithi063.stdout:4/3: creat f0 x:0 0 0 2024-03-22T07:15:50.064 INFO:tasks.workunit.client.0.smithi063.stdout:4/4: write f0 [240008,109010] 0 2024-03-22T07:15:50.080 INFO:tasks.workunit.client.0.smithi063.stdout:4/5: dwrite f0 [0,4194304] 0 2024-03-22T07:15:50.081 INFO:tasks.workunit.client.0.smithi063.stdout:4/6: mkdir d1 0 2024-03-22T07:15:50.081 INFO:tasks.workunit.client.0.smithi063.stdout:4/7: truncate f0 4387478 0 2024-03-22T07:15:50.083 INFO:tasks.workunit.client.0.smithi063.stdout:4/8: link f0 d1/f2 0 2024-03-22T07:15:50.089 INFO:tasks.workunit.client.0.smithi063.stdout:4/9: dread f0 [0,4194304] 0 2024-03-22T07:15:50.090 INFO:tasks.workunit.client.0.smithi063.stdout:4/10: creat d1/f3 x:0 0 0 2024-03-22T07:15:50.091 INFO:tasks.workunit.client.0.smithi063.stdout:4/11: stat d1 0 2024-03-22T07:15:50.262 INFO:tasks.workunit.client.0.smithi063.stdout:4/12: dwrite d1/f3 [0,4194304] 0 2024-03-22T07:15:50.269 INFO:tasks.workunit.client.0.smithi063.stdout:3/33: sync 2024-03-22T07:15:50.269 INFO:tasks.workunit.client.0.smithi063.stdout:2/43: sync 2024-03-22T07:15:50.270 INFO:tasks.workunit.client.0.smithi063.stdout:2/44: chown d0/fc 48216 1 2024-03-22T07:15:50.270 INFO:tasks.workunit.client.0.smithi063.stdout:3/34: truncate da/fb 1896554 0 2024-03-22T07:15:50.272 INFO:tasks.workunit.client.0.smithi063.stdout:3/35: creat da/fd x:0 0 0 2024-03-22T07:15:50.273 INFO:tasks.workunit.client.0.smithi063.stdout:2/45: mknod d0/cf 0 2024-03-22T07:15:50.273 INFO:tasks.workunit.client.0.smithi063.stdout:2/46: chown d0/cd 33980176 1 2024-03-22T07:15:50.275 INFO:tasks.workunit.client.0.smithi063.stdout:3/36: mkdir da/de 0 2024-03-22T07:15:50.276 INFO:tasks.workunit.client.0.smithi063.stdout:3/37: rmdir da/de 0 2024-03-22T07:15:50.290 INFO:tasks.workunit.client.0.smithi063.stdout:2/47: dread d0/f9 [0,4194304] 0 2024-03-22T07:15:50.291 INFO:tasks.workunit.client.0.smithi063.stdout:2/48: truncate d0/f4 631004 0 2024-03-22T07:15:50.291 INFO:tasks.workunit.client.0.smithi063.stdout:2/49: write d0/f9 [401312,104308] 0 2024-03-22T07:15:50.291 INFO:tasks.workunit.client.0.smithi063.stdout:2/50: fdatasync d0/fa 0 2024-03-22T07:15:50.291 INFO:tasks.workunit.client.0.smithi063.stdout:2/51: truncate d0/f9 1115291 0 2024-03-22T07:15:50.292 INFO:tasks.workunit.client.0.smithi063.stdout:2/52: chown d0/cf 269428 1 2024-03-22T07:15:50.297 INFO:tasks.workunit.client.0.smithi063.stdout:4/13: write d1/f3 [3466162,50925] 0 2024-03-22T07:15:50.297 INFO:tasks.workunit.client.0.smithi063.stdout:4/14: mknod d1/c4 0 2024-03-22T07:15:50.297 INFO:tasks.workunit.client.0.smithi063.stdout:4/15: chown d1/c4 5642239 1 2024-03-22T07:15:50.298 INFO:tasks.workunit.client.0.smithi063.stdout:4/16: mkdir d1/d5 0 2024-03-22T07:15:50.299 INFO:tasks.workunit.client.0.smithi063.stdout:4/17: truncate d1/f2 3918831 0 2024-03-22T07:15:50.307 INFO:tasks.workunit.client.0.smithi063.stdout:4/18: truncate d1/f3 2480347 0 2024-03-22T07:15:50.313 INFO:tasks.workunit.client.0.smithi063.stdout:2/53: dwrite d0/fb [0,4194304] 0 2024-03-22T07:15:50.314 INFO:tasks.workunit.client.0.smithi063.stdout:2/54: read d0/f9 [850384,112728] 0 2024-03-22T07:15:50.315 INFO:tasks.workunit.client.0.smithi063.stdout:4/19: read f0 [3500925,16162] 0 2024-03-22T07:15:50.315 INFO:tasks.workunit.client.0.smithi063.stdout:4/20: fdatasync f0 0 2024-03-22T07:15:50.315 INFO:tasks.workunit.client.0.smithi063.stdout:4/21: creat d1/f6 x:0 0 0 2024-03-22T07:15:50.318 INFO:tasks.workunit.client.0.smithi063.stdout:2/55: creat d0/f10 x:0 0 0 2024-03-22T07:15:50.320 INFO:tasks.workunit.client.0.smithi063.stdout:0/109: sync 2024-03-22T07:15:50.320 INFO:tasks.workunit.client.0.smithi063.stdout:9/43: sync 2024-03-22T07:15:50.320 INFO:tasks.workunit.client.0.smithi063.stdout:9/44: stat d8/fa 0 2024-03-22T07:15:50.320 INFO:tasks.workunit.client.0.smithi063.stdout:6/70: sync 2024-03-22T07:15:50.320 INFO:tasks.workunit.client.0.smithi063.stdout:1/117: sync 2024-03-22T07:15:50.320 INFO:tasks.workunit.client.0.smithi063.stdout:2/56: dread d0/f4 [0,4194304] 0 2024-03-22T07:15:50.320 INFO:tasks.workunit.client.0.smithi063.stdout:8/91: sync 2024-03-22T07:15:50.321 INFO:tasks.workunit.client.0.smithi063.stdout:6/71: dread - d3/f18 zero size 2024-03-22T07:15:50.321 INFO:tasks.workunit.client.0.smithi063.stdout:2/57: stat d0/f8 0 2024-03-22T07:15:50.321 INFO:tasks.workunit.client.0.smithi063.stdout:6/72: fsync d3/f4 0 2024-03-22T07:15:50.321 INFO:tasks.workunit.client.0.smithi063.stdout:6/73: stat d3/d15/d16 0 2024-03-22T07:15:50.321 INFO:tasks.workunit.client.0.smithi063.stdout:6/74: read - d3/d6/f11 zero size 2024-03-22T07:15:50.321 INFO:tasks.workunit.client.0.smithi063.stdout:5/71: sync 2024-03-22T07:15:50.321 INFO:tasks.workunit.client.0.smithi063.stdout:7/67: sync 2024-03-22T07:15:50.321 INFO:tasks.workunit.client.0.smithi063.stdout:3/38: write da/fb [792,117015] 0 2024-03-22T07:15:50.321 INFO:tasks.workunit.client.0.smithi063.stdout:4/22: link d1/f3 d1/d5/f7 0 2024-03-22T07:15:50.322 INFO:tasks.workunit.client.0.smithi063.stdout:4/23: write d1/d5/f7 [17349,46425] 0 2024-03-22T07:15:50.322 INFO:tasks.workunit.client.1.smithi069.stdout:4/13: dwrite f3 [0,4194304] 0 2024-03-22T07:15:50.322 INFO:tasks.workunit.client.0.smithi063.stdout:0/110: mkdir da/de/d17/d22 0 2024-03-22T07:15:50.325 INFO:tasks.workunit.client.1.smithi069.stdout:4/14: creat f4 x:0 0 0 2024-03-22T07:15:50.326 INFO:tasks.workunit.client.0.smithi063.stdout:5/72: dread d8/f10 [0,4194304] 0 2024-03-22T07:15:50.326 INFO:tasks.workunit.client.0.smithi063.stdout:5/73: write d8/f16 [560430,58482] 0 2024-03-22T07:15:50.327 INFO:tasks.workunit.client.0.smithi063.stdout:9/45: symlink d8/le 0 2024-03-22T07:15:50.327 INFO:tasks.workunit.client.0.smithi063.stdout:9/46: read - d8/fb zero size 2024-03-22T07:15:50.327 INFO:tasks.workunit.client.0.smithi063.stdout:4/24: dread d1/f2 [0,4194304] 0 2024-03-22T07:15:50.327 INFO:tasks.workunit.client.0.smithi063.stdout:4/25: write d1/d5/f7 [1692193,112713] 0 2024-03-22T07:15:50.328 INFO:tasks.workunit.client.0.smithi063.stdout:2/58: mknod d0/de/c11 0 2024-03-22T07:15:50.329 INFO:tasks.workunit.client.0.smithi063.stdout:6/75: mknod d3/d15/d16/c1a 0 2024-03-22T07:15:50.330 INFO:tasks.workunit.client.1.smithi069.stdout:4/15: write f3 [687013,26057] 0 2024-03-22T07:15:50.331 INFO:tasks.workunit.client.1.smithi069.stdout:4/16: rmdir - no directory 2024-03-22T07:15:50.332 INFO:tasks.workunit.client.0.smithi063.stdout:0/111: write f9 [286858,72172] 0 2024-03-22T07:15:50.332 INFO:tasks.workunit.client.0.smithi063.stdout:7/68: mknod d5/c19 0 2024-03-22T07:15:50.332 INFO:tasks.workunit.client.0.smithi063.stdout:0/112: write f1 [1416033,113270] 0 2024-03-22T07:15:50.332 INFO:tasks.workunit.client.0.smithi063.stdout:7/69: write d5/f18 [25594,40216] 0 2024-03-22T07:15:50.332 INFO:tasks.workunit.client.0.smithi063.stdout:0/113: write f9 [5156116,1639] 0 2024-03-22T07:15:50.333 INFO:tasks.workunit.client.0.smithi063.stdout:7/70: readlink l1 0 2024-03-22T07:15:50.333 INFO:tasks.workunit.client.0.smithi063.stdout:7/71: write d5/f17 [56960,73502] 0 2024-03-22T07:15:50.334 INFO:tasks.workunit.client.0.smithi063.stdout:9/47: mknod d8/cf 0 2024-03-22T07:15:50.334 INFO:tasks.workunit.client.0.smithi063.stdout:2/59: mkdir d0/de/d12 0 2024-03-22T07:15:50.336 INFO:tasks.workunit.client.0.smithi063.stdout:5/74: write d8/f12 [1929765,30237] 0 2024-03-22T07:15:50.336 INFO:tasks.workunit.client.0.smithi063.stdout:5/75: dread - d8/f14 zero size 2024-03-22T07:15:50.337 INFO:tasks.workunit.client.0.smithi063.stdout:9/48: rmdir d8 39 2024-03-22T07:15:50.337 INFO:tasks.workunit.client.0.smithi063.stdout:2/60: creat d0/f13 x:0 0 0 2024-03-22T07:15:50.339 INFO:tasks.workunit.client.0.smithi063.stdout:5/76: creat d8/d13/f18 x:0 0 0 2024-03-22T07:15:50.339 INFO:tasks.workunit.client.0.smithi063.stdout:5/77: write d8/f12 [1943149,26706] 0 2024-03-22T07:15:50.340 INFO:tasks.workunit.client.0.smithi063.stdout:9/49: rename d8/f9 to d8/f10 0 2024-03-22T07:15:50.342 INFO:tasks.workunit.client.0.smithi063.stdout:2/61: link d0/f13 d0/de/f14 0 2024-03-22T07:15:50.346 INFO:tasks.workunit.client.0.smithi063.stdout:9/50: symlink d8/l11 0 2024-03-22T07:15:50.346 INFO:tasks.workunit.client.0.smithi063.stdout:1/118: dwrite d4/d14/f1b [0,4194304] 0 2024-03-22T07:15:50.347 INFO:tasks.workunit.client.0.smithi063.stdout:2/62: creat d0/de/f15 x:0 0 0 2024-03-22T07:15:50.347 INFO:tasks.workunit.client.0.smithi063.stdout:2/63: truncate d0/f4 1578690 0 2024-03-22T07:15:50.347 INFO:tasks.workunit.client.0.smithi063.stdout:2/64: readlink d0/l7 0 2024-03-22T07:15:50.348 INFO:tasks.workunit.client.0.smithi063.stdout:2/65: fdatasync d0/fa 0 2024-03-22T07:15:50.348 INFO:tasks.workunit.client.0.smithi063.stdout:7/72: dread d5/f13 [0,4194304] 0 2024-03-22T07:15:50.349 INFO:tasks.workunit.client.0.smithi063.stdout:9/51: mknod d8/c12 0 2024-03-22T07:15:50.349 INFO:tasks.workunit.client.0.smithi063.stdout:9/52: truncate d8/fd 105667 0 2024-03-22T07:15:50.349 INFO:tasks.workunit.client.0.smithi063.stdout:9/53: truncate d8/fb 209962 0 2024-03-22T07:15:50.355 INFO:tasks.workunit.client.0.smithi063.stdout:7/73: link d5/f7 d5/d14/f1a 0 2024-03-22T07:15:50.357 INFO:tasks.workunit.client.0.smithi063.stdout:0/114: dwrite da/fc [0,4194304] 0 2024-03-22T07:15:50.358 INFO:tasks.workunit.client.0.smithi063.stdout:4/26: dwrite d1/f2 [0,4194304] 0 2024-03-22T07:15:50.358 INFO:tasks.workunit.client.0.smithi063.stdout:9/54: rmdir d8 39 2024-03-22T07:15:50.360 INFO:tasks.workunit.client.0.smithi063.stdout:1/119: rmdir d4 39 2024-03-22T07:15:50.361 INFO:tasks.workunit.client.0.smithi063.stdout:5/78: getdents d8/d13 0 2024-03-22T07:15:50.363 INFO:tasks.workunit.client.0.smithi063.stdout:2/66: symlink d0/l16 0 2024-03-22T07:15:50.366 INFO:tasks.workunit.client.0.smithi063.stdout:4/27: dread d1/f2 [0,4194304] 0 2024-03-22T07:15:50.370 INFO:tasks.workunit.client.0.smithi063.stdout:7/74: mkdir d5/d16/d1b 0 2024-03-22T07:15:50.370 INFO:tasks.workunit.client.0.smithi063.stdout:7/75: write d5/f9 [134609,28175] 0 2024-03-22T07:15:50.371 INFO:tasks.workunit.client.0.smithi063.stdout:2/67: truncate d0/fb 1892684 0 2024-03-22T07:15:50.371 INFO:tasks.workunit.client.0.smithi063.stdout:2/68: chown d0/c6 219616 1 2024-03-22T07:15:50.372 INFO:tasks.workunit.client.0.smithi063.stdout:4/28: symlink d1/l8 0 2024-03-22T07:15:50.373 INFO:tasks.workunit.client.0.smithi063.stdout:7/76: mknod d5/d16/d1b/c1c 0 2024-03-22T07:15:50.374 INFO:tasks.workunit.client.0.smithi063.stdout:9/55: dwrite d8/fd [0,4194304] 0 2024-03-22T07:15:50.375 INFO:tasks.workunit.client.1.smithi069.stdout:2/12: rename l0 to l4 0 2024-03-22T07:15:50.375 INFO:tasks.workunit.client.1.smithi069.stdout:2/13: rmdir - no directory 2024-03-22T07:15:50.375 INFO:tasks.workunit.client.0.smithi063.stdout:5/79: rename d8/l9 to d8/d13/l19 0 2024-03-22T07:15:50.377 INFO:tasks.workunit.client.0.smithi063.stdout:7/77: dread d5/f9 [0,4194304] 0 2024-03-22T07:15:50.377 INFO:tasks.workunit.client.0.smithi063.stdout:7/78: fdatasync d5/f18 0 2024-03-22T07:15:50.377 INFO:tasks.workunit.client.0.smithi063.stdout:7/79: dread - d5/da/f11 zero size 2024-03-22T07:15:50.378 INFO:tasks.workunit.client.0.smithi063.stdout:2/69: unlink d0/fc 0 2024-03-22T07:15:50.379 INFO:tasks.workunit.client.0.smithi063.stdout:4/29: dread d1/d5/f7 [0,4194304] 0 2024-03-22T07:15:50.382 INFO:tasks.workunit.client.0.smithi063.stdout:5/80: link d8/cd d8/d13/c1a 0 2024-03-22T07:15:50.385 INFO:tasks.workunit.client.0.smithi063.stdout:4/30: creat d1/d5/f9 x:0 0 0 2024-03-22T07:15:50.387 INFO:tasks.workunit.client.0.smithi063.stdout:2/70: symlink d0/l17 0 2024-03-22T07:15:50.387 INFO:tasks.workunit.client.0.smithi063.stdout:2/71: truncate d0/f13 491794 0 2024-03-22T07:15:50.387 INFO:tasks.workunit.client.0.smithi063.stdout:2/72: readlink d0/l7 0 2024-03-22T07:15:50.387 INFO:tasks.workunit.client.0.smithi063.stdout:2/73: write d0/f9 [1201823,43944] 0 2024-03-22T07:15:50.388 INFO:tasks.workunit.client.0.smithi063.stdout:4/31: symlink d1/d5/la 0 2024-03-22T07:15:50.388 INFO:tasks.workunit.client.0.smithi063.stdout:4/32: chown d1 341681619 1 2024-03-22T07:15:50.389 INFO:tasks.workunit.client.0.smithi063.stdout:4/33: chown f0 11877233 1 2024-03-22T07:15:50.389 INFO:tasks.workunit.client.0.smithi063.stdout:4/34: write d1/f3 [1180051,64706] 0 2024-03-22T07:15:50.389 INFO:tasks.workunit.client.0.smithi063.stdout:2/74: mknod d0/de/d12/c18 0 2024-03-22T07:15:50.392 INFO:tasks.workunit.client.0.smithi063.stdout:4/35: write d1/f2 [3299951,62020] 0 2024-03-22T07:15:50.392 INFO:tasks.workunit.client.0.smithi063.stdout:4/36: read - d1/d5/f9 zero size 2024-03-22T07:15:50.401 INFO:tasks.workunit.client.0.smithi063.stdout:7/80: dwrite d5/d14/f1a [0,4194304] 0 2024-03-22T07:15:50.401 INFO:tasks.workunit.client.0.smithi063.stdout:7/81: fdatasync d5/f13 0 2024-03-22T07:15:50.402 INFO:tasks.workunit.client.0.smithi063.stdout:7/82: mkdir d5/d14/d1d 0 2024-03-22T07:15:50.402 INFO:tasks.workunit.client.0.smithi063.stdout:7/83: fdatasync d5/f17 0 2024-03-22T07:15:50.402 INFO:tasks.workunit.client.0.smithi063.stdout:9/56: dwrite f2 [0,4194304] 0 2024-03-22T07:15:50.415 INFO:tasks.workunit.client.0.smithi063.stdout:9/57: dread f2 [0,4194304] 0 2024-03-22T07:15:50.415 INFO:tasks.workunit.client.0.smithi063.stdout:9/58: chown c6 216784 1 2024-03-22T07:15:50.415 INFO:tasks.workunit.client.0.smithi063.stdout:9/59: write d8/f10 [59163,30911] 0 2024-03-22T07:15:50.435 INFO:tasks.workunit.client.0.smithi063.stdout:2/75: dwrite d0/f4 [0,4194304] 0 2024-03-22T07:15:50.435 INFO:tasks.workunit.client.0.smithi063.stdout:2/76: write d0/f13 [630628,795] 0 2024-03-22T07:15:50.435 INFO:tasks.workunit.client.0.smithi063.stdout:5/81: dwrite d8/d13/f18 [0,4194304] 0 2024-03-22T07:15:50.435 INFO:tasks.workunit.client.0.smithi063.stdout:5/82: chown d8/f16 822749 1 2024-03-22T07:15:50.435 INFO:tasks.workunit.client.0.smithi063.stdout:4/37: dwrite d1/f6 [0,4194304] 0 2024-03-22T07:15:50.435 INFO:tasks.workunit.client.0.smithi063.stdout:7/84: dwrite d5/f17 [0,4194304] 0 2024-03-22T07:15:50.447 INFO:tasks.workunit.client.0.smithi063.stdout:7/85: creat d5/d16/f1e x:0 0 0 2024-03-22T07:15:50.450 INFO:tasks.workunit.client.0.smithi063.stdout:2/77: rmdir d0 39 2024-03-22T07:15:50.450 INFO:tasks.workunit.client.0.smithi063.stdout:4/38: getdents d1 0 2024-03-22T07:15:50.450 INFO:tasks.workunit.client.0.smithi063.stdout:4/39: truncate d1/d5/f9 522080 0 2024-03-22T07:15:50.451 INFO:tasks.workunit.client.0.smithi063.stdout:4/40: chown f0 367924 1 2024-03-22T07:15:50.451 INFO:tasks.workunit.client.0.smithi063.stdout:7/86: creat d5/d16/d1b/f1f x:0 0 0 2024-03-22T07:15:50.452 INFO:tasks.workunit.client.0.smithi063.stdout:5/83: dwrite d8/d13/f18 [0,4194304] 0 2024-03-22T07:15:50.452 INFO:tasks.workunit.client.0.smithi063.stdout:2/78: unlink d0/de/c11 0 2024-03-22T07:15:50.452 INFO:tasks.workunit.client.0.smithi063.stdout:2/79: readlink d0/l7 0 2024-03-22T07:15:50.453 INFO:tasks.workunit.client.0.smithi063.stdout:2/80: fdatasync d0/f5 0 2024-03-22T07:15:50.453 INFO:tasks.workunit.client.0.smithi063.stdout:4/41: mknod d1/cb 0 2024-03-22T07:15:50.453 INFO:tasks.workunit.client.0.smithi063.stdout:4/42: chown d1 253636 1 2024-03-22T07:15:50.455 INFO:tasks.workunit.client.0.smithi063.stdout:7/87: rename d5/l6 to d5/d14/d1d/l20 0 2024-03-22T07:15:50.464 INFO:tasks.workunit.client.0.smithi063.stdout:2/81: dread d0/fa [0,4194304] 0 2024-03-22T07:15:50.465 INFO:tasks.workunit.client.0.smithi063.stdout:2/82: rename d0/cf to d0/c19 0 2024-03-22T07:15:50.466 INFO:tasks.workunit.client.0.smithi063.stdout:2/83: mknod d0/de/c1a 0 2024-03-22T07:15:50.476 INFO:tasks.workunit.client.0.smithi063.stdout:5/84: dwrite d8/f16 [0,4194304] 0 2024-03-22T07:15:50.482 INFO:tasks.workunit.client.0.smithi063.stdout:4/43: dwrite d1/f2 [0,4194304] 0 2024-03-22T07:15:50.485 INFO:tasks.workunit.client.0.smithi063.stdout:5/85: dread d8/f12 [0,4194304] 0 2024-03-22T07:15:50.489 INFO:tasks.workunit.client.0.smithi063.stdout:5/86: dread d8/fe [0,4194304] 0 2024-03-22T07:15:50.489 INFO:tasks.workunit.client.0.smithi063.stdout:7/88: dwrite d5/f13 [4194304,4194304] 0 2024-03-22T07:15:50.489 INFO:tasks.workunit.client.0.smithi063.stdout:2/84: dwrite d0/de/f15 [0,4194304] 0 2024-03-22T07:15:50.490 INFO:tasks.workunit.client.0.smithi063.stdout:5/87: rename d8/c15 to d8/d13/c1b 0 2024-03-22T07:15:50.490 INFO:tasks.workunit.client.0.smithi063.stdout:5/88: write d8/f14 [677060,31897] 0 2024-03-22T07:15:50.491 INFO:tasks.workunit.client.0.smithi063.stdout:5/89: write d8/f14 [13923,65609] 0 2024-03-22T07:15:50.492 INFO:tasks.workunit.client.0.smithi063.stdout:7/89: mkdir d5/d16/d1b/d21 0 2024-03-22T07:15:50.492 INFO:tasks.workunit.client.0.smithi063.stdout:7/90: fsync d5/f18 0 2024-03-22T07:15:50.495 INFO:tasks.workunit.client.0.smithi063.stdout:2/85: getdents d0/de/d12 0 2024-03-22T07:15:50.496 INFO:tasks.workunit.client.0.smithi063.stdout:5/90: creat d8/f1c x:0 0 0 2024-03-22T07:15:50.497 INFO:tasks.workunit.client.0.smithi063.stdout:2/86: mkdir d0/de/d1b 0 2024-03-22T07:15:50.497 INFO:tasks.workunit.client.0.smithi063.stdout:2/87: read d0/f5 [862142,9311] 0 2024-03-22T07:15:50.504 INFO:tasks.workunit.client.0.smithi063.stdout:4/44: dwrite d1/f3 [0,4194304] 0 2024-03-22T07:15:50.510 INFO:tasks.workunit.client.0.smithi063.stdout:7/91: dread d5/f13 [4194304,4194304] 0 2024-03-22T07:15:50.514 INFO:tasks.workunit.client.0.smithi063.stdout:7/92: dread d5/da/ff [0,4194304] 0 2024-03-22T07:15:50.514 INFO:tasks.workunit.client.0.smithi063.stdout:7/93: dread - d5/d16/f1e zero size 2024-03-22T07:15:50.514 INFO:tasks.workunit.client.0.smithi063.stdout:7/94: fsync d5/f9 0 2024-03-22T07:15:50.514 INFO:tasks.workunit.client.0.smithi063.stdout:7/95: chown c2 55775551 1 2024-03-22T07:15:50.515 INFO:tasks.workunit.client.0.smithi063.stdout:2/88: dwrite d0/fa [4194304,4194304] 0 2024-03-22T07:15:50.515 INFO:tasks.workunit.client.0.smithi063.stdout:7/96: getdents d5 0 2024-03-22T07:15:50.536 INFO:tasks.workunit.client.0.smithi063.stdout:5/91: dwrite d8/f14 [0,4194304] 0 2024-03-22T07:15:50.539 INFO:tasks.workunit.client.0.smithi063.stdout:4/45: dwrite d1/d5/f9 [0,4194304] 0 2024-03-22T07:15:50.540 INFO:tasks.workunit.client.0.smithi063.stdout:5/92: rename c5 to d8/c1d 0 2024-03-22T07:15:50.541 INFO:tasks.workunit.client.0.smithi063.stdout:5/93: truncate d8/f1c 787743 0 2024-03-22T07:15:50.546 INFO:tasks.workunit.client.0.smithi063.stdout:4/46: mkdir d1/d5/dc 0 2024-03-22T07:15:50.547 INFO:tasks.workunit.client.0.smithi063.stdout:5/94: mkdir d8/d13/d1e 0 2024-03-22T07:15:50.547 INFO:tasks.workunit.client.0.smithi063.stdout:5/95: fdatasync d8/d13/f18 0 2024-03-22T07:15:50.548 INFO:tasks.workunit.client.0.smithi063.stdout:4/47: link d1/d5/la d1/d5/dc/ld 0 2024-03-22T07:15:50.550 INFO:tasks.workunit.client.0.smithi063.stdout:5/96: symlink d8/l1f 0 2024-03-22T07:15:50.550 INFO:tasks.workunit.client.0.smithi063.stdout:5/97: stat d8 0 2024-03-22T07:15:50.550 INFO:tasks.workunit.client.0.smithi063.stdout:5/98: stat d8/d13 0 2024-03-22T07:15:50.567 INFO:tasks.workunit.client.0.smithi063.stdout:7/97: dwrite d5/da/ff [4194304,4194304] 0 2024-03-22T07:15:50.567 INFO:tasks.workunit.client.0.smithi063.stdout:7/98: readlink d5/lc 0 2024-03-22T07:15:50.567 INFO:tasks.workunit.client.0.smithi063.stdout:7/99: read - d5/d16/f1e zero size 2024-03-22T07:15:50.570 INFO:tasks.workunit.client.0.smithi063.stdout:2/89: dread d0/f13 [0,4194304] 0 2024-03-22T07:15:50.574 INFO:tasks.workunit.client.0.smithi063.stdout:4/48: dwrite d1/f6 [4194304,4194304] 0 2024-03-22T07:15:50.574 INFO:tasks.workunit.client.0.smithi063.stdout:4/49: fsync d1/d5/f7 0 2024-03-22T07:15:50.575 INFO:tasks.workunit.client.0.smithi063.stdout:4/50: creat d1/d5/dc/fe x:0 0 0 2024-03-22T07:15:50.576 INFO:tasks.workunit.client.0.smithi063.stdout:4/51: creat d1/d5/dc/ff x:0 0 0 2024-03-22T07:15:50.577 INFO:tasks.workunit.client.0.smithi063.stdout:2/90: dread d0/f9 [0,4194304] 0 2024-03-22T07:15:50.578 INFO:tasks.workunit.client.0.smithi063.stdout:2/91: write d0/f8 [537503,47157] 0 2024-03-22T07:15:50.578 INFO:tasks.workunit.client.0.smithi063.stdout:2/92: fsync d0/f13 0 2024-03-22T07:15:50.581 INFO:tasks.workunit.client.0.smithi063.stdout:3/39: sync 2024-03-22T07:15:50.581 INFO:tasks.workunit.client.0.smithi063.stdout:8/92: sync 2024-03-22T07:15:50.581 INFO:tasks.workunit.client.0.smithi063.stdout:8/93: readlink d0/l3 0 2024-03-22T07:15:50.582 INFO:tasks.workunit.client.0.smithi063.stdout:8/94: chown d0/f19 1 1 2024-03-22T07:15:50.582 INFO:tasks.workunit.client.0.smithi063.stdout:2/93: dread d0/f9 [0,4194304] 0 2024-03-22T07:15:50.582 INFO:tasks.workunit.client.0.smithi063.stdout:2/94: chown d0/de/c1a 0 1 2024-03-22T07:15:50.582 INFO:tasks.workunit.client.0.smithi063.stdout:2/95: truncate d0/f5 1810393 0 2024-03-22T07:15:50.583 INFO:tasks.workunit.client.0.smithi063.stdout:3/40: unlink f6 0 2024-03-22T07:15:50.587 INFO:tasks.workunit.client.0.smithi063.stdout:8/95: mknod d0/d1f/c21 0 2024-03-22T07:15:50.587 INFO:tasks.workunit.client.0.smithi063.stdout:8/96: dread - d0/d1b/f1e zero size 2024-03-22T07:15:50.587 INFO:tasks.workunit.client.0.smithi063.stdout:8/97: chown d0/f5 38889 1 2024-03-22T07:15:50.587 INFO:tasks.workunit.client.0.smithi063.stdout:8/98: chown d0/cf 133918 1 2024-03-22T07:15:50.587 INFO:tasks.workunit.client.0.smithi063.stdout:2/96: creat d0/f1c x:0 0 0 2024-03-22T07:15:50.587 INFO:tasks.workunit.client.0.smithi063.stdout:2/97: truncate d0/de/f14 812585 0 2024-03-22T07:15:50.588 INFO:tasks.workunit.client.0.smithi063.stdout:3/41: mknod da/cf 0 2024-03-22T07:15:50.588 INFO:tasks.workunit.client.0.smithi063.stdout:3/42: truncate da/fb 2289730 0 2024-03-22T07:15:50.588 INFO:tasks.workunit.client.0.smithi063.stdout:3/43: write da/fb [1876684,7432] 0 2024-03-22T07:15:50.588 INFO:tasks.workunit.client.0.smithi063.stdout:3/44: chown da/cf 134 1 2024-03-22T07:15:50.589 INFO:tasks.workunit.client.0.smithi063.stdout:3/45: creat da/f10 x:0 0 0 2024-03-22T07:15:50.589 INFO:tasks.workunit.client.0.smithi063.stdout:3/46: chown c3 50647 1 2024-03-22T07:15:50.589 INFO:tasks.workunit.client.0.smithi063.stdout:3/47: fsync da/fd 0 2024-03-22T07:15:50.589 INFO:tasks.workunit.client.0.smithi063.stdout:3/48: write da/f10 [890781,77586] 0 2024-03-22T07:15:50.589 INFO:tasks.workunit.client.0.smithi063.stdout:3/49: readlink l8 0 2024-03-22T07:15:50.589 INFO:tasks.workunit.client.0.smithi063.stdout:3/50: dread - da/fd zero size 2024-03-22T07:15:50.589 INFO:tasks.workunit.client.0.smithi063.stdout:8/99: rename d0/fd to d0/f22 0 2024-03-22T07:15:50.591 INFO:tasks.workunit.client.0.smithi063.stdout:8/100: truncate d0/f4 1910494 0 2024-03-22T07:15:50.591 INFO:tasks.workunit.client.0.smithi063.stdout:8/101: chown d0/d18 2068133044 1 2024-03-22T07:15:50.592 INFO:tasks.workunit.client.0.smithi063.stdout:2/98: getdents d0/de 0 2024-03-22T07:15:50.593 INFO:tasks.workunit.client.0.smithi063.stdout:8/102: symlink d0/d18/l23 0 2024-03-22T07:15:50.595 INFO:tasks.workunit.client.0.smithi063.stdout:2/99: dread d0/f5 [0,4194304] 0 2024-03-22T07:15:50.596 INFO:tasks.workunit.client.0.smithi063.stdout:2/100: symlink d0/de/l1d 0 2024-03-22T07:15:50.602 INFO:tasks.workunit.client.0.smithi063.stdout:4/52: dwrite d1/f6 [0,4194304] 0 2024-03-22T07:15:50.602 INFO:tasks.workunit.client.0.smithi063.stdout:2/101: write d0/fb [1564095,98325] 0 2024-03-22T07:15:50.602 INFO:tasks.workunit.client.0.smithi063.stdout:2/102: chown d0/cd 50306262 1 2024-03-22T07:15:50.602 INFO:tasks.workunit.client.0.smithi063.stdout:2/103: chown d0/f4 1967877550 1 2024-03-22T07:15:50.602 INFO:tasks.workunit.client.0.smithi063.stdout:5/99: dwrite d8/f12 [0,4194304] 0 2024-03-22T07:15:50.604 INFO:tasks.workunit.client.0.smithi063.stdout:7/100: dwrite d5/d16/d1b/f1f [0,4194304] 0 2024-03-22T07:15:50.604 INFO:tasks.workunit.client.0.smithi063.stdout:7/101: getdents d5 0 2024-03-22T07:15:50.604 INFO:tasks.workunit.client.0.smithi063.stdout:7/102: chown d5/l15 298617147 1 2024-03-22T07:15:50.604 INFO:tasks.workunit.client.0.smithi063.stdout:7/103: creat d5/d16/f22 x:0 0 0 2024-03-22T07:15:50.606 INFO:tasks.workunit.client.0.smithi063.stdout:2/104: truncate d0/f4 2894500 0 2024-03-22T07:15:50.613 INFO:tasks.workunit.client.0.smithi063.stdout:2/105: symlink d0/de/d12/l1e 0 2024-03-22T07:15:50.619 INFO:tasks.workunit.client.0.smithi063.stdout:5/100: dread d8/f12 [0,4194304] 0 2024-03-22T07:15:50.621 INFO:tasks.workunit.client.0.smithi063.stdout:4/53: write d1/f3 [3761649,21828] 0 2024-03-22T07:15:50.629 INFO:tasks.workunit.client.0.smithi063.stdout:8/103: dwrite d0/d1b/f1e [0,4194304] 0 2024-03-22T07:15:50.629 INFO:tasks.workunit.client.0.smithi063.stdout:8/104: chown d0/fc 127 1 2024-03-22T07:15:50.629 INFO:tasks.workunit.client.0.smithi063.stdout:3/51: dwrite da/f10 [0,4194304] 0 2024-03-22T07:15:50.629 INFO:tasks.workunit.client.0.smithi063.stdout:7/104: dwrite d5/f9 [0,4194304] 0 2024-03-22T07:15:50.637 INFO:tasks.workunit.client.0.smithi063.stdout:5/101: symlink d8/d13/l20 0 2024-03-22T07:15:50.638 INFO:tasks.workunit.client.0.smithi063.stdout:5/102: fsync d8/d13/f18 0 2024-03-22T07:15:50.638 INFO:tasks.workunit.client.0.smithi063.stdout:5/103: write d8/fe [3146823,3053] 0 2024-03-22T07:15:50.641 INFO:tasks.workunit.client.0.smithi063.stdout:8/105: dread d0/f12 [0,4194304] 0 2024-03-22T07:15:50.646 INFO:tasks.workunit.client.0.smithi063.stdout:5/104: dread d8/f1c [0,4194304] 0 2024-03-22T07:15:50.646 INFO:tasks.workunit.client.0.smithi063.stdout:5/105: fsync d8/f1c 0 2024-03-22T07:15:50.650 INFO:tasks.workunit.client.0.smithi063.stdout:4/54: dwrite d1/f3 [0,4194304] 0 2024-03-22T07:15:50.650 INFO:tasks.workunit.client.0.smithi063.stdout:4/55: dread - d1/d5/dc/fe zero size 2024-03-22T07:15:50.656 INFO:tasks.workunit.client.0.smithi063.stdout:3/52: dwrite da/fb [0,4194304] 0 2024-03-22T07:15:50.666 INFO:tasks.workunit.client.0.smithi063.stdout:2/106: dwrite d0/f4 [0,4194304] 0 2024-03-22T07:15:50.675 INFO:tasks.workunit.client.0.smithi063.stdout:3/53: dwrite da/f10 [0,4194304] 0 2024-03-22T07:15:50.675 INFO:tasks.workunit.client.0.smithi063.stdout:3/54: creat da/f11 x:0 0 0 2024-03-22T07:15:50.688 INFO:tasks.workunit.client.0.smithi063.stdout:7/105: mknod d5/d16/d1b/c23 0 2024-03-22T07:15:50.688 INFO:tasks.workunit.client.0.smithi063.stdout:7/106: chown d5/da 3553 1 2024-03-22T07:15:50.694 INFO:tasks.workunit.client.0.smithi063.stdout:8/106: unlink d0/c13 0 2024-03-22T07:15:50.695 INFO:tasks.workunit.client.0.smithi063.stdout:4/56: rename d1/d5/dc/fe to d1/d5/dc/f10 0 2024-03-22T07:15:50.698 INFO:tasks.workunit.client.0.smithi063.stdout:2/107: rename d0/f1c to d0/de/d12/f1f 0 2024-03-22T07:15:50.706 INFO:tasks.workunit.client.0.smithi063.stdout:7/107: mknod d5/d16/c24 0 2024-03-22T07:15:50.706 INFO:tasks.workunit.client.0.smithi063.stdout:7/108: truncate d5/f18 692047 0 2024-03-22T07:15:50.710 INFO:tasks.workunit.client.0.smithi063.stdout:8/107: mkdir d0/d24 0 2024-03-22T07:15:50.711 INFO:tasks.workunit.client.0.smithi063.stdout:4/57: stat d1/c4 0 2024-03-22T07:15:50.712 INFO:tasks.workunit.client.0.smithi063.stdout:3/55: rename c3 to da/c12 0 2024-03-22T07:15:50.714 INFO:tasks.workunit.client.0.smithi063.stdout:2/108: getdents d0/de 0 2024-03-22T07:15:50.715 INFO:tasks.workunit.client.0.smithi063.stdout:8/108: dread d0/f5 [0,4194304] 0 2024-03-22T07:15:50.715 INFO:tasks.workunit.client.0.smithi063.stdout:8/109: fsync d0/f4 0 2024-03-22T07:15:50.716 INFO:tasks.workunit.client.0.smithi063.stdout:7/109: creat d5/da/f25 x:0 0 0 2024-03-22T07:15:50.716 INFO:tasks.workunit.client.0.smithi063.stdout:7/110: read d5/f18 [403834,69206] 0 2024-03-22T07:15:50.716 INFO:tasks.workunit.client.0.smithi063.stdout:7/111: chown d5/f9 10 1 2024-03-22T07:15:50.720 INFO:tasks.workunit.client.0.smithi063.stdout:3/56: symlink da/l13 0 2024-03-22T07:15:50.720 INFO:tasks.workunit.client.0.smithi063.stdout:3/57: chown c4 69437 1 2024-03-22T07:15:50.724 INFO:tasks.workunit.client.0.smithi063.stdout:7/112: mknod d5/da/c26 0 2024-03-22T07:15:50.726 INFO:tasks.workunit.client.0.smithi063.stdout:7/113: symlink d5/d16/l27 0 2024-03-22T07:15:50.730 INFO:tasks.workunit.client.0.smithi063.stdout:3/58: dread da/fb [0,4194304] 0 2024-03-22T07:15:50.733 INFO:tasks.workunit.client.0.smithi063.stdout:4/58: getdents d1/d5/dc 0 2024-03-22T07:15:50.734 INFO:tasks.workunit.client.0.smithi063.stdout:4/59: dread - d1/d5/dc/f10 zero size 2024-03-22T07:15:50.734 INFO:tasks.workunit.client.0.smithi063.stdout:8/110: dwrite d0/f22 [0,4194304] 0 2024-03-22T07:15:50.735 INFO:tasks.workunit.client.0.smithi063.stdout:8/111: mknod d0/d1f/c25 0 2024-03-22T07:15:50.735 INFO:tasks.workunit.client.0.smithi063.stdout:8/112: stat d0/d1b/f1e 0 2024-03-22T07:15:50.735 INFO:tasks.workunit.client.0.smithi063.stdout:8/113: write d0/f19 [875299,86774] 0 2024-03-22T07:15:50.737 INFO:tasks.workunit.client.0.smithi063.stdout:8/114: mkdir d0/d26 0 2024-03-22T07:15:50.738 INFO:tasks.workunit.client.0.smithi063.stdout:2/109: getdents d0/de/d12 0 2024-03-22T07:15:50.738 INFO:tasks.workunit.client.0.smithi063.stdout:2/110: stat d0/l17 0 2024-03-22T07:15:50.738 INFO:tasks.workunit.client.0.smithi063.stdout:2/111: write d0/de/d12/f1f [601089,20669] 0 2024-03-22T07:15:50.741 INFO:tasks.workunit.client.0.smithi063.stdout:8/115: creat d0/d26/f27 x:0 0 0 2024-03-22T07:15:50.741 INFO:tasks.workunit.client.0.smithi063.stdout:8/116: write d0/d1b/f20 [58744,101338] 0 2024-03-22T07:15:50.741 INFO:tasks.workunit.client.0.smithi063.stdout:2/112: rmdir d0/de 39 2024-03-22T07:15:50.742 INFO:tasks.workunit.client.0.smithi063.stdout:5/106: truncate d8/fe 2539476 0 2024-03-22T07:15:50.745 INFO:tasks.workunit.client.0.smithi063.stdout:5/107: creat d8/f21 x:0 0 0 2024-03-22T07:15:50.750 INFO:tasks.workunit.client.0.smithi063.stdout:5/108: mknod d8/c22 0 2024-03-22T07:15:50.750 INFO:tasks.workunit.client.0.smithi063.stdout:5/109: write d8/f1c [1019488,82103] 0 2024-03-22T07:15:50.757 INFO:tasks.workunit.client.0.smithi063.stdout:2/113: write d0/f4 [743110,15882] 0 2024-03-22T07:15:50.758 INFO:tasks.workunit.client.0.smithi063.stdout:2/114: mkdir d0/de/d12/d20 0 2024-03-22T07:15:50.759 INFO:tasks.workunit.client.0.smithi063.stdout:5/110: dread d8/f14 [0,4194304] 0 2024-03-22T07:15:50.760 INFO:tasks.workunit.client.0.smithi063.stdout:2/115: mknod d0/c21 0 2024-03-22T07:15:50.769 INFO:tasks.workunit.client.0.smithi063.stdout:7/114: dwrite d5/d14/f1a [4194304,4194304] 0 2024-03-22T07:15:50.769 INFO:tasks.workunit.client.0.smithi063.stdout:7/115: readlink l1 0 2024-03-22T07:15:50.769 INFO:tasks.workunit.client.0.smithi063.stdout:7/116: dread - d5/d16/f22 zero size 2024-03-22T07:15:50.769 INFO:tasks.workunit.client.0.smithi063.stdout:8/117: dwrite d0/f17 [0,4194304] 0 2024-03-22T07:15:50.770 INFO:tasks.workunit.client.0.smithi063.stdout:7/117: rmdir d5/d14 39 2024-03-22T07:15:50.770 INFO:tasks.workunit.client.0.smithi063.stdout:4/60: dwrite d1/d5/dc/f10 [0,4194304] 0 2024-03-22T07:15:50.770 INFO:tasks.workunit.client.0.smithi063.stdout:4/61: dread - d1/d5/dc/ff zero size 2024-03-22T07:15:50.772 INFO:tasks.workunit.client.0.smithi063.stdout:2/116: write d0/fa [8289739,6725] 0 2024-03-22T07:15:50.773 INFO:tasks.workunit.client.0.smithi063.stdout:0/115: sync 2024-03-22T07:15:50.773 INFO:tasks.workunit.client.0.smithi063.stdout:6/76: sync 2024-03-22T07:15:50.773 INFO:tasks.workunit.client.0.smithi063.stdout:6/77: chown d3/d6/f10 245 1 2024-03-22T07:15:50.773 INFO:tasks.workunit.client.0.smithi063.stdout:6/78: truncate d3/d6/f11 703355 0 2024-03-22T07:15:50.774 INFO:tasks.workunit.client.0.smithi063.stdout:9/60: sync 2024-03-22T07:15:50.774 INFO:tasks.workunit.client.0.smithi063.stdout:1/120: sync 2024-03-22T07:15:50.776 INFO:tasks.workunit.client.0.smithi063.stdout:2/117: fdatasync d0/fa 0 2024-03-22T07:15:50.776 INFO:tasks.workunit.client.0.smithi063.stdout:7/118: mkdir d5/da/d28 0 2024-03-22T07:15:50.779 INFO:tasks.workunit.client.0.smithi063.stdout:4/62: creat d1/f11 x:0 0 0 2024-03-22T07:15:50.781 INFO:tasks.workunit.client.0.smithi063.stdout:0/116: rename da/f11 to da/de/f23 0 2024-03-22T07:15:50.781 INFO:tasks.workunit.client.0.smithi063.stdout:0/117: chown l7 65417252 1 2024-03-22T07:15:50.782 INFO:tasks.workunit.client.0.smithi063.stdout:6/79: dread d3/f4 [0,4194304] 0 2024-03-22T07:15:50.782 INFO:tasks.workunit.client.0.smithi063.stdout:6/80: truncate d3/f19 1563163 0 2024-03-22T07:15:50.782 INFO:tasks.workunit.client.0.smithi063.stdout:9/61: creat d8/f13 x:0 0 0 2024-03-22T07:15:50.784 INFO:tasks.workunit.client.0.smithi063.stdout:1/121: mkdir d4/df/d13/d2b 0 2024-03-22T07:15:50.784 INFO:tasks.workunit.client.0.smithi063.stdout:1/122: truncate d4/d14/f23 679342 0 2024-03-22T07:15:50.784 INFO:tasks.workunit.client.0.smithi063.stdout:5/111: dwrite d8/f14 [0,4194304] 0 2024-03-22T07:15:50.786 INFO:tasks.workunit.client.0.smithi063.stdout:8/118: rmdir d0/d26 39 2024-03-22T07:15:50.786 INFO:tasks.workunit.client.0.smithi063.stdout:8/119: chown d0/d18/c1c 84298 1 2024-03-22T07:15:50.786 INFO:tasks.workunit.client.0.smithi063.stdout:8/120: readlink d0/l10 0 2024-03-22T07:15:50.787 INFO:tasks.workunit.client.0.smithi063.stdout:7/119: mkdir d5/d14/d1d/d29 0 2024-03-22T07:15:50.789 INFO:tasks.workunit.client.0.smithi063.stdout:2/118: dread d0/de/f15 [0,4194304] 0 2024-03-22T07:15:50.789 INFO:tasks.workunit.client.0.smithi063.stdout:2/119: write d0/f10 [248728,123634] 0 2024-03-22T07:15:50.789 INFO:tasks.workunit.client.0.smithi063.stdout:4/63: rename f0 to d1/d5/f12 0 2024-03-22T07:15:50.791 INFO:tasks.workunit.client.0.smithi063.stdout:0/118: unlink da/f13 0 2024-03-22T07:15:50.791 INFO:tasks.workunit.client.0.smithi063.stdout:2/120: dread d0/f5 [0,4194304] 0 2024-03-22T07:15:50.793 INFO:tasks.workunit.client.0.smithi063.stdout:6/81: mkdir d3/d15/d16/d1b 0 2024-03-22T07:15:50.796 INFO:tasks.workunit.client.0.smithi063.stdout:1/123: mkdir d4/df/d2c 0 2024-03-22T07:15:50.797 INFO:tasks.workunit.client.0.smithi063.stdout:1/124: write d4/d14/f23 [1242176,40580] 0 2024-03-22T07:15:50.797 INFO:tasks.workunit.client.0.smithi063.stdout:1/125: chown d4/d14/f23 12673315 1 2024-03-22T07:15:50.797 INFO:tasks.workunit.client.0.smithi063.stdout:1/126: readlink d4/l16 0 2024-03-22T07:15:50.797 INFO:tasks.workunit.client.0.smithi063.stdout:1/127: write d4/d14/f23 [1578218,41251] 0 2024-03-22T07:15:50.798 INFO:tasks.workunit.client.0.smithi063.stdout:8/121: unlink d0/d1f/c21 0 2024-03-22T07:15:50.798 INFO:tasks.workunit.client.0.smithi063.stdout:8/122: chown d0/l3 26 1 2024-03-22T07:15:50.800 INFO:tasks.workunit.client.0.smithi063.stdout:1/128: dread d4/df/f15 [0,4194304] 0 2024-03-22T07:15:50.801 INFO:tasks.workunit.client.0.smithi063.stdout:1/129: write d4/df/f15 [2405044,48594] 0 2024-03-22T07:15:50.801 INFO:tasks.workunit.client.0.smithi063.stdout:7/120: creat d5/d16/d1b/d21/f2a x:0 0 0 2024-03-22T07:15:50.803 INFO:tasks.workunit.client.0.smithi063.stdout:9/62: fsync d8/f13 0 2024-03-22T07:15:50.803 INFO:tasks.workunit.client.0.smithi063.stdout:9/63: dread - d8/f13 zero size 2024-03-22T07:15:50.804 INFO:tasks.workunit.client.0.smithi063.stdout:9/64: dread d8/fb [0,4194304] 0 2024-03-22T07:15:50.810 INFO:tasks.workunit.client.0.smithi063.stdout:6/82: mkdir d3/d1c 0 2024-03-22T07:15:50.811 INFO:tasks.workunit.client.0.smithi063.stdout:8/123: write d0/f17 [1957727,5256] 0 2024-03-22T07:15:50.811 INFO:tasks.workunit.client.0.smithi063.stdout:8/124: fdatasync d0/f1a 0 2024-03-22T07:15:50.811 INFO:tasks.workunit.client.0.smithi063.stdout:8/125: chown d0/d1b 1 1 2024-03-22T07:15:50.811 INFO:tasks.workunit.client.0.smithi063.stdout:1/130: symlink d4/df/d13/l2d 0 2024-03-22T07:15:50.812 INFO:tasks.workunit.client.0.smithi063.stdout:9/65: unlink d8/l11 0 2024-03-22T07:15:50.813 INFO:tasks.workunit.client.0.smithi063.stdout:6/83: link d3/f5 d3/d1c/f1d 0 2024-03-22T07:15:50.814 INFO:tasks.workunit.client.0.smithi063.stdout:7/121: dread d5/f9 [0,4194304] 0 2024-03-22T07:15:50.818 INFO:tasks.workunit.client.0.smithi063.stdout:7/122: fsync d5/d16/d1b/d21/f2a 0 2024-03-22T07:15:50.819 INFO:tasks.workunit.client.0.smithi063.stdout:1/131: write d4/d14/f1b [3125891,120615] 0 2024-03-22T07:15:50.821 INFO:tasks.workunit.client.0.smithi063.stdout:4/64: dwrite d1/d5/f9 [0,4194304] 0 2024-03-22T07:15:50.821 INFO:tasks.workunit.client.0.smithi063.stdout:4/65: fsync d1/d5/dc/f10 0 2024-03-22T07:15:50.821 INFO:tasks.workunit.client.0.smithi063.stdout:2/121: dwrite d0/de/f15 [0,4194304] 0 2024-03-22T07:15:50.821 INFO:tasks.workunit.client.0.smithi063.stdout:2/122: fdatasync d0/de/f15 0 2024-03-22T07:15:50.821 INFO:tasks.workunit.client.0.smithi063.stdout:9/66: rmdir d8 39 2024-03-22T07:15:50.822 INFO:tasks.workunit.client.0.smithi063.stdout:8/126: write d0/d1b/f1e [3965087,86323] 0 2024-03-22T07:15:50.825 INFO:tasks.workunit.client.0.smithi063.stdout:2/123: creat d0/de/f22 x:0 0 0 2024-03-22T07:15:50.828 INFO:tasks.workunit.client.0.smithi063.stdout:9/67: link d8/fa d8/f14 0 2024-03-22T07:15:50.828 INFO:tasks.workunit.client.0.smithi063.stdout:9/68: chown f3 902 1 2024-03-22T07:15:50.828 INFO:tasks.workunit.client.0.smithi063.stdout:9/69: chown f2 4 1 2024-03-22T07:15:50.831 INFO:tasks.workunit.client.0.smithi063.stdout:1/132: dread d4/fb [0,4194304] 0 2024-03-22T07:15:50.832 INFO:tasks.workunit.client.0.smithi063.stdout:8/127: mkdir d0/d1b/d28 0 2024-03-22T07:15:50.832 INFO:tasks.workunit.client.0.smithi063.stdout:8/128: stat d0/f22 0 2024-03-22T07:15:50.832 INFO:tasks.workunit.client.0.smithi063.stdout:4/66: mkdir d1/d13 0 2024-03-22T07:15:50.832 INFO:tasks.workunit.client.0.smithi063.stdout:4/67: write d1/f11 [301111,52854] 0 2024-03-22T07:15:50.832 INFO:tasks.workunit.client.0.smithi063.stdout:7/123: write d5/d14/f1a [277689,126132] 0 2024-03-22T07:15:50.832 INFO:tasks.workunit.client.0.smithi063.stdout:7/124: creat d5/d16/f2b x:0 0 0 2024-03-22T07:15:50.833 INFO:tasks.workunit.client.0.smithi063.stdout:9/70: mkdir d8/d15 0 2024-03-22T07:15:50.833 INFO:tasks.workunit.client.0.smithi063.stdout:9/71: read - d8/fa zero size 2024-03-22T07:15:50.834 INFO:tasks.workunit.client.0.smithi063.stdout:1/133: rename d4/l26 to d4/df/d13/d2b/l2e 0 2024-03-22T07:15:50.835 INFO:tasks.workunit.client.0.smithi063.stdout:8/129: creat d0/d24/f29 x:0 0 0 2024-03-22T07:15:50.837 INFO:tasks.workunit.client.0.smithi063.stdout:4/68: creat d1/d13/f14 x:0 0 0 2024-03-22T07:15:50.840 INFO:tasks.workunit.client.0.smithi063.stdout:2/124: rmdir d0/de 39 2024-03-22T07:15:50.840 INFO:tasks.workunit.client.0.smithi063.stdout:2/125: readlink d0/l7 0 2024-03-22T07:15:50.841 INFO:tasks.workunit.client.0.smithi063.stdout:2/126: read d0/de/f14 [24303,1065] 0 2024-03-22T07:15:50.841 INFO:tasks.workunit.client.0.smithi063.stdout:2/127: dread - d0/de/f22 zero size 2024-03-22T07:15:50.854 INFO:tasks.workunit.client.0.smithi063.stdout:7/125: rename d5/d16/d1b/f1f to d5/da/f2c 0 2024-03-22T07:15:50.854 INFO:tasks.workunit.client.0.smithi063.stdout:8/130: mknod d0/c2a 0 2024-03-22T07:15:50.855 INFO:tasks.workunit.client.0.smithi063.stdout:4/69: truncate d1/d5/f12 3739157 0 2024-03-22T07:15:50.855 INFO:tasks.workunit.client.0.smithi063.stdout:4/70: dread - d1/d13/f14 zero size 2024-03-22T07:15:50.856 INFO:tasks.workunit.client.0.smithi063.stdout:4/71: chown d1/f3 580682057 1 2024-03-22T07:15:50.856 INFO:tasks.workunit.client.0.smithi063.stdout:8/131: unlink d0/f5 0 2024-03-22T07:15:50.858 INFO:tasks.workunit.client.0.smithi063.stdout:7/126: dread d5/f9 [0,4194304] 0 2024-03-22T07:15:50.858 INFO:tasks.workunit.client.0.smithi063.stdout:7/127: creat d5/d16/d1b/f2d x:0 0 0 2024-03-22T07:15:50.858 INFO:tasks.workunit.client.0.smithi063.stdout:7/128: stat d5/d16/d1b/c23 0 2024-03-22T07:15:50.863 INFO:tasks.workunit.client.0.smithi063.stdout:8/132: dread d0/fc [0,4194304] 0 2024-03-22T07:15:50.866 INFO:tasks.workunit.client.0.smithi063.stdout:1/134: dwrite d4/d14/f18 [0,4194304] 0 2024-03-22T07:15:50.868 INFO:tasks.workunit.client.0.smithi063.stdout:8/133: write d0/f12 [2304165,50198] 0 2024-03-22T07:15:50.869 INFO:tasks.workunit.client.0.smithi063.stdout:8/134: symlink d0/d26/l2b 0 2024-03-22T07:15:50.871 INFO:tasks.workunit.client.0.smithi063.stdout:8/135: rename d0/c2a to d0/d26/c2c 0 2024-03-22T07:15:50.871 INFO:tasks.workunit.client.0.smithi063.stdout:8/136: readlink d0/d18/l23 0 2024-03-22T07:15:50.871 INFO:tasks.workunit.client.0.smithi063.stdout:8/137: write d0/d1b/f1e [4971536,96624] 0 2024-03-22T07:15:50.871 INFO:tasks.workunit.client.0.smithi063.stdout:1/135: dread d4/df/f29 [0,4194304] 0 2024-03-22T07:15:50.872 INFO:tasks.workunit.client.0.smithi063.stdout:1/136: write d4/df/d13/f25 [122267,28713] 0 2024-03-22T07:15:50.872 INFO:tasks.workunit.client.0.smithi063.stdout:1/137: stat d4/df/d13 0 2024-03-22T07:15:50.874 INFO:tasks.workunit.client.0.smithi063.stdout:8/138: dread d0/d1b/f20 [0,4194304] 0 2024-03-22T07:15:50.874 INFO:tasks.workunit.client.0.smithi063.stdout:9/72: dwrite d8/fc [0,4194304] 0 2024-03-22T07:15:50.876 INFO:tasks.workunit.client.0.smithi063.stdout:2/128: dwrite d0/de/d12/f1f [0,4194304] 0 2024-03-22T07:15:50.876 INFO:tasks.workunit.client.0.smithi063.stdout:4/72: dwrite d1/f3 [0,4194304] 0 2024-03-22T07:15:50.876 INFO:tasks.workunit.client.0.smithi063.stdout:4/73: fsync d1/d13/f14 0 2024-03-22T07:15:50.876 INFO:tasks.workunit.client.0.smithi063.stdout:4/74: truncate d1/d13/f14 386212 0 2024-03-22T07:15:50.881 INFO:tasks.workunit.client.1.smithi069.stdout:3/18: rmdir d5 39 2024-03-22T07:15:50.886 INFO:tasks.workunit.client.0.smithi063.stdout:2/129: dread d0/de/f15 [0,4194304] 0 2024-03-22T07:15:50.886 INFO:tasks.workunit.client.0.smithi063.stdout:8/139: dread d0/d1b/f1e [0,4194304] 0 2024-03-22T07:15:50.927 INFO:tasks.workunit.client.0.smithi063.stdout:1/138: rename d4/cd to d4/d14/c2f 0 2024-03-22T07:15:50.927 INFO:tasks.workunit.client.0.smithi063.stdout:1/139: fsync d4/d14/f21 0 2024-03-22T07:15:50.928 INFO:tasks.workunit.client.0.smithi063.stdout:9/73: mkdir d8/d16 0 2024-03-22T07:15:50.930 INFO:tasks.workunit.client.0.smithi063.stdout:9/74: dread d8/f10 [0,4194304] 0 2024-03-22T07:15:50.933 INFO:tasks.workunit.client.0.smithi063.stdout:4/75: symlink d1/d13/l15 0 2024-03-22T07:15:50.933 INFO:tasks.workunit.client.0.smithi063.stdout:4/76: fdatasync d1/d5/f12 0 2024-03-22T07:15:50.934 INFO:tasks.workunit.client.0.smithi063.stdout:2/130: creat d0/f23 x:0 0 0 2024-03-22T07:15:50.936 INFO:tasks.workunit.client.0.smithi063.stdout:8/140: mkdir d0/d26/d2d 0 2024-03-22T07:15:50.939 INFO:tasks.workunit.client.0.smithi063.stdout:9/75: mkdir d8/d17 0 2024-03-22T07:15:50.939 INFO:tasks.workunit.client.0.smithi063.stdout:9/76: write d8/fb [1150512,27489] 0 2024-03-22T07:15:50.940 INFO:tasks.workunit.client.0.smithi063.stdout:2/131: creat d0/f24 x:0 0 0 2024-03-22T07:15:50.940 INFO:tasks.workunit.client.0.smithi063.stdout:4/77: read d1/d5/f12 [1477048,108780] 0 2024-03-22T07:15:50.940 INFO:tasks.workunit.client.0.smithi063.stdout:4/78: fdatasync d1/d5/f9 0 2024-03-22T07:15:50.940 INFO:tasks.workunit.client.0.smithi063.stdout:4/79: fdatasync d1/f3 0 2024-03-22T07:15:50.942 INFO:tasks.workunit.client.0.smithi063.stdout:8/141: creat d0/d26/f2e x:0 0 0 2024-03-22T07:15:50.943 INFO:tasks.workunit.client.1.smithi069.stdout:6/3: creat f0 x:0 0 0 2024-03-22T07:15:50.943 INFO:tasks.workunit.client.1.smithi069.stdout:7/8: creat f0 x:0 0 0 2024-03-22T07:15:50.943 INFO:tasks.workunit.client.1.smithi069.stdout:7/9: dread - f0 zero size 2024-03-22T07:15:50.943 INFO:tasks.workunit.client.1.smithi069.stdout:7/10: fdatasync f0 0 2024-03-22T07:15:50.943 INFO:tasks.workunit.client.1.smithi069.stdout:7/11: chown f0 239 1 2024-03-22T07:15:50.944 INFO:tasks.workunit.client.1.smithi069.stdout:7/12: stat f0 0 2024-03-22T07:15:50.944 INFO:tasks.workunit.client.1.smithi069.stdout:9/0: mknod c0 0 2024-03-22T07:15:50.944 INFO:tasks.workunit.client.1.smithi069.stdout:9/1: stat c0 0 2024-03-22T07:15:50.944 INFO:tasks.workunit.client.0.smithi063.stdout:1/140: dwrite d4/f1d [0,4194304] 0 2024-03-22T07:15:50.947 INFO:tasks.workunit.client.0.smithi063.stdout:9/77: mknod d8/d16/c18 0 2024-03-22T07:15:50.949 INFO:tasks.workunit.client.0.smithi063.stdout:4/80: mknod d1/d5/dc/c16 0 2024-03-22T07:15:50.961 INFO:tasks.workunit.client.0.smithi063.stdout:1/141: truncate d4/df/f11 1330163 0 2024-03-22T07:15:50.961 INFO:tasks.workunit.client.0.smithi063.stdout:1/142: write d4/f9 [423923,124450] 0 2024-03-22T07:15:50.962 INFO:tasks.workunit.client.0.smithi063.stdout:9/78: mkdir d8/d19 0 2024-03-22T07:15:50.963 INFO:tasks.workunit.client.0.smithi063.stdout:9/79: write d8/fb [692883,91776] 0 2024-03-22T07:15:50.963 INFO:tasks.workunit.client.0.smithi063.stdout:2/132: symlink d0/l25 0 2024-03-22T07:15:50.963 INFO:tasks.workunit.client.0.smithi063.stdout:9/80: write f3 [1419333,118386] 0 2024-03-22T07:15:50.963 INFO:tasks.workunit.client.0.smithi063.stdout:9/81: dread - d8/f13 zero size 2024-03-22T07:15:50.964 INFO:tasks.workunit.client.0.smithi063.stdout:1/143: creat d4/d14/f30 x:0 0 0 2024-03-22T07:15:50.964 INFO:tasks.workunit.client.0.smithi063.stdout:2/133: mknod d0/de/d12/c26 0 2024-03-22T07:15:50.965 INFO:tasks.workunit.client.0.smithi063.stdout:9/82: rmdir d8 39 2024-03-22T07:15:50.965 INFO:tasks.workunit.client.0.smithi063.stdout:9/83: chown c6 127214470 1 2024-03-22T07:15:50.966 INFO:tasks.workunit.client.0.smithi063.stdout:1/144: read d4/d14/f23 [677082,56949] 0 2024-03-22T07:15:50.966 INFO:tasks.workunit.client.0.smithi063.stdout:1/145: dread - d4/d14/f30 zero size 2024-03-22T07:15:50.967 INFO:tasks.workunit.client.0.smithi063.stdout:2/134: creat d0/de/d12/f27 x:0 0 0 2024-03-22T07:15:50.967 INFO:tasks.workunit.client.0.smithi063.stdout:9/84: rmdir d8 39 2024-03-22T07:15:50.969 INFO:tasks.workunit.client.0.smithi063.stdout:1/146: mknod d4/d14/c31 0 2024-03-22T07:15:50.969 INFO:tasks.workunit.client.0.smithi063.stdout:2/135: creat d0/de/d12/d20/f28 x:0 0 0 2024-03-22T07:15:50.970 INFO:tasks.workunit.client.1.smithi069.stdout:7/13: dwrite f0 [0,4194304] 0 2024-03-22T07:15:50.971 INFO:tasks.workunit.client.0.smithi063.stdout:1/147: creat d4/f32 x:0 0 0 2024-03-22T07:15:50.971 INFO:tasks.workunit.client.0.smithi063.stdout:1/148: truncate d4/df/d13/f27 669810 0 2024-03-22T07:15:50.971 INFO:tasks.workunit.client.0.smithi063.stdout:1/149: dread - d4/d14/f21 zero size 2024-03-22T07:15:50.972 INFO:tasks.workunit.client.0.smithi063.stdout:9/85: dread f7 [0,4194304] 0 2024-03-22T07:15:50.973 INFO:tasks.workunit.client.0.smithi063.stdout:1/150: mknod d4/df/c33 0 2024-03-22T07:15:50.980 INFO:tasks.workunit.client.0.smithi063.stdout:4/81: dwrite d1/f2 [4194304,4194304] 0 2024-03-22T07:15:50.980 INFO:tasks.workunit.client.0.smithi063.stdout:4/82: creat d1/d13/f17 x:0 0 0 2024-03-22T07:15:50.980 INFO:tasks.workunit.client.0.smithi063.stdout:9/86: fdatasync f3 0 2024-03-22T07:15:50.980 INFO:tasks.workunit.client.0.smithi063.stdout:9/87: chown d8/d17 30752 1 2024-03-22T07:15:50.981 INFO:tasks.workunit.client.0.smithi063.stdout:9/88: chown f3 2585691 1 2024-03-22T07:15:50.983 INFO:tasks.workunit.client.0.smithi063.stdout:8/142: dwrite d0/f19 [0,4194304] 0 2024-03-22T07:15:50.988 INFO:tasks.workunit.client.0.smithi063.stdout:2/136: getdents d0/de/d12 0 2024-03-22T07:15:50.990 INFO:tasks.workunit.client.0.smithi063.stdout:9/89: creat d8/f1a x:0 0 0 2024-03-22T07:15:50.991 INFO:tasks.workunit.client.0.smithi063.stdout:4/83: write d1/d5/f9 [1404019,60294] 0 2024-03-22T07:15:50.996 INFO:tasks.workunit.client.0.smithi063.stdout:1/151: dwrite d4/f1e [0,4194304] 0 2024-03-22T07:15:50.996 INFO:tasks.workunit.client.0.smithi063.stdout:1/152: fsync d4/f1d 0 2024-03-22T07:15:50.996 INFO:tasks.workunit.client.0.smithi063.stdout:8/143: read d0/f12 [3747752,78954] 0 2024-03-22T07:15:50.997 INFO:tasks.workunit.client.0.smithi063.stdout:2/137: link d0/de/d12/f27 d0/de/d12/d20/f29 0 2024-03-22T07:15:50.998 INFO:tasks.workunit.client.0.smithi063.stdout:9/90: symlink d8/d17/l1b 0 2024-03-22T07:15:50.998 INFO:tasks.workunit.client.0.smithi063.stdout:9/91: dread - d8/f14 zero size 2024-03-22T07:15:50.999 INFO:tasks.workunit.client.0.smithi063.stdout:9/92: fdatasync d8/f10 0 2024-03-22T07:15:50.999 INFO:tasks.workunit.client.0.smithi063.stdout:9/93: chown d8/d17/l1b 1408388 1 2024-03-22T07:15:51.002 INFO:tasks.workunit.client.0.smithi063.stdout:1/153: creat d4/df/d13/f34 x:0 0 0 2024-03-22T07:15:51.003 INFO:tasks.workunit.client.0.smithi063.stdout:4/84: rmdir d1/d13 39 2024-03-22T07:15:51.003 INFO:tasks.workunit.client.0.smithi063.stdout:4/85: chown d1/d5/dc 31374 1 2024-03-22T07:15:51.006 INFO:tasks.workunit.client.0.smithi063.stdout:1/154: rename d4/l6 to d4/l35 0 2024-03-22T07:15:51.006 INFO:tasks.workunit.client.0.smithi063.stdout:1/155: write d4/df/f29 [2174978,28630] 0 2024-03-22T07:15:51.022 INFO:tasks.workunit.client.0.smithi063.stdout:8/144: mknod d0/d1b/d28/c2f 0 2024-03-22T07:15:51.023 INFO:tasks.workunit.client.0.smithi063.stdout:4/86: mknod d1/d5/c18 0 2024-03-22T07:15:51.023 INFO:tasks.workunit.client.0.smithi063.stdout:1/156: creat d4/df/d2c/f36 x:0 0 0 2024-03-22T07:15:51.024 INFO:tasks.workunit.client.0.smithi063.stdout:2/138: dwrite d0/f23 [0,4194304] 0 2024-03-22T07:15:51.024 INFO:tasks.workunit.client.0.smithi063.stdout:2/139: dread - d0/f24 zero size 2024-03-22T07:15:51.024 INFO:tasks.workunit.client.0.smithi063.stdout:2/140: write d0/f5 [2327989,56086] 0 2024-03-22T07:15:51.024 INFO:tasks.workunit.client.0.smithi063.stdout:2/141: write d0/f4 [2280846,625] 0 2024-03-22T07:15:51.026 INFO:tasks.workunit.client.0.smithi063.stdout:1/157: symlink d4/d14/l37 0 2024-03-22T07:15:51.026 INFO:tasks.workunit.client.0.smithi063.stdout:1/158: stat d4/f1d 0 2024-03-22T07:15:51.026 INFO:tasks.workunit.client.0.smithi063.stdout:3/59: sync 2024-03-22T07:15:51.026 INFO:tasks.workunit.client.0.smithi063.stdout:6/84: sync 2024-03-22T07:15:51.026 INFO:tasks.workunit.client.0.smithi063.stdout:5/112: sync 2024-03-22T07:15:51.026 INFO:tasks.workunit.client.0.smithi063.stdout:0/119: sync 2024-03-22T07:15:51.026 INFO:tasks.workunit.client.0.smithi063.stdout:5/113: readlink d8/d13/l19 0 2024-03-22T07:15:51.026 INFO:tasks.workunit.client.0.smithi063.stdout:3/60: fdatasync da/f10 0 2024-03-22T07:15:51.026 INFO:tasks.workunit.client.0.smithi063.stdout:0/120: write da/ff [1222270,80773] 0 2024-03-22T07:15:51.027 INFO:tasks.workunit.client.0.smithi063.stdout:0/121: write f9 [4254918,80588] 0 2024-03-22T07:15:51.032 INFO:tasks.workunit.client.0.smithi063.stdout:4/87: write d1/f3 [2650281,37628] 0 2024-03-22T07:15:51.032 INFO:tasks.workunit.client.0.smithi063.stdout:8/145: mknod d0/d24/c30 0 2024-03-22T07:15:51.032 INFO:tasks.workunit.client.0.smithi063.stdout:4/88: write d1/d13/f14 [65183,101379] 0 2024-03-22T07:15:51.034 INFO:tasks.workunit.client.0.smithi063.stdout:1/159: dread d4/f1f [0,4194304] 0 2024-03-22T07:15:51.035 INFO:tasks.workunit.client.0.smithi063.stdout:2/142: link d0/f5 d0/de/d12/d20/f2a 0 2024-03-22T07:15:51.041 INFO:tasks.workunit.client.0.smithi063.stdout:3/61: symlink da/l14 0 2024-03-22T07:15:51.043 INFO:tasks.workunit.client.0.smithi063.stdout:0/122: link da/de/f23 da/de/f24 0 2024-03-22T07:15:51.044 INFO:tasks.workunit.client.0.smithi063.stdout:8/146: readlink d0/d18/l23 0 2024-03-22T07:15:51.046 INFO:tasks.workunit.client.0.smithi063.stdout:6/85: dwrite d3/f9 [4194304,4194304] 0 2024-03-22T07:15:51.046 INFO:tasks.workunit.client.0.smithi063.stdout:4/89: creat d1/d13/f19 x:0 0 0 2024-03-22T07:15:51.046 INFO:tasks.workunit.client.0.smithi063.stdout:4/90: chown d1/d5/dc/f10 551388771 1 2024-03-22T07:15:51.047 INFO:tasks.workunit.client.0.smithi063.stdout:2/143: symlink d0/de/d12/l2b 0 2024-03-22T07:15:51.047 INFO:tasks.workunit.client.0.smithi063.stdout:2/144: chown d0/l25 1039304343 1 2024-03-22T07:15:51.047 INFO:tasks.workunit.client.0.smithi063.stdout:2/145: chown d0/de 5433382 1 2024-03-22T07:15:51.047 INFO:tasks.workunit.client.0.smithi063.stdout:2/146: creat d0/de/d12/d20/f2c x:0 0 0 2024-03-22T07:15:51.047 INFO:tasks.workunit.client.0.smithi063.stdout:2/147: write d0/fa [6858889,47592] 0 2024-03-22T07:15:51.049 INFO:tasks.workunit.client.0.smithi063.stdout:5/114: rename d8/l1f to d8/l23 0 2024-03-22T07:15:51.051 INFO:tasks.workunit.client.0.smithi063.stdout:3/62: write da/f10 [21224,11292] 0 2024-03-22T07:15:51.052 INFO:tasks.workunit.client.0.smithi063.stdout:6/86: mkdir d3/d15/d16/d1e 0 2024-03-22T07:15:51.053 INFO:tasks.workunit.client.0.smithi063.stdout:4/91: getdents d1/d5/dc 0 2024-03-22T07:15:51.053 INFO:tasks.workunit.client.0.smithi063.stdout:4/92: fdatasync d1/f3 0 2024-03-22T07:15:51.053 INFO:tasks.workunit.client.0.smithi063.stdout:4/93: chown d1/d5/la 748723420 1 2024-03-22T07:15:51.054 INFO:tasks.workunit.client.0.smithi063.stdout:1/160: dwrite d4/d14/f2a [0,4194304] 0 2024-03-22T07:15:51.054 INFO:tasks.workunit.client.0.smithi063.stdout:8/147: dread d0/f12 [0,4194304] 0 2024-03-22T07:15:51.054 INFO:tasks.workunit.client.0.smithi063.stdout:1/161: readlink d4/df/d13/l2d 0 2024-03-22T07:15:51.056 INFO:tasks.workunit.client.0.smithi063.stdout:5/115: mkdir d8/d13/d24 0 2024-03-22T07:15:51.056 INFO:tasks.workunit.client.0.smithi063.stdout:5/116: write d8/f21 [415042,116800] 0 2024-03-22T07:15:51.056 INFO:tasks.workunit.client.0.smithi063.stdout:3/63: truncate da/fb 1824805 0 2024-03-22T07:15:51.057 INFO:tasks.workunit.client.0.smithi063.stdout:6/87: rmdir d3 39 2024-03-22T07:15:51.061 INFO:tasks.workunit.client.0.smithi063.stdout:4/94: symlink d1/d5/dc/l1a 0 2024-03-22T07:15:51.061 INFO:tasks.workunit.client.0.smithi063.stdout:4/95: chown d1/c4 160 1 2024-03-22T07:15:51.061 INFO:tasks.workunit.client.0.smithi063.stdout:4/96: readlink d1/d13/l15 0 2024-03-22T07:15:51.066 INFO:tasks.workunit.client.0.smithi063.stdout:1/162: link d4/d14/f18 d4/f38 0 2024-03-22T07:15:51.066 INFO:tasks.workunit.client.0.smithi063.stdout:1/163: chown d4/df/d13/l20 365932968 1 2024-03-22T07:15:51.067 INFO:tasks.workunit.client.0.smithi063.stdout:2/148: mknod d0/c2d 0 2024-03-22T07:15:51.067 INFO:tasks.workunit.client.0.smithi063.stdout:3/64: symlink da/l15 0 2024-03-22T07:15:51.069 INFO:tasks.workunit.client.0.smithi063.stdout:6/88: mknod d3/d15/c1f 0 2024-03-22T07:15:51.069 INFO:tasks.workunit.client.0.smithi063.stdout:2/149: write d0/f9 [81429,1427] 0 2024-03-22T07:15:51.074 INFO:tasks.workunit.client.0.smithi063.stdout:1/164: write d4/fb [1661671,111328] 0 2024-03-22T07:15:51.075 INFO:tasks.workunit.client.0.smithi063.stdout:6/89: creat d3/d15/d16/d1e/f20 x:0 0 0 2024-03-22T07:15:51.076 INFO:tasks.workunit.client.0.smithi063.stdout:6/90: getdents d3/d15/d16/d1b 0 2024-03-22T07:15:51.076 INFO:tasks.workunit.client.0.smithi063.stdout:6/91: readlink d3/lf 0 2024-03-22T07:15:51.106 INFO:tasks.workunit.client.0.smithi063.stdout:1/165: dread d4/f12 [0,4194304] 0 2024-03-22T07:15:51.106 INFO:tasks.workunit.client.0.smithi063.stdout:6/92: dread d3/f18 [0,4194304] 0 2024-03-22T07:15:51.110 INFO:tasks.workunit.client.0.smithi063.stdout:3/65: dread da/fb [0,4194304] 0 2024-03-22T07:15:51.111 INFO:tasks.workunit.client.0.smithi063.stdout:6/93: mknod d3/d15/d16/d1e/c21 0 2024-03-22T07:15:51.111 INFO:tasks.workunit.client.0.smithi063.stdout:6/94: rmdir d3/d15/d16/d1b 0 2024-03-22T07:15:51.111 INFO:tasks.workunit.client.0.smithi063.stdout:6/95: creat d3/d15/f22 x:0 0 0 2024-03-22T07:15:51.112 INFO:tasks.workunit.client.1.smithi069.stdout:2/14: rename f2 to f5 0 2024-03-22T07:15:51.112 INFO:tasks.workunit.client.1.smithi069.stdout:2/15: dread - f1 zero size 2024-03-22T07:15:51.112 INFO:tasks.workunit.client.1.smithi069.stdout:2/16: creat f6 x:0 0 0 2024-03-22T07:15:51.112 INFO:tasks.workunit.client.1.smithi069.stdout:2/17: rmdir - no directory 2024-03-22T07:15:51.112 INFO:tasks.workunit.client.1.smithi069.stdout:3/19: creat d5/f7 x:0 0 0 2024-03-22T07:15:51.113 INFO:tasks.workunit.client.1.smithi069.stdout:3/20: write d5/f7 [649364,59934] 0 2024-03-22T07:15:51.114 INFO:tasks.workunit.client.1.smithi069.stdout:6/4: link f0 f1 0 2024-03-22T07:15:51.114 INFO:tasks.workunit.client.1.smithi069.stdout:6/5: truncate f0 367610 0 2024-03-22T07:15:51.114 INFO:tasks.workunit.client.1.smithi069.stdout:6/6: readlink - no filename 2024-03-22T07:15:51.114 INFO:tasks.workunit.client.0.smithi063.stdout:5/117: dwrite d8/f14 [0,4194304] 0 2024-03-22T07:15:51.114 INFO:tasks.workunit.client.0.smithi063.stdout:6/96: dread d3/f4 [0,4194304] 0 2024-03-22T07:15:51.115 INFO:tasks.workunit.client.0.smithi063.stdout:6/97: creat d3/d15/f23 x:0 0 0 2024-03-22T07:15:51.115 INFO:tasks.workunit.client.1.smithi069.stdout:9/2: link c0 c1 0 2024-03-22T07:15:51.117 INFO:tasks.workunit.client.0.smithi063.stdout:6/98: mknod d3/d15/d16/d1e/c24 0 2024-03-22T07:15:51.118 INFO:tasks.workunit.client.1.smithi069.stdout:4/17: rename f4 to f5 0 2024-03-22T07:15:51.118 INFO:tasks.workunit.client.0.smithi063.stdout:6/99: creat d3/d15/d16/f25 x:0 0 0 2024-03-22T07:15:51.118 INFO:tasks.workunit.client.0.smithi063.stdout:6/100: truncate d3/d15/f22 681286 0 2024-03-22T07:15:51.121 INFO:tasks.workunit.client.1.smithi069.stdout:3/21: mknod d5/c8 0 2024-03-22T07:15:51.122 INFO:tasks.workunit.client.0.smithi063.stdout:8/148: dwrite d0/f1a [0,4194304] 0 2024-03-22T07:15:51.122 INFO:tasks.workunit.client.1.smithi069.stdout:6/7: rename f0 to f2 0 2024-03-22T07:15:51.123 INFO:tasks.workunit.client.1.smithi069.stdout:6/8: rmdir - no directory 2024-03-22T07:15:51.123 INFO:tasks.workunit.client.1.smithi069.stdout:6/9: creat f3 x:0 0 0 2024-03-22T07:15:51.123 INFO:tasks.workunit.client.1.smithi069.stdout:4/18: mkdir d6 0 2024-03-22T07:15:51.123 INFO:tasks.workunit.client.1.smithi069.stdout:4/19: chown l2 337446 1 2024-03-22T07:15:51.123 INFO:tasks.workunit.client.0.smithi063.stdout:8/149: rename d0/l10 to d0/d1b/d28/l31 0 2024-03-22T07:15:51.125 INFO:tasks.workunit.client.0.smithi063.stdout:8/150: symlink d0/d1b/l32 0 2024-03-22T07:15:51.126 INFO:tasks.workunit.client.0.smithi063.stdout:1/166: write d4/d14/f18 [3189087,121464] 0 2024-03-22T07:15:51.126 INFO:tasks.workunit.client.0.smithi063.stdout:8/151: mkdir d0/d24/d33 0 2024-03-22T07:15:51.127 INFO:tasks.workunit.client.1.smithi069.stdout:6/10: mkdir d4 0 2024-03-22T07:15:51.127 INFO:tasks.workunit.client.1.smithi069.stdout:2/18: dread f3 [0,4194304] 0 2024-03-22T07:15:51.127 INFO:tasks.workunit.client.1.smithi069.stdout:2/19: creat f7 x:0 0 0 2024-03-22T07:15:51.128 INFO:tasks.workunit.client.0.smithi063.stdout:3/66: dwrite da/fb [0,4194304] 0 2024-03-22T07:15:51.128 INFO:tasks.workunit.client.0.smithi063.stdout:3/67: read - da/fd zero size 2024-03-22T07:15:51.129 INFO:tasks.workunit.client.1.smithi069.stdout:6/11: rename f1 to d4/f5 0 2024-03-22T07:15:51.129 INFO:tasks.workunit.client.0.smithi063.stdout:2/150: dwrite d0/fb [0,4194304] 0 2024-03-22T07:15:51.129 INFO:tasks.workunit.client.0.smithi063.stdout:1/167: rmdir d4/df 39 2024-03-22T07:15:51.129 INFO:tasks.workunit.client.0.smithi063.stdout:1/168: stat d4/f1f 0 2024-03-22T07:15:51.131 INFO:tasks.workunit.client.0.smithi063.stdout:3/68: mkdir da/d16 0 2024-03-22T07:15:51.131 INFO:tasks.workunit.client.0.smithi063.stdout:3/69: dread - da/f11 zero size 2024-03-22T07:15:51.133 INFO:tasks.workunit.client.0.smithi063.stdout:2/151: symlink d0/de/l2e 0 2024-03-22T07:15:51.133 INFO:tasks.workunit.client.0.smithi063.stdout:2/152: chown d0/de/d12/c26 3402234 1 2024-03-22T07:15:51.135 INFO:tasks.workunit.client.0.smithi063.stdout:3/70: mknod da/c17 0 2024-03-22T07:15:51.137 INFO:tasks.workunit.client.0.smithi063.stdout:3/71: rename da/f10 to da/f18 0 2024-03-22T07:15:51.137 INFO:tasks.workunit.client.0.smithi063.stdout:3/72: write da/f11 [117478,17828] 0 2024-03-22T07:15:51.152 INFO:tasks.workunit.client.0.smithi063.stdout:2/153: dread d0/de/d12/f1f [0,4194304] 0 2024-03-22T07:15:51.154 INFO:tasks.workunit.client.0.smithi063.stdout:2/154: rename d0/c21 to d0/de/d1b/c2f 0 2024-03-22T07:15:51.158 INFO:tasks.workunit.client.0.smithi063.stdout:3/73: dread da/f18 [0,4194304] 0 2024-03-22T07:15:51.161 INFO:tasks.workunit.client.1.smithi069.stdout:3/22: dwrite d5/f7 [0,4194304] 0 2024-03-22T07:15:51.162 INFO:tasks.workunit.client.1.smithi069.stdout:6/12: dwrite d4/f5 [0,4194304] 0 2024-03-22T07:15:51.162 INFO:tasks.workunit.client.0.smithi063.stdout:2/155: dread d0/de/d12/f1f [0,4194304] 0 2024-03-22T07:15:51.162 INFO:tasks.workunit.client.0.smithi063.stdout:6/101: dwrite d3/d6/f11 [0,4194304] 0 2024-03-22T07:15:51.163 INFO:tasks.workunit.client.1.smithi069.stdout:3/23: mkdir d5/d9 0 2024-03-22T07:15:51.163 INFO:tasks.workunit.client.1.smithi069.stdout:3/24: chown l4 10985 1 2024-03-22T07:15:51.163 INFO:tasks.workunit.client.1.smithi069.stdout:3/25: chown d5/d9 4811482 1 2024-03-22T07:15:51.163 INFO:tasks.workunit.client.1.smithi069.stdout:3/26: write d5/f6 [1942536,117227] 0 2024-03-22T07:15:51.165 INFO:tasks.workunit.client.0.smithi063.stdout:6/102: rename d3/d6/f11 to d3/d15/d16/f26 0 2024-03-22T07:15:51.165 INFO:tasks.workunit.client.0.smithi063.stdout:6/103: dread - d3/d15/f23 zero size 2024-03-22T07:15:51.165 INFO:tasks.workunit.client.0.smithi063.stdout:6/104: rename d3/d6 to d3/d6/d27 22 2024-03-22T07:15:51.165 INFO:tasks.workunit.client.0.smithi063.stdout:2/156: creat d0/de/f30 x:0 0 0 2024-03-22T07:15:51.167 INFO:tasks.workunit.client.1.smithi069.stdout:3/27: creat d5/d9/fa x:0 0 0 2024-03-22T07:15:51.169 INFO:tasks.workunit.client.1.smithi069.stdout:3/28: mkdir d5/d9/db 0 2024-03-22T07:15:51.172 INFO:tasks.workunit.client.1.smithi069.stdout:3/29: creat d5/d9/db/fc x:0 0 0 2024-03-22T07:15:51.172 INFO:tasks.workunit.client.1.smithi069.stdout:3/30: creat d5/d9/fd x:0 0 0 2024-03-22T07:15:51.172 INFO:tasks.workunit.client.1.smithi069.stdout:3/31: chown d5/d9/db 1660132 1 2024-03-22T07:15:51.172 INFO:tasks.workunit.client.1.smithi069.stdout:3/32: creat d5/d9/fe x:0 0 0 2024-03-22T07:15:51.174 INFO:tasks.workunit.client.0.smithi063.stdout:1/169: dwrite d4/f32 [0,4194304] 0 2024-03-22T07:15:51.179 INFO:tasks.workunit.client.0.smithi063.stdout:1/170: mkdir d4/df/d13/d39 0 2024-03-22T07:15:51.179 INFO:tasks.workunit.client.0.smithi063.stdout:1/171: write d4/df/d13/f34 [599895,92025] 0 2024-03-22T07:15:51.180 INFO:tasks.workunit.client.0.smithi063.stdout:2/157: rmdir d0 39 2024-03-22T07:15:51.181 INFO:tasks.workunit.client.0.smithi063.stdout:6/105: dwrite d3/f19 [0,4194304] 0 2024-03-22T07:15:51.189 INFO:tasks.workunit.client.1.smithi069.stdout:3/33: dwrite d5/d9/fa [0,4194304] 0 2024-03-22T07:15:51.193 INFO:tasks.workunit.client.1.smithi069.stdout:3/34: rename l3 to d5/lf 0 2024-03-22T07:15:51.221 INFO:tasks.workunit.client.1.smithi069.stdout:7/14: fsync f0 0 2024-03-22T07:15:51.221 INFO:tasks.workunit.client.1.smithi069.stdout:7/15: readlink - no filename 2024-03-22T07:15:51.221 INFO:tasks.workunit.client.0.smithi063.stdout:2/158: getdents d0/de/d12 0 2024-03-22T07:15:51.222 INFO:tasks.workunit.client.0.smithi063.stdout:7/129: sync 2024-03-22T07:15:51.222 INFO:tasks.workunit.client.0.smithi063.stdout:7/130: chown d5/d16/d1b 1068292716 1 2024-03-22T07:15:51.223 INFO:tasks.workunit.client.0.smithi063.stdout:1/172: write d4/f32 [4559727,7490] 0 2024-03-22T07:15:51.224 INFO:tasks.workunit.client.0.smithi063.stdout:2/159: truncate d0/f5 1630552 0 2024-03-22T07:15:51.226 INFO:tasks.workunit.client.0.smithi063.stdout:6/106: getdents d3/d15 0 2024-03-22T07:15:51.229 INFO:tasks.workunit.client.0.smithi063.stdout:6/107: creat d3/d6/f28 x:0 0 0 2024-03-22T07:15:51.229 INFO:tasks.workunit.client.0.smithi063.stdout:6/108: dread - d3/d6/f28 zero size 2024-03-22T07:15:51.229 INFO:tasks.workunit.client.0.smithi063.stdout:6/109: readlink d3/d6/l13 0 2024-03-22T07:15:51.229 INFO:tasks.workunit.client.0.smithi063.stdout:6/110: write f2 [9264272,97783] 0 2024-03-22T07:15:51.230 INFO:tasks.workunit.client.0.smithi063.stdout:6/111: mkdir d3/d15/d29 0 2024-03-22T07:15:51.231 INFO:tasks.workunit.client.0.smithi063.stdout:6/112: mkdir d3/d6/d2a 0 2024-03-22T07:15:51.231 INFO:tasks.workunit.client.0.smithi063.stdout:6/113: mknod d3/d15/d29/c2b 0 2024-03-22T07:15:51.232 INFO:tasks.workunit.client.0.smithi063.stdout:2/160: dread d0/f23 [0,4194304] 0 2024-03-22T07:15:51.233 INFO:tasks.workunit.client.0.smithi063.stdout:6/114: rmdir d3/d15 39 2024-03-22T07:15:51.233 INFO:tasks.workunit.client.0.smithi063.stdout:6/115: chown d3/d1c 12326 1 2024-03-22T07:15:51.233 INFO:tasks.workunit.client.0.smithi063.stdout:6/116: fsync d3/d6/fc 0 2024-03-22T07:15:51.233 INFO:tasks.workunit.client.0.smithi063.stdout:2/161: getdents d0 0 2024-03-22T07:15:51.233 INFO:tasks.workunit.client.0.smithi063.stdout:2/162: chown d0/c2d 1 1 2024-03-22T07:15:51.235 INFO:tasks.workunit.client.0.smithi063.stdout:2/163: unlink d0/l25 0 2024-03-22T07:15:51.235 INFO:tasks.workunit.client.0.smithi063.stdout:2/164: write d0/f24 [768908,68855] 0 2024-03-22T07:15:51.236 INFO:tasks.workunit.client.0.smithi063.stdout:6/117: getdents d3/d15/d16 0 2024-03-22T07:15:51.236 INFO:tasks.workunit.client.0.smithi063.stdout:6/118: read - d3/d15/d16/f25 zero size 2024-03-22T07:15:51.237 INFO:tasks.workunit.client.0.smithi063.stdout:2/165: rename d0/de/f22 to d0/de/f31 0 2024-03-22T07:15:51.239 INFO:tasks.workunit.client.0.smithi063.stdout:6/119: rename f2 to d3/d6/f2c 0 2024-03-22T07:15:51.240 INFO:tasks.workunit.client.0.smithi063.stdout:2/166: mkdir d0/de/d1b/d32 0 2024-03-22T07:15:51.241 INFO:tasks.workunit.client.0.smithi063.stdout:6/120: rename d3/d15/d16/c1a to d3/d1c/c2d 0 2024-03-22T07:15:51.257 INFO:tasks.workunit.client.0.smithi063.stdout:6/121: write d3/f19 [1017640,104255] 0 2024-03-22T07:15:51.257 INFO:tasks.workunit.client.0.smithi063.stdout:6/122: dread - d3/d6/f28 zero size 2024-03-22T07:15:51.258 INFO:tasks.workunit.client.0.smithi063.stdout:6/123: mkdir d3/d6/d2a/d2e 0 2024-03-22T07:15:51.260 INFO:tasks.workunit.client.0.smithi063.stdout:1/173: dwrite d4/df/d13/f27 [0,4194304] 0 2024-03-22T07:15:51.261 INFO:tasks.workunit.client.0.smithi063.stdout:7/131: dwrite d5/f13 [4194304,4194304] 0 2024-03-22T07:15:51.261 INFO:tasks.workunit.client.0.smithi063.stdout:7/132: creat d5/d16/f2e x:0 0 0 2024-03-22T07:15:51.261 INFO:tasks.workunit.client.0.smithi063.stdout:2/167: dwrite d0/de/d12/f27 [0,4194304] 0 2024-03-22T07:15:51.261 INFO:tasks.workunit.client.0.smithi063.stdout:2/168: dread - d0/de/f30 zero size 2024-03-22T07:15:51.261 INFO:tasks.workunit.client.0.smithi063.stdout:2/169: chown d0/de/d12/c26 19 1 2024-03-22T07:15:51.262 INFO:tasks.workunit.client.0.smithi063.stdout:2/170: write d0/de/d12/d20/f2c [54060,120480] 0 2024-03-22T07:15:51.262 INFO:tasks.workunit.client.0.smithi063.stdout:2/171: write d0/f13 [1752809,52770] 0 2024-03-22T07:15:51.262 INFO:tasks.workunit.client.0.smithi063.stdout:6/124: fsync d3/d6/f28 0 2024-03-22T07:15:51.263 INFO:tasks.workunit.client.0.smithi063.stdout:1/174: mkdir d4/df/d13/d2b/d3a 0 2024-03-22T07:15:51.263 INFO:tasks.workunit.client.0.smithi063.stdout:1/175: creat d4/f3b x:0 0 0 2024-03-22T07:15:51.267 INFO:tasks.workunit.client.0.smithi063.stdout:7/133: rename d5/lc to d5/l2f 0 2024-03-22T07:15:51.274 INFO:tasks.workunit.client.0.smithi063.stdout:2/172: symlink d0/de/d1b/l33 0 2024-03-22T07:15:51.274 INFO:tasks.workunit.client.0.smithi063.stdout:2/173: readlink d0/de/l2e 0 2024-03-22T07:15:51.274 INFO:tasks.workunit.client.0.smithi063.stdout:6/125: dwrite d3/d15/f22 [0,4194304] 0 2024-03-22T07:15:51.277 INFO:tasks.workunit.client.0.smithi063.stdout:1/176: stat d4/d14/c2f 0 2024-03-22T07:15:51.277 INFO:tasks.workunit.client.0.smithi063.stdout:1/177: stat d4/df/d13/l22 0 2024-03-22T07:15:51.277 INFO:tasks.workunit.client.0.smithi063.stdout:1/178: truncate d4/d14/f2a 4283522 0 2024-03-22T07:15:51.277 INFO:tasks.workunit.client.0.smithi063.stdout:1/179: creat d4/df/d13/f3c x:0 0 0 2024-03-22T07:15:51.279 INFO:tasks.workunit.client.0.smithi063.stdout:2/174: mknod d0/de/d1b/d32/c34 0 2024-03-22T07:15:51.280 INFO:tasks.workunit.client.0.smithi063.stdout:2/175: write d0/f5 [2056961,64617] 0 2024-03-22T07:15:51.281 INFO:tasks.workunit.client.0.smithi063.stdout:6/126: symlink d3/d15/d16/d1e/l2f 0 2024-03-22T07:15:51.283 INFO:tasks.workunit.client.0.smithi063.stdout:1/180: creat d4/df/d13/d2b/d3a/f3d x:0 0 0 2024-03-22T07:15:51.284 INFO:tasks.workunit.client.0.smithi063.stdout:9/94: sync 2024-03-22T07:15:51.284 INFO:tasks.workunit.client.0.smithi063.stdout:0/123: sync 2024-03-22T07:15:51.285 INFO:tasks.workunit.client.0.smithi063.stdout:2/176: mkdir d0/de/d35 0 2024-03-22T07:15:51.285 INFO:tasks.workunit.client.0.smithi063.stdout:2/177: chown d0/de/f30 0 1 2024-03-22T07:15:51.286 INFO:tasks.workunit.client.0.smithi063.stdout:6/127: dread d3/f4 [0,4194304] 0 2024-03-22T07:15:51.286 INFO:tasks.workunit.client.0.smithi063.stdout:6/128: dread - d3/d15/d16/f25 zero size 2024-03-22T07:15:51.286 INFO:tasks.workunit.client.0.smithi063.stdout:1/181: rename d4/df/d13/l22 to d4/d14/l3e 0 2024-03-22T07:15:51.288 INFO:tasks.workunit.client.0.smithi063.stdout:2/178: unlink d0/f5 0 2024-03-22T07:15:51.289 INFO:tasks.workunit.client.0.smithi063.stdout:6/129: truncate d3/f19 2075346 0 2024-03-22T07:15:51.290 INFO:tasks.workunit.client.0.smithi063.stdout:1/182: symlink d4/df/l3f 0 2024-03-22T07:15:51.290 INFO:tasks.workunit.client.0.smithi063.stdout:1/183: creat d4/df/d13/f40 x:0 0 0 2024-03-22T07:15:51.290 INFO:tasks.workunit.client.0.smithi063.stdout:1/184: creat d4/d14/f41 x:0 0 0 2024-03-22T07:15:51.290 INFO:tasks.workunit.client.1.smithi069.stdout:9/3: getdents . 0 2024-03-22T07:15:51.290 INFO:tasks.workunit.client.1.smithi069.stdout:9/4: dread - no filename 2024-03-22T07:15:51.291 INFO:tasks.workunit.client.0.smithi063.stdout:2/179: unlink d0/f4 0 2024-03-22T07:15:51.291 INFO:tasks.workunit.client.0.smithi063.stdout:2/180: write d0/de/d12/d20/f28 [465512,107335] 0 2024-03-22T07:15:51.292 INFO:tasks.workunit.client.0.smithi063.stdout:6/130: rmdir d3/d15/d16 39 2024-03-22T07:15:51.293 INFO:tasks.workunit.client.1.smithi069.stdout:9/5: mkdir d2 0 2024-03-22T07:15:51.293 INFO:tasks.workunit.client.1.smithi069.stdout:9/6: dwrite - no filename 2024-03-22T07:15:51.293 INFO:tasks.workunit.client.1.smithi069.stdout:9/7: fdatasync - no filename 2024-03-22T07:15:51.294 INFO:tasks.workunit.client.0.smithi063.stdout:1/185: dread d4/df/f11 [0,4194304] 0 2024-03-22T07:15:51.294 INFO:tasks.workunit.client.0.smithi063.stdout:1/186: write d4/f12 [866198,102859] 0 2024-03-22T07:15:51.294 INFO:tasks.workunit.client.0.smithi063.stdout:2/181: mkdir d0/de/d35/d36 0 2024-03-22T07:15:51.295 INFO:tasks.workunit.client.1.smithi069.stdout:9/8: unlink c1 0 2024-03-22T07:15:51.295 INFO:tasks.workunit.client.1.smithi069.stdout:9/9: rename d2 to d2/d3 22 2024-03-22T07:15:51.295 INFO:tasks.workunit.client.0.smithi063.stdout:1/187: unlink d4/d14/f2a 0 2024-03-22T07:15:51.295 INFO:tasks.workunit.client.0.smithi063.stdout:1/188: chown d4/df/d13 4176724 1 2024-03-22T07:15:51.296 INFO:tasks.workunit.client.0.smithi063.stdout:2/182: symlink d0/de/d12/d20/l37 0 2024-03-22T07:15:51.296 INFO:tasks.workunit.client.0.smithi063.stdout:2/183: chown d0/de/d12/d20/f28 44721986 1 2024-03-22T07:15:51.297 INFO:tasks.workunit.client.1.smithi069.stdout:9/10: symlink d2/l4 0 2024-03-22T07:15:51.297 INFO:tasks.workunit.client.0.smithi063.stdout:2/184: truncate d0/de/f14 1148756 0 2024-03-22T07:15:51.299 INFO:tasks.workunit.client.1.smithi069.stdout:9/11: creat d2/f5 x:0 0 0 2024-03-22T07:15:51.299 INFO:tasks.workunit.client.1.smithi069.stdout:9/12: readlink d2/l4 0 2024-03-22T07:15:51.299 INFO:tasks.workunit.client.1.smithi069.stdout:9/13: truncate d2/f5 1010560 0 2024-03-22T07:15:51.300 INFO:tasks.workunit.client.1.smithi069.stdout:9/14: mknod d2/c6 0 2024-03-22T07:15:51.300 INFO:tasks.workunit.client.1.smithi069.stdout:9/15: chown d2/f5 24196289 1 2024-03-22T07:15:51.301 INFO:tasks.workunit.client.1.smithi069.stdout:9/16: symlink d2/l7 0 2024-03-22T07:15:51.303 INFO:tasks.workunit.client.1.smithi069.stdout:9/17: dread d2/f5 [0,4194304] 0 2024-03-22T07:15:51.308 INFO:tasks.workunit.client.1.smithi069.stdout:6/13: truncate d4/f5 2118174 0 2024-03-22T07:15:51.308 INFO:tasks.workunit.client.1.smithi069.stdout:6/14: fdatasync d4/f5 0 2024-03-22T07:15:51.308 INFO:tasks.workunit.client.1.smithi069.stdout:3/35: fsync d5/d9/fd 0 2024-03-22T07:15:51.308 INFO:tasks.workunit.client.1.smithi069.stdout:3/36: dread - d5/d9/fe zero size 2024-03-22T07:15:51.308 INFO:tasks.workunit.client.1.smithi069.stdout:3/37: rename d5/d9 to d5/d9/d10 22 2024-03-22T07:15:51.310 INFO:tasks.workunit.client.1.smithi069.stdout:6/15: rmdir d4 39 2024-03-22T07:15:51.312 INFO:tasks.workunit.client.1.smithi069.stdout:3/38: symlink d5/l11 0 2024-03-22T07:15:51.312 INFO:tasks.workunit.client.1.smithi069.stdout:3/39: write d5/d9/db/fc [500500,33166] 0 2024-03-22T07:15:51.324 INFO:tasks.workunit.client.1.smithi069.stdout:3/40: dread d5/d9/fa [0,4194304] 0 2024-03-22T07:15:51.329 INFO:tasks.workunit.client.0.smithi063.stdout:7/134: dwrite d5/d16/d1b/d21/f2a [0,4194304] 0 2024-03-22T07:15:51.329 INFO:tasks.workunit.client.0.smithi063.stdout:9/95: dwrite d8/fb [0,4194304] 0 2024-03-22T07:15:51.330 INFO:tasks.workunit.client.0.smithi063.stdout:1/189: dwrite d4/d14/f41 [0,4194304] 0 2024-03-22T07:15:51.331 INFO:tasks.workunit.client.1.smithi069.stdout:9/18: dwrite d2/f5 [0,4194304] 0 2024-03-22T07:15:51.331 INFO:tasks.workunit.client.0.smithi063.stdout:0/124: dwrite f1 [0,4194304] 0 2024-03-22T07:15:51.331 INFO:tasks.workunit.client.0.smithi063.stdout:0/125: fsync f5 0 2024-03-22T07:15:51.336 INFO:tasks.workunit.client.0.smithi063.stdout:1/190: dread d4/df/d13/f34 [0,4194304] 0 2024-03-22T07:15:51.338 INFO:tasks.workunit.client.0.smithi063.stdout:6/131: write d3/d1c/f1d [31980,87192] 0 2024-03-22T07:15:51.338 INFO:tasks.workunit.client.0.smithi063.stdout:6/132: dread - d3/d15/d16/d1e/f20 zero size 2024-03-22T07:15:51.338 INFO:tasks.workunit.client.1.smithi069.stdout:6/16: rename d4/f5 to d4/f6 0 2024-03-22T07:15:51.338 INFO:tasks.workunit.client.1.smithi069.stdout:6/17: dread - f3 zero size 2024-03-22T07:15:51.340 INFO:tasks.workunit.client.0.smithi063.stdout:9/96: dread d8/fc [0,4194304] 0 2024-03-22T07:15:51.340 INFO:tasks.workunit.client.0.smithi063.stdout:9/97: readlink l5 0 2024-03-22T07:15:51.340 INFO:tasks.workunit.client.0.smithi063.stdout:9/98: write d8/fa [222639,68097] 0 2024-03-22T07:15:51.340 INFO:tasks.workunit.client.0.smithi063.stdout:9/99: chown c0 591369 1 2024-03-22T07:15:51.342 INFO:tasks.workunit.client.0.smithi063.stdout:7/135: symlink d5/d14/d1d/d29/l30 0 2024-03-22T07:15:51.345 INFO:tasks.workunit.client.1.smithi069.stdout:9/19: write d2/f5 [1858073,52077] 0 2024-03-22T07:15:51.351 INFO:tasks.workunit.client.0.smithi063.stdout:1/191: rename d4/f1d to d4/df/d13/d2b/d3a/f42 0 2024-03-22T07:15:51.353 INFO:tasks.workunit.client.0.smithi063.stdout:0/126: dwrite da/fc [0,4194304] 0 2024-03-22T07:15:51.353 INFO:tasks.workunit.client.0.smithi063.stdout:6/133: symlink d3/l30 0 2024-03-22T07:15:51.354 INFO:tasks.workunit.client.0.smithi063.stdout:7/136: dread d5/f17 [0,4194304] 0 2024-03-22T07:15:51.354 INFO:tasks.workunit.client.0.smithi063.stdout:7/137: chown d5/f18 0 1 2024-03-22T07:15:51.358 INFO:tasks.workunit.client.0.smithi063.stdout:2/185: write d0/f13 [856823,109572] 0 2024-03-22T07:15:51.358 INFO:tasks.workunit.client.1.smithi069.stdout:6/18: dwrite d4/f6 [0,4194304] 0 2024-03-22T07:15:51.359 INFO:tasks.workunit.client.0.smithi063.stdout:9/100: creat d8/d15/f1c x:0 0 0 2024-03-22T07:15:51.359 INFO:tasks.workunit.client.0.smithi063.stdout:4/97: sync 2024-03-22T07:15:51.359 INFO:tasks.workunit.client.1.smithi069.stdout:6/19: mkdir d4/d7 0 2024-03-22T07:15:51.361 INFO:tasks.workunit.client.0.smithi063.stdout:0/127: rmdir da/de/d17/d22 0 2024-03-22T07:15:51.362 INFO:tasks.workunit.client.1.smithi069.stdout:6/20: unlink f2 0 2024-03-22T07:15:51.364 INFO:tasks.workunit.client.0.smithi063.stdout:6/134: symlink d3/d15/d16/l31 0 2024-03-22T07:15:51.367 INFO:tasks.workunit.client.0.smithi063.stdout:7/138: dread d5/f9 [0,4194304] 0 2024-03-22T07:15:51.367 INFO:tasks.workunit.client.0.smithi063.stdout:7/139: write d5/da/f11 [456377,85986] 0 2024-03-22T07:15:51.367 INFO:tasks.workunit.client.0.smithi063.stdout:7/140: chown d5/d14/d1d 3633067 1 2024-03-22T07:15:51.367 INFO:tasks.workunit.client.1.smithi069.stdout:6/21: truncate d4/f6 2739905 0 2024-03-22T07:15:51.367 INFO:tasks.workunit.client.1.smithi069.stdout:6/22: readlink - no filename 2024-03-22T07:15:51.368 INFO:tasks.workunit.client.0.smithi063.stdout:9/101: creat d8/d17/f1d x:0 0 0 2024-03-22T07:15:51.370 INFO:tasks.workunit.client.0.smithi063.stdout:8/152: sync 2024-03-22T07:15:51.370 INFO:tasks.workunit.client.0.smithi063.stdout:3/74: sync 2024-03-22T07:15:51.371 INFO:tasks.workunit.client.0.smithi063.stdout:5/118: sync 2024-03-22T07:15:51.371 INFO:tasks.workunit.client.0.smithi063.stdout:5/119: chown d8/d13/c1a 2 1 2024-03-22T07:15:51.371 INFO:tasks.workunit.client.0.smithi063.stdout:5/120: getdents d8/d13/d24 0 2024-03-22T07:15:51.373 INFO:tasks.workunit.client.0.smithi063.stdout:0/128: mkdir da/de/d17/d25 0 2024-03-22T07:15:51.373 INFO:tasks.workunit.client.0.smithi063.stdout:0/129: chown da/fc 76035 1 2024-03-22T07:15:51.376 INFO:tasks.workunit.client.0.smithi063.stdout:6/135: mkdir d3/d1c/d32 0 2024-03-22T07:15:51.378 INFO:tasks.workunit.client.0.smithi063.stdout:2/186: getdents d0/de/d35 0 2024-03-22T07:15:51.380 INFO:tasks.workunit.client.0.smithi063.stdout:7/141: mkdir d5/da/d28/d31 0 2024-03-22T07:15:51.382 INFO:tasks.workunit.client.0.smithi063.stdout:9/102: mkdir d8/d17/d1e 0 2024-03-22T07:15:51.383 INFO:tasks.workunit.client.0.smithi063.stdout:8/153: read d0/f4 [1680823,4870] 0 2024-03-22T07:15:51.384 INFO:tasks.workunit.client.0.smithi063.stdout:0/130: symlink da/de/d17/l26 0 2024-03-22T07:15:51.385 INFO:tasks.workunit.client.0.smithi063.stdout:6/136: rename d3/lf to d3/d15/d29/l33 0 2024-03-22T07:15:51.386 INFO:tasks.workunit.client.0.smithi063.stdout:2/187: creat d0/de/f38 x:0 0 0 2024-03-22T07:15:51.387 INFO:tasks.workunit.client.0.smithi063.stdout:5/121: rename d8/f21 to d8/f25 0 2024-03-22T07:15:51.387 INFO:tasks.workunit.client.0.smithi063.stdout:0/131: creat da/de/d17/d25/f27 x:0 0 0 2024-03-22T07:15:51.394 INFO:tasks.workunit.client.0.smithi063.stdout:6/137: read d3/d6/fc [2433514,83851] 0 2024-03-22T07:15:51.415 INFO:tasks.workunit.client.0.smithi063.stdout:4/98: dwrite d1/d5/f12 [4194304,4194304] 0 2024-03-22T07:15:51.415 INFO:tasks.workunit.client.1.smithi069.stdout:9/20: dwrite d2/f5 [0,4194304] 0 2024-03-22T07:15:51.415 INFO:tasks.workunit.client.1.smithi069.stdout:9/21: chown d2 2008968 1 2024-03-22T07:15:51.416 INFO:tasks.workunit.client.0.smithi063.stdout:0/132: dwrite da/de/f23 [0,4194304] 0 2024-03-22T07:15:51.416 INFO:tasks.workunit.client.0.smithi063.stdout:6/138: dwrite d3/d6/f10 [0,4194304] 0 2024-03-22T07:15:51.416 INFO:tasks.workunit.client.0.smithi063.stdout:6/139: fsync d3/f19 0 2024-03-22T07:15:51.416 INFO:tasks.workunit.client.0.smithi063.stdout:6/140: dread - d3/d15/f23 zero size 2024-03-22T07:15:51.416 INFO:tasks.workunit.client.0.smithi063.stdout:4/99: rename d1/d5 to d1/d5/dc/d1b 22 2024-03-22T07:15:51.417 INFO:tasks.workunit.client.1.smithi069.stdout:9/22: mknod d2/c8 0 2024-03-22T07:15:51.417 INFO:tasks.workunit.client.0.smithi063.stdout:2/188: dread d0/de/f14 [0,4194304] 0 2024-03-22T07:15:51.418 INFO:tasks.workunit.client.1.smithi069.stdout:9/23: mknod d2/c9 0 2024-03-22T07:15:51.423 INFO:tasks.workunit.client.0.smithi063.stdout:6/141: mknod d3/d6/d2a/c34 0 2024-03-22T07:15:51.423 INFO:tasks.workunit.client.0.smithi063.stdout:2/189: mkdir d0/de/d12/d39 0 2024-03-22T07:15:51.423 INFO:tasks.workunit.client.0.smithi063.stdout:6/142: mkdir d3/d6/d35 0 2024-03-22T07:15:51.423 INFO:tasks.workunit.client.0.smithi063.stdout:6/143: dread - d3/d6/f28 zero size 2024-03-22T07:15:51.423 INFO:tasks.workunit.client.0.smithi063.stdout:6/144: chown d3/d6/fc 31997 1 2024-03-22T07:15:51.424 INFO:tasks.workunit.client.0.smithi063.stdout:6/145: symlink d3/d1c/d32/l36 0 2024-03-22T07:15:51.424 INFO:tasks.workunit.client.0.smithi063.stdout:2/190: rename d0/f24 to d0/de/d35/d36/f3a 0 2024-03-22T07:15:51.430 INFO:tasks.workunit.client.1.smithi069.stdout:9/24: write d2/f5 [1591657,14114] 0 2024-03-22T07:15:51.430 INFO:tasks.workunit.client.0.smithi063.stdout:6/146: write d3/d6/f10 [2023065,91190] 0 2024-03-22T07:15:51.431 INFO:tasks.workunit.client.1.smithi069.stdout:9/25: rename d2/c6 to d2/ca 0 2024-03-22T07:15:51.431 INFO:tasks.workunit.client.1.smithi069.stdout:9/26: chown d2/l4 2688 1 2024-03-22T07:15:51.432 INFO:tasks.workunit.client.1.smithi069.stdout:9/27: symlink d2/lb 0 2024-03-22T07:15:51.433 INFO:tasks.workunit.client.1.smithi069.stdout:9/28: rename d2/l7 to d2/lc 0 2024-03-22T07:15:51.433 INFO:tasks.workunit.client.1.smithi069.stdout:9/29: write d2/f5 [3331675,81748] 0 2024-03-22T07:15:51.433 INFO:tasks.workunit.client.1.smithi069.stdout:9/30: write d2/f5 [5204847,106756] 0 2024-03-22T07:15:51.442 INFO:tasks.workunit.client.0.smithi063.stdout:6/147: dwrite d3/d15/f23 [0,4194304] 0 2024-03-22T07:15:51.444 INFO:tasks.workunit.client.1.smithi069.stdout:9/31: truncate d2/f5 5269017 0 2024-03-22T07:15:51.543 INFO:tasks.workunit.client.0.smithi063.stdout:3/75: dwrite da/f11 [0,4194304] 0 2024-03-22T07:15:51.547 INFO:tasks.workunit.client.0.smithi063.stdout:8/154: write d0/f1a [593370,121174] 0 2024-03-22T07:15:51.547 INFO:tasks.workunit.client.0.smithi063.stdout:8/155: truncate d0/d24/f29 397689 0 2024-03-22T07:15:51.547 INFO:tasks.workunit.client.0.smithi063.stdout:2/191: write d0/de/d12/d20/f2a [1061568,12570] 0 2024-03-22T07:15:51.551 INFO:tasks.workunit.client.0.smithi063.stdout:8/156: rmdir d0/d24 39 2024-03-22T07:15:51.552 INFO:tasks.workunit.client.0.smithi063.stdout:8/157: creat d0/d26/f34 x:0 0 0 2024-03-22T07:15:51.555 INFO:tasks.workunit.client.0.smithi063.stdout:8/158: stat d0/d26/c2c 0 2024-03-22T07:15:51.558 INFO:tasks.workunit.client.0.smithi063.stdout:6/148: dread d3/d6/f2c [4194304,4194304] 0 2024-03-22T07:15:51.558 INFO:tasks.workunit.client.0.smithi063.stdout:6/149: dread - d3/d15/d16/f25 zero size 2024-03-22T07:15:51.558 INFO:tasks.workunit.client.0.smithi063.stdout:6/150: chown d3/d1c/f1d 98085873 1 2024-03-22T07:15:51.561 INFO:tasks.workunit.client.0.smithi063.stdout:3/76: write da/fb [2860413,15678] 0 2024-03-22T07:15:51.561 INFO:tasks.workunit.client.0.smithi063.stdout:3/77: chown da/cf 1255019804 1 2024-03-22T07:15:51.562 INFO:tasks.workunit.client.0.smithi063.stdout:7/142: dwrite d5/d16/f1e [0,4194304] 0 2024-03-22T07:15:51.562 INFO:tasks.workunit.client.0.smithi063.stdout:7/143: chown d5/d16/d1b/d21/f2a 380178 1 2024-03-22T07:15:51.562 INFO:tasks.workunit.client.0.smithi063.stdout:7/144: dread - d5/d16/d1b/f2d zero size 2024-03-22T07:15:51.562 INFO:tasks.workunit.client.0.smithi063.stdout:7/145: fsync d5/d16/f22 0 2024-03-22T07:15:51.562 INFO:tasks.workunit.client.0.smithi063.stdout:5/122: dwrite d8/f1c [0,4194304] 0 2024-03-22T07:15:51.563 INFO:tasks.workunit.client.0.smithi063.stdout:6/151: symlink d3/d15/d16/l37 0 2024-03-22T07:15:51.563 INFO:tasks.workunit.client.0.smithi063.stdout:4/100: dwrite d1/d13/f19 [0,4194304] 0 2024-03-22T07:15:51.563 INFO:tasks.workunit.client.0.smithi063.stdout:4/101: write d1/d5/f7 [3745328,98662] 0 2024-03-22T07:15:51.563 INFO:tasks.workunit.client.0.smithi063.stdout:8/159: rmdir d0/d1f 39 2024-03-22T07:15:51.564 INFO:tasks.workunit.client.0.smithi063.stdout:3/78: mknod da/c19 0 2024-03-22T07:15:51.570 INFO:tasks.workunit.client.0.smithi063.stdout:6/152: mkdir d3/d6/d2a/d38 0 2024-03-22T07:15:51.570 INFO:tasks.workunit.client.0.smithi063.stdout:6/153: write d3/d15/d16/f25 [424312,60918] 0 2024-03-22T07:15:51.571 INFO:tasks.workunit.client.1.smithi069.stdout:9/32: dread d2/f5 [0,4194304] 0 2024-03-22T07:15:51.572 INFO:tasks.workunit.client.0.smithi063.stdout:7/146: dread d5/d16/d1b/d21/f2a [0,4194304] 0 2024-03-22T07:15:51.572 INFO:tasks.workunit.client.1.smithi069.stdout:9/33: mkdir d2/dd 0 2024-03-22T07:15:51.573 INFO:tasks.workunit.client.0.smithi063.stdout:5/123: dread d8/f16 [0,4194304] 0 2024-03-22T07:15:51.573 INFO:tasks.workunit.client.1.smithi069.stdout:9/34: mknod d2/dd/ce 0 2024-03-22T07:15:51.574 INFO:tasks.workunit.client.0.smithi063.stdout:3/79: creat da/d16/f1a x:0 0 0 2024-03-22T07:15:51.575 INFO:tasks.workunit.client.0.smithi063.stdout:3/80: write da/fb [698437,6788] 0 2024-03-22T07:15:51.575 INFO:tasks.workunit.client.0.smithi063.stdout:5/124: mknod d8/d13/c26 0 2024-03-22T07:15:51.576 INFO:tasks.workunit.client.0.smithi063.stdout:4/102: symlink d1/d5/dc/l1c 0 2024-03-22T07:15:51.577 INFO:tasks.workunit.client.0.smithi063.stdout:5/125: creat d8/d13/d1e/f27 x:0 0 0 2024-03-22T07:15:51.578 INFO:tasks.workunit.client.0.smithi063.stdout:4/103: write d1/d5/dc/ff [660052,112283] 0 2024-03-22T07:15:51.578 INFO:tasks.workunit.client.0.smithi063.stdout:4/104: chown d1/c4 1508482634 1 2024-03-22T07:15:51.578 INFO:tasks.workunit.client.0.smithi063.stdout:4/105: dread - d1/d13/f17 zero size 2024-03-22T07:15:51.579 INFO:tasks.workunit.client.0.smithi063.stdout:4/106: stat d1/d5/c18 0 2024-03-22T07:15:51.579 INFO:tasks.workunit.client.0.smithi063.stdout:5/126: rmdir d8/d13 39 2024-03-22T07:15:51.580 INFO:tasks.workunit.client.0.smithi063.stdout:5/127: mkdir d8/d13/d28 0 2024-03-22T07:15:51.580 INFO:tasks.workunit.client.1.smithi069.stdout:9/35: getdents d2/dd 0 2024-03-22T07:15:51.581 INFO:tasks.workunit.client.0.smithi063.stdout:6/154: dread d3/f9 [4194304,4194304] 0 2024-03-22T07:15:51.581 INFO:tasks.workunit.client.0.smithi063.stdout:5/128: truncate d8/f14 2781521 0 2024-03-22T07:15:51.581 INFO:tasks.workunit.client.0.smithi063.stdout:5/129: chown d8/f10 20 1 2024-03-22T07:15:51.581 INFO:tasks.workunit.client.0.smithi063.stdout:5/130: symlink d8/d13/d1e/l29 0 2024-03-22T07:15:51.581 INFO:tasks.workunit.client.0.smithi063.stdout:5/131: dread - d8/d13/d1e/f27 zero size 2024-03-22T07:15:51.582 INFO:tasks.workunit.client.0.smithi063.stdout:5/132: chown d8/d13/c1a 476998278 1 2024-03-22T07:15:51.582 INFO:tasks.workunit.client.0.smithi063.stdout:6/155: symlink d3/d6/l39 0 2024-03-22T07:15:51.583 INFO:tasks.workunit.client.0.smithi063.stdout:6/156: mkdir d3/d6/d35/d3a 0 2024-03-22T07:15:51.586 INFO:tasks.workunit.client.0.smithi063.stdout:4/107: fdatasync d1/f3 0 2024-03-22T07:15:51.586 INFO:tasks.workunit.client.0.smithi063.stdout:8/160: dwrite d0/d24/f29 [0,4194304] 0 2024-03-22T07:15:51.587 INFO:tasks.workunit.client.0.smithi063.stdout:9/103: sync 2024-03-22T07:15:51.589 INFO:tasks.workunit.client.0.smithi063.stdout:7/147: write d5/f17 [2592651,119244] 0 2024-03-22T07:15:51.589 INFO:tasks.workunit.client.0.smithi063.stdout:2/192: dwrite d0/f23 [0,4194304] 0 2024-03-22T07:15:51.589 INFO:tasks.workunit.client.0.smithi063.stdout:6/157: dread d3/d1c/f1d [0,4194304] 0 2024-03-22T07:15:51.589 INFO:tasks.workunit.client.0.smithi063.stdout:6/158: dread - d3/d6/f28 zero size 2024-03-22T07:15:51.589 INFO:tasks.workunit.client.0.smithi063.stdout:1/192: sync 2024-03-22T07:15:51.589 INFO:tasks.workunit.client.0.smithi063.stdout:0/133: sync 2024-03-22T07:15:51.590 INFO:tasks.workunit.client.0.smithi063.stdout:0/134: creat da/de/d17/f28 x:0 0 0 2024-03-22T07:15:51.590 INFO:tasks.workunit.client.0.smithi063.stdout:0/135: write f9 [2005712,81780] 0 2024-03-22T07:15:51.590 INFO:tasks.workunit.client.0.smithi063.stdout:0/136: fsync da/de/d17/f21 0 2024-03-22T07:15:51.591 INFO:tasks.workunit.client.0.smithi063.stdout:8/161: creat d0/d24/f35 x:0 0 0 2024-03-22T07:15:51.595 INFO:tasks.workunit.client.0.smithi063.stdout:7/148: dread d5/d16/d1b/d21/f2a [0,4194304] 0 2024-03-22T07:15:51.595 INFO:tasks.workunit.client.0.smithi063.stdout:7/149: creat d5/d16/f32 x:0 0 0 2024-03-22T07:15:51.595 INFO:tasks.workunit.client.0.smithi063.stdout:2/193: creat d0/de/d1b/d32/f3b x:0 0 0 2024-03-22T07:15:51.596 INFO:tasks.workunit.client.0.smithi063.stdout:5/133: fsync d8/d13/d1e/f27 0 2024-03-22T07:15:51.596 INFO:tasks.workunit.client.0.smithi063.stdout:5/134: getdents d8/d13/d28 0 2024-03-22T07:15:51.598 INFO:tasks.workunit.client.0.smithi063.stdout:6/159: mknod d3/d15/d16/d1e/c3b 0 2024-03-22T07:15:51.598 INFO:tasks.workunit.client.0.smithi063.stdout:6/160: write d3/d6/fa [999180,74466] 0 2024-03-22T07:15:51.602 INFO:tasks.workunit.client.0.smithi063.stdout:0/137: mknod da/de/d17/c29 0 2024-03-22T07:15:51.606 INFO:tasks.workunit.client.0.smithi063.stdout:8/162: unlink d0/f4 0 2024-03-22T07:15:51.607 INFO:tasks.workunit.client.0.smithi063.stdout:7/150: mkdir d5/da/d28/d31/d33 0 2024-03-22T07:15:51.607 INFO:tasks.workunit.client.0.smithi063.stdout:2/194: rmdir d0/de/d12/d20 39 2024-03-22T07:15:51.609 INFO:tasks.workunit.client.0.smithi063.stdout:5/135: creat d8/d13/f2a x:0 0 0 2024-03-22T07:15:51.610 INFO:tasks.workunit.client.0.smithi063.stdout:4/108: dwrite d1/d5/dc/ff [0,4194304] 0 2024-03-22T07:15:51.612 INFO:tasks.workunit.client.0.smithi063.stdout:6/161: symlink d3/d15/d16/d1e/l3c 0 2024-03-22T07:15:51.613 INFO:tasks.workunit.client.0.smithi063.stdout:8/163: mknod d0/d26/c36 0 2024-03-22T07:15:51.614 INFO:tasks.workunit.client.0.smithi063.stdout:7/151: mkdir d5/d14/d34 0 2024-03-22T07:15:51.616 INFO:tasks.workunit.client.0.smithi063.stdout:2/195: dread d0/f23 [0,4194304] 0 2024-03-22T07:15:51.623 INFO:tasks.workunit.client.0.smithi063.stdout:6/162: link d3/d15/d16/d1e/c21 d3/d15/d29/c3d 0 2024-03-22T07:15:51.623 INFO:tasks.workunit.client.0.smithi063.stdout:1/193: dwrite d4/f38 [0,4194304] 0 2024-03-22T07:15:51.623 INFO:tasks.workunit.client.0.smithi063.stdout:7/152: unlink c4 0 2024-03-22T07:15:51.623 INFO:tasks.workunit.client.0.smithi063.stdout:7/153: read - d5/d16/f2e zero size 2024-03-22T07:15:51.623 INFO:tasks.workunit.client.0.smithi063.stdout:7/154: creat d5/da/f35 x:0 0 0 2024-03-22T07:15:51.623 INFO:tasks.workunit.client.0.smithi063.stdout:0/138: dread f3 [0,4194304] 0 2024-03-22T07:15:51.623 INFO:tasks.workunit.client.0.smithi063.stdout:0/139: truncate da/ff 3466634 0 2024-03-22T07:15:51.624 INFO:tasks.workunit.client.0.smithi063.stdout:0/140: fdatasync da/de/f23 0 2024-03-22T07:15:51.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:51 smithi063 ceph-mon[29200]: pgmap v248: 65 pgs: 65 active+clean; 242 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 6.2 MiB/s rd, 14 MiB/s wr, 996 op/s 2024-03-22T07:15:51.624 INFO:tasks.workunit.client.0.smithi063.stdout:2/196: symlink d0/de/d35/l3c 0 2024-03-22T07:15:51.626 INFO:tasks.workunit.client.0.smithi063.stdout:6/163: dread d3/d1c/f1d [0,4194304] 0 2024-03-22T07:15:51.626 INFO:tasks.workunit.client.0.smithi063.stdout:6/164: stat d3/d15/c1f 0 2024-03-22T07:15:51.626 INFO:tasks.workunit.client.0.smithi063.stdout:6/165: creat d3/d6/f3e x:0 0 0 2024-03-22T07:15:51.626 INFO:tasks.workunit.client.0.smithi063.stdout:4/109: unlink d1/d5/dc/c16 0 2024-03-22T07:15:51.626 INFO:tasks.workunit.client.0.smithi063.stdout:1/194: symlink d4/df/d13/d39/l43 0 2024-03-22T07:15:51.628 INFO:tasks.workunit.client.0.smithi063.stdout:7/155: truncate d5/f13 1237098 0 2024-03-22T07:15:51.628 INFO:tasks.workunit.client.0.smithi063.stdout:7/156: fdatasync d5/da/f2c 0 2024-03-22T07:15:51.630 INFO:tasks.workunit.client.0.smithi063.stdout:0/141: getdents da/de 0 2024-03-22T07:15:51.634 INFO:tasks.workunit.client.0.smithi063.stdout:0/142: creat da/f2a x:0 0 0 2024-03-22T07:15:51.634 INFO:tasks.workunit.client.0.smithi063.stdout:0/143: creat da/f2b x:0 0 0 2024-03-22T07:15:51.635 INFO:tasks.workunit.client.0.smithi063.stdout:2/197: symlink d0/de/d12/d39/l3d 0 2024-03-22T07:15:51.635 INFO:tasks.workunit.client.0.smithi063.stdout:5/136: dwrite d8/f12 [0,4194304] 0 2024-03-22T07:15:51.635 INFO:tasks.workunit.client.0.smithi063.stdout:6/166: rename d3/d15/c1f to d3/d6/d2a/d38/c3f 0 2024-03-22T07:15:51.635 INFO:tasks.workunit.client.0.smithi063.stdout:8/164: dwrite d0/d26/f2e [0,4194304] 0 2024-03-22T07:15:51.636 INFO:tasks.workunit.client.0.smithi063.stdout:1/195: rmdir d4/df 39 2024-03-22T07:15:51.636 INFO:tasks.workunit.client.0.smithi063.stdout:7/157: write d5/f9 [3640431,38452] 0 2024-03-22T07:15:51.637 INFO:tasks.workunit.client.0.smithi063.stdout:7/158: dread d5/f18 [0,4194304] 0 2024-03-22T07:15:51.638 INFO:tasks.workunit.client.0.smithi063.stdout:0/144: unlink da/de/d17/c1f 0 2024-03-22T07:15:51.639 INFO:tasks.workunit.client.0.smithi063.stdout:2/198: rename d0/de/d12/d20/l37 to d0/de/d12/d39/l3e 0 2024-03-22T07:15:51.640 INFO:tasks.workunit.client.0.smithi063.stdout:2/199: chown d0/de 19856312 1 2024-03-22T07:15:51.641 INFO:tasks.workunit.client.0.smithi063.stdout:6/167: link d3/f19 d3/d6/d35/d3a/f40 0 2024-03-22T07:15:51.641 INFO:tasks.workunit.client.0.smithi063.stdout:6/168: write d3/d6/f28 [667609,129370] 0 2024-03-22T07:15:51.643 INFO:tasks.workunit.client.0.smithi063.stdout:4/110: link d1/f2 d1/d5/dc/f1d 0 2024-03-22T07:15:51.643 INFO:tasks.workunit.client.0.smithi063.stdout:4/111: truncate d1/d13/f17 148227 0 2024-03-22T07:15:51.644 INFO:tasks.workunit.client.0.smithi063.stdout:4/112: fdatasync d1/d5/f7 0 2024-03-22T07:15:51.644 INFO:tasks.workunit.client.0.smithi063.stdout:8/165: dread d0/f19 [0,4194304] 0 2024-03-22T07:15:51.644 INFO:tasks.workunit.client.0.smithi063.stdout:1/196: creat d4/df/f44 x:0 0 0 2024-03-22T07:15:51.647 INFO:tasks.workunit.client.0.smithi063.stdout:1/197: dread d4/d14/f23 [0,4194304] 0 2024-03-22T07:15:51.647 INFO:tasks.workunit.client.0.smithi063.stdout:7/159: creat d5/d14/d1d/d29/f36 x:0 0 0 2024-03-22T07:15:51.648 INFO:tasks.workunit.client.0.smithi063.stdout:0/145: symlink da/de/l2c 0 2024-03-22T07:15:51.648 INFO:tasks.workunit.client.0.smithi063.stdout:0/146: fsync f1 0 2024-03-22T07:15:51.653 INFO:tasks.workunit.client.0.smithi063.stdout:4/113: dread d1/d5/f7 [0,4194304] 0 2024-03-22T07:15:51.654 INFO:tasks.workunit.client.0.smithi063.stdout:0/147: symlink da/de/l2d 0 2024-03-22T07:15:51.654 INFO:tasks.workunit.client.0.smithi063.stdout:0/148: write da/f2a [905801,128811] 0 2024-03-22T07:15:51.655 INFO:tasks.workunit.client.0.smithi063.stdout:1/198: dread d4/f9 [0,4194304] 0 2024-03-22T07:15:51.656 INFO:tasks.workunit.client.0.smithi063.stdout:6/169: write d3/d6/f2c [3158785,84551] 0 2024-03-22T07:15:51.656 INFO:tasks.workunit.client.0.smithi063.stdout:6/170: fsync d3/d15/f22 0 2024-03-22T07:15:51.656 INFO:tasks.workunit.client.0.smithi063.stdout:6/171: write d3/d15/f17 [88613,8291] 0 2024-03-22T07:15:51.662 INFO:tasks.workunit.client.0.smithi063.stdout:5/137: rename d8/d13/f2a to d8/f2b 0 2024-03-22T07:15:51.672 INFO:tasks.workunit.client.0.smithi063.stdout:6/172: dread d3/d15/f23 [0,4194304] 0 2024-03-22T07:15:51.672 INFO:tasks.workunit.client.0.smithi063.stdout:5/138: dread d8/fe [0,4194304] 0 2024-03-22T07:15:51.674 INFO:tasks.workunit.client.0.smithi063.stdout:5/139: symlink d8/l2c 0 2024-03-22T07:15:51.674 INFO:tasks.workunit.client.0.smithi063.stdout:5/140: dread - d8/f2b zero size 2024-03-22T07:15:51.675 INFO:tasks.workunit.client.0.smithi063.stdout:5/141: unlink d8/d13/c1b 0 2024-03-22T07:15:51.679 INFO:tasks.workunit.client.0.smithi063.stdout:4/114: dwrite d1/f6 [0,4194304] 0 2024-03-22T07:15:51.681 INFO:tasks.workunit.client.0.smithi063.stdout:4/115: mknod d1/d5/dc/c1e 0 2024-03-22T07:15:51.682 INFO:tasks.workunit.client.0.smithi063.stdout:4/116: rename d1/cb to d1/c1f 0 2024-03-22T07:15:51.683 INFO:tasks.workunit.client.0.smithi063.stdout:4/117: rename d1/d13/f19 to d1/d5/f20 0 2024-03-22T07:15:51.683 INFO:tasks.workunit.client.0.smithi063.stdout:4/118: chown d1/f11 5528 1 2024-03-22T07:15:51.702 INFO:tasks.workunit.client.0.smithi063.stdout:4/119: read d1/d5/f20 [378191,60277] 0 2024-03-22T07:15:51.704 INFO:tasks.workunit.client.0.smithi063.stdout:4/120: link d1/f2 d1/f21 0 2024-03-22T07:15:51.704 INFO:tasks.workunit.client.0.smithi063.stdout:4/121: write d1/d5/f12 [2579854,90923] 0 2024-03-22T07:15:51.704 INFO:tasks.workunit.client.0.smithi063.stdout:4/122: creat d1/d5/f22 x:0 0 0 2024-03-22T07:15:51.704 INFO:tasks.workunit.client.0.smithi063.stdout:4/123: chown d1/d5/f22 987924 1 2024-03-22T07:15:51.704 INFO:tasks.workunit.client.0.smithi063.stdout:4/124: chown d1/d5/f12 12 1 2024-03-22T07:15:51.705 INFO:tasks.workunit.client.0.smithi063.stdout:4/125: creat d1/d5/f23 x:0 0 0 2024-03-22T07:15:51.706 INFO:tasks.workunit.client.0.smithi063.stdout:4/126: creat d1/f24 x:0 0 0 2024-03-22T07:15:51.706 INFO:tasks.workunit.client.0.smithi063.stdout:4/127: write d1/d5/f7 [4715897,82184] 0 2024-03-22T07:15:51.708 INFO:tasks.workunit.client.0.smithi063.stdout:4/128: link d1/d5/dc/ld d1/d13/l25 0 2024-03-22T07:15:51.708 INFO:tasks.workunit.client.0.smithi063.stdout:4/129: creat d1/f26 x:0 0 0 2024-03-22T07:15:51.709 INFO:tasks.workunit.client.0.smithi063.stdout:4/130: truncate d1/d5/f20 2931518 0 2024-03-22T07:15:51.709 INFO:tasks.workunit.client.0.smithi063.stdout:4/131: write d1/d5/f12 [3260760,56699] 0 2024-03-22T07:15:51.709 INFO:tasks.workunit.client.0.smithi063.stdout:4/132: readlink d1/d5/dc/ld 0 2024-03-22T07:15:51.710 INFO:tasks.workunit.client.0.smithi063.stdout:4/133: mkdir d1/d13/d27 0 2024-03-22T07:15:51.711 INFO:tasks.workunit.client.0.smithi063.stdout:4/134: dread d1/d13/f17 [0,4194304] 0 2024-03-22T07:15:51.711 INFO:tasks.workunit.client.0.smithi063.stdout:4/135: creat d1/d5/f28 x:0 0 0 2024-03-22T07:15:51.711 INFO:tasks.workunit.client.0.smithi063.stdout:4/136: chown d1/d5/f23 165694 1 2024-03-22T07:15:51.713 INFO:tasks.workunit.client.0.smithi063.stdout:4/137: rename d1/f24 to d1/d13/f29 0 2024-03-22T07:15:51.714 INFO:tasks.workunit.client.0.smithi063.stdout:4/138: mknod d1/d13/d27/c2a 0 2024-03-22T07:15:51.714 INFO:tasks.workunit.client.0.smithi063.stdout:4/139: fdatasync d1/d5/f20 0 2024-03-22T07:15:51.721 INFO:tasks.workunit.client.0.smithi063.stdout:2/200: dwrite d0/f8 [0,4194304] 0 2024-03-22T07:15:51.721 INFO:tasks.workunit.client.0.smithi063.stdout:2/201: truncate d0/de/f30 478734 0 2024-03-22T07:15:51.721 INFO:tasks.workunit.client.0.smithi063.stdout:6/173: dwrite d3/f18 [0,4194304] 0 2024-03-22T07:15:51.722 INFO:tasks.workunit.client.0.smithi063.stdout:6/174: truncate d3/d15/d16/d1e/f20 987037 0 2024-03-22T07:15:51.722 INFO:tasks.workunit.client.0.smithi063.stdout:7/160: dwrite d5/d14/d1d/d29/f36 [0,4194304] 0 2024-03-22T07:15:51.722 INFO:tasks.workunit.client.0.smithi063.stdout:8/166: dwrite d0/d1b/f1e [0,4194304] 0 2024-03-22T07:15:51.726 INFO:tasks.workunit.client.0.smithi063.stdout:1/199: getdents d4/df 0 2024-03-22T07:15:51.726 INFO:tasks.workunit.client.0.smithi063.stdout:1/200: truncate d4/d14/f21 968285 0 2024-03-22T07:15:51.727 INFO:tasks.workunit.client.0.smithi063.stdout:6/175: creat d3/d15/f41 x:0 0 0 2024-03-22T07:15:51.728 INFO:tasks.workunit.client.0.smithi063.stdout:4/140: dwrite d1/f11 [0,4194304] 0 2024-03-22T07:15:51.730 INFO:tasks.workunit.client.0.smithi063.stdout:7/161: creat d5/d14/f37 x:0 0 0 2024-03-22T07:15:51.730 INFO:tasks.workunit.client.0.smithi063.stdout:7/162: write d5/d16/f22 [279906,17028] 0 2024-03-22T07:15:51.730 INFO:tasks.workunit.client.0.smithi063.stdout:7/163: stat d5/d16/f1e 0 2024-03-22T07:15:51.733 INFO:tasks.workunit.client.0.smithi063.stdout:2/202: dread d0/de/d12/f27 [0,4194304] 0 2024-03-22T07:15:51.748 INFO:tasks.workunit.client.0.smithi063.stdout:1/201: symlink d4/df/d13/d39/l45 0 2024-03-22T07:15:51.749 INFO:tasks.workunit.client.0.smithi063.stdout:6/176: symlink d3/l42 0 2024-03-22T07:15:51.752 INFO:tasks.workunit.client.0.smithi063.stdout:2/203: creat d0/de/d12/d20/f3f x:0 0 0 2024-03-22T07:15:51.753 INFO:tasks.workunit.client.0.smithi063.stdout:2/204: dread - d0/de/f31 zero size 2024-03-22T07:15:51.753 INFO:tasks.workunit.client.0.smithi063.stdout:1/202: truncate d4/df/f15 159435 0 2024-03-22T07:15:51.755 INFO:tasks.workunit.client.0.smithi063.stdout:7/164: dwrite d5/d16/f22 [0,4194304] 0 2024-03-22T07:15:51.755 INFO:tasks.workunit.client.0.smithi063.stdout:8/167: dwrite d0/f22 [4194304,4194304] 0 2024-03-22T07:15:51.755 INFO:tasks.workunit.client.0.smithi063.stdout:4/141: rmdir d1/d5/dc 39 2024-03-22T07:15:51.757 INFO:tasks.workunit.client.0.smithi063.stdout:2/205: rename d0/de/d1b/d32/c34 to d0/de/c40 0 2024-03-22T07:15:51.770 INFO:tasks.workunit.client.0.smithi063.stdout:7/165: dwrite d5/f9 [4194304,4194304] 0 2024-03-22T07:15:51.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:51 smithi069 ceph-mon[36441]: pgmap v248: 65 pgs: 65 active+clean; 242 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 6.2 MiB/s rd, 14 MiB/s wr, 996 op/s 2024-03-22T07:15:51.786 INFO:tasks.workunit.client.0.smithi063.stdout:6/177: getdents d3/d15 0 2024-03-22T07:15:51.791 INFO:tasks.workunit.client.0.smithi063.stdout:4/142: mknod d1/d13/d27/c2b 0 2024-03-22T07:15:51.792 INFO:tasks.workunit.client.0.smithi063.stdout:2/206: fsync d0/f13 0 2024-03-22T07:15:51.793 INFO:tasks.workunit.client.0.smithi063.stdout:8/168: rmdir d0/d26 39 2024-03-22T07:15:51.797 INFO:tasks.workunit.client.0.smithi063.stdout:6/178: link d3/d6/d35/d3a/f40 d3/d15/d16/f43 0 2024-03-22T07:15:51.797 INFO:tasks.workunit.client.0.smithi063.stdout:6/179: read d3/d1c/f1d [561843,43915] 0 2024-03-22T07:15:51.798 INFO:tasks.workunit.client.0.smithi063.stdout:4/143: mkdir d1/d13/d27/d2c 0 2024-03-22T07:15:51.798 INFO:tasks.workunit.client.0.smithi063.stdout:4/144: creat d1/d5/f2d x:0 0 0 2024-03-22T07:15:51.800 INFO:tasks.workunit.client.0.smithi063.stdout:2/207: creat d0/f41 x:0 0 0 2024-03-22T07:15:51.801 INFO:tasks.workunit.client.0.smithi063.stdout:8/169: write d0/d24/f29 [3514440,90766] 0 2024-03-22T07:15:51.801 INFO:tasks.workunit.client.0.smithi063.stdout:8/170: chown d0/d18/c1d 11 1 2024-03-22T07:15:51.801 INFO:tasks.workunit.client.0.smithi063.stdout:6/180: mknod d3/d6/c44 0 2024-03-22T07:15:51.801 INFO:tasks.workunit.client.0.smithi063.stdout:6/181: write d3/d6/fc [5009889,31940] 0 2024-03-22T07:15:51.801 INFO:tasks.workunit.client.0.smithi063.stdout:6/182: write d3/d15/d16/f43 [2957927,117934] 0 2024-03-22T07:15:51.803 INFO:tasks.workunit.client.0.smithi063.stdout:4/145: stat d1/d5/dc/f10 0 2024-03-22T07:15:51.803 INFO:tasks.workunit.client.0.smithi063.stdout:4/146: write d1/d5/dc/f1d [3212771,42591] 0 2024-03-22T07:15:51.804 INFO:tasks.workunit.client.0.smithi063.stdout:4/147: creat d1/d13/f2e x:0 0 0 2024-03-22T07:15:51.804 INFO:tasks.workunit.client.0.smithi063.stdout:4/148: read - d1/d5/f28 zero size 2024-03-22T07:15:51.805 INFO:tasks.workunit.client.0.smithi063.stdout:8/171: mknod d0/d1b/c37 0 2024-03-22T07:15:51.807 INFO:tasks.workunit.client.0.smithi063.stdout:6/183: getdents d3/d1c 0 2024-03-22T07:15:51.817 INFO:tasks.workunit.client.0.smithi063.stdout:6/184: rename d3/d15/d29/c3d to d3/c45 0 2024-03-22T07:15:51.817 INFO:tasks.workunit.client.0.smithi063.stdout:6/185: chown d3/d15/d29/c2b 158950378 1 2024-03-22T07:15:51.827 INFO:tasks.workunit.client.0.smithi063.stdout:9/104: sync 2024-03-22T07:15:51.827 INFO:tasks.workunit.client.0.smithi063.stdout:3/81: sync 2024-03-22T07:15:51.827 INFO:tasks.workunit.client.0.smithi063.stdout:9/105: stat f3 0 2024-03-22T07:15:51.827 INFO:tasks.workunit.client.0.smithi063.stdout:3/82: read da/fb [1761752,88353] 0 2024-03-22T07:15:51.827 INFO:tasks.workunit.client.0.smithi063.stdout:9/106: rename d8/d17/l1b to d8/d17/d1e/l1f 0 2024-03-22T07:15:51.827 INFO:tasks.workunit.client.0.smithi063.stdout:3/83: rename c4 to da/c1b 0 2024-03-22T07:15:51.827 INFO:tasks.workunit.client.0.smithi063.stdout:3/84: creat da/f1c x:0 0 0 2024-03-22T07:15:51.827 INFO:tasks.workunit.client.0.smithi063.stdout:3/85: write da/f18 [5149190,22296] 0 2024-03-22T07:15:51.831 INFO:tasks.workunit.client.0.smithi063.stdout:4/149: dwrite d1/f21 [4194304,4194304] 0 2024-03-22T07:15:51.833 INFO:tasks.workunit.client.0.smithi063.stdout:9/107: read d8/fb [730234,62269] 0 2024-03-22T07:15:51.833 INFO:tasks.workunit.client.0.smithi063.stdout:9/108: stat d8/fd 0 2024-03-22T07:15:51.833 INFO:tasks.workunit.client.0.smithi063.stdout:8/172: dwrite d0/f22 [4194304,4194304] 0 2024-03-22T07:15:51.834 INFO:tasks.workunit.client.0.smithi063.stdout:8/173: write d0/d26/f34 [280881,53797] 0 2024-03-22T07:15:51.834 INFO:tasks.workunit.client.0.smithi063.stdout:8/174: chown d0/d1b 20 1 2024-03-22T07:15:51.834 INFO:tasks.workunit.client.0.smithi063.stdout:8/175: fsync d0/d24/f35 0 2024-03-22T07:15:51.838 INFO:tasks.workunit.client.0.smithi063.stdout:9/109: rename d8/d16 to d8/d19/d20 0 2024-03-22T07:15:51.839 INFO:tasks.workunit.client.0.smithi063.stdout:9/110: truncate d8/fa 1256136 0 2024-03-22T07:15:51.839 INFO:tasks.workunit.client.0.smithi063.stdout:9/111: write f3 [385453,90209] 0 2024-03-22T07:15:51.840 INFO:tasks.workunit.client.0.smithi063.stdout:8/176: symlink d0/l38 0 2024-03-22T07:15:51.841 INFO:tasks.workunit.client.0.smithi063.stdout:8/177: truncate d0/d24/f35 347743 0 2024-03-22T07:15:51.842 INFO:tasks.workunit.client.0.smithi063.stdout:9/112: creat d8/d19/d20/f21 x:0 0 0 2024-03-22T07:15:51.845 INFO:tasks.workunit.client.0.smithi063.stdout:8/178: creat d0/d1b/d28/f39 x:0 0 0 2024-03-22T07:15:51.846 INFO:tasks.workunit.client.0.smithi063.stdout:8/179: write d0/d24/f35 [999762,90728] 0 2024-03-22T07:15:51.846 INFO:tasks.workunit.client.0.smithi063.stdout:8/180: readlink d0/d26/l2b 0 2024-03-22T07:15:51.846 INFO:tasks.workunit.client.0.smithi063.stdout:4/150: rmdir d1/d5 39 2024-03-22T07:15:51.846 INFO:tasks.workunit.client.0.smithi063.stdout:6/186: dread d3/f19 [0,4194304] 0 2024-03-22T07:15:51.846 INFO:tasks.workunit.client.0.smithi063.stdout:9/113: mkdir d8/d19/d20/d22 0 2024-03-22T07:15:51.846 INFO:tasks.workunit.client.0.smithi063.stdout:9/114: fsync d8/fc 0 2024-03-22T07:15:51.846 INFO:tasks.workunit.client.0.smithi063.stdout:9/115: write d8/f1a [162986,17801] 0 2024-03-22T07:15:51.846 INFO:tasks.workunit.client.0.smithi063.stdout:4/151: creat d1/d5/f2f x:0 0 0 2024-03-22T07:15:51.849 INFO:tasks.workunit.client.0.smithi063.stdout:8/181: write d0/f19 [1458142,124715] 0 2024-03-22T07:15:51.854 INFO:tasks.workunit.client.0.smithi063.stdout:0/149: sync 2024-03-22T07:15:51.855 INFO:tasks.workunit.client.0.smithi063.stdout:3/86: dwrite da/f18 [0,4194304] 0 2024-03-22T07:15:51.860 INFO:tasks.workunit.client.0.smithi063.stdout:0/150: mkdir da/d2e 0 2024-03-22T07:15:51.860 INFO:tasks.workunit.client.0.smithi063.stdout:8/182: truncate d0/f22 4055700 0 2024-03-22T07:15:51.860 INFO:tasks.workunit.client.0.smithi063.stdout:3/87: mknod da/c1d 0 2024-03-22T07:15:51.862 INFO:tasks.workunit.client.0.smithi063.stdout:3/88: mknod da/c1e 0 2024-03-22T07:15:51.862 INFO:tasks.workunit.client.0.smithi063.stdout:3/89: truncate da/fd 576620 0 2024-03-22T07:15:51.862 INFO:tasks.workunit.client.0.smithi063.stdout:3/90: chown da/c1b 1 1 2024-03-22T07:15:51.862 INFO:tasks.workunit.client.0.smithi063.stdout:3/91: write da/f1c [661889,76264] 0 2024-03-22T07:15:51.873 INFO:tasks.workunit.client.0.smithi063.stdout:0/151: dread f1 [0,4194304] 0 2024-03-22T07:15:51.873 INFO:tasks.workunit.client.0.smithi063.stdout:4/152: write d1/f11 [3755745,61755] 0 2024-03-22T07:15:51.873 INFO:tasks.workunit.client.0.smithi063.stdout:0/152: creat da/de/d17/f2f x:0 0 0 2024-03-22T07:15:51.877 INFO:tasks.workunit.client.0.smithi063.stdout:3/92: read da/f18 [1554219,70595] 0 2024-03-22T07:15:51.877 INFO:tasks.workunit.client.0.smithi063.stdout:0/153: unlink da/de/d17/l26 0 2024-03-22T07:15:51.878 INFO:tasks.workunit.client.0.smithi063.stdout:3/93: dread da/fd [0,4194304] 0 2024-03-22T07:15:51.878 INFO:tasks.workunit.client.0.smithi063.stdout:3/94: chown da/l13 7867 1 2024-03-22T07:15:51.879 INFO:tasks.workunit.client.0.smithi063.stdout:6/187: dwrite d3/d15/f22 [0,4194304] 0 2024-03-22T07:15:51.879 INFO:tasks.workunit.client.0.smithi063.stdout:9/116: dwrite d8/d19/d20/f21 [0,4194304] 0 2024-03-22T07:15:51.879 INFO:tasks.workunit.client.0.smithi063.stdout:6/188: readlink d3/l30 0 2024-03-22T07:15:51.881 INFO:tasks.workunit.client.0.smithi063.stdout:3/95: rename l8 to da/d16/l1f 0 2024-03-22T07:15:51.883 INFO:tasks.workunit.client.0.smithi063.stdout:4/153: dread d1/d5/f7 [0,4194304] 0 2024-03-22T07:15:51.883 INFO:tasks.workunit.client.0.smithi063.stdout:4/154: write d1/d5/f9 [3508045,41917] 0 2024-03-22T07:15:51.885 INFO:tasks.workunit.client.0.smithi063.stdout:6/189: link d3/d15/d16/d1e/c3b d3/d1c/d32/c46 0 2024-03-22T07:15:51.887 INFO:tasks.workunit.client.0.smithi063.stdout:8/183: write d0/d1b/f1e [1423615,6408] 0 2024-03-22T07:15:51.888 INFO:tasks.workunit.client.0.smithi063.stdout:8/184: creat d0/d24/f3a x:0 0 0 2024-03-22T07:15:51.888 INFO:tasks.workunit.client.0.smithi063.stdout:0/154: dread f9 [0,4194304] 0 2024-03-22T07:15:51.889 INFO:tasks.workunit.client.0.smithi063.stdout:0/155: fdatasync f1 0 2024-03-22T07:15:51.889 INFO:tasks.workunit.client.0.smithi063.stdout:0/156: write f5 [3782582,105193] 0 2024-03-22T07:15:51.891 INFO:tasks.workunit.client.0.smithi063.stdout:3/96: rename da/l13 to da/d16/l20 0 2024-03-22T07:15:51.893 INFO:tasks.workunit.client.0.smithi063.stdout:4/155: creat d1/d13/d27/f30 x:0 0 0 2024-03-22T07:15:51.893 INFO:tasks.workunit.client.0.smithi063.stdout:4/156: dread - d1/d5/f2f zero size 2024-03-22T07:15:51.899 INFO:tasks.workunit.client.0.smithi063.stdout:8/185: write d0/f17 [2275785,3407] 0 2024-03-22T07:15:51.901 INFO:tasks.workunit.client.0.smithi063.stdout:0/157: symlink da/l30 0 2024-03-22T07:15:51.902 INFO:tasks.workunit.client.0.smithi063.stdout:3/97: rename da/fd to da/f21 0 2024-03-22T07:15:51.904 INFO:tasks.workunit.client.0.smithi063.stdout:0/158: unlink da/c16 0 2024-03-22T07:15:51.905 INFO:tasks.workunit.client.0.smithi063.stdout:3/98: dread da/f21 [0,4194304] 0 2024-03-22T07:15:51.905 INFO:tasks.workunit.client.0.smithi063.stdout:3/99: fsync da/f1c 0 2024-03-22T07:15:51.906 INFO:tasks.workunit.client.0.smithi063.stdout:0/159: rename f9 to da/de/f31 0 2024-03-22T07:15:51.907 INFO:tasks.workunit.client.0.smithi063.stdout:3/100: symlink da/d16/l22 0 2024-03-22T07:15:51.907 INFO:tasks.workunit.client.0.smithi063.stdout:3/101: write da/d16/f1a [114666,110904] 0 2024-03-22T07:15:51.907 INFO:tasks.workunit.client.0.smithi063.stdout:3/102: fsync da/fb 0 2024-03-22T07:15:51.908 INFO:tasks.workunit.client.0.smithi063.stdout:0/160: creat da/d2e/f32 x:0 0 0 2024-03-22T07:15:51.908 INFO:tasks.workunit.client.0.smithi063.stdout:0/161: stat da/de/f18 0 2024-03-22T07:15:51.924 INFO:tasks.workunit.client.0.smithi063.stdout:6/190: dwrite d3/d6/f28 [0,4194304] 0 2024-03-22T07:15:51.927 INFO:tasks.workunit.client.0.smithi063.stdout:6/191: mkdir d3/d15/d16/d47 0 2024-03-22T07:15:51.927 INFO:tasks.workunit.client.0.smithi063.stdout:6/192: getdents d3/d6/d2a/d38 0 2024-03-22T07:15:51.928 INFO:tasks.workunit.client.0.smithi063.stdout:6/193: creat d3/d6/d2a/f48 x:0 0 0 2024-03-22T07:15:51.941 INFO:tasks.workunit.client.0.smithi063.stdout:3/103: dwrite da/f18 [4194304,4194304] 0 2024-03-22T07:15:51.942 INFO:tasks.workunit.client.0.smithi063.stdout:4/157: dwrite d1/d5/f23 [0,4194304] 0 2024-03-22T07:15:51.942 INFO:tasks.workunit.client.0.smithi063.stdout:3/104: symlink da/d16/l23 0 2024-03-22T07:15:51.945 INFO:tasks.workunit.client.0.smithi063.stdout:4/158: link d1/d5/f2d d1/d13/f31 0 2024-03-22T07:15:51.947 INFO:tasks.workunit.client.0.smithi063.stdout:4/159: creat d1/f32 x:0 0 0 2024-03-22T07:15:51.947 INFO:tasks.workunit.client.0.smithi063.stdout:4/160: dread - d1/d13/d27/f30 zero size 2024-03-22T07:15:51.947 INFO:tasks.workunit.client.0.smithi063.stdout:0/162: rmdir da/d2e 39 2024-03-22T07:15:51.949 INFO:tasks.workunit.client.0.smithi063.stdout:8/186: dwrite d0/d1b/f20 [0,4194304] 0 2024-03-22T07:15:51.949 INFO:tasks.workunit.client.0.smithi063.stdout:8/187: read - d0/d26/f27 zero size 2024-03-22T07:15:51.949 INFO:tasks.workunit.client.0.smithi063.stdout:8/188: write d0/d1b/f1e [1691375,107698] 0 2024-03-22T07:15:51.949 INFO:tasks.workunit.client.0.smithi063.stdout:8/189: getdents d0/d24/d33 0 2024-03-22T07:15:51.949 INFO:tasks.workunit.client.0.smithi063.stdout:3/105: getdents da 0 2024-03-22T07:15:51.949 INFO:tasks.workunit.client.0.smithi063.stdout:6/194: dwrite d3/d6/f3e [0,4194304] 0 2024-03-22T07:15:51.949 INFO:tasks.workunit.client.0.smithi063.stdout:6/195: truncate d3/f14 380322 0 2024-03-22T07:15:51.949 INFO:tasks.workunit.client.0.smithi063.stdout:4/161: getdents d1/d5/dc 0 2024-03-22T07:15:51.951 INFO:tasks.workunit.client.0.smithi063.stdout:0/163: rename da/d2e/f32 to da/d2e/f33 0 2024-03-22T07:15:51.956 INFO:tasks.workunit.client.0.smithi063.stdout:8/190: read d0/fc [160124,69510] 0 2024-03-22T07:15:51.957 INFO:tasks.workunit.client.0.smithi063.stdout:6/196: dread d3/d6/f10 [0,4194304] 0 2024-03-22T07:15:51.957 INFO:tasks.workunit.client.0.smithi063.stdout:3/106: rename da/c19 to da/c24 0 2024-03-22T07:15:51.960 INFO:tasks.workunit.client.0.smithi063.stdout:5/142: sync 2024-03-22T07:15:51.962 INFO:tasks.workunit.client.0.smithi063.stdout:4/162: link d1/d5/f9 d1/d5/f33 0 2024-03-22T07:15:51.963 INFO:tasks.workunit.client.0.smithi063.stdout:0/164: mknod da/de/d17/c34 0 2024-03-22T07:15:51.963 INFO:tasks.workunit.client.0.smithi063.stdout:0/165: read - da/de/d17/f2f zero size 2024-03-22T07:15:51.963 INFO:tasks.workunit.client.0.smithi063.stdout:0/166: fsync f3 0 2024-03-22T07:15:51.964 INFO:tasks.workunit.client.0.smithi063.stdout:8/191: rename d0/d26/f2e to d0/d1b/d28/f3b 0 2024-03-22T07:15:51.966 INFO:tasks.workunit.client.0.smithi063.stdout:3/107: dread da/f21 [0,4194304] 0 2024-03-22T07:15:51.968 INFO:tasks.workunit.client.0.smithi063.stdout:0/167: symlink da/de/l35 0 2024-03-22T07:15:51.968 INFO:tasks.workunit.client.0.smithi063.stdout:0/168: stat da/de/d17/d25 0 2024-03-22T07:15:51.969 INFO:tasks.workunit.client.0.smithi063.stdout:8/192: rename d0/d26/c2c to d0/d24/c3c 0 2024-03-22T07:15:51.969 INFO:tasks.workunit.client.0.smithi063.stdout:8/193: dread - d0/d1b/d28/f39 zero size 2024-03-22T07:15:51.970 INFO:tasks.workunit.client.0.smithi063.stdout:3/108: chown da/d16/l23 303753374 1 2024-03-22T07:15:51.972 INFO:tasks.workunit.client.0.smithi063.stdout:0/169: mkdir da/de/d17/d25/d36 0 2024-03-22T07:15:51.978 INFO:tasks.workunit.client.0.smithi063.stdout:6/197: dwrite d3/d15/f22 [4194304,4194304] 0 2024-03-22T07:15:51.978 INFO:tasks.workunit.client.0.smithi063.stdout:6/198: fsync d3/d6/fa 0 2024-03-22T07:15:51.978 INFO:tasks.workunit.client.0.smithi063.stdout:6/199: readlink d3/d15/d16/l31 0 2024-03-22T07:15:51.978 INFO:tasks.workunit.client.0.smithi063.stdout:6/200: rmdir d3/d1c 39 2024-03-22T07:15:51.980 INFO:tasks.workunit.client.0.smithi063.stdout:6/201: creat d3/d6/d2a/d38/f49 x:0 0 0 2024-03-22T07:15:51.981 INFO:tasks.workunit.client.0.smithi063.stdout:6/202: symlink d3/d15/d16/d47/l4a 0 2024-03-22T07:15:51.981 INFO:tasks.workunit.client.0.smithi063.stdout:6/203: write d3/d15/d16/d1e/f20 [1131443,57415] 0 2024-03-22T07:15:51.981 INFO:tasks.workunit.client.0.smithi063.stdout:8/194: dread d0/f22 [0,4194304] 0 2024-03-22T07:15:51.981 INFO:tasks.workunit.client.0.smithi063.stdout:5/143: dwrite d8/f10 [0,4194304] 0 2024-03-22T07:15:51.982 INFO:tasks.workunit.client.0.smithi063.stdout:5/144: chown d8/d13/d28 23464 1 2024-03-22T07:15:51.982 INFO:tasks.workunit.client.0.smithi063.stdout:4/163: getdents d1/d5 0 2024-03-22T07:15:51.983 INFO:tasks.workunit.client.0.smithi063.stdout:3/109: read da/f11 [573754,71528] 0 2024-03-22T07:15:51.984 INFO:tasks.workunit.client.0.smithi063.stdout:8/195: mkdir d0/d26/d2d/d3d 0 2024-03-22T07:15:51.984 INFO:tasks.workunit.client.0.smithi063.stdout:3/110: dread da/f21 [0,4194304] 0 2024-03-22T07:15:51.985 INFO:tasks.workunit.client.0.smithi063.stdout:5/145: mknod d8/d13/c2d 0 2024-03-22T07:15:51.985 INFO:tasks.workunit.client.0.smithi063.stdout:5/146: chown d8/d13/l19 46837 1 2024-03-22T07:15:51.988 INFO:tasks.workunit.client.0.smithi063.stdout:8/196: mkdir d0/d1b/d28/d3e 0 2024-03-22T07:15:51.988 INFO:tasks.workunit.client.0.smithi063.stdout:8/197: chown d0/d1b/d28/l31 5524 1 2024-03-22T07:15:51.989 INFO:tasks.workunit.client.0.smithi063.stdout:3/111: fsync da/f18 0 2024-03-22T07:15:51.990 INFO:tasks.workunit.client.0.smithi063.stdout:0/170: getdents da/de/d17/d25 0 2024-03-22T07:15:51.995 INFO:tasks.workunit.client.0.smithi063.stdout:6/204: write d3/d15/f22 [5217183,113790] 0 2024-03-22T07:15:51.995 INFO:tasks.workunit.client.0.smithi063.stdout:3/112: rename da/d16/f1a to da/d16/f25 0 2024-03-22T07:15:51.995 INFO:tasks.workunit.client.0.smithi063.stdout:0/171: mkdir da/d37 0 2024-03-22T07:15:51.996 INFO:tasks.workunit.client.0.smithi063.stdout:6/205: creat d3/d1c/f4b x:0 0 0 2024-03-22T07:15:51.997 INFO:tasks.workunit.client.0.smithi063.stdout:3/113: mkdir da/d16/d26 0 2024-03-22T07:15:51.998 INFO:tasks.workunit.client.0.smithi063.stdout:3/114: dread da/f21 [0,4194304] 0 2024-03-22T07:15:51.998 INFO:tasks.workunit.client.0.smithi063.stdout:3/115: write da/f18 [738313,87710] 0 2024-03-22T07:15:51.998 INFO:tasks.workunit.client.0.smithi063.stdout:0/172: creat da/de/d17/d25/f38 x:0 0 0 2024-03-22T07:15:51.999 INFO:tasks.workunit.client.0.smithi063.stdout:3/116: creat da/d16/d26/f27 x:0 0 0 2024-03-22T07:15:52.000 INFO:tasks.workunit.client.0.smithi063.stdout:0/173: mkdir da/de/d39 0 2024-03-22T07:15:52.000 INFO:tasks.workunit.client.0.smithi063.stdout:0/174: creat da/de/d17/f3a x:0 0 0 2024-03-22T07:15:52.000 INFO:tasks.workunit.client.0.smithi063.stdout:0/175: readlink da/de/l35 0 2024-03-22T07:15:52.000 INFO:tasks.workunit.client.0.smithi063.stdout:3/117: dread da/d16/f25 [0,4194304] 0 2024-03-22T07:15:52.006 INFO:tasks.workunit.client.0.smithi063.stdout:5/147: rmdir d8/d13 39 2024-03-22T07:15:52.006 INFO:tasks.workunit.client.0.smithi063.stdout:5/148: readlink d8/d13/l19 0 2024-03-22T07:15:52.006 INFO:tasks.workunit.client.0.smithi063.stdout:0/176: dread da/de/f31 [0,4194304] 0 2024-03-22T07:15:52.008 INFO:tasks.workunit.client.0.smithi063.stdout:6/206: write d3/f9 [6730112,58883] 0 2024-03-22T07:15:52.008 INFO:tasks.workunit.client.0.smithi063.stdout:0/177: rename da/de/d17/c29 to da/de/d39/c3b 0 2024-03-22T07:15:52.008 INFO:tasks.workunit.client.0.smithi063.stdout:6/207: fsync d3/d6/d2a/d38/f49 0 2024-03-22T07:15:52.009 INFO:tasks.workunit.client.0.smithi063.stdout:4/164: dwrite d1/d5/f12 [4194304,4194304] 0 2024-03-22T07:15:52.009 INFO:tasks.workunit.client.0.smithi063.stdout:4/165: dread - d1/d5/f2d zero size 2024-03-22T07:15:52.009 INFO:tasks.workunit.client.0.smithi063.stdout:0/178: write da/f2a [1143415,122585] 0 2024-03-22T07:15:52.010 INFO:tasks.workunit.client.0.smithi063.stdout:4/166: symlink d1/d5/dc/l34 0 2024-03-22T07:15:52.010 INFO:tasks.workunit.client.0.smithi063.stdout:4/167: chown d1/d5/f20 199 1 2024-03-22T07:15:52.011 INFO:tasks.workunit.client.0.smithi063.stdout:4/168: creat d1/d13/f35 x:0 0 0 2024-03-22T07:15:52.011 INFO:tasks.workunit.client.0.smithi063.stdout:6/208: dread d3/d6/f2c [8388608,4194304] 0 2024-03-22T07:15:52.015 INFO:tasks.workunit.client.0.smithi063.stdout:3/118: truncate da/f18 1797023 0 2024-03-22T07:15:52.017 INFO:tasks.workunit.client.0.smithi063.stdout:5/149: dread d8/f10 [0,4194304] 0 2024-03-22T07:15:52.017 INFO:tasks.workunit.client.0.smithi063.stdout:5/150: stat d8/d13/d1e/l29 0 2024-03-22T07:15:52.017 INFO:tasks.workunit.client.0.smithi063.stdout:3/119: mknod da/d16/d26/c28 0 2024-03-22T07:15:52.018 INFO:tasks.workunit.client.0.smithi063.stdout:5/151: creat d8/f2e x:0 0 0 2024-03-22T07:15:52.019 INFO:tasks.workunit.client.0.smithi063.stdout:3/120: stat da/c24 0 2024-03-22T07:15:52.019 INFO:tasks.workunit.client.0.smithi063.stdout:3/121: dread - da/d16/d26/f27 zero size 2024-03-22T07:15:52.019 INFO:tasks.workunit.client.0.smithi063.stdout:3/122: creat da/d16/d26/f29 x:0 0 0 2024-03-22T07:15:52.019 INFO:tasks.workunit.client.0.smithi063.stdout:5/152: creat d8/d13/d1e/f2f x:0 0 0 2024-03-22T07:15:52.023 INFO:tasks.workunit.client.0.smithi063.stdout:4/169: write d1/d5/f23 [2936446,10039] 0 2024-03-22T07:15:52.023 INFO:tasks.workunit.client.0.smithi063.stdout:4/170: rmdir d1/d13/d27 39 2024-03-22T07:15:52.023 INFO:tasks.workunit.client.0.smithi063.stdout:4/171: write d1/d5/f28 [3300,40837] 0 2024-03-22T07:15:52.023 INFO:tasks.workunit.client.0.smithi063.stdout:4/172: write d1/d13/d27/f30 [585100,125283] 0 2024-03-22T07:15:52.024 INFO:tasks.workunit.client.0.smithi063.stdout:4/173: symlink d1/l36 0 2024-03-22T07:15:52.025 INFO:tasks.workunit.client.0.smithi063.stdout:4/174: symlink d1/d5/dc/l37 0 2024-03-22T07:15:52.030 INFO:tasks.workunit.client.0.smithi063.stdout:4/175: mkdir d1/d5/d38 0 2024-03-22T07:15:52.030 INFO:tasks.workunit.client.0.smithi063.stdout:4/176: rename d1/d5/f22 to d1/d13/f39 0 2024-03-22T07:15:52.030 INFO:tasks.workunit.client.0.smithi063.stdout:4/177: dread - d1/d13/f31 zero size 2024-03-22T07:15:52.030 INFO:tasks.workunit.client.0.smithi063.stdout:4/178: readlink d1/d5/la 0 2024-03-22T07:15:52.031 INFO:tasks.workunit.client.0.smithi063.stdout:4/179: rename d1/d5/dc/c1e to d1/d13/d27/d2c/c3a 0 2024-03-22T07:15:52.031 INFO:tasks.workunit.client.0.smithi063.stdout:5/153: dread d8/f10 [0,4194304] 0 2024-03-22T07:15:52.031 INFO:tasks.workunit.client.0.smithi063.stdout:8/198: dwrite d0/d1b/f1e [4194304,4194304] 0 2024-03-22T07:15:52.031 INFO:tasks.workunit.client.0.smithi063.stdout:5/154: truncate d8/fe 3184243 0 2024-03-22T07:15:52.033 INFO:tasks.workunit.client.0.smithi063.stdout:4/180: unlink d1/d5/dc/l1a 0 2024-03-22T07:15:52.039 INFO:tasks.workunit.client.0.smithi063.stdout:0/179: dwrite f1 [4194304,4194304] 0 2024-03-22T07:15:52.040 INFO:tasks.workunit.client.0.smithi063.stdout:8/199: unlink d0/d1f/c25 0 2024-03-22T07:15:52.041 INFO:tasks.workunit.client.0.smithi063.stdout:6/209: dwrite d3/d15/f41 [0,4194304] 0 2024-03-22T07:15:52.041 INFO:tasks.workunit.client.0.smithi063.stdout:5/155: getdents d8/d13 0 2024-03-22T07:15:52.041 INFO:tasks.workunit.client.0.smithi063.stdout:5/156: fdatasync d8/f14 0 2024-03-22T07:15:52.041 INFO:tasks.workunit.client.0.smithi063.stdout:5/157: dread - d8/d13/d1e/f27 zero size 2024-03-22T07:15:52.043 INFO:tasks.workunit.client.0.smithi063.stdout:4/181: symlink d1/d5/dc/l3b 0 2024-03-22T07:15:52.050 INFO:tasks.workunit.client.0.smithi063.stdout:8/200: unlink d0/f17 0 2024-03-22T07:15:52.052 INFO:tasks.workunit.client.0.smithi063.stdout:3/123: dwrite da/f18 [0,4194304] 0 2024-03-22T07:15:52.052 INFO:tasks.workunit.client.0.smithi063.stdout:3/124: creat da/d16/d26/f2a x:0 0 0 2024-03-22T07:15:52.054 INFO:tasks.workunit.client.0.smithi063.stdout:0/180: write da/de/f23 [4174577,48754] 0 2024-03-22T07:15:52.058 INFO:tasks.workunit.client.0.smithi063.stdout:6/210: getdents d3/d15/d29 0 2024-03-22T07:15:52.059 INFO:tasks.workunit.client.0.smithi063.stdout:8/201: mkdir d0/d18/d3f 0 2024-03-22T07:15:52.059 INFO:tasks.workunit.client.0.smithi063.stdout:8/202: dread - d0/d26/f27 zero size 2024-03-22T07:15:52.059 INFO:tasks.workunit.client.0.smithi063.stdout:8/203: write d0/d1b/f1e [9420375,125481] 0 2024-03-22T07:15:52.062 INFO:tasks.workunit.client.0.smithi063.stdout:0/181: mkdir da/de/d17/d25/d3c 0 2024-03-22T07:15:52.062 INFO:tasks.workunit.client.0.smithi063.stdout:0/182: readlink da/de/d17/l1b 0 2024-03-22T07:15:52.064 INFO:tasks.workunit.client.0.smithi063.stdout:0/183: mknod da/d2e/c3d 0 2024-03-22T07:15:52.064 INFO:tasks.workunit.client.0.smithi063.stdout:0/184: creat da/de/d17/d25/f3e x:0 0 0 2024-03-22T07:15:52.064 INFO:tasks.workunit.client.0.smithi063.stdout:0/185: chown da/de/l2d 0 1 2024-03-22T07:15:52.064 INFO:tasks.workunit.client.0.smithi063.stdout:0/186: write da/de/f24 [394040,105382] 0 2024-03-22T07:15:52.064 INFO:tasks.workunit.client.0.smithi063.stdout:0/187: write da/de/f31 [232291,10144] 0 2024-03-22T07:15:52.067 INFO:tasks.workunit.client.0.smithi063.stdout:0/188: rename f3 to da/de/d17/d25/d3c/f3f 0 2024-03-22T07:15:52.067 INFO:tasks.workunit.client.0.smithi063.stdout:0/189: creat da/f40 x:0 0 0 2024-03-22T07:15:52.067 INFO:tasks.workunit.client.0.smithi063.stdout:8/204: dread d0/d24/f35 [0,4194304] 0 2024-03-22T07:15:52.069 INFO:tasks.workunit.client.0.smithi063.stdout:0/190: creat da/de/d39/f41 x:0 0 0 2024-03-22T07:15:52.069 INFO:tasks.workunit.client.0.smithi063.stdout:0/191: chown da/d2e 530 1 2024-03-22T07:15:52.070 INFO:tasks.workunit.client.0.smithi063.stdout:1/203: sync 2024-03-22T07:15:52.070 INFO:tasks.workunit.client.0.smithi063.stdout:7/166: sync 2024-03-22T07:15:52.070 INFO:tasks.workunit.client.0.smithi063.stdout:2/208: sync 2024-03-22T07:15:52.071 INFO:tasks.workunit.client.0.smithi063.stdout:8/205: rename d0/f12 to d0/d1f/f40 0 2024-03-22T07:15:52.072 INFO:tasks.workunit.client.0.smithi063.stdout:6/211: dread d3/d15/f23 [0,4194304] 0 2024-03-22T07:15:52.072 INFO:tasks.workunit.client.0.smithi063.stdout:7/167: mkdir d5/d38 0 2024-03-22T07:15:52.075 INFO:tasks.workunit.client.0.smithi063.stdout:7/168: creat d5/f39 x:0 0 0 2024-03-22T07:15:52.079 INFO:tasks.workunit.client.0.smithi063.stdout:5/158: dwrite d8/fe [0,4194304] 0 2024-03-22T07:15:52.079 INFO:tasks.workunit.client.0.smithi063.stdout:4/182: dwrite d1/f21 [4194304,4194304] 0 2024-03-22T07:15:52.080 INFO:tasks.workunit.client.0.smithi063.stdout:8/206: link d0/c7 d0/d1b/d28/c41 0 2024-03-22T07:15:52.080 INFO:tasks.workunit.client.0.smithi063.stdout:8/207: write d0/d24/f35 [1232992,95060] 0 2024-03-22T07:15:52.085 INFO:tasks.workunit.client.0.smithi063.stdout:5/159: readlink d8/lc 0 2024-03-22T07:15:52.085 INFO:tasks.workunit.client.0.smithi063.stdout:5/160: fdatasync d8/f2e 0 2024-03-22T07:15:52.085 INFO:tasks.workunit.client.0.smithi063.stdout:5/161: dread - d8/d13/d1e/f2f zero size 2024-03-22T07:15:52.087 INFO:tasks.workunit.client.0.smithi063.stdout:0/192: truncate f1 1937792 0 2024-03-22T07:15:52.087 INFO:tasks.workunit.client.0.smithi063.stdout:0/193: readlink da/de/l2d 0 2024-03-22T07:15:52.088 INFO:tasks.workunit.client.0.smithi063.stdout:0/194: read da/de/d17/d25/d3c/f3f [1784746,120131] 0 2024-03-22T07:15:52.088 INFO:tasks.workunit.client.0.smithi063.stdout:4/183: mkdir d1/d5/d3c 0 2024-03-22T07:15:52.088 INFO:tasks.workunit.client.0.smithi063.stdout:4/184: chown d1/d13/f2e 21266 1 2024-03-22T07:15:52.088 INFO:tasks.workunit.client.0.smithi063.stdout:4/185: stat d1/d5/d3c 0 2024-03-22T07:15:52.090 INFO:tasks.workunit.client.0.smithi063.stdout:8/208: chown d0/l16 101294 1 2024-03-22T07:15:52.093 INFO:tasks.workunit.client.0.smithi063.stdout:9/117: sync 2024-03-22T07:15:52.096 INFO:tasks.workunit.client.0.smithi063.stdout:3/125: rmdir da/d16 39 2024-03-22T07:15:52.100 INFO:tasks.workunit.client.0.smithi063.stdout:3/126: write da/f1c [438673,103707] 0 2024-03-22T07:15:52.100 INFO:tasks.workunit.client.0.smithi063.stdout:3/127: write da/f11 [238790,108331] 0 2024-03-22T07:15:52.103 INFO:tasks.workunit.client.0.smithi063.stdout:3/128: rmdir da/d16 39 2024-03-22T07:15:52.103 INFO:tasks.workunit.client.0.smithi063.stdout:9/118: dread d8/fc [0,4194304] 0 2024-03-22T07:15:52.103 INFO:tasks.workunit.client.0.smithi063.stdout:9/119: readlink l5 0 2024-03-22T07:15:52.105 INFO:tasks.workunit.client.0.smithi063.stdout:3/129: unlink da/c24 0 2024-03-22T07:15:52.105 INFO:tasks.workunit.client.0.smithi063.stdout:8/209: dread d0/f19 [0,4194304] 0 2024-03-22T07:15:52.106 INFO:tasks.workunit.client.0.smithi063.stdout:8/210: fdatasync d0/d1b/d28/f3b 0 2024-03-22T07:15:52.106 INFO:tasks.workunit.client.0.smithi063.stdout:8/211: write d0/f22 [4415482,72206] 0 2024-03-22T07:15:52.106 INFO:tasks.workunit.client.0.smithi063.stdout:8/212: dread - d0/d1b/d28/f39 zero size 2024-03-22T07:15:52.106 INFO:tasks.workunit.client.0.smithi063.stdout:8/213: creat d0/d24/f42 x:0 0 0 2024-03-22T07:15:52.106 INFO:tasks.workunit.client.0.smithi063.stdout:9/120: creat d8/d19/f23 x:0 0 0 2024-03-22T07:15:52.106 INFO:tasks.workunit.client.0.smithi063.stdout:3/130: write da/fb [3477712,52541] 0 2024-03-22T07:15:52.108 INFO:tasks.workunit.client.0.smithi063.stdout:9/121: mkdir d8/d19/d20/d22/d24 0 2024-03-22T07:15:52.109 INFO:tasks.workunit.client.0.smithi063.stdout:3/131: mknod da/c2b 0 2024-03-22T07:15:52.110 INFO:tasks.workunit.client.0.smithi063.stdout:3/132: symlink da/d16/d26/l2c 0 2024-03-22T07:15:52.110 INFO:tasks.workunit.client.0.smithi063.stdout:3/133: write da/d16/d26/f27 [929214,71228] 0 2024-03-22T07:15:52.110 INFO:tasks.workunit.client.0.smithi063.stdout:3/134: readlink da/d16/l20 0 2024-03-22T07:15:52.110 INFO:tasks.workunit.client.0.smithi063.stdout:3/135: chown da/f21 14200 1 2024-03-22T07:15:52.110 INFO:tasks.workunit.client.0.smithi063.stdout:3/136: write da/d16/d26/f2a [380454,117671] 0 2024-03-22T07:15:52.115 INFO:tasks.workunit.client.0.smithi063.stdout:1/204: dwrite d4/df/f1a [0,4194304] 0 2024-03-22T07:15:52.117 INFO:tasks.workunit.client.0.smithi063.stdout:1/205: link d4/df/f44 d4/d14/f46 0 2024-03-22T07:15:52.118 INFO:tasks.workunit.client.0.smithi063.stdout:1/206: fsync d4/f1e 0 2024-03-22T07:15:52.119 INFO:tasks.workunit.client.0.smithi063.stdout:1/207: truncate d4/fb 4114272 0 2024-03-22T07:15:52.122 INFO:tasks.workunit.client.0.smithi063.stdout:1/208: getdents d4/df/d2c 0 2024-03-22T07:15:52.122 INFO:tasks.workunit.client.0.smithi063.stdout:1/209: chown d4/df/d13/f25 151680041 1 2024-03-22T07:15:52.125 INFO:tasks.workunit.client.0.smithi063.stdout:7/169: dwrite d5/f17 [0,4194304] 0 2024-03-22T07:15:52.126 INFO:tasks.workunit.client.0.smithi063.stdout:1/210: dread d4/df/d13/f25 [0,4194304] 0 2024-03-22T07:15:52.126 INFO:tasks.workunit.client.0.smithi063.stdout:6/212: dwrite d3/d15/f41 [0,4194304] 0 2024-03-22T07:15:52.128 INFO:tasks.workunit.client.0.smithi063.stdout:1/211: rename d4/d14/f41 to d4/d14/f47 0 2024-03-22T07:15:52.128 INFO:tasks.workunit.client.0.smithi063.stdout:1/212: fdatasync d4/d14/f21 0 2024-03-22T07:15:52.131 INFO:tasks.workunit.client.0.smithi063.stdout:1/213: creat d4/df/f48 x:0 0 0 2024-03-22T07:15:52.131 INFO:tasks.workunit.client.0.smithi063.stdout:1/214: readlink d4/df/l19 0 2024-03-22T07:15:52.133 INFO:tasks.workunit.client.0.smithi063.stdout:6/213: rename d3/d15/d16/d1e to d3/d15/d29/d4c 0 2024-03-22T07:15:52.133 INFO:tasks.workunit.client.0.smithi063.stdout:6/214: chown d3/d15/d29/d4c/l3c 0 1 2024-03-22T07:15:52.135 INFO:tasks.workunit.client.0.smithi063.stdout:6/215: getdents d3/d6/d2a 0 2024-03-22T07:15:52.137 INFO:tasks.workunit.client.0.smithi063.stdout:6/216: unlink d3/d15/d16/f43 0 2024-03-22T07:15:52.137 INFO:tasks.workunit.client.0.smithi063.stdout:6/217: fsync d3/d1c/f1d 0 2024-03-22T07:15:52.137 INFO:tasks.workunit.client.0.smithi063.stdout:6/218: creat d3/d6/d2a/d38/f4d x:0 0 0 2024-03-22T07:15:52.143 INFO:tasks.workunit.client.0.smithi063.stdout:6/219: dread d3/d15/f23 [0,4194304] 0 2024-03-22T07:15:52.143 INFO:tasks.workunit.client.0.smithi063.stdout:0/195: dwrite da/de/d17/f2f [0,4194304] 0 2024-03-22T07:15:52.143 INFO:tasks.workunit.client.0.smithi063.stdout:5/162: dwrite d8/d13/d1e/f27 [0,4194304] 0 2024-03-22T07:15:52.144 INFO:tasks.workunit.client.0.smithi063.stdout:1/215: write d4/f1e [3800941,79960] 0 2024-03-22T07:15:52.144 INFO:tasks.workunit.client.0.smithi063.stdout:5/163: fdatasync d8/f2e 0 2024-03-22T07:15:52.144 INFO:tasks.workunit.client.0.smithi063.stdout:0/196: write da/f2b [893109,102134] 0 2024-03-22T07:15:52.144 INFO:tasks.workunit.client.0.smithi063.stdout:6/220: mkdir d3/d15/d29/d4e 0 2024-03-22T07:15:52.146 INFO:tasks.workunit.client.0.smithi063.stdout:1/216: truncate d4/f32 1758143 0 2024-03-22T07:15:52.146 INFO:tasks.workunit.client.0.smithi063.stdout:5/164: rename d8 to d8/d30 22 2024-03-22T07:15:52.147 INFO:tasks.workunit.client.0.smithi063.stdout:0/197: symlink da/de/d39/l42 0 2024-03-22T07:15:52.147 INFO:tasks.workunit.client.0.smithi063.stdout:0/198: read da/de/f23 [489813,19139] 0 2024-03-22T07:15:52.148 INFO:tasks.workunit.client.0.smithi063.stdout:6/221: mknod d3/d6/d35/c4f 0 2024-03-22T07:15:52.148 INFO:tasks.workunit.client.0.smithi063.stdout:1/217: creat d4/df/d13/d2b/f49 x:0 0 0 2024-03-22T07:15:52.149 INFO:tasks.workunit.client.0.smithi063.stdout:1/218: readlink d4/df/l3f 0 2024-03-22T07:15:52.149 INFO:tasks.workunit.client.0.smithi063.stdout:5/165: mkdir d8/d13/d1e/d31 0 2024-03-22T07:15:52.150 INFO:tasks.workunit.client.0.smithi063.stdout:0/199: mkdir da/de/d17/d25/d3c/d43 0 2024-03-22T07:15:52.152 INFO:tasks.workunit.client.0.smithi063.stdout:1/219: mknod d4/df/d13/d2b/c4a 0 2024-03-22T07:15:52.152 INFO:tasks.workunit.client.0.smithi063.stdout:1/220: chown d4/df/l19 0 1 2024-03-22T07:15:52.152 INFO:tasks.workunit.client.0.smithi063.stdout:4/186: dwrite d1/f11 [0,4194304] 0 2024-03-22T07:15:52.153 INFO:tasks.workunit.client.0.smithi063.stdout:7/170: dwrite d5/d16/f32 [0,4194304] 0 2024-03-22T07:15:52.153 INFO:tasks.workunit.client.0.smithi063.stdout:0/200: rmdir da/de 39 2024-03-22T07:15:52.153 INFO:tasks.workunit.client.0.smithi063.stdout:0/201: fdatasync da/de/d17/f28 0 2024-03-22T07:15:52.153 INFO:tasks.workunit.client.0.smithi063.stdout:0/202: readlink da/de/l2d 0 2024-03-22T07:15:52.154 INFO:tasks.workunit.client.0.smithi063.stdout:9/122: dwrite d8/fa [0,4194304] 0 2024-03-22T07:15:52.154 INFO:tasks.workunit.client.0.smithi063.stdout:8/214: dwrite d0/d26/f27 [0,4194304] 0 2024-03-22T07:15:52.154 INFO:tasks.workunit.client.0.smithi063.stdout:0/203: read da/f2b [359399,40739] 0 2024-03-22T07:15:52.154 INFO:tasks.workunit.client.0.smithi063.stdout:0/204: stat da/f2a 0 2024-03-22T07:15:52.155 INFO:tasks.workunit.client.0.smithi063.stdout:0/205: chown da/de/d17/f1a 5453333 1 2024-03-22T07:15:52.155 INFO:tasks.workunit.client.0.smithi063.stdout:3/137: dwrite da/d16/d26/f29 [0,4194304] 0 2024-03-22T07:15:52.155 INFO:tasks.workunit.client.0.smithi063.stdout:1/221: creat d4/df/d2c/f4b x:0 0 0 2024-03-22T07:15:52.157 INFO:tasks.workunit.client.0.smithi063.stdout:4/187: mkdir d1/d13/d27/d2c/d3d 0 2024-03-22T07:15:52.158 INFO:tasks.workunit.client.0.smithi063.stdout:5/166: symlink d8/l32 0 2024-03-22T07:15:52.165 INFO:tasks.workunit.client.0.smithi063.stdout:6/222: dwrite d3/d1c/f4b [0,4194304] 0 2024-03-22T07:15:52.166 INFO:tasks.workunit.client.0.smithi063.stdout:9/123: dread d8/fd [0,4194304] 0 2024-03-22T07:15:52.167 INFO:tasks.workunit.client.0.smithi063.stdout:6/223: dread d3/d15/d29/d4c/f20 [0,4194304] 0 2024-03-22T07:15:52.172 INFO:tasks.workunit.client.0.smithi063.stdout:0/206: mknod da/de/d17/d25/d3c/d43/c44 0 2024-03-22T07:15:52.173 INFO:tasks.workunit.client.0.smithi063.stdout:8/215: dwrite d0/d26/f27 [0,4194304] 0 2024-03-22T07:15:52.176 INFO:tasks.workunit.client.0.smithi063.stdout:3/138: rmdir da/d16 39 2024-03-22T07:15:52.176 INFO:tasks.workunit.client.0.smithi063.stdout:3/139: stat da/d16/d26/f2a 0 2024-03-22T07:15:52.176 INFO:tasks.workunit.client.0.smithi063.stdout:3/140: chown c2 0 1 2024-03-22T07:15:52.177 INFO:tasks.workunit.client.0.smithi063.stdout:3/141: write da/d16/d26/f27 [660221,23792] 0 2024-03-22T07:15:52.177 INFO:tasks.workunit.client.0.smithi063.stdout:3/142: chown da/f1c 697 1 2024-03-22T07:15:52.177 INFO:tasks.workunit.client.0.smithi063.stdout:3/143: write da/f11 [4223589,14287] 0 2024-03-22T07:15:52.177 INFO:tasks.workunit.client.0.smithi063.stdout:3/144: chown da/d16/d26 500 1 2024-03-22T07:15:52.181 INFO:tasks.workunit.client.0.smithi063.stdout:0/207: write da/de/f18 [1291670,100221] 0 2024-03-22T07:15:52.181 INFO:tasks.workunit.client.0.smithi063.stdout:0/208: creat da/de/d17/f45 x:0 0 0 2024-03-22T07:15:52.181 INFO:tasks.workunit.client.0.smithi063.stdout:0/209: chown da/de/d39/c3b 14956838 1 2024-03-22T07:15:52.183 INFO:tasks.workunit.client.0.smithi063.stdout:1/222: mknod d4/c4c 0 2024-03-22T07:15:52.186 INFO:tasks.workunit.client.0.smithi063.stdout:5/167: creat d8/d13/d24/f33 x:0 0 0 2024-03-22T07:15:52.186 INFO:tasks.workunit.client.0.smithi063.stdout:7/171: rmdir d5/d16/d1b/d21 39 2024-03-22T07:15:52.190 INFO:tasks.workunit.client.0.smithi063.stdout:9/124: mkdir d8/d19/d20/d22/d25 0 2024-03-22T07:15:52.193 INFO:tasks.workunit.client.0.smithi063.stdout:6/224: creat d3/d15/d29/d4e/f50 x:0 0 0 2024-03-22T07:15:52.193 INFO:tasks.workunit.client.0.smithi063.stdout:8/216: chown d0/l15 5471930 1 2024-03-22T07:15:52.196 INFO:tasks.workunit.client.0.smithi063.stdout:3/145: creat da/d16/f2d x:0 0 0 2024-03-22T07:15:52.199 INFO:tasks.workunit.client.0.smithi063.stdout:0/210: mknod da/de/d17/d25/d36/c46 0 2024-03-22T07:15:52.199 INFO:tasks.workunit.client.0.smithi063.stdout:1/223: mknod d4/d14/c4d 0 2024-03-22T07:15:52.203 INFO:tasks.workunit.client.0.smithi063.stdout:9/125: rmdir d8/d19/d20/d22/d25 0 2024-03-22T07:15:52.204 INFO:tasks.workunit.client.0.smithi063.stdout:6/225: mkdir d3/d1c/d32/d51 0 2024-03-22T07:15:52.205 INFO:tasks.workunit.client.0.smithi063.stdout:6/226: chown d3/d15 1184054 1 2024-03-22T07:15:52.205 INFO:tasks.workunit.client.0.smithi063.stdout:3/146: rename c2 to da/d16/c2e 0 2024-03-22T07:15:52.206 INFO:tasks.workunit.client.0.smithi063.stdout:6/227: creat d3/d6/d35/f52 x:0 0 0 2024-03-22T07:15:52.207 INFO:tasks.workunit.client.0.smithi063.stdout:6/228: dread - d3/d6/d2a/d38/f49 zero size 2024-03-22T07:15:52.208 INFO:tasks.workunit.client.0.smithi063.stdout:8/217: dread d0/f1a [0,4194304] 0 2024-03-22T07:15:52.208 INFO:tasks.workunit.client.0.smithi063.stdout:8/218: creat d0/d1b/d28/f43 x:0 0 0 2024-03-22T07:15:52.208 INFO:tasks.workunit.client.0.smithi063.stdout:8/219: chown d0/f19 6048 1 2024-03-22T07:15:52.208 INFO:tasks.workunit.client.0.smithi063.stdout:8/220: read d0/d1f/f40 [2854265,93200] 0 2024-03-22T07:15:52.208 INFO:tasks.workunit.client.0.smithi063.stdout:8/221: fsync d0/d26/f34 0 2024-03-22T07:15:52.208 INFO:tasks.workunit.client.0.smithi063.stdout:8/222: chown d0/d1b/c37 89303 1 2024-03-22T07:15:52.208 INFO:tasks.workunit.client.0.smithi063.stdout:8/223: write d0/d1b/d28/f43 [207241,121176] 0 2024-03-22T07:15:52.208 INFO:tasks.workunit.client.0.smithi063.stdout:0/211: dread da/fc [0,4194304] 0 2024-03-22T07:15:52.209 INFO:tasks.workunit.client.0.smithi063.stdout:9/126: rename d8/fb to d8/d17/f26 0 2024-03-22T07:15:52.209 INFO:tasks.workunit.client.0.smithi063.stdout:7/172: dwrite d5/d14/f37 [0,4194304] 0 2024-03-22T07:15:52.209 INFO:tasks.workunit.client.0.smithi063.stdout:7/173: write d5/da/f25 [324345,33462] 0 2024-03-22T07:15:52.210 INFO:tasks.workunit.client.0.smithi063.stdout:5/168: dwrite d8/d13/d24/f33 [0,4194304] 0 2024-03-22T07:15:52.210 INFO:tasks.workunit.client.0.smithi063.stdout:3/147: creat da/f2f x:0 0 0 2024-03-22T07:15:52.212 INFO:tasks.workunit.client.0.smithi063.stdout:6/229: symlink d3/d15/d29/d4c/l53 0 2024-03-22T07:15:52.213 INFO:tasks.workunit.client.0.smithi063.stdout:3/148: dread da/f1c [0,4194304] 0 2024-03-22T07:15:52.221 INFO:tasks.workunit.client.0.smithi063.stdout:0/212: mknod da/d2e/c47 0 2024-03-22T07:15:52.221 INFO:tasks.workunit.client.0.smithi063.stdout:9/127: truncate d8/fd 2180637 0 2024-03-22T07:15:52.221 INFO:tasks.workunit.client.0.smithi063.stdout:9/128: truncate d8/f1a 809127 0 2024-03-22T07:15:52.223 INFO:tasks.workunit.client.0.smithi063.stdout:1/224: truncate d4/f1e 2349346 0 2024-03-22T07:15:52.223 INFO:tasks.workunit.client.0.smithi063.stdout:1/225: creat d4/df/f4e x:0 0 0 2024-03-22T07:15:52.223 INFO:tasks.workunit.client.0.smithi063.stdout:1/226: dread - d4/df/f4e zero size 2024-03-22T07:15:52.224 INFO:tasks.workunit.client.0.smithi063.stdout:5/169: mknod d8/d13/d24/c34 0 2024-03-22T07:15:52.225 INFO:tasks.workunit.client.0.smithi063.stdout:4/188: truncate d1/d5/f12 3852996 0 2024-03-22T07:15:52.226 INFO:tasks.workunit.client.0.smithi063.stdout:0/213: symlink da/de/d17/d25/l48 0 2024-03-22T07:15:52.228 INFO:tasks.workunit.client.0.smithi063.stdout:1/227: symlink d4/df/d13/l4f 0 2024-03-22T07:15:52.229 INFO:tasks.workunit.client.0.smithi063.stdout:5/170: truncate d8/f25 183469 0 2024-03-22T07:15:52.233 INFO:tasks.workunit.client.0.smithi063.stdout:3/149: symlink da/l30 0 2024-03-22T07:15:52.233 INFO:tasks.workunit.client.0.smithi063.stdout:3/150: write da/f11 [1933791,125195] 0 2024-03-22T07:15:52.233 INFO:tasks.workunit.client.0.smithi063.stdout:3/151: chown da/f1c 0 1 2024-03-22T07:15:52.233 INFO:tasks.workunit.client.0.smithi063.stdout:3/152: fsync da/f1c 0 2024-03-22T07:15:52.233 INFO:tasks.workunit.client.0.smithi063.stdout:3/153: fsync da/d16/f25 0 2024-03-22T07:15:52.234 INFO:tasks.workunit.client.0.smithi063.stdout:0/214: mkdir da/d49 0 2024-03-22T07:15:52.238 INFO:tasks.workunit.client.0.smithi063.stdout:5/171: mkdir d8/d13/d35 0 2024-03-22T07:15:52.238 INFO:tasks.workunit.client.0.smithi063.stdout:0/215: mknod da/d2e/c4a 0 2024-03-22T07:15:52.239 INFO:tasks.workunit.client.0.smithi063.stdout:5/172: symlink d8/d13/l36 0 2024-03-22T07:15:52.239 INFO:tasks.workunit.client.0.smithi063.stdout:0/216: rmdir da/de/d17/d25/d3c 39 2024-03-22T07:15:52.241 INFO:tasks.workunit.client.0.smithi063.stdout:0/217: creat da/de/d17/d25/d36/f4b x:0 0 0 2024-03-22T07:15:52.241 INFO:tasks.workunit.client.0.smithi063.stdout:0/218: write da/d2e/f33 [1001273,44645] 0 2024-03-22T07:15:52.242 INFO:tasks.workunit.client.0.smithi063.stdout:0/219: mknod da/de/d17/d25/c4c 0 2024-03-22T07:15:52.245 INFO:tasks.workunit.client.0.smithi063.stdout:3/154: write da/d16/d26/f29 [1562516,13249] 0 2024-03-22T07:15:52.245 INFO:tasks.workunit.client.0.smithi063.stdout:3/155: write da/f18 [4401687,59408] 0 2024-03-22T07:15:52.247 INFO:tasks.workunit.client.0.smithi063.stdout:8/224: dwrite d0/d24/f35 [0,4194304] 0 2024-03-22T07:15:52.248 INFO:tasks.workunit.client.0.smithi063.stdout:7/174: dwrite d5/da/ff [0,4194304] 0 2024-03-22T07:15:52.248 INFO:tasks.workunit.client.0.smithi063.stdout:7/175: fdatasync d5/da/f11 0 2024-03-22T07:15:52.249 INFO:tasks.workunit.client.0.smithi063.stdout:1/228: dwrite d4/df/f29 [0,4194304] 0 2024-03-22T07:15:52.250 INFO:tasks.workunit.client.0.smithi063.stdout:1/229: write d4/d14/f23 [1966240,29435] 0 2024-03-22T07:15:52.250 INFO:tasks.workunit.client.0.smithi063.stdout:1/230: creat d4/d14/f50 x:0 0 0 2024-03-22T07:15:52.250 INFO:tasks.workunit.client.0.smithi063.stdout:1/231: write d4/d14/f47 [5219929,43305] 0 2024-03-22T07:15:52.250 INFO:tasks.workunit.client.0.smithi063.stdout:6/230: dwrite d3/f18 [0,4194304] 0 2024-03-22T07:15:52.252 INFO:tasks.workunit.client.0.smithi063.stdout:7/176: symlink d5/d16/l3a 0 2024-03-22T07:15:52.253 INFO:tasks.workunit.client.0.smithi063.stdout:9/129: write d8/fd [874914,57921] 0 2024-03-22T07:15:52.256 INFO:tasks.workunit.client.0.smithi063.stdout:4/189: rmdir d1 39 2024-03-22T07:15:52.261 INFO:tasks.workunit.client.0.smithi063.stdout:1/232: symlink d4/l51 0 2024-03-22T07:15:52.262 INFO:tasks.workunit.client.0.smithi063.stdout:5/173: write d8/f25 [467544,452] 0 2024-03-22T07:15:52.272 INFO:tasks.workunit.client.0.smithi063.stdout:9/130: creat d8/d19/d20/d22/f27 x:0 0 0 2024-03-22T07:15:52.272 INFO:tasks.workunit.client.0.smithi063.stdout:9/131: truncate f3 1996296 0 2024-03-22T07:15:52.272 INFO:tasks.workunit.client.0.smithi063.stdout:7/177: write d5/da/f2c [1317376,80719] 0 2024-03-22T07:15:52.273 INFO:tasks.workunit.client.0.smithi063.stdout:1/233: rename d4/df/d13/d2b/d3a/f3d to d4/df/d13/d2b/f52 0 2024-03-22T07:15:52.273 INFO:tasks.workunit.client.0.smithi063.stdout:1/234: write d4/df/d13/d2b/f49 [416125,39042] 0 2024-03-22T07:15:52.274 INFO:tasks.workunit.client.0.smithi063.stdout:3/156: dwrite da/d16/d26/f2a [0,4194304] 0 2024-03-22T07:15:52.274 INFO:tasks.workunit.client.0.smithi063.stdout:8/225: dwrite d0/d24/f42 [0,4194304] 0 2024-03-22T07:15:52.274 INFO:tasks.workunit.client.0.smithi063.stdout:5/174: symlink d8/d13/d28/l37 0 2024-03-22T07:15:52.276 INFO:tasks.workunit.client.0.smithi063.stdout:1/235: symlink d4/df/d13/d2b/d3a/l53 0 2024-03-22T07:15:52.276 INFO:tasks.workunit.client.0.smithi063.stdout:1/236: chown d4/lc 15989247 1 2024-03-22T07:15:52.276 INFO:tasks.workunit.client.0.smithi063.stdout:1/237: dread - d4/df/f44 zero size 2024-03-22T07:15:52.276 INFO:tasks.workunit.client.0.smithi063.stdout:1/238: chown d4/d14/f21 4304183 1 2024-03-22T07:15:52.276 INFO:tasks.workunit.client.0.smithi063.stdout:1/239: chown d4/df/d13/l2d 11 1 2024-03-22T07:15:52.276 INFO:tasks.workunit.client.0.smithi063.stdout:1/240: creat d4/df/d2c/f54 x:0 0 0 2024-03-22T07:15:52.277 INFO:tasks.workunit.client.0.smithi063.stdout:9/132: write d8/fc [3546868,32182] 0 2024-03-22T07:15:52.277 INFO:tasks.workunit.client.0.smithi063.stdout:9/133: stat d8/d15/f1c 0 2024-03-22T07:15:52.277 INFO:tasks.workunit.client.0.smithi063.stdout:9/134: dread - d8/d19/f23 zero size 2024-03-22T07:15:52.277 INFO:tasks.workunit.client.0.smithi063.stdout:9/135: readlink d8/le 0 2024-03-22T07:15:52.279 INFO:tasks.workunit.client.0.smithi063.stdout:3/157: link da/d16/d26/l2c da/d16/l31 0 2024-03-22T07:15:52.279 INFO:tasks.workunit.client.0.smithi063.stdout:3/158: creat da/d16/f32 x:0 0 0 2024-03-22T07:15:52.279 INFO:tasks.workunit.client.0.smithi063.stdout:3/159: dread - da/f2f zero size 2024-03-22T07:15:52.280 INFO:tasks.workunit.client.0.smithi063.stdout:7/178: dread d5/da/ff [0,4194304] 0 2024-03-22T07:15:52.285 INFO:tasks.workunit.client.0.smithi063.stdout:5/175: write d8/f1c [2976819,23972] 0 2024-03-22T07:15:52.285 INFO:tasks.workunit.client.0.smithi063.stdout:1/241: unlink d4/c4c 0 2024-03-22T07:15:52.289 INFO:tasks.workunit.client.0.smithi063.stdout:9/136: dread d8/fa [0,4194304] 0 2024-03-22T07:15:52.289 INFO:tasks.workunit.client.0.smithi063.stdout:9/137: fdatasync d8/d19/d20/d22/f27 0 2024-03-22T07:15:52.289 INFO:tasks.workunit.client.0.smithi063.stdout:3/160: symlink da/d16/l33 0 2024-03-22T07:15:52.291 INFO:tasks.workunit.client.0.smithi063.stdout:7/179: mkdir d5/d14/d1d/d29/d3b 0 2024-03-22T07:15:52.291 INFO:tasks.workunit.client.0.smithi063.stdout:7/180: truncate d5/da/f35 372608 0 2024-03-22T07:15:52.291 INFO:tasks.workunit.client.0.smithi063.stdout:7/181: rename d5/d16/d1b/d21 to d5/d16/d1b/d21/d3c 22 2024-03-22T07:15:52.291 INFO:tasks.workunit.client.0.smithi063.stdout:7/182: read d5/f7 [2329467,39206] 0 2024-03-22T07:15:52.295 INFO:tasks.workunit.client.0.smithi063.stdout:8/226: rmdir d0 39 2024-03-22T07:15:52.296 INFO:tasks.workunit.client.0.smithi063.stdout:1/242: write d4/df/f29 [3118144,71310] 0 2024-03-22T07:15:52.296 INFO:tasks.workunit.client.0.smithi063.stdout:9/138: dread d8/d19/d20/f21 [0,4194304] 0 2024-03-22T07:15:52.297 INFO:tasks.workunit.client.0.smithi063.stdout:5/176: rmdir d8/d13/d35 0 2024-03-22T07:15:52.299 INFO:tasks.workunit.client.0.smithi063.stdout:2/209: sync 2024-03-22T07:15:52.303 INFO:tasks.workunit.client.0.smithi063.stdout:5/177: dread d8/f14 [0,4194304] 0 2024-03-22T07:15:52.303 INFO:tasks.workunit.client.0.smithi063.stdout:3/161: chown da/l15 3928273 1 2024-03-22T07:15:52.305 INFO:tasks.workunit.client.0.smithi063.stdout:8/227: creat d0/d26/d2d/d3d/f44 x:0 0 0 2024-03-22T07:15:52.305 INFO:tasks.workunit.client.0.smithi063.stdout:8/228: write d0/d24/f29 [4798850,122152] 0 2024-03-22T07:15:52.306 INFO:tasks.workunit.client.0.smithi063.stdout:1/243: mknod d4/df/d13/d2b/c55 0 2024-03-22T07:15:52.307 INFO:tasks.workunit.client.0.smithi063.stdout:1/244: write d4/d14/f46 [607141,74278] 0 2024-03-22T07:15:52.308 INFO:tasks.workunit.client.0.smithi063.stdout:9/139: link d8/le d8/l28 0 2024-03-22T07:15:52.322 INFO:tasks.workunit.client.0.smithi063.stdout:5/178: dwrite d8/f12 [0,4194304] 0 2024-03-22T07:15:52.322 INFO:tasks.workunit.client.0.smithi063.stdout:8/229: dwrite d0/d1b/d28/f43 [0,4194304] 0 2024-03-22T07:15:52.322 INFO:tasks.workunit.client.0.smithi063.stdout:8/230: chown d0/d24/c3c 8273103 1 2024-03-22T07:15:52.326 INFO:tasks.workunit.client.0.smithi063.stdout:5/179: write d8/f10 [1969816,128126] 0 2024-03-22T07:15:52.326 INFO:tasks.workunit.client.0.smithi063.stdout:5/180: chown d8/d13/d1e 448498 1 2024-03-22T07:15:52.327 INFO:tasks.workunit.client.0.smithi063.stdout:5/181: creat d8/d13/d1e/f38 x:0 0 0 2024-03-22T07:15:52.333 INFO:tasks.workunit.client.0.smithi063.stdout:5/182: write d8/d13/d24/f33 [557176,23918] 0 2024-03-22T07:15:52.377 INFO:tasks.workunit.client.0.smithi063.stdout:3/162: creat da/d16/d26/f34 x:0 0 0 2024-03-22T07:15:52.378 INFO:tasks.workunit.client.0.smithi063.stdout:1/245: symlink d4/df/d13/d39/l56 0 2024-03-22T07:15:52.378 INFO:tasks.workunit.client.0.smithi063.stdout:1/246: chown d4/d14/c31 364 1 2024-03-22T07:15:52.379 INFO:tasks.workunit.client.0.smithi063.stdout:9/140: mkdir d8/d19/d20/d22/d29 0 2024-03-22T07:15:52.379 INFO:tasks.workunit.client.0.smithi063.stdout:9/141: chown d8/l28 168434 1 2024-03-22T07:15:52.379 INFO:tasks.workunit.client.0.smithi063.stdout:9/142: creat d8/d15/f2a x:0 0 0 2024-03-22T07:15:52.380 INFO:tasks.workunit.client.0.smithi063.stdout:2/210: getdents d0/de/d1b/d32 0 2024-03-22T07:15:52.383 INFO:tasks.workunit.client.0.smithi063.stdout:8/231: readlink d0/l3 0 2024-03-22T07:15:52.392 INFO:tasks.workunit.client.0.smithi063.stdout:3/163: creat da/f35 x:0 0 0 2024-03-22T07:15:52.393 INFO:tasks.workunit.client.0.smithi063.stdout:1/247: mkdir d4/d14/d57 0 2024-03-22T07:15:52.395 INFO:tasks.workunit.client.0.smithi063.stdout:9/143: mkdir d8/d19/d2b 0 2024-03-22T07:15:52.395 INFO:tasks.workunit.client.0.smithi063.stdout:9/144: fsync d8/f1a 0 2024-03-22T07:15:52.395 INFO:tasks.workunit.client.0.smithi063.stdout:8/232: mkdir d0/d18/d45 0 2024-03-22T07:15:52.397 INFO:tasks.workunit.client.0.smithi063.stdout:3/164: truncate da/d16/d26/f2a 1931382 0 2024-03-22T07:15:52.398 INFO:tasks.workunit.client.0.smithi063.stdout:8/233: truncate d0/d1b/f20 1787137 0 2024-03-22T07:15:52.398 INFO:tasks.workunit.client.0.smithi063.stdout:8/234: stat d0/d1b/d28/c2f 0 2024-03-22T07:15:52.400 INFO:tasks.workunit.client.0.smithi063.stdout:1/248: dread f2 [0,4194304] 0 2024-03-22T07:15:52.400 INFO:tasks.workunit.client.0.smithi063.stdout:2/211: dwrite d0/f23 [0,4194304] 0 2024-03-22T07:15:52.400 INFO:tasks.workunit.client.0.smithi063.stdout:9/145: write d8/f14 [348762,103172] 0 2024-03-22T07:15:52.404 INFO:tasks.workunit.client.0.smithi063.stdout:1/249: creat d4/d14/d57/f58 x:0 0 0 2024-03-22T07:15:52.404 INFO:tasks.workunit.client.0.smithi063.stdout:1/250: creat d4/f59 x:0 0 0 2024-03-22T07:15:52.404 INFO:tasks.workunit.client.0.smithi063.stdout:1/251: fdatasync d4/df/f48 0 2024-03-22T07:15:52.405 INFO:tasks.workunit.client.0.smithi063.stdout:2/212: stat d0/c3 0 2024-03-22T07:15:52.407 INFO:tasks.workunit.client.0.smithi063.stdout:9/146: mknod d8/d19/d2b/c2c 0 2024-03-22T07:15:52.409 INFO:tasks.workunit.client.0.smithi063.stdout:9/147: dread d8/f10 [0,4194304] 0 2024-03-22T07:15:52.412 INFO:tasks.workunit.client.0.smithi063.stdout:2/213: rename d0/de/d12/d20/f28 to d0/de/d35/f42 0 2024-03-22T07:15:52.417 INFO:tasks.workunit.client.0.smithi063.stdout:2/214: write d0/de/d12/d20/f29 [4057856,59029] 0 2024-03-22T07:15:52.421 INFO:tasks.workunit.client.0.smithi063.stdout:3/165: dwrite da/f11 [0,4194304] 0 2024-03-22T07:15:52.421 INFO:tasks.workunit.client.0.smithi063.stdout:3/166: creat da/d16/d26/f36 x:0 0 0 2024-03-22T07:15:52.423 INFO:tasks.workunit.client.0.smithi063.stdout:3/167: mknod da/d16/d26/c37 0 2024-03-22T07:15:52.423 INFO:tasks.workunit.client.0.smithi063.stdout:3/168: write da/d16/d26/f29 [2193899,127869] 0 2024-03-22T07:15:52.438 INFO:tasks.workunit.client.0.smithi063.stdout:8/235: dwrite d0/d24/f35 [0,4194304] 0 2024-03-22T07:15:52.438 INFO:tasks.workunit.client.0.smithi063.stdout:8/236: stat d0/d24 0 2024-03-22T07:15:52.441 INFO:tasks.workunit.client.0.smithi063.stdout:9/148: dwrite f2 [0,4194304] 0 2024-03-22T07:15:52.441 INFO:tasks.workunit.client.0.smithi063.stdout:1/252: dwrite d4/df/d13/f34 [0,4194304] 0 2024-03-22T07:15:52.441 INFO:tasks.workunit.client.0.smithi063.stdout:1/253: chown d4/l35 137883002 1 2024-03-22T07:15:52.448 INFO:tasks.workunit.client.0.smithi063.stdout:1/254: mkdir d4/d5a 0 2024-03-22T07:15:52.448 INFO:tasks.workunit.client.0.smithi063.stdout:1/255: fdatasync d4/df/f1a 0 2024-03-22T07:15:52.450 INFO:tasks.workunit.client.0.smithi063.stdout:1/256: mknod d4/d14/d57/c5b 0 2024-03-22T07:15:52.450 INFO:tasks.workunit.client.0.smithi063.stdout:1/257: creat d4/df/d13/d2b/f5c x:0 0 0 2024-03-22T07:15:52.450 INFO:tasks.workunit.client.0.smithi063.stdout:1/258: creat d4/d14/d57/f5d x:0 0 0 2024-03-22T07:15:52.450 INFO:tasks.workunit.client.0.smithi063.stdout:1/259: chown d4/f1f 188103 1 2024-03-22T07:15:52.450 INFO:tasks.workunit.client.0.smithi063.stdout:1/260: fsync d4/df/d13/f3c 0 2024-03-22T07:15:52.452 INFO:tasks.workunit.client.0.smithi063.stdout:1/261: unlink d4/df/d13/d2b/f52 0 2024-03-22T07:15:52.453 INFO:tasks.workunit.client.0.smithi063.stdout:3/169: truncate da/d16/d26/f29 4039356 0 2024-03-22T07:15:52.453 INFO:tasks.workunit.client.0.smithi063.stdout:3/170: dread - da/d16/d26/f36 zero size 2024-03-22T07:15:52.456 INFO:tasks.workunit.client.0.smithi063.stdout:8/237: dwrite d0/d1b/f20 [0,4194304] 0 2024-03-22T07:15:52.466 INFO:tasks.workunit.client.0.smithi063.stdout:8/238: dread d0/d1b/f20 [0,4194304] 0 2024-03-22T07:15:52.466 INFO:tasks.workunit.client.0.smithi063.stdout:9/149: dwrite d8/f1a [0,4194304] 0 2024-03-22T07:15:52.466 INFO:tasks.workunit.client.0.smithi063.stdout:9/150: getdents d8/d19/d20/d22/d29 0 2024-03-22T07:15:52.466 INFO:tasks.workunit.client.0.smithi063.stdout:9/151: fdatasync d8/d19/f23 0 2024-03-22T07:15:52.466 INFO:tasks.workunit.client.0.smithi063.stdout:9/152: creat d8/d19/d20/d22/f2d x:0 0 0 2024-03-22T07:15:52.467 INFO:tasks.workunit.client.0.smithi063.stdout:9/153: read d8/fd [305686,15095] 0 2024-03-22T07:15:52.467 INFO:tasks.workunit.client.0.smithi063.stdout:8/239: creat d0/d18/d45/f46 x:0 0 0 2024-03-22T07:15:52.467 INFO:tasks.workunit.client.0.smithi063.stdout:8/240: write d0/d1b/d28/f3b [4560930,64341] 0 2024-03-22T07:15:52.472 INFO:tasks.workunit.client.0.smithi063.stdout:0/220: sync 2024-03-22T07:15:52.486 INFO:tasks.workunit.client.0.smithi063.stdout:0/221: dwrite da/f40 [0,4194304] 0 2024-03-22T07:15:52.488 INFO:tasks.workunit.client.0.smithi063.stdout:0/222: rmdir da/d37 0 2024-03-22T07:15:52.488 INFO:tasks.workunit.client.0.smithi063.stdout:0/223: dread - da/de/d17/f28 zero size 2024-03-22T07:15:52.488 INFO:tasks.workunit.client.0.smithi063.stdout:0/224: dread - da/de/d17/f21 zero size 2024-03-22T07:15:52.489 INFO:tasks.workunit.client.0.smithi063.stdout:0/225: truncate da/f2b 568674 0 2024-03-22T07:15:52.494 INFO:tasks.workunit.client.0.smithi063.stdout:9/154: rmdir d8 39 2024-03-22T07:15:52.495 INFO:tasks.workunit.client.0.smithi063.stdout:8/241: write d0/d24/f42 [4084490,97789] 0 2024-03-22T07:15:52.498 INFO:tasks.workunit.client.0.smithi063.stdout:9/155: write d8/d19/d20/f21 [606161,20619] 0 2024-03-22T07:15:52.498 INFO:tasks.workunit.client.0.smithi063.stdout:9/156: stat d8/f1a 0 2024-03-22T07:15:52.499 INFO:tasks.workunit.client.0.smithi063.stdout:9/157: read - d8/f13 zero size 2024-03-22T07:15:52.499 INFO:tasks.workunit.client.0.smithi063.stdout:9/158: fdatasync d8/d17/f1d 0 2024-03-22T07:15:52.499 INFO:tasks.workunit.client.0.smithi063.stdout:9/159: chown d8/f13 348 1 2024-03-22T07:15:52.499 INFO:tasks.workunit.client.0.smithi063.stdout:9/160: chown d8/d17/d1e/l1f 5 1 2024-03-22T07:15:52.499 INFO:tasks.workunit.client.0.smithi063.stdout:9/161: write f7 [578262,39617] 0 2024-03-22T07:15:52.500 INFO:tasks.workunit.client.0.smithi063.stdout:9/162: symlink d8/d17/d1e/l2e 0 2024-03-22T07:15:52.500 INFO:tasks.workunit.client.0.smithi063.stdout:9/163: stat d8/d19 0 2024-03-22T07:15:52.500 INFO:tasks.workunit.client.0.smithi063.stdout:9/164: truncate d8/d19/d20/d22/f2d 1016083 0 2024-03-22T07:15:52.500 INFO:tasks.workunit.client.0.smithi063.stdout:9/165: stat d8/d19/d20/d22/d24 0 2024-03-22T07:15:52.501 INFO:tasks.workunit.client.0.smithi063.stdout:0/226: dread da/f2b [0,4194304] 0 2024-03-22T07:15:52.501 INFO:tasks.workunit.client.0.smithi063.stdout:9/166: read f7 [537583,47812] 0 2024-03-22T07:15:52.501 INFO:tasks.workunit.client.0.smithi063.stdout:0/227: mknod da/de/d17/d25/d3c/d43/c4d 0 2024-03-22T07:15:52.507 INFO:tasks.workunit.client.0.smithi063.stdout:0/228: dread da/de/f31 [4194304,4194304] 0 2024-03-22T07:15:52.507 INFO:tasks.workunit.client.0.smithi063.stdout:0/229: fsync da/de/d17/f21 0 2024-03-22T07:15:52.511 INFO:tasks.workunit.client.0.smithi063.stdout:9/167: dread d8/fa [0,4194304] 0 2024-03-22T07:15:52.512 INFO:tasks.workunit.client.0.smithi063.stdout:9/168: readlink d8/le 0 2024-03-22T07:15:52.524 INFO:tasks.workunit.client.0.smithi063.stdout:8/242: dwrite d0/d1b/d28/f3b [4194304,4194304] 0 2024-03-22T07:15:52.524 INFO:tasks.workunit.client.0.smithi063.stdout:8/243: chown d0/d1b/d28/c41 478824 1 2024-03-22T07:15:52.528 INFO:tasks.workunit.client.0.smithi063.stdout:0/230: dwrite da/de/f24 [4194304,4194304] 0 2024-03-22T07:15:52.533 INFO:tasks.workunit.client.0.smithi063.stdout:9/169: dwrite d8/d17/f1d [0,4194304] 0 2024-03-22T07:15:52.553 INFO:tasks.workunit.client.0.smithi063.stdout:8/244: dwrite d0/d1b/d28/f39 [0,4194304] 0 2024-03-22T07:15:52.553 INFO:tasks.workunit.client.0.smithi063.stdout:0/231: dwrite da/de/f23 [4194304,4194304] 0 2024-03-22T07:15:52.553 INFO:tasks.workunit.client.0.smithi063.stdout:0/232: dread - da/de/d39/f41 zero size 2024-03-22T07:15:52.554 INFO:tasks.workunit.client.0.smithi063.stdout:9/170: dwrite d8/d19/f23 [0,4194304] 0 2024-03-22T07:15:52.554 INFO:tasks.workunit.client.0.smithi063.stdout:9/171: fdatasync d8/f10 0 2024-03-22T07:15:52.554 INFO:tasks.workunit.client.0.smithi063.stdout:9/172: readlink d8/le 0 2024-03-22T07:15:52.554 INFO:tasks.workunit.client.0.smithi063.stdout:9/173: truncate d8/f14 4759240 0 2024-03-22T07:15:52.554 INFO:tasks.workunit.client.0.smithi063.stdout:9/174: dread - d8/d19/d20/d22/f27 zero size 2024-03-22T07:15:52.557 INFO:tasks.workunit.client.0.smithi063.stdout:0/233: mkdir da/de/d17/d4e 0 2024-03-22T07:15:52.558 INFO:tasks.workunit.client.0.smithi063.stdout:0/234: dread - da/de/d17/f28 zero size 2024-03-22T07:15:52.560 INFO:tasks.workunit.client.0.smithi063.stdout:9/175: mkdir d8/d15/d2f 0 2024-03-22T07:15:52.560 INFO:tasks.workunit.client.0.smithi063.stdout:9/176: chown l5 30 1 2024-03-22T07:15:52.560 INFO:tasks.workunit.client.0.smithi063.stdout:6/231: sync 2024-03-22T07:15:52.561 INFO:tasks.workunit.client.0.smithi063.stdout:8/245: rename d0/l16 to d0/l47 0 2024-03-22T07:15:52.562 INFO:tasks.workunit.client.0.smithi063.stdout:0/235: mknod da/de/d17/d25/c4f 0 2024-03-22T07:15:52.563 INFO:tasks.workunit.client.0.smithi063.stdout:6/232: getdents d3 0 2024-03-22T07:15:52.563 INFO:tasks.workunit.client.0.smithi063.stdout:8/246: mknod d0/d1b/d28/d3e/c48 0 2024-03-22T07:15:52.564 INFO:tasks.workunit.client.0.smithi063.stdout:6/233: mkdir d3/d54 0 2024-03-22T07:15:52.568 INFO:tasks.workunit.client.0.smithi063.stdout:9/177: write d8/d17/f1d [2481807,18363] 0 2024-03-22T07:15:52.569 INFO:tasks.workunit.client.0.smithi063.stdout:0/236: dread da/d2e/f33 [0,4194304] 0 2024-03-22T07:15:52.569 INFO:tasks.workunit.client.0.smithi063.stdout:0/237: dread - da/de/d17/d25/d36/f4b zero size 2024-03-22T07:15:52.570 INFO:tasks.workunit.client.0.smithi063.stdout:6/234: rename d3/d1c/f1d to d3/d6/d35/d3a/f55 0 2024-03-22T07:15:52.571 INFO:tasks.workunit.client.0.smithi063.stdout:6/235: dread - d3/d6/d2a/f48 zero size 2024-03-22T07:15:52.571 INFO:tasks.workunit.client.0.smithi063.stdout:9/178: mknod d8/d19/d2b/c30 0 2024-03-22T07:15:52.572 INFO:tasks.workunit.client.0.smithi063.stdout:8/247: dread d0/d24/f35 [0,4194304] 0 2024-03-22T07:15:52.573 INFO:tasks.workunit.client.0.smithi063.stdout:6/236: mknod d3/d6/d2a/c56 0 2024-03-22T07:15:52.574 INFO:tasks.workunit.client.0.smithi063.stdout:6/237: read d3/d15/f17 [5945,89509] 0 2024-03-22T07:15:52.574 INFO:tasks.workunit.client.0.smithi063.stdout:6/238: stat d3/d1c/c2d 0 2024-03-22T07:15:52.574 INFO:tasks.workunit.client.0.smithi063.stdout:6/239: chown d3/d15/d29/d4c 228190550 1 2024-03-22T07:15:52.575 INFO:tasks.workunit.client.0.smithi063.stdout:8/248: link d0/d1b/c37 d0/d24/d33/c49 0 2024-03-22T07:15:52.577 INFO:tasks.workunit.client.0.smithi063.stdout:6/240: chown d3/f4 5 1 2024-03-22T07:15:52.577 INFO:tasks.workunit.client.0.smithi063.stdout:6/241: write d3/d6/f28 [4743238,57555] 0 2024-03-22T07:15:52.577 INFO:tasks.workunit.client.0.smithi063.stdout:6/242: readlink d3/d6/l8 0 2024-03-22T07:15:52.577 INFO:tasks.workunit.client.0.smithi063.stdout:8/249: truncate d0/d1f/f40 1309070 0 2024-03-22T07:15:52.579 INFO:tasks.workunit.client.0.smithi063.stdout:6/243: unlink d3/l42 0 2024-03-22T07:15:52.579 INFO:tasks.workunit.client.0.smithi063.stdout:8/250: dread d0/fc [0,4194304] 0 2024-03-22T07:15:52.582 INFO:tasks.workunit.client.0.smithi063.stdout:8/251: unlink d0/d1b/d28/f39 0 2024-03-22T07:15:52.582 INFO:tasks.workunit.client.0.smithi063.stdout:8/252: read - d0/d26/d2d/d3d/f44 zero size 2024-03-22T07:15:52.583 INFO:tasks.workunit.client.0.smithi063.stdout:6/244: write d3/f4 [2888445,46960] 0 2024-03-22T07:15:52.584 INFO:tasks.workunit.client.0.smithi063.stdout:6/245: mkdir d3/d15/d16/d57 0 2024-03-22T07:15:52.598 INFO:tasks.workunit.client.0.smithi063.stdout:8/253: dread d0/d1b/f1e [4194304,4194304] 0 2024-03-22T07:15:52.606 INFO:tasks.workunit.client.0.smithi063.stdout:0/238: dwrite da/de/d17/d25/f3e [0,4194304] 0 2024-03-22T07:15:52.607 INFO:tasks.workunit.client.0.smithi063.stdout:6/246: dwrite d3/d15/d29/d4e/f50 [0,4194304] 0 2024-03-22T07:15:52.607 INFO:tasks.workunit.client.0.smithi063.stdout:6/247: write d3/d6/d35/f52 [389272,53530] 0 2024-03-22T07:15:52.607 INFO:tasks.workunit.client.0.smithi063.stdout:6/248: chown d3/d6/d2a/d38/f4d 211 1 2024-03-22T07:15:52.607 INFO:tasks.workunit.client.0.smithi063.stdout:9/179: dwrite d8/fa [0,4194304] 0 2024-03-22T07:15:52.607 INFO:tasks.workunit.client.0.smithi063.stdout:9/180: write d8/f10 [892050,77629] 0 2024-03-22T07:15:52.612 INFO:tasks.workunit.client.0.smithi063.stdout:6/249: rename d3/d15/d29/d4c/c21 to d3/d15/d29/c58 0 2024-03-22T07:15:52.613 INFO:tasks.workunit.client.0.smithi063.stdout:6/250: truncate d3/f14 1153834 0 2024-03-22T07:15:52.613 INFO:tasks.workunit.client.0.smithi063.stdout:6/251: fdatasync d3/d15/d29/d4e/f50 0 2024-03-22T07:15:52.614 INFO:tasks.workunit.client.0.smithi063.stdout:8/254: dwrite d0/d26/f34 [0,4194304] 0 2024-03-22T07:15:52.614 INFO:tasks.workunit.client.0.smithi063.stdout:8/255: stat d0/d1b/d28/f3b 0 2024-03-22T07:15:52.615 INFO:tasks.workunit.client.0.smithi063.stdout:6/252: creat d3/d15/d16/f59 x:0 0 0 2024-03-22T07:15:52.622 INFO:tasks.workunit.client.0.smithi063.stdout:8/256: mkdir d0/d18/d4a 0 2024-03-22T07:15:52.623 INFO:tasks.workunit.client.0.smithi063.stdout:8/257: symlink d0/d26/l4b 0 2024-03-22T07:15:52.623 INFO:tasks.workunit.client.0.smithi063.stdout:6/253: truncate d3/f18 3460837 0 2024-03-22T07:15:52.628 INFO:tasks.workunit.client.0.smithi063.stdout:0/239: dwrite da/de/d17/f28 [0,4194304] 0 2024-03-22T07:15:52.628 INFO:tasks.workunit.client.0.smithi063.stdout:9/181: dwrite d8/f1a [4194304,4194304] 0 2024-03-22T07:15:52.633 INFO:tasks.workunit.client.0.smithi063.stdout:6/254: dread d3/f4 [0,4194304] 0 2024-03-22T07:15:52.633 INFO:tasks.workunit.client.0.smithi063.stdout:6/255: chown d3/d6 580 1 2024-03-22T07:15:52.635 INFO:tasks.workunit.client.0.smithi063.stdout:9/182: mkdir d8/d19/d20/d22/d29/d31 0 2024-03-22T07:15:52.637 INFO:tasks.workunit.client.0.smithi063.stdout:9/183: creat d8/d19/d20/d22/d29/d31/f32 x:0 0 0 2024-03-22T07:15:52.638 INFO:tasks.workunit.client.0.smithi063.stdout:9/184: symlink d8/d17/l33 0 2024-03-22T07:15:52.639 INFO:tasks.workunit.client.0.smithi063.stdout:5/183: sync 2024-03-22T07:15:52.639 INFO:tasks.workunit.client.0.smithi063.stdout:7/183: sync 2024-03-22T07:15:52.639 INFO:tasks.workunit.client.0.smithi063.stdout:7/184: write d5/d14/f1a [408895,33733] 0 2024-03-22T07:15:52.639 INFO:tasks.workunit.client.0.smithi063.stdout:4/190: sync 2024-03-22T07:15:52.639 INFO:tasks.workunit.client.0.smithi063.stdout:2/215: sync 2024-03-22T07:15:52.639 INFO:tasks.workunit.client.0.smithi063.stdout:2/216: truncate d0/de/d35/d36/f3a 1113592 0 2024-03-22T07:15:52.639 INFO:tasks.workunit.client.0.smithi063.stdout:2/217: readlink d0/de/d12/d39/l3d 0 2024-03-22T07:15:52.640 INFO:tasks.workunit.client.0.smithi063.stdout:2/218: readlink d0/de/l2e 0 2024-03-22T07:15:52.640 INFO:tasks.workunit.client.0.smithi063.stdout:9/185: creat d8/d19/d20/f34 x:0 0 0 2024-03-22T07:15:52.642 INFO:tasks.workunit.client.0.smithi063.stdout:7/185: link d5/d14/d1d/l20 d5/d14/d1d/d29/d3b/l3d 0 2024-03-22T07:15:52.646 INFO:tasks.workunit.client.0.smithi063.stdout:4/191: rename d1/d5/f7 to d1/d13/d27/d2c/f3e 0 2024-03-22T07:15:52.647 INFO:tasks.workunit.client.0.smithi063.stdout:4/192: chown d1/d13/f39 2 1 2024-03-22T07:15:52.647 INFO:tasks.workunit.client.0.smithi063.stdout:2/219: creat d0/de/d1b/d32/f43 x:0 0 0 2024-03-22T07:15:52.647 INFO:tasks.workunit.client.0.smithi063.stdout:6/256: unlink d3/f18 0 2024-03-22T07:15:52.647 INFO:tasks.workunit.client.0.smithi063.stdout:9/186: creat d8/d19/f35 x:0 0 0 2024-03-22T07:15:52.648 INFO:tasks.workunit.client.0.smithi063.stdout:1/262: sync 2024-03-22T07:15:52.654 INFO:tasks.workunit.client.0.smithi063.stdout:8/258: dwrite d0/f1a [0,4194304] 0 2024-03-22T07:15:52.654 INFO:tasks.workunit.client.0.smithi063.stdout:4/193: mknod d1/d5/c3f 0 2024-03-22T07:15:52.654 INFO:tasks.workunit.client.0.smithi063.stdout:4/194: write d1/d5/f9 [2976819,126094] 0 2024-03-22T07:15:52.654 INFO:tasks.workunit.client.0.smithi063.stdout:5/184: dread d8/d13/f18 [0,4194304] 0 2024-03-22T07:15:52.654 INFO:tasks.workunit.client.0.smithi063.stdout:0/240: dwrite da/de/f24 [0,4194304] 0 2024-03-22T07:15:52.654 INFO:tasks.workunit.client.0.smithi063.stdout:5/185: write d8/f12 [4343162,55858] 0 2024-03-22T07:15:52.655 INFO:tasks.workunit.client.0.smithi063.stdout:5/186: chown d8/d13/f18 239102 1 2024-03-22T07:15:52.655 INFO:tasks.workunit.client.0.smithi063.stdout:0/241: truncate da/de/f31 6103883 0 2024-03-22T07:15:52.655 INFO:tasks.workunit.client.0.smithi063.stdout:7/186: dread d5/d16/f22 [0,4194304] 0 2024-03-22T07:15:52.655 INFO:tasks.workunit.client.0.smithi063.stdout:7/187: chown d5/da/ff 89 1 2024-03-22T07:15:52.655 INFO:tasks.workunit.client.0.smithi063.stdout:7/188: readlink d5/d14/d1d/l20 0 2024-03-22T07:15:52.655 INFO:tasks.workunit.client.0.smithi063.stdout:6/257: mkdir d3/d15/d29/d4e/d5a 0 2024-03-22T07:15:52.655 INFO:tasks.workunit.client.0.smithi063.stdout:5/187: dread d8/f16 [0,4194304] 0 2024-03-22T07:15:52.658 INFO:tasks.workunit.client.0.smithi063.stdout:9/187: creat d8/d19/d20/d22/d29/f36 x:0 0 0 2024-03-22T07:15:52.661 INFO:tasks.workunit.client.0.smithi063.stdout:8/259: truncate d0/d1b/d28/f3b 1049276 0 2024-03-22T07:15:52.663 INFO:tasks.workunit.client.0.smithi063.stdout:0/242: write da/de/d17/f28 [3196916,48610] 0 2024-03-22T07:15:52.664 INFO:tasks.workunit.client.0.smithi063.stdout:4/195: symlink d1/d13/d27/l40 0 2024-03-22T07:15:52.664 INFO:tasks.workunit.client.0.smithi063.stdout:4/196: write d1/f26 [946394,79200] 0 2024-03-22T07:15:52.666 INFO:tasks.workunit.client.0.smithi063.stdout:2/220: dread d0/fa [4194304,4194304] 0 2024-03-22T07:15:52.669 INFO:tasks.workunit.client.0.smithi063.stdout:7/189: write d5/d14/d1d/d29/f36 [3365364,99742] 0 2024-03-22T07:15:52.672 INFO:tasks.workunit.client.0.smithi063.stdout:8/260: write d0/f22 [1817896,15565] 0 2024-03-22T07:15:52.674 INFO:tasks.workunit.client.0.smithi063.stdout:5/188: creat d8/d13/f39 x:0 0 0 2024-03-22T07:15:52.675 INFO:tasks.workunit.client.0.smithi063.stdout:9/188: mkdir d8/d19/d20/d22/d29/d31/d37 0 2024-03-22T07:15:52.676 INFO:tasks.workunit.client.0.smithi063.stdout:4/197: creat d1/d5/dc/f41 x:0 0 0 2024-03-22T07:15:52.676 INFO:tasks.workunit.client.0.smithi063.stdout:4/198: chown d1/d13/d27/d2c/d3d 11173 1 2024-03-22T07:15:52.678 INFO:tasks.workunit.client.0.smithi063.stdout:2/221: creat d0/de/d35/f44 x:0 0 0 2024-03-22T07:15:52.679 INFO:tasks.workunit.client.0.smithi063.stdout:7/190: link d5/l15 d5/l3e 0 2024-03-22T07:15:52.680 INFO:tasks.workunit.client.0.smithi063.stdout:8/261: mkdir d0/d24/d33/d4c 0 2024-03-22T07:15:52.680 INFO:tasks.workunit.client.0.smithi063.stdout:8/262: chown d0/d18/d4a 0 1 2024-03-22T07:15:52.682 INFO:tasks.workunit.client.0.smithi063.stdout:5/189: mknod d8/d13/d28/c3a 0 2024-03-22T07:15:52.682 INFO:tasks.workunit.client.0.smithi063.stdout:5/190: readlink d8/d13/d1e/l29 0 2024-03-22T07:15:52.683 INFO:tasks.workunit.client.0.smithi063.stdout:5/191: dread - d8/f2b zero size 2024-03-22T07:15:52.683 INFO:tasks.workunit.client.0.smithi063.stdout:5/192: creat d8/d13/d24/f3b x:0 0 0 2024-03-22T07:15:52.683 INFO:tasks.workunit.client.0.smithi063.stdout:5/193: chown d8/f25 351778 1 2024-03-22T07:15:52.683 INFO:tasks.workunit.client.0.smithi063.stdout:5/194: fdatasync d8/d13/f39 0 2024-03-22T07:15:52.683 INFO:tasks.workunit.client.0.smithi063.stdout:4/199: mknod d1/d13/d27/d2c/d3d/c42 0 2024-03-22T07:15:52.683 INFO:tasks.workunit.client.0.smithi063.stdout:2/222: mknod d0/de/d12/d20/c45 0 2024-03-22T07:15:52.685 INFO:tasks.workunit.client.0.smithi063.stdout:5/195: write d8/f14 [400567,63038] 0 2024-03-22T07:15:52.686 INFO:tasks.workunit.client.0.smithi063.stdout:0/243: dwrite da/f40 [4194304,4194304] 0 2024-03-22T07:15:52.687 INFO:tasks.workunit.client.0.smithi063.stdout:2/223: dread d0/de/f14 [0,4194304] 0 2024-03-22T07:15:52.688 INFO:tasks.workunit.client.0.smithi063.stdout:5/196: rmdir d8/d13/d1e 39 2024-03-22T07:15:52.689 INFO:tasks.workunit.client.0.smithi063.stdout:5/197: truncate d8/f1c 3085321 0 2024-03-22T07:15:52.691 INFO:tasks.workunit.client.0.smithi063.stdout:1/263: dwrite d4/f1e [0,4194304] 0 2024-03-22T07:15:52.714 INFO:tasks.workunit.client.0.smithi063.stdout:8/263: dread d0/f22 [0,4194304] 0 2024-03-22T07:15:52.715 INFO:tasks.workunit.client.0.smithi063.stdout:8/264: creat d0/d26/d2d/f4d x:0 0 0 2024-03-22T07:15:52.715 INFO:tasks.workunit.client.0.smithi063.stdout:8/265: chown d0/d1b/f1e 393139409 1 2024-03-22T07:15:52.718 INFO:tasks.workunit.client.0.smithi063.stdout:0/244: write da/f40 [3336297,52494] 0 2024-03-22T07:15:52.718 INFO:tasks.workunit.client.0.smithi063.stdout:3/171: sync 2024-03-22T07:15:52.719 INFO:tasks.workunit.client.0.smithi063.stdout:0/245: mknod da/de/d17/d25/c50 0 2024-03-22T07:15:52.722 INFO:tasks.workunit.client.0.smithi063.stdout:0/246: creat da/d49/f51 x:0 0 0 2024-03-22T07:15:52.722 INFO:tasks.workunit.client.0.smithi063.stdout:0/247: creat da/de/d17/d25/d36/f52 x:0 0 0 2024-03-22T07:15:52.723 INFO:tasks.workunit.client.0.smithi063.stdout:3/172: dread da/d16/d26/f27 [0,4194304] 0 2024-03-22T07:15:52.725 INFO:tasks.workunit.client.0.smithi063.stdout:4/200: rename d1/d13 to d1/d5/d43 0 2024-03-22T07:15:52.729 INFO:tasks.workunit.client.0.smithi063.stdout:8/266: dread d0/f1a [0,4194304] 0 2024-03-22T07:15:52.729 INFO:tasks.workunit.client.0.smithi063.stdout:8/267: truncate d0/d26/d2d/d3d/f44 806475 0 2024-03-22T07:15:52.731 INFO:tasks.workunit.client.0.smithi063.stdout:5/198: dwrite d8/fe [0,4194304] 0 2024-03-22T07:15:52.731 INFO:tasks.workunit.client.0.smithi063.stdout:7/191: dwrite d5/d16/f2b [0,4194304] 0 2024-03-22T07:15:52.731 INFO:tasks.workunit.client.0.smithi063.stdout:7/192: chown d5/d16/d1b/c23 40860036 1 2024-03-22T07:15:52.732 INFO:tasks.workunit.client.0.smithi063.stdout:4/201: dread d1/f2 [0,4194304] 0 2024-03-22T07:15:52.732 INFO:tasks.workunit.client.0.smithi063.stdout:4/202: chown d1/f6 815999365 1 2024-03-22T07:15:52.732 INFO:tasks.workunit.client.0.smithi063.stdout:4/203: fsync d1/f11 0 2024-03-22T07:15:52.732 INFO:tasks.workunit.client.0.smithi063.stdout:4/204: stat d1/d5/dc/l34 0 2024-03-22T07:15:52.732 INFO:tasks.workunit.client.0.smithi063.stdout:4/205: write d1/d5/dc/f1d [2921449,49326] 0 2024-03-22T07:15:52.732 INFO:tasks.workunit.client.0.smithi063.stdout:4/206: chown d1/d5/d43 844 1 2024-03-22T07:15:52.733 INFO:tasks.workunit.client.0.smithi063.stdout:2/224: dwrite d0/de/d12/d20/f29 [0,4194304] 0 2024-03-22T07:15:52.733 INFO:tasks.workunit.client.0.smithi063.stdout:3/173: dread da/fb [0,4194304] 0 2024-03-22T07:15:52.734 INFO:tasks.workunit.client.0.smithi063.stdout:3/174: creat da/d16/f38 x:0 0 0 2024-03-22T07:15:52.734 INFO:tasks.workunit.client.0.smithi063.stdout:3/175: creat da/d16/d26/f39 x:0 0 0 2024-03-22T07:15:52.734 INFO:tasks.workunit.client.0.smithi063.stdout:3/176: write da/d16/d26/f36 [896779,36392] 0 2024-03-22T07:15:52.735 INFO:tasks.workunit.client.0.smithi063.stdout:3/177: chown da/d16/d26/c28 18 1 2024-03-22T07:15:52.735 INFO:tasks.workunit.client.0.smithi063.stdout:0/248: write da/de/f23 [2613392,56079] 0 2024-03-22T07:15:52.735 INFO:tasks.workunit.client.0.smithi063.stdout:7/193: creat d5/d14/d1d/f3f x:0 0 0 2024-03-22T07:15:52.735 INFO:tasks.workunit.client.0.smithi063.stdout:7/194: readlink d5/d14/d1d/l20 0 2024-03-22T07:15:52.735 INFO:tasks.workunit.client.0.smithi063.stdout:2/225: symlink d0/de/l46 0 2024-03-22T07:15:52.735 INFO:tasks.workunit.client.0.smithi063.stdout:2/226: dread - d0/de/d1b/d32/f3b zero size 2024-03-22T07:15:52.736 INFO:tasks.workunit.client.0.smithi063.stdout:5/199: write d8/f16 [4089328,130840] 0 2024-03-22T07:15:52.736 INFO:tasks.workunit.client.0.smithi063.stdout:4/207: dread d1/f26 [0,4194304] 0 2024-03-22T07:15:52.738 INFO:tasks.workunit.client.0.smithi063.stdout:0/249: mkdir da/de/d17/d25/d3c/d53 0 2024-03-22T07:15:52.738 INFO:tasks.workunit.client.0.smithi063.stdout:0/250: fsync da/f2b 0 2024-03-22T07:15:52.738 INFO:tasks.workunit.client.0.smithi063.stdout:0/251: creat da/f54 x:0 0 0 2024-03-22T07:15:52.739 INFO:tasks.workunit.client.0.smithi063.stdout:8/268: dread d0/d1b/d28/f43 [0,4194304] 0 2024-03-22T07:15:52.739 INFO:tasks.workunit.client.0.smithi063.stdout:8/269: write d0/d1b/f20 [4551579,40546] 0 2024-03-22T07:15:52.739 INFO:tasks.workunit.client.0.smithi063.stdout:8/270: chown d0/d26/d2d/d3d 264179 1 2024-03-22T07:15:52.739 INFO:tasks.workunit.client.0.smithi063.stdout:7/195: mkdir d5/d14/d1d/d40 0 2024-03-22T07:15:52.740 INFO:tasks.workunit.client.0.smithi063.stdout:4/208: truncate d1/f11 2375284 0 2024-03-22T07:15:52.743 INFO:tasks.workunit.client.0.smithi063.stdout:0/252: creat da/d2e/f55 x:0 0 0 2024-03-22T07:15:52.745 INFO:tasks.workunit.client.0.smithi063.stdout:8/271: creat d0/d24/d33/d4c/f4e x:0 0 0 2024-03-22T07:15:52.745 INFO:tasks.workunit.client.0.smithi063.stdout:8/272: write d0/d24/f3a [1028607,113670] 0 2024-03-22T07:15:52.747 INFO:tasks.workunit.client.0.smithi063.stdout:5/200: rmdir d8/d13/d24 39 2024-03-22T07:15:52.747 INFO:tasks.workunit.client.0.smithi063.stdout:5/201: fdatasync d8/f12 0 2024-03-22T07:15:52.748 INFO:tasks.workunit.client.0.smithi063.stdout:3/178: write da/f18 [1790477,13765] 0 2024-03-22T07:15:52.750 INFO:tasks.workunit.client.0.smithi063.stdout:0/253: mkdir da/d2e/d56 0 2024-03-22T07:15:52.750 INFO:tasks.workunit.client.0.smithi063.stdout:0/254: fdatasync da/de/d17/d25/d36/f52 0 2024-03-22T07:15:52.753 INFO:tasks.workunit.client.0.smithi063.stdout:7/196: rmdir d5/d16 39 2024-03-22T07:15:52.755 INFO:tasks.workunit.client.0.smithi063.stdout:4/209: read d1/d5/dc/ff [3853056,4350] 0 2024-03-22T07:15:52.757 INFO:tasks.workunit.client.0.smithi063.stdout:0/255: creat da/de/d17/d4e/f57 x:0 0 0 2024-03-22T07:15:52.758 INFO:tasks.workunit.client.0.smithi063.stdout:2/227: dwrite d0/de/d1b/d32/f43 [0,4194304] 0 2024-03-22T07:15:52.758 INFO:tasks.workunit.client.0.smithi063.stdout:2/228: creat d0/de/d12/d20/f47 x:0 0 0 2024-03-22T07:15:52.759 INFO:tasks.workunit.client.0.smithi063.stdout:7/197: creat d5/d14/d1d/d29/d3b/f41 x:0 0 0 2024-03-22T07:15:52.759 INFO:tasks.workunit.client.0.smithi063.stdout:7/198: creat d5/d14/f42 x:0 0 0 2024-03-22T07:15:52.759 INFO:tasks.workunit.client.0.smithi063.stdout:7/199: truncate d5/f18 1709796 0 2024-03-22T07:15:52.765 INFO:tasks.workunit.client.0.smithi063.stdout:3/179: getdents da/d16/d26 0 2024-03-22T07:15:52.767 INFO:tasks.workunit.client.0.smithi063.stdout:7/200: mkdir d5/d16/d43 0 2024-03-22T07:15:52.770 INFO:tasks.workunit.client.0.smithi063.stdout:2/229: unlink d0/c6 0 2024-03-22T07:15:52.773 INFO:tasks.workunit.client.0.smithi063.stdout:8/273: dwrite d0/d26/f34 [0,4194304] 0 2024-03-22T07:15:52.773 INFO:tasks.workunit.client.0.smithi063.stdout:4/210: write d1/f11 [3206781,111639] 0 2024-03-22T07:15:52.773 INFO:tasks.workunit.client.0.smithi063.stdout:4/211: chown d1/d5/f12 1452935786 1 2024-03-22T07:15:52.773 INFO:tasks.workunit.client.0.smithi063.stdout:4/212: truncate d1/d5/d43/f31 620769 0 2024-03-22T07:15:52.774 INFO:tasks.workunit.client.0.smithi063.stdout:5/202: dwrite d8/f2b [0,4194304] 0 2024-03-22T07:15:52.775 INFO:tasks.workunit.client.0.smithi063.stdout:2/230: creat d0/de/d1b/f48 x:0 0 0 2024-03-22T07:15:52.777 INFO:tasks.workunit.client.0.smithi063.stdout:5/203: truncate d8/d13/f18 3743160 0 2024-03-22T07:15:52.787 INFO:tasks.workunit.client.0.smithi063.stdout:0/256: dwrite da/de/d39/f41 [0,4194304] 0 2024-03-22T07:15:52.788 INFO:tasks.workunit.client.0.smithi063.stdout:8/274: dread d0/d24/f42 [0,4194304] 0 2024-03-22T07:15:52.788 INFO:tasks.workunit.client.0.smithi063.stdout:0/257: write da/de/d17/f3a [136320,10296] 0 2024-03-22T07:15:52.788 INFO:tasks.workunit.client.0.smithi063.stdout:0/258: write da/de/d17/f3a [151557,58210] 0 2024-03-22T07:15:52.788 INFO:tasks.workunit.client.0.smithi063.stdout:2/231: dread d0/de/d1b/d32/f43 [0,4194304] 0 2024-03-22T07:15:52.794 INFO:tasks.workunit.client.0.smithi063.stdout:7/201: dwrite d5/da/ff [4194304,4194304] 0 2024-03-22T07:15:52.794 INFO:tasks.workunit.client.0.smithi063.stdout:5/204: dread d8/f12 [0,4194304] 0 2024-03-22T07:15:52.795 INFO:tasks.workunit.client.0.smithi063.stdout:7/202: mknod d5/d14/c44 0 2024-03-22T07:15:52.799 INFO:tasks.workunit.client.0.smithi063.stdout:8/275: write d0/d24/f35 [3832049,41941] 0 2024-03-22T07:15:52.799 INFO:tasks.workunit.client.0.smithi063.stdout:3/180: dwrite da/f11 [0,4194304] 0 2024-03-22T07:15:52.799 INFO:tasks.workunit.client.0.smithi063.stdout:3/181: write da/f2f [958912,48708] 0 2024-03-22T07:15:52.799 INFO:tasks.workunit.client.0.smithi063.stdout:3/182: fsync da/f35 0 2024-03-22T07:15:52.799 INFO:tasks.workunit.client.0.smithi063.stdout:3/183: stat da/f18 0 2024-03-22T07:15:52.803 INFO:tasks.workunit.client.0.smithi063.stdout:7/203: unlink l1 0 2024-03-22T07:15:52.803 INFO:tasks.workunit.client.0.smithi063.stdout:7/204: chown d5/d16/d1b 261128 1 2024-03-22T07:15:52.806 INFO:tasks.workunit.client.0.smithi063.stdout:8/276: mkdir d0/d1b/d4f 0 2024-03-22T07:15:52.812 INFO:tasks.workunit.client.0.smithi063.stdout:8/277: dread d0/f19 [0,4194304] 0 2024-03-22T07:15:52.812 INFO:tasks.workunit.client.0.smithi063.stdout:8/278: getdents d0/d1b/d4f 0 2024-03-22T07:15:52.812 INFO:tasks.workunit.client.0.smithi063.stdout:8/279: readlink d0/l15 0 2024-03-22T07:15:52.813 INFO:tasks.workunit.client.0.smithi063.stdout:0/259: dwrite da/de/d17/d25/d3c/f3f [0,4194304] 0 2024-03-22T07:15:52.813 INFO:tasks.workunit.client.0.smithi063.stdout:0/260: fdatasync da/d2e/f33 0 2024-03-22T07:15:52.815 INFO:tasks.workunit.client.0.smithi063.stdout:3/184: rename da/f18 to da/f3a 0 2024-03-22T07:15:52.815 INFO:tasks.workunit.client.0.smithi063.stdout:3/185: chown da/d16/d26/c37 0 1 2024-03-22T07:15:52.815 INFO:tasks.workunit.client.0.smithi063.stdout:3/186: chown da/l30 8 1 2024-03-22T07:15:52.816 INFO:tasks.workunit.client.0.smithi063.stdout:7/205: read d5/f17 [2715450,30691] 0 2024-03-22T07:15:52.816 INFO:tasks.workunit.client.0.smithi063.stdout:8/280: mknod d0/d24/d33/c50 0 2024-03-22T07:15:52.818 INFO:tasks.workunit.client.0.smithi063.stdout:0/261: mkdir da/de/d17/d25/d36/d58 0 2024-03-22T07:15:52.818 INFO:tasks.workunit.client.0.smithi063.stdout:0/262: chown da/de/d17/d25 75228515 1 2024-03-22T07:15:52.819 INFO:tasks.workunit.client.0.smithi063.stdout:7/206: mknod d5/d16/d1b/d21/c45 0 2024-03-22T07:15:52.821 INFO:tasks.workunit.client.0.smithi063.stdout:0/263: rename da/de/d17/l1b to da/de/d17/l59 0 2024-03-22T07:15:52.821 INFO:tasks.workunit.client.0.smithi063.stdout:0/264: chown da/de/d17/f2f 0 1 2024-03-22T07:15:52.823 INFO:tasks.workunit.client.0.smithi063.stdout:4/213: dwrite d1/f11 [0,4194304] 0 2024-03-22T07:15:52.823 INFO:tasks.workunit.client.0.smithi063.stdout:4/214: chown d1/d5/d43/f2e 13786528 1 2024-03-22T07:15:52.823 INFO:tasks.workunit.client.0.smithi063.stdout:5/205: dwrite d8/f10 [0,4194304] 0 2024-03-22T07:15:52.823 INFO:tasks.workunit.client.0.smithi063.stdout:3/187: mknod da/c3b 0 2024-03-22T07:15:52.823 INFO:tasks.workunit.client.0.smithi063.stdout:3/188: readlink l9 0 2024-03-22T07:15:52.824 INFO:tasks.workunit.client.0.smithi063.stdout:7/207: dread d5/da/f11 [0,4194304] 0 2024-03-22T07:15:52.824 INFO:tasks.workunit.client.0.smithi063.stdout:7/208: creat d5/f46 x:0 0 0 2024-03-22T07:15:52.824 INFO:tasks.workunit.client.0.smithi063.stdout:7/209: write d5/f13 [1456401,124906] 0 2024-03-22T07:15:52.827 INFO:tasks.workunit.client.0.smithi063.stdout:8/281: rename d0/cf to d0/d18/d4a/c51 0 2024-03-22T07:15:52.827 INFO:tasks.workunit.client.0.smithi063.stdout:8/282: write d0/f19 [1788657,113768] 0 2024-03-22T07:15:52.828 INFO:tasks.workunit.client.0.smithi063.stdout:4/215: dread d1/d5/f12 [0,4194304] 0 2024-03-22T07:15:52.829 INFO:tasks.workunit.client.0.smithi063.stdout:0/265: rmdir da/de/d17/d25/d3c/d43 39 2024-03-22T07:15:52.829 INFO:tasks.workunit.client.0.smithi063.stdout:0/266: fsync da/de/d17/d25/f3e 0 2024-03-22T07:15:52.831 INFO:tasks.workunit.client.0.smithi063.stdout:0/267: dread da/d2e/f33 [0,4194304] 0 2024-03-22T07:15:52.831 INFO:tasks.workunit.client.0.smithi063.stdout:0/268: fsync da/de/d17/d25/d3c/f3f 0 2024-03-22T07:15:52.831 INFO:tasks.workunit.client.0.smithi063.stdout:0/269: write da/de/d17/d25/f38 [406749,21704] 0 2024-03-22T07:15:52.831 INFO:tasks.workunit.client.0.smithi063.stdout:0/270: write da/d2e/f33 [1326573,16659] 0 2024-03-22T07:15:52.832 INFO:tasks.workunit.client.0.smithi063.stdout:5/206: mkdir d8/d13/d1e/d3c 0 2024-03-22T07:15:52.832 INFO:tasks.workunit.client.0.smithi063.stdout:5/207: getdents d8/d13/d1e/d3c 0 2024-03-22T07:15:52.833 INFO:tasks.workunit.client.0.smithi063.stdout:3/189: mkdir da/d16/d3c 0 2024-03-22T07:15:52.835 INFO:tasks.workunit.client.0.smithi063.stdout:8/283: getdents d0/d26 0 2024-03-22T07:15:52.837 INFO:tasks.workunit.client.0.smithi063.stdout:5/208: creat d8/f3d x:0 0 0 2024-03-22T07:15:52.838 INFO:tasks.workunit.client.0.smithi063.stdout:3/190: dread da/fb [0,4194304] 0 2024-03-22T07:15:52.841 INFO:tasks.workunit.client.0.smithi063.stdout:8/284: mknod d0/c52 0 2024-03-22T07:15:52.842 INFO:tasks.workunit.client.0.smithi063.stdout:5/209: getdents d8/d13 0 2024-03-22T07:15:52.843 INFO:tasks.workunit.client.0.smithi063.stdout:0/271: dread da/de/d17/f2f [0,4194304] 0 2024-03-22T07:15:52.844 INFO:tasks.workunit.client.0.smithi063.stdout:8/285: creat d0/d1f/f53 x:0 0 0 2024-03-22T07:15:52.844 INFO:tasks.workunit.client.0.smithi063.stdout:8/286: stat d0/d26/f34 0 2024-03-22T07:15:52.844 INFO:tasks.workunit.client.0.smithi063.stdout:8/287: stat d0/f22 0 2024-03-22T07:15:52.845 INFO:tasks.workunit.client.0.smithi063.stdout:7/210: dwrite d5/d16/f22 [4194304,4194304] 0 2024-03-22T07:15:52.845 INFO:tasks.workunit.client.0.smithi063.stdout:0/272: mkdir da/de/d39/d5a 0 2024-03-22T07:15:52.849 INFO:tasks.workunit.client.0.smithi063.stdout:3/191: write da/d16/d26/f2a [486231,19275] 0 2024-03-22T07:15:52.851 INFO:tasks.workunit.client.0.smithi063.stdout:7/211: unlink d5/f46 0 2024-03-22T07:15:52.851 INFO:tasks.workunit.client.0.smithi063.stdout:7/212: creat d5/d14/d1d/d29/f47 x:0 0 0 2024-03-22T07:15:52.851 INFO:tasks.workunit.client.0.smithi063.stdout:7/213: getdents d5/da/d28/d31/d33 0 2024-03-22T07:15:52.853 INFO:tasks.workunit.client.0.smithi063.stdout:3/192: creat da/f3d x:0 0 0 2024-03-22T07:15:52.854 INFO:tasks.workunit.client.0.smithi063.stdout:3/193: getdents da/d16/d3c 0 2024-03-22T07:15:52.857 INFO:tasks.workunit.client.0.smithi063.stdout:3/194: mknod da/d16/c3e 0 2024-03-22T07:15:52.858 INFO:tasks.workunit.client.0.smithi063.stdout:3/195: mkdir da/d16/d3f 0 2024-03-22T07:15:52.860 INFO:tasks.workunit.client.0.smithi063.stdout:3/196: symlink da/d16/l40 0 2024-03-22T07:15:52.860 INFO:tasks.workunit.client.0.smithi063.stdout:7/214: dread d5/d16/f1e [0,4194304] 0 2024-03-22T07:15:52.860 INFO:tasks.workunit.client.0.smithi063.stdout:7/215: write d5/da/f25 [1221835,40738] 0 2024-03-22T07:15:52.862 INFO:tasks.workunit.client.0.smithi063.stdout:7/216: symlink d5/da/d28/d31/l48 0 2024-03-22T07:15:52.863 INFO:tasks.workunit.client.0.smithi063.stdout:3/197: link da/d16/d26/l2c da/l41 0 2024-03-22T07:15:52.863 INFO:tasks.workunit.client.0.smithi063.stdout:3/198: creat da/d16/f42 x:0 0 0 2024-03-22T07:15:52.865 INFO:tasks.workunit.client.0.smithi063.stdout:8/288: dread d0/d26/f34 [0,4194304] 0 2024-03-22T07:15:52.865 INFO:tasks.workunit.client.0.smithi063.stdout:5/210: dwrite d8/d13/d24/f33 [0,4194304] 0 2024-03-22T07:15:52.865 INFO:tasks.workunit.client.0.smithi063.stdout:5/211: readlink d8/l23 0 2024-03-22T07:15:52.865 INFO:tasks.workunit.client.0.smithi063.stdout:7/217: unlink d5/da/d28/d31/l48 0 2024-03-22T07:15:52.871 INFO:tasks.workunit.client.0.smithi063.stdout:7/218: symlink d5/d14/d1d/d40/l49 0 2024-03-22T07:15:52.872 INFO:tasks.workunit.client.0.smithi063.stdout:3/199: mknod da/c43 0 2024-03-22T07:15:52.873 INFO:tasks.workunit.client.0.smithi063.stdout:3/200: fsync da/d16/f32 0 2024-03-22T07:15:52.874 INFO:tasks.workunit.client.0.smithi063.stdout:7/219: mkdir d5/d14/d1d/d29/d4a 0 2024-03-22T07:15:52.879 INFO:tasks.workunit.client.0.smithi063.stdout:5/212: dwrite d8/f25 [0,4194304] 0 2024-03-22T07:15:52.889 INFO:tasks.workunit.client.0.smithi063.stdout:3/201: dwrite da/d16/d26/f39 [0,4194304] 0 2024-03-22T07:15:52.889 INFO:tasks.workunit.client.0.smithi063.stdout:3/202: write da/f21 [1007811,40324] 0 2024-03-22T07:15:52.893 INFO:tasks.workunit.client.0.smithi063.stdout:5/213: mknod d8/d13/d28/c3e 0 2024-03-22T07:15:52.893 INFO:tasks.workunit.client.0.smithi063.stdout:5/214: readlink d8/d13/l19 0 2024-03-22T07:15:52.893 INFO:tasks.workunit.client.0.smithi063.stdout:5/215: truncate d8/d13/d24/f33 4420261 0 2024-03-22T07:15:52.893 INFO:tasks.workunit.client.0.smithi063.stdout:5/216: dread - d8/d13/d24/f3b zero size 2024-03-22T07:15:52.896 INFO:tasks.workunit.client.0.smithi063.stdout:5/217: link d8/d13/c26 d8/d13/d28/c3f 0 2024-03-22T07:15:52.896 INFO:tasks.workunit.client.0.smithi063.stdout:5/218: write d8/d13/d1e/f38 [321531,36998] 0 2024-03-22T07:15:52.896 INFO:tasks.workunit.client.0.smithi063.stdout:5/219: chown d8/d13/c1a 15 1 2024-03-22T07:15:52.897 INFO:tasks.workunit.client.0.smithi063.stdout:5/220: symlink d8/d13/d28/l40 0 2024-03-22T07:15:52.898 INFO:tasks.workunit.client.0.smithi063.stdout:5/221: mknod d8/d13/d1e/c41 0 2024-03-22T07:15:52.898 INFO:tasks.workunit.client.0.smithi063.stdout:5/222: chown d8/lb 1367428 1 2024-03-22T07:15:52.900 INFO:tasks.workunit.client.0.smithi063.stdout:5/223: stat d8/l23 0 2024-03-22T07:15:52.900 INFO:tasks.workunit.client.0.smithi063.stdout:5/224: mknod d8/d13/d28/c42 0 2024-03-22T07:15:52.903 INFO:tasks.workunit.client.0.smithi063.stdout:5/225: creat d8/d13/d28/f43 x:0 0 0 2024-03-22T07:15:52.903 INFO:tasks.workunit.client.0.smithi063.stdout:5/226: chown d8/d13/d24/c34 80477120 1 2024-03-22T07:15:52.903 INFO:tasks.workunit.client.0.smithi063.stdout:5/227: read - d8/d13/d24/f3b zero size 2024-03-22T07:15:52.973 INFO:tasks.workunit.client.0.smithi063.stdout:0/273: dwrite da/fc [0,4194304] 0 2024-03-22T07:15:52.974 INFO:tasks.workunit.client.0.smithi063.stdout:0/274: creat da/de/d17/d25/d3c/d43/f5b x:0 0 0 2024-03-22T07:15:52.975 INFO:tasks.workunit.client.0.smithi063.stdout:0/275: rmdir da/d49 39 2024-03-22T07:15:52.977 INFO:tasks.workunit.client.0.smithi063.stdout:0/276: rename da/f2a to da/de/f5c 0 2024-03-22T07:15:52.977 INFO:tasks.workunit.client.0.smithi063.stdout:0/277: chown da/d2e 6096 1 2024-03-22T07:15:52.981 INFO:tasks.workunit.client.0.smithi063.stdout:0/278: dread da/de/f23 [0,4194304] 0 2024-03-22T07:15:52.981 INFO:tasks.workunit.client.0.smithi063.stdout:0/279: fsync da/de/d17/d25/d36/f4b 0 2024-03-22T07:15:52.981 INFO:tasks.workunit.client.0.smithi063.stdout:0/280: fdatasync da/de/d17/d25/f3e 0 2024-03-22T07:15:52.982 INFO:tasks.workunit.client.0.smithi063.stdout:0/281: rmdir da/de/d39/d5a 0 2024-03-22T07:15:52.982 INFO:tasks.workunit.client.0.smithi063.stdout:0/282: creat da/de/d17/f5d x:0 0 0 2024-03-22T07:15:52.988 INFO:tasks.workunit.client.0.smithi063.stdout:0/283: truncate da/fc 3288750 0 2024-03-22T07:15:52.988 INFO:tasks.workunit.client.0.smithi063.stdout:0/284: fsync da/de/f23 0 2024-03-22T07:15:52.988 INFO:tasks.workunit.client.0.smithi063.stdout:0/285: symlink da/de/d39/l5e 0 2024-03-22T07:15:52.989 INFO:tasks.workunit.client.0.smithi063.stdout:0/286: getdents da/de/d17/d25/d36 0 2024-03-22T07:15:52.989 INFO:tasks.workunit.client.0.smithi063.stdout:0/287: fsync da/de/d17/d25/d3c/d43/f5b 0 2024-03-22T07:15:52.991 INFO:tasks.workunit.client.0.smithi063.stdout:0/288: rename da/de/d17/d25/d36 to da/de/d17/d25/d3c/d5f 0 2024-03-22T07:15:52.991 INFO:tasks.workunit.client.0.smithi063.stdout:0/289: dread - da/de/d17/d25/d3c/d5f/f52 zero size 2024-03-22T07:15:52.991 INFO:tasks.workunit.client.0.smithi063.stdout:0/290: stat da/de/f18 0 2024-03-22T07:15:52.991 INFO:tasks.workunit.client.0.smithi063.stdout:0/291: rmdir da/de/d17/d25/d3c/d5f/d58 0 2024-03-22T07:15:53.053 INFO:tasks.workunit.client.0.smithi063.stdout:0/292: dwrite da/de/d17/f3a [0,4194304] 0 2024-03-22T07:15:53.053 INFO:tasks.workunit.client.0.smithi063.stdout:0/293: truncate da/de/d17/d25/f27 355954 0 2024-03-22T07:15:53.055 INFO:tasks.workunit.client.0.smithi063.stdout:0/294: rename da/d2e to da/de/d60 0 2024-03-22T07:15:53.057 INFO:tasks.workunit.client.0.smithi063.stdout:0/295: rename da/de/d17/d25/f38 to da/de/d17/d25/d3c/d53/f61 0 2024-03-22T07:15:53.058 INFO:tasks.workunit.client.0.smithi063.stdout:0/296: creat da/de/f62 x:0 0 0 2024-03-22T07:15:53.059 INFO:tasks.workunit.client.0.smithi063.stdout:0/297: getdents da/de/d60 0 2024-03-22T07:15:53.059 INFO:tasks.workunit.client.0.smithi063.stdout:0/298: stat da/de/d17/f2f 0 2024-03-22T07:15:53.059 INFO:tasks.workunit.client.0.smithi063.stdout:0/299: readlink da/de/l2c 0 2024-03-22T07:15:53.091 INFO:tasks.workunit.client.0.smithi063.stdout:1/264: sync 2024-03-22T07:15:53.091 INFO:tasks.workunit.client.0.smithi063.stdout:1/265: chown d4/d14/f47 4923 1 2024-03-22T07:15:53.091 INFO:tasks.workunit.client.0.smithi063.stdout:9/189: sync 2024-03-22T07:15:53.091 INFO:tasks.workunit.client.0.smithi063.stdout:9/190: chown d8/d17 25655 1 2024-03-22T07:15:53.091 INFO:tasks.workunit.client.0.smithi063.stdout:6/258: sync 2024-03-22T07:15:53.091 INFO:tasks.workunit.client.0.smithi063.stdout:1/266: creat d4/df/d13/d39/f5e x:0 0 0 2024-03-22T07:15:53.091 INFO:tasks.workunit.client.0.smithi063.stdout:1/267: chown d4/d14/c31 12807532 1 2024-03-22T07:15:53.091 INFO:tasks.workunit.client.0.smithi063.stdout:1/268: write d4/df/d13/f25 [977510,10093] 0 2024-03-22T07:15:53.091 INFO:tasks.workunit.client.0.smithi063.stdout:1/269: getdents d4/d5a 0 2024-03-22T07:15:53.091 INFO:tasks.workunit.client.0.smithi063.stdout:1/270: dread - d4/f3b zero size 2024-03-22T07:15:53.091 INFO:tasks.workunit.client.0.smithi063.stdout:1/271: chown d4/df/d2c/f54 7939004 1 2024-03-22T07:15:53.092 INFO:tasks.workunit.client.0.smithi063.stdout:6/259: creat d3/d6/d2a/d2e/f5b x:0 0 0 2024-03-22T07:15:53.092 INFO:tasks.workunit.client.0.smithi063.stdout:1/272: rename d4/df/f29 to d4/df/d13/d39/f5f 0 2024-03-22T07:15:53.104 INFO:tasks.workunit.client.0.smithi063.stdout:6/260: dread d3/d1c/f4b [0,4194304] 0 2024-03-22T07:15:53.106 INFO:tasks.workunit.client.0.smithi063.stdout:6/261: dread d3/d15/d29/d4c/f20 [0,4194304] 0 2024-03-22T07:15:53.106 INFO:tasks.workunit.client.0.smithi063.stdout:6/262: write d3/d6/d35/f52 [1122746,4084] 0 2024-03-22T07:15:53.106 INFO:tasks.workunit.client.0.smithi063.stdout:6/263: write d3/f4 [3919899,69005] 0 2024-03-22T07:15:53.106 INFO:tasks.workunit.client.0.smithi063.stdout:6/264: chown d3/f19 381547319 1 2024-03-22T07:15:53.106 INFO:tasks.workunit.client.0.smithi063.stdout:6/265: truncate d3/d6/d2a/d38/f49 546843 0 2024-03-22T07:15:53.117 INFO:tasks.workunit.client.0.smithi063.stdout:6/266: write d3/d6/fc [3174546,51224] 0 2024-03-22T07:15:53.117 INFO:tasks.workunit.client.0.smithi063.stdout:9/191: dwrite d8/f1a [0,4194304] 0 2024-03-22T07:15:53.117 INFO:tasks.workunit.client.0.smithi063.stdout:1/273: dwrite f2 [0,4194304] 0 2024-03-22T07:15:53.118 INFO:tasks.workunit.client.0.smithi063.stdout:6/267: truncate d3/d6/f3e 72283 0 2024-03-22T07:15:53.119 INFO:tasks.workunit.client.0.smithi063.stdout:1/274: dread d4/df/f44 [0,4194304] 0 2024-03-22T07:15:53.120 INFO:tasks.workunit.client.0.smithi063.stdout:9/192: unlink f3 0 2024-03-22T07:15:53.120 INFO:tasks.workunit.client.0.smithi063.stdout:9/193: dread - d8/d15/f1c zero size 2024-03-22T07:15:53.120 INFO:tasks.workunit.client.0.smithi063.stdout:9/194: rmdir d8/d17/d1e 39 2024-03-22T07:15:53.124 INFO:tasks.workunit.client.0.smithi063.stdout:6/268: dread d3/d6/d35/d3a/f55 [0,4194304] 0 2024-03-22T07:15:53.126 INFO:tasks.workunit.client.0.smithi063.stdout:6/269: mknod d3/d6/c5c 0 2024-03-22T07:15:53.126 INFO:tasks.workunit.client.0.smithi063.stdout:6/270: creat d3/d1c/f5d x:0 0 0 2024-03-22T07:15:53.127 INFO:tasks.workunit.client.0.smithi063.stdout:6/271: readlink d3/d6/ld 0 2024-03-22T07:15:53.128 INFO:tasks.workunit.client.0.smithi063.stdout:6/272: write d3/f14 [567446,76019] 0 2024-03-22T07:15:53.128 INFO:tasks.workunit.client.0.smithi063.stdout:6/273: chown d3/d15 716335 1 2024-03-22T07:15:53.129 INFO:tasks.workunit.client.0.smithi063.stdout:9/195: dread f2 [0,4194304] 0 2024-03-22T07:15:53.130 INFO:tasks.workunit.client.0.smithi063.stdout:9/196: mknod d8/d17/c38 0 2024-03-22T07:15:53.133 INFO:tasks.workunit.client.0.smithi063.stdout:6/274: dread d3/d6/d35/d3a/f55 [0,4194304] 0 2024-03-22T07:15:53.134 INFO:tasks.workunit.client.0.smithi063.stdout:6/275: link d3/d15/d16/f26 d3/d6/d2a/f5e 0 2024-03-22T07:15:53.135 INFO:tasks.workunit.client.0.smithi063.stdout:1/275: dwrite d4/df/d13/f3c [0,4194304] 0 2024-03-22T07:15:53.137 INFO:tasks.workunit.client.0.smithi063.stdout:6/276: mkdir d3/d15/d29/d4e/d5f 0 2024-03-22T07:15:53.137 INFO:tasks.workunit.client.0.smithi063.stdout:6/277: write d3/d15/d29/d4c/f20 [2224701,43729] 0 2024-03-22T07:15:53.140 INFO:tasks.workunit.client.0.smithi063.stdout:6/278: truncate d3/d6/d35/d3a/f40 1075367 0 2024-03-22T07:15:53.144 INFO:tasks.workunit.client.0.smithi063.stdout:1/276: write d4/df/d13/d39/f5f [2342036,71909] 0 2024-03-22T07:15:53.147 INFO:tasks.workunit.client.0.smithi063.stdout:1/277: symlink d4/df/d13/l60 0 2024-03-22T07:15:53.147 INFO:tasks.workunit.client.0.smithi063.stdout:1/278: chown d4/df/d13/f25 579114 1 2024-03-22T07:15:53.149 INFO:tasks.workunit.client.0.smithi063.stdout:1/279: unlink d4/df/d13/d2b/f5c 0 2024-03-22T07:15:53.149 INFO:tasks.workunit.client.0.smithi063.stdout:1/280: write d4/d14/f50 [412038,48597] 0 2024-03-22T07:15:53.149 INFO:tasks.workunit.client.0.smithi063.stdout:1/281: write d4/df/d13/d2b/d3a/f42 [4222129,119259] 0 2024-03-22T07:15:53.160 INFO:tasks.workunit.client.0.smithi063.stdout:9/197: dwrite d8/d17/f26 [0,4194304] 0 2024-03-22T07:15:53.160 INFO:tasks.workunit.client.0.smithi063.stdout:9/198: truncate d8/d15/f2a 447159 0 2024-03-22T07:15:53.160 INFO:tasks.workunit.client.0.smithi063.stdout:9/199: chown d8 14063 1 2024-03-22T07:15:53.164 INFO:tasks.workunit.client.0.smithi063.stdout:6/279: dwrite d3/d1c/f4b [0,4194304] 0 2024-03-22T07:15:53.166 INFO:tasks.workunit.client.0.smithi063.stdout:9/200: dread d8/d19/d20/f21 [0,4194304] 0 2024-03-22T07:15:53.169 INFO:tasks.workunit.client.0.smithi063.stdout:9/201: rename d8/d19/d20/d22/d24 to d8/d19/d39 0 2024-03-22T07:15:53.169 INFO:tasks.workunit.client.0.smithi063.stdout:9/202: write f7 [1478960,114551] 0 2024-03-22T07:15:53.170 INFO:tasks.workunit.client.0.smithi063.stdout:6/280: truncate d3/f5 455735 0 2024-03-22T07:15:53.170 INFO:tasks.workunit.client.0.smithi063.stdout:6/281: chown d3/d15/d16/f25 168 1 2024-03-22T07:15:53.171 INFO:tasks.workunit.client.0.smithi063.stdout:1/282: dwrite d4/df/d13/f25 [0,4194304] 0 2024-03-22T07:15:53.171 INFO:tasks.workunit.client.0.smithi063.stdout:6/282: mkdir d3/d15/d16/d60 0 2024-03-22T07:15:53.172 INFO:tasks.workunit.client.0.smithi063.stdout:1/283: mknod d4/d5a/c61 0 2024-03-22T07:15:53.172 INFO:tasks.workunit.client.0.smithi063.stdout:1/284: creat d4/d14/f62 x:0 0 0 2024-03-22T07:15:53.176 INFO:tasks.workunit.client.0.smithi063.stdout:1/285: mkdir d4/df/d13/d39/d63 0 2024-03-22T07:15:53.178 INFO:tasks.workunit.client.0.smithi063.stdout:6/283: dread d3/d6/d2a/f5e [0,4194304] 0 2024-03-22T07:15:53.178 INFO:tasks.workunit.client.0.smithi063.stdout:6/284: write d3/d6/f3e [437438,51667] 0 2024-03-22T07:15:53.178 INFO:tasks.workunit.client.0.smithi063.stdout:6/285: creat d3/d6/d2a/d38/f61 x:0 0 0 2024-03-22T07:15:53.178 INFO:tasks.workunit.client.0.smithi063.stdout:6/286: chown d3/d15/d16/f26 60 1 2024-03-22T07:15:53.180 INFO:tasks.workunit.client.0.smithi063.stdout:6/287: creat d3/d15/f62 x:0 0 0 2024-03-22T07:15:53.180 INFO:tasks.workunit.client.0.smithi063.stdout:6/288: write d3/d15/d29/d4e/f50 [4852798,11956] 0 2024-03-22T07:15:53.180 INFO:tasks.workunit.client.0.smithi063.stdout:6/289: creat d3/d1c/f63 x:0 0 0 2024-03-22T07:15:53.181 INFO:tasks.workunit.client.0.smithi063.stdout:6/290: truncate d3/d6/f2c 4483049 0 2024-03-22T07:15:53.181 INFO:tasks.workunit.client.0.smithi063.stdout:6/291: chown d3/d15/d29/l33 152971211 1 2024-03-22T07:15:53.190 INFO:tasks.workunit.client.0.smithi063.stdout:2/232: sync 2024-03-22T07:15:53.204 INFO:tasks.workunit.client.0.smithi063.stdout:2/233: dread d0/de/d12/d20/f29 [0,4194304] 0 2024-03-22T07:15:53.205 INFO:tasks.workunit.client.0.smithi063.stdout:2/234: link d0/de/d1b/d32/f3b d0/de/d1b/d32/f49 0 2024-03-22T07:15:53.205 INFO:tasks.workunit.client.0.smithi063.stdout:2/235: write d0/de/f31 [553000,33143] 0 2024-03-22T07:15:53.208 INFO:tasks.workunit.client.0.smithi063.stdout:9/203: dwrite d8/fc [0,4194304] 0 2024-03-22T07:15:53.208 INFO:tasks.workunit.client.0.smithi063.stdout:1/286: dwrite d4/f12 [0,4194304] 0 2024-03-22T07:15:53.209 INFO:tasks.workunit.client.0.smithi063.stdout:1/287: write d4/d14/f21 [1287617,108020] 0 2024-03-22T07:15:53.209 INFO:tasks.workunit.client.0.smithi063.stdout:1/288: creat d4/df/f64 x:0 0 0 2024-03-22T07:15:53.209 INFO:tasks.workunit.client.0.smithi063.stdout:1/289: stat d4/df/d13/d2b/c4a 0 2024-03-22T07:15:53.209 INFO:tasks.workunit.client.0.smithi063.stdout:6/292: dwrite d3/d6/fa [0,4194304] 0 2024-03-22T07:15:53.209 INFO:tasks.workunit.client.0.smithi063.stdout:9/204: read d8/f10 [484093,63781] 0 2024-03-22T07:15:53.209 INFO:tasks.workunit.client.0.smithi063.stdout:9/205: write d8/d19/f23 [4344172,115041] 0 2024-03-22T07:15:53.210 INFO:tasks.workunit.client.0.smithi063.stdout:2/236: write d0/fa [97079,65216] 0 2024-03-22T07:15:53.211 INFO:tasks.workunit.client.0.smithi063.stdout:2/237: fdatasync d0/de/f30 0 2024-03-22T07:15:53.211 INFO:tasks.workunit.client.0.smithi063.stdout:1/290: link d4/fb d4/d5a/f65 0 2024-03-22T07:15:53.211 INFO:tasks.workunit.client.0.smithi063.stdout:1/291: write d4/d14/f46 [1386280,32310] 0 2024-03-22T07:15:53.214 INFO:tasks.workunit.client.0.smithi063.stdout:1/292: fdatasync d4/d14/f46 0 2024-03-22T07:15:53.214 INFO:tasks.workunit.client.0.smithi063.stdout:1/293: chown d4/df/d13/d39/d63 1769 1 2024-03-22T07:15:53.219 INFO:tasks.workunit.client.0.smithi063.stdout:1/294: creat d4/df/d13/d39/d63/f66 x:0 0 0 2024-03-22T07:15:53.221 INFO:tasks.workunit.client.0.smithi063.stdout:6/293: dread d3/d6/f2c [0,4194304] 0 2024-03-22T07:15:53.222 INFO:tasks.workunit.client.0.smithi063.stdout:6/294: mkdir d3/d1c/d32/d51/d64 0 2024-03-22T07:15:53.223 INFO:tasks.workunit.client.0.smithi063.stdout:6/295: creat d3/d15/d16/d60/f65 x:0 0 0 2024-03-22T07:15:53.231 INFO:tasks.workunit.client.0.smithi063.stdout:4/216: sync 2024-03-22T07:15:53.231 INFO:tasks.workunit.client.0.smithi063.stdout:4/217: write d1/d5/d43/f2e [526369,60300] 0 2024-03-22T07:15:53.233 INFO:tasks.workunit.client.0.smithi063.stdout:9/206: dwrite d8/d19/d20/f34 [0,4194304] 0 2024-03-22T07:15:53.233 INFO:tasks.workunit.client.0.smithi063.stdout:9/207: creat d8/d19/f3a x:0 0 0 2024-03-22T07:15:53.233 INFO:tasks.workunit.client.0.smithi063.stdout:9/208: dread - d8/d15/f1c zero size 2024-03-22T07:15:53.235 INFO:tasks.workunit.client.0.smithi063.stdout:1/295: dread d4/df/d13/f25 [0,4194304] 0 2024-03-22T07:15:53.235 INFO:tasks.workunit.client.0.smithi063.stdout:2/238: dwrite d0/f13 [0,4194304] 0 2024-03-22T07:15:53.238 INFO:tasks.workunit.client.0.smithi063.stdout:2/239: dread d0/de/d12/d20/f2c [0,4194304] 0 2024-03-22T07:15:53.242 INFO:tasks.workunit.client.0.smithi063.stdout:6/296: dwrite d3/f5 [0,4194304] 0 2024-03-22T07:15:53.242 INFO:tasks.workunit.client.0.smithi063.stdout:6/297: fsync d3/d6/d2a/f5e 0 2024-03-22T07:15:53.243 INFO:tasks.workunit.client.0.smithi063.stdout:6/298: write d3/d6/d35/f52 [1092525,107609] 0 2024-03-22T07:15:53.243 INFO:tasks.workunit.client.0.smithi063.stdout:6/299: fsync d3/d15/d16/f59 0 2024-03-22T07:15:53.243 INFO:tasks.workunit.client.0.smithi063.stdout:2/240: rename d0/de/d12/d39/l3e to d0/de/d12/l4a 0 2024-03-22T07:15:53.243 INFO:tasks.workunit.client.0.smithi063.stdout:2/241: write d0/de/d35/d36/f3a [1399842,7883] 0 2024-03-22T07:15:53.244 INFO:tasks.workunit.client.0.smithi063.stdout:6/300: dread d3/d6/d2a/d38/f49 [0,4194304] 0 2024-03-22T07:15:53.244 INFO:tasks.workunit.client.0.smithi063.stdout:6/301: creat d3/d15/f66 x:0 0 0 2024-03-22T07:15:53.244 INFO:tasks.workunit.client.0.smithi063.stdout:6/302: read d3/f14 [138447,6147] 0 2024-03-22T07:15:53.245 INFO:tasks.workunit.client.0.smithi063.stdout:6/303: creat d3/d6/d35/f67 x:0 0 0 2024-03-22T07:15:53.245 INFO:tasks.workunit.client.0.smithi063.stdout:6/304: write d3/d6/d2a/d38/f49 [11583,92050] 0 2024-03-22T07:15:53.250 INFO:tasks.workunit.client.0.smithi063.stdout:0/300: sync 2024-03-22T07:15:53.251 INFO:tasks.workunit.client.0.smithi063.stdout:8/289: sync 2024-03-22T07:15:53.251 INFO:tasks.workunit.client.0.smithi063.stdout:7/220: sync 2024-03-22T07:15:53.251 INFO:tasks.workunit.client.0.smithi063.stdout:5/228: sync 2024-03-22T07:15:53.252 INFO:tasks.workunit.client.0.smithi063.stdout:7/221: truncate d5/d16/f1e 2067157 0 2024-03-22T07:15:53.252 INFO:tasks.workunit.client.0.smithi063.stdout:5/229: creat d8/f44 x:0 0 0 2024-03-22T07:15:53.252 INFO:tasks.workunit.client.0.smithi063.stdout:5/230: stat d8/d13/d1e/f38 0 2024-03-22T07:15:53.252 INFO:tasks.workunit.client.0.smithi063.stdout:5/231: truncate d8/d13/d1e/f2f 587965 0 2024-03-22T07:15:53.258 INFO:tasks.workunit.client.0.smithi063.stdout:1/296: dwrite d4/df/d13/f25 [0,4194304] 0 2024-03-22T07:15:53.259 INFO:tasks.workunit.client.0.smithi063.stdout:9/209: dwrite d8/d19/f3a [0,4194304] 0 2024-03-22T07:15:53.259 INFO:tasks.workunit.client.0.smithi063.stdout:3/203: sync 2024-03-22T07:15:53.260 INFO:tasks.workunit.client.0.smithi063.stdout:1/297: mknod d4/df/c67 0 2024-03-22T07:15:53.261 INFO:tasks.workunit.client.0.smithi063.stdout:1/298: fdatasync d4/d14/d57/f58 0 2024-03-22T07:15:53.263 INFO:tasks.workunit.client.0.smithi063.stdout:6/305: write d3/f19 [1934106,123287] 0 2024-03-22T07:15:53.263 INFO:tasks.workunit.client.0.smithi063.stdout:6/306: write d3/d6/d35/d3a/f40 [2145727,64784] 0 2024-03-22T07:15:53.267 INFO:tasks.workunit.client.0.smithi063.stdout:5/232: dread d8/d13/d24/f33 [0,4194304] 0 2024-03-22T07:15:53.267 INFO:tasks.workunit.client.0.smithi063.stdout:6/307: fdatasync d3/d6/d35/d3a/f40 0 2024-03-22T07:15:53.272 INFO:tasks.workunit.client.0.smithi063.stdout:9/210: write d8/d19/d20/f34 [583697,117685] 0 2024-03-22T07:15:53.273 INFO:tasks.workunit.client.0.smithi063.stdout:9/211: fsync d8/f13 0 2024-03-22T07:15:53.273 INFO:tasks.workunit.client.0.smithi063.stdout:5/233: mkdir d8/d13/d28/d45 0 2024-03-22T07:15:53.274 INFO:tasks.workunit.client.0.smithi063.stdout:9/212: symlink d8/d17/d1e/l3b 0 2024-03-22T07:15:53.276 INFO:tasks.workunit.client.0.smithi063.stdout:5/234: rmdir d8/d13/d28/d45 0 2024-03-22T07:15:53.276 INFO:tasks.workunit.client.0.smithi063.stdout:9/213: truncate d8/d19/d20/d22/f2d 269618 0 2024-03-22T07:15:53.277 INFO:tasks.workunit.client.0.smithi063.stdout:5/235: creat d8/d13/d1e/d31/f46 x:0 0 0 2024-03-22T07:15:53.278 INFO:tasks.workunit.client.0.smithi063.stdout:9/214: symlink d8/d19/d2b/l3c 0 2024-03-22T07:15:53.280 INFO:tasks.workunit.client.0.smithi063.stdout:5/236: rename d8/d13/c17 to d8/d13/d28/c47 0 2024-03-22T07:15:53.281 INFO:tasks.workunit.client.0.smithi063.stdout:5/237: creat d8/d13/f48 x:0 0 0 2024-03-22T07:15:53.283 INFO:tasks.workunit.client.0.smithi063.stdout:5/238: creat d8/d13/d1e/d3c/f49 x:0 0 0 2024-03-22T07:15:53.283 INFO:tasks.workunit.client.0.smithi063.stdout:5/239: truncate d8/f10 4916143 0 2024-03-22T07:15:53.283 INFO:tasks.workunit.client.0.smithi063.stdout:5/240: write d8/f12 [5409948,77829] 0 2024-03-22T07:15:53.283 INFO:tasks.workunit.client.0.smithi063.stdout:5/241: chown d8/d13/c1a 3 1 2024-03-22T07:15:53.283 INFO:tasks.workunit.client.0.smithi063.stdout:5/242: write d8/f1c [122776,109222] 0 2024-03-22T07:15:53.283 INFO:tasks.workunit.client.0.smithi063.stdout:5/243: chown d8/d13/l36 68 1 2024-03-22T07:15:53.283 INFO:tasks.workunit.client.0.smithi063.stdout:5/244: write d8/f16 [672301,96462] 0 2024-03-22T07:15:53.288 INFO:tasks.workunit.client.0.smithi063.stdout:5/245: write d8/f12 [2576618,42756] 0 2024-03-22T07:15:53.288 INFO:tasks.workunit.client.0.smithi063.stdout:5/246: write d8/f2e [153069,1842] 0 2024-03-22T07:15:53.291 INFO:tasks.workunit.client.0.smithi063.stdout:3/204: dwrite da/f35 [0,4194304] 0 2024-03-22T07:15:53.291 INFO:tasks.workunit.client.0.smithi063.stdout:3/205: write da/f2f [1303361,55684] 0 2024-03-22T07:15:53.291 INFO:tasks.workunit.client.0.smithi063.stdout:3/206: chown da/f2f 2108471957 1 2024-03-22T07:15:53.292 INFO:tasks.workunit.client.0.smithi063.stdout:3/207: chown da/d16/d26/c37 109616728 1 2024-03-22T07:15:53.292 INFO:tasks.workunit.client.0.smithi063.stdout:0/301: dwrite da/de/f62 [0,4194304] 0 2024-03-22T07:15:53.292 INFO:tasks.workunit.client.0.smithi063.stdout:1/299: dwrite d4/d14/f47 [0,4194304] 0 2024-03-22T07:15:53.292 INFO:tasks.workunit.client.0.smithi063.stdout:6/308: dwrite d3/d15/f41 [0,4194304] 0 2024-03-22T07:15:53.299 INFO:tasks.workunit.client.0.smithi063.stdout:3/208: rename da/d16/d26/f34 to da/d16/d3f/f44 0 2024-03-22T07:15:53.299 INFO:tasks.workunit.client.0.smithi063.stdout:3/209: readlink da/l41 0 2024-03-22T07:15:53.299 INFO:tasks.workunit.client.0.smithi063.stdout:3/210: truncate da/f21 1232714 0 2024-03-22T07:15:53.299 INFO:tasks.workunit.client.0.smithi063.stdout:3/211: write da/d16/f32 [726833,63928] 0 2024-03-22T07:15:53.299 INFO:tasks.workunit.client.0.smithi063.stdout:3/212: fdatasync da/f3a 0 2024-03-22T07:15:53.300 INFO:tasks.workunit.client.0.smithi063.stdout:0/302: creat da/d49/f63 x:0 0 0 2024-03-22T07:15:53.302 INFO:tasks.workunit.client.0.smithi063.stdout:1/300: symlink d4/df/l68 0 2024-03-22T07:15:53.302 INFO:tasks.workunit.client.0.smithi063.stdout:1/301: read - d4/f59 zero size 2024-03-22T07:15:53.306 INFO:tasks.workunit.client.0.smithi063.stdout:3/213: symlink da/d16/l45 0 2024-03-22T07:15:53.306 INFO:tasks.workunit.client.0.smithi063.stdout:5/247: dwrite d8/d13/d24/f3b [0,4194304] 0 2024-03-22T07:15:53.310 INFO:tasks.workunit.client.0.smithi063.stdout:0/303: mkdir da/de/d17/d25/d3c/d64 0 2024-03-22T07:15:53.325 INFO:tasks.workunit.client.0.smithi063.stdout:1/302: dwrite d4/df/d2c/f36 [0,4194304] 0 2024-03-22T07:15:53.333 INFO:tasks.workunit.client.0.smithi063.stdout:3/214: dwrite da/d16/d26/f39 [0,4194304] 0 2024-03-22T07:15:53.333 INFO:tasks.workunit.client.0.smithi063.stdout:3/215: stat da/c2b 0 2024-03-22T07:15:53.333 INFO:tasks.workunit.client.0.smithi063.stdout:5/248: dwrite d8/f16 [0,4194304] 0 2024-03-22T07:15:53.334 INFO:tasks.workunit.client.0.smithi063.stdout:3/216: dread da/f1c [0,4194304] 0 2024-03-22T07:15:53.338 INFO:tasks.workunit.client.0.smithi063.stdout:1/303: dread d4/df/d13/f34 [0,4194304] 0 2024-03-22T07:15:53.338 INFO:tasks.workunit.client.0.smithi063.stdout:1/304: read - d4/f59 zero size 2024-03-22T07:15:53.340 INFO:tasks.workunit.client.0.smithi063.stdout:5/249: rmdir d8/d13/d1e/d3c 39 2024-03-22T07:15:53.342 INFO:tasks.workunit.client.0.smithi063.stdout:0/304: rmdir da/de 39 2024-03-22T07:15:53.343 INFO:tasks.workunit.client.0.smithi063.stdout:1/305: mknod d4/d14/c69 0 2024-03-22T07:15:53.347 INFO:tasks.workunit.client.0.smithi063.stdout:5/250: symlink d8/d13/l4a 0 2024-03-22T07:15:53.347 INFO:tasks.workunit.client.0.smithi063.stdout:5/251: write d8/d13/f18 [3913442,31111] 0 2024-03-22T07:15:53.347 INFO:tasks.workunit.client.0.smithi063.stdout:5/252: write d8/d13/d28/f43 [727418,128915] 0 2024-03-22T07:15:53.347 INFO:tasks.workunit.client.0.smithi063.stdout:5/253: creat d8/d13/f4b x:0 0 0 2024-03-22T07:15:53.349 INFO:tasks.workunit.client.0.smithi063.stdout:0/305: creat da/de/d17/d25/f65 x:0 0 0 2024-03-22T07:15:53.349 INFO:tasks.workunit.client.0.smithi063.stdout:0/306: dread - da/de/d17/d25/d3c/d5f/f52 zero size 2024-03-22T07:15:53.350 INFO:tasks.workunit.client.0.smithi063.stdout:1/306: mknod d4/df/c6a 0 2024-03-22T07:15:53.354 INFO:tasks.workunit.client.0.smithi063.stdout:5/254: creat d8/d13/d1e/d3c/f4c x:0 0 0 2024-03-22T07:15:53.356 INFO:tasks.workunit.client.0.smithi063.stdout:0/307: creat da/de/d17/d25/d3c/f66 x:0 0 0 2024-03-22T07:15:53.356 INFO:tasks.workunit.client.0.smithi063.stdout:0/308: chown da/de/d17/f21 476957992 1 2024-03-22T07:15:53.356 INFO:tasks.workunit.client.0.smithi063.stdout:0/309: stat da/de/d60/c3d 0 2024-03-22T07:15:53.356 INFO:tasks.workunit.client.0.smithi063.stdout:0/310: getdents da/de/d60 0 2024-03-22T07:15:53.358 INFO:tasks.workunit.client.0.smithi063.stdout:5/255: mknod d8/d13/d1e/c4d 0 2024-03-22T07:15:53.358 INFO:tasks.workunit.client.0.smithi063.stdout:5/256: creat d8/d13/d1e/d31/f4e x:0 0 0 2024-03-22T07:15:53.360 INFO:tasks.workunit.client.0.smithi063.stdout:0/311: unlink da/de/d17/d25/c50 0 2024-03-22T07:15:53.365 INFO:tasks.workunit.client.0.smithi063.stdout:5/257: mknod d8/d13/d1e/d3c/c4f 0 2024-03-22T07:15:53.365 INFO:tasks.workunit.client.0.smithi063.stdout:5/258: dread - d8/f3d zero size 2024-03-22T07:15:53.365 INFO:tasks.workunit.client.0.smithi063.stdout:5/259: chown d8/f1c 336417 1 2024-03-22T07:15:53.365 INFO:tasks.workunit.client.0.smithi063.stdout:5/260: dread - d8/d13/d1e/d31/f4e zero size 2024-03-22T07:15:53.370 INFO:tasks.workunit.client.0.smithi063.stdout:1/307: dwrite d4/df/d13/d2b/f49 [0,4194304] 0 2024-03-22T07:15:53.375 INFO:tasks.workunit.client.0.smithi063.stdout:5/261: dread d8/fe [0,4194304] 0 2024-03-22T07:15:53.381 INFO:tasks.workunit.client.0.smithi063.stdout:3/217: truncate da/d16/f32 593551 0 2024-03-22T07:15:53.383 INFO:tasks.workunit.client.0.smithi063.stdout:5/262: creat d8/d13/d24/f50 x:0 0 0 2024-03-22T07:15:53.383 INFO:tasks.workunit.client.0.smithi063.stdout:5/263: dread - d8/f3d zero size 2024-03-22T07:15:53.384 INFO:tasks.workunit.client.0.smithi063.stdout:3/218: creat da/f46 x:0 0 0 2024-03-22T07:15:53.386 INFO:tasks.workunit.client.0.smithi063.stdout:5/264: symlink d8/d13/d1e/d31/l51 0 2024-03-22T07:15:53.386 INFO:tasks.workunit.client.0.smithi063.stdout:5/265: fdatasync d8/d13/d1e/f2f 0 2024-03-22T07:15:53.387 INFO:tasks.workunit.client.0.smithi063.stdout:3/219: link da/d16/l31 da/d16/d3c/l47 0 2024-03-22T07:15:53.389 INFO:tasks.workunit.client.0.smithi063.stdout:3/220: creat da/d16/d26/f48 x:0 0 0 2024-03-22T07:15:53.390 INFO:tasks.workunit.client.0.smithi063.stdout:3/221: mknod da/d16/d26/c49 0 2024-03-22T07:15:53.392 INFO:tasks.workunit.client.0.smithi063.stdout:0/312: dwrite da/de/d17/d25/d3c/d53/f61 [0,4194304] 0 2024-03-22T07:15:53.392 INFO:tasks.workunit.client.0.smithi063.stdout:3/222: unlink da/d16/l33 0 2024-03-22T07:15:53.395 INFO:tasks.workunit.client.0.smithi063.stdout:0/313: mkdir da/de/d67 0 2024-03-22T07:15:53.395 INFO:tasks.workunit.client.0.smithi063.stdout:0/314: stat f5 0 2024-03-22T07:15:53.396 INFO:tasks.workunit.client.0.smithi063.stdout:3/223: chown da/l14 870144 1 2024-03-22T07:15:53.396 INFO:tasks.workunit.client.0.smithi063.stdout:3/224: stat da/d16/d26 0 2024-03-22T07:15:53.397 INFO:tasks.workunit.client.0.smithi063.stdout:3/225: symlink da/d16/l4a 0 2024-03-22T07:15:53.399 INFO:tasks.workunit.client.0.smithi063.stdout:3/226: mkdir da/d16/d3c/d4b 0 2024-03-22T07:15:53.400 INFO:tasks.workunit.client.0.smithi063.stdout:3/227: creat da/d16/d3f/f4c x:0 0 0 2024-03-22T07:15:53.400 INFO:tasks.workunit.client.0.smithi063.stdout:3/228: fsync da/d16/f25 0 2024-03-22T07:15:53.402 INFO:tasks.workunit.client.0.smithi063.stdout:3/229: mknod da/d16/c4d 0 2024-03-22T07:15:53.403 INFO:tasks.workunit.client.0.smithi063.stdout:3/230: rename da/d16/d26/c49 to da/d16/d3c/d4b/c4e 0 2024-03-22T07:15:53.405 INFO:tasks.workunit.client.0.smithi063.stdout:3/231: link da/c2b da/d16/d3c/d4b/c4f 0 2024-03-22T07:15:53.405 INFO:tasks.workunit.client.0.smithi063.stdout:3/232: chown da/d16/l31 494 1 2024-03-22T07:15:53.405 INFO:tasks.workunit.client.0.smithi063.stdout:3/233: creat da/d16/d3f/f50 x:0 0 0 2024-03-22T07:15:53.422 INFO:tasks.workunit.client.0.smithi063.stdout:3/234: dwrite da/d16/f38 [0,4194304] 0 2024-03-22T07:15:53.489 INFO:tasks.workunit.client.0.smithi063.stdout:1/308: dwrite d4/df/d13/f3c [0,4194304] 0 2024-03-22T07:15:53.490 INFO:tasks.workunit.client.0.smithi063.stdout:1/309: write d4/f9 [319925,80639] 0 2024-03-22T07:15:53.514 INFO:tasks.workunit.client.0.smithi063.stdout:1/310: rename d4/df/d13/d2b/f49 to d4/df/d13/d39/f6b 0 2024-03-22T07:15:53.515 INFO:tasks.workunit.client.0.smithi063.stdout:1/311: chown d4 88095 1 2024-03-22T07:15:53.519 INFO:tasks.workunit.client.0.smithi063.stdout:3/235: symlink da/l51 0 2024-03-22T07:15:53.519 INFO:tasks.workunit.client.0.smithi063.stdout:3/236: dread da/d16/f25 [0,4194304] 0 2024-03-22T07:15:53.520 INFO:tasks.workunit.client.0.smithi063.stdout:1/312: mkdir d4/df/d13/d2b/d6c 0 2024-03-22T07:15:53.523 INFO:tasks.workunit.client.0.smithi063.stdout:1/313: truncate d4/df/f11 1307765 0 2024-03-22T07:15:53.523 INFO:tasks.workunit.client.0.smithi063.stdout:1/314: chown d4/df/d13/d39/d63 158 1 2024-03-22T07:15:53.526 INFO:tasks.workunit.client.0.smithi063.stdout:1/315: mkdir d4/df/d13/d39/d6d 0 2024-03-22T07:15:53.530 INFO:tasks.workunit.client.0.smithi063.stdout:1/316: getdents d4/df/d13/d39/d63 0 2024-03-22T07:15:53.530 INFO:tasks.workunit.client.0.smithi063.stdout:1/317: fdatasync d4/df/f15 0 2024-03-22T07:15:53.532 INFO:tasks.workunit.client.0.smithi063.stdout:1/318: link d4/d14/d57/f5d d4/df/d13/d2b/d3a/f6e 0 2024-03-22T07:15:53.547 INFO:tasks.workunit.client.0.smithi063.stdout:1/319: dwrite f2 [0,4194304] 0 2024-03-22T07:15:53.560 INFO:tasks.workunit.client.0.smithi063.stdout:1/320: dwrite d4/d14/f21 [0,4194304] 0 2024-03-22T07:15:53.560 INFO:tasks.workunit.client.0.smithi063.stdout:1/321: write d4/d14/d57/f5d [81457,291] 0 2024-03-22T07:15:53.560 INFO:tasks.workunit.client.0.smithi063.stdout:1/322: write d4/d14/d57/f58 [232030,81566] 0 2024-03-22T07:15:53.569 INFO:tasks.workunit.client.0.smithi063.stdout:1/323: mkdir d4/df/d13/d39/d6f 0 2024-03-22T07:15:53.569 INFO:tasks.workunit.client.0.smithi063.stdout:1/324: creat d4/df/d13/d2b/d3a/f70 x:0 0 0 2024-03-22T07:15:53.590 INFO:tasks.workunit.client.0.smithi063.stdout:1/325: dwrite d4/fb [0,4194304] 0 2024-03-22T07:15:53.591 INFO:tasks.workunit.client.0.smithi063.stdout:1/326: stat d4/d5a/c61 0 2024-03-22T07:15:53.591 INFO:tasks.workunit.client.0.smithi063.stdout:1/327: write d4/df/d2c/f4b [661889,37968] 0 2024-03-22T07:15:53.591 INFO:tasks.workunit.client.0.smithi063.stdout:8/290: sync 2024-03-22T07:15:53.591 INFO:tasks.workunit.client.0.smithi063.stdout:4/218: sync 2024-03-22T07:15:53.591 INFO:tasks.workunit.client.0.smithi063.stdout:2/242: sync 2024-03-22T07:15:53.591 INFO:tasks.workunit.client.0.smithi063.stdout:4/219: dread - d1/d5/f2f zero size 2024-03-22T07:15:53.591 INFO:tasks.workunit.client.0.smithi063.stdout:2/243: read d0/de/d12/d20/f2c [69848,121587] 0 2024-03-22T07:15:53.591 INFO:tasks.workunit.client.0.smithi063.stdout:7/222: sync 2024-03-22T07:15:53.594 INFO:tasks.workunit.client.0.smithi063.stdout:1/328: dread d4/d14/d57/f5d [0,4194304] 0 2024-03-22T07:15:53.597 INFO:tasks.workunit.client.0.smithi063.stdout:8/291: mkdir d0/d18/d45/d54 0 2024-03-22T07:15:53.599 INFO:tasks.workunit.client.0.smithi063.stdout:4/220: link d1/d5/d43/f17 d1/d5/d38/f44 0 2024-03-22T07:15:53.600 INFO:tasks.workunit.client.0.smithi063.stdout:3/237: getdents da 0 2024-03-22T07:15:53.602 INFO:tasks.workunit.client.0.smithi063.stdout:3/238: dread da/d16/f32 [0,4194304] 0 2024-03-22T07:15:53.602 INFO:tasks.workunit.client.0.smithi063.stdout:2/244: link d0/de/d1b/d32/f43 d0/de/d35/d36/f4b 0 2024-03-22T07:15:53.606 INFO:tasks.workunit.client.0.smithi063.stdout:1/329: rmdir d4/df/d13/d39/d6d 0 2024-03-22T07:15:53.609 INFO:tasks.workunit.client.0.smithi063.stdout:8/292: rmdir d0/d18/d3f 0 2024-03-22T07:15:53.610 INFO:tasks.workunit.client.0.smithi063.stdout:8/293: stat d0/d1f/f40 0 2024-03-22T07:15:53.613 INFO:tasks.workunit.client.0.smithi063.stdout:3/239: creat da/f52 x:0 0 0 2024-03-22T07:15:53.613 INFO:tasks.workunit.client.0.smithi063.stdout:3/240: readlink da/d16/d26/l2c 0 2024-03-22T07:15:53.613 INFO:tasks.workunit.client.0.smithi063.stdout:3/241: fdatasync da/d16/d26/f39 0 2024-03-22T07:15:53.613 INFO:tasks.workunit.client.0.smithi063.stdout:3/242: readlink da/l15 0 2024-03-22T07:15:53.613 INFO:tasks.workunit.client.0.smithi063.stdout:3/243: chown da/c1d 40781748 1 2024-03-22T07:15:53.614 INFO:tasks.workunit.client.0.smithi063.stdout:2/245: rename d0/de/d35/f42 to d0/de/f4c 0 2024-03-22T07:15:53.615 INFO:tasks.workunit.client.0.smithi063.stdout:9/215: sync 2024-03-22T07:15:53.618 INFO:tasks.workunit.client.0.smithi063.stdout:1/330: unlink d4/df/d13/d39/l56 0 2024-03-22T07:15:53.618 INFO:tasks.workunit.client.0.smithi063.stdout:1/331: readlink d4/df/d13/d2b/l2e 0 2024-03-22T07:15:53.620 INFO:tasks.workunit.client.0.smithi063.stdout:8/294: symlink d0/d18/l55 0 2024-03-22T07:15:53.620 INFO:tasks.workunit.client.0.smithi063.stdout:8/295: write d0/d1b/f20 [5259428,128169] 0 2024-03-22T07:15:53.620 INFO:tasks.workunit.client.0.smithi063.stdout:8/296: fsync d0/d26/d2d/f4d 0 2024-03-22T07:15:53.622 INFO:tasks.workunit.client.0.smithi063.stdout:3/244: mknod da/c53 0 2024-03-22T07:15:53.622 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:53 smithi063 ceph-mon[29200]: pgmap v249: 65 pgs: 65 active+clean; 399 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 24 MiB/s rd, 31 MiB/s wr, 856 op/s 2024-03-22T07:15:53.624 INFO:tasks.workunit.client.0.smithi063.stdout:2/246: symlink d0/de/d35/l4d 0 2024-03-22T07:15:53.625 INFO:tasks.workunit.client.0.smithi063.stdout:2/247: creat d0/de/f4e x:0 0 0 2024-03-22T07:15:53.627 INFO:tasks.workunit.client.0.smithi063.stdout:9/216: truncate f2 659989 0 2024-03-22T07:15:53.629 INFO:tasks.workunit.client.0.smithi063.stdout:1/332: mkdir d4/d71 0 2024-03-22T07:15:53.635 INFO:tasks.workunit.client.0.smithi063.stdout:8/297: dwrite d0/d1b/d28/f43 [0,4194304] 0 2024-03-22T07:15:53.638 INFO:tasks.workunit.client.0.smithi063.stdout:2/248: symlink d0/de/d35/d36/l4f 0 2024-03-22T07:15:53.651 INFO:tasks.workunit.client.0.smithi063.stdout:6/309: sync 2024-03-22T07:15:53.651 INFO:tasks.workunit.client.0.smithi063.stdout:5/266: sync 2024-03-22T07:15:53.651 INFO:tasks.workunit.client.0.smithi063.stdout:1/333: dwrite d4/df/d2c/f4b [0,4194304] 0 2024-03-22T07:15:53.653 INFO:tasks.workunit.client.0.smithi063.stdout:2/249: link d0/de/d12/d20/f2a d0/de/d1b/f50 0 2024-03-22T07:15:53.654 INFO:tasks.workunit.client.0.smithi063.stdout:6/310: write d3/f9 [7051953,122570] 0 2024-03-22T07:15:53.654 INFO:tasks.workunit.client.0.smithi063.stdout:6/311: dread - d3/d15/f62 zero size 2024-03-22T07:15:53.654 INFO:tasks.workunit.client.0.smithi063.stdout:8/298: dwrite d0/d1f/f53 [0,4194304] 0 2024-03-22T07:15:53.658 INFO:tasks.workunit.client.0.smithi063.stdout:5/267: rename d8/f3d to d8/d13/d1e/d31/f52 0 2024-03-22T07:15:53.663 INFO:tasks.workunit.client.0.smithi063.stdout:8/299: mkdir d0/d26/d2d/d56 0 2024-03-22T07:15:53.663 INFO:tasks.workunit.client.0.smithi063.stdout:8/300: write d0/f22 [3015788,6494] 0 2024-03-22T07:15:53.667 INFO:tasks.workunit.client.0.smithi063.stdout:8/301: unlink d0/d26/l2b 0 2024-03-22T07:15:53.670 INFO:tasks.workunit.client.0.smithi063.stdout:8/302: mknod d0/d26/d2d/d3d/c57 0 2024-03-22T07:15:53.677 INFO:tasks.workunit.client.0.smithi063.stdout:0/315: sync 2024-03-22T07:15:53.679 INFO:tasks.workunit.client.0.smithi063.stdout:0/316: link da/de/d17/f3a da/de/d17/d25/f68 0 2024-03-22T07:15:53.680 INFO:tasks.workunit.client.0.smithi063.stdout:0/317: creat da/de/d60/f69 x:0 0 0 2024-03-22T07:15:53.681 INFO:tasks.workunit.client.0.smithi063.stdout:0/318: creat da/de/d17/d25/f6a x:0 0 0 2024-03-22T07:15:53.681 INFO:tasks.workunit.client.0.smithi063.stdout:0/319: dread - da/de/d17/f1a zero size 2024-03-22T07:15:53.693 INFO:tasks.workunit.client.0.smithi063.stdout:8/303: dwrite d0/fc [0,4194304] 0 2024-03-22T07:15:53.693 INFO:tasks.workunit.client.0.smithi063.stdout:8/304: dread - d0/d26/d2d/f4d zero size 2024-03-22T07:15:53.693 INFO:tasks.workunit.client.0.smithi063.stdout:3/245: rmdir da 39 2024-03-22T07:15:53.697 INFO:tasks.workunit.client.0.smithi063.stdout:3/246: rename da/d16/l40 to da/d16/l54 0 2024-03-22T07:15:53.697 INFO:tasks.workunit.client.0.smithi063.stdout:3/247: write da/fb [1451953,86979] 0 2024-03-22T07:15:53.697 INFO:tasks.workunit.client.0.smithi063.stdout:3/248: readlink da/d16/l22 0 2024-03-22T07:15:53.697 INFO:tasks.workunit.client.0.smithi063.stdout:3/249: dread - da/d16/d3f/f4c zero size 2024-03-22T07:15:53.697 INFO:tasks.workunit.client.0.smithi063.stdout:3/250: truncate da/d16/d26/f2a 2959606 0 2024-03-22T07:15:53.697 INFO:tasks.workunit.client.0.smithi063.stdout:3/251: chown da/d16/l1f 3 1 2024-03-22T07:15:53.698 INFO:tasks.workunit.client.0.smithi063.stdout:8/305: dread d0/d1b/f1e [8388608,4194304] 0 2024-03-22T07:15:53.698 INFO:tasks.workunit.client.0.smithi063.stdout:9/217: link f2 d8/d19/f3d 0 2024-03-22T07:15:53.700 INFO:tasks.workunit.client.0.smithi063.stdout:8/306: rename d0/d24/f3a to d0/d18/d45/f58 0 2024-03-22T07:15:53.701 INFO:tasks.workunit.client.0.smithi063.stdout:8/307: stat d0/ca 0 2024-03-22T07:15:53.704 INFO:tasks.workunit.client.0.smithi063.stdout:3/252: dread da/d16/d26/f29 [0,4194304] 0 2024-03-22T07:15:53.705 INFO:tasks.workunit.client.0.smithi063.stdout:3/253: link da/l51 da/d16/d26/l55 0 2024-03-22T07:15:53.706 INFO:tasks.workunit.client.0.smithi063.stdout:3/254: creat da/d16/d3f/f56 x:0 0 0 2024-03-22T07:15:53.707 INFO:tasks.workunit.client.0.smithi063.stdout:3/255: creat da/f57 x:0 0 0 2024-03-22T07:15:53.716 INFO:tasks.workunit.client.0.smithi063.stdout:3/256: dread da/d16/d26/f39 [0,4194304] 0 2024-03-22T07:15:53.717 INFO:tasks.workunit.client.0.smithi063.stdout:3/257: getdents da/d16/d3c/d4b 0 2024-03-22T07:15:53.725 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:53 smithi069 ceph-mon[36441]: pgmap v249: 65 pgs: 65 active+clean; 399 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 24 MiB/s rd, 31 MiB/s wr, 856 op/s 2024-03-22T07:15:53.835 INFO:tasks.workunit.client.0.smithi063.stdout:2/250: dwrite d0/de/d12/d20/f47 [0,4194304] 0 2024-03-22T07:15:53.835 INFO:tasks.workunit.client.0.smithi063.stdout:2/251: chown d0/de/d12/d39/l3d 14712651 1 2024-03-22T07:15:53.836 INFO:tasks.workunit.client.0.smithi063.stdout:5/268: getdents d8/d13/d1e/d31 0 2024-03-22T07:15:53.836 INFO:tasks.workunit.client.0.smithi063.stdout:5/269: write d8/f2e [357682,86017] 0 2024-03-22T07:15:53.844 INFO:tasks.workunit.client.0.smithi063.stdout:2/252: write d0/de/d12/f1f [3301130,46271] 0 2024-03-22T07:15:53.846 INFO:tasks.workunit.client.0.smithi063.stdout:8/308: rmdir d0/d18/d45 39 2024-03-22T07:15:53.848 INFO:tasks.workunit.client.0.smithi063.stdout:8/309: mkdir d0/d26/d2d/d59 0 2024-03-22T07:15:53.849 INFO:tasks.workunit.client.0.smithi063.stdout:2/253: write d0/de/d1b/d32/f43 [360620,103635] 0 2024-03-22T07:15:53.850 INFO:tasks.workunit.client.0.smithi063.stdout:6/312: dwrite d3/d15/d29/d4e/f50 [4194304,4194304] 0 2024-03-22T07:15:53.850 INFO:tasks.workunit.client.0.smithi063.stdout:6/313: read d3/d6/fc [1814593,86934] 0 2024-03-22T07:15:53.850 INFO:tasks.workunit.client.0.smithi063.stdout:8/310: rmdir d0/d18/d45/d54 0 2024-03-22T07:15:53.851 INFO:tasks.workunit.client.0.smithi063.stdout:3/258: dwrite da/f52 [0,4194304] 0 2024-03-22T07:15:53.852 INFO:tasks.workunit.client.0.smithi063.stdout:0/320: dwrite da/de/d17/f3a [0,4194304] 0 2024-03-22T07:15:53.852 INFO:tasks.workunit.client.0.smithi063.stdout:2/254: creat d0/f51 x:0 0 0 2024-03-22T07:15:53.853 INFO:tasks.workunit.client.0.smithi063.stdout:5/270: dwrite d8/d13/d1e/d31/f52 [0,4194304] 0 2024-03-22T07:15:53.853 INFO:tasks.workunit.client.0.smithi063.stdout:5/271: chown d8/d13/d28/l40 547378 1 2024-03-22T07:15:53.854 INFO:tasks.workunit.client.0.smithi063.stdout:6/314: rename d3/f5 to d3/d15/f68 0 2024-03-22T07:15:53.859 INFO:tasks.workunit.client.1.smithi069.stdout:3/41: sync 2024-03-22T07:15:53.859 INFO:tasks.workunit.client.1.smithi069.stdout:0/13: sync 2024-03-22T07:15:53.859 INFO:tasks.workunit.client.1.smithi069.stdout:7/16: sync 2024-03-22T07:15:53.859 INFO:tasks.workunit.client.1.smithi069.stdout:0/14: fsync - no filename 2024-03-22T07:15:53.859 INFO:tasks.workunit.client.1.smithi069.stdout:0/15: dwrite - no filename 2024-03-22T07:15:53.859 INFO:tasks.workunit.client.1.smithi069.stdout:4/20: sync 2024-03-22T07:15:53.859 INFO:tasks.workunit.client.1.smithi069.stdout:6/23: sync 2024-03-22T07:15:53.859 INFO:tasks.workunit.client.1.smithi069.stdout:7/17: stat f0 0 2024-03-22T07:15:53.859 INFO:tasks.workunit.client.1.smithi069.stdout:3/42: readlink d5/lf 0 2024-03-22T07:15:53.859 INFO:tasks.workunit.client.1.smithi069.stdout:4/21: chown f5 7 1 2024-03-22T07:15:53.859 INFO:tasks.workunit.client.1.smithi069.stdout:4/22: dread - f5 zero size 2024-03-22T07:15:53.859 INFO:tasks.workunit.client.1.smithi069.stdout:9/36: sync 2024-03-22T07:15:53.859 INFO:tasks.workunit.client.1.smithi069.stdout:5/2: sync 2024-03-22T07:15:53.859 INFO:tasks.workunit.client.1.smithi069.stdout:5/3: dwrite - no filename 2024-03-22T07:15:53.860 INFO:tasks.workunit.client.1.smithi069.stdout:1/13: sync 2024-03-22T07:15:53.860 INFO:tasks.workunit.client.1.smithi069.stdout:8/14: sync 2024-03-22T07:15:53.860 INFO:tasks.workunit.client.1.smithi069.stdout:2/20: sync 2024-03-22T07:15:53.860 INFO:tasks.workunit.client.1.smithi069.stdout:2/21: creat f8 x:0 0 0 2024-03-22T07:15:53.860 INFO:tasks.workunit.client.1.smithi069.stdout:2/22: creat f9 x:0 0 0 2024-03-22T07:15:53.860 INFO:tasks.workunit.client.1.smithi069.stdout:2/23: fsync f6 0 2024-03-22T07:15:53.860 INFO:tasks.workunit.client.1.smithi069.stdout:6/24: chown d4/f6 1658614 1 2024-03-22T07:15:53.860 INFO:tasks.workunit.client.0.smithi063.stdout:8/311: rmdir d0/d26/d2d/d56 0 2024-03-22T07:15:53.861 INFO:tasks.workunit.client.1.smithi069.stdout:4/23: symlink d6/l7 0 2024-03-22T07:15:53.861 INFO:tasks.workunit.client.1.smithi069.stdout:4/24: dread - f5 zero size 2024-03-22T07:15:53.861 INFO:tasks.workunit.client.0.smithi063.stdout:5/272: mkdir d8/d13/d24/d53 0 2024-03-22T07:15:53.862 INFO:tasks.workunit.client.0.smithi063.stdout:6/315: mkdir d3/d15/d16/d69 0 2024-03-22T07:15:53.864 INFO:tasks.workunit.client.1.smithi069.stdout:9/37: symlink d2/lf 0 2024-03-22T07:15:53.864 INFO:tasks.workunit.client.1.smithi069.stdout:9/38: readlink d2/lb 0 2024-03-22T07:15:53.866 INFO:tasks.workunit.client.1.smithi069.stdout:5/4: creat f0 x:0 0 0 2024-03-22T07:15:53.866 INFO:tasks.workunit.client.0.smithi063.stdout:2/255: rename d0/de/f15 to d0/de/d12/d20/f52 0 2024-03-22T07:15:53.867 INFO:tasks.workunit.client.0.smithi063.stdout:5/273: mknod d8/d13/c54 0 2024-03-22T07:15:53.868 INFO:tasks.workunit.client.0.smithi063.stdout:2/256: getdents d0/de/d12/d39 0 2024-03-22T07:15:53.868 INFO:tasks.workunit.client.1.smithi069.stdout:8/15: creat f0 x:0 0 0 2024-03-22T07:15:53.869 INFO:tasks.workunit.client.0.smithi063.stdout:8/312: dread d0/f22 [0,4194304] 0 2024-03-22T07:15:53.869 INFO:tasks.workunit.client.1.smithi069.stdout:1/14: mkdir d1 0 2024-03-22T07:15:53.870 INFO:tasks.workunit.client.0.smithi063.stdout:5/274: unlink d8/d13/d28/l37 0 2024-03-22T07:15:53.870 INFO:tasks.workunit.client.0.smithi063.stdout:5/275: write d8/f12 [1297223,62571] 0 2024-03-22T07:15:53.871 INFO:tasks.workunit.client.0.smithi063.stdout:2/257: creat d0/f53 x:0 0 0 2024-03-22T07:15:53.872 INFO:tasks.workunit.client.1.smithi069.stdout:6/25: chown d4/d7 1 1 2024-03-22T07:15:53.872 INFO:tasks.workunit.client.1.smithi069.stdout:6/26: dread - f3 zero size 2024-03-22T07:15:53.872 INFO:tasks.workunit.client.1.smithi069.stdout:6/27: readlink - no filename 2024-03-22T07:15:53.873 INFO:tasks.workunit.client.1.smithi069.stdout:6/28: dread - f3 zero size 2024-03-22T07:15:53.874 INFO:tasks.workunit.client.0.smithi063.stdout:8/313: creat d0/d18/d4a/f5a x:0 0 0 2024-03-22T07:15:53.874 INFO:tasks.workunit.client.0.smithi063.stdout:8/314: chown d0/d24/c30 1005 1 2024-03-22T07:15:53.874 INFO:tasks.workunit.client.0.smithi063.stdout:3/259: dwrite da/d16/d3f/f56 [0,4194304] 0 2024-03-22T07:15:53.874 INFO:tasks.workunit.client.1.smithi069.stdout:0/16: getdents . 0 2024-03-22T07:15:53.875 INFO:tasks.workunit.client.1.smithi069.stdout:0/17: readlink - no filename 2024-03-22T07:15:53.875 INFO:tasks.workunit.client.1.smithi069.stdout:0/18: dwrite - no filename 2024-03-22T07:15:53.875 INFO:tasks.workunit.client.1.smithi069.stdout:0/19: fsync - no filename 2024-03-22T07:15:53.875 INFO:tasks.workunit.client.1.smithi069.stdout:0/20: truncate - no filename 2024-03-22T07:15:53.875 INFO:tasks.workunit.client.1.smithi069.stdout:0/21: fdatasync - no filename 2024-03-22T07:15:53.875 INFO:tasks.workunit.client.1.smithi069.stdout:4/25: symlink d6/l8 0 2024-03-22T07:15:53.875 INFO:tasks.workunit.client.1.smithi069.stdout:4/26: truncate f5 125068 0 2024-03-22T07:15:53.875 INFO:tasks.workunit.client.1.smithi069.stdout:7/18: write f0 [2153894,99695] 0 2024-03-22T07:15:53.875 INFO:tasks.workunit.client.1.smithi069.stdout:7/19: write f0 [2996767,84370] 0 2024-03-22T07:15:53.876 INFO:tasks.workunit.client.0.smithi063.stdout:5/276: rename d8/f16 to d8/d13/d1e/d3c/f55 0 2024-03-22T07:15:53.876 INFO:tasks.workunit.client.0.smithi063.stdout:5/277: chown d8/l23 100874231 1 2024-03-22T07:15:53.876 INFO:tasks.workunit.client.1.smithi069.stdout:5/5: mkdir d1 0 2024-03-22T07:15:53.876 INFO:tasks.workunit.client.1.smithi069.stdout:5/6: truncate f0 750432 0 2024-03-22T07:15:53.877 INFO:tasks.workunit.client.1.smithi069.stdout:5/7: dread f0 [0,4194304] 0 2024-03-22T07:15:53.877 INFO:tasks.workunit.client.1.smithi069.stdout:3/43: getdents d5 0 2024-03-22T07:15:53.877 INFO:tasks.workunit.client.1.smithi069.stdout:3/44: dread - d5/d9/fe zero size 2024-03-22T07:15:53.877 INFO:tasks.workunit.client.1.smithi069.stdout:3/45: rename d5 to d5/d12 22 2024-03-22T07:15:53.878 INFO:tasks.workunit.client.1.smithi069.stdout:8/16: mkdir d1 0 2024-03-22T07:15:53.879 INFO:tasks.workunit.client.1.smithi069.stdout:1/15: creat d1/f2 x:0 0 0 2024-03-22T07:15:53.881 INFO:tasks.workunit.client.0.smithi063.stdout:3/260: creat da/d16/d26/f58 x:0 0 0 2024-03-22T07:15:53.883 INFO:tasks.workunit.client.0.smithi063.stdout:5/278: rename d8/d13/d1e/d31/l51 to d8/d13/d28/l56 0 2024-03-22T07:15:53.883 INFO:tasks.workunit.client.0.smithi063.stdout:5/279: chown d8/d13/d1e/d31 5 1 2024-03-22T07:15:53.883 INFO:tasks.workunit.client.0.smithi063.stdout:5/280: fsync d8/d13/d1e/f38 0 2024-03-22T07:15:53.883 INFO:tasks.workunit.client.0.smithi063.stdout:5/281: truncate d8/d13/d24/f3b 4478586 0 2024-03-22T07:15:53.884 INFO:tasks.workunit.client.1.smithi069.stdout:2/24: unlink f8 0 2024-03-22T07:15:53.884 INFO:tasks.workunit.client.0.smithi063.stdout:0/321: dwrite da/de/d17/d25/f6a [0,4194304] 0 2024-03-22T07:15:53.884 INFO:tasks.workunit.client.0.smithi063.stdout:2/258: getdents d0/de/d35 0 2024-03-22T07:15:53.884 INFO:tasks.workunit.client.0.smithi063.stdout:2/259: creat d0/de/d1b/d32/f54 x:0 0 0 2024-03-22T07:15:53.885 INFO:tasks.workunit.client.1.smithi069.stdout:0/22: mknod c2 0 2024-03-22T07:15:53.885 INFO:tasks.workunit.client.1.smithi069.stdout:0/23: chown c2 35572901 1 2024-03-22T07:15:53.885 INFO:tasks.workunit.client.1.smithi069.stdout:4/27: creat d6/f9 x:0 0 0 2024-03-22T07:15:53.886 INFO:tasks.workunit.client.1.smithi069.stdout:7/20: unlink f0 0 2024-03-22T07:15:53.887 INFO:tasks.workunit.client.1.smithi069.stdout:7/21: chown . 30900337 1 2024-03-22T07:15:53.887 INFO:tasks.workunit.client.1.smithi069.stdout:7/22: rmdir - no directory 2024-03-22T07:15:53.887 INFO:tasks.workunit.client.1.smithi069.stdout:7/23: dwrite - no filename 2024-03-22T07:15:53.887 INFO:tasks.workunit.client.1.smithi069.stdout:7/24: unlink - no file 2024-03-22T07:15:53.887 INFO:tasks.workunit.client.1.smithi069.stdout:7/25: link - no file 2024-03-22T07:15:53.887 INFO:tasks.workunit.client.1.smithi069.stdout:7/26: creat f1 x:0 0 0 2024-03-22T07:15:53.889 INFO:tasks.workunit.client.0.smithi063.stdout:0/322: truncate da/de/d17/d25/f27 298371 0 2024-03-22T07:15:53.889 INFO:tasks.workunit.client.0.smithi063.stdout:0/323: write da/de/f24 [1596073,95358] 0 2024-03-22T07:15:53.890 INFO:tasks.workunit.client.1.smithi069.stdout:3/46: truncate d5/f7 2847156 0 2024-03-22T07:15:53.892 INFO:tasks.workunit.client.1.smithi069.stdout:8/17: creat d1/f2 x:0 0 0 2024-03-22T07:15:53.893 INFO:tasks.workunit.client.0.smithi063.stdout:0/324: mknod da/de/d67/c6b 0 2024-03-22T07:15:53.893 INFO:tasks.workunit.client.0.smithi063.stdout:0/325: truncate da/de/d17/d25/d3c/d43/f5b 700742 0 2024-03-22T07:15:53.895 INFO:tasks.workunit.client.1.smithi069.stdout:1/16: unlink d1/f2 0 2024-03-22T07:15:53.895 INFO:tasks.workunit.client.1.smithi069.stdout:1/17: unlink - no file 2024-03-22T07:15:53.895 INFO:tasks.workunit.client.1.smithi069.stdout:1/18: unlink - no file 2024-03-22T07:15:53.895 INFO:tasks.workunit.client.1.smithi069.stdout:1/19: creat d1/f3 x:0 0 0 2024-03-22T07:15:53.895 INFO:tasks.workunit.client.1.smithi069.stdout:0/24: mkdir d3 0 2024-03-22T07:15:53.895 INFO:tasks.workunit.client.1.smithi069.stdout:0/25: chown c2 12515 1 2024-03-22T07:15:53.895 INFO:tasks.workunit.client.1.smithi069.stdout:0/26: dread - no filename 2024-03-22T07:15:53.896 INFO:tasks.workunit.client.1.smithi069.stdout:4/28: mknod d6/ca 0 2024-03-22T07:15:53.896 INFO:tasks.workunit.client.1.smithi069.stdout:4/29: write f3 [270962,26288] 0 2024-03-22T07:15:53.897 INFO:tasks.workunit.client.0.smithi063.stdout:0/326: unlink da/de/d60/c47 0 2024-03-22T07:15:53.900 INFO:tasks.workunit.client.1.smithi069.stdout:5/8: unlink f0 0 2024-03-22T07:15:53.901 INFO:tasks.workunit.client.0.smithi063.stdout:0/327: rename da/fc to da/d49/f6c 0 2024-03-22T07:15:53.901 INFO:tasks.workunit.client.0.smithi063.stdout:0/328: stat da/de/d17/d25/f27 0 2024-03-22T07:15:53.901 INFO:tasks.workunit.client.0.smithi063.stdout:0/329: dread - da/de/d17/f21 zero size 2024-03-22T07:15:53.901 INFO:tasks.workunit.client.0.smithi063.stdout:0/330: readlink da/de/l1d 0 2024-03-22T07:15:53.902 INFO:tasks.workunit.client.0.smithi063.stdout:0/331: rename da/de/d17/d25/d3c/d53/f61 to da/d49/f6d 0 2024-03-22T07:15:53.904 INFO:tasks.workunit.client.1.smithi069.stdout:0/27: creat d3/f4 x:0 0 0 2024-03-22T07:15:53.904 INFO:tasks.workunit.client.1.smithi069.stdout:0/28: truncate d3/f4 748418 0 2024-03-22T07:15:53.904 INFO:tasks.workunit.client.0.smithi063.stdout:0/332: symlink da/de/d17/d25/d3c/d53/l6e 0 2024-03-22T07:15:53.906 INFO:tasks.workunit.client.1.smithi069.stdout:5/9: creat d1/f2 x:0 0 0 2024-03-22T07:15:53.906 INFO:tasks.workunit.client.1.smithi069.stdout:9/39: dwrite d2/f5 [0,4194304] 0 2024-03-22T07:15:53.907 INFO:tasks.workunit.client.1.smithi069.stdout:3/47: rmdir d5/d9/db 39 2024-03-22T07:15:53.910 INFO:tasks.workunit.client.1.smithi069.stdout:0/29: symlink d3/l5 0 2024-03-22T07:15:53.910 INFO:tasks.workunit.client.1.smithi069.stdout:0/30: creat d3/f6 x:0 0 0 2024-03-22T07:15:53.910 INFO:tasks.workunit.client.0.smithi063.stdout:2/260: truncate d0/de/d12/f1f 2282720 0 2024-03-22T07:15:53.911 INFO:tasks.workunit.client.0.smithi063.stdout:2/261: chown d0/de/d12 4412459 1 2024-03-22T07:15:53.911 INFO:tasks.workunit.client.0.smithi063.stdout:2/262: dread - d0/f53 zero size 2024-03-22T07:15:53.911 INFO:tasks.workunit.client.1.smithi069.stdout:9/40: creat d2/dd/f10 x:0 0 0 2024-03-22T07:15:53.913 INFO:tasks.workunit.client.1.smithi069.stdout:6/29: dwrite d4/f6 [0,4194304] 0 2024-03-22T07:15:53.914 INFO:tasks.workunit.client.1.smithi069.stdout:0/31: mkdir d3/d7 0 2024-03-22T07:15:53.916 INFO:tasks.workunit.client.1.smithi069.stdout:9/41: symlink d2/dd/l11 0 2024-03-22T07:15:53.919 INFO:tasks.workunit.client.1.smithi069.stdout:6/30: mknod d4/d7/c8 0 2024-03-22T07:15:53.919 INFO:tasks.workunit.client.1.smithi069.stdout:6/31: write f3 [55363,10092] 0 2024-03-22T07:15:53.925 INFO:tasks.workunit.client.0.smithi063.stdout:8/315: dwrite d0/d26/d2d/d3d/f44 [0,4194304] 0 2024-03-22T07:15:53.925 INFO:tasks.workunit.client.0.smithi063.stdout:8/316: readlink d0/d26/l4b 0 2024-03-22T07:15:53.925 INFO:tasks.workunit.client.0.smithi063.stdout:3/261: dwrite da/d16/d26/f58 [0,4194304] 0 2024-03-22T07:15:53.925 INFO:tasks.workunit.client.0.smithi063.stdout:0/333: dread da/de/d17/d25/f68 [0,4194304] 0 2024-03-22T07:15:53.933 INFO:tasks.workunit.client.1.smithi069.stdout:7/27: fsync f1 0 2024-03-22T07:15:53.933 INFO:tasks.workunit.client.1.smithi069.stdout:7/28: creat f2 x:0 0 0 2024-03-22T07:15:53.933 INFO:tasks.workunit.client.1.smithi069.stdout:7/29: creat f3 x:0 0 0 2024-03-22T07:15:53.933 INFO:tasks.workunit.client.1.smithi069.stdout:7/30: dread - f2 zero size 2024-03-22T07:15:53.937 INFO:tasks.workunit.client.1.smithi069.stdout:7/31: rename f1 to f4 0 2024-03-22T07:15:53.937 INFO:tasks.workunit.client.1.smithi069.stdout:7/32: chown f2 773626 1 2024-03-22T07:15:53.939 INFO:tasks.workunit.client.0.smithi063.stdout:3/262: unlink da/f46 0 2024-03-22T07:15:53.939 INFO:tasks.workunit.client.0.smithi063.stdout:3/263: chown da/c12 7 1 2024-03-22T07:15:53.939 INFO:tasks.workunit.client.1.smithi069.stdout:7/33: symlink l5 0 2024-03-22T07:15:53.939 INFO:tasks.workunit.client.1.smithi069.stdout:7/34: chown f2 78501859 1 2024-03-22T07:15:53.939 INFO:tasks.workunit.client.1.smithi069.stdout:7/35: dread - f2 zero size 2024-03-22T07:15:53.939 INFO:tasks.workunit.client.1.smithi069.stdout:7/36: creat f6 x:0 0 0 2024-03-22T07:15:53.939 INFO:tasks.workunit.client.1.smithi069.stdout:7/37: chown f4 447 1 2024-03-22T07:15:53.941 INFO:tasks.workunit.client.0.smithi063.stdout:8/317: rename d0/d26/d2d/d3d to d0/d24/d5b 0 2024-03-22T07:15:53.948 INFO:tasks.workunit.client.1.smithi069.stdout:8/18: dwrite f0 [0,4194304] 0 2024-03-22T07:15:53.948 INFO:tasks.workunit.client.1.smithi069.stdout:8/19: fdatasync f0 0 2024-03-22T07:15:53.948 INFO:tasks.workunit.client.1.smithi069.stdout:8/20: rename d1 to d1/d3 22 2024-03-22T07:15:53.948 INFO:tasks.workunit.client.1.smithi069.stdout:8/21: dread - d1/f2 zero size 2024-03-22T07:15:53.948 INFO:tasks.workunit.client.1.smithi069.stdout:2/25: dwrite f9 [0,4194304] 0 2024-03-22T07:15:53.953 INFO:tasks.workunit.client.1.smithi069.stdout:8/22: rmdir d1 39 2024-03-22T07:15:53.954 INFO:tasks.workunit.client.1.smithi069.stdout:1/20: fsync d1/f3 0 2024-03-22T07:15:53.954 INFO:tasks.workunit.client.1.smithi069.stdout:1/21: rename d1 to d1/d4 22 2024-03-22T07:15:53.955 INFO:tasks.workunit.client.1.smithi069.stdout:8/23: creat d1/f4 x:0 0 0 2024-03-22T07:15:53.955 INFO:tasks.workunit.client.1.smithi069.stdout:8/24: dread - d1/f2 zero size 2024-03-22T07:15:53.955 INFO:tasks.workunit.client.1.smithi069.stdout:0/32: fsync d3/f4 0 2024-03-22T07:15:53.955 INFO:tasks.workunit.client.1.smithi069.stdout:0/33: creat d3/f8 x:0 0 0 2024-03-22T07:15:53.955 INFO:tasks.workunit.client.1.smithi069.stdout:1/22: mkdir d1/d5 0 2024-03-22T07:15:53.957 INFO:tasks.workunit.client.1.smithi069.stdout:0/34: creat d3/d7/f9 x:0 0 0 2024-03-22T07:15:53.957 INFO:tasks.workunit.client.0.smithi063.stdout:3/264: dread da/f52 [0,4194304] 0 2024-03-22T07:15:53.958 INFO:tasks.workunit.client.1.smithi069.stdout:1/23: rmdir d1/d5 0 2024-03-22T07:15:53.960 INFO:tasks.workunit.client.1.smithi069.stdout:0/35: mkdir d3/da 0 2024-03-22T07:15:53.960 INFO:tasks.workunit.client.1.smithi069.stdout:0/36: creat d3/d7/fb x:0 0 0 2024-03-22T07:15:53.960 INFO:tasks.workunit.client.1.smithi069.stdout:0/37: truncate d3/f6 433287 0 2024-03-22T07:15:53.961 INFO:tasks.workunit.client.1.smithi069.stdout:0/38: chown d3/d7/fb 122077895 1 2024-03-22T07:15:53.961 INFO:tasks.workunit.client.0.smithi063.stdout:3/265: rename da/c1e to da/d16/d26/c59 0 2024-03-22T07:15:53.962 INFO:tasks.workunit.client.1.smithi069.stdout:1/24: link d1/f3 d1/f6 0 2024-03-22T07:15:53.962 INFO:tasks.workunit.client.1.smithi069.stdout:1/25: creat d1/f7 x:0 0 0 2024-03-22T07:15:53.962 INFO:tasks.workunit.client.1.smithi069.stdout:1/26: truncate d1/f6 372296 0 2024-03-22T07:15:53.962 INFO:tasks.workunit.client.1.smithi069.stdout:0/39: mknod d3/d7/cc 0 2024-03-22T07:15:53.963 INFO:tasks.workunit.client.1.smithi069.stdout:1/27: mknod d1/c8 0 2024-03-22T07:15:53.965 INFO:tasks.workunit.client.1.smithi069.stdout:0/40: symlink d3/ld 0 2024-03-22T07:15:53.965 INFO:tasks.workunit.client.1.smithi069.stdout:0/41: symlink d3/d7/le 0 2024-03-22T07:15:53.965 INFO:tasks.workunit.client.1.smithi069.stdout:0/42: truncate d3/f6 939494 0 2024-03-22T07:15:53.966 INFO:tasks.workunit.client.1.smithi069.stdout:0/43: symlink d3/d7/lf 0 2024-03-22T07:15:53.967 INFO:tasks.workunit.client.1.smithi069.stdout:8/25: write f0 [2015438,118567] 0 2024-03-22T07:15:53.967 INFO:tasks.workunit.client.1.smithi069.stdout:1/28: dread d1/f6 [0,4194304] 0 2024-03-22T07:15:53.967 INFO:tasks.workunit.client.1.smithi069.stdout:1/29: write d1/f7 [15914,45894] 0 2024-03-22T07:15:53.967 INFO:tasks.workunit.client.1.smithi069.stdout:1/30: chown d1 341022860 1 2024-03-22T07:15:53.968 INFO:tasks.workunit.client.0.smithi063.stdout:2/263: dwrite d0/f41 [0,4194304] 0 2024-03-22T07:15:53.968 INFO:tasks.workunit.client.1.smithi069.stdout:0/44: mknod d3/d7/c10 0 2024-03-22T07:15:53.969 INFO:tasks.workunit.client.0.smithi063.stdout:2/264: mknod d0/de/d35/d36/c55 0 2024-03-22T07:15:53.969 INFO:tasks.workunit.client.0.smithi063.stdout:2/265: write d0/de/d1b/d32/f3b [676922,1853] 0 2024-03-22T07:15:53.969 INFO:tasks.workunit.client.0.smithi063.stdout:2/266: dread - d0/f53 zero size 2024-03-22T07:15:53.970 INFO:tasks.workunit.client.1.smithi069.stdout:4/30: dwrite d6/f9 [0,4194304] 0 2024-03-22T07:15:53.970 INFO:tasks.workunit.client.1.smithi069.stdout:5/10: dwrite d1/f2 [0,4194304] 0 2024-03-22T07:15:53.970 INFO:tasks.workunit.client.1.smithi069.stdout:5/11: creat d1/f3 x:0 0 0 2024-03-22T07:15:53.971 INFO:tasks.workunit.client.0.smithi063.stdout:2/267: unlink d0/de/d1b/d32/f43 0 2024-03-22T07:15:53.978 INFO:tasks.workunit.client.0.smithi063.stdout:3/266: write da/d16/f38 [1962307,29005] 0 2024-03-22T07:15:53.980 INFO:tasks.workunit.client.1.smithi069.stdout:3/48: dwrite d5/d9/fa [0,4194304] 0 2024-03-22T07:15:53.981 INFO:tasks.workunit.client.0.smithi063.stdout:3/267: getdents da/d16 0 2024-03-22T07:15:53.983 INFO:tasks.workunit.client.1.smithi069.stdout:3/49: rmdir d5/d9 39 2024-03-22T07:15:53.986 INFO:tasks.workunit.client.0.smithi063.stdout:3/268: write da/d16/d26/f39 [205241,31822] 0 2024-03-22T07:15:53.988 INFO:tasks.workunit.client.0.smithi063.stdout:3/269: dread da/f1c [0,4194304] 0 2024-03-22T07:15:53.988 INFO:tasks.workunit.client.0.smithi063.stdout:3/270: write da/d16/d26/f48 [595150,53012] 0 2024-03-22T07:15:53.988 INFO:tasks.workunit.client.1.smithi069.stdout:0/45: fsync d3/f6 0 2024-03-22T07:15:53.988 INFO:tasks.workunit.client.1.smithi069.stdout:0/46: dread - d3/f8 zero size 2024-03-22T07:15:53.994 INFO:tasks.workunit.client.1.smithi069.stdout:9/42: dwrite d2/dd/f10 [0,4194304] 0 2024-03-22T07:15:53.997 INFO:tasks.workunit.client.1.smithi069.stdout:9/43: link c0 d2/dd/c12 0 2024-03-22T07:15:54.002 INFO:tasks.workunit.client.1.smithi069.stdout:4/31: dread d6/f9 [0,4194304] 0 2024-03-22T07:15:54.003 INFO:tasks.workunit.client.1.smithi069.stdout:0/47: read d3/f6 [290110,2868] 0 2024-03-22T07:15:54.003 INFO:tasks.workunit.client.1.smithi069.stdout:4/32: symlink d6/lb 0 2024-03-22T07:15:54.003 INFO:tasks.workunit.client.1.smithi069.stdout:4/33: chown d6/lb 30899 1 2024-03-22T07:15:54.004 INFO:tasks.workunit.client.1.smithi069.stdout:4/34: chown d6/l8 2683350 1 2024-03-22T07:15:54.004 INFO:tasks.workunit.client.0.smithi063.stdout:0/334: dwrite da/de/d17/f3a [4194304,4194304] 0 2024-03-22T07:15:54.005 INFO:tasks.workunit.client.1.smithi069.stdout:6/32: dwrite d4/f6 [0,4194304] 0 2024-03-22T07:15:54.005 INFO:tasks.workunit.client.1.smithi069.stdout:6/33: chown d4/d7 30 1 2024-03-22T07:15:54.005 INFO:tasks.workunit.client.1.smithi069.stdout:1/31: dwrite d1/f6 [0,4194304] 0 2024-03-22T07:15:54.005 INFO:tasks.workunit.client.1.smithi069.stdout:1/32: chown d1/c8 0 1 2024-03-22T07:15:54.005 INFO:tasks.workunit.client.1.smithi069.stdout:7/38: fsync f2 0 2024-03-22T07:15:54.005 INFO:tasks.workunit.client.1.smithi069.stdout:7/39: stat f2 0 2024-03-22T07:15:54.005 INFO:tasks.workunit.client.1.smithi069.stdout:7/40: creat f7 x:0 0 0 2024-03-22T07:15:54.006 INFO:tasks.workunit.client.0.smithi063.stdout:8/318: dwrite d0/d26/f34 [0,4194304] 0 2024-03-22T07:15:54.006 INFO:tasks.workunit.client.0.smithi063.stdout:8/319: fsync d0/d26/f27 0 2024-03-22T07:15:54.006 INFO:tasks.workunit.client.0.smithi063.stdout:5/282: dwrite d8/f12 [0,4194304] 0 2024-03-22T07:15:54.007 INFO:tasks.workunit.client.0.smithi063.stdout:5/283: chown d8/f10 9929 1 2024-03-22T07:15:54.007 INFO:tasks.workunit.client.0.smithi063.stdout:5/284: write d8/fe [4734217,83483] 0 2024-03-22T07:15:54.007 INFO:tasks.workunit.client.1.smithi069.stdout:2/26: dwrite f1 [0,4194304] 0 2024-03-22T07:15:54.007 INFO:tasks.workunit.client.1.smithi069.stdout:2/27: fdatasync f9 0 2024-03-22T07:15:54.007 INFO:tasks.workunit.client.1.smithi069.stdout:2/28: read - f5 zero size 2024-03-22T07:15:54.007 INFO:tasks.workunit.client.1.smithi069.stdout:2/29: rmdir - no directory 2024-03-22T07:15:54.007 INFO:tasks.workunit.client.1.smithi069.stdout:2/30: creat fa x:0 0 0 2024-03-22T07:15:54.007 INFO:tasks.workunit.client.1.smithi069.stdout:8/26: dwrite d1/f4 [0,4194304] 0 2024-03-22T07:15:54.010 INFO:tasks.workunit.client.0.smithi063.stdout:5/285: truncate d8/d13/d1e/f2f 96723 0 2024-03-22T07:15:54.014 INFO:tasks.workunit.client.0.smithi063.stdout:0/335: dread da/d49/f6c [0,4194304] 0 2024-03-22T07:15:54.016 INFO:tasks.workunit.client.1.smithi069.stdout:6/34: dread f3 [0,4194304] 0 2024-03-22T07:15:54.017 INFO:tasks.workunit.client.1.smithi069.stdout:2/31: mknod cb 0 2024-03-22T07:15:54.017 INFO:tasks.workunit.client.0.smithi063.stdout:0/336: symlink da/de/d17/d25/d3c/d43/l6f 0 2024-03-22T07:15:54.017 INFO:tasks.workunit.client.0.smithi063.stdout:0/337: write da/f54 [861292,89896] 0 2024-03-22T07:15:54.017 INFO:tasks.workunit.client.0.smithi063.stdout:0/338: creat da/de/d17/d25/f70 x:0 0 0 2024-03-22T07:15:54.017 INFO:tasks.workunit.client.0.smithi063.stdout:8/320: dread d0/d1f/f53 [0,4194304] 0 2024-03-22T07:15:54.017 INFO:tasks.workunit.client.0.smithi063.stdout:8/321: chown d0/l15 3 1 2024-03-22T07:15:54.018 INFO:tasks.workunit.client.0.smithi063.stdout:0/339: mknod da/de/d17/d25/d3c/d64/c71 0 2024-03-22T07:15:54.027 INFO:tasks.workunit.client.1.smithi069.stdout:7/41: rename f7 to f8 0 2024-03-22T07:15:54.029 INFO:tasks.workunit.client.0.smithi063.stdout:5/286: dwrite d8/f2e [0,4194304] 0 2024-03-22T07:15:54.030 INFO:tasks.workunit.client.1.smithi069.stdout:6/35: creat d4/f9 x:0 0 0 2024-03-22T07:15:54.032 INFO:tasks.workunit.client.1.smithi069.stdout:2/32: mknod cc 0 2024-03-22T07:15:54.032 INFO:tasks.workunit.client.1.smithi069.stdout:1/33: fsync d1/f7 0 2024-03-22T07:15:54.034 INFO:tasks.workunit.client.1.smithi069.stdout:8/27: rename d1/f4 to d1/f5 0 2024-03-22T07:15:54.034 INFO:tasks.workunit.client.1.smithi069.stdout:8/28: write d1/f2 [583601,72190] 0 2024-03-22T07:15:54.034 INFO:tasks.workunit.client.0.smithi063.stdout:5/287: mkdir d8/d13/d28/d57 0 2024-03-22T07:15:54.035 INFO:tasks.workunit.client.1.smithi069.stdout:6/36: mkdir d4/d7/da 0 2024-03-22T07:15:54.036 INFO:tasks.workunit.client.1.smithi069.stdout:6/37: fsync f3 0 2024-03-22T07:15:54.039 INFO:tasks.workunit.client.1.smithi069.stdout:1/34: symlink d1/l9 0 2024-03-22T07:15:54.039 INFO:tasks.workunit.client.1.smithi069.stdout:1/35: creat d1/fa x:0 0 0 2024-03-22T07:15:54.040 INFO:tasks.workunit.client.0.smithi063.stdout:3/271: rmdir da 39 2024-03-22T07:15:54.042 INFO:tasks.workunit.client.0.smithi063.stdout:8/322: dwrite d0/d24/f42 [0,4194304] 0 2024-03-22T07:15:54.042 INFO:tasks.workunit.client.0.smithi063.stdout:8/323: chown d0/d18/d4a/f5a 844454 1 2024-03-22T07:15:54.043 INFO:tasks.workunit.client.0.smithi063.stdout:3/272: unlink da/f35 0 2024-03-22T07:15:54.043 INFO:tasks.workunit.client.1.smithi069.stdout:0/48: getdents d3/d7 0 2024-03-22T07:15:54.043 INFO:tasks.workunit.client.1.smithi069.stdout:0/49: chown d3/d7/c10 6945 1 2024-03-22T07:15:54.043 INFO:tasks.workunit.client.1.smithi069.stdout:0/50: write d3/d7/fb [583117,112175] 0 2024-03-22T07:15:54.044 INFO:tasks.workunit.client.1.smithi069.stdout:6/38: creat d4/fb x:0 0 0 2024-03-22T07:15:54.044 INFO:tasks.workunit.client.0.smithi063.stdout:8/324: rmdir d0/d24/d33 39 2024-03-22T07:15:54.045 INFO:tasks.workunit.client.0.smithi063.stdout:3/273: dread da/d16/f32 [0,4194304] 0 2024-03-22T07:15:54.046 INFO:tasks.workunit.client.0.smithi063.stdout:3/274: rmdir da/d16/d3c/d4b 39 2024-03-22T07:15:54.046 INFO:tasks.workunit.client.1.smithi069.stdout:8/29: dread d1/f2 [0,4194304] 0 2024-03-22T07:15:54.047 INFO:tasks.workunit.client.1.smithi069.stdout:8/30: chown f0 9929201 1 2024-03-22T07:15:54.048 INFO:tasks.workunit.client.0.smithi063.stdout:3/275: symlink da/d16/d3c/d4b/l5a 0 2024-03-22T07:15:54.048 INFO:tasks.workunit.client.0.smithi063.stdout:3/276: dread - da/d16/f42 zero size 2024-03-22T07:15:54.049 INFO:tasks.workunit.client.1.smithi069.stdout:1/36: dread d1/f3 [0,4194304] 0 2024-03-22T07:15:54.050 INFO:tasks.workunit.client.0.smithi063.stdout:8/325: dread d0/d24/f29 [0,4194304] 0 2024-03-22T07:15:54.051 INFO:tasks.workunit.client.1.smithi069.stdout:0/51: creat d3/da/f11 x:0 0 0 2024-03-22T07:15:54.052 INFO:tasks.workunit.client.0.smithi063.stdout:3/277: mknod da/d16/d26/c5b 0 2024-03-22T07:15:54.053 INFO:tasks.workunit.client.1.smithi069.stdout:6/39: unlink d4/f9 0 2024-03-22T07:15:54.053 INFO:tasks.workunit.client.1.smithi069.stdout:5/12: write d1/f2 [5108882,76845] 0 2024-03-22T07:15:54.053 INFO:tasks.workunit.client.1.smithi069.stdout:5/13: write d1/f3 [874354,116967] 0 2024-03-22T07:15:54.054 INFO:tasks.workunit.client.1.smithi069.stdout:1/37: dread d1/f6 [0,4194304] 0 2024-03-22T07:15:54.054 INFO:tasks.workunit.client.1.smithi069.stdout:1/38: creat d1/fb x:0 0 0 2024-03-22T07:15:54.054 INFO:tasks.workunit.client.0.smithi063.stdout:8/326: getdents d0/d18/d45 0 2024-03-22T07:15:54.054 INFO:tasks.workunit.client.1.smithi069.stdout:0/52: creat d3/d7/f12 x:0 0 0 2024-03-22T07:15:54.054 INFO:tasks.workunit.client.1.smithi069.stdout:0/53: readlink d3/ld 0 2024-03-22T07:15:54.055 INFO:tasks.workunit.client.1.smithi069.stdout:6/40: symlink d4/d7/da/lc 0 2024-03-22T07:15:54.056 INFO:tasks.workunit.client.0.smithi063.stdout:3/278: rename da/d16/d26/f39 to da/d16/f5c 0 2024-03-22T07:15:54.058 INFO:tasks.workunit.client.1.smithi069.stdout:1/39: dread d1/f3 [0,4194304] 0 2024-03-22T07:15:54.058 INFO:tasks.workunit.client.1.smithi069.stdout:1/40: dread - d1/fa zero size 2024-03-22T07:15:54.058 INFO:tasks.workunit.client.1.smithi069.stdout:1/41: fdatasync d1/f3 0 2024-03-22T07:15:54.058 INFO:tasks.workunit.client.1.smithi069.stdout:1/42: write d1/f7 [360492,27394] 0 2024-03-22T07:15:54.058 INFO:tasks.workunit.client.0.smithi063.stdout:8/327: getdents d0/d18 0 2024-03-22T07:15:54.058 INFO:tasks.workunit.client.1.smithi069.stdout:3/50: truncate d5/d9/fa 3232773 0 2024-03-22T07:15:54.060 INFO:tasks.workunit.client.1.smithi069.stdout:0/54: rename d3/d7/f9 to d3/f13 0 2024-03-22T07:15:54.060 INFO:tasks.workunit.client.0.smithi063.stdout:3/279: mknod da/d16/d3f/c5d 0 2024-03-22T07:15:54.061 INFO:tasks.workunit.client.1.smithi069.stdout:0/55: dread d3/f6 [0,4194304] 0 2024-03-22T07:15:54.061 INFO:tasks.workunit.client.1.smithi069.stdout:0/56: write d3/da/f11 [133379,38086] 0 2024-03-22T07:15:54.061 INFO:tasks.workunit.client.1.smithi069.stdout:0/57: chown d3/f6 47499 1 2024-03-22T07:15:54.061 INFO:tasks.workunit.client.1.smithi069.stdout:0/58: write d3/f8 [766293,100245] 0 2024-03-22T07:15:54.062 INFO:tasks.workunit.client.1.smithi069.stdout:1/43: dread d1/f6 [0,4194304] 0 2024-03-22T07:15:54.062 INFO:tasks.workunit.client.1.smithi069.stdout:3/51: creat d5/d9/f13 x:0 0 0 2024-03-22T07:15:54.063 INFO:tasks.workunit.client.0.smithi063.stdout:8/328: dread d0/d24/f35 [0,4194304] 0 2024-03-22T07:15:54.063 INFO:tasks.workunit.client.0.smithi063.stdout:5/288: dwrite d8/d13/d1e/d31/f46 [0,4194304] 0 2024-03-22T07:15:54.063 INFO:tasks.workunit.client.0.smithi063.stdout:8/329: write d0/d18/d4a/f5a [878103,101434] 0 2024-03-22T07:15:54.063 INFO:tasks.workunit.client.0.smithi063.stdout:8/330: creat d0/d1f/f5c x:0 0 0 2024-03-22T07:15:54.063 INFO:tasks.workunit.client.0.smithi063.stdout:8/331: write d0/f22 [3036467,82500] 0 2024-03-22T07:15:54.063 INFO:tasks.workunit.client.0.smithi063.stdout:3/280: unlink da/d16/d3c/d4b/c4e 0 2024-03-22T07:15:54.064 INFO:tasks.workunit.client.1.smithi069.stdout:7/42: dwrite f2 [0,4194304] 0 2024-03-22T07:15:54.065 INFO:tasks.workunit.client.0.smithi063.stdout:4/221: sync 2024-03-22T07:15:54.065 INFO:tasks.workunit.client.0.smithi063.stdout:7/223: sync 2024-03-22T07:15:54.065 INFO:tasks.workunit.client.0.smithi063.stdout:7/224: readlink d5/le 0 2024-03-22T07:15:54.065 INFO:tasks.workunit.client.0.smithi063.stdout:7/225: write d5/f17 [11452,2473] 0 2024-03-22T07:15:54.065 INFO:tasks.workunit.client.0.smithi063.stdout:1/334: sync 2024-03-22T07:15:54.066 INFO:tasks.workunit.client.1.smithi069.stdout:9/44: rmdir d2/dd 39 2024-03-22T07:15:54.066 INFO:tasks.workunit.client.1.smithi069.stdout:1/44: symlink d1/lc 0 2024-03-22T07:15:54.066 INFO:tasks.workunit.client.1.smithi069.stdout:4/35: getdents d6 0 2024-03-22T07:15:54.067 INFO:tasks.workunit.client.1.smithi069.stdout:8/31: dread f0 [0,4194304] 0 2024-03-22T07:15:54.067 INFO:tasks.workunit.client.1.smithi069.stdout:8/32: write d1/f2 [935467,44913] 0 2024-03-22T07:15:54.068 INFO:tasks.workunit.client.0.smithi063.stdout:9/218: sync 2024-03-22T07:15:54.069 INFO:tasks.workunit.client.1.smithi069.stdout:3/52: unlink c1 0 2024-03-22T07:15:54.069 INFO:tasks.workunit.client.1.smithi069.stdout:3/53: chown d5/d9 2028137 1 2024-03-22T07:15:54.069 INFO:tasks.workunit.client.0.smithi063.stdout:3/281: getdents da/d16/d3c 0 2024-03-22T07:15:54.073 INFO:tasks.workunit.client.1.smithi069.stdout:2/33: truncate f1 761658 0 2024-03-22T07:15:54.073 INFO:tasks.workunit.client.0.smithi063.stdout:4/222: creat d1/f45 x:0 0 0 2024-03-22T07:15:54.075 INFO:tasks.workunit.client.1.smithi069.stdout:7/43: mkdir d9 0 2024-03-22T07:15:54.075 INFO:tasks.workunit.client.1.smithi069.stdout:7/44: fsync f6 0 2024-03-22T07:15:54.075 INFO:tasks.workunit.client.0.smithi063.stdout:9/219: dread d8/d19/d20/f34 [0,4194304] 0 2024-03-22T07:15:54.076 INFO:tasks.workunit.client.0.smithi063.stdout:8/332: dread d0/d1b/f1e [0,4194304] 0 2024-03-22T07:15:54.076 INFO:tasks.workunit.client.0.smithi063.stdout:8/333: chown d0/d1f 15373895 1 2024-03-22T07:15:54.076 INFO:tasks.workunit.client.0.smithi063.stdout:1/335: dread d4/d14/f50 [0,4194304] 0 2024-03-22T07:15:54.076 INFO:tasks.workunit.client.0.smithi063.stdout:1/336: fdatasync d4/f38 0 2024-03-22T07:15:54.076 INFO:tasks.workunit.client.0.smithi063.stdout:1/337: dread - d4/f3b zero size 2024-03-22T07:15:54.077 INFO:tasks.workunit.client.0.smithi063.stdout:1/338: chown d4/df/l19 487 1 2024-03-22T07:15:54.078 INFO:tasks.workunit.client.0.smithi063.stdout:7/226: mkdir d5/da/d28/d4b 0 2024-03-22T07:15:54.079 INFO:tasks.workunit.client.1.smithi069.stdout:9/45: creat d2/dd/f13 x:0 0 0 2024-03-22T07:15:54.079 INFO:tasks.workunit.client.1.smithi069.stdout:9/46: dread - d2/dd/f13 zero size 2024-03-22T07:15:54.081 INFO:tasks.workunit.client.0.smithi063.stdout:5/289: truncate d8/f12 1485918 0 2024-03-22T07:15:54.081 INFO:tasks.workunit.client.0.smithi063.stdout:5/290: truncate d8/d13/d1e/d31/f4e 811454 0 2024-03-22T07:15:54.081 INFO:tasks.workunit.client.1.smithi069.stdout:1/45: symlink d1/ld 0 2024-03-22T07:15:54.082 INFO:tasks.workunit.client.1.smithi069.stdout:4/36: mkdir d6/dc 0 2024-03-22T07:15:54.086 INFO:tasks.workunit.client.0.smithi063.stdout:4/223: creat d1/d5/d43/d27/d2c/d3d/f46 x:0 0 0 2024-03-22T07:15:54.086 INFO:tasks.workunit.client.0.smithi063.stdout:4/224: write d1/d5/f2d [1333277,53007] 0 2024-03-22T07:15:54.086 INFO:tasks.workunit.client.1.smithi069.stdout:8/33: rename f0 to d1/f6 0 2024-03-22T07:15:54.088 INFO:tasks.workunit.client.1.smithi069.stdout:3/54: mknod d5/c14 0 2024-03-22T07:15:54.092 INFO:tasks.workunit.client.0.smithi063.stdout:7/227: symlink d5/d16/l4c 0 2024-03-22T07:15:54.092 INFO:tasks.workunit.client.0.smithi063.stdout:7/228: write d5/f17 [4027443,24517] 0 2024-03-22T07:15:54.092 INFO:tasks.workunit.client.0.smithi063.stdout:8/334: dread d0/d26/f27 [0,4194304] 0 2024-03-22T07:15:54.094 INFO:tasks.workunit.client.1.smithi069.stdout:2/34: mknod cd 0 2024-03-22T07:15:54.094 INFO:tasks.workunit.client.1.smithi069.stdout:7/45: symlink d9/la 0 2024-03-22T07:15:54.094 INFO:tasks.workunit.client.1.smithi069.stdout:9/47: mknod d2/c14 0 2024-03-22T07:15:54.094 INFO:tasks.workunit.client.1.smithi069.stdout:9/48: write d2/dd/f13 [100418,116575] 0 2024-03-22T07:15:54.094 INFO:tasks.workunit.client.1.smithi069.stdout:9/49: write d2/dd/f10 [5020257,48257] 0 2024-03-22T07:15:54.097 INFO:tasks.workunit.client.0.smithi063.stdout:7/229: unlink d5/d14/f42 0 2024-03-22T07:15:54.097 INFO:tasks.workunit.client.1.smithi069.stdout:2/35: rename cd to ce 0 2024-03-22T07:15:54.097 INFO:tasks.workunit.client.1.smithi069.stdout:2/36: stat cb 0 2024-03-22T07:15:54.098 INFO:tasks.workunit.client.1.smithi069.stdout:2/37: fsync f7 0 2024-03-22T07:15:54.098 INFO:tasks.workunit.client.1.smithi069.stdout:5/14: dwrite d1/f2 [4194304,4194304] 0 2024-03-22T07:15:54.098 INFO:tasks.workunit.client.1.smithi069.stdout:5/15: stat d1/f3 0 2024-03-22T07:15:54.099 INFO:tasks.workunit.client.0.smithi063.stdout:8/335: creat d0/d24/f5d x:0 0 0 2024-03-22T07:15:54.100 INFO:tasks.workunit.client.1.smithi069.stdout:7/46: mkdir d9/db 0 2024-03-22T07:15:54.105 INFO:tasks.workunit.client.0.smithi063.stdout:7/230: rename d5/d14/c44 to d5/d16/d1b/c4d 0 2024-03-22T07:15:54.108 INFO:tasks.workunit.client.1.smithi069.stdout:1/46: fdatasync d1/f7 0 2024-03-22T07:15:54.108 INFO:tasks.workunit.client.1.smithi069.stdout:1/47: dread - d1/fa zero size 2024-03-22T07:15:54.111 INFO:tasks.workunit.client.0.smithi063.stdout:3/282: dwrite da/f2f [0,4194304] 0 2024-03-22T07:15:54.111 INFO:tasks.workunit.client.0.smithi063.stdout:3/283: write da/f57 [470973,126802] 0 2024-03-22T07:15:54.111 INFO:tasks.workunit.client.0.smithi063.stdout:3/284: chown da/d16/d3c/d4b 3393 1 2024-03-22T07:15:54.111 INFO:tasks.workunit.client.0.smithi063.stdout:3/285: readlink da/l51 0 2024-03-22T07:15:54.111 INFO:tasks.workunit.client.0.smithi063.stdout:3/286: write da/f3a [1825162,78242] 0 2024-03-22T07:15:54.113 INFO:tasks.workunit.client.0.smithi063.stdout:3/287: creat da/d16/d3c/f5e x:0 0 0 2024-03-22T07:15:54.113 INFO:tasks.workunit.client.1.smithi069.stdout:1/48: write d1/f6 [1134188,45558] 0 2024-03-22T07:15:54.114 INFO:tasks.workunit.client.1.smithi069.stdout:7/47: mknod d9/db/cc 0 2024-03-22T07:15:54.114 INFO:tasks.workunit.client.1.smithi069.stdout:5/16: mknod d1/c4 0 2024-03-22T07:15:54.114 INFO:tasks.workunit.client.1.smithi069.stdout:5/17: stat d1/f2 0 2024-03-22T07:15:54.114 INFO:tasks.workunit.client.1.smithi069.stdout:4/37: dread d6/f9 [0,4194304] 0 2024-03-22T07:15:54.114 INFO:tasks.workunit.client.1.smithi069.stdout:0/59: dwrite d3/f8 [0,4194304] 0 2024-03-22T07:15:54.115 INFO:tasks.workunit.client.1.smithi069.stdout:4/38: fsync f3 0 2024-03-22T07:15:54.115 INFO:tasks.workunit.client.0.smithi063.stdout:3/288: truncate da/fb 960426 0 2024-03-22T07:15:54.117 INFO:tasks.workunit.client.1.smithi069.stdout:7/48: symlink d9/ld 0 2024-03-22T07:15:54.117 INFO:tasks.workunit.client.1.smithi069.stdout:1/49: rename d1/ld to d1/le 0 2024-03-22T07:15:54.119 INFO:tasks.workunit.client.1.smithi069.stdout:4/39: mkdir d6/dc/dd 0 2024-03-22T07:15:54.120 INFO:tasks.workunit.client.1.smithi069.stdout:7/49: creat d9/fe x:0 0 0 2024-03-22T07:15:54.120 INFO:tasks.workunit.client.1.smithi069.stdout:7/50: truncate f4 905418 0 2024-03-22T07:15:54.122 INFO:tasks.workunit.client.1.smithi069.stdout:4/40: rmdir d6/dc/dd 0 2024-03-22T07:15:54.125 INFO:tasks.workunit.client.1.smithi069.stdout:7/51: creat d9/db/ff x:0 0 0 2024-03-22T07:15:54.125 INFO:tasks.workunit.client.1.smithi069.stdout:7/52: dread - f3 zero size 2024-03-22T07:15:54.125 INFO:tasks.workunit.client.1.smithi069.stdout:7/53: readlink d9/ld 0 2024-03-22T07:15:54.127 INFO:tasks.workunit.client.1.smithi069.stdout:2/38: write f9 [1251177,47141] 0 2024-03-22T07:15:54.127 INFO:tasks.workunit.client.1.smithi069.stdout:4/41: symlink d6/le 0 2024-03-22T07:15:54.128 INFO:tasks.workunit.client.1.smithi069.stdout:4/42: read f5 [7305,52392] 0 2024-03-22T07:15:54.130 INFO:tasks.workunit.client.1.smithi069.stdout:2/39: link f5 ff 0 2024-03-22T07:15:54.132 INFO:tasks.workunit.client.1.smithi069.stdout:2/40: symlink l10 0 2024-03-22T07:15:54.138 INFO:tasks.workunit.client.1.smithi069.stdout:5/18: write d1/f2 [3086771,89539] 0 2024-03-22T07:15:54.139 INFO:tasks.workunit.client.1.smithi069.stdout:5/19: write d1/f3 [1521167,62434] 0 2024-03-22T07:15:54.149 INFO:tasks.workunit.client.0.smithi063.stdout:7/231: read d5/da/ff [6740272,56050] 0 2024-03-22T07:15:54.150 INFO:tasks.workunit.client.0.smithi063.stdout:7/232: write d5/d16/d1b/f2d [449602,36182] 0 2024-03-22T07:15:54.150 INFO:tasks.workunit.client.0.smithi063.stdout:7/233: getdents d5/da 0 2024-03-22T07:15:54.153 INFO:tasks.workunit.client.0.smithi063.stdout:7/234: dread d5/d16/f1e [0,4194304] 0 2024-03-22T07:15:54.176 INFO:tasks.workunit.client.0.smithi063.stdout:1/339: dwrite d4/df/f48 [0,4194304] 0 2024-03-22T07:15:54.178 INFO:tasks.workunit.client.0.smithi063.stdout:1/340: creat d4/df/d13/d39/d6f/f72 x:0 0 0 2024-03-22T07:15:54.179 INFO:tasks.workunit.client.1.smithi069.stdout:3/55: dwrite f2 [0,4194304] 0 2024-03-22T07:15:54.179 INFO:tasks.workunit.client.1.smithi069.stdout:3/56: creat d5/d9/f15 x:0 0 0 2024-03-22T07:15:54.181 INFO:tasks.workunit.client.0.smithi063.stdout:3/289: dwrite da/d16/f25 [0,4194304] 0 2024-03-22T07:15:54.181 INFO:tasks.workunit.client.0.smithi063.stdout:3/290: write da/d16/d3f/f4c [1003329,74164] 0 2024-03-22T07:15:54.181 INFO:tasks.workunit.client.0.smithi063.stdout:8/336: dwrite d0/f1a [0,4194304] 0 2024-03-22T07:15:54.181 INFO:tasks.workunit.client.0.smithi063.stdout:7/235: dwrite d5/d16/d1b/d21/f2a [0,4194304] 0 2024-03-22T07:15:54.181 INFO:tasks.workunit.client.0.smithi063.stdout:5/291: dwrite d8/f12 [0,4194304] 0 2024-03-22T07:15:54.183 INFO:tasks.workunit.client.1.smithi069.stdout:8/34: dwrite d1/f6 [0,4194304] 0 2024-03-22T07:15:54.183 INFO:tasks.workunit.client.0.smithi063.stdout:4/225: dwrite d1/f11 [4194304,4194304] 0 2024-03-22T07:15:54.183 INFO:tasks.workunit.client.1.smithi069.stdout:2/41: dwrite f7 [0,4194304] 0 2024-03-22T07:15:54.183 INFO:tasks.workunit.client.1.smithi069.stdout:4/43: dwrite d6/f9 [4194304,4194304] 0 2024-03-22T07:15:54.184 INFO:tasks.workunit.client.1.smithi069.stdout:4/44: fdatasync f3 0 2024-03-22T07:15:54.184 INFO:tasks.workunit.client.0.smithi063.stdout:3/291: mknod da/d16/d3f/c5f 0 2024-03-22T07:15:54.185 INFO:tasks.workunit.client.1.smithi069.stdout:3/57: symlink d5/l16 0 2024-03-22T07:15:54.186 INFO:tasks.workunit.client.1.smithi069.stdout:2/42: fdatasync f9 0 2024-03-22T07:15:54.187 INFO:tasks.workunit.client.0.smithi063.stdout:8/337: rename d0/d24/d33/d4c/f4e to d0/d24/d33/f5e 0 2024-03-22T07:15:54.189 INFO:tasks.workunit.client.1.smithi069.stdout:8/35: fsync d1/f6 0 2024-03-22T07:15:54.191 INFO:tasks.workunit.client.0.smithi063.stdout:1/341: dread d4/df/f15 [0,4194304] 0 2024-03-22T07:15:54.191 INFO:tasks.workunit.client.0.smithi063.stdout:1/342: readlink d4/l51 0 2024-03-22T07:15:54.193 INFO:tasks.workunit.client.0.smithi063.stdout:1/343: write d4/df/d13/f40 [422206,74031] 0 2024-03-22T07:15:54.193 INFO:tasks.workunit.client.0.smithi063.stdout:1/344: creat d4/f73 x:0 0 0 2024-03-22T07:15:54.194 INFO:tasks.workunit.client.0.smithi063.stdout:7/236: truncate d5/f13 412985 0 2024-03-22T07:15:54.194 INFO:tasks.workunit.client.0.smithi063.stdout:7/237: dread - d5/d14/d1d/f3f zero size 2024-03-22T07:15:54.194 INFO:tasks.workunit.client.0.smithi063.stdout:7/238: readlink d5/le 0 2024-03-22T07:15:54.196 INFO:tasks.workunit.client.0.smithi063.stdout:5/292: mkdir d8/d13/d1e/d31/d58 0 2024-03-22T07:15:54.197 INFO:tasks.workunit.client.1.smithi069.stdout:4/45: creat d6/dc/ff x:0 0 0 2024-03-22T07:15:54.197 INFO:tasks.workunit.client.1.smithi069.stdout:3/58: symlink d5/l17 0 2024-03-22T07:15:54.197 INFO:tasks.workunit.client.1.smithi069.stdout:2/43: rename f7 to f11 0 2024-03-22T07:15:54.198 INFO:tasks.workunit.client.1.smithi069.stdout:1/50: dwrite d1/f6 [0,4194304] 0 2024-03-22T07:15:54.198 INFO:tasks.workunit.client.1.smithi069.stdout:1/51: write d1/f7 [683593,97200] 0 2024-03-22T07:15:54.198 INFO:tasks.workunit.client.1.smithi069.stdout:4/46: mkdir d6/dc/d10 0 2024-03-22T07:15:54.199 INFO:tasks.workunit.client.1.smithi069.stdout:4/47: dread - d6/dc/ff zero size 2024-03-22T07:15:54.199 INFO:tasks.workunit.client.1.smithi069.stdout:5/20: dread d1/f2 [4194304,4194304] 0 2024-03-22T07:15:54.202 INFO:tasks.workunit.client.1.smithi069.stdout:3/59: getdents d5/d9/db 0 2024-03-22T07:15:54.204 INFO:tasks.workunit.client.0.smithi063.stdout:3/292: symlink da/d16/d3f/l60 0 2024-03-22T07:15:54.206 INFO:tasks.workunit.client.0.smithi063.stdout:4/226: rename d1/f6 to d1/d5/d3c/f47 0 2024-03-22T07:15:54.206 INFO:tasks.workunit.client.0.smithi063.stdout:4/227: read d1/f2 [2806531,93452] 0 2024-03-22T07:15:54.207 INFO:tasks.workunit.client.0.smithi063.stdout:8/338: mkdir d0/d1b/d28/d5f 0 2024-03-22T07:15:54.207 INFO:tasks.workunit.client.0.smithi063.stdout:8/339: chown d0 34467 1 2024-03-22T07:15:54.207 INFO:tasks.workunit.client.0.smithi063.stdout:8/340: readlink d0/d18/l55 0 2024-03-22T07:15:54.208 INFO:tasks.workunit.client.1.smithi069.stdout:4/48: mknod d6/dc/d10/c11 0 2024-03-22T07:15:54.208 INFO:tasks.workunit.client.1.smithi069.stdout:4/49: write d6/dc/ff [342757,130058] 0 2024-03-22T07:15:54.209 INFO:tasks.workunit.client.0.smithi063.stdout:7/239: mkdir d5/da/d4e 0 2024-03-22T07:15:54.213 INFO:tasks.workunit.client.0.smithi063.stdout:4/228: symlink d1/d5/l48 0 2024-03-22T07:15:54.213 INFO:tasks.workunit.client.0.smithi063.stdout:4/229: stat d1/d5/d43/f31 0 2024-03-22T07:15:54.215 INFO:tasks.workunit.client.0.smithi063.stdout:8/341: creat d0/d24/d5b/f60 x:0 0 0 2024-03-22T07:15:54.215 INFO:tasks.workunit.client.0.smithi063.stdout:8/342: stat d0/d1f/f5c 0 2024-03-22T07:15:54.216 INFO:tasks.workunit.client.1.smithi069.stdout:7/54: getdents d9/db 0 2024-03-22T07:15:54.216 INFO:tasks.workunit.client.1.smithi069.stdout:7/55: fsync f3 0 2024-03-22T07:15:54.216 INFO:tasks.workunit.client.1.smithi069.stdout:7/56: chown l5 3743 1 2024-03-22T07:15:54.216 INFO:tasks.workunit.client.1.smithi069.stdout:7/57: fsync f3 0 2024-03-22T07:15:54.218 INFO:tasks.workunit.client.1.smithi069.stdout:7/58: mknod d9/c10 0 2024-03-22T07:15:54.224 INFO:tasks.workunit.client.0.smithi063.stdout:1/345: dwrite d4/f1f [0,4194304] 0 2024-03-22T07:15:54.224 INFO:tasks.workunit.client.0.smithi063.stdout:1/346: write d4/df/d13/d2b/d3a/f6e [429631,55652] 0 2024-03-22T07:15:54.225 INFO:tasks.workunit.client.1.smithi069.stdout:3/60: truncate f2 899482 0 2024-03-22T07:15:54.225 INFO:tasks.workunit.client.1.smithi069.stdout:3/61: fdatasync d5/f6 0 2024-03-22T07:15:54.227 INFO:tasks.workunit.client.1.smithi069.stdout:3/62: creat d5/f18 x:0 0 0 2024-03-22T07:15:54.228 INFO:tasks.workunit.client.1.smithi069.stdout:3/63: write d5/f6 [1164117,41897] 0 2024-03-22T07:15:54.230 INFO:tasks.workunit.client.1.smithi069.stdout:3/64: link d5/d9/fd d5/d9/f19 0 2024-03-22T07:15:54.232 INFO:tasks.workunit.client.1.smithi069.stdout:3/65: mkdir d5/d9/d1a 0 2024-03-22T07:15:54.232 INFO:tasks.workunit.client.1.smithi069.stdout:3/66: chown d5/l17 1 1 2024-03-22T07:15:54.232 INFO:tasks.workunit.client.1.smithi069.stdout:3/67: dread - d5/d9/fd zero size 2024-03-22T07:15:54.232 INFO:tasks.workunit.client.1.smithi069.stdout:3/68: mknod d5/d9/d1a/c1b 0 2024-03-22T07:15:54.232 INFO:tasks.workunit.client.1.smithi069.stdout:3/69: readlink d5/l16 0 2024-03-22T07:15:54.251 INFO:tasks.workunit.client.0.smithi063.stdout:5/293: rmdir d8/d13 39 2024-03-22T07:15:54.252 INFO:tasks.workunit.client.1.smithi069.stdout:8/36: dwrite d1/f2 [0,4194304] 0 2024-03-22T07:15:54.254 INFO:tasks.workunit.client.1.smithi069.stdout:8/37: symlink d1/l7 0 2024-03-22T07:15:54.255 INFO:tasks.workunit.client.0.smithi063.stdout:4/230: dwrite d1/d5/f12 [4194304,4194304] 0 2024-03-22T07:15:54.256 INFO:tasks.workunit.client.0.smithi063.stdout:4/231: chown d1/d5/f23 23498543 1 2024-03-22T07:15:54.257 INFO:tasks.workunit.client.0.smithi063.stdout:4/232: creat d1/d5/d38/f49 x:0 0 0 2024-03-22T07:15:54.258 INFO:tasks.workunit.client.0.smithi063.stdout:4/233: mkdir d1/d5/d43/d27/d2c/d4a 0 2024-03-22T07:15:54.260 INFO:tasks.workunit.client.0.smithi063.stdout:3/293: dwrite da/d16/d3c/f5e [0,4194304] 0 2024-03-22T07:15:54.260 INFO:tasks.workunit.client.0.smithi063.stdout:3/294: chown da/d16/d26/c59 27 1 2024-03-22T07:15:54.260 INFO:tasks.workunit.client.0.smithi063.stdout:5/294: dread d8/f2b [0,4194304] 0 2024-03-22T07:15:54.260 INFO:tasks.workunit.client.0.smithi063.stdout:5/295: write d8/d13/d1e/f38 [1018680,24258] 0 2024-03-22T07:15:54.261 INFO:tasks.workunit.client.1.smithi069.stdout:2/44: dwrite f6 [0,4194304] 0 2024-03-22T07:15:54.261 INFO:tasks.workunit.client.1.smithi069.stdout:2/45: stat f9 0 2024-03-22T07:15:54.261 INFO:tasks.workunit.client.1.smithi069.stdout:1/52: dwrite d1/fa [0,4194304] 0 2024-03-22T07:15:54.261 INFO:tasks.workunit.client.0.smithi063.stdout:1/347: dwrite d4/df/f15 [0,4194304] 0 2024-03-22T07:15:54.261 INFO:tasks.workunit.client.0.smithi063.stdout:1/348: fsync d4/f38 0 2024-03-22T07:15:54.261 INFO:tasks.workunit.client.0.smithi063.stdout:7/240: dwrite d5/da/ff [4194304,4194304] 0 2024-03-22T07:15:54.262 INFO:tasks.workunit.client.0.smithi063.stdout:8/343: dwrite d0/d1b/d28/f3b [0,4194304] 0 2024-03-22T07:15:54.265 INFO:tasks.workunit.client.1.smithi069.stdout:2/46: rename f9 to f12 0 2024-03-22T07:15:54.266 INFO:tasks.workunit.client.0.smithi063.stdout:3/295: rename da/cf to da/d16/d3c/d4b/c61 0 2024-03-22T07:15:54.266 INFO:tasks.workunit.client.1.smithi069.stdout:3/70: dread d5/f6 [0,4194304] 0 2024-03-22T07:15:54.269 INFO:tasks.workunit.client.1.smithi069.stdout:3/71: dread d5/d9/db/fc [0,4194304] 0 2024-03-22T07:15:54.269 INFO:tasks.workunit.client.1.smithi069.stdout:3/72: truncate d5/d9/fd 360581 0 2024-03-22T07:15:54.269 INFO:tasks.workunit.client.1.smithi069.stdout:3/73: chown d5/l11 2909597 1 2024-03-22T07:15:54.270 INFO:tasks.workunit.client.0.smithi063.stdout:7/241: mkdir d5/d14/d1d/d4f 0 2024-03-22T07:15:54.270 INFO:tasks.workunit.client.0.smithi063.stdout:8/344: dread d0/f19 [0,4194304] 0 2024-03-22T07:15:54.271 INFO:tasks.workunit.client.1.smithi069.stdout:4/50: getdents d6/dc/d10 0 2024-03-22T07:15:54.273 INFO:tasks.workunit.client.1.smithi069.stdout:5/21: truncate d1/f3 1403975 0 2024-03-22T07:15:54.273 INFO:tasks.workunit.client.0.smithi063.stdout:1/349: getdents d4/df/d13/d39/d6f 0 2024-03-22T07:15:54.439 INFO:tasks.workunit.client.0.smithi063.stdout:1/350: write d4/d14/d57/f58 [1210115,53469] 0 2024-03-22T07:15:54.440 INFO:tasks.workunit.client.0.smithi063.stdout:1/351: creat d4/df/d13/d2b/d3a/f74 x:0 0 0 2024-03-22T07:15:54.452 INFO:tasks.workunit.client.1.smithi069.stdout:3/74: dread d5/f7 [0,4194304] 0 2024-03-22T07:15:54.452 INFO:tasks.workunit.client.1.smithi069.stdout:3/75: read d5/d9/db/fc [356647,19889] 0 2024-03-22T07:15:54.452 INFO:tasks.workunit.client.1.smithi069.stdout:1/53: symlink d1/lf 0 2024-03-22T07:15:54.453 INFO:tasks.workunit.client.1.smithi069.stdout:3/76: dread d5/f6 [0,4194304] 0 2024-03-22T07:15:54.453 INFO:tasks.workunit.client.1.smithi069.stdout:2/47: mknod c13 0 2024-03-22T07:15:54.453 INFO:tasks.workunit.client.1.smithi069.stdout:5/22: link d1/c4 d1/c5 0 2024-03-22T07:15:54.453 INFO:tasks.workunit.client.1.smithi069.stdout:1/54: dread d1/f7 [0,4194304] 0 2024-03-22T07:15:54.453 INFO:tasks.workunit.client.1.smithi069.stdout:1/55: chown d1/l9 3 1 2024-03-22T07:15:54.453 INFO:tasks.workunit.client.1.smithi069.stdout:3/77: rename d5/d9/db/fc to d5/d9/db/f1c 0 2024-03-22T07:15:54.453 INFO:tasks.workunit.client.1.smithi069.stdout:4/51: mkdir d6/d12 0 2024-03-22T07:15:54.453 INFO:tasks.workunit.client.1.smithi069.stdout:5/23: write d1/f2 [5963761,7448] 0 2024-03-22T07:15:54.453 INFO:tasks.workunit.client.1.smithi069.stdout:5/24: write d1/f2 [6594786,81488] 0 2024-03-22T07:15:54.454 INFO:tasks.workunit.client.1.smithi069.stdout:2/48: write f3 [2076957,61807] 0 2024-03-22T07:15:54.454 INFO:tasks.workunit.client.1.smithi069.stdout:3/78: rename d5/lf to d5/d9/l1d 0 2024-03-22T07:15:54.454 INFO:tasks.workunit.client.1.smithi069.stdout:5/25: write d1/f3 [2382522,111658] 0 2024-03-22T07:15:54.454 INFO:tasks.workunit.client.1.smithi069.stdout:5/26: readlink - no filename 2024-03-22T07:15:54.454 INFO:tasks.workunit.client.1.smithi069.stdout:1/56: dread d1/fa [0,4194304] 0 2024-03-22T07:15:54.454 INFO:tasks.workunit.client.1.smithi069.stdout:1/57: write d1/f7 [1675790,3664] 0 2024-03-22T07:15:54.454 INFO:tasks.workunit.client.1.smithi069.stdout:1/58: readlink d1/lf 0 2024-03-22T07:15:54.454 INFO:tasks.workunit.client.1.smithi069.stdout:1/59: fsync d1/f3 0 2024-03-22T07:15:54.454 INFO:tasks.workunit.client.1.smithi069.stdout:3/79: rename d5/d9/d1a to d5/d9/d1e 0 2024-03-22T07:15:54.454 INFO:tasks.workunit.client.1.smithi069.stdout:3/80: dread d5/d9/f19 [0,4194304] 0 2024-03-22T07:15:54.455 INFO:tasks.workunit.client.1.smithi069.stdout:3/81: dread - d5/d9/fe zero size 2024-03-22T07:15:54.455 INFO:tasks.workunit.client.1.smithi069.stdout:3/82: symlink d5/d9/l1f 0 2024-03-22T07:15:54.455 INFO:tasks.workunit.client.1.smithi069.stdout:2/49: dread f11 [0,4194304] 0 2024-03-22T07:15:54.455 INFO:tasks.workunit.client.1.smithi069.stdout:2/50: link cb c14 0 2024-03-22T07:15:54.455 INFO:tasks.workunit.client.1.smithi069.stdout:5/27: fdatasync d1/f3 0 2024-03-22T07:15:54.455 INFO:tasks.workunit.client.1.smithi069.stdout:5/28: fdatasync d1/f3 0 2024-03-22T07:15:54.455 INFO:tasks.workunit.client.1.smithi069.stdout:4/52: dwrite f3 [0,4194304] 0 2024-03-22T07:15:54.455 INFO:tasks.workunit.client.1.smithi069.stdout:2/51: dwrite f6 [4194304,4194304] 0 2024-03-22T07:15:54.455 INFO:tasks.workunit.client.1.smithi069.stdout:1/60: dwrite d1/fa [0,4194304] 0 2024-03-22T07:15:54.455 INFO:tasks.workunit.client.1.smithi069.stdout:3/83: getdents d5 0 2024-03-22T07:15:54.455 INFO:tasks.workunit.client.1.smithi069.stdout:1/61: write d1/f7 [1038769,121410] 0 2024-03-22T07:15:54.456 INFO:tasks.workunit.client.1.smithi069.stdout:1/62: write d1/f3 [2595020,53843] 0 2024-03-22T07:15:54.456 INFO:tasks.workunit.client.1.smithi069.stdout:1/63: symlink d1/l10 0 2024-03-22T07:15:54.456 INFO:tasks.workunit.client.1.smithi069.stdout:1/64: rename d1/lc to d1/l11 0 2024-03-22T07:15:54.456 INFO:tasks.workunit.client.1.smithi069.stdout:5/29: dwrite d1/f2 [0,4194304] 0 2024-03-22T07:15:54.457 INFO:tasks.workunit.client.1.smithi069.stdout:2/52: dwrite f3 [0,4194304] 0 2024-03-22T07:15:54.457 INFO:tasks.workunit.client.1.smithi069.stdout:2/53: creat f15 x:0 0 0 2024-03-22T07:15:54.457 INFO:tasks.workunit.client.1.smithi069.stdout:2/54: chown fa 12665290 1 2024-03-22T07:15:54.457 INFO:tasks.workunit.client.1.smithi069.stdout:5/30: mkdir d1/d6 0 2024-03-22T07:15:54.457 INFO:tasks.workunit.client.1.smithi069.stdout:4/53: dwrite d6/f9 [0,4194304] 0 2024-03-22T07:15:54.457 INFO:tasks.workunit.client.1.smithi069.stdout:4/54: chown f5 8713512 1 2024-03-22T07:15:54.457 INFO:tasks.workunit.client.1.smithi069.stdout:4/55: fsync d6/dc/ff 0 2024-03-22T07:15:54.457 INFO:tasks.workunit.client.1.smithi069.stdout:2/55: symlink l16 0 2024-03-22T07:15:54.457 INFO:tasks.workunit.client.1.smithi069.stdout:6/41: sync 2024-03-22T07:15:54.464 INFO:tasks.workunit.client.1.smithi069.stdout:4/56: symlink d6/dc/l13 0 2024-03-22T07:15:54.464 INFO:tasks.workunit.client.1.smithi069.stdout:6/42: mkdir d4/dd 0 2024-03-22T07:15:54.465 INFO:tasks.workunit.client.1.smithi069.stdout:6/43: truncate d4/fb 812711 0 2024-03-22T07:15:54.465 INFO:tasks.workunit.client.1.smithi069.stdout:2/56: read f3 [3910510,97546] 0 2024-03-22T07:15:54.465 INFO:tasks.workunit.client.1.smithi069.stdout:2/57: mknod c17 0 2024-03-22T07:15:54.465 INFO:tasks.workunit.client.1.smithi069.stdout:2/58: stat f11 0 2024-03-22T07:15:54.465 INFO:tasks.workunit.client.1.smithi069.stdout:2/59: mkdir d18 0 2024-03-22T07:15:54.465 INFO:tasks.workunit.client.1.smithi069.stdout:2/60: creat d18/f19 x:0 0 0 2024-03-22T07:15:54.465 INFO:tasks.workunit.client.1.smithi069.stdout:2/61: truncate ff 871884 0 2024-03-22T07:15:54.465 INFO:tasks.workunit.client.1.smithi069.stdout:2/62: symlink d18/l1a 0 2024-03-22T07:15:54.465 INFO:tasks.workunit.client.1.smithi069.stdout:2/63: creat d18/f1b x:0 0 0 2024-03-22T07:15:54.472 INFO:tasks.workunit.client.1.smithi069.stdout:2/64: dread f5 [0,4194304] 0 2024-03-22T07:15:54.474 INFO:tasks.workunit.client.1.smithi069.stdout:4/57: dread d6/f9 [4194304,4194304] 0 2024-03-22T07:15:54.474 INFO:tasks.workunit.client.1.smithi069.stdout:4/58: creat d6/dc/d10/f14 x:0 0 0 2024-03-22T07:15:54.477 INFO:tasks.workunit.client.0.smithi063.stdout:3/296: dread da/d16/d26/f29 [0,4194304] 0 2024-03-22T07:15:54.478 INFO:tasks.workunit.client.0.smithi063.stdout:3/297: fsync da/d16/f5c 0 2024-03-22T07:15:54.478 INFO:tasks.workunit.client.0.smithi063.stdout:7/242: link d5/d16/c24 d5/d14/d1d/d29/d3b/c50 0 2024-03-22T07:15:54.478 INFO:tasks.workunit.client.0.smithi063.stdout:5/296: write d8/d13/d1e/f27 [1975417,39341] 0 2024-03-22T07:15:54.479 INFO:tasks.workunit.client.0.smithi063.stdout:8/345: rename d0/d24/d33/c50 to d0/d26/d2d/d59/c61 0 2024-03-22T07:15:54.479 INFO:tasks.workunit.client.0.smithi063.stdout:6/316: sync 2024-03-22T07:15:54.479 INFO:tasks.workunit.client.0.smithi063.stdout:2/268: sync 2024-03-22T07:15:54.479 INFO:tasks.workunit.client.0.smithi063.stdout:0/340: sync 2024-03-22T07:15:54.479 INFO:tasks.workunit.client.0.smithi063.stdout:0/341: readlink da/de/d17/l59 0 2024-03-22T07:15:54.479 INFO:tasks.workunit.client.0.smithi063.stdout:1/352: symlink d4/d5a/l75 0 2024-03-22T07:15:54.479 INFO:tasks.workunit.client.0.smithi063.stdout:8/346: creat d0/f62 x:0 0 0 2024-03-22T07:15:54.479 INFO:tasks.workunit.client.0.smithi063.stdout:9/220: sync 2024-03-22T07:15:54.480 INFO:tasks.workunit.client.0.smithi063.stdout:9/221: truncate d8/d19/d20/d22/d29/d31/f32 438965 0 2024-03-22T07:15:54.480 INFO:tasks.workunit.client.0.smithi063.stdout:2/269: write d0/de/d35/d36/f3a [1045172,79754] 0 2024-03-22T07:15:54.480 INFO:tasks.workunit.client.0.smithi063.stdout:2/270: fdatasync d0/f8 0 2024-03-22T07:15:54.480 INFO:tasks.workunit.client.0.smithi063.stdout:1/353: rename d4/d14/d57/f58 to d4/f76 0 2024-03-22T07:15:54.480 INFO:tasks.workunit.client.0.smithi063.stdout:6/317: symlink d3/l6a 0 2024-03-22T07:15:54.480 INFO:tasks.workunit.client.0.smithi063.stdout:6/318: fdatasync d3/d6/d35/d3a/f40 0 2024-03-22T07:15:54.480 INFO:tasks.workunit.client.0.smithi063.stdout:6/319: truncate d3/d15/d29/d4e/f50 9097175 0 2024-03-22T07:15:54.480 INFO:tasks.workunit.client.0.smithi063.stdout:6/320: fsync d3/d6/d2a/d38/f61 0 2024-03-22T07:15:54.480 INFO:tasks.workunit.client.0.smithi063.stdout:6/321: chown d3/d15/d29/d4c/l2f 1476917558 1 2024-03-22T07:15:54.480 INFO:tasks.workunit.client.0.smithi063.stdout:6/322: truncate d3/d6/f10 3851891 0 2024-03-22T07:15:54.481 INFO:tasks.workunit.client.0.smithi063.stdout:6/323: rename d3/d6 to d3/d6/d2a/d2e/d6b 22 2024-03-22T07:15:54.481 INFO:tasks.workunit.client.0.smithi063.stdout:2/271: rmdir d0/de/d1b 39 2024-03-22T07:15:54.481 INFO:tasks.workunit.client.0.smithi063.stdout:1/354: getdents d4/df/d13/d39 0 2024-03-22T07:15:54.481 INFO:tasks.workunit.client.0.smithi063.stdout:1/355: fsync d4/df/f44 0 2024-03-22T07:15:54.481 INFO:tasks.workunit.client.0.smithi063.stdout:1/356: chown d4/df/d13/d2b/d3a/f74 1 1 2024-03-22T07:15:54.481 INFO:tasks.workunit.client.0.smithi063.stdout:1/357: fsync d4/df/d13/d39/f6b 0 2024-03-22T07:15:54.481 INFO:tasks.workunit.client.0.smithi063.stdout:1/358: creat d4/df/d13/d39/d63/f77 x:0 0 0 2024-03-22T07:15:54.481 INFO:tasks.workunit.client.0.smithi063.stdout:2/272: mknod d0/c56 0 2024-03-22T07:15:54.481 INFO:tasks.workunit.client.0.smithi063.stdout:1/359: rename d4/df/d2c/f36 to d4/d71/f78 0 2024-03-22T07:15:54.481 INFO:tasks.workunit.client.0.smithi063.stdout:2/273: mknod d0/de/d1b/c57 0 2024-03-22T07:15:54.481 INFO:tasks.workunit.client.0.smithi063.stdout:1/360: symlink d4/d14/d57/l79 0 2024-03-22T07:15:54.481 INFO:tasks.workunit.client.0.smithi063.stdout:2/274: mkdir d0/de/d58 0 2024-03-22T07:15:54.482 INFO:tasks.workunit.client.0.smithi063.stdout:1/361: symlink d4/d5a/l7a 0 2024-03-22T07:15:54.482 INFO:tasks.workunit.client.0.smithi063.stdout:2/275: mknod d0/de/d58/c59 0 2024-03-22T07:15:54.482 INFO:tasks.workunit.client.0.smithi063.stdout:1/362: symlink d4/df/d13/d2b/d3a/l7b 0 2024-03-22T07:15:54.482 INFO:tasks.workunit.client.0.smithi063.stdout:2/276: readlink d0/l17 0 2024-03-22T07:15:54.482 INFO:tasks.workunit.client.0.smithi063.stdout:2/277: chown d0/f13 9077412 1 2024-03-22T07:15:54.482 INFO:tasks.workunit.client.0.smithi063.stdout:2/278: mkdir d0/de/d12/d5a 0 2024-03-22T07:15:54.482 INFO:tasks.workunit.client.0.smithi063.stdout:2/279: fsync d0/de/d12/d20/f2a 0 2024-03-22T07:15:54.483 INFO:tasks.workunit.client.0.smithi063.stdout:2/280: stat d0/l17 0 2024-03-22T07:15:54.483 INFO:tasks.workunit.client.0.smithi063.stdout:1/363: dread d4/df/d13/d2b/d3a/f6e [0,4194304] 0 2024-03-22T07:15:54.483 INFO:tasks.workunit.client.0.smithi063.stdout:2/281: symlink d0/de/l5b 0 2024-03-22T07:15:54.483 INFO:tasks.workunit.client.0.smithi063.stdout:2/282: fsync d0/de/d1b/f50 0 2024-03-22T07:15:54.483 INFO:tasks.workunit.client.0.smithi063.stdout:2/283: write d0/de/d12/d20/f2a [2451536,95702] 0 2024-03-22T07:15:54.483 INFO:tasks.workunit.client.0.smithi063.stdout:1/364: mknod d4/df/d13/d2b/d6c/c7c 0 2024-03-22T07:15:54.483 INFO:tasks.workunit.client.0.smithi063.stdout:2/284: symlink d0/de/d12/d5a/l5c 0 2024-03-22T07:15:54.483 INFO:tasks.workunit.client.0.smithi063.stdout:2/285: chown d0/de/d1b/l33 160 1 2024-03-22T07:15:54.483 INFO:tasks.workunit.client.0.smithi063.stdout:8/347: write d0/f1a [1860521,31537] 0 2024-03-22T07:15:54.483 INFO:tasks.workunit.client.0.smithi063.stdout:2/286: link d0/de/d1b/d32/f3b d0/de/d12/d20/f5d 0 2024-03-22T07:15:54.483 INFO:tasks.workunit.client.0.smithi063.stdout:8/348: mknod d0/d1b/d28/d5f/c63 0 2024-03-22T07:15:54.483 INFO:tasks.workunit.client.0.smithi063.stdout:8/349: stat d0/d26/f34 0 2024-03-22T07:15:54.484 INFO:tasks.workunit.client.0.smithi063.stdout:1/365: dread d4/fb [0,4194304] 0 2024-03-22T07:15:54.484 INFO:tasks.workunit.client.0.smithi063.stdout:1/366: rename d4/df/d13/d2b/d6c/c7c to d4/df/c7d 0 2024-03-22T07:15:54.484 INFO:tasks.workunit.client.0.smithi063.stdout:1/367: symlink d4/df/l7e 0 2024-03-22T07:15:54.484 INFO:tasks.workunit.client.0.smithi063.stdout:3/298: dwrite da/d16/f5c [0,4194304] 0 2024-03-22T07:15:54.484 INFO:tasks.workunit.client.0.smithi063.stdout:3/299: dread da/f21 [0,4194304] 0 2024-03-22T07:15:54.484 INFO:tasks.workunit.client.0.smithi063.stdout:3/300: mknod da/d16/d3f/c62 0 2024-03-22T07:15:54.484 INFO:tasks.workunit.client.0.smithi063.stdout:1/368: dread d4/d14/f1b [0,4194304] 0 2024-03-22T07:15:54.484 INFO:tasks.workunit.client.0.smithi063.stdout:1/369: write d4/d5a/f65 [3077281,46269] 0 2024-03-22T07:15:54.484 INFO:tasks.workunit.client.0.smithi063.stdout:1/370: write d4/df/d13/d39/d63/f66 [880074,2249] 0 2024-03-22T07:15:54.484 INFO:tasks.workunit.client.0.smithi063.stdout:1/371: chown d4/df/d13/d2b/d3a/f70 1598424 1 2024-03-22T07:15:54.484 INFO:tasks.workunit.client.0.smithi063.stdout:1/372: write d4/f76 [84010,24030] 0 2024-03-22T07:15:54.484 INFO:tasks.workunit.client.0.smithi063.stdout:3/301: write da/f2f [3757458,97544] 0 2024-03-22T07:15:54.484 INFO:tasks.workunit.client.0.smithi063.stdout:0/342: dwrite da/de/d60/f55 [0,4194304] 0 2024-03-22T07:15:54.484 INFO:tasks.workunit.client.0.smithi063.stdout:0/343: creat da/de/d39/f72 x:0 0 0 2024-03-22T07:15:54.485 INFO:tasks.workunit.client.0.smithi063.stdout:7/243: dwrite d5/d14/d1d/f3f [0,4194304] 0 2024-03-22T07:15:54.485 INFO:tasks.workunit.client.0.smithi063.stdout:7/244: unlink d5/d16/l3a 0 2024-03-22T07:15:54.485 INFO:tasks.workunit.client.0.smithi063.stdout:8/350: rmdir d0/d26 39 2024-03-22T07:15:54.485 INFO:tasks.workunit.client.0.smithi063.stdout:8/351: creat d0/f64 x:0 0 0 2024-03-22T07:15:54.485 INFO:tasks.workunit.client.0.smithi063.stdout:8/352: rmdir d0/d24 39 2024-03-22T07:15:54.485 INFO:tasks.workunit.client.0.smithi063.stdout:9/222: dwrite d8/d15/f1c [0,4194304] 0 2024-03-22T07:15:54.485 INFO:tasks.workunit.client.0.smithi063.stdout:2/287: dwrite d0/de/f4c [0,4194304] 0 2024-03-22T07:15:54.485 INFO:tasks.workunit.client.0.smithi063.stdout:2/288: chown d0/de/d12/d20/c45 191 1 2024-03-22T07:15:54.485 INFO:tasks.workunit.client.0.smithi063.stdout:9/223: mkdir d8/d3e 0 2024-03-22T07:15:54.485 INFO:tasks.workunit.client.0.smithi063.stdout:9/224: dread d8/d15/f2a [0,4194304] 0 2024-03-22T07:15:54.486 INFO:tasks.workunit.client.0.smithi063.stdout:2/289: truncate d0/fb 4624611 0 2024-03-22T07:15:54.486 INFO:tasks.workunit.client.0.smithi063.stdout:9/225: unlink d8/fc 0 2024-03-22T07:15:54.486 INFO:tasks.workunit.client.0.smithi063.stdout:6/324: dwrite d3/d6/f3e [0,4194304] 0 2024-03-22T07:15:54.486 INFO:tasks.workunit.client.0.smithi063.stdout:2/290: symlink d0/de/d1b/d32/l5e 0 2024-03-22T07:15:54.486 INFO:tasks.workunit.client.0.smithi063.stdout:1/373: dwrite d4/df/d13/d39/d63/f77 [0,4194304] 0 2024-03-22T07:15:54.486 INFO:tasks.workunit.client.0.smithi063.stdout:1/374: fdatasync d4/f73 0 2024-03-22T07:15:54.486 INFO:tasks.workunit.client.0.smithi063.stdout:7/245: dwrite d5/d16/f2b [4194304,4194304] 0 2024-03-22T07:15:54.486 INFO:tasks.workunit.client.0.smithi063.stdout:7/246: creat d5/d14/d1d/f51 x:0 0 0 2024-03-22T07:15:54.486 INFO:tasks.workunit.client.0.smithi063.stdout:9/226: mknod d8/d15/c3f 0 2024-03-22T07:15:54.487 INFO:tasks.workunit.client.0.smithi063.stdout:2/291: rename d0/de/d12/l2b to d0/de/d12/d5a/l5f 0 2024-03-22T07:15:54.487 INFO:tasks.workunit.client.0.smithi063.stdout:3/302: dwrite da/f3d [0,4194304] 0 2024-03-22T07:15:54.487 INFO:tasks.workunit.client.0.smithi063.stdout:8/353: dwrite d0/d26/d2d/f4d [0,4194304] 0 2024-03-22T07:15:54.487 INFO:tasks.workunit.client.0.smithi063.stdout:0/344: dwrite da/de/d17/d25/f3e [0,4194304] 0 2024-03-22T07:15:54.487 INFO:tasks.workunit.client.0.smithi063.stdout:9/227: unlink d8/d19/f3d 0 2024-03-22T07:15:54.487 INFO:tasks.workunit.client.0.smithi063.stdout:9/228: truncate d8/d17/f1d 4995543 0 2024-03-22T07:15:54.487 INFO:tasks.workunit.client.0.smithi063.stdout:1/375: write d4/d14/f21 [1966180,68022] 0 2024-03-22T07:15:54.487 INFO:tasks.workunit.client.0.smithi063.stdout:1/376: write d4/d14/f21 [5043293,114233] 0 2024-03-22T07:15:54.488 INFO:tasks.workunit.client.0.smithi063.stdout:8/354: creat d0/d26/d2d/d59/f65 x:0 0 0 2024-03-22T07:15:54.488 INFO:tasks.workunit.client.0.smithi063.stdout:8/355: readlink d0/d26/l4b 0 2024-03-22T07:15:54.488 INFO:tasks.workunit.client.0.smithi063.stdout:0/345: unlink da/de/d17/d25/d3c/d43/f5b 0 2024-03-22T07:15:54.488 INFO:tasks.workunit.client.0.smithi063.stdout:9/229: mknod d8/d19/d2b/c40 0 2024-03-22T07:15:54.488 INFO:tasks.workunit.client.0.smithi063.stdout:9/230: chown d8/d15/f2a 1 1 2024-03-22T07:15:54.488 INFO:tasks.workunit.client.0.smithi063.stdout:9/231: chown d8/d17/d1e 1238 1 2024-03-22T07:15:54.488 INFO:tasks.workunit.client.0.smithi063.stdout:8/356: write d0/d1f/f53 [1422470,11327] 0 2024-03-22T07:15:54.488 INFO:tasks.workunit.client.0.smithi063.stdout:1/377: symlink d4/d5a/l7f 0 2024-03-22T07:15:54.488 INFO:tasks.workunit.client.0.smithi063.stdout:1/378: write d4/f59 [577471,10029] 0 2024-03-22T07:15:54.489 INFO:tasks.workunit.client.0.smithi063.stdout:1/379: chown d4/df/d13/d39/d63 38938 1 2024-03-22T07:15:54.489 INFO:tasks.workunit.client.0.smithi063.stdout:1/380: write d4/d14/f1b [758315,96009] 0 2024-03-22T07:15:54.489 INFO:tasks.workunit.client.0.smithi063.stdout:2/292: link d0/de/d12/f27 d0/f60 0 2024-03-22T07:15:54.489 INFO:tasks.workunit.client.0.smithi063.stdout:0/346: mknod da/de/d17/d25/d3c/d64/c73 0 2024-03-22T07:15:54.489 INFO:tasks.workunit.client.0.smithi063.stdout:0/347: write da/de/d17/f1a [189580,39412] 0 2024-03-22T07:15:54.489 INFO:tasks.workunit.client.0.smithi063.stdout:9/232: mkdir d8/d19/d41 0 2024-03-22T07:15:54.489 INFO:tasks.workunit.client.0.smithi063.stdout:0/348: dread da/de/f31 [4194304,4194304] 0 2024-03-22T07:15:54.489 INFO:tasks.workunit.client.0.smithi063.stdout:0/349: dread - da/de/d17/d25/f65 zero size 2024-03-22T07:15:54.490 INFO:tasks.workunit.client.0.smithi063.stdout:0/350: stat da/de/d17/f21 0 2024-03-22T07:15:54.490 INFO:tasks.workunit.client.0.smithi063.stdout:0/351: write da/de/d17/d4e/f57 [941748,13470] 0 2024-03-22T07:15:54.490 INFO:tasks.workunit.client.0.smithi063.stdout:0/352: write da/d49/f63 [410993,85810] 0 2024-03-22T07:15:54.490 INFO:tasks.workunit.client.0.smithi063.stdout:0/353: truncate da/de/d17/d25/d3c/d5f/f52 448562 0 2024-03-22T07:15:54.490 INFO:tasks.workunit.client.0.smithi063.stdout:2/293: truncate d0/f60 985192 0 2024-03-22T07:15:54.490 INFO:tasks.workunit.client.0.smithi063.stdout:2/294: write d0/de/d12/d20/f47 [4398504,18797] 0 2024-03-22T07:15:54.490 INFO:tasks.workunit.client.0.smithi063.stdout:8/357: symlink d0/l66 0 2024-03-22T07:15:54.490 INFO:tasks.workunit.client.0.smithi063.stdout:8/358: write d0/d26/f34 [4511911,3803] 0 2024-03-22T07:15:54.491 INFO:tasks.workunit.client.0.smithi063.stdout:2/295: write d0/f9 [131490,76336] 0 2024-03-22T07:15:54.491 INFO:tasks.workunit.client.0.smithi063.stdout:3/303: dwrite da/f3a [0,4194304] 0 2024-03-22T07:15:54.491 INFO:tasks.workunit.client.0.smithi063.stdout:8/359: getdents d0/d24/d33 0 2024-03-22T07:15:54.491 INFO:tasks.workunit.client.0.smithi063.stdout:0/354: dread da/de/f23 [0,4194304] 0 2024-03-22T07:15:54.491 INFO:tasks.workunit.client.0.smithi063.stdout:0/355: creat da/d49/f74 x:0 0 0 2024-03-22T07:15:54.491 INFO:tasks.workunit.client.0.smithi063.stdout:2/296: dread d0/de/d1b/d32/f49 [0,4194304] 0 2024-03-22T07:15:54.491 INFO:tasks.workunit.client.0.smithi063.stdout:3/304: dread da/fb [0,4194304] 0 2024-03-22T07:15:54.491 INFO:tasks.workunit.client.0.smithi063.stdout:3/305: fsync da/d16/d26/f27 0 2024-03-22T07:15:54.491 INFO:tasks.workunit.client.0.smithi063.stdout:3/306: write da/d16/d3f/f44 [763468,99852] 0 2024-03-22T07:15:54.491 INFO:tasks.workunit.client.0.smithi063.stdout:8/360: link d0/d1b/d28/d3e/c48 d0/d18/d4a/c67 0 2024-03-22T07:15:54.491 INFO:tasks.workunit.client.0.smithi063.stdout:0/356: symlink da/de/d60/d56/l75 0 2024-03-22T07:15:54.491 INFO:tasks.workunit.client.0.smithi063.stdout:2/297: symlink d0/de/d12/d20/l61 0 2024-03-22T07:15:54.491 INFO:tasks.workunit.client.0.smithi063.stdout:3/307: mkdir da/d63 0 2024-03-22T07:15:54.491 INFO:tasks.workunit.client.0.smithi063.stdout:3/308: chown da/d16/d3c/f5e 6 1 2024-03-22T07:15:54.492 INFO:tasks.workunit.client.0.smithi063.stdout:8/361: mknod d0/d1f/c68 0 2024-03-22T07:15:54.492 INFO:tasks.workunit.client.0.smithi063.stdout:3/309: write da/d16/d26/f48 [57522,76336] 0 2024-03-22T07:15:54.492 INFO:tasks.workunit.client.0.smithi063.stdout:8/362: truncate d0/d26/d2d/f4d 211822 0 2024-03-22T07:15:54.492 INFO:tasks.workunit.client.0.smithi063.stdout:2/298: mknod d0/de/d35/d36/c62 0 2024-03-22T07:15:54.492 INFO:tasks.workunit.client.0.smithi063.stdout:8/363: creat d0/d1b/f69 x:0 0 0 2024-03-22T07:15:54.492 INFO:tasks.workunit.client.0.smithi063.stdout:8/364: write d0/f19 [572580,82871] 0 2024-03-22T07:15:54.492 INFO:tasks.workunit.client.0.smithi063.stdout:8/365: creat d0/d18/d45/f6a x:0 0 0 2024-03-22T07:15:54.492 INFO:tasks.workunit.client.0.smithi063.stdout:0/357: dread da/de/f31 [0,4194304] 0 2024-03-22T07:15:54.495 INFO:tasks.workunit.client.0.smithi063.stdout:0/358: write da/de/d60/f33 [1173775,54955] 0 2024-03-22T07:15:54.495 INFO:tasks.workunit.client.0.smithi063.stdout:3/310: write da/f3d [3992802,126500] 0 2024-03-22T07:15:54.495 INFO:tasks.workunit.client.0.smithi063.stdout:1/381: dwrite d4/df/f48 [0,4194304] 0 2024-03-22T07:15:54.495 INFO:tasks.workunit.client.0.smithi063.stdout:9/233: dwrite d8/d19/d20/d22/f27 [0,4194304] 0 2024-03-22T07:15:54.495 INFO:tasks.workunit.client.0.smithi063.stdout:2/299: write d0/de/f4c [2870048,23077] 0 2024-03-22T07:15:54.495 INFO:tasks.workunit.client.0.smithi063.stdout:8/366: read d0/fc [2194824,63222] 0 2024-03-22T07:15:54.495 INFO:tasks.workunit.client.0.smithi063.stdout:3/311: dread da/d16/d3f/f4c [0,4194304] 0 2024-03-22T07:15:54.495 INFO:tasks.workunit.client.0.smithi063.stdout:3/312: chown da/c43 383736560 1 2024-03-22T07:15:54.496 INFO:tasks.workunit.client.1.smithi069.stdout:4/59: dread d6/dc/ff [0,4194304] 0 2024-03-22T07:15:54.496 INFO:tasks.workunit.client.1.smithi069.stdout:4/60: fsync d6/f9 0 2024-03-22T07:15:54.496 INFO:tasks.workunit.client.1.smithi069.stdout:5/31: dwrite d1/f2 [0,4194304] 0 2024-03-22T07:15:54.496 INFO:tasks.workunit.client.1.smithi069.stdout:1/65: truncate d1/f7 425744 0 2024-03-22T07:15:54.496 INFO:tasks.workunit.client.1.smithi069.stdout:5/32: mkdir d1/d7 0 2024-03-22T07:15:54.496 INFO:tasks.workunit.client.1.smithi069.stdout:6/44: dwrite f3 [0,4194304] 0 2024-03-22T07:15:54.496 INFO:tasks.workunit.client.1.smithi069.stdout:0/60: sync 2024-03-22T07:15:54.496 INFO:tasks.workunit.client.1.smithi069.stdout:7/59: sync 2024-03-22T07:15:54.496 INFO:tasks.workunit.client.1.smithi069.stdout:8/38: sync 2024-03-22T07:15:54.497 INFO:tasks.workunit.client.1.smithi069.stdout:9/50: sync 2024-03-22T07:15:54.497 INFO:tasks.workunit.client.1.smithi069.stdout:1/66: unlink d1/f3 0 2024-03-22T07:15:54.497 INFO:tasks.workunit.client.1.smithi069.stdout:5/33: mkdir d1/d7/d8 0 2024-03-22T07:15:54.502 INFO:tasks.workunit.client.1.smithi069.stdout:2/65: fsync d18/f19 0 2024-03-22T07:15:54.504 INFO:tasks.workunit.client.0.smithi063.stdout:2/300: mkdir d0/de/d1b/d63 0 2024-03-22T07:15:54.504 INFO:tasks.workunit.client.0.smithi063.stdout:2/301: write d0/de/d12/d20/f2a [3326971,100751] 0 2024-03-22T07:15:54.505 INFO:tasks.workunit.client.1.smithi069.stdout:9/51: symlink d2/dd/l15 0 2024-03-22T07:15:54.505 INFO:tasks.workunit.client.1.smithi069.stdout:9/52: creat d2/f16 x:0 0 0 2024-03-22T07:15:54.506 INFO:tasks.workunit.client.0.smithi063.stdout:0/359: dwrite da/de/d17/f28 [0,4194304] 0 2024-03-22T07:15:54.506 INFO:tasks.workunit.client.0.smithi063.stdout:0/360: chown f5 426491 1 2024-03-22T07:15:54.507 INFO:tasks.workunit.client.0.smithi063.stdout:2/302: link d0/de/d35/l4d d0/de/d12/d5a/l64 0 2024-03-22T07:15:54.510 INFO:tasks.workunit.client.1.smithi069.stdout:5/34: rename d1/f2 to d1/f9 0 2024-03-22T07:15:54.510 INFO:tasks.workunit.client.1.smithi069.stdout:5/35: chown d1/f9 9 1 2024-03-22T07:15:54.510 INFO:tasks.workunit.client.1.smithi069.stdout:5/36: fdatasync d1/f3 0 2024-03-22T07:15:54.511 INFO:tasks.workunit.client.1.smithi069.stdout:2/66: mkdir d18/d1c 0 2024-03-22T07:15:54.511 INFO:tasks.workunit.client.1.smithi069.stdout:2/67: truncate f15 643787 0 2024-03-22T07:15:54.511 INFO:tasks.workunit.client.0.smithi063.stdout:0/361: mkdir da/de/d17/d25/d3c/d43/d76 0 2024-03-22T07:15:54.511 INFO:tasks.workunit.client.0.smithi063.stdout:3/313: dread da/d16/d26/f48 [0,4194304] 0 2024-03-22T07:15:54.512 INFO:tasks.workunit.client.0.smithi063.stdout:0/362: readlink da/de/d17/d25/l48 0 2024-03-22T07:15:54.512 INFO:tasks.workunit.client.0.smithi063.stdout:1/382: dread d4/d14/f18 [0,4194304] 0 2024-03-22T07:15:54.512 INFO:tasks.workunit.client.1.smithi069.stdout:0/61: getdents d3/da 0 2024-03-22T07:15:54.514 INFO:tasks.workunit.client.0.smithi063.stdout:3/314: creat da/d16/d3f/f64 x:0 0 0 2024-03-22T07:15:54.514 INFO:tasks.workunit.client.0.smithi063.stdout:2/303: getdents d0/de/d12 0 2024-03-22T07:15:54.514 INFO:tasks.workunit.client.0.smithi063.stdout:2/304: creat d0/de/f65 x:0 0 0 2024-03-22T07:15:54.516 INFO:tasks.workunit.client.1.smithi069.stdout:4/61: dwrite d6/dc/ff [0,4194304] 0 2024-03-22T07:15:54.517 INFO:tasks.workunit.client.0.smithi063.stdout:2/305: truncate d0/f8 2268010 0 2024-03-22T07:15:54.518 INFO:tasks.workunit.client.0.smithi063.stdout:2/306: getdents d0/de/d35 0 2024-03-22T07:15:54.519 INFO:tasks.workunit.client.0.smithi063.stdout:8/367: rmdir d0/d1b 39 2024-03-22T07:15:54.519 INFO:tasks.workunit.client.0.smithi063.stdout:8/368: dread - d0/d18/d45/f46 zero size 2024-03-22T07:15:54.519 INFO:tasks.workunit.client.0.smithi063.stdout:2/307: symlink d0/de/d1b/l66 0 2024-03-22T07:15:54.521 INFO:tasks.workunit.client.1.smithi069.stdout:2/68: rename f11 to d18/f1d 0 2024-03-22T07:15:54.521 INFO:tasks.workunit.client.1.smithi069.stdout:2/69: creat d18/f1e x:0 0 0 2024-03-22T07:15:54.521 INFO:tasks.workunit.client.1.smithi069.stdout:2/70: truncate f15 987334 0 2024-03-22T07:15:54.524 INFO:tasks.workunit.client.1.smithi069.stdout:0/62: creat d3/da/f14 x:0 0 0 2024-03-22T07:15:54.526 INFO:tasks.workunit.client.0.smithi063.stdout:2/308: rename d0/de/d35/l4d to d0/l67 0 2024-03-22T07:15:54.528 INFO:tasks.workunit.client.0.smithi063.stdout:8/369: link d0/c2 d0/c6b 0 2024-03-22T07:15:54.528 INFO:tasks.workunit.client.1.smithi069.stdout:4/62: symlink d6/d12/l15 0 2024-03-22T07:15:54.531 INFO:tasks.workunit.client.0.smithi063.stdout:2/309: rename d0/de/d1b/l33 to d0/de/d58/l68 0 2024-03-22T07:15:54.533 INFO:tasks.workunit.client.0.smithi063.stdout:2/310: mknod d0/de/d12/c69 0 2024-03-22T07:15:54.533 INFO:tasks.workunit.client.0.smithi063.stdout:2/311: write d0/f53 [669380,30452] 0 2024-03-22T07:15:54.536 INFO:tasks.workunit.client.0.smithi063.stdout:2/312: write d0/f60 [630921,26505] 0 2024-03-22T07:15:54.537 INFO:tasks.workunit.client.1.smithi069.stdout:0/63: mkdir d3/d7/d15 0 2024-03-22T07:15:54.537 INFO:tasks.workunit.client.1.smithi069.stdout:0/64: write d3/da/f14 [144254,27593] 0 2024-03-22T07:15:54.538 INFO:tasks.workunit.client.0.smithi063.stdout:4/234: sync 2024-03-22T07:15:54.538 INFO:tasks.workunit.client.0.smithi063.stdout:5/297: sync 2024-03-22T07:15:54.539 INFO:tasks.workunit.client.0.smithi063.stdout:8/370: read d0/d1b/d28/f43 [3165297,112451] 0 2024-03-22T07:15:54.539 INFO:tasks.workunit.client.0.smithi063.stdout:8/371: stat d0/l3 0 2024-03-22T07:15:54.541 INFO:tasks.workunit.client.1.smithi069.stdout:7/60: dwrite f6 [0,4194304] 0 2024-03-22T07:15:54.541 INFO:tasks.workunit.client.1.smithi069.stdout:7/61: chown d9/la 15894618 1 2024-03-22T07:15:54.541 INFO:tasks.workunit.client.0.smithi063.stdout:4/235: dread d1/d5/d38/f44 [0,4194304] 0 2024-03-22T07:15:54.541 INFO:tasks.workunit.client.0.smithi063.stdout:8/372: symlink d0/d26/l6c 0 2024-03-22T07:15:54.542 INFO:tasks.workunit.client.1.smithi069.stdout:4/63: mkdir d6/d16 0 2024-03-22T07:15:54.542 INFO:tasks.workunit.client.1.smithi069.stdout:4/64: dread - d6/dc/d10/f14 zero size 2024-03-22T07:15:54.543 INFO:tasks.workunit.client.0.smithi063.stdout:4/236: creat d1/d5/d43/d27/f4b x:0 0 0 2024-03-22T07:15:54.543 INFO:tasks.workunit.client.0.smithi063.stdout:4/237: write d1/d5/f12 [3189184,49029] 0 2024-03-22T07:15:54.546 INFO:tasks.workunit.client.1.smithi069.stdout:7/62: truncate f2 898661 0 2024-03-22T07:15:54.546 INFO:tasks.workunit.client.1.smithi069.stdout:0/65: symlink d3/da/l16 0 2024-03-22T07:15:54.546 INFO:tasks.workunit.client.1.smithi069.stdout:4/65: chown d6/ca 6967 1 2024-03-22T07:15:54.546 INFO:tasks.workunit.client.1.smithi069.stdout:7/63: mknod d9/c11 0 2024-03-22T07:15:54.547 INFO:tasks.workunit.client.0.smithi063.stdout:8/373: mknod d0/d24/d5b/c6d 0 2024-03-22T07:15:54.547 INFO:tasks.workunit.client.0.smithi063.stdout:4/238: truncate d1/d5/d43/f2e 507647 0 2024-03-22T07:15:54.547 INFO:tasks.workunit.client.0.smithi063.stdout:4/239: creat d1/d5/dc/f4c x:0 0 0 2024-03-22T07:15:54.547 INFO:tasks.workunit.client.0.smithi063.stdout:8/374: rename d0/d26/d2d/d59/f65 to d0/d1b/f6e 0 2024-03-22T07:15:54.548 INFO:tasks.workunit.client.0.smithi063.stdout:4/240: creat d1/d5/d43/d27/d2c/d4a/f4d x:0 0 0 2024-03-22T07:15:54.548 INFO:tasks.workunit.client.1.smithi069.stdout:0/66: mkdir d3/d7/d17 0 2024-03-22T07:15:54.548 INFO:tasks.workunit.client.1.smithi069.stdout:4/66: truncate f3 2503672 0 2024-03-22T07:15:54.548 INFO:tasks.workunit.client.1.smithi069.stdout:4/67: creat d6/dc/d10/f17 x:0 0 0 2024-03-22T07:15:54.549 INFO:tasks.workunit.client.1.smithi069.stdout:7/64: symlink d9/l12 0 2024-03-22T07:15:54.549 INFO:tasks.workunit.client.1.smithi069.stdout:7/65: truncate f8 304043 0 2024-03-22T07:15:54.550 INFO:tasks.workunit.client.1.smithi069.stdout:0/67: creat d3/d7/d17/f18 x:0 0 0 2024-03-22T07:15:54.550 INFO:tasks.workunit.client.1.smithi069.stdout:0/68: truncate d3/d7/fb 1507823 0 2024-03-22T07:15:54.551 INFO:tasks.workunit.client.1.smithi069.stdout:1/67: dwrite d1/f7 [0,4194304] 0 2024-03-22T07:15:54.551 INFO:tasks.workunit.client.1.smithi069.stdout:4/68: mkdir d6/d12/d18 0 2024-03-22T07:15:54.553 INFO:tasks.workunit.client.1.smithi069.stdout:7/66: mkdir d9/db/d13 0 2024-03-22T07:15:54.553 INFO:tasks.workunit.client.1.smithi069.stdout:7/67: stat d9/db/cc 0 2024-03-22T07:15:54.553 INFO:tasks.workunit.client.1.smithi069.stdout:7/68: stat f6 0 2024-03-22T07:15:54.553 INFO:tasks.workunit.client.1.smithi069.stdout:7/69: fdatasync f4 0 2024-03-22T07:15:54.553 INFO:tasks.workunit.client.1.smithi069.stdout:9/53: rmdir d2 39 2024-03-22T07:15:54.555 INFO:tasks.workunit.client.0.smithi063.stdout:4/241: dread d1/d5/d43/f14 [0,4194304] 0 2024-03-22T07:15:54.556 INFO:tasks.workunit.client.0.smithi063.stdout:9/234: dwrite d8/d19/f3a [0,4194304] 0 2024-03-22T07:15:54.556 INFO:tasks.workunit.client.0.smithi063.stdout:4/242: rename d1/d5/d43/l25 to d1/d5/d38/l4e 0 2024-03-22T07:15:54.556 INFO:tasks.workunit.client.1.smithi069.stdout:8/39: dwrite d1/f5 [0,4194304] 0 2024-03-22T07:15:54.556 INFO:tasks.workunit.client.1.smithi069.stdout:8/40: creat d1/f8 x:0 0 0 2024-03-22T07:15:54.556 INFO:tasks.workunit.client.1.smithi069.stdout:0/69: rename d3/da/l16 to d3/d7/l19 0 2024-03-22T07:15:54.556 INFO:tasks.workunit.client.1.smithi069.stdout:9/54: dread d2/dd/f13 [0,4194304] 0 2024-03-22T07:15:54.557 INFO:tasks.workunit.client.1.smithi069.stdout:9/55: chown c0 4305546 1 2024-03-22T07:15:54.560 INFO:tasks.workunit.client.1.smithi069.stdout:0/70: link d3/da/f14 d3/d7/d17/f1a 0 2024-03-22T07:15:54.561 INFO:tasks.workunit.client.0.smithi063.stdout:8/375: write d0/d26/f27 [2585970,96962] 0 2024-03-22T07:15:54.561 INFO:tasks.workunit.client.0.smithi063.stdout:9/235: symlink d8/d19/d2b/l42 0 2024-03-22T07:15:54.561 INFO:tasks.workunit.client.0.smithi063.stdout:9/236: fdatasync d8/d19/f3a 0 2024-03-22T07:15:54.561 INFO:tasks.workunit.client.1.smithi069.stdout:9/56: mknod d2/dd/c17 0 2024-03-22T07:15:54.561 INFO:tasks.workunit.client.1.smithi069.stdout:9/57: chown d2/c14 336 1 2024-03-22T07:15:54.562 INFO:tasks.workunit.client.0.smithi063.stdout:4/243: mknod d1/d5/c4f 0 2024-03-22T07:15:54.562 INFO:tasks.workunit.client.1.smithi069.stdout:6/45: dwrite f3 [4194304,4194304] 0 2024-03-22T07:15:54.563 INFO:tasks.workunit.client.0.smithi063.stdout:3/315: dwrite da/d16/d3f/f56 [4194304,4194304] 0 2024-03-22T07:15:54.563 INFO:tasks.workunit.client.0.smithi063.stdout:1/383: dwrite d4/d71/f78 [0,4194304] 0 2024-03-22T07:15:54.563 INFO:tasks.workunit.client.0.smithi063.stdout:0/363: dwrite da/de/d17/d4e/f57 [0,4194304] 0 2024-03-22T07:15:54.563 INFO:tasks.workunit.client.0.smithi063.stdout:0/364: dread - da/de/d17/f45 zero size 2024-03-22T07:15:54.563 INFO:tasks.workunit.client.0.smithi063.stdout:0/365: chown da/de/d17/d25/d3c/d53/l6e 3469 1 2024-03-22T07:15:54.563 INFO:tasks.workunit.client.0.smithi063.stdout:0/366: readlink da/de/d17/l59 0 2024-03-22T07:15:54.563 INFO:tasks.workunit.client.0.smithi063.stdout:1/384: write d4/df/d13/d39/d63/f66 [1571014,63092] 0 2024-03-22T07:15:54.563 INFO:tasks.workunit.client.0.smithi063.stdout:5/298: dwrite d8/d13/d24/f33 [0,4194304] 0 2024-03-22T07:15:54.564 INFO:tasks.workunit.client.1.smithi069.stdout:9/58: mkdir d2/dd/d18 0 2024-03-22T07:15:54.564 INFO:tasks.workunit.client.0.smithi063.stdout:8/376: mknod d0/d18/d45/c6f 0 2024-03-22T07:15:54.566 INFO:tasks.workunit.client.1.smithi069.stdout:4/69: write d6/dc/ff [4975629,14074] 0 2024-03-22T07:15:54.566 INFO:tasks.workunit.client.1.smithi069.stdout:4/70: write d6/dc/d10/f17 [285651,22637] 0 2024-03-22T07:15:54.567 INFO:tasks.workunit.client.1.smithi069.stdout:4/71: symlink d6/d12/d18/l19 0 2024-03-22T07:15:54.567 INFO:tasks.workunit.client.1.smithi069.stdout:4/72: creat d6/dc/f1a x:0 0 0 2024-03-22T07:15:54.569 INFO:tasks.workunit.client.1.smithi069.stdout:4/73: mknod d6/dc/d10/c1b 0 2024-03-22T07:15:54.570 INFO:tasks.workunit.client.1.smithi069.stdout:2/71: rmdir d18 39 2024-03-22T07:15:54.572 INFO:tasks.workunit.client.1.smithi069.stdout:1/68: creat d1/f12 x:0 0 0 2024-03-22T07:15:54.580 INFO:tasks.workunit.client.1.smithi069.stdout:4/74: fdatasync d6/dc/d10/f17 0 2024-03-22T07:15:54.580 INFO:tasks.workunit.client.0.smithi063.stdout:0/367: unlink da/d49/f6c 0 2024-03-22T07:15:54.581 INFO:tasks.workunit.client.1.smithi069.stdout:4/75: fsync f3 0 2024-03-22T07:15:54.582 INFO:tasks.workunit.client.1.smithi069.stdout:4/76: dread - d6/dc/f1a zero size 2024-03-22T07:15:54.582 INFO:tasks.workunit.client.0.smithi063.stdout:5/299: truncate d8/f2b 2528558 0 2024-03-22T07:15:54.583 INFO:tasks.workunit.client.1.smithi069.stdout:9/59: dread d2/dd/f10 [0,4194304] 0 2024-03-22T07:15:54.585 INFO:tasks.workunit.client.1.smithi069.stdout:9/60: mknod d2/dd/d18/c19 0 2024-03-22T07:15:54.585 INFO:tasks.workunit.client.1.smithi069.stdout:9/61: stat d2/lf 0 2024-03-22T07:15:54.587 INFO:tasks.workunit.client.1.smithi069.stdout:9/62: rename d2/dd/ce to d2/dd/c1a 0 2024-03-22T07:15:54.588 INFO:tasks.workunit.client.1.smithi069.stdout:6/46: write d4/f6 [350041,62210] 0 2024-03-22T07:15:54.588 INFO:tasks.workunit.client.1.smithi069.stdout:6/47: read d4/f6 [731466,31594] 0 2024-03-22T07:15:54.589 INFO:tasks.workunit.client.0.smithi063.stdout:8/377: symlink d0/d26/l70 0 2024-03-22T07:15:54.589 INFO:tasks.workunit.client.0.smithi063.stdout:4/244: rmdir d1/d5/dc 39 2024-03-22T07:15:54.589 INFO:tasks.workunit.client.1.smithi069.stdout:8/41: truncate d1/f5 3910373 0 2024-03-22T07:15:54.590 INFO:tasks.workunit.client.0.smithi063.stdout:0/368: mkdir da/de/d60/d56/d77 0 2024-03-22T07:15:54.591 INFO:tasks.workunit.client.1.smithi069.stdout:6/48: mknod d4/d7/ce 0 2024-03-22T07:15:54.592 INFO:tasks.workunit.client.0.smithi063.stdout:5/300: creat d8/d13/d28/d57/f59 x:0 0 0 2024-03-22T07:15:54.592 INFO:tasks.workunit.client.0.smithi063.stdout:5/301: chown d8/d13/f39 25263 1 2024-03-22T07:15:54.592 INFO:tasks.workunit.client.0.smithi063.stdout:5/302: truncate d8/d13/d1e/d3c/f4c 496561 0 2024-03-22T07:15:54.592 INFO:tasks.workunit.client.0.smithi063.stdout:5/303: stat d8/d13/f18 0 2024-03-22T07:15:54.593 INFO:tasks.workunit.client.0.smithi063.stdout:8/378: mknod d0/d26/d2d/d59/c71 0 2024-03-22T07:15:54.594 INFO:tasks.workunit.client.1.smithi069.stdout:6/49: creat d4/d7/da/ff x:0 0 0 2024-03-22T07:15:54.594 INFO:tasks.workunit.client.1.smithi069.stdout:6/50: chown d4/d7/da 480 1 2024-03-22T07:15:54.595 INFO:tasks.workunit.client.0.smithi063.stdout:4/245: mknod d1/d5/dc/c50 0 2024-03-22T07:15:54.596 INFO:tasks.workunit.client.0.smithi063.stdout:0/369: mkdir da/de/d17/d25/d3c/d64/d78 0 2024-03-22T07:15:54.597 INFO:tasks.workunit.client.0.smithi063.stdout:8/379: symlink d0/d1b/d28/d3e/l72 0 2024-03-22T07:15:54.599 INFO:tasks.workunit.client.0.smithi063.stdout:4/246: creat d1/d5/d3c/f51 x:0 0 0 2024-03-22T07:15:54.599 INFO:tasks.workunit.client.0.smithi063.stdout:4/247: read - d1/f45 zero size 2024-03-22T07:15:54.599 INFO:tasks.workunit.client.0.smithi063.stdout:4/248: creat d1/d5/d3c/f52 x:0 0 0 2024-03-22T07:15:54.600 INFO:tasks.workunit.client.0.smithi063.stdout:4/249: rename d1/d5/d43/d27/d2c/d4a to d1/d5/d43/d27/d2c/d4a/d53 22 2024-03-22T07:15:54.600 INFO:tasks.workunit.client.0.smithi063.stdout:8/380: dread d0/d26/d2d/f4d [0,4194304] 0 2024-03-22T07:15:54.600 INFO:tasks.workunit.client.0.smithi063.stdout:8/381: write d0/d1b/f6e [467963,65159] 0 2024-03-22T07:15:54.600 INFO:tasks.workunit.client.0.smithi063.stdout:8/382: truncate d0/f64 124650 0 2024-03-22T07:15:54.601 INFO:tasks.workunit.client.0.smithi063.stdout:8/383: truncate d0/d24/d33/f5e 496536 0 2024-03-22T07:15:54.601 INFO:tasks.workunit.client.0.smithi063.stdout:8/384: readlink d0/l66 0 2024-03-22T07:15:54.601 INFO:tasks.workunit.client.0.smithi063.stdout:8/385: chown d0/c7 0 1 2024-03-22T07:15:54.601 INFO:tasks.workunit.client.0.smithi063.stdout:0/370: mknod da/c79 0 2024-03-22T07:15:54.601 INFO:tasks.workunit.client.0.smithi063.stdout:4/250: mkdir d1/d5/d43/d27/d2c/d3d/d54 0 2024-03-22T07:15:54.602 INFO:tasks.workunit.client.0.smithi063.stdout:8/386: mknod d0/d1b/c73 0 2024-03-22T07:15:54.603 INFO:tasks.workunit.client.0.smithi063.stdout:0/371: mknod da/de/d60/d56/c7a 0 2024-03-22T07:15:54.605 INFO:tasks.workunit.client.0.smithi063.stdout:6/325: sync 2024-03-22T07:15:54.605 INFO:tasks.workunit.client.0.smithi063.stdout:6/326: dread - d3/d15/f66 zero size 2024-03-22T07:15:54.606 INFO:tasks.workunit.client.0.smithi063.stdout:4/251: link d1/l8 d1/d5/d43/d27/d2c/d4a/l55 0 2024-03-22T07:15:54.606 INFO:tasks.workunit.client.0.smithi063.stdout:4/252: chown d1/d5/d3c/f51 0 1 2024-03-22T07:15:54.607 INFO:tasks.workunit.client.0.smithi063.stdout:5/304: dread d8/f12 [0,4194304] 0 2024-03-22T07:15:54.608 INFO:tasks.workunit.client.0.smithi063.stdout:4/253: link d1/d5/d38/l4e d1/d5/d38/l56 0 2024-03-22T07:15:54.609 INFO:tasks.workunit.client.0.smithi063.stdout:4/254: chown d1/d5/d43/l15 1349 1 2024-03-22T07:15:54.610 INFO:tasks.workunit.client.1.smithi069.stdout:4/77: write d6/dc/ff [2959409,90314] 0 2024-03-22T07:15:54.610 INFO:tasks.workunit.client.1.smithi069.stdout:4/78: truncate d6/dc/d10/f17 581916 0 2024-03-22T07:15:54.610 INFO:tasks.workunit.client.1.smithi069.stdout:4/79: chown d6/dc/d10/f14 0 1 2024-03-22T07:15:54.610 INFO:tasks.workunit.client.0.smithi063.stdout:5/305: symlink d8/d13/d1e/d31/d58/l5a 0 2024-03-22T07:15:54.611 INFO:tasks.workunit.client.0.smithi063.stdout:9/237: dwrite d8/d19/d20/d22/d29/f36 [0,4194304] 0 2024-03-22T07:15:54.611 INFO:tasks.workunit.client.0.smithi063.stdout:4/255: mknod d1/d5/d43/d27/d2c/d3d/d54/c57 0 2024-03-22T07:15:54.611 INFO:tasks.workunit.client.0.smithi063.stdout:4/256: dread - d1/d5/d3c/f51 zero size 2024-03-22T07:15:54.612 INFO:tasks.workunit.client.0.smithi063.stdout:1/385: dwrite d4/df/d13/d39/d6f/f72 [0,4194304] 0 2024-03-22T07:15:54.612 INFO:tasks.workunit.client.0.smithi063.stdout:1/386: creat d4/d14/d57/f80 x:0 0 0 2024-03-22T07:15:54.612 INFO:tasks.workunit.client.0.smithi063.stdout:1/387: creat d4/df/d13/d2b/d3a/f81 x:0 0 0 2024-03-22T07:15:54.613 INFO:tasks.workunit.client.1.smithi069.stdout:2/72: dwrite fa [0,4194304] 0 2024-03-22T07:15:54.613 INFO:tasks.workunit.client.1.smithi069.stdout:9/63: dwrite d2/f16 [0,4194304] 0 2024-03-22T07:15:54.614 INFO:tasks.workunit.client.1.smithi069.stdout:9/64: write d2/dd/f10 [4444492,80148] 0 2024-03-22T07:15:54.614 INFO:tasks.workunit.client.0.smithi063.stdout:4/257: mknod d1/d5/d43/d27/d2c/c58 0 2024-03-22T07:15:54.616 INFO:tasks.workunit.client.1.smithi069.stdout:7/70: dread f2 [0,4194304] 0 2024-03-22T07:15:54.616 INFO:tasks.workunit.client.1.smithi069.stdout:7/71: write f8 [441689,54870] 0 2024-03-22T07:15:54.616 INFO:tasks.workunit.client.1.smithi069.stdout:7/72: write d9/db/ff [138505,21500] 0 2024-03-22T07:15:54.621 INFO:tasks.workunit.client.0.smithi063.stdout:5/306: read d8/f25 [4021496,2011] 0 2024-03-22T07:15:54.621 INFO:tasks.workunit.client.0.smithi063.stdout:5/307: fdatasync d8/d13/d24/f33 0 2024-03-22T07:15:54.621 INFO:tasks.workunit.client.0.smithi063.stdout:3/316: dwrite da/d16/f42 [0,4194304] 0 2024-03-22T07:15:54.622 INFO:tasks.workunit.client.1.smithi069.stdout:9/65: mknod d2/dd/c1b 0 2024-03-22T07:15:54.623 INFO:tasks.workunit.client.0.smithi063.stdout:5/308: symlink d8/d13/d1e/l5b 0 2024-03-22T07:15:54.625 INFO:tasks.workunit.client.1.smithi069.stdout:0/71: dwrite d3/f6 [0,4194304] 0 2024-03-22T07:15:54.625 INFO:tasks.workunit.client.1.smithi069.stdout:0/72: dread - d3/d7/f12 zero size 2024-03-22T07:15:54.625 INFO:tasks.workunit.client.1.smithi069.stdout:0/73: dread - d3/d7/f12 zero size 2024-03-22T07:15:54.625 INFO:tasks.workunit.client.1.smithi069.stdout:0/74: dread - d3/f13 zero size 2024-03-22T07:15:54.625 INFO:tasks.workunit.client.1.smithi069.stdout:4/80: write d6/f9 [105369,35786] 0 2024-03-22T07:15:54.625 INFO:tasks.workunit.client.0.smithi063.stdout:7/247: sync 2024-03-22T07:15:54.626 INFO:tasks.workunit.client.1.smithi069.stdout:9/66: dread d2/dd/f13 [0,4194304] 0 2024-03-22T07:15:54.626 INFO:tasks.workunit.client.1.smithi069.stdout:9/67: truncate d2/dd/f13 1008997 0 2024-03-22T07:15:54.626 INFO:tasks.workunit.client.1.smithi069.stdout:9/68: rename d2 to d2/dd/d18/d1c 22 2024-03-22T07:15:54.627 INFO:tasks.workunit.client.1.smithi069.stdout:7/73: unlink d9/la 0 2024-03-22T07:15:54.627 INFO:tasks.workunit.client.1.smithi069.stdout:7/74: creat d9/f14 x:0 0 0 2024-03-22T07:15:54.628 INFO:tasks.workunit.client.0.smithi063.stdout:3/317: getdents da/d16 0 2024-03-22T07:15:54.630 INFO:tasks.workunit.client.0.smithi063.stdout:4/258: dread d1/f2 [4194304,4194304] 0 2024-03-22T07:15:54.630 INFO:tasks.workunit.client.0.smithi063.stdout:4/259: stat d1/d5/d43/f29 0 2024-03-22T07:15:54.631 INFO:tasks.workunit.client.0.smithi063.stdout:7/248: link d5/f18 d5/d16/d1b/f52 0 2024-03-22T07:15:54.634 INFO:tasks.workunit.client.0.smithi063.stdout:4/260: mkdir d1/d5/d38/d59 0 2024-03-22T07:15:54.636 INFO:tasks.workunit.client.0.smithi063.stdout:4/261: creat d1/d5/dc/f5a x:0 0 0 2024-03-22T07:15:54.636 INFO:tasks.workunit.client.0.smithi063.stdout:4/262: chown d1/d5/dc/f41 7707 1 2024-03-22T07:15:54.636 INFO:tasks.workunit.client.0.smithi063.stdout:4/263: truncate d1/d5/f33 4879116 0 2024-03-22T07:15:54.652 INFO:tasks.workunit.client.1.smithi069.stdout:7/75: unlink d9/fe 0 2024-03-22T07:15:54.652 INFO:tasks.workunit.client.1.smithi069.stdout:7/76: stat f2 0 2024-03-22T07:15:54.653 INFO:tasks.workunit.client.1.smithi069.stdout:1/69: unlink d1/f7 0 2024-03-22T07:15:54.654 INFO:tasks.workunit.client.1.smithi069.stdout:0/75: rmdir d3/da 39 2024-03-22T07:15:54.654 INFO:tasks.workunit.client.1.smithi069.stdout:0/76: stat d3/d7/d17/f1a 0 2024-03-22T07:15:54.656 INFO:tasks.workunit.client.1.smithi069.stdout:0/77: link d3/f4 d3/d7/d15/f1b 0 2024-03-22T07:15:54.656 INFO:tasks.workunit.client.1.smithi069.stdout:0/78: dread - d3/d7/f12 zero size 2024-03-22T07:15:54.658 INFO:tasks.workunit.client.1.smithi069.stdout:0/79: mkdir d3/d7/d17/d1c 0 2024-03-22T07:15:54.667 INFO:tasks.workunit.client.0.smithi063.stdout:7/249: write d5/d14/f37 [1434645,88295] 0 2024-03-22T07:15:54.669 INFO:tasks.workunit.client.0.smithi063.stdout:7/250: link d5/d14/f1a d5/da/d28/d31/d33/f53 0 2024-03-22T07:15:54.671 INFO:tasks.workunit.client.0.smithi063.stdout:7/251: unlink d5/l2f 0 2024-03-22T07:15:54.671 INFO:tasks.workunit.client.0.smithi063.stdout:7/252: fdatasync d5/d16/f32 0 2024-03-22T07:15:54.671 INFO:tasks.workunit.client.0.smithi063.stdout:7/253: fsync d5/d14/d1d/d29/f36 0 2024-03-22T07:15:54.680 INFO:tasks.workunit.client.1.smithi069.stdout:7/77: write f6 [169079,93822] 0 2024-03-22T07:15:54.680 INFO:tasks.workunit.client.1.smithi069.stdout:7/78: write f4 [640361,57278] 0 2024-03-22T07:15:54.682 INFO:tasks.workunit.client.1.smithi069.stdout:7/79: dread f2 [0,4194304] 0 2024-03-22T07:15:54.684 INFO:tasks.workunit.client.1.smithi069.stdout:1/70: dread d1/fa [0,4194304] 0 2024-03-22T07:15:54.684 INFO:tasks.workunit.client.1.smithi069.stdout:7/80: rename d9/db/cc to d9/c15 0 2024-03-22T07:15:54.685 INFO:tasks.workunit.client.1.smithi069.stdout:1/71: readlink d1/l10 0 2024-03-22T07:15:54.686 INFO:tasks.workunit.client.1.smithi069.stdout:1/72: mkdir d1/d13 0 2024-03-22T07:15:54.687 INFO:tasks.workunit.client.1.smithi069.stdout:1/73: mknod d1/d13/c14 0 2024-03-22T07:15:54.689 INFO:tasks.workunit.client.1.smithi069.stdout:1/74: rename d1/c8 to d1/d13/c15 0 2024-03-22T07:15:54.690 INFO:tasks.workunit.client.0.smithi063.stdout:8/387: dwrite d0/d24/f29 [4194304,4194304] 0 2024-03-22T07:15:54.690 INFO:tasks.workunit.client.0.smithi063.stdout:0/372: dwrite da/de/d17/f21 [0,4194304] 0 2024-03-22T07:15:54.691 INFO:tasks.workunit.client.1.smithi069.stdout:7/81: read d9/db/ff [120346,109901] 0 2024-03-22T07:15:54.691 INFO:tasks.workunit.client.1.smithi069.stdout:7/82: truncate f3 92374 0 2024-03-22T07:15:54.693 INFO:tasks.workunit.client.0.smithi063.stdout:8/388: creat d0/d24/f74 x:0 0 0 2024-03-22T07:15:54.694 INFO:tasks.workunit.client.0.smithi063.stdout:1/388: dwrite d4/f12 [0,4194304] 0 2024-03-22T07:15:54.694 INFO:tasks.workunit.client.1.smithi069.stdout:6/51: dwrite d4/f6 [4194304,4194304] 0 2024-03-22T07:15:54.702 INFO:tasks.workunit.client.0.smithi063.stdout:5/309: dwrite d8/d13/d24/f3b [0,4194304] 0 2024-03-22T07:15:54.702 INFO:tasks.workunit.client.0.smithi063.stdout:5/310: stat d8/d13/d1e/d31/d58/l5a 0 2024-03-22T07:15:54.702 INFO:tasks.workunit.client.0.smithi063.stdout:5/311: dread - d8/f44 zero size 2024-03-22T07:15:54.702 INFO:tasks.workunit.client.0.smithi063.stdout:5/312: chown d8/d13/d28/c47 6336324 1 2024-03-22T07:15:54.702 INFO:tasks.workunit.client.0.smithi063.stdout:3/318: dwrite da/d16/d26/f58 [0,4194304] 0 2024-03-22T07:15:54.702 INFO:tasks.workunit.client.0.smithi063.stdout:5/313: read d8/d13/d1e/f27 [2236937,1918] 0 2024-03-22T07:15:54.702 INFO:tasks.workunit.client.0.smithi063.stdout:5/314: read - d8/f44 zero size 2024-03-22T07:15:54.702 INFO:tasks.workunit.client.0.smithi063.stdout:7/254: dwrite d5/f7 [4194304,4194304] 0 2024-03-22T07:15:54.703 INFO:tasks.workunit.client.0.smithi063.stdout:7/255: chown d5/d16 654428473 1 2024-03-22T07:15:54.704 INFO:tasks.workunit.client.1.smithi069.stdout:4/81: dread d6/dc/ff [0,4194304] 0 2024-03-22T07:15:54.705 INFO:tasks.workunit.client.0.smithi063.stdout:1/389: creat d4/df/d13/d39/f82 x:0 0 0 2024-03-22T07:15:54.707 INFO:tasks.workunit.client.0.smithi063.stdout:3/319: truncate da/d16/d26/f29 2711467 0 2024-03-22T07:15:54.708 INFO:tasks.workunit.client.1.smithi069.stdout:9/69: dread d2/dd/f10 [0,4194304] 0 2024-03-22T07:15:54.710 INFO:tasks.workunit.client.0.smithi063.stdout:7/256: symlink d5/d16/l54 0 2024-03-22T07:15:54.711 INFO:tasks.workunit.client.1.smithi069.stdout:9/70: truncate d2/f5 2416593 0 2024-03-22T07:15:54.714 INFO:tasks.workunit.client.1.smithi069.stdout:2/73: dwrite fa [0,4194304] 0 2024-03-22T07:15:54.714 INFO:tasks.workunit.client.0.smithi063.stdout:1/390: link d4/d5a/c61 d4/df/d13/d39/d63/c83 0 2024-03-22T07:15:54.715 INFO:tasks.workunit.client.0.smithi063.stdout:1/391: creat d4/df/d13/d2b/f84 x:0 0 0 2024-03-22T07:15:54.715 INFO:tasks.workunit.client.1.smithi069.stdout:9/71: rmdir d2/dd 39 2024-03-22T07:15:54.716 INFO:tasks.workunit.client.1.smithi069.stdout:9/72: truncate d2/dd/f13 1753461 0 2024-03-22T07:15:54.721 INFO:tasks.workunit.client.1.smithi069.stdout:9/73: mknod d2/dd/c1d 0 2024-03-22T07:15:54.722 INFO:tasks.workunit.client.1.smithi069.stdout:9/74: chown d2/c9 70431545 1 2024-03-22T07:15:54.722 INFO:tasks.workunit.client.1.smithi069.stdout:9/75: readlink d2/dd/l11 0 2024-03-22T07:15:54.722 INFO:tasks.workunit.client.1.smithi069.stdout:0/80: dwrite d3/f13 [0,4194304] 0 2024-03-22T07:15:54.723 INFO:tasks.workunit.client.1.smithi069.stdout:9/76: mknod d2/dd/d18/c1e 0 2024-03-22T07:15:54.728 INFO:tasks.workunit.client.1.smithi069.stdout:0/81: symlink d3/d7/d17/d1c/l1d 0 2024-03-22T07:15:54.728 INFO:tasks.workunit.client.1.smithi069.stdout:0/82: chown d3/ld 163 1 2024-03-22T07:15:54.733 INFO:tasks.workunit.client.1.smithi069.stdout:9/77: rename d2/dd/f13 to d2/dd/d18/f1f 0 2024-03-22T07:15:54.734 INFO:tasks.workunit.client.0.smithi063.stdout:4/264: dwrite d1/d5/f23 [4194304,4194304] 0 2024-03-22T07:15:54.738 INFO:tasks.workunit.client.0.smithi063.stdout:4/265: rename d1/d5/f33 to d1/f5b 0 2024-03-22T07:15:54.738 INFO:tasks.workunit.client.0.smithi063.stdout:4/266: write d1/d5/d3c/f51 [60530,11291] 0 2024-03-22T07:15:54.738 INFO:tasks.workunit.client.0.smithi063.stdout:4/267: write d1/d5/f23 [3974199,83718] 0 2024-03-22T07:15:54.738 INFO:tasks.workunit.client.1.smithi069.stdout:0/83: rename d3/d7/d17/f18 to d3/d7/f1e 0 2024-03-22T07:15:54.740 INFO:tasks.workunit.client.0.smithi063.stdout:4/268: rename d1/d5/dc/ld to d1/d5/d43/d27/d2c/d4a/l5c 0 2024-03-22T07:15:54.743 INFO:tasks.workunit.client.0.smithi063.stdout:4/269: link d1/d5/c3f d1/d5/c5d 0 2024-03-22T07:15:54.744 INFO:tasks.workunit.client.0.smithi063.stdout:4/270: symlink d1/d5/d43/d27/d2c/d4a/l5e 0 2024-03-22T07:15:54.759 INFO:tasks.workunit.client.1.smithi069.stdout:0/84: dread d3/f6 [0,4194304] 0 2024-03-22T07:15:54.759 INFO:tasks.workunit.client.1.smithi069.stdout:0/85: chown d3/d7/fb 3629 1 2024-03-22T07:15:54.759 INFO:tasks.workunit.client.1.smithi069.stdout:0/86: fdatasync d3/f6 0 2024-03-22T07:15:54.760 INFO:tasks.workunit.client.1.smithi069.stdout:0/87: unlink d3/f4 0 2024-03-22T07:15:54.762 INFO:tasks.workunit.client.1.smithi069.stdout:0/88: mkdir d3/d7/d15/d1f 0 2024-03-22T07:15:54.762 INFO:tasks.workunit.client.0.smithi063.stdout:7/257: dwrite d5/d16/f22 [0,4194304] 0 2024-03-22T07:15:54.763 INFO:tasks.workunit.client.1.smithi069.stdout:0/89: creat d3/d7/d15/f20 x:0 0 0 2024-03-22T07:15:54.763 INFO:tasks.workunit.client.0.smithi063.stdout:7/258: creat d5/d14/d1d/d29/d4a/f55 x:0 0 0 2024-03-22T07:15:54.764 INFO:tasks.workunit.client.1.smithi069.stdout:0/90: creat d3/d7/d15/d1f/f21 x:0 0 0 2024-03-22T07:15:54.764 INFO:tasks.workunit.client.0.smithi063.stdout:7/259: truncate d5/d16/d1b/f52 287251 0 2024-03-22T07:15:54.765 INFO:tasks.workunit.client.1.smithi069.stdout:0/91: mknod d3/c22 0 2024-03-22T07:15:54.766 INFO:tasks.workunit.client.1.smithi069.stdout:0/92: readlink d3/d7/l19 0 2024-03-22T07:15:54.772 INFO:tasks.workunit.client.1.smithi069.stdout:0/93: dread d3/da/f14 [0,4194304] 0 2024-03-22T07:15:54.774 INFO:tasks.workunit.client.1.smithi069.stdout:0/94: mknod d3/da/c23 0 2024-03-22T07:15:54.794 INFO:tasks.workunit.client.0.smithi063.stdout:4/271: read d1/d5/d3c/f47 [5098945,91733] 0 2024-03-22T07:15:54.795 INFO:tasks.workunit.client.0.smithi063.stdout:4/272: dread - d1/d5/d43/d27/f4b zero size 2024-03-22T07:15:54.796 INFO:tasks.workunit.client.0.smithi063.stdout:4/273: mkdir d1/d5/d43/d27/d2c/d3d/d54/d5f 0 2024-03-22T07:15:54.796 INFO:tasks.workunit.client.0.smithi063.stdout:4/274: creat d1/d5/d38/f60 x:0 0 0 2024-03-22T07:15:54.796 INFO:tasks.workunit.client.0.smithi063.stdout:4/275: read - d1/d5/d43/f39 zero size 2024-03-22T07:15:54.799 INFO:tasks.workunit.client.0.smithi063.stdout:4/276: unlink d1/d5/d43/d27/d2c/d3d/f46 0 2024-03-22T07:15:54.800 INFO:tasks.workunit.client.0.smithi063.stdout:4/277: symlink d1/d5/d38/l61 0 2024-03-22T07:15:54.800 INFO:tasks.workunit.client.0.smithi063.stdout:4/278: fdatasync d1/f32 0 2024-03-22T07:15:54.800 INFO:tasks.workunit.client.0.smithi063.stdout:4/279: chown d1/l36 813836232 1 2024-03-22T07:15:54.800 INFO:tasks.workunit.client.0.smithi063.stdout:4/280: dread - d1/d5/d3c/f52 zero size 2024-03-22T07:15:54.804 INFO:tasks.workunit.client.0.smithi063.stdout:3/320: dwrite da/f3d [0,4194304] 0 2024-03-22T07:15:54.804 INFO:tasks.workunit.client.1.smithi069.stdout:7/83: dwrite f6 [0,4194304] 0 2024-03-22T07:15:54.804 INFO:tasks.workunit.client.1.smithi069.stdout:7/84: chown f2 1 1 2024-03-22T07:15:54.804 INFO:tasks.workunit.client.1.smithi069.stdout:7/85: chown d9/c11 333903431 1 2024-03-22T07:15:54.804 INFO:tasks.workunit.client.1.smithi069.stdout:7/86: stat d9/c10 0 2024-03-22T07:15:54.804 INFO:tasks.workunit.client.1.smithi069.stdout:7/87: chown f6 1 1 2024-03-22T07:15:54.806 INFO:tasks.workunit.client.0.smithi063.stdout:8/389: dwrite d0/d24/d33/f5e [0,4194304] 0 2024-03-22T07:15:54.807 INFO:tasks.workunit.client.1.smithi069.stdout:6/52: dwrite d4/d7/da/ff [0,4194304] 0 2024-03-22T07:15:54.807 INFO:tasks.workunit.client.1.smithi069.stdout:7/88: dread f2 [0,4194304] 0 2024-03-22T07:15:54.807 INFO:tasks.workunit.client.1.smithi069.stdout:4/82: dwrite d6/f9 [4194304,4194304] 0 2024-03-22T07:15:54.808 INFO:tasks.workunit.client.1.smithi069.stdout:6/53: dread d4/fb [0,4194304] 0 2024-03-22T07:15:54.808 INFO:tasks.workunit.client.1.smithi069.stdout:6/54: write d4/f6 [1342751,19899] 0 2024-03-22T07:15:54.808 INFO:tasks.workunit.client.1.smithi069.stdout:6/55: readlink d4/d7/da/lc 0 2024-03-22T07:15:54.808 INFO:tasks.workunit.client.1.smithi069.stdout:6/56: stat d4/d7/da/ff 0 2024-03-22T07:15:54.808 INFO:tasks.workunit.client.0.smithi063.stdout:1/392: dwrite d4/d14/f50 [0,4194304] 0 2024-03-22T07:15:54.808 INFO:tasks.workunit.client.0.smithi063.stdout:0/373: dwrite da/de/d39/f72 [0,4194304] 0 2024-03-22T07:15:54.810 INFO:tasks.workunit.client.1.smithi069.stdout:1/75: dwrite d1/f12 [0,4194304] 0 2024-03-22T07:15:54.810 INFO:tasks.workunit.client.1.smithi069.stdout:9/78: dwrite d2/f16 [0,4194304] 0 2024-03-22T07:15:54.810 INFO:tasks.workunit.client.1.smithi069.stdout:9/79: write d2/f16 [5149121,112295] 0 2024-03-22T07:15:54.811 INFO:tasks.workunit.client.0.smithi063.stdout:8/390: creat d0/d24/d33/d4c/f75 x:0 0 0 2024-03-22T07:15:54.811 INFO:tasks.workunit.client.0.smithi063.stdout:5/315: dwrite d8/d13/d1e/d31/f46 [4194304,4194304] 0 2024-03-22T07:15:54.811 INFO:tasks.workunit.client.0.smithi063.stdout:5/316: read - d8/f44 zero size 2024-03-22T07:15:54.811 INFO:tasks.workunit.client.0.smithi063.stdout:5/317: stat d8/d13/d28/c42 0 2024-03-22T07:15:54.811 INFO:tasks.workunit.client.1.smithi069.stdout:2/74: dwrite d18/f19 [0,4194304] 0 2024-03-22T07:15:54.812 INFO:tasks.workunit.client.0.smithi063.stdout:5/318: dread d8/d13/d1e/d3c/f4c [0,4194304] 0 2024-03-22T07:15:54.814 INFO:tasks.workunit.client.1.smithi069.stdout:7/89: creat d9/db/f16 x:0 0 0 2024-03-22T07:15:54.814 INFO:tasks.workunit.client.1.smithi069.stdout:7/90: write f3 [1129392,66640] 0 2024-03-22T07:15:54.814 INFO:tasks.workunit.client.1.smithi069.stdout:7/91: creat d9/db/f17 x:0 0 0 2024-03-22T07:15:54.814 INFO:tasks.workunit.client.1.smithi069.stdout:7/92: write f2 [1192959,95928] 0 2024-03-22T07:15:54.815 INFO:tasks.workunit.client.0.smithi063.stdout:7/260: dwrite d5/d14/d1d/f51 [0,4194304] 0 2024-03-22T07:15:54.815 INFO:tasks.workunit.client.1.smithi069.stdout:1/76: dread d1/fa [0,4194304] 0 2024-03-22T07:15:54.815 INFO:tasks.workunit.client.1.smithi069.stdout:1/77: readlink d1/l10 0 2024-03-22T07:15:54.818 INFO:tasks.workunit.client.0.smithi063.stdout:0/374: dread da/f40 [4194304,4194304] 0 2024-03-22T07:15:54.818 INFO:tasks.workunit.client.0.smithi063.stdout:0/375: write da/de/d17/d25/d3c/d5f/f4b [852723,50810] 0 2024-03-22T07:15:54.820 INFO:tasks.workunit.client.0.smithi063.stdout:1/393: dread d4/d14/f46 [0,4194304] 0 2024-03-22T07:15:54.820 INFO:tasks.workunit.client.0.smithi063.stdout:3/321: dread da/d16/f25 [0,4194304] 0 2024-03-22T07:15:54.820 INFO:tasks.workunit.client.0.smithi063.stdout:3/322: truncate da/f3a 4829303 0 2024-03-22T07:15:54.821 INFO:tasks.workunit.client.1.smithi069.stdout:6/57: mkdir d4/d7/d10 0 2024-03-22T07:15:54.828 INFO:tasks.workunit.client.0.smithi063.stdout:4/281: dwrite d1/d5/f20 [0,4194304] 0 2024-03-22T07:15:54.830 INFO:tasks.workunit.client.1.smithi069.stdout:3/84: sync 2024-03-22T07:15:54.835 INFO:tasks.workunit.client.0.smithi063.stdout:0/376: mknod da/de/d17/d25/d3c/d43/c7b 0 2024-03-22T07:15:54.835 INFO:tasks.workunit.client.1.smithi069.stdout:2/75: symlink d18/l1f 0 2024-03-22T07:15:54.835 INFO:tasks.workunit.client.1.smithi069.stdout:6/58: dread f3 [0,4194304] 0 2024-03-22T07:15:54.835 INFO:tasks.workunit.client.0.smithi063.stdout:4/282: dread d1/d5/d43/d27/d2c/f3e [0,4194304] 0 2024-03-22T07:15:54.835 INFO:tasks.workunit.client.0.smithi063.stdout:4/283: creat d1/d5/d43/d27/d2c/d3d/f62 x:0 0 0 2024-03-22T07:15:54.837 INFO:tasks.workunit.client.0.smithi063.stdout:1/394: mkdir d4/df/d13/d2b/d6c/d85 0 2024-03-22T07:15:54.839 INFO:tasks.workunit.client.1.smithi069.stdout:9/80: dread d2/f5 [0,4194304] 0 2024-03-22T07:15:54.841 INFO:tasks.workunit.client.1.smithi069.stdout:1/78: readlink d1/l11 0 2024-03-22T07:15:54.843 INFO:tasks.workunit.client.1.smithi069.stdout:6/59: creat d4/dd/f11 x:0 0 0 2024-03-22T07:15:54.843 INFO:tasks.workunit.client.1.smithi069.stdout:7/93: dwrite d9/db/f16 [0,4194304] 0 2024-03-22T07:15:54.844 INFO:tasks.workunit.client.0.smithi063.stdout:0/377: mkdir da/de/d17/d25/d7c 0 2024-03-22T07:15:54.844 INFO:tasks.workunit.client.0.smithi063.stdout:0/378: stat da/de/d17/d25/d3c/d64 0 2024-03-22T07:15:54.844 INFO:tasks.workunit.client.0.smithi063.stdout:0/379: fdatasync da/de/d17/f2f 0 2024-03-22T07:15:54.844 INFO:tasks.workunit.client.0.smithi063.stdout:0/380: chown da/de/d17/f21 29876330 1 2024-03-22T07:15:54.845 INFO:tasks.workunit.client.0.smithi063.stdout:8/391: rmdir d0/d24 39 2024-03-22T07:15:54.845 INFO:tasks.workunit.client.0.smithi063.stdout:8/392: stat d0/d26/d2d/f4d 0 2024-03-22T07:15:54.845 INFO:tasks.workunit.client.0.smithi063.stdout:8/393: write d0/fc [3335518,11956] 0 2024-03-22T07:15:54.846 INFO:tasks.workunit.client.0.smithi063.stdout:7/261: dwrite d5/d14/f37 [0,4194304] 0 2024-03-22T07:15:54.849 INFO:tasks.workunit.client.1.smithi069.stdout:1/79: mkdir d1/d13/d16 0 2024-03-22T07:15:54.851 INFO:tasks.workunit.client.1.smithi069.stdout:2/76: dread f6 [0,4194304] 0 2024-03-22T07:15:54.854 INFO:tasks.workunit.client.0.smithi063.stdout:0/381: truncate da/de/d17/d25/d3c/d5f/f52 243751 0 2024-03-22T07:15:54.854 INFO:tasks.workunit.client.0.smithi063.stdout:0/382: write da/d49/f63 [1154154,43946] 0 2024-03-22T07:15:54.859 INFO:tasks.workunit.client.0.smithi063.stdout:3/323: rename da/d16/d3c to da/d63/d65 0 2024-03-22T07:15:54.861 INFO:tasks.workunit.client.0.smithi063.stdout:3/324: creat da/d63/d65/f66 x:0 0 0 2024-03-22T07:15:54.861 INFO:tasks.workunit.client.0.smithi063.stdout:3/325: creat da/d16/d3f/f67 x:0 0 0 2024-03-22T07:15:54.862 INFO:tasks.workunit.client.1.smithi069.stdout:1/80: creat d1/d13/f17 x:0 0 0 2024-03-22T07:15:54.862 INFO:tasks.workunit.client.1.smithi069.stdout:2/77: dread fa [0,4194304] 0 2024-03-22T07:15:54.863 INFO:tasks.workunit.client.0.smithi063.stdout:3/326: mkdir da/d63/d65/d4b/d68 0 2024-03-22T07:15:54.864 INFO:tasks.workunit.client.0.smithi063.stdout:0/383: dread da/de/d17/d25/d3c/f3f [0,4194304] 0 2024-03-22T07:15:54.864 INFO:tasks.workunit.client.0.smithi063.stdout:7/262: dread d5/d14/d1d/f3f [0,4194304] 0 2024-03-22T07:15:54.864 INFO:tasks.workunit.client.1.smithi069.stdout:1/81: truncate d1/fb 860064 0 2024-03-22T07:15:54.865 INFO:tasks.workunit.client.0.smithi063.stdout:6/327: sync 2024-03-22T07:15:54.865 INFO:tasks.workunit.client.0.smithi063.stdout:2/313: sync 2024-03-22T07:15:54.865 INFO:tasks.workunit.client.0.smithi063.stdout:2/314: chown d0/de/d12/f27 2 1 2024-03-22T07:15:54.865 INFO:tasks.workunit.client.0.smithi063.stdout:3/327: rmdir da/d16/d26 39 2024-03-22T07:15:54.866 INFO:tasks.workunit.client.1.smithi069.stdout:2/78: rename c17 to d18/c20 0 2024-03-22T07:15:54.866 INFO:tasks.workunit.client.1.smithi069.stdout:2/79: chown d18/c20 597 1 2024-03-22T07:15:54.866 INFO:tasks.workunit.client.0.smithi063.stdout:3/328: write da/d16/d3f/f44 [687707,51914] 0 2024-03-22T07:15:54.866 INFO:tasks.workunit.client.0.smithi063.stdout:6/328: link d3/d1c/f4b d3/d6/d2a/d38/f6c 0 2024-03-22T07:15:54.868 INFO:tasks.workunit.client.1.smithi069.stdout:1/82: unlink d1/f12 0 2024-03-22T07:15:54.868 INFO:tasks.workunit.client.1.smithi069.stdout:1/83: read d1/fb [746917,63334] 0 2024-03-22T07:15:54.870 INFO:tasks.workunit.client.1.smithi069.stdout:2/80: rename ce to d18/d1c/c21 0 2024-03-22T07:15:54.873 INFO:tasks.workunit.client.1.smithi069.stdout:2/81: creat d18/d1c/f22 x:0 0 0 2024-03-22T07:15:54.875 INFO:tasks.workunit.client.1.smithi069.stdout:8/42: sync 2024-03-22T07:15:54.875 INFO:tasks.workunit.client.1.smithi069.stdout:8/43: readlink d1/l7 0 2024-03-22T07:15:54.877 INFO:tasks.workunit.client.1.smithi069.stdout:2/82: symlink d18/l23 0 2024-03-22T07:15:54.878 INFO:tasks.workunit.client.0.smithi063.stdout:0/384: dread f1 [0,4194304] 0 2024-03-22T07:15:54.879 INFO:tasks.workunit.client.1.smithi069.stdout:1/84: rename d1/f6 to d1/f18 0 2024-03-22T07:15:54.880 INFO:tasks.workunit.client.1.smithi069.stdout:2/83: rename d18/f1d to d18/d1c/f24 0 2024-03-22T07:15:54.887 INFO:tasks.workunit.client.0.smithi063.stdout:8/394: dwrite d0/d18/d45/f6a [0,4194304] 0 2024-03-22T07:15:54.887 INFO:tasks.workunit.client.1.smithi069.stdout:3/85: dwrite d5/d9/fd [0,4194304] 0 2024-03-22T07:15:54.887 INFO:tasks.workunit.client.1.smithi069.stdout:6/60: dwrite d4/dd/f11 [0,4194304] 0 2024-03-22T07:15:54.889 INFO:tasks.workunit.client.0.smithi063.stdout:0/385: write da/de/d17/f28 [3588159,4661] 0 2024-03-22T07:15:54.889 INFO:tasks.workunit.client.1.smithi069.stdout:5/37: sync 2024-03-22T07:15:54.890 INFO:tasks.workunit.client.1.smithi069.stdout:5/38: fdatasync d1/f3 0 2024-03-22T07:15:54.890 INFO:tasks.workunit.client.1.smithi069.stdout:5/39: rename d1 to d1/d7/d8/da 22 2024-03-22T07:15:54.890 INFO:tasks.workunit.client.0.smithi063.stdout:8/395: symlink d0/d1f/l76 0 2024-03-22T07:15:54.891 INFO:tasks.workunit.client.0.smithi063.stdout:6/329: dwrite d3/d6/f2c [0,4194304] 0 2024-03-22T07:15:54.891 INFO:tasks.workunit.client.0.smithi063.stdout:6/330: dread - d3/d6/d2a/f48 zero size 2024-03-22T07:15:54.891 INFO:tasks.workunit.client.0.smithi063.stdout:4/284: dwrite d1/d5/f9 [0,4194304] 0 2024-03-22T07:15:54.891 INFO:tasks.workunit.client.0.smithi063.stdout:4/285: chown d1/d5/c5d 28607199 1 2024-03-22T07:15:54.892 INFO:tasks.workunit.client.1.smithi069.stdout:7/94: dwrite d9/db/f16 [0,4194304] 0 2024-03-22T07:15:54.893 INFO:tasks.workunit.client.0.smithi063.stdout:8/396: truncate d0/d18/d4a/f5a 106229 0 2024-03-22T07:15:54.897 INFO:tasks.workunit.client.0.smithi063.stdout:0/386: dread da/d49/f63 [0,4194304] 0 2024-03-22T07:15:54.897 INFO:tasks.workunit.client.0.smithi063.stdout:6/331: chown d3/c12 312 1 2024-03-22T07:15:54.898 INFO:tasks.workunit.client.0.smithi063.stdout:4/286: mknod d1/d5/d38/d59/c63 0 2024-03-22T07:15:54.898 INFO:tasks.workunit.client.1.smithi069.stdout:5/40: dread d1/f3 [0,4194304] 0 2024-03-22T07:15:54.899 INFO:tasks.workunit.client.0.smithi063.stdout:8/397: link d0/d26/c36 d0/d26/c77 0 2024-03-22T07:15:54.901 INFO:tasks.workunit.client.0.smithi063.stdout:0/387: rmdir da/de/d17/d25/d7c 0 2024-03-22T07:15:54.904 INFO:tasks.workunit.client.0.smithi063.stdout:3/329: rmdir da/d63/d65 39 2024-03-22T07:15:54.904 INFO:tasks.workunit.client.0.smithi063.stdout:6/332: dread d3/f4 [0,4194304] 0 2024-03-22T07:15:54.905 INFO:tasks.workunit.client.0.smithi063.stdout:4/287: dread d1/d5/d43/f31 [0,4194304] 0 2024-03-22T07:15:54.905 INFO:tasks.workunit.client.0.smithi063.stdout:4/288: write d1/d5/dc/ff [3326158,51151] 0 2024-03-22T07:15:54.906 INFO:tasks.workunit.client.1.smithi069.stdout:3/86: write d5/d9/fd [3299743,4373] 0 2024-03-22T07:15:54.907 INFO:tasks.workunit.client.0.smithi063.stdout:6/333: unlink d3/d6/d35/f52 0 2024-03-22T07:15:54.907 INFO:tasks.workunit.client.0.smithi063.stdout:6/334: write d3/f4 [3104406,117061] 0 2024-03-22T07:15:54.910 INFO:tasks.workunit.client.1.smithi069.stdout:7/95: dwrite d9/f14 [0,4194304] 0 2024-03-22T07:15:54.911 INFO:tasks.workunit.client.1.smithi069.stdout:6/61: dwrite d4/fb [0,4194304] 0 2024-03-22T07:15:54.911 INFO:tasks.workunit.client.0.smithi063.stdout:6/335: rename d3/d15/d29 to d3/d15/d16/d60/d6d 0 2024-03-22T07:15:54.911 INFO:tasks.workunit.client.0.smithi063.stdout:6/336: dread - d3/d6/d2a/d38/f61 zero size 2024-03-22T07:15:54.912 INFO:tasks.workunit.client.0.smithi063.stdout:6/337: write d3/f9 [5800494,44548] 0 2024-03-22T07:15:54.912 INFO:tasks.workunit.client.0.smithi063.stdout:6/338: write d3/d6/d2a/d38/f61 [192878,49984] 0 2024-03-22T07:15:54.914 INFO:tasks.workunit.client.0.smithi063.stdout:6/339: symlink d3/d1c/d32/d51/l6e 0 2024-03-22T07:15:54.916 INFO:tasks.workunit.client.0.smithi063.stdout:6/340: rename d3/d15/f66 to d3/d15/d16/d60/d6d/d4c/f6f 0 2024-03-22T07:15:54.916 INFO:tasks.workunit.client.0.smithi063.stdout:6/341: readlink d3/d6/l8 0 2024-03-22T07:15:54.918 INFO:tasks.workunit.client.0.smithi063.stdout:6/342: symlink d3/d15/d16/d57/l70 0 2024-03-22T07:15:54.918 INFO:tasks.workunit.client.0.smithi063.stdout:6/343: fdatasync d3/d15/d16/d60/d6d/d4c/f6f 0 2024-03-22T07:15:54.930 INFO:tasks.workunit.client.1.smithi069.stdout:5/41: link d1/c4 d1/d7/cb 0 2024-03-22T07:15:54.930 INFO:tasks.workunit.client.1.smithi069.stdout:5/42: chown d1/d7 219245488 1 2024-03-22T07:15:54.930 INFO:tasks.workunit.client.1.smithi069.stdout:5/43: stat d1 0 2024-03-22T07:15:54.930 INFO:tasks.workunit.client.1.smithi069.stdout:5/44: getdents d1/d7/d8 0 2024-03-22T07:15:54.932 INFO:tasks.workunit.client.1.smithi069.stdout:3/87: symlink d5/d9/l20 0 2024-03-22T07:15:54.932 INFO:tasks.workunit.client.1.smithi069.stdout:3/88: dread - d5/f18 zero size 2024-03-22T07:15:54.938 INFO:tasks.workunit.client.1.smithi069.stdout:7/96: truncate d9/db/ff 123668 0 2024-03-22T07:15:54.938 INFO:tasks.workunit.client.1.smithi069.stdout:7/97: chown d9/ld 161 1 2024-03-22T07:15:54.938 INFO:tasks.workunit.client.1.smithi069.stdout:7/98: readlink d9/l12 0 2024-03-22T07:15:54.938 INFO:tasks.workunit.client.1.smithi069.stdout:7/99: readlink l5 0 2024-03-22T07:15:54.938 INFO:tasks.workunit.client.0.smithi063.stdout:0/388: dwrite da/de/d17/d25/f3e [0,4194304] 0 2024-03-22T07:15:54.938 INFO:tasks.workunit.client.0.smithi063.stdout:0/389: truncate da/f2b 935660 0 2024-03-22T07:15:54.938 INFO:tasks.workunit.client.0.smithi063.stdout:0/390: readlink da/de/l2c 0 2024-03-22T07:15:54.938 INFO:tasks.workunit.client.0.smithi063.stdout:6/344: dwrite d3/d6/d2a/d38/f49 [0,4194304] 0 2024-03-22T07:15:54.939 INFO:tasks.workunit.client.0.smithi063.stdout:3/330: dwrite da/f3a [4194304,4194304] 0 2024-03-22T07:15:54.939 INFO:tasks.workunit.client.0.smithi063.stdout:3/331: fdatasync da/d16/f38 0 2024-03-22T07:15:54.939 INFO:tasks.workunit.client.1.smithi069.stdout:1/85: fsync d1/f18 0 2024-03-22T07:15:54.939 INFO:tasks.workunit.client.0.smithi063.stdout:0/391: mknod da/de/d60/d56/c7d 0 2024-03-22T07:15:54.940 INFO:tasks.workunit.client.1.smithi069.stdout:2/84: getdents d18/d1c 0 2024-03-22T07:15:54.940 INFO:tasks.workunit.client.0.smithi063.stdout:3/332: dread da/d16/d26/f27 [0,4194304] 0 2024-03-22T07:15:54.940 INFO:tasks.workunit.client.0.smithi063.stdout:3/333: chown da/d16/f38 477479985 1 2024-03-22T07:15:54.942 INFO:tasks.workunit.client.0.smithi063.stdout:6/345: symlink d3/d15/d16/d60/d6d/d4c/l71 0 2024-03-22T07:15:54.943 INFO:tasks.workunit.client.1.smithi069.stdout:6/62: mkdir d4/dd/d12 0 2024-03-22T07:15:54.944 INFO:tasks.workunit.client.0.smithi063.stdout:3/334: mkdir da/d16/d26/d69 0 2024-03-22T07:15:54.946 INFO:tasks.workunit.client.1.smithi069.stdout:3/89: creat d5/d9/d1e/f21 x:0 0 0 2024-03-22T07:15:54.946 INFO:tasks.workunit.client.0.smithi063.stdout:6/346: link d3/d6/f10 d3/d6/d2a/d2e/f72 0 2024-03-22T07:15:54.947 INFO:tasks.workunit.client.1.smithi069.stdout:9/81: truncate d2/f5 1921829 0 2024-03-22T07:15:54.947 INFO:tasks.workunit.client.0.smithi063.stdout:0/392: getdents da/de/d17/d25 0 2024-03-22T07:15:54.948 INFO:tasks.workunit.client.0.smithi063.stdout:0/393: chown da/de/d17/d25/d3c 292 1 2024-03-22T07:15:54.948 INFO:tasks.workunit.client.1.smithi069.stdout:9/82: write d2/dd/d18/f1f [887675,55832] 0 2024-03-22T07:15:54.949 INFO:tasks.workunit.client.1.smithi069.stdout:7/100: rename f3 to d9/f18 0 2024-03-22T07:15:54.950 INFO:tasks.workunit.client.1.smithi069.stdout:3/90: read f2 [406767,84595] 0 2024-03-22T07:15:54.950 INFO:tasks.workunit.client.1.smithi069.stdout:3/91: write d5/d9/d1e/f21 [329619,6552] 0 2024-03-22T07:15:54.950 INFO:tasks.workunit.client.1.smithi069.stdout:1/86: creat d1/d13/d16/f19 x:0 0 0 2024-03-22T07:15:54.951 INFO:tasks.workunit.client.1.smithi069.stdout:2/85: rmdir d18 39 2024-03-22T07:15:54.952 INFO:tasks.workunit.client.0.smithi063.stdout:3/335: creat da/f6a x:0 0 0 2024-03-22T07:15:54.952 INFO:tasks.workunit.client.0.smithi063.stdout:6/347: mkdir d3/d15/d16/d60/d73 0 2024-03-22T07:15:54.953 INFO:tasks.workunit.client.0.smithi063.stdout:6/348: truncate d3/d6/d35/d3a/f55 4326698 0 2024-03-22T07:15:54.953 INFO:tasks.workunit.client.0.smithi063.stdout:0/394: symlink da/de/d17/d25/d3c/l7e 0 2024-03-22T07:15:54.954 INFO:tasks.workunit.client.1.smithi069.stdout:6/63: mkdir d4/dd/d12/d13 0 2024-03-22T07:15:54.956 INFO:tasks.workunit.client.0.smithi063.stdout:3/336: link da/d16/d3f/f44 da/d16/d26/d69/f6b 0 2024-03-22T07:15:54.956 INFO:tasks.workunit.client.0.smithi063.stdout:3/337: write da/d63/d65/f66 [180548,103119] 0 2024-03-22T07:15:54.956 INFO:tasks.workunit.client.0.smithi063.stdout:3/338: chown da/d63/d65/d4b/c4f 1257788 1 2024-03-22T07:15:54.956 INFO:tasks.workunit.client.0.smithi063.stdout:6/349: mknod d3/d15/d16/d47/c74 0 2024-03-22T07:15:54.956 INFO:tasks.workunit.client.0.smithi063.stdout:3/339: write da/d16/d3f/f4c [393453,616] 0 2024-03-22T07:15:54.957 INFO:tasks.workunit.client.1.smithi069.stdout:1/87: write d1/fa [2234281,682] 0 2024-03-22T07:15:54.957 INFO:tasks.workunit.client.0.smithi063.stdout:0/395: dread da/f54 [0,4194304] 0 2024-03-22T07:15:54.957 INFO:tasks.workunit.client.1.smithi069.stdout:9/83: rename d2/dd/d18/f1f to d2/dd/d18/f20 0 2024-03-22T07:15:54.958 INFO:tasks.workunit.client.1.smithi069.stdout:2/86: mkdir d18/d1c/d25 0 2024-03-22T07:15:54.959 INFO:tasks.workunit.client.0.smithi063.stdout:0/396: mknod da/de/d60/d56/c7f 0 2024-03-22T07:15:54.959 INFO:tasks.workunit.client.0.smithi063.stdout:0/397: write da/d49/f63 [1855113,65864] 0 2024-03-22T07:15:54.959 INFO:tasks.workunit.client.0.smithi063.stdout:0/398: truncate da/f54 1737831 0 2024-03-22T07:15:54.959 INFO:tasks.workunit.client.0.smithi063.stdout:0/399: readlink da/de/d17/d25/l48 0 2024-03-22T07:15:54.961 INFO:tasks.workunit.client.1.smithi069.stdout:6/64: creat d4/f14 x:0 0 0 2024-03-22T07:15:54.961 INFO:tasks.workunit.client.1.smithi069.stdout:6/65: symlink d4/dd/l15 0 2024-03-22T07:15:54.961 INFO:tasks.workunit.client.0.smithi063.stdout:0/400: mkdir da/de/d17/d25/d3c/d43/d80 0 2024-03-22T07:15:54.962 INFO:tasks.workunit.client.0.smithi063.stdout:0/401: chown da/de/d17/d25/c4c 115905 1 2024-03-22T07:15:54.962 INFO:tasks.workunit.client.1.smithi069.stdout:3/92: fdatasync d5/d9/d1e/f21 0 2024-03-22T07:15:54.962 INFO:tasks.workunit.client.1.smithi069.stdout:3/93: write f2 [324612,50302] 0 2024-03-22T07:15:54.962 INFO:tasks.workunit.client.1.smithi069.stdout:3/94: readlink d5/d9/l1d 0 2024-03-22T07:15:54.962 INFO:tasks.workunit.client.1.smithi069.stdout:3/95: dread - d5/f18 zero size 2024-03-22T07:15:54.962 INFO:tasks.workunit.client.1.smithi069.stdout:3/96: creat d5/d9/d1e/f22 x:0 0 0 2024-03-22T07:15:54.965 INFO:tasks.workunit.client.1.smithi069.stdout:6/66: mknod d4/dd/d12/c16 0 2024-03-22T07:15:54.970 INFO:tasks.workunit.client.1.smithi069.stdout:6/67: unlink d4/dd/d12/c16 0 2024-03-22T07:15:54.971 INFO:tasks.workunit.client.1.smithi069.stdout:6/68: symlink d4/dd/d12/l17 0 2024-03-22T07:15:54.971 INFO:tasks.workunit.client.1.smithi069.stdout:6/69: readlink d4/dd/d12/l17 0 2024-03-22T07:15:54.974 INFO:tasks.workunit.client.1.smithi069.stdout:1/88: dread d1/fa [0,4194304] 0 2024-03-22T07:15:54.974 INFO:tasks.workunit.client.1.smithi069.stdout:1/89: truncate d1/d13/d16/f19 692185 0 2024-03-22T07:15:54.974 INFO:tasks.workunit.client.1.smithi069.stdout:1/90: truncate d1/d13/f17 976536 0 2024-03-22T07:15:54.974 INFO:tasks.workunit.client.1.smithi069.stdout:1/91: stat d1/f18 0 2024-03-22T07:15:54.975 INFO:tasks.workunit.client.1.smithi069.stdout:1/92: fsync d1/fb 0 2024-03-22T07:15:54.975 INFO:tasks.workunit.client.1.smithi069.stdout:1/93: write d1/fb [1338633,22871] 0 2024-03-22T07:15:54.975 INFO:tasks.workunit.client.1.smithi069.stdout:1/94: write d1/f18 [2394394,91758] 0 2024-03-22T07:15:54.977 INFO:tasks.workunit.client.1.smithi069.stdout:1/95: symlink d1/d13/d16/l1a 0 2024-03-22T07:15:54.979 INFO:tasks.workunit.client.1.smithi069.stdout:1/96: link d1/fb d1/d13/f1b 0 2024-03-22T07:15:54.981 INFO:tasks.workunit.client.1.smithi069.stdout:1/97: symlink d1/d13/l1c 0 2024-03-22T07:15:54.990 INFO:tasks.workunit.client.1.smithi069.stdout:5/45: dwrite d1/f9 [0,4194304] 0 2024-03-22T07:15:54.990 INFO:tasks.workunit.client.1.smithi069.stdout:2/87: dwrite d18/d1c/f22 [0,4194304] 0 2024-03-22T07:15:54.992 INFO:tasks.workunit.client.0.smithi063.stdout:3/340: dwrite da/d16/f32 [0,4194304] 0 2024-03-22T07:15:54.992 INFO:tasks.workunit.client.0.smithi063.stdout:6/350: dwrite d3/d1c/f63 [0,4194304] 0 2024-03-22T07:15:54.993 INFO:tasks.workunit.client.1.smithi069.stdout:6/70: truncate d4/fb 4035719 0 2024-03-22T07:15:54.993 INFO:tasks.workunit.client.1.smithi069.stdout:6/71: write d4/f14 [543507,99327] 0 2024-03-22T07:15:54.996 INFO:tasks.workunit.client.1.smithi069.stdout:2/88: rename ff to d18/d1c/f26 0 2024-03-22T07:15:54.997 INFO:tasks.workunit.client.0.smithi063.stdout:3/341: mknod da/d63/c6c 0 2024-03-22T07:15:55.001 INFO:tasks.workunit.client.0.smithi063.stdout:6/351: unlink d3/d15/d16/d60/d6d/d4e/f50 0 2024-03-22T07:15:55.001 INFO:tasks.workunit.client.0.smithi063.stdout:3/342: creat da/d16/f6d x:0 0 0 2024-03-22T07:15:55.001 INFO:tasks.workunit.client.1.smithi069.stdout:6/72: mknod d4/dd/c18 0 2024-03-22T07:15:55.001 INFO:tasks.workunit.client.1.smithi069.stdout:7/101: dwrite d9/f18 [0,4194304] 0 2024-03-22T07:15:55.001 INFO:tasks.workunit.client.1.smithi069.stdout:7/102: write d9/db/ff [249827,15335] 0 2024-03-22T07:15:55.003 INFO:tasks.workunit.client.1.smithi069.stdout:2/89: unlink d18/c20 0 2024-03-22T07:15:55.004 INFO:tasks.workunit.client.0.smithi063.stdout:6/352: creat d3/d15/d16/d60/d6d/d4c/f75 x:0 0 0 2024-03-22T07:15:55.007 INFO:tasks.workunit.client.0.smithi063.stdout:6/353: rename d3/d6/d2a/d2e to d3/d76 0 2024-03-22T07:15:55.007 INFO:tasks.workunit.client.0.smithi063.stdout:6/354: chown d3/d6/d2a/c34 0 1 2024-03-22T07:15:55.007 INFO:tasks.workunit.client.0.smithi063.stdout:6/355: fsync d3/d1c/f4b 0 2024-03-22T07:15:55.007 INFO:tasks.workunit.client.0.smithi063.stdout:6/356: read - d3/d6/d2a/f48 zero size 2024-03-22T07:15:55.008 INFO:tasks.workunit.client.0.smithi063.stdout:6/357: truncate d3/d15/d16/f26 1943338 0 2024-03-22T07:15:55.018 INFO:tasks.workunit.client.1.smithi069.stdout:2/90: dread d18/f19 [0,4194304] 0 2024-03-22T07:15:55.019 INFO:tasks.workunit.client.1.smithi069.stdout:6/73: rename d4/d7/c8 to d4/dd/d12/c19 0 2024-03-22T07:15:55.021 INFO:tasks.workunit.client.1.smithi069.stdout:9/84: read d2/f5 [1921524,55955] 0 2024-03-22T07:15:55.024 INFO:tasks.workunit.client.1.smithi069.stdout:3/97: dwrite d5/d9/d1e/f22 [0,4194304] 0 2024-03-22T07:15:55.026 INFO:tasks.workunit.client.0.smithi063.stdout:0/402: dwrite da/de/f62 [0,4194304] 0 2024-03-22T07:15:55.026 INFO:tasks.workunit.client.0.smithi063.stdout:0/403: stat da/de/d17/f45 0 2024-03-22T07:15:55.031 INFO:tasks.workunit.client.1.smithi069.stdout:6/74: truncate d4/f6 748620 0 2024-03-22T07:15:55.031 INFO:tasks.workunit.client.1.smithi069.stdout:6/75: getdents d4/dd/d12/d13 0 2024-03-22T07:15:55.032 INFO:tasks.workunit.client.1.smithi069.stdout:9/85: getdents d2 0 2024-03-22T07:15:55.032 INFO:tasks.workunit.client.1.smithi069.stdout:1/98: dwrite d1/d13/d16/f19 [0,4194304] 0 2024-03-22T07:15:55.033 INFO:tasks.workunit.client.0.smithi063.stdout:0/404: getdents da/de/d17 0 2024-03-22T07:15:55.033 INFO:tasks.workunit.client.0.smithi063.stdout:0/405: read - da/de/d17/f5d zero size 2024-03-22T07:15:55.033 INFO:tasks.workunit.client.0.smithi063.stdout:0/406: read - da/de/d17/d25/d3c/f66 zero size 2024-03-22T07:15:55.033 INFO:tasks.workunit.client.0.smithi063.stdout:0/407: rename da/de/d60/d56/c7d to da/de/d17/d25/d3c/d5f/c81 0 2024-03-22T07:15:55.036 INFO:tasks.workunit.client.0.smithi063.stdout:0/408: unlink da/de/d17/d25/d3c/d64/c73 0 2024-03-22T07:15:55.036 INFO:tasks.workunit.client.0.smithi063.stdout:0/409: chown da/de/f24 199821802 1 2024-03-22T07:15:55.038 INFO:tasks.workunit.client.1.smithi069.stdout:3/98: write d5/d9/d1e/f22 [1213920,2762] 0 2024-03-22T07:15:55.038 INFO:tasks.workunit.client.1.smithi069.stdout:3/99: read d5/d9/f19 [2987772,122826] 0 2024-03-22T07:15:55.039 INFO:tasks.workunit.client.0.smithi063.stdout:6/358: write d3/d15/d16/f26 [1945043,16140] 0 2024-03-22T07:15:55.039 INFO:tasks.workunit.client.0.smithi063.stdout:6/359: mknod d3/d15/d16/d60/d6d/d4e/d5a/c77 0 2024-03-22T07:15:55.040 INFO:tasks.workunit.client.0.smithi063.stdout:6/360: rmdir d3/d54 0 2024-03-22T07:15:55.040 INFO:tasks.workunit.client.0.smithi063.stdout:6/361: creat d3/d15/f78 x:0 0 0 2024-03-22T07:15:55.041 INFO:tasks.workunit.client.0.smithi063.stdout:6/362: chown d3/d1c/c2d 334774 1 2024-03-22T07:15:55.041 INFO:tasks.workunit.client.0.smithi063.stdout:6/363: truncate d3/d76/f5b 691986 0 2024-03-22T07:15:55.041 INFO:tasks.workunit.client.0.smithi063.stdout:6/364: write d3/d15/f62 [219398,31819] 0 2024-03-22T07:15:55.042 INFO:tasks.workunit.client.1.smithi069.stdout:1/99: symlink d1/d13/d16/l1d 0 2024-03-22T07:15:55.043 INFO:tasks.workunit.client.0.smithi063.stdout:6/365: symlink d3/d1c/d32/d51/l79 0 2024-03-22T07:15:55.044 INFO:tasks.workunit.client.1.smithi069.stdout:1/100: mkdir d1/d13/d16/d1e 0 2024-03-22T07:15:55.045 INFO:tasks.workunit.client.0.smithi063.stdout:6/366: mknod d3/d15/d16/d60/d6d/d4e/d5a/c7a 0 2024-03-22T07:15:55.046 INFO:tasks.workunit.client.0.smithi063.stdout:6/367: rmdir d3/d15/d16/d60/d73 0 2024-03-22T07:15:55.046 INFO:tasks.workunit.client.0.smithi063.stdout:3/343: dwrite da/f3d [4194304,4194304] 0 2024-03-22T07:15:55.047 INFO:tasks.workunit.client.0.smithi063.stdout:6/368: mkdir d3/d15/d16/d60/d7b 0 2024-03-22T07:15:55.047 INFO:tasks.workunit.client.0.smithi063.stdout:6/369: readlink d3/l6a 0 2024-03-22T07:15:55.047 INFO:tasks.workunit.client.1.smithi069.stdout:1/101: rename d1/d13/d16/l1a to d1/d13/l1f 0 2024-03-22T07:15:55.048 INFO:tasks.workunit.client.0.smithi063.stdout:3/344: mknod da/d16/d3f/c6e 0 2024-03-22T07:15:55.049 INFO:tasks.workunit.client.1.smithi069.stdout:1/102: creat d1/d13/d16/d1e/f20 x:0 0 0 2024-03-22T07:15:55.049 INFO:tasks.workunit.client.0.smithi063.stdout:6/370: creat d3/f7c x:0 0 0 2024-03-22T07:15:55.056 INFO:tasks.workunit.client.1.smithi069.stdout:5/46: dwrite d1/f9 [4194304,4194304] 0 2024-03-22T07:15:55.057 INFO:tasks.workunit.client.1.smithi069.stdout:2/91: rmdir d18 39 2024-03-22T07:15:55.058 INFO:tasks.workunit.client.0.smithi063.stdout:9/238: sync 2024-03-22T07:15:55.058 INFO:tasks.workunit.client.1.smithi069.stdout:5/47: write d1/f3 [365745,50216] 0 2024-03-22T07:15:55.058 INFO:tasks.workunit.client.0.smithi063.stdout:9/239: creat d8/d17/f43 x:0 0 0 2024-03-22T07:15:55.062 INFO:tasks.workunit.client.1.smithi069.stdout:2/92: mkdir d18/d1c/d27 0 2024-03-22T07:15:55.063 INFO:tasks.workunit.client.1.smithi069.stdout:2/93: dread d18/d1c/f26 [0,4194304] 0 2024-03-22T07:15:55.063 INFO:tasks.workunit.client.1.smithi069.stdout:2/94: write d18/d1c/f24 [4362076,118666] 0 2024-03-22T07:15:55.065 INFO:tasks.workunit.client.0.smithi063.stdout:6/371: dread d3/d15/f68 [0,4194304] 0 2024-03-22T07:15:55.065 INFO:tasks.workunit.client.0.smithi063.stdout:6/372: fsync d3/d15/d16/f25 0 2024-03-22T07:15:55.065 INFO:tasks.workunit.client.0.smithi063.stdout:6/373: chown d3/d1c/f5d 8 1 2024-03-22T07:15:55.065 INFO:tasks.workunit.client.0.smithi063.stdout:6/374: fdatasync d3/f14 0 2024-03-22T07:15:55.065 INFO:tasks.workunit.client.0.smithi063.stdout:6/375: write d3/d15/d16/f25 [758967,15989] 0 2024-03-22T07:15:55.065 INFO:tasks.workunit.client.0.smithi063.stdout:6/376: creat d3/d15/d16/d60/d6d/d4c/f7d x:0 0 0 2024-03-22T07:15:55.066 INFO:tasks.workunit.client.1.smithi069.stdout:5/48: symlink d1/d7/d8/lc 0 2024-03-22T07:15:55.066 INFO:tasks.workunit.client.1.smithi069.stdout:7/103: dwrite f2 [0,4194304] 0 2024-03-22T07:15:55.066 INFO:tasks.workunit.client.1.smithi069.stdout:7/104: truncate f6 4470931 0 2024-03-22T07:15:55.070 INFO:tasks.workunit.client.1.smithi069.stdout:2/95: creat d18/d1c/f28 x:0 0 0 2024-03-22T07:15:55.071 INFO:tasks.workunit.client.1.smithi069.stdout:5/49: mknod d1/d7/cd 0 2024-03-22T07:15:55.071 INFO:tasks.workunit.client.1.smithi069.stdout:5/50: chown d1/d6 6849976 1 2024-03-22T07:15:55.072 INFO:tasks.workunit.client.1.smithi069.stdout:9/86: dwrite d2/dd/d18/f20 [0,4194304] 0 2024-03-22T07:15:55.073 INFO:tasks.workunit.client.0.smithi063.stdout:6/377: dread d3/d6/d35/d3a/f55 [0,4194304] 0 2024-03-22T07:15:55.073 INFO:tasks.workunit.client.0.smithi063.stdout:0/410: dwrite da/de/d17/f21 [0,4194304] 0 2024-03-22T07:15:55.073 INFO:tasks.workunit.client.0.smithi063.stdout:6/378: getdents d3/d1c/d32/d51/d64 0 2024-03-22T07:15:55.077 INFO:tasks.workunit.client.0.smithi063.stdout:3/345: dwrite da/f3d [0,4194304] 0 2024-03-22T07:15:55.077 INFO:tasks.workunit.client.0.smithi063.stdout:3/346: fdatasync da/d16/d3f/f56 0 2024-03-22T07:15:55.077 INFO:tasks.workunit.client.1.smithi069.stdout:6/76: truncate d4/f6 1051934 0 2024-03-22T07:15:55.078 INFO:tasks.workunit.client.1.smithi069.stdout:0/95: sync 2024-03-22T07:15:55.078 INFO:tasks.workunit.client.1.smithi069.stdout:2/96: mknod d18/d1c/d27/c29 0 2024-03-22T07:15:55.080 INFO:tasks.workunit.client.1.smithi069.stdout:5/51: mkdir d1/d6/de 0 2024-03-22T07:15:55.080 INFO:tasks.workunit.client.1.smithi069.stdout:5/52: chown d1/d7/cb 973351 1 2024-03-22T07:15:55.080 INFO:tasks.workunit.client.1.smithi069.stdout:5/53: chown d1/d6 53269937 1 2024-03-22T07:15:55.082 INFO:tasks.workunit.client.0.smithi063.stdout:9/240: dwrite d8/f14 [0,4194304] 0 2024-03-22T07:15:55.082 INFO:tasks.workunit.client.0.smithi063.stdout:9/241: mknod d8/d19/d20/d22/d29/c44 0 2024-03-22T07:15:55.083 INFO:tasks.workunit.client.0.smithi063.stdout:9/242: creat d8/d19/f45 x:0 0 0 2024-03-22T07:15:55.083 INFO:tasks.workunit.client.1.smithi069.stdout:1/103: dread d1/f18 [0,4194304] 0 2024-03-22T07:15:55.083 INFO:tasks.workunit.client.1.smithi069.stdout:1/104: write d1/fa [2965495,207] 0 2024-03-22T07:15:55.083 INFO:tasks.workunit.client.1.smithi069.stdout:1/105: write d1/f18 [1977197,111781] 0 2024-03-22T07:15:55.084 INFO:tasks.workunit.client.0.smithi063.stdout:3/347: rmdir da 39 2024-03-22T07:15:55.084 INFO:tasks.workunit.client.1.smithi069.stdout:7/105: link d9/f18 d9/db/f19 0 2024-03-22T07:15:55.085 INFO:tasks.workunit.client.0.smithi063.stdout:3/348: creat da/d16/d3f/f6f x:0 0 0 2024-03-22T07:15:55.086 INFO:tasks.workunit.client.0.smithi063.stdout:3/349: read da/d16/d26/f2a [430397,32063] 0 2024-03-22T07:15:55.086 INFO:tasks.workunit.client.0.smithi063.stdout:3/350: dread - da/d16/d3f/f6f zero size 2024-03-22T07:15:55.086 INFO:tasks.workunit.client.0.smithi063.stdout:3/351: creat da/d16/d3f/f70 x:0 0 0 2024-03-22T07:15:55.086 INFO:tasks.workunit.client.0.smithi063.stdout:3/352: stat da/d16/d26/d69/f6b 0 2024-03-22T07:15:55.087 INFO:tasks.workunit.client.1.smithi069.stdout:2/97: symlink d18/d1c/l2a 0 2024-03-22T07:15:55.092 INFO:tasks.workunit.client.1.smithi069.stdout:7/106: mkdir d9/db/d1a 0 2024-03-22T07:15:55.093 INFO:tasks.workunit.client.1.smithi069.stdout:6/77: write f3 [7463623,115409] 0 2024-03-22T07:15:55.095 INFO:tasks.workunit.client.1.smithi069.stdout:2/98: creat d18/f2b x:0 0 0 2024-03-22T07:15:55.096 INFO:tasks.workunit.client.1.smithi069.stdout:0/96: dread d3/da/f11 [0,4194304] 0 2024-03-22T07:15:55.096 INFO:tasks.workunit.client.0.smithi063.stdout:9/243: dread d8/d17/f1d [0,4194304] 0 2024-03-22T07:15:55.097 INFO:tasks.workunit.client.1.smithi069.stdout:7/107: unlink d9/db/ff 0 2024-03-22T07:15:55.097 INFO:tasks.workunit.client.0.smithi063.stdout:9/244: truncate f2 834251 0 2024-03-22T07:15:55.098 INFO:tasks.workunit.client.1.smithi069.stdout:2/99: creat d18/d1c/d25/f2c x:0 0 0 2024-03-22T07:15:55.099 INFO:tasks.workunit.client.0.smithi063.stdout:3/353: dread da/d16/d26/f2a [0,4194304] 0 2024-03-22T07:15:55.099 INFO:tasks.workunit.client.0.smithi063.stdout:5/319: sync 2024-03-22T07:15:55.099 INFO:tasks.workunit.client.0.smithi063.stdout:5/320: chown d8/d13/d28/c47 14230335 1 2024-03-22T07:15:55.099 INFO:tasks.workunit.client.0.smithi063.stdout:5/321: creat d8/d13/d28/d57/f5c x:0 0 0 2024-03-22T07:15:55.099 INFO:tasks.workunit.client.0.smithi063.stdout:5/322: chown d8/d13/d1e/l5b 142229 1 2024-03-22T07:15:55.099 INFO:tasks.workunit.client.1.smithi069.stdout:0/97: mkdir d3/d7/d24 0 2024-03-22T07:15:55.099 INFO:tasks.workunit.client.1.smithi069.stdout:0/98: chown d3/d7/l19 98530602 1 2024-03-22T07:15:55.101 INFO:tasks.workunit.client.0.smithi063.stdout:3/354: truncate da/d16/f25 1598139 0 2024-03-22T07:15:55.102 INFO:tasks.workunit.client.1.smithi069.stdout:0/99: write d3/d7/d15/f1b [222743,42553] 0 2024-03-22T07:15:55.102 INFO:tasks.workunit.client.0.smithi063.stdout:5/323: creat d8/d13/f5d x:0 0 0 2024-03-22T07:15:55.102 INFO:tasks.workunit.client.0.smithi063.stdout:5/324: chown d8/d13/c26 0 1 2024-03-22T07:15:55.103 INFO:tasks.workunit.client.1.smithi069.stdout:0/100: truncate d3/d7/d15/d1f/f21 1013153 0 2024-03-22T07:15:55.104 INFO:tasks.workunit.client.0.smithi063.stdout:9/245: dread d8/d19/d20/f21 [0,4194304] 0 2024-03-22T07:15:55.104 INFO:tasks.workunit.client.0.smithi063.stdout:3/355: creat da/f71 x:0 0 0 2024-03-22T07:15:55.104 INFO:tasks.workunit.client.0.smithi063.stdout:3/356: chown da/d16/d26/d69 62308222 1 2024-03-22T07:15:55.104 INFO:tasks.workunit.client.0.smithi063.stdout:9/246: link d8/d19/f35 d8/d15/f46 0 2024-03-22T07:15:55.105 INFO:tasks.workunit.client.0.smithi063.stdout:3/357: link l9 da/d16/l72 0 2024-03-22T07:15:55.106 INFO:tasks.workunit.client.0.smithi063.stdout:5/325: rename d8/d13/d24/d53 to d8/d5e 0 2024-03-22T07:15:55.107 INFO:tasks.workunit.client.1.smithi069.stdout:4/83: sync 2024-03-22T07:15:55.107 INFO:tasks.workunit.client.1.smithi069.stdout:8/44: sync 2024-03-22T07:15:55.107 INFO:tasks.workunit.client.1.smithi069.stdout:4/84: chown d6/dc 129248 1 2024-03-22T07:15:55.107 INFO:tasks.workunit.client.0.smithi063.stdout:9/247: link d8/fa d8/d15/d2f/f47 0 2024-03-22T07:15:55.108 INFO:tasks.workunit.client.0.smithi063.stdout:3/358: mknod da/d63/d65/d4b/c73 0 2024-03-22T07:15:55.110 INFO:tasks.workunit.client.0.smithi063.stdout:5/326: mkdir d8/d13/d28/d57/d5f 0 2024-03-22T07:15:55.110 INFO:tasks.workunit.client.0.smithi063.stdout:5/327: truncate d8/d13/d1e/d3c/f49 499039 0 2024-03-22T07:15:55.110 INFO:tasks.workunit.client.0.smithi063.stdout:5/328: fdatasync d8/d13/f48 0 2024-03-22T07:15:55.110 INFO:tasks.workunit.client.1.smithi069.stdout:6/78: dread d4/fb [0,4194304] 0 2024-03-22T07:15:55.111 INFO:tasks.workunit.client.1.smithi069.stdout:4/85: mkdir d6/d12/d18/d1c 0 2024-03-22T07:15:55.112 INFO:tasks.workunit.client.0.smithi063.stdout:3/359: rename da/d16/d26/c59 to da/d63/d65/c74 0 2024-03-22T07:15:55.113 INFO:tasks.workunit.client.1.smithi069.stdout:8/45: rmdir d1 39 2024-03-22T07:15:55.114 INFO:tasks.workunit.client.0.smithi063.stdout:6/379: dwrite d3/f9 [0,4194304] 0 2024-03-22T07:15:55.115 INFO:tasks.workunit.client.0.smithi063.stdout:6/380: fsync d3/d15/f23 0 2024-03-22T07:15:55.115 INFO:tasks.workunit.client.0.smithi063.stdout:6/381: symlink d3/d15/d16/d69/l7e 0 2024-03-22T07:15:55.115 INFO:tasks.workunit.client.0.smithi063.stdout:6/382: read d3/f14 [563832,51125] 0 2024-03-22T07:15:55.115 INFO:tasks.workunit.client.0.smithi063.stdout:6/383: creat d3/d15/f7f x:0 0 0 2024-03-22T07:15:55.115 INFO:tasks.workunit.client.0.smithi063.stdout:6/384: write d3/d15/d16/f59 [202768,71878] 0 2024-03-22T07:15:55.115 INFO:tasks.workunit.client.0.smithi063.stdout:6/385: dread - d3/d6/d2a/f48 zero size 2024-03-22T07:15:55.116 INFO:tasks.workunit.client.1.smithi069.stdout:7/108: write f6 [2714258,105368] 0 2024-03-22T07:15:55.116 INFO:tasks.workunit.client.1.smithi069.stdout:4/86: rmdir d6/d12/d18 39 2024-03-22T07:15:55.116 INFO:tasks.workunit.client.1.smithi069.stdout:4/87: chown d6/d12/d18/l19 20 1 2024-03-22T07:15:55.116 INFO:tasks.workunit.client.1.smithi069.stdout:4/88: write d6/f9 [9237766,62742] 0 2024-03-22T07:15:55.116 INFO:tasks.workunit.client.1.smithi069.stdout:4/89: fdatasync d6/dc/d10/f17 0 2024-03-22T07:15:55.117 INFO:tasks.workunit.client.1.smithi069.stdout:4/90: write d6/dc/d10/f14 [54803,109571] 0 2024-03-22T07:15:55.117 INFO:tasks.workunit.client.1.smithi069.stdout:8/46: getdents d1 0 2024-03-22T07:15:55.117 INFO:tasks.workunit.client.1.smithi069.stdout:8/47: stat d1 0 2024-03-22T07:15:55.118 INFO:tasks.workunit.client.0.smithi063.stdout:5/329: dread d8/d13/d1e/d31/f4e [0,4194304] 0 2024-03-22T07:15:55.118 INFO:tasks.workunit.client.0.smithi063.stdout:5/330: truncate d8/d13/d24/f50 584304 0 2024-03-22T07:15:55.118 INFO:tasks.workunit.client.0.smithi063.stdout:5/331: dread - d8/d13/f5d zero size 2024-03-22T07:15:55.118 INFO:tasks.workunit.client.0.smithi063.stdout:5/332: chown d8/d13/d28/c47 93300 1 2024-03-22T07:15:55.118 INFO:tasks.workunit.client.0.smithi063.stdout:0/411: dwrite da/d49/f63 [0,4194304] 0 2024-03-22T07:15:55.118 INFO:tasks.workunit.client.0.smithi063.stdout:0/412: stat da/de/d39/l42 0 2024-03-22T07:15:55.119 INFO:tasks.workunit.client.0.smithi063.stdout:0/413: write da/de/d17/d25/f27 [302840,36987] 0 2024-03-22T07:15:55.119 INFO:tasks.workunit.client.1.smithi069.stdout:7/109: symlink d9/db/d1a/l1b 0 2024-03-22T07:15:55.123 INFO:tasks.workunit.client.1.smithi069.stdout:4/91: mkdir d6/dc/d1d 0 2024-03-22T07:15:55.123 INFO:tasks.workunit.client.1.smithi069.stdout:4/92: write d6/dc/ff [3011396,62024] 0 2024-03-22T07:15:55.123 INFO:tasks.workunit.client.1.smithi069.stdout:4/93: write f5 [675702,41645] 0 2024-03-22T07:15:55.124 INFO:tasks.workunit.client.1.smithi069.stdout:8/48: rmdir d1 39 2024-03-22T07:15:55.125 INFO:tasks.workunit.client.1.smithi069.stdout:8/49: chown d1/f2 47307030 1 2024-03-22T07:15:55.127 INFO:tasks.workunit.client.0.smithi063.stdout:5/333: dread d8/f2b [0,4194304] 0 2024-03-22T07:15:55.129 INFO:tasks.workunit.client.0.smithi063.stdout:6/386: dread d3/d6/d2a/d38/f6c [0,4194304] 0 2024-03-22T07:15:55.129 INFO:tasks.workunit.client.0.smithi063.stdout:6/387: stat d3/d15/d16/d60/d6d/d4c/l53 0 2024-03-22T07:15:55.129 INFO:tasks.workunit.client.1.smithi069.stdout:0/101: dwrite d3/d7/d15/d1f/f21 [0,4194304] 0 2024-03-22T07:15:55.138 INFO:tasks.workunit.client.1.smithi069.stdout:7/110: mknod d9/c1c 0 2024-03-22T07:15:55.139 INFO:tasks.workunit.client.1.smithi069.stdout:2/100: getdents d18/d1c/d25 0 2024-03-22T07:15:55.144 INFO:tasks.workunit.client.0.smithi063.stdout:0/414: dread da/de/d17/f21 [0,4194304] 0 2024-03-22T07:15:55.144 INFO:tasks.workunit.client.1.smithi069.stdout:8/50: write d1/f5 [1650716,51939] 0 2024-03-22T07:15:55.146 INFO:tasks.workunit.client.1.smithi069.stdout:0/102: write d3/f8 [2594358,104719] 0 2024-03-22T07:15:55.146 INFO:tasks.workunit.client.1.smithi069.stdout:2/101: write f5 [226298,69249] 0 2024-03-22T07:15:55.147 INFO:tasks.workunit.client.1.smithi069.stdout:0/103: rmdir d3/d7/d15/d1f 39 2024-03-22T07:15:55.150 INFO:tasks.workunit.client.1.smithi069.stdout:0/104: link d3/d7/lf d3/da/l25 0 2024-03-22T07:15:55.151 INFO:tasks.workunit.client.1.smithi069.stdout:0/105: chown d3/da/f14 837638771 1 2024-03-22T07:15:55.152 INFO:tasks.workunit.client.1.smithi069.stdout:7/111: write f2 [142129,53633] 0 2024-03-22T07:15:55.153 INFO:tasks.workunit.client.1.smithi069.stdout:2/102: write f1 [225867,71903] 0 2024-03-22T07:15:55.153 INFO:tasks.workunit.client.1.smithi069.stdout:2/103: chown cc 16 1 2024-03-22T07:15:55.153 INFO:tasks.workunit.client.1.smithi069.stdout:0/106: creat d3/d7/d15/d1f/f26 x:0 0 0 2024-03-22T07:15:55.156 INFO:tasks.workunit.client.1.smithi069.stdout:0/107: symlink d3/d7/l27 0 2024-03-22T07:15:55.166 INFO:tasks.workunit.client.0.smithi063.stdout:0/415: dread da/de/d17/d4e/f57 [0,4194304] 0 2024-03-22T07:15:55.166 INFO:tasks.workunit.client.0.smithi063.stdout:0/416: dread - da/de/d17/d25/f70 zero size 2024-03-22T07:15:55.166 INFO:tasks.workunit.client.0.smithi063.stdout:0/417: creat da/f82 x:0 0 0 2024-03-22T07:15:55.166 INFO:tasks.workunit.client.0.smithi063.stdout:0/418: chown da/f2b 323083 1 2024-03-22T07:15:55.166 INFO:tasks.workunit.client.0.smithi063.stdout:9/248: dwrite d8/d17/f26 [0,4194304] 0 2024-03-22T07:15:55.169 INFO:tasks.workunit.client.0.smithi063.stdout:5/334: dwrite d8/f14 [0,4194304] 0 2024-03-22T07:15:55.169 INFO:tasks.workunit.client.0.smithi063.stdout:9/249: getdents d8/d19/d2b 0 2024-03-22T07:15:55.170 INFO:tasks.workunit.client.1.smithi069.stdout:7/112: write d9/f14 [3992808,41248] 0 2024-03-22T07:15:55.170 INFO:tasks.workunit.client.1.smithi069.stdout:7/113: read f6 [3483339,106482] 0 2024-03-22T07:15:55.170 INFO:tasks.workunit.client.1.smithi069.stdout:2/104: read d18/d1c/f22 [2898219,88139] 0 2024-03-22T07:15:55.170 INFO:tasks.workunit.client.1.smithi069.stdout:1/106: dwrite d1/f18 [0,4194304] 0 2024-03-22T07:15:55.170 INFO:tasks.workunit.client.1.smithi069.stdout:1/107: truncate d1/fb 2322173 0 2024-03-22T07:15:55.170 INFO:tasks.workunit.client.1.smithi069.stdout:1/108: readlink d1/l10 0 2024-03-22T07:15:55.170 INFO:tasks.workunit.client.0.smithi063.stdout:3/360: dwrite da/d16/d26/f27 [0,4194304] 0 2024-03-22T07:15:55.171 INFO:tasks.workunit.client.1.smithi069.stdout:6/79: dwrite d4/fb [4194304,4194304] 0 2024-03-22T07:15:55.171 INFO:tasks.workunit.client.0.smithi063.stdout:9/250: mkdir d8/d19/d41/d48 0 2024-03-22T07:15:55.173 INFO:tasks.workunit.client.0.smithi063.stdout:6/388: dwrite d3/d1c/f63 [0,4194304] 0 2024-03-22T07:15:55.173 INFO:tasks.workunit.client.1.smithi069.stdout:2/105: rename f3 to d18/d1c/f2d 0 2024-03-22T07:15:55.173 INFO:tasks.workunit.client.1.smithi069.stdout:0/108: dread d3/d7/d15/f1b [0,4194304] 0 2024-03-22T07:15:55.173 INFO:tasks.workunit.client.1.smithi069.stdout:1/109: dread d1/d13/f17 [0,4194304] 0 2024-03-22T07:15:55.173 INFO:tasks.workunit.client.1.smithi069.stdout:1/110: write d1/fa [2693862,80380] 0 2024-03-22T07:15:55.174 INFO:tasks.workunit.client.0.smithi063.stdout:5/335: dread d8/d13/d1e/f38 [0,4194304] 0 2024-03-22T07:15:55.177 INFO:tasks.workunit.client.1.smithi069.stdout:6/80: truncate d4/dd/f11 3128905 0 2024-03-22T07:15:55.178 INFO:tasks.workunit.client.1.smithi069.stdout:1/111: truncate d1/d13/f17 493572 0 2024-03-22T07:15:55.183 INFO:tasks.workunit.client.1.smithi069.stdout:6/81: rmdir d4/d7/d10 0 2024-03-22T07:15:55.183 INFO:tasks.workunit.client.1.smithi069.stdout:6/82: chown d4/fb 14 1 2024-03-22T07:15:55.183 INFO:tasks.workunit.client.1.smithi069.stdout:6/83: rename d4/d7 to d4/d7/da/d1a 22 2024-03-22T07:15:55.185 INFO:tasks.workunit.client.1.smithi069.stdout:2/106: dread d18/d1c/f22 [0,4194304] 0 2024-03-22T07:15:55.186 INFO:tasks.workunit.client.1.smithi069.stdout:1/112: symlink d1/l21 0 2024-03-22T07:15:55.186 INFO:tasks.workunit.client.1.smithi069.stdout:1/113: truncate d1/d13/d16/f19 4432575 0 2024-03-22T07:15:55.188 INFO:tasks.workunit.client.1.smithi069.stdout:7/114: write d9/f18 [1354353,49397] 0 2024-03-22T07:15:55.189 INFO:tasks.workunit.client.1.smithi069.stdout:1/114: creat d1/f22 x:0 0 0 2024-03-22T07:15:55.190 INFO:tasks.workunit.client.1.smithi069.stdout:7/115: mkdir d9/db/d13/d1d 0 2024-03-22T07:15:55.191 INFO:tasks.workunit.client.0.smithi063.stdout:3/361: rmdir da/d63/d65/d4b/d68 0 2024-03-22T07:15:55.191 INFO:tasks.workunit.client.1.smithi069.stdout:1/115: stat d1/le 0 2024-03-22T07:15:55.192 INFO:tasks.workunit.client.1.smithi069.stdout:8/51: truncate d1/f5 2973226 0 2024-03-22T07:15:55.194 INFO:tasks.workunit.client.0.smithi063.stdout:9/251: unlink d8/d19/d20/d22/f2d 0 2024-03-22T07:15:55.195 INFO:tasks.workunit.client.0.smithi063.stdout:9/252: write d8/d17/f43 [731528,119719] 0 2024-03-22T07:15:55.195 INFO:tasks.workunit.client.1.smithi069.stdout:7/116: symlink d9/db/d13/l1e 0 2024-03-22T07:15:55.195 INFO:tasks.workunit.client.1.smithi069.stdout:8/52: creat d1/f9 x:0 0 0 2024-03-22T07:15:55.195 INFO:tasks.workunit.client.1.smithi069.stdout:7/117: mknod d9/db/d13/c1f 0 2024-03-22T07:15:55.197 INFO:tasks.workunit.client.1.smithi069.stdout:8/53: mknod d1/ca 0 2024-03-22T07:15:55.198 INFO:tasks.workunit.client.0.smithi063.stdout:6/389: link d3/d15/d16/d60/d6d/d4c/l3c d3/d15/d16/d60/d6d/d4e/d5f/l80 0 2024-03-22T07:15:55.200 INFO:tasks.workunit.client.1.smithi069.stdout:7/118: creat d9/db/d13/f20 x:0 0 0 2024-03-22T07:15:55.200 INFO:tasks.workunit.client.1.smithi069.stdout:7/119: dread - d9/db/f17 zero size 2024-03-22T07:15:55.200 INFO:tasks.workunit.client.1.smithi069.stdout:8/54: symlink d1/lb 0 2024-03-22T07:15:55.200 INFO:tasks.workunit.client.1.smithi069.stdout:8/55: write d1/f9 [968367,101999] 0 2024-03-22T07:15:55.201 INFO:tasks.workunit.client.1.smithi069.stdout:8/56: fsync d1/f6 0 2024-03-22T07:15:55.201 INFO:tasks.workunit.client.1.smithi069.stdout:8/57: truncate d1/f8 784616 0 2024-03-22T07:15:55.201 INFO:tasks.workunit.client.1.smithi069.stdout:7/120: getdents d9/db/d1a 0 2024-03-22T07:15:55.201 INFO:tasks.workunit.client.1.smithi069.stdout:7/121: write d9/db/f16 [5126939,25724] 0 2024-03-22T07:15:55.201 INFO:tasks.workunit.client.1.smithi069.stdout:7/122: write d9/f18 [863766,58477] 0 2024-03-22T07:15:55.202 INFO:tasks.workunit.client.1.smithi069.stdout:7/123: creat d9/db/d1a/f21 x:0 0 0 2024-03-22T07:15:55.203 INFO:tasks.workunit.client.0.smithi063.stdout:3/362: creat da/d63/f75 x:0 0 0 2024-03-22T07:15:55.203 INFO:tasks.workunit.client.0.smithi063.stdout:9/253: symlink d8/d19/l49 0 2024-03-22T07:15:55.203 INFO:tasks.workunit.client.0.smithi063.stdout:6/390: symlink d3/d15/d16/d60/d6d/l81 0 2024-03-22T07:15:55.203 INFO:tasks.workunit.client.0.smithi063.stdout:9/254: creat d8/d19/d41/f4a x:0 0 0 2024-03-22T07:15:55.206 INFO:tasks.workunit.client.0.smithi063.stdout:6/391: creat d3/f82 x:0 0 0 2024-03-22T07:15:55.206 INFO:tasks.workunit.client.0.smithi063.stdout:6/392: write d3/d6/d2a/f5e [1620972,28171] 0 2024-03-22T07:15:55.206 INFO:tasks.workunit.client.0.smithi063.stdout:9/255: mkdir d8/d19/d20/d22/d29/d31/d4b 0 2024-03-22T07:15:55.206 INFO:tasks.workunit.client.0.smithi063.stdout:6/393: rmdir d3/d1c/d32 39 2024-03-22T07:15:55.206 INFO:tasks.workunit.client.0.smithi063.stdout:9/256: symlink d8/l4c 0 2024-03-22T07:15:55.207 INFO:tasks.workunit.client.1.smithi069.stdout:7/124: mknod d9/db/d1a/c22 0 2024-03-22T07:15:55.207 INFO:tasks.workunit.client.1.smithi069.stdout:7/125: mknod d9/db/d13/c23 0 2024-03-22T07:15:55.207 INFO:tasks.workunit.client.1.smithi069.stdout:7/126: creat d9/db/f24 x:0 0 0 2024-03-22T07:15:55.207 INFO:tasks.workunit.client.1.smithi069.stdout:7/127: write f2 [2181299,94139] 0 2024-03-22T07:15:55.207 INFO:tasks.workunit.client.1.smithi069.stdout:1/116: read d1/f18 [2958797,42279] 0 2024-03-22T07:15:55.210 INFO:tasks.workunit.client.1.smithi069.stdout:7/128: link d9/db/d13/l1e d9/db/d13/l25 0 2024-03-22T07:15:55.210 INFO:tasks.workunit.client.1.smithi069.stdout:7/129: chown d9/db/d13/c23 2 1 2024-03-22T07:15:55.211 INFO:tasks.workunit.client.1.smithi069.stdout:7/130: creat d9/db/f26 x:0 0 0 2024-03-22T07:15:55.211 INFO:tasks.workunit.client.1.smithi069.stdout:1/117: chown d1/lf 376 1 2024-03-22T07:15:55.212 INFO:tasks.workunit.client.0.smithi063.stdout:9/257: creat d8/d19/d20/d22/d29/d31/d4b/f4d x:0 0 0 2024-03-22T07:15:55.212 INFO:tasks.workunit.client.0.smithi063.stdout:9/258: dread - d8/d19/d20/d22/d29/d31/d4b/f4d zero size 2024-03-22T07:15:55.212 INFO:tasks.workunit.client.0.smithi063.stdout:9/259: creat d8/d19/d20/d22/d29/d31/f4e x:0 0 0 2024-03-22T07:15:55.212 INFO:tasks.workunit.client.0.smithi063.stdout:6/394: getdents d3/d6/d35 0 2024-03-22T07:15:55.212 INFO:tasks.workunit.client.0.smithi063.stdout:0/419: dwrite da/de/f31 [4194304,4194304] 0 2024-03-22T07:15:55.213 INFO:tasks.workunit.client.0.smithi063.stdout:0/420: dread - da/de/d60/f69 zero size 2024-03-22T07:15:55.213 INFO:tasks.workunit.client.0.smithi063.stdout:6/395: link d3/d6/d2a/d38/f6c d3/d15/d16/d69/f83 0 2024-03-22T07:15:55.213 INFO:tasks.workunit.client.0.smithi063.stdout:0/421: readlink da/de/d17/d25/d3c/d43/l6f 0 2024-03-22T07:15:55.213 INFO:tasks.workunit.client.0.smithi063.stdout:0/422: write da/de/d17/f28 [3628037,68813] 0 2024-03-22T07:15:55.213 INFO:tasks.workunit.client.0.smithi063.stdout:0/423: stat da/de/d17 0 2024-03-22T07:15:55.213 INFO:tasks.workunit.client.0.smithi063.stdout:5/336: dwrite d8/d13/d24/f50 [0,4194304] 0 2024-03-22T07:15:55.214 INFO:tasks.workunit.client.0.smithi063.stdout:0/424: link da/de/d17/f5d da/de/d17/d25/d3c/d43/f83 0 2024-03-22T07:15:55.217 INFO:tasks.workunit.client.1.smithi069.stdout:0/109: dwrite d3/d7/d15/d1f/f21 [0,4194304] 0 2024-03-22T07:15:55.217 INFO:tasks.workunit.client.1.smithi069.stdout:0/110: dread - d3/d7/d15/d1f/f26 zero size 2024-03-22T07:15:55.217 INFO:tasks.workunit.client.1.smithi069.stdout:4/94: dwrite d6/f9 [8388608,4194304] 0 2024-03-22T07:15:55.217 INFO:tasks.workunit.client.1.smithi069.stdout:0/111: write d3/d7/fb [1464340,61278] 0 2024-03-22T07:15:55.217 INFO:tasks.workunit.client.1.smithi069.stdout:4/95: write d6/dc/ff [5946510,96240] 0 2024-03-22T07:15:55.218 INFO:tasks.workunit.client.1.smithi069.stdout:7/131: rename d9/c11 to d9/c27 0 2024-03-22T07:15:55.218 INFO:tasks.workunit.client.1.smithi069.stdout:2/107: dread d18/d1c/f26 [0,4194304] 0 2024-03-22T07:15:55.218 INFO:tasks.workunit.client.1.smithi069.stdout:1/118: symlink d1/d13/l23 0 2024-03-22T07:15:55.218 INFO:tasks.workunit.client.1.smithi069.stdout:1/119: write d1/f22 [565811,41533] 0 2024-03-22T07:15:55.218 INFO:tasks.workunit.client.1.smithi069.stdout:1/120: chown d1/d13/l1c 867481136 1 2024-03-22T07:15:55.218 INFO:tasks.workunit.client.1.smithi069.stdout:1/121: fdatasync d1/d13/f1b 0 2024-03-22T07:15:55.219 INFO:tasks.workunit.client.1.smithi069.stdout:1/122: rename d1 to d1/d13/d16/d24 22 2024-03-22T07:15:55.219 INFO:tasks.workunit.client.1.smithi069.stdout:2/108: dread f5 [0,4194304] 0 2024-03-22T07:15:55.219 INFO:tasks.workunit.client.1.smithi069.stdout:8/58: dread d1/f9 [0,4194304] 0 2024-03-22T07:15:55.219 INFO:tasks.workunit.client.1.smithi069.stdout:4/96: dread d6/dc/d10/f17 [0,4194304] 0 2024-03-22T07:15:55.219 INFO:tasks.workunit.client.1.smithi069.stdout:4/97: write d6/dc/ff [2106723,122235] 0 2024-03-22T07:15:55.221 INFO:tasks.workunit.client.0.smithi063.stdout:5/337: symlink d8/d13/l60 0 2024-03-22T07:15:55.222 INFO:tasks.workunit.client.0.smithi063.stdout:5/338: dread - d8/d13/f39 zero size 2024-03-22T07:15:55.222 INFO:tasks.workunit.client.0.smithi063.stdout:5/339: symlink d8/d13/d28/d57/d5f/l61 0 2024-03-22T07:15:55.222 INFO:tasks.workunit.client.0.smithi063.stdout:5/340: mkdir d8/d13/d24/d62 0 2024-03-22T07:15:55.222 INFO:tasks.workunit.client.0.smithi063.stdout:2/315: sync 2024-03-22T07:15:55.222 INFO:tasks.workunit.client.0.smithi063.stdout:4/289: sync 2024-03-22T07:15:55.222 INFO:tasks.workunit.client.0.smithi063.stdout:8/398: sync 2024-03-22T07:15:55.223 INFO:tasks.workunit.client.0.smithi063.stdout:1/395: sync 2024-03-22T07:15:55.223 INFO:tasks.workunit.client.0.smithi063.stdout:4/290: fsync d1/f45 0 2024-03-22T07:15:55.223 INFO:tasks.workunit.client.0.smithi063.stdout:8/399: write d0/fc [3445430,66665] 0 2024-03-22T07:15:55.223 INFO:tasks.workunit.client.0.smithi063.stdout:7/263: sync 2024-03-22T07:15:55.223 INFO:tasks.workunit.client.0.smithi063.stdout:8/400: fsync d0/f62 0 2024-03-22T07:15:55.224 INFO:tasks.workunit.client.1.smithi069.stdout:7/132: symlink d9/db/l28 0 2024-03-22T07:15:55.224 INFO:tasks.workunit.client.0.smithi063.stdout:0/425: write da/de/d17/f21 [420455,34138] 0 2024-03-22T07:15:55.225 INFO:tasks.workunit.client.0.smithi063.stdout:1/396: mkdir d4/df/d13/d2b/d86 0 2024-03-22T07:15:55.225 INFO:tasks.workunit.client.1.smithi069.stdout:2/109: creat d18/d1c/d25/f2e x:0 0 0 2024-03-22T07:15:55.225 INFO:tasks.workunit.client.0.smithi063.stdout:2/316: creat d0/de/d12/f6a x:0 0 0 2024-03-22T07:15:55.227 INFO:tasks.workunit.client.1.smithi069.stdout:8/59: rename d1/f8 to d1/fc 0 2024-03-22T07:15:55.229 INFO:tasks.workunit.client.1.smithi069.stdout:0/112: rmdir d3/d7 39 2024-03-22T07:15:55.230 INFO:tasks.workunit.client.0.smithi063.stdout:7/264: rename d5/d16/d1b/f2d to d5/d16/d43/f56 0 2024-03-22T07:15:55.230 INFO:tasks.workunit.client.0.smithi063.stdout:7/265: write d5/d16/d1b/f52 [517973,50990] 0 2024-03-22T07:15:55.230 INFO:tasks.workunit.client.0.smithi063.stdout:7/266: truncate d5/da/f35 1124910 0 2024-03-22T07:15:55.233 INFO:tasks.workunit.client.0.smithi063.stdout:0/426: mknod da/de/d17/c84 0 2024-03-22T07:15:55.233 INFO:tasks.workunit.client.1.smithi069.stdout:1/123: mkdir d1/d25 0 2024-03-22T07:15:55.234 INFO:tasks.workunit.client.1.smithi069.stdout:1/124: write d1/f22 [1009706,43339] 0 2024-03-22T07:15:55.234 INFO:tasks.workunit.client.1.smithi069.stdout:1/125: readlink d1/l11 0 2024-03-22T07:15:55.234 INFO:tasks.workunit.client.1.smithi069.stdout:1/126: write d1/f22 [1418740,29378] 0 2024-03-22T07:15:55.235 INFO:tasks.workunit.client.1.smithi069.stdout:1/127: dread d1/d13/f17 [0,4194304] 0 2024-03-22T07:15:55.236 INFO:tasks.workunit.client.1.smithi069.stdout:1/128: dread - d1/d13/d16/d1e/f20 zero size 2024-03-22T07:15:55.236 INFO:tasks.workunit.client.1.smithi069.stdout:1/129: truncate d1/d13/d16/d1e/f20 102981 0 2024-03-22T07:15:55.236 INFO:tasks.workunit.client.1.smithi069.stdout:1/130: creat d1/d13/f26 x:0 0 0 2024-03-22T07:15:55.236 INFO:tasks.workunit.client.1.smithi069.stdout:1/131: creat d1/d13/f27 x:0 0 0 2024-03-22T07:15:55.236 INFO:tasks.workunit.client.1.smithi069.stdout:8/60: rename d1/f5 to d1/fd 0 2024-03-22T07:15:55.237 INFO:tasks.workunit.client.0.smithi063.stdout:2/317: mknod d0/de/c6b 0 2024-03-22T07:15:55.237 INFO:tasks.workunit.client.0.smithi063.stdout:4/291: dread d1/f21 [0,4194304] 0 2024-03-22T07:15:55.239 INFO:tasks.workunit.client.1.smithi069.stdout:1/132: creat d1/d25/f28 x:0 0 0 2024-03-22T07:15:55.239 INFO:tasks.workunit.client.0.smithi063.stdout:8/401: dread d0/d26/f27 [0,4194304] 0 2024-03-22T07:15:55.240 INFO:tasks.workunit.client.0.smithi063.stdout:1/397: rename d4/d14/f30 to d4/df/d13/d2b/d86/f87 0 2024-03-22T07:15:55.240 INFO:tasks.workunit.client.0.smithi063.stdout:1/398: chown d4/d14/f50 1039929 1 2024-03-22T07:15:55.240 INFO:tasks.workunit.client.0.smithi063.stdout:8/402: read - d0/d24/d33/d4c/f75 zero size 2024-03-22T07:15:55.240 INFO:tasks.workunit.client.0.smithi063.stdout:8/403: fdatasync d0/d24/d33/f5e 0 2024-03-22T07:15:55.240 INFO:tasks.workunit.client.0.smithi063.stdout:8/404: fdatasync d0/d24/d33/d4c/f75 0 2024-03-22T07:15:55.241 INFO:tasks.workunit.client.1.smithi069.stdout:8/61: mkdir d1/de 0 2024-03-22T07:15:55.241 INFO:tasks.workunit.client.1.smithi069.stdout:8/62: read d1/f9 [990072,123453] 0 2024-03-22T07:15:55.241 INFO:tasks.workunit.client.1.smithi069.stdout:8/63: readlink d1/lb 0 2024-03-22T07:15:55.244 INFO:tasks.workunit.client.1.smithi069.stdout:8/64: unlink d1/ca 0 2024-03-22T07:15:55.244 INFO:tasks.workunit.client.0.smithi063.stdout:4/292: creat d1/d5/d38/d59/f64 x:0 0 0 2024-03-22T07:15:55.245 INFO:tasks.workunit.client.0.smithi063.stdout:1/399: mkdir d4/d71/d88 0 2024-03-22T07:15:55.245 INFO:tasks.workunit.client.0.smithi063.stdout:1/400: readlink d4/d5a/l7f 0 2024-03-22T07:15:55.246 INFO:tasks.workunit.client.1.smithi069.stdout:1/133: rmdir d1/d13/d16 39 2024-03-22T07:15:55.247 INFO:tasks.workunit.client.1.smithi069.stdout:8/65: truncate d1/f2 2420940 0 2024-03-22T07:15:55.248 INFO:tasks.workunit.client.0.smithi063.stdout:8/405: mkdir d0/d24/d33/d4c/d78 0 2024-03-22T07:15:55.248 INFO:tasks.workunit.client.0.smithi063.stdout:8/406: stat d0/d1b/d28/f3b 0 2024-03-22T07:15:55.253 INFO:tasks.workunit.client.0.smithi063.stdout:9/260: truncate d8/d17/f43 119450 0 2024-03-22T07:15:55.253 INFO:tasks.workunit.client.0.smithi063.stdout:1/401: unlink d4/df/d13/l4f 0 2024-03-22T07:15:55.253 INFO:tasks.workunit.client.0.smithi063.stdout:8/407: getdents d0/d26/d2d 0 2024-03-22T07:15:55.255 INFO:tasks.workunit.client.0.smithi063.stdout:9/261: creat d8/d19/d20/d22/f4f x:0 0 0 2024-03-22T07:15:55.255 INFO:tasks.workunit.client.0.smithi063.stdout:9/262: write d8/fd [2470003,81761] 0 2024-03-22T07:15:55.256 INFO:tasks.workunit.client.0.smithi063.stdout:2/318: write d0/de/f14 [730177,60484] 0 2024-03-22T07:15:55.259 INFO:tasks.workunit.client.0.smithi063.stdout:1/402: unlink d4/f59 0 2024-03-22T07:15:55.259 INFO:tasks.workunit.client.0.smithi063.stdout:1/403: stat d4/df/d13/d2b/d86/f87 0 2024-03-22T07:15:55.259 INFO:tasks.workunit.client.0.smithi063.stdout:9/263: symlink d8/d17/d1e/l50 0 2024-03-22T07:15:55.260 INFO:tasks.workunit.client.0.smithi063.stdout:9/264: fsync d8/d19/d20/d22/d29/d31/d4b/f4d 0 2024-03-22T07:15:55.262 INFO:tasks.workunit.client.0.smithi063.stdout:2/319: creat d0/de/d1b/d63/f6c x:0 0 0 2024-03-22T07:15:55.263 INFO:tasks.workunit.client.1.smithi069.stdout:7/133: dread d9/db/f16 [0,4194304] 0 2024-03-22T07:15:55.263 INFO:tasks.workunit.client.0.smithi063.stdout:2/320: creat d0/de/d35/d36/f6d x:0 0 0 2024-03-22T07:15:55.265 INFO:tasks.workunit.client.0.smithi063.stdout:2/321: mkdir d0/de/d1b/d63/d6e 0 2024-03-22T07:15:55.265 INFO:tasks.workunit.client.0.smithi063.stdout:2/322: write d0/f60 [1849508,129044] 0 2024-03-22T07:15:55.269 INFO:tasks.workunit.client.1.smithi069.stdout:4/98: dwrite d6/dc/f1a [0,4194304] 0 2024-03-22T07:15:55.269 INFO:tasks.workunit.client.1.smithi069.stdout:4/99: getdents d6/dc/d1d 0 2024-03-22T07:15:55.274 INFO:tasks.workunit.client.1.smithi069.stdout:4/100: link d6/l8 d6/dc/l1e 0 2024-03-22T07:15:55.275 INFO:tasks.workunit.client.1.smithi069.stdout:4/101: rmdir d6/d12/d18/d1c 0 2024-03-22T07:15:55.275 INFO:tasks.workunit.client.0.smithi063.stdout:1/404: dread d4/df/f15 [0,4194304] 0 2024-03-22T07:15:55.276 INFO:tasks.workunit.client.0.smithi063.stdout:9/265: write d8/d15/f1c [4189996,113955] 0 2024-03-22T07:15:55.277 INFO:tasks.workunit.client.0.smithi063.stdout:1/405: creat d4/df/d13/d2b/d6c/f89 x:0 0 0 2024-03-22T07:15:55.279 INFO:tasks.workunit.client.0.smithi063.stdout:1/406: mknod d4/d71/d88/c8a 0 2024-03-22T07:15:55.279 INFO:tasks.workunit.client.0.smithi063.stdout:1/407: write d4/df/f1a [1363493,121326] 0 2024-03-22T07:15:55.282 INFO:tasks.workunit.client.0.smithi063.stdout:2/323: dread d0/de/d12/d20/f2a [0,4194304] 0 2024-03-22T07:15:55.282 INFO:tasks.workunit.client.0.smithi063.stdout:2/324: getdents d0/de/d58 0 2024-03-22T07:15:55.283 INFO:tasks.workunit.client.0.smithi063.stdout:5/341: dwrite d8/d13/d24/f33 [4194304,4194304] 0 2024-03-22T07:15:55.284 INFO:tasks.workunit.client.1.smithi069.stdout:2/110: dwrite d18/d1c/f28 [0,4194304] 0 2024-03-22T07:15:55.284 INFO:tasks.workunit.client.1.smithi069.stdout:2/111: write f5 [1795635,77463] 0 2024-03-22T07:15:55.286 INFO:tasks.workunit.client.1.smithi069.stdout:2/112: rename l16 to d18/l2f 0 2024-03-22T07:15:55.286 INFO:tasks.workunit.client.1.smithi069.stdout:2/113: stat d18/l1a 0 2024-03-22T07:15:55.287 INFO:tasks.workunit.client.0.smithi063.stdout:2/325: creat d0/de/d35/f6f x:0 0 0 2024-03-22T07:15:55.287 INFO:tasks.workunit.client.0.smithi063.stdout:2/326: stat d0/de/f65 0 2024-03-22T07:15:55.293 INFO:tasks.workunit.client.0.smithi063.stdout:7/267: rmdir d5/d16 39 2024-03-22T07:15:55.293 INFO:tasks.workunit.client.1.smithi069.stdout:1/134: rmdir d1 39 2024-03-22T07:15:55.297 INFO:tasks.workunit.client.1.smithi069.stdout:1/135: symlink d1/l29 0 2024-03-22T07:15:55.298 INFO:tasks.workunit.client.0.smithi063.stdout:1/408: write d4/df/d13/d39/d63/f77 [2047112,60039] 0 2024-03-22T07:15:55.300 INFO:tasks.workunit.client.0.smithi063.stdout:5/342: dread d8/d13/d1e/f27 [0,4194304] 0 2024-03-22T07:15:55.300 INFO:tasks.workunit.client.1.smithi069.stdout:0/113: dwrite d3/d7/d15/d1f/f21 [0,4194304] 0 2024-03-22T07:15:55.301 INFO:tasks.workunit.client.0.smithi063.stdout:0/427: dwrite da/de/d17/d25/f3e [0,4194304] 0 2024-03-22T07:15:55.301 INFO:tasks.workunit.client.0.smithi063.stdout:4/293: dwrite d1/f21 [4194304,4194304] 0 2024-03-22T07:15:55.302 INFO:tasks.workunit.client.0.smithi063.stdout:1/409: mkdir d4/d8b 0 2024-03-22T07:15:55.302 INFO:tasks.workunit.client.0.smithi063.stdout:1/410: chown d4/f12 1192696 1 2024-03-22T07:15:55.302 INFO:tasks.workunit.client.0.smithi063.stdout:1/411: creat d4/f8c x:0 0 0 2024-03-22T07:15:55.303 INFO:tasks.workunit.client.0.smithi063.stdout:1/412: readlink d4/df/l3f 0 2024-03-22T07:15:55.303 INFO:tasks.workunit.client.1.smithi069.stdout:4/102: dwrite f3 [0,4194304] 0 2024-03-22T07:15:55.303 INFO:tasks.workunit.client.1.smithi069.stdout:8/66: dwrite d1/fd [0,4194304] 0 2024-03-22T07:15:55.303 INFO:tasks.workunit.client.1.smithi069.stdout:7/134: dwrite d9/f14 [0,4194304] 0 2024-03-22T07:15:55.305 INFO:tasks.workunit.client.0.smithi063.stdout:8/408: dwrite d0/d24/f5d [0,4194304] 0 2024-03-22T07:15:55.305 INFO:tasks.workunit.client.0.smithi063.stdout:0/428: creat da/de/d17/d25/d3c/d64/d78/f85 x:0 0 0 2024-03-22T07:15:55.305 INFO:tasks.workunit.client.0.smithi063.stdout:0/429: dread - da/de/d17/d25/d3c/d64/d78/f85 zero size 2024-03-22T07:15:55.306 INFO:tasks.workunit.client.1.smithi069.stdout:4/103: rmdir d6 39 2024-03-22T07:15:55.306 INFO:tasks.workunit.client.0.smithi063.stdout:4/294: mkdir d1/d5/d3c/d65 0 2024-03-22T07:15:55.307 INFO:tasks.workunit.client.0.smithi063.stdout:1/413: write d4/df/d13/d39/f5f [1508538,23588] 0 2024-03-22T07:15:55.307 INFO:tasks.workunit.client.0.smithi063.stdout:1/414: creat d4/df/d13/d39/f8d x:0 0 0 2024-03-22T07:15:55.307 INFO:tasks.workunit.client.0.smithi063.stdout:1/415: creat d4/df/d13/d39/f8e x:0 0 0 2024-03-22T07:15:55.308 INFO:tasks.workunit.client.1.smithi069.stdout:8/67: rename d1/lb to d1/de/lf 0 2024-03-22T07:15:55.309 INFO:tasks.workunit.client.0.smithi063.stdout:7/268: dread d5/d16/d1b/d21/f2a [0,4194304] 0 2024-03-22T07:15:55.312 INFO:tasks.workunit.client.1.smithi069.stdout:7/135: unlink d9/db/d13/l1e 0 2024-03-22T07:15:55.313 INFO:tasks.workunit.client.0.smithi063.stdout:8/409: rename d0/d18/d45/f6a to d0/d1b/d4f/f79 0 2024-03-22T07:15:55.314 INFO:tasks.workunit.client.0.smithi063.stdout:9/266: fsync d8/d17/f43 0 2024-03-22T07:15:55.314 INFO:tasks.workunit.client.0.smithi063.stdout:9/267: chown d8/d19 0 1 2024-03-22T07:15:55.314 INFO:tasks.workunit.client.0.smithi063.stdout:9/268: creat d8/d19/d20/d22/d29/d31/f51 x:0 0 0 2024-03-22T07:15:55.315 INFO:tasks.workunit.client.1.smithi069.stdout:7/136: symlink d9/db/d13/d1d/l29 0 2024-03-22T07:15:55.315 INFO:tasks.workunit.client.1.smithi069.stdout:7/137: write d9/db/f26 [99760,34941] 0 2024-03-22T07:15:55.316 INFO:tasks.workunit.client.0.smithi063.stdout:5/343: dread d8/f14 [0,4194304] 0 2024-03-22T07:15:55.316 INFO:tasks.workunit.client.0.smithi063.stdout:0/430: creat da/de/d17/d25/d3c/d43/d80/f86 x:0 0 0 2024-03-22T07:15:55.319 INFO:tasks.workunit.client.0.smithi063.stdout:4/295: write d1/d5/dc/f1d [4461431,107280] 0 2024-03-22T07:15:55.319 INFO:tasks.workunit.client.1.smithi069.stdout:8/68: dread d1/f6 [0,4194304] 0 2024-03-22T07:15:55.320 INFO:tasks.workunit.client.1.smithi069.stdout:8/69: symlink d1/l10 0 2024-03-22T07:15:55.320 INFO:tasks.workunit.client.1.smithi069.stdout:8/70: write d1/f9 [1634507,96147] 0 2024-03-22T07:15:55.325 INFO:tasks.workunit.client.0.smithi063.stdout:2/327: dwrite d0/f10 [0,4194304] 0 2024-03-22T07:15:55.325 INFO:tasks.workunit.client.0.smithi063.stdout:2/328: creat d0/de/d12/d20/f70 x:0 0 0 2024-03-22T07:15:55.325 INFO:tasks.workunit.client.0.smithi063.stdout:2/329: chown d0/de/d12/d20/l61 46964630 1 2024-03-22T07:15:55.325 INFO:tasks.workunit.client.0.smithi063.stdout:2/330: truncate d0/de/d12/f1f 3268350 0 2024-03-22T07:15:55.326 INFO:tasks.workunit.client.1.smithi069.stdout:2/114: dwrite d18/f1e [0,4194304] 0 2024-03-22T07:15:55.327 INFO:tasks.workunit.client.1.smithi069.stdout:2/115: fsync f15 0 2024-03-22T07:15:55.329 INFO:tasks.workunit.client.0.smithi063.stdout:5/344: write d8/d13/d24/f33 [4361367,110655] 0 2024-03-22T07:15:55.330 INFO:tasks.workunit.client.0.smithi063.stdout:8/410: truncate d0/f1a 3965569 0 2024-03-22T07:15:55.331 INFO:tasks.workunit.client.0.smithi063.stdout:9/269: symlink d8/d19/l52 0 2024-03-22T07:15:55.332 INFO:tasks.workunit.client.0.smithi063.stdout:0/431: symlink da/l87 0 2024-03-22T07:15:55.333 INFO:tasks.workunit.client.1.smithi069.stdout:2/116: dread d18/d1c/f26 [0,4194304] 0 2024-03-22T07:15:55.333 INFO:tasks.workunit.client.0.smithi063.stdout:4/296: creat d1/d5/d43/d27/d2c/d3d/d54/d5f/f66 x:0 0 0 2024-03-22T07:15:55.335 INFO:tasks.workunit.client.0.smithi063.stdout:1/416: getdents d4/df/d13/d2b 0 2024-03-22T07:15:55.335 INFO:tasks.workunit.client.0.smithi063.stdout:1/417: write d4/df/f64 [140924,53179] 0 2024-03-22T07:15:55.335 INFO:tasks.workunit.client.1.smithi069.stdout:1/136: dwrite d1/d13/d16/f19 [4194304,4194304] 0 2024-03-22T07:15:55.335 INFO:tasks.workunit.client.1.smithi069.stdout:1/137: chown d1/d13/l1c 97 1 2024-03-22T07:15:55.337 INFO:tasks.workunit.client.1.smithi069.stdout:7/138: dwrite d9/db/f17 [0,4194304] 0 2024-03-22T07:15:55.340 INFO:tasks.workunit.client.1.smithi069.stdout:8/71: write d1/f2 [2175701,37543] 0 2024-03-22T07:15:55.340 INFO:tasks.workunit.client.1.smithi069.stdout:8/72: readlink d1/l10 0 2024-03-22T07:15:55.341 INFO:tasks.workunit.client.0.smithi063.stdout:5/345: rename d8/f2e to d8/d13/f63 0 2024-03-22T07:15:55.342 INFO:tasks.workunit.client.0.smithi063.stdout:8/411: mkdir d0/d1f/d7a 0 2024-03-22T07:15:55.344 INFO:tasks.workunit.client.1.smithi069.stdout:2/117: rename f15 to d18/d1c/f30 0 2024-03-22T07:15:55.344 INFO:tasks.workunit.client.0.smithi063.stdout:9/270: mkdir d8/d19/d20/d22/d29/d31/d53 0 2024-03-22T07:15:55.345 INFO:tasks.workunit.client.0.smithi063.stdout:0/432: link da/de/l2c da/de/l88 0 2024-03-22T07:15:55.345 INFO:tasks.workunit.client.0.smithi063.stdout:0/433: readlink da/l30 0 2024-03-22T07:15:55.347 INFO:tasks.workunit.client.0.smithi063.stdout:4/297: creat d1/d5/f67 x:0 0 0 2024-03-22T07:15:55.347 INFO:tasks.workunit.client.1.smithi069.stdout:2/118: dread d18/d1c/f26 [0,4194304] 0 2024-03-22T07:15:55.347 INFO:tasks.workunit.client.0.smithi063.stdout:1/418: getdents d4/d71/d88 0 2024-03-22T07:15:55.350 INFO:tasks.workunit.client.0.smithi063.stdout:5/346: dread d8/f1c [0,4194304] 0 2024-03-22T07:15:55.352 INFO:tasks.workunit.client.1.smithi069.stdout:7/139: mkdir d9/db/d13/d1d/d2a 0 2024-03-22T07:15:55.356 INFO:tasks.workunit.client.0.smithi063.stdout:2/331: dwrite d0/f13 [0,4194304] 0 2024-03-22T07:15:55.356 INFO:tasks.workunit.client.1.smithi069.stdout:1/138: dwrite d1/d13/f27 [0,4194304] 0 2024-03-22T07:15:55.356 INFO:tasks.workunit.client.1.smithi069.stdout:1/139: write d1/d25/f28 [819836,4462] 0 2024-03-22T07:15:55.357 INFO:tasks.workunit.client.1.smithi069.stdout:8/73: write d1/fd [2081591,61920] 0 2024-03-22T07:15:55.358 INFO:tasks.workunit.client.0.smithi063.stdout:9/271: write d8/fa [1335438,427] 0 2024-03-22T07:15:55.359 INFO:tasks.workunit.client.1.smithi069.stdout:2/119: rename c14 to d18/d1c/c31 0 2024-03-22T07:15:55.359 INFO:tasks.workunit.client.1.smithi069.stdout:2/120: chown d18/f2b 16768403 1 2024-03-22T07:15:55.359 INFO:tasks.workunit.client.1.smithi069.stdout:2/121: write f5 [1798295,24247] 0 2024-03-22T07:15:55.360 INFO:tasks.workunit.client.0.smithi063.stdout:8/412: rename d0/d18 to d0/d1b/d28/d7b 0 2024-03-22T07:15:55.360 INFO:tasks.workunit.client.0.smithi063.stdout:8/413: dread - d0/d1f/f5c zero size 2024-03-22T07:15:55.366 INFO:tasks.workunit.client.0.smithi063.stdout:8/414: creat d0/f7c x:0 0 0 2024-03-22T07:15:55.366 INFO:tasks.workunit.client.0.smithi063.stdout:4/298: rename d1/d5/c4f to d1/d5/d3c/c68 0 2024-03-22T07:15:55.368 INFO:tasks.workunit.client.0.smithi063.stdout:8/415: rename d0/d24/d33/d4c/f75 to d0/d24/d33/d4c/f7d 0 2024-03-22T07:15:55.368 INFO:tasks.workunit.client.0.smithi063.stdout:8/416: write d0/d24/f74 [153086,47043] 0 2024-03-22T07:15:55.370 INFO:tasks.workunit.client.0.smithi063.stdout:4/299: rename d1/d5/d3c/f51 to d1/d5/d3c/d65/f69 0 2024-03-22T07:15:55.372 INFO:tasks.workunit.client.1.smithi069.stdout:8/74: mknod d1/c11 0 2024-03-22T07:15:55.372 INFO:tasks.workunit.client.0.smithi063.stdout:4/300: creat d1/f6a x:0 0 0 2024-03-22T07:15:55.375 INFO:tasks.workunit.client.1.smithi069.stdout:8/75: mkdir d1/de/d12 0 2024-03-22T07:15:55.383 INFO:tasks.workunit.client.1.smithi069.stdout:7/140: rename d9/db/d13 to d9/d2b 0 2024-03-22T07:15:55.384 INFO:tasks.workunit.client.1.smithi069.stdout:7/141: creat d9/db/d1a/f2c x:0 0 0 2024-03-22T07:15:55.385 INFO:tasks.workunit.client.1.smithi069.stdout:7/142: truncate d9/db/f16 5063706 0 2024-03-22T07:15:55.385 INFO:tasks.workunit.client.1.smithi069.stdout:7/143: fdatasync d9/db/f24 0 2024-03-22T07:15:55.386 INFO:tasks.workunit.client.1.smithi069.stdout:7/144: symlink d9/d2b/l2d 0 2024-03-22T07:15:55.387 INFO:tasks.workunit.client.0.smithi063.stdout:1/419: dwrite d4/d14/f1b [0,4194304] 0 2024-03-22T07:15:55.387 INFO:tasks.workunit.client.0.smithi063.stdout:1/420: readlink d4/df/d13/d39/l45 0 2024-03-22T07:15:55.389 INFO:tasks.workunit.client.0.smithi063.stdout:1/421: unlink d4/df/d13/d2b/d3a/f74 0 2024-03-22T07:15:55.390 INFO:tasks.workunit.client.0.smithi063.stdout:1/422: mknod d4/df/c8f 0 2024-03-22T07:15:55.394 INFO:tasks.workunit.client.0.smithi063.stdout:5/347: dwrite d8/d13/d24/f50 [0,4194304] 0 2024-03-22T07:15:55.396 INFO:tasks.workunit.client.0.smithi063.stdout:5/348: unlink d8/d13/d24/c34 0 2024-03-22T07:15:55.397 INFO:tasks.workunit.client.0.smithi063.stdout:2/332: dwrite d0/de/d12/d20/f52 [0,4194304] 0 2024-03-22T07:15:55.398 INFO:tasks.workunit.client.0.smithi063.stdout:5/349: mkdir d8/d13/d1e/d31/d64 0 2024-03-22T07:15:55.400 INFO:tasks.workunit.client.0.smithi063.stdout:1/423: write d4/df/d13/d2b/d3a/f42 [1184522,101542] 0 2024-03-22T07:15:55.400 INFO:tasks.workunit.client.0.smithi063.stdout:1/424: fsync d4/df/f1a 0 2024-03-22T07:15:55.400 INFO:tasks.workunit.client.0.smithi063.stdout:4/301: dwrite d1/d5/d43/f17 [0,4194304] 0 2024-03-22T07:15:55.400 INFO:tasks.workunit.client.0.smithi063.stdout:1/425: chown d4/df/d13/d39/d63/c83 28629 1 2024-03-22T07:15:55.400 INFO:tasks.workunit.client.0.smithi063.stdout:1/426: stat d4/df/d13/f3c 0 2024-03-22T07:15:55.401 INFO:tasks.workunit.client.1.smithi069.stdout:8/76: dread d1/f9 [0,4194304] 0 2024-03-22T07:15:55.402 INFO:tasks.workunit.client.1.smithi069.stdout:8/77: mknod d1/de/c13 0 2024-03-22T07:15:55.402 INFO:tasks.workunit.client.1.smithi069.stdout:8/78: stat d1 0 2024-03-22T07:15:55.403 INFO:tasks.workunit.client.1.smithi069.stdout:8/79: creat d1/f14 x:0 0 0 2024-03-22T07:15:55.404 INFO:tasks.workunit.client.0.smithi063.stdout:4/302: link d1/d5/d43/f14 d1/d5/d43/d27/d2c/d3d/d54/f6b 0 2024-03-22T07:15:55.406 INFO:tasks.workunit.client.1.smithi069.stdout:8/80: read d1/f9 [290789,75870] 0 2024-03-22T07:15:55.406 INFO:tasks.workunit.client.1.smithi069.stdout:8/81: read d1/fc [254489,42779] 0 2024-03-22T07:15:55.406 INFO:tasks.workunit.client.1.smithi069.stdout:8/82: write d1/f2 [899927,102793] 0 2024-03-22T07:15:55.406 INFO:tasks.workunit.client.1.smithi069.stdout:8/83: chown d1/fd 0 1 2024-03-22T07:15:55.407 INFO:tasks.workunit.client.0.smithi063.stdout:4/303: mknod d1/c6c 0 2024-03-22T07:15:55.407 INFO:tasks.workunit.client.1.smithi069.stdout:8/84: link d1/l10 d1/de/l15 0 2024-03-22T07:15:55.407 INFO:tasks.workunit.client.1.smithi069.stdout:8/85: readlink d1/de/lf 0 2024-03-22T07:15:55.408 INFO:tasks.workunit.client.0.smithi063.stdout:4/304: mknod d1/d5/dc/c6d 0 2024-03-22T07:15:55.408 INFO:tasks.workunit.client.0.smithi063.stdout:4/305: chown d1/d5/d43/d27/d2c/d3d/f62 104 1 2024-03-22T07:15:55.408 INFO:tasks.workunit.client.0.smithi063.stdout:5/350: dread d8/f14 [0,4194304] 0 2024-03-22T07:15:55.409 INFO:tasks.workunit.client.0.smithi063.stdout:5/351: dread d8/d13/d1e/d31/f4e [0,4194304] 0 2024-03-22T07:15:55.410 INFO:tasks.workunit.client.0.smithi063.stdout:4/306: unlink d1/f11 0 2024-03-22T07:15:55.410 INFO:tasks.workunit.client.1.smithi069.stdout:8/86: rename d1/f2 to d1/de/d12/f16 0 2024-03-22T07:15:55.410 INFO:tasks.workunit.client.1.smithi069.stdout:8/87: fdatasync d1/fc 0 2024-03-22T07:15:55.410 INFO:tasks.workunit.client.1.smithi069.stdout:8/88: chown d1/c11 6459 1 2024-03-22T07:15:55.410 INFO:tasks.workunit.client.1.smithi069.stdout:8/89: write d1/f9 [206561,62642] 0 2024-03-22T07:15:55.411 INFO:tasks.workunit.client.0.smithi063.stdout:5/352: mkdir d8/d13/d1e/d31/d64/d65 0 2024-03-22T07:15:55.412 INFO:tasks.workunit.client.1.smithi069.stdout:5/54: sync 2024-03-22T07:15:55.412 INFO:tasks.workunit.client.1.smithi069.stdout:5/55: chown d1/d7/d8 241 1 2024-03-22T07:15:55.412 INFO:tasks.workunit.client.1.smithi069.stdout:9/87: sync 2024-03-22T07:15:55.413 INFO:tasks.workunit.client.1.smithi069.stdout:3/100: sync 2024-03-22T07:15:55.413 INFO:tasks.workunit.client.0.smithi063.stdout:2/333: write d0/fb [557123,127473] 0 2024-03-22T07:15:55.413 INFO:tasks.workunit.client.0.smithi063.stdout:2/334: chown d0/de/d12/d5a 57902181 1 2024-03-22T07:15:55.414 INFO:tasks.workunit.client.1.smithi069.stdout:8/90: creat d1/de/f17 x:0 0 0 2024-03-22T07:15:55.414 INFO:tasks.workunit.client.1.smithi069.stdout:5/56: rmdir d1/d7/d8 39 2024-03-22T07:15:55.414 INFO:tasks.workunit.client.1.smithi069.stdout:5/57: write d1/f3 [718313,57029] 0 2024-03-22T07:15:55.415 INFO:tasks.workunit.client.0.smithi063.stdout:4/307: rename d1/f32 to d1/d5/d38/f6e 0 2024-03-22T07:15:55.415 INFO:tasks.workunit.client.1.smithi069.stdout:3/101: mkdir d5/d9/d23 0 2024-03-22T07:15:55.415 INFO:tasks.workunit.client.1.smithi069.stdout:5/58: write d1/f3 [1183928,34690] 0 2024-03-22T07:15:55.416 INFO:tasks.workunit.client.1.smithi069.stdout:8/91: mknod d1/de/d12/c18 0 2024-03-22T07:15:55.416 INFO:tasks.workunit.client.1.smithi069.stdout:8/92: write d1/de/d12/f16 [601396,25296] 0 2024-03-22T07:15:55.416 INFO:tasks.workunit.client.1.smithi069.stdout:7/145: dwrite d9/db/f26 [0,4194304] 0 2024-03-22T07:15:55.417 INFO:tasks.workunit.client.1.smithi069.stdout:3/102: symlink d5/d9/d1e/l24 0 2024-03-22T07:15:55.418 INFO:tasks.workunit.client.0.smithi063.stdout:5/353: mkdir d8/d13/d28/d66 0 2024-03-22T07:15:55.418 INFO:tasks.workunit.client.1.smithi069.stdout:5/59: creat d1/d6/ff x:0 0 0 2024-03-22T07:15:55.420 INFO:tasks.workunit.client.0.smithi063.stdout:4/308: link d1/d5/d43/d27/d2c/c58 d1/d5/d43/d27/d2c/d3d/c6f 0 2024-03-22T07:15:55.420 INFO:tasks.workunit.client.1.smithi069.stdout:8/93: creat d1/de/d12/f19 x:0 0 0 2024-03-22T07:15:55.422 INFO:tasks.workunit.client.0.smithi063.stdout:4/309: link d1/d5/d43/d27/d2c/d4a/l5c d1/d5/d43/d27/d2c/d4a/l70 0 2024-03-22T07:15:55.422 INFO:tasks.workunit.client.0.smithi063.stdout:4/310: dread - d1/d5/d43/f35 zero size 2024-03-22T07:15:55.422 INFO:tasks.workunit.client.0.smithi063.stdout:4/311: creat d1/d5/d38/d59/f71 x:0 0 0 2024-03-22T07:15:55.422 INFO:tasks.workunit.client.0.smithi063.stdout:4/312: chown d1/d5/d43/d27/d2c/c3a 6 1 2024-03-22T07:15:55.423 INFO:tasks.workunit.client.1.smithi069.stdout:5/60: mkdir d1/d10 0 2024-03-22T07:15:55.423 INFO:tasks.workunit.client.1.smithi069.stdout:5/61: chown d1/d6/ff 1306 1 2024-03-22T07:15:55.424 INFO:tasks.workunit.client.1.smithi069.stdout:7/146: mkdir d9/d2b/d2e 0 2024-03-22T07:15:55.424 INFO:tasks.workunit.client.1.smithi069.stdout:7/147: readlink d9/ld 0 2024-03-22T07:15:55.425 INFO:tasks.workunit.client.1.smithi069.stdout:5/62: rmdir d1/d6/de 0 2024-03-22T07:15:55.426 INFO:tasks.workunit.client.1.smithi069.stdout:5/63: rename d1/d7/d8 to d1/d7/d8/d11 22 2024-03-22T07:15:55.427 INFO:tasks.workunit.client.0.smithi063.stdout:4/313: rename d1/c6c to d1/d5/d43/c72 0 2024-03-22T07:15:55.427 INFO:tasks.workunit.client.0.smithi063.stdout:4/314: creat d1/d5/d3c/d65/f73 x:0 0 0 2024-03-22T07:15:55.428 INFO:tasks.workunit.client.1.smithi069.stdout:5/64: creat d1/d7/d8/f12 x:0 0 0 2024-03-22T07:15:55.428 INFO:tasks.workunit.client.1.smithi069.stdout:5/65: chown d1/d7/cd 12068637 1 2024-03-22T07:15:55.428 INFO:tasks.workunit.client.1.smithi069.stdout:5/66: chown d1/c4 2 1 2024-03-22T07:15:55.428 INFO:tasks.workunit.client.1.smithi069.stdout:5/67: stat d1/d6/ff 0 2024-03-22T07:15:55.428 INFO:tasks.workunit.client.0.smithi063.stdout:1/427: dwrite d4/df/d13/f40 [0,4194304] 0 2024-03-22T07:15:55.429 INFO:tasks.workunit.client.1.smithi069.stdout:7/148: mkdir d9/d2b/d2f 0 2024-03-22T07:15:55.429 INFO:tasks.workunit.client.1.smithi069.stdout:3/103: dread d5/d9/fa [0,4194304] 0 2024-03-22T07:15:55.429 INFO:tasks.workunit.client.1.smithi069.stdout:3/104: stat d5/f7 0 2024-03-22T07:15:55.430 INFO:tasks.workunit.client.0.smithi063.stdout:4/315: mknod d1/d5/d43/d27/d2c/c74 0 2024-03-22T07:15:55.430 INFO:tasks.workunit.client.0.smithi063.stdout:4/316: write d1/d5/d38/f49 [961010,83233] 0 2024-03-22T07:15:55.430 INFO:tasks.workunit.client.0.smithi063.stdout:4/317: fdatasync d1/d5/f2f 0 2024-03-22T07:15:55.430 INFO:tasks.workunit.client.1.smithi069.stdout:3/105: mknod d5/d9/d23/c25 0 2024-03-22T07:15:55.431 INFO:tasks.workunit.client.1.smithi069.stdout:3/106: dread - d5/d9/f15 zero size 2024-03-22T07:15:55.432 INFO:tasks.workunit.client.1.smithi069.stdout:7/149: mkdir d9/d2b/d1d/d2a/d30 0 2024-03-22T07:15:55.432 INFO:tasks.workunit.client.1.smithi069.stdout:7/150: dread - d9/db/d1a/f2c zero size 2024-03-22T07:15:55.437 INFO:tasks.workunit.client.1.smithi069.stdout:7/151: mknod d9/c31 0 2024-03-22T07:15:55.438 INFO:tasks.workunit.client.0.smithi063.stdout:1/428: rename d4/d71/f78 to d4/df/d2c/f90 0 2024-03-22T07:15:55.438 INFO:tasks.workunit.client.1.smithi069.stdout:3/107: write d5/d9/fa [2358471,58862] 0 2024-03-22T07:15:55.440 INFO:tasks.workunit.client.0.smithi063.stdout:1/429: mkdir d4/d5a/d91 0 2024-03-22T07:15:55.440 INFO:tasks.workunit.client.0.smithi063.stdout:1/430: creat d4/df/d13/f92 x:0 0 0 2024-03-22T07:15:55.440 INFO:tasks.workunit.client.0.smithi063.stdout:1/431: fdatasync d4/df/d13/d39/f6b 0 2024-03-22T07:15:55.445 INFO:tasks.workunit.client.1.smithi069.stdout:3/108: write d5/f7 [2632860,94116] 0 2024-03-22T07:15:55.445 INFO:tasks.workunit.client.1.smithi069.stdout:3/109: write d5/d9/fd [2881219,99849] 0 2024-03-22T07:15:55.446 INFO:tasks.workunit.client.1.smithi069.stdout:1/140: dwrite d1/d13/d16/f19 [4194304,4194304] 0 2024-03-22T07:15:55.446 INFO:tasks.workunit.client.1.smithi069.stdout:2/122: dread d18/f1e [0,4194304] 0 2024-03-22T07:15:55.447 INFO:tasks.workunit.client.1.smithi069.stdout:1/141: dread d1/d13/d16/d1e/f20 [0,4194304] 0 2024-03-22T07:15:55.448 INFO:tasks.workunit.client.1.smithi069.stdout:3/110: mkdir d5/d9/d26 0 2024-03-22T07:15:55.449 INFO:tasks.workunit.client.0.smithi063.stdout:2/335: dwrite d0/f41 [0,4194304] 0 2024-03-22T07:15:55.450 INFO:tasks.workunit.client.0.smithi063.stdout:5/354: dwrite d8/d13/d1e/d31/f52 [0,4194304] 0 2024-03-22T07:15:55.451 INFO:tasks.workunit.client.1.smithi069.stdout:1/142: mkdir d1/d2a 0 2024-03-22T07:15:55.453 INFO:tasks.workunit.client.0.smithi063.stdout:4/318: getdents d1 0 2024-03-22T07:15:55.453 INFO:tasks.workunit.client.1.smithi069.stdout:3/111: dread f2 [0,4194304] 0 2024-03-22T07:15:55.453 INFO:tasks.workunit.client.1.smithi069.stdout:3/112: chown d5 2526 1 2024-03-22T07:15:55.454 INFO:tasks.workunit.client.1.smithi069.stdout:1/143: creat d1/d2a/f2b x:0 0 0 2024-03-22T07:15:55.455 INFO:tasks.workunit.client.1.smithi069.stdout:3/113: mknod d5/d9/db/c27 0 2024-03-22T07:15:55.458 INFO:tasks.workunit.client.0.smithi063.stdout:2/336: mknod d0/de/d1b/d32/c71 0 2024-03-22T07:15:55.458 INFO:tasks.workunit.client.0.smithi063.stdout:2/337: write d0/de/f65 [237118,116505] 0 2024-03-22T07:15:55.459 INFO:tasks.workunit.client.0.smithi063.stdout:2/338: getdents d0/de/d58 0 2024-03-22T07:15:55.460 INFO:tasks.workunit.client.0.smithi063.stdout:4/319: symlink d1/d5/d43/d27/l75 0 2024-03-22T07:15:55.460 INFO:tasks.workunit.client.0.smithi063.stdout:4/320: chown d1/d5/d43/f17 59147 1 2024-03-22T07:15:55.460 INFO:tasks.workunit.client.0.smithi063.stdout:4/321: fdatasync d1/f6a 0 2024-03-22T07:15:55.460 INFO:tasks.workunit.client.0.smithi063.stdout:4/322: read d1/f2 [3045302,31430] 0 2024-03-22T07:15:55.461 INFO:tasks.workunit.client.1.smithi069.stdout:1/144: dread d1/fb [0,4194304] 0 2024-03-22T07:15:55.462 INFO:tasks.workunit.client.1.smithi069.stdout:1/145: write d1/d13/f26 [613602,48706] 0 2024-03-22T07:15:55.463 INFO:tasks.workunit.client.1.smithi069.stdout:7/152: dwrite d9/f14 [0,4194304] 0 2024-03-22T07:15:55.463 INFO:tasks.workunit.client.1.smithi069.stdout:7/153: dread - d9/db/d1a/f21 zero size 2024-03-22T07:15:55.463 INFO:tasks.workunit.client.0.smithi063.stdout:2/339: creat d0/de/d1b/d63/d6e/f72 x:0 0 0 2024-03-22T07:15:55.463 INFO:tasks.workunit.client.0.smithi063.stdout:2/340: chown d0/de/d12/c26 46 1 2024-03-22T07:15:55.463 INFO:tasks.workunit.client.0.smithi063.stdout:4/323: mknod d1/d5/d43/d27/d2c/d3d/d54/d5f/c76 0 2024-03-22T07:15:55.464 INFO:tasks.workunit.client.1.smithi069.stdout:7/154: mknod d9/c32 0 2024-03-22T07:15:55.464 INFO:tasks.workunit.client.1.smithi069.stdout:7/155: read d9/f18 [473690,22945] 0 2024-03-22T07:15:55.464 INFO:tasks.workunit.client.1.smithi069.stdout:7/156: chown d9/d2b/d1d/d2a 14012 1 2024-03-22T07:15:55.464 INFO:tasks.workunit.client.1.smithi069.stdout:7/157: chown d9/db/d1a/f2c 2950010 1 2024-03-22T07:15:55.464 INFO:tasks.workunit.client.1.smithi069.stdout:7/158: rmdir d9/d2b 39 2024-03-22T07:15:55.468 INFO:tasks.workunit.client.1.smithi069.stdout:8/94: dwrite d1/f9 [0,4194304] 0 2024-03-22T07:15:55.468 INFO:tasks.workunit.client.1.smithi069.stdout:8/95: dread - d1/de/d12/f19 zero size 2024-03-22T07:15:55.468 INFO:tasks.workunit.client.1.smithi069.stdout:8/96: dread - d1/de/d12/f19 zero size 2024-03-22T07:15:55.476 INFO:tasks.workunit.client.1.smithi069.stdout:8/97: dread d1/f6 [0,4194304] 0 2024-03-22T07:15:55.476 INFO:tasks.workunit.client.1.smithi069.stdout:8/98: write d1/fc [437694,56066] 0 2024-03-22T07:15:55.476 INFO:tasks.workunit.client.1.smithi069.stdout:8/99: creat d1/de/f1a x:0 0 0 2024-03-22T07:15:55.476 INFO:tasks.workunit.client.0.smithi063.stdout:5/355: dwrite d8/d13/f4b [0,4194304] 0 2024-03-22T07:15:55.477 INFO:tasks.workunit.client.1.smithi069.stdout:2/123: dwrite d18/d1c/f2d [0,4194304] 0 2024-03-22T07:15:55.479 INFO:tasks.workunit.client.0.smithi063.stdout:5/356: unlink d8/d13/l60 0 2024-03-22T07:15:55.479 INFO:tasks.workunit.client.0.smithi063.stdout:5/357: readlink d8/d13/l20 0 2024-03-22T07:15:55.481 INFO:tasks.workunit.client.0.smithi063.stdout:5/358: unlink d8/d13/d28/d57/f59 0 2024-03-22T07:15:55.482 INFO:tasks.workunit.client.0.smithi063.stdout:5/359: mknod d8/d13/d1e/d31/d64/c67 0 2024-03-22T07:15:55.482 INFO:tasks.workunit.client.0.smithi063.stdout:5/360: write d8/d13/d1e/d3c/f49 [254600,108679] 0 2024-03-22T07:15:55.483 INFO:tasks.workunit.client.1.smithi069.stdout:2/124: getdents d18/d1c/d27 0 2024-03-22T07:15:55.485 INFO:tasks.workunit.client.0.smithi063.stdout:5/361: dread d8/d13/d1e/f38 [0,4194304] 0 2024-03-22T07:15:55.487 INFO:tasks.workunit.client.0.smithi063.stdout:5/362: creat d8/d13/d1e/d31/d58/f68 x:0 0 0 2024-03-22T07:15:55.488 INFO:tasks.workunit.client.0.smithi063.stdout:5/363: symlink d8/d13/d28/d57/l69 0 2024-03-22T07:15:55.489 INFO:tasks.workunit.client.0.smithi063.stdout:5/364: write d8/d13/d1e/f2f [557638,88519] 0 2024-03-22T07:15:55.489 INFO:tasks.workunit.client.0.smithi063.stdout:5/365: dread - d8/d13/f5d zero size 2024-03-22T07:15:55.489 INFO:tasks.workunit.client.0.smithi063.stdout:5/366: mkdir d8/d13/d1e/d31/d64/d65/d6a 0 2024-03-22T07:15:55.490 INFO:tasks.workunit.client.0.smithi063.stdout:5/367: symlink d8/d5e/l6b 0 2024-03-22T07:15:55.511 INFO:tasks.workunit.client.1.smithi069.stdout:7/159: dwrite f4 [0,4194304] 0 2024-03-22T07:15:55.511 INFO:tasks.workunit.client.1.smithi069.stdout:7/160: rename d9/d2b to d9/d2b/d1d/d33 22 2024-03-22T07:15:55.511 INFO:tasks.workunit.client.1.smithi069.stdout:7/161: write d9/f14 [4634988,111005] 0 2024-03-22T07:15:55.514 INFO:tasks.workunit.client.1.smithi069.stdout:7/162: mknod d9/d2b/d2e/c34 0 2024-03-22T07:15:55.514 INFO:tasks.workunit.client.0.smithi063.stdout:2/341: dwrite d0/de/d35/d36/f3a [0,4194304] 0 2024-03-22T07:15:55.516 INFO:tasks.workunit.client.0.smithi063.stdout:2/342: creat d0/de/d58/f73 x:0 0 0 2024-03-22T07:15:55.521 INFO:tasks.workunit.client.0.smithi063.stdout:4/324: dwrite d1/d5/d43/d27/d2c/d3d/d54/d5f/f66 [0,4194304] 0 2024-03-22T07:15:55.523 INFO:tasks.workunit.client.0.smithi063.stdout:4/325: symlink d1/d5/d43/d27/d2c/d3d/l77 0 2024-03-22T07:15:55.523 INFO:tasks.workunit.client.0.smithi063.stdout:4/326: dread - d1/f45 zero size 2024-03-22T07:15:55.525 INFO:tasks.workunit.client.0.smithi063.stdout:2/343: dread d0/de/d1b/f50 [0,4194304] 0 2024-03-22T07:15:55.525 INFO:tasks.workunit.client.1.smithi069.stdout:7/163: read d9/f14 [147442,75004] 0 2024-03-22T07:15:55.527 INFO:tasks.workunit.client.0.smithi063.stdout:2/344: mkdir d0/de/d74 0 2024-03-22T07:15:55.528 INFO:tasks.workunit.client.0.smithi063.stdout:2/345: rename d0/de/d1b/l66 to d0/de/d1b/l75 0 2024-03-22T07:15:55.529 INFO:tasks.workunit.client.0.smithi063.stdout:2/346: creat d0/de/d12/d39/f76 x:0 0 0 2024-03-22T07:15:55.529 INFO:tasks.workunit.client.0.smithi063.stdout:2/347: stat d0/de/l5b 0 2024-03-22T07:15:55.530 INFO:tasks.workunit.client.0.smithi063.stdout:2/348: mknod d0/de/d1b/d32/c77 0 2024-03-22T07:15:55.532 INFO:tasks.workunit.client.0.smithi063.stdout:2/349: mkdir d0/de/d1b/d63/d78 0 2024-03-22T07:15:55.533 INFO:tasks.workunit.client.1.smithi069.stdout:8/100: dwrite d1/de/d12/f19 [0,4194304] 0 2024-03-22T07:15:55.537 INFO:tasks.workunit.client.0.smithi063.stdout:2/350: dread d0/de/d1b/f50 [0,4194304] 0 2024-03-22T07:15:55.537 INFO:tasks.workunit.client.1.smithi069.stdout:2/125: dwrite d18/d1c/d25/f2e [0,4194304] 0 2024-03-22T07:15:55.537 INFO:tasks.workunit.client.0.smithi063.stdout:5/368: dwrite d8/d13/d28/d57/f5c [0,4194304] 0 2024-03-22T07:15:55.537 INFO:tasks.workunit.client.0.smithi063.stdout:4/327: dwrite d1/d5/d43/f2e [0,4194304] 0 2024-03-22T07:15:55.543 INFO:tasks.workunit.client.0.smithi063.stdout:4/328: mkdir d1/d5/d43/d27/d2c/d4a/d78 0 2024-03-22T07:15:55.543 INFO:tasks.workunit.client.0.smithi063.stdout:4/329: dread - d1/d5/d38/f6e zero size 2024-03-22T07:15:55.544 INFO:tasks.workunit.client.0.smithi063.stdout:2/351: mknod d0/c79 0 2024-03-22T07:15:55.545 INFO:tasks.workunit.client.0.smithi063.stdout:2/352: readlink d0/de/d12/d39/l3d 0 2024-03-22T07:15:55.546 INFO:tasks.workunit.client.0.smithi063.stdout:4/330: mkdir d1/d5/d43/d27/d2c/d3d/d54/d5f/d79 0 2024-03-22T07:15:55.547 INFO:tasks.workunit.client.0.smithi063.stdout:2/353: chown d0/c2d 23 1 2024-03-22T07:15:55.549 INFO:tasks.workunit.client.0.smithi063.stdout:6/396: sync 2024-03-22T07:15:55.549 INFO:tasks.workunit.client.0.smithi063.stdout:3/363: sync 2024-03-22T07:15:55.550 INFO:tasks.workunit.client.0.smithi063.stdout:3/364: link da/f3a da/d16/d26/f76 0 2024-03-22T07:15:55.551 INFO:tasks.workunit.client.0.smithi063.stdout:3/365: write da/f21 [1750946,108226] 0 2024-03-22T07:15:55.551 INFO:tasks.workunit.client.0.smithi063.stdout:6/397: truncate d3/d15/d16/d60/d6d/d4c/f20 1952275 0 2024-03-22T07:15:55.552 INFO:tasks.workunit.client.0.smithi063.stdout:3/366: creat da/d63/d65/f77 x:0 0 0 2024-03-22T07:15:55.554 INFO:tasks.workunit.client.0.smithi063.stdout:6/398: creat d3/d15/d16/d60/d6d/d4e/d5a/f84 x:0 0 0 2024-03-22T07:15:55.554 INFO:tasks.workunit.client.0.smithi063.stdout:6/399: creat d3/d15/f85 x:0 0 0 2024-03-22T07:15:55.554 INFO:tasks.workunit.client.0.smithi063.stdout:6/400: dread - d3/f7c zero size 2024-03-22T07:15:55.560 INFO:tasks.workunit.client.0.smithi063.stdout:6/401: dread d3/d15/f22 [4194304,4194304] 0 2024-03-22T07:15:55.562 INFO:tasks.workunit.client.0.smithi063.stdout:6/402: dread d3/d6/d2a/d38/f61 [0,4194304] 0 2024-03-22T07:15:55.563 INFO:tasks.workunit.client.0.smithi063.stdout:6/403: write d3/f14 [1293619,18069] 0 2024-03-22T07:15:55.563 INFO:tasks.workunit.client.0.smithi063.stdout:6/404: symlink d3/d6/l86 0 2024-03-22T07:15:55.565 INFO:tasks.workunit.client.0.smithi063.stdout:6/405: link d3/d15/f23 d3/d15/d16/d47/f87 0 2024-03-22T07:15:55.565 INFO:tasks.workunit.client.0.smithi063.stdout:6/406: chown d3/d6/d2a 5 1 2024-03-22T07:15:55.565 INFO:tasks.workunit.client.0.smithi063.stdout:6/407: fsync d3/d76/f5b 0 2024-03-22T07:15:55.572 INFO:tasks.workunit.client.0.smithi063.stdout:3/367: dwrite da/d16/d26/f27 [0,4194304] 0 2024-03-22T07:15:55.572 INFO:tasks.workunit.client.0.smithi063.stdout:3/368: creat da/d63/d65/f78 x:0 0 0 2024-03-22T07:15:55.572 INFO:tasks.workunit.client.0.smithi063.stdout:6/408: write d3/d6/fa [1448022,59534] 0 2024-03-22T07:15:55.576 INFO:tasks.workunit.client.0.smithi063.stdout:3/369: rename da/d16/f32 to da/d16/d3f/f79 0 2024-03-22T07:15:55.576 INFO:tasks.workunit.client.0.smithi063.stdout:3/370: chown da/d63 47873908 1 2024-03-22T07:15:55.577 INFO:tasks.workunit.client.0.smithi063.stdout:6/409: truncate d3/d15/f23 1128221 0 2024-03-22T07:15:55.577 INFO:tasks.workunit.client.0.smithi063.stdout:6/410: dread - d3/d15/d16/d60/d6d/d4e/d5a/f84 zero size 2024-03-22T07:15:55.577 INFO:tasks.workunit.client.0.smithi063.stdout:3/371: getdents da/d16/d26/d69 0 2024-03-22T07:15:55.578 INFO:tasks.workunit.client.0.smithi063.stdout:3/372: mknod da/d16/d26/c7a 0 2024-03-22T07:15:55.579 INFO:tasks.workunit.client.0.smithi063.stdout:3/373: mkdir da/d63/d65/d7b 0 2024-03-22T07:15:55.580 INFO:tasks.workunit.client.0.smithi063.stdout:3/374: mknod da/d63/c7c 0 2024-03-22T07:15:55.584 INFO:tasks.workunit.client.0.smithi063.stdout:3/375: symlink da/d16/l7d 0 2024-03-22T07:15:55.584 INFO:tasks.workunit.client.0.smithi063.stdout:3/376: truncate da/d16/d3f/f64 704808 0 2024-03-22T07:15:55.586 INFO:tasks.workunit.client.0.smithi063.stdout:6/411: write d3/d6/f28 [4173439,71633] 0 2024-03-22T07:15:55.586 INFO:tasks.workunit.client.0.smithi063.stdout:6/412: fsync d3/d6/d2a/f48 0 2024-03-22T07:15:55.609 INFO:tasks.workunit.client.1.smithi069.stdout:7/164: mknod d9/d2b/d2e/c35 0 2024-03-22T07:15:55.611 INFO:tasks.workunit.client.1.smithi069.stdout:8/101: symlink d1/de/l1b 0 2024-03-22T07:15:55.611 INFO:tasks.workunit.client.1.smithi069.stdout:8/102: read d1/fd [3835107,119971] 0 2024-03-22T07:15:55.611 INFO:tasks.workunit.client.1.smithi069.stdout:8/103: chown d1/c11 2522 1 2024-03-22T07:15:55.628 INFO:tasks.workunit.client.0.smithi063.stdout:3/377: link da/d16/d26/l55 da/d63/d65/d4b/l7e 0 2024-03-22T07:15:55.649 INFO:tasks.workunit.client.0.smithi063.stdout:6/413: mkdir d3/d15/d16/d60/d6d/d4e/d5a/d88 0 2024-03-22T07:15:55.653 INFO:tasks.workunit.client.0.smithi063.stdout:3/378: symlink da/d63/d65/d7b/l7f 0 2024-03-22T07:15:55.668 INFO:tasks.workunit.client.0.smithi063.stdout:6/414: dwrite d3/d15/d16/d60/d6d/d4c/f75 [0,4194304] 0 2024-03-22T07:15:55.670 INFO:tasks.workunit.client.1.smithi069.stdout:2/126: rename cb to d18/c32 0 2024-03-22T07:15:55.672 INFO:tasks.workunit.client.1.smithi069.stdout:8/104: mkdir d1/d1c 0 2024-03-22T07:15:55.677 INFO:tasks.workunit.client.1.smithi069.stdout:7/165: mknod d9/d2b/c36 0 2024-03-22T07:15:55.677 INFO:tasks.workunit.client.1.smithi069.stdout:7/166: dread - d9/db/d1a/f21 zero size 2024-03-22T07:15:55.677 INFO:tasks.workunit.client.1.smithi069.stdout:8/105: dread d1/de/d12/f16 [0,4194304] 0 2024-03-22T07:15:55.677 INFO:tasks.workunit.client.1.smithi069.stdout:8/106: chown d1/de/l1b 38307 1 2024-03-22T07:15:55.679 INFO:tasks.workunit.client.1.smithi069.stdout:8/107: rename d1/f6 to d1/f1d 0 2024-03-22T07:15:55.679 INFO:tasks.workunit.client.1.smithi069.stdout:8/108: stat d1/de/l1b 0 2024-03-22T07:15:55.682 INFO:tasks.workunit.client.1.smithi069.stdout:7/167: mknod d9/d2b/c37 0 2024-03-22T07:15:55.682 INFO:tasks.workunit.client.1.smithi069.stdout:7/168: chown d9/db/d1a/c22 8650169 1 2024-03-22T07:15:55.683 INFO:tasks.workunit.client.1.smithi069.stdout:7/169: readlink d9/l12 0 2024-03-22T07:15:55.683 INFO:tasks.workunit.client.1.smithi069.stdout:8/109: symlink d1/de/l1e 0 2024-03-22T07:15:55.683 INFO:tasks.workunit.client.1.smithi069.stdout:7/170: rmdir d9/d2b/d2f 0 2024-03-22T07:15:55.687 INFO:tasks.workunit.client.0.smithi063.stdout:3/379: symlink da/d16/l80 0 2024-03-22T07:15:55.687 INFO:tasks.workunit.client.1.smithi069.stdout:8/110: dread d1/f1d [0,4194304] 0 2024-03-22T07:15:55.688 INFO:tasks.workunit.client.1.smithi069.stdout:8/111: mknod d1/de/c1f 0 2024-03-22T07:15:55.688 INFO:tasks.workunit.client.1.smithi069.stdout:8/112: dread - d1/f14 zero size 2024-03-22T07:15:55.690 INFO:tasks.workunit.client.1.smithi069.stdout:8/113: rmdir d1/de/d12 39 2024-03-22T07:15:55.690 INFO:tasks.workunit.client.1.smithi069.stdout:8/114: write d1/f14 [485903,113680] 0 2024-03-22T07:15:55.691 INFO:tasks.workunit.client.1.smithi069.stdout:8/115: mkdir d1/de/d20 0 2024-03-22T07:15:55.691 INFO:tasks.workunit.client.0.smithi063.stdout:3/380: getdents da/d63/d65/d4b 0 2024-03-22T07:15:55.696 INFO:tasks.workunit.client.0.smithi063.stdout:3/381: dread da/d16/d26/f29 [0,4194304] 0 2024-03-22T07:15:55.696 INFO:tasks.workunit.client.0.smithi063.stdout:3/382: chown da/d16/c2e 2758679 1 2024-03-22T07:15:55.696 INFO:tasks.workunit.client.0.smithi063.stdout:3/383: read da/f21 [1818484,69654] 0 2024-03-22T07:15:55.697 INFO:tasks.workunit.client.0.smithi063.stdout:3/384: unlink da/f11 0 2024-03-22T07:15:55.697 INFO:tasks.workunit.client.0.smithi063.stdout:3/385: fsync da/d63/d65/f77 0 2024-03-22T07:15:55.700 INFO:tasks.workunit.client.0.smithi063.stdout:3/386: dread da/d16/d26/d69/f6b [0,4194304] 0 2024-03-22T07:15:55.703 INFO:tasks.workunit.client.0.smithi063.stdout:3/387: mkdir da/d63/d65/d81 0 2024-03-22T07:15:55.707 INFO:tasks.workunit.client.1.smithi069.stdout:7/171: dwrite d9/db/d1a/f2c [0,4194304] 0 2024-03-22T07:15:55.708 INFO:tasks.workunit.client.0.smithi063.stdout:7/269: sync 2024-03-22T07:15:55.708 INFO:tasks.workunit.client.0.smithi063.stdout:0/434: sync 2024-03-22T07:15:55.708 INFO:tasks.workunit.client.0.smithi063.stdout:0/435: chown da/d49/f51 1 1 2024-03-22T07:15:55.708 INFO:tasks.workunit.client.0.smithi063.stdout:0/436: stat da/de/d17/d25/f27 0 2024-03-22T07:15:55.708 INFO:tasks.workunit.client.0.smithi063.stdout:7/270: getdents d5/da/d28/d31/d33 0 2024-03-22T07:15:55.708 INFO:tasks.workunit.client.0.smithi063.stdout:7/271: dread - d5/d16/f2e zero size 2024-03-22T07:15:55.708 INFO:tasks.workunit.client.0.smithi063.stdout:7/272: fdatasync d5/da/f2c 0 2024-03-22T07:15:55.713 INFO:tasks.workunit.client.1.smithi069.stdout:7/172: link d9/d2b/f20 d9/d2b/f38 0 2024-03-22T07:15:55.714 INFO:tasks.workunit.client.1.smithi069.stdout:7/173: chown d9/d2b/c1f 20 1 2024-03-22T07:15:55.719 INFO:tasks.workunit.client.1.smithi069.stdout:7/174: dread d9/f14 [0,4194304] 0 2024-03-22T07:15:55.720 INFO:tasks.workunit.client.1.smithi069.stdout:7/175: getdents d9/d2b/d2e 0 2024-03-22T07:15:55.720 INFO:tasks.workunit.client.1.smithi069.stdout:7/176: creat d9/db/d1a/f39 x:0 0 0 2024-03-22T07:15:55.720 INFO:tasks.workunit.client.1.smithi069.stdout:7/177: chown d9/c1c 20270 1 2024-03-22T07:15:55.721 INFO:tasks.workunit.client.1.smithi069.stdout:7/178: creat d9/d2b/d1d/d2a/d30/f3a x:0 0 0 2024-03-22T07:15:55.724 INFO:tasks.workunit.client.0.smithi063.stdout:6/415: creat d3/f89 x:0 0 0 2024-03-22T07:15:55.725 INFO:tasks.workunit.client.0.smithi063.stdout:0/437: dread da/d49/f6d [0,4194304] 0 2024-03-22T07:15:55.727 INFO:tasks.workunit.client.0.smithi063.stdout:0/438: truncate da/de/f18 266385 0 2024-03-22T07:15:55.728 INFO:tasks.workunit.client.0.smithi063.stdout:6/416: rename d3/d15/f41 to d3/d6/d2a/d38/f8a 0 2024-03-22T07:15:55.730 INFO:tasks.workunit.client.0.smithi063.stdout:0/439: mkdir da/de/d17/d25/d3c/d53/d89 0 2024-03-22T07:15:55.730 INFO:tasks.workunit.client.0.smithi063.stdout:0/440: readlink da/de/d17/d25/d3c/d53/l6e 0 2024-03-22T07:15:55.732 INFO:tasks.workunit.client.0.smithi063.stdout:6/417: mkdir d3/d15/d16/d60/d6d/d4c/d8b 0 2024-03-22T07:15:55.733 INFO:tasks.workunit.client.0.smithi063.stdout:0/441: symlink da/de/d17/d25/d3c/d43/l8a 0 2024-03-22T07:15:55.734 INFO:tasks.workunit.client.0.smithi063.stdout:0/442: write da/f40 [9206334,111952] 0 2024-03-22T07:15:55.734 INFO:tasks.workunit.client.0.smithi063.stdout:7/273: dwrite d5/d16/f2e [0,4194304] 0 2024-03-22T07:15:55.735 INFO:tasks.workunit.client.1.smithi069.stdout:8/116: dwrite d1/f14 [0,4194304] 0 2024-03-22T07:15:55.738 INFO:tasks.workunit.client.0.smithi063.stdout:7/274: mkdir d5/da/d28/d31/d33/d57 0 2024-03-22T07:15:55.739 INFO:tasks.workunit.client.0.smithi063.stdout:6/418: dread d3/d6/d2a/f5e [0,4194304] 0 2024-03-22T07:15:55.739 INFO:tasks.workunit.client.0.smithi063.stdout:6/419: chown d3/d15/d16/d60/d6d/d4e 134383885 1 2024-03-22T07:15:55.740 INFO:tasks.workunit.client.0.smithi063.stdout:6/420: mkdir d3/d15/d16/d69/d8c 0 2024-03-22T07:15:55.745 INFO:tasks.workunit.client.0.smithi063.stdout:6/421: dread d3/d15/d16/d60/d6d/d4c/f20 [0,4194304] 0 2024-03-22T07:15:55.747 INFO:tasks.workunit.client.1.smithi069.stdout:8/117: write d1/f9 [1867696,108971] 0 2024-03-22T07:15:55.747 INFO:tasks.workunit.client.1.smithi069.stdout:7/179: dwrite d9/db/f17 [0,4194304] 0 2024-03-22T07:15:55.748 INFO:tasks.workunit.client.1.smithi069.stdout:7/180: chown d9/c10 599107 1 2024-03-22T07:15:55.748 INFO:tasks.workunit.client.0.smithi063.stdout:6/422: link d3/d76/f5b d3/d15/d16/d60/d6d/d4e/d5a/d88/f8d 0 2024-03-22T07:15:55.748 INFO:tasks.workunit.client.1.smithi069.stdout:8/118: mknod d1/d1c/c21 0 2024-03-22T07:15:55.749 INFO:tasks.workunit.client.0.smithi063.stdout:6/423: mknod d3/d15/d16/d60/d6d/d4e/d5a/d88/c8e 0 2024-03-22T07:15:55.749 INFO:tasks.workunit.client.0.smithi063.stdout:6/424: chown d3/d15/d16/d60/d6d/d4c/c24 13330 1 2024-03-22T07:15:55.750 INFO:tasks.workunit.client.1.smithi069.stdout:4/104: sync 2024-03-22T07:15:55.750 INFO:tasks.workunit.client.1.smithi069.stdout:4/105: write f3 [4461833,120140] 0 2024-03-22T07:15:55.750 INFO:tasks.workunit.client.1.smithi069.stdout:0/114: sync 2024-03-22T07:15:55.750 INFO:tasks.workunit.client.1.smithi069.stdout:8/119: creat d1/d1c/f22 x:0 0 0 2024-03-22T07:15:55.750 INFO:tasks.workunit.client.1.smithi069.stdout:8/120: dread - d1/de/f1a zero size 2024-03-22T07:15:55.751 INFO:tasks.workunit.client.0.smithi063.stdout:6/425: mknod d3/d15/d16/d69/c8f 0 2024-03-22T07:15:55.751 INFO:tasks.workunit.client.0.smithi063.stdout:6/426: getdents d3/d6/d2a 0 2024-03-22T07:15:55.752 INFO:tasks.workunit.client.1.smithi069.stdout:4/106: mkdir d6/dc/d10/d1f 0 2024-03-22T07:15:55.752 INFO:tasks.workunit.client.0.smithi063.stdout:6/427: rename d3/d6/d35/d3a/f55 to d3/d1c/f90 0 2024-03-22T07:15:55.752 INFO:tasks.workunit.client.0.smithi063.stdout:7/275: read d5/d14/f37 [2351731,43913] 0 2024-03-22T07:15:55.753 INFO:tasks.workunit.client.1.smithi069.stdout:0/115: mknod d3/d7/d17/d1c/c28 0 2024-03-22T07:15:55.753 INFO:tasks.workunit.client.1.smithi069.stdout:0/116: chown d3/da/c23 75493764 1 2024-03-22T07:15:55.753 INFO:tasks.workunit.client.0.smithi063.stdout:7/276: symlink d5/da/d28/d31/d33/d57/l58 0 2024-03-22T07:15:55.754 INFO:tasks.workunit.client.0.smithi063.stdout:7/277: chown d5/d16/f2e 495 1 2024-03-22T07:15:55.755 INFO:tasks.workunit.client.1.smithi069.stdout:0/117: mkdir d3/d7/d15/d29 0 2024-03-22T07:15:55.756 INFO:tasks.workunit.client.1.smithi069.stdout:0/118: chown d3/d7/cc 1 1 2024-03-22T07:15:55.757 INFO:tasks.workunit.client.1.smithi069.stdout:4/107: dread d6/dc/d10/f14 [0,4194304] 0 2024-03-22T07:15:55.757 INFO:tasks.workunit.client.1.smithi069.stdout:6/84: sync 2024-03-22T07:15:55.758 INFO:tasks.workunit.client.1.smithi069.stdout:6/85: chown d4/d7/da/lc 95203427 1 2024-03-22T07:15:55.758 INFO:tasks.workunit.client.1.smithi069.stdout:6/86: readlink d4/dd/l15 0 2024-03-22T07:15:55.758 INFO:tasks.workunit.client.1.smithi069.stdout:0/119: rename d3/l5 to d3/d7/d17/l2a 0 2024-03-22T07:15:55.758 INFO:tasks.workunit.client.1.smithi069.stdout:4/108: truncate d6/dc/d10/f17 451511 0 2024-03-22T07:15:55.761 INFO:tasks.workunit.client.1.smithi069.stdout:6/87: unlink d4/dd/f11 0 2024-03-22T07:15:55.761 INFO:tasks.workunit.client.1.smithi069.stdout:6/88: creat d4/d7/da/f1b x:0 0 0 2024-03-22T07:15:55.762 INFO:tasks.workunit.client.1.smithi069.stdout:7/181: dread f4 [0,4194304] 0 2024-03-22T07:15:55.763 INFO:tasks.workunit.client.0.smithi063.stdout:7/278: dread d5/d16/d1b/d21/f2a [0,4194304] 0 2024-03-22T07:15:55.764 INFO:tasks.workunit.client.0.smithi063.stdout:7/279: readlink d5/le 0 2024-03-22T07:15:55.764 INFO:tasks.workunit.client.1.smithi069.stdout:0/120: rename d3/f6 to d3/d7/d15/d1f/f2b 0 2024-03-22T07:15:55.765 INFO:tasks.workunit.client.1.smithi069.stdout:7/182: rmdir d9/d2b/d2e 39 2024-03-22T07:15:55.765 INFO:tasks.workunit.client.1.smithi069.stdout:7/183: chown d9/d2b/d2e/c35 7 1 2024-03-22T07:15:55.765 INFO:tasks.workunit.client.1.smithi069.stdout:7/184: fdatasync d9/d2b/f20 0 2024-03-22T07:15:55.767 INFO:tasks.workunit.client.1.smithi069.stdout:7/185: read f8 [339206,103080] 0 2024-03-22T07:15:55.767 INFO:tasks.workunit.client.1.smithi069.stdout:0/121: fsync d3/d7/d15/f20 0 2024-03-22T07:15:55.768 INFO:tasks.workunit.client.1.smithi069.stdout:4/109: rename d6/d12/l15 to d6/l20 0 2024-03-22T07:15:55.770 INFO:tasks.workunit.client.1.smithi069.stdout:7/186: link f2 d9/f3b 0 2024-03-22T07:15:55.770 INFO:tasks.workunit.client.0.smithi063.stdout:0/443: dwrite da/de/d17/f1a [0,4194304] 0 2024-03-22T07:15:55.772 INFO:tasks.workunit.client.1.smithi069.stdout:0/122: link d3/da/l25 d3/d7/d17/l2c 0 2024-03-22T07:15:55.772 INFO:tasks.workunit.client.1.smithi069.stdout:4/110: mkdir d6/d12/d18/d21 0 2024-03-22T07:15:55.773 INFO:tasks.workunit.client.1.smithi069.stdout:7/187: creat d9/db/d1a/f3c x:0 0 0 2024-03-22T07:15:55.773 INFO:tasks.workunit.client.1.smithi069.stdout:7/188: truncate d9/db/d1a/f2c 4705914 0 2024-03-22T07:15:55.774 INFO:tasks.workunit.client.0.smithi063.stdout:7/280: dread d5/d14/d1d/f51 [0,4194304] 0 2024-03-22T07:15:55.775 INFO:tasks.workunit.client.1.smithi069.stdout:8/121: dwrite d1/de/d12/f19 [0,4194304] 0 2024-03-22T07:15:55.779 INFO:tasks.workunit.client.1.smithi069.stdout:8/122: link d1/f9 d1/de/f23 0 2024-03-22T07:15:55.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:55 smithi069 ceph-mon[36441]: pgmap v250: 65 pgs: 65 active+clean; 501 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 36 MiB/s rd, 44 MiB/s wr, 639 op/s 2024-03-22T07:15:55.781 INFO:tasks.workunit.client.1.smithi069.stdout:8/123: getdents d1/d1c 0 2024-03-22T07:15:55.781 INFO:tasks.workunit.client.1.smithi069.stdout:8/124: fsync d1/f14 0 2024-03-22T07:15:55.781 INFO:tasks.workunit.client.1.smithi069.stdout:7/189: write d9/db/f16 [3095726,67467] 0 2024-03-22T07:15:55.782 INFO:tasks.workunit.client.0.smithi063.stdout:6/428: rmdir d3/d6 39 2024-03-22T07:15:55.782 INFO:tasks.workunit.client.0.smithi063.stdout:6/429: readlink d3/d15/d16/d60/d6d/d4e/d5f/l80 0 2024-03-22T07:15:55.782 INFO:tasks.workunit.client.0.smithi063.stdout:6/430: getdents d3/d15/d16/d60/d6d/d4c/d8b 0 2024-03-22T07:15:55.783 INFO:tasks.workunit.client.0.smithi063.stdout:6/431: mknod d3/d6/c91 0 2024-03-22T07:15:55.784 INFO:tasks.workunit.client.0.smithi063.stdout:6/432: truncate d3/d15/d16/d60/d6d/d4c/f75 3650797 0 2024-03-22T07:15:55.784 INFO:tasks.workunit.client.0.smithi063.stdout:6/433: write d3/d15/f85 [266045,12390] 0 2024-03-22T07:15:55.785 INFO:tasks.workunit.client.1.smithi069.stdout:8/125: dread d1/fc [0,4194304] 0 2024-03-22T07:15:55.786 INFO:tasks.workunit.client.1.smithi069.stdout:4/111: fsync d6/dc/d10/f17 0 2024-03-22T07:15:55.787 INFO:tasks.workunit.client.1.smithi069.stdout:7/190: mkdir d9/d2b/d1d/d2a/d30/d3d 0 2024-03-22T07:15:55.789 INFO:tasks.workunit.client.1.smithi069.stdout:6/89: dwrite d4/f14 [0,4194304] 0 2024-03-22T07:15:55.789 INFO:tasks.workunit.client.1.smithi069.stdout:8/126: symlink d1/de/d12/l24 0 2024-03-22T07:15:55.789 INFO:tasks.workunit.client.0.smithi063.stdout:7/281: dread d5/d16/f2b [4194304,4194304] 0 2024-03-22T07:15:55.789 INFO:tasks.workunit.client.0.smithi063.stdout:7/282: getdents d5/d14/d1d/d4f 0 2024-03-22T07:15:55.790 INFO:tasks.workunit.client.0.smithi063.stdout:0/444: dwrite da/de/d60/f69 [0,4194304] 0 2024-03-22T07:15:55.790 INFO:tasks.workunit.client.1.smithi069.stdout:7/191: symlink d9/d2b/d1d/d2a/l3e 0 2024-03-22T07:15:55.792 INFO:tasks.workunit.client.1.smithi069.stdout:0/123: write d3/d7/d15/f20 [600064,109425] 0 2024-03-22T07:15:55.794 INFO:tasks.workunit.client.0.smithi063.stdout:0/445: mkdir da/de/d60/d56/d8b 0 2024-03-22T07:15:55.794 INFO:tasks.workunit.client.0.smithi063.stdout:0/446: write da/de/d17/f5d [516936,9941] 0 2024-03-22T07:15:55.794 INFO:tasks.workunit.client.1.smithi069.stdout:6/90: mknod d4/dd/d12/c1c 0 2024-03-22T07:15:55.796 INFO:tasks.workunit.client.1.smithi069.stdout:8/127: rename d1/de/c1f to d1/de/c25 0 2024-03-22T07:15:55.799 INFO:tasks.workunit.client.1.smithi069.stdout:7/192: rename d9/d2b/d1d/d2a/l3e to d9/d2b/d1d/d2a/d30/d3d/l3f 0 2024-03-22T07:15:55.802 INFO:tasks.workunit.client.1.smithi069.stdout:0/124: link d3/d7/lf d3/d7/d15/d29/l2d 0 2024-03-22T07:15:55.802 INFO:tasks.workunit.client.1.smithi069.stdout:0/125: chown d3/d7/d17/d1c 546108 1 2024-03-22T07:15:55.803 INFO:tasks.workunit.client.1.smithi069.stdout:0/126: stat d3/d7/le 0 2024-03-22T07:15:55.804 INFO:tasks.workunit.client.0.smithi063.stdout:7/283: dread d5/da/ff [4194304,4194304] 0 2024-03-22T07:15:55.804 INFO:tasks.workunit.client.0.smithi063.stdout:0/447: read f1 [990214,42273] 0 2024-03-22T07:15:55.804 INFO:tasks.workunit.client.0.smithi063.stdout:0/448: creat da/de/d39/f8c x:0 0 0 2024-03-22T07:15:55.804 INFO:tasks.workunit.client.0.smithi063.stdout:0/449: readlink da/de/d17/d25/d3c/l7e 0 2024-03-22T07:15:55.805 INFO:tasks.workunit.client.1.smithi069.stdout:6/91: creat d4/dd/d12/f1d x:0 0 0 2024-03-22T07:15:55.805 INFO:tasks.workunit.client.0.smithi063.stdout:7/284: creat d5/d16/d1b/d21/f59 x:0 0 0 2024-03-22T07:15:55.806 INFO:tasks.workunit.client.1.smithi069.stdout:7/193: rename d9/db/f26 to d9/d2b/d1d/d2a/d30/d3d/f40 0 2024-03-22T07:15:55.806 INFO:tasks.workunit.client.1.smithi069.stdout:7/194: truncate d9/db/d1a/f21 302618 0 2024-03-22T07:15:55.807 INFO:tasks.workunit.client.0.smithi063.stdout:0/450: rename l7 to da/de/d17/d25/d3c/d53/d89/l8d 0 2024-03-22T07:15:55.809 INFO:tasks.workunit.client.0.smithi063.stdout:0/451: mknod da/de/d17/d25/d3c/d64/d78/c8e 0 2024-03-22T07:15:55.809 INFO:tasks.workunit.client.0.smithi063.stdout:0/452: write da/de/d17/d25/f70 [116456,238] 0 2024-03-22T07:15:55.809 INFO:tasks.workunit.client.0.smithi063.stdout:0/453: truncate da/de/d17/d25/f27 1294461 0 2024-03-22T07:15:55.809 INFO:tasks.workunit.client.0.smithi063.stdout:0/454: readlink da/de/l1d 0 2024-03-22T07:15:55.809 INFO:tasks.workunit.client.0.smithi063.stdout:0/455: chown da/de/f5c 1088480 1 2024-03-22T07:15:55.810 INFO:tasks.workunit.client.1.smithi069.stdout:6/92: mknod d4/dd/d12/c1e 0 2024-03-22T07:15:55.810 INFO:tasks.workunit.client.1.smithi069.stdout:6/93: write d4/f6 [1828884,92715] 0 2024-03-22T07:15:55.810 INFO:tasks.workunit.client.1.smithi069.stdout:6/94: getdents d4/dd/d12/d13 0 2024-03-22T07:15:55.810 INFO:tasks.workunit.client.1.smithi069.stdout:4/112: dwrite d6/dc/f1a [0,4194304] 0 2024-03-22T07:15:55.810 INFO:tasks.workunit.client.1.smithi069.stdout:4/113: stat d6/d12/d18/d21 0 2024-03-22T07:15:55.811 INFO:tasks.workunit.client.1.smithi069.stdout:7/195: mkdir d9/d41 0 2024-03-22T07:15:55.811 INFO:tasks.workunit.client.1.smithi069.stdout:7/196: write d9/d2b/f20 [997970,39339] 0 2024-03-22T07:15:55.814 INFO:tasks.workunit.client.1.smithi069.stdout:0/127: mknod d3/d7/d15/c2e 0 2024-03-22T07:15:55.814 INFO:tasks.workunit.client.0.smithi063.stdout:9/272: sync 2024-03-22T07:15:55.822 INFO:tasks.workunit.client.1.smithi069.stdout:7/197: mkdir d9/d42 0 2024-03-22T07:15:55.822 INFO:tasks.workunit.client.1.smithi069.stdout:8/128: dwrite d1/f14 [0,4194304] 0 2024-03-22T07:15:55.823 INFO:tasks.workunit.client.1.smithi069.stdout:4/114: read d6/dc/f1a [2674085,11063] 0 2024-03-22T07:15:55.823 INFO:tasks.workunit.client.1.smithi069.stdout:4/115: fsync d6/dc/f1a 0 2024-03-22T07:15:55.824 INFO:tasks.workunit.client.0.smithi063.stdout:0/456: write da/de/d17/d25/f6a [2022921,64391] 0 2024-03-22T07:15:55.824 INFO:tasks.workunit.client.0.smithi063.stdout:0/457: fsync da/de/d17/f1a 0 2024-03-22T07:15:55.824 INFO:tasks.workunit.client.1.smithi069.stdout:0/128: link d3/d7/l19 d3/d7/d15/d1f/l2f 0 2024-03-22T07:15:55.825 INFO:tasks.workunit.client.1.smithi069.stdout:7/198: truncate d9/db/f17 3568760 0 2024-03-22T07:15:55.825 INFO:tasks.workunit.client.0.smithi063.stdout:0/458: link da/de/d39/l5e da/de/d17/d25/d3c/d64/d78/l8f 0 2024-03-22T07:15:55.825 INFO:tasks.workunit.client.0.smithi063.stdout:0/459: creat da/de/d39/f90 x:0 0 0 2024-03-22T07:15:55.825 INFO:tasks.workunit.client.0.smithi063.stdout:0/460: chown da/c1e 710866302 1 2024-03-22T07:15:55.826 INFO:tasks.workunit.client.0.smithi063.stdout:0/461: creat da/de/d60/d56/d8b/f91 x:0 0 0 2024-03-22T07:15:55.827 INFO:tasks.workunit.client.0.smithi063.stdout:0/462: mkdir da/de/d17/d25/d3c/d43/d76/d92 0 2024-03-22T07:15:55.827 INFO:tasks.workunit.client.0.smithi063.stdout:0/463: fsync da/de/d17/f1a 0 2024-03-22T07:15:55.828 INFO:tasks.workunit.client.1.smithi069.stdout:4/116: rename d6/dc/l1e to d6/dc/d1d/l22 0 2024-03-22T07:15:55.828 INFO:tasks.workunit.client.1.smithi069.stdout:4/117: chown d6/f9 634024 1 2024-03-22T07:15:55.828 INFO:tasks.workunit.client.1.smithi069.stdout:4/118: fsync f5 0 2024-03-22T07:15:55.828 INFO:tasks.workunit.client.1.smithi069.stdout:0/129: creat d3/d7/d15/d29/f30 x:0 0 0 2024-03-22T07:15:55.830 INFO:tasks.workunit.client.1.smithi069.stdout:4/119: mkdir d6/dc/d23 0 2024-03-22T07:15:55.830 INFO:tasks.workunit.client.1.smithi069.stdout:4/120: write d6/dc/f1a [3195899,43864] 0 2024-03-22T07:15:55.833 INFO:tasks.workunit.client.1.smithi069.stdout:4/121: mknod d6/dc/c24 0 2024-03-22T07:15:55.834 INFO:tasks.workunit.client.1.smithi069.stdout:4/122: mknod d6/d12/c25 0 2024-03-22T07:15:55.835 INFO:tasks.workunit.client.1.smithi069.stdout:4/123: stat d6/l20 0 2024-03-22T07:15:55.837 INFO:tasks.workunit.client.1.smithi069.stdout:8/129: write d1/f14 [2962754,71226] 0 2024-03-22T07:15:55.839 INFO:tasks.workunit.client.1.smithi069.stdout:8/130: creat d1/de/d20/f26 x:0 0 0 2024-03-22T07:15:55.840 INFO:tasks.workunit.client.0.smithi063.stdout:7/285: dwrite d5/f39 [0,4194304] 0 2024-03-22T07:15:55.844 INFO:tasks.workunit.client.0.smithi063.stdout:7/286: link d5/l15 d5/da/d28/d31/l5a 0 2024-03-22T07:15:55.846 INFO:tasks.workunit.client.0.smithi063.stdout:7/287: mknod d5/da/c5b 0 2024-03-22T07:15:55.850 INFO:tasks.workunit.client.1.smithi069.stdout:4/124: write d6/f9 [11097866,50146] 0 2024-03-22T07:15:55.855 INFO:tasks.workunit.client.1.smithi069.stdout:6/95: dwrite f3 [0,4194304] 0 2024-03-22T07:15:55.856 INFO:tasks.workunit.client.0.smithi063.stdout:9/273: dwrite d8/d19/f45 [0,4194304] 0 2024-03-22T07:15:55.856 INFO:tasks.workunit.client.0.smithi063.stdout:9/274: chown d8/d15 0 1 2024-03-22T07:15:55.859 INFO:tasks.workunit.client.1.smithi069.stdout:6/96: unlink d4/d7/da/ff 0 2024-03-22T07:15:55.860 INFO:tasks.workunit.client.1.smithi069.stdout:6/97: creat d4/dd/d12/f1f x:0 0 0 2024-03-22T07:15:55.860 INFO:tasks.workunit.client.0.smithi063.stdout:0/464: dwrite da/de/d17/d25/d3c/d64/d78/f85 [0,4194304] 0 2024-03-22T07:15:55.860 INFO:tasks.workunit.client.0.smithi063.stdout:0/465: stat da/de/d17/d25/d3c/d43 0 2024-03-22T07:15:55.861 INFO:tasks.workunit.client.1.smithi069.stdout:0/130: dwrite d3/d7/d17/f1a [0,4194304] 0 2024-03-22T07:15:55.864 INFO:tasks.workunit.client.0.smithi063.stdout:9/275: write d8/d19/d20/d22/d29/f36 [3662334,100894] 0 2024-03-22T07:15:55.865 INFO:tasks.workunit.client.0.smithi063.stdout:7/288: dwrite d5/d14/d1d/d29/f36 [0,4194304] 0 2024-03-22T07:15:55.865 INFO:tasks.workunit.client.0.smithi063.stdout:0/466: read da/d49/f6d [3737639,71530] 0 2024-03-22T07:15:55.866 INFO:tasks.workunit.client.1.smithi069.stdout:4/125: dread d6/dc/f1a [0,4194304] 0 2024-03-22T07:15:55.866 INFO:tasks.workunit.client.0.smithi063.stdout:9/276: unlink d8/le 0 2024-03-22T07:15:55.867 INFO:tasks.workunit.client.1.smithi069.stdout:6/98: mknod d4/dd/d12/d13/c20 0 2024-03-22T07:15:55.869 INFO:tasks.workunit.client.0.smithi063.stdout:7/289: creat d5/da/d28/d31/f5c x:0 0 0 2024-03-22T07:15:55.869 INFO:tasks.workunit.client.0.smithi063.stdout:7/290: readlink d5/l3e 0 2024-03-22T07:15:55.870 INFO:tasks.workunit.client.0.smithi063.stdout:0/467: symlink da/de/d60/l93 0 2024-03-22T07:15:55.870 INFO:tasks.workunit.client.0.smithi063.stdout:0/468: fsync da/de/d17/f2f 0 2024-03-22T07:15:55.871 INFO:tasks.workunit.client.0.smithi063.stdout:9/277: creat d8/f54 x:0 0 0 2024-03-22T07:15:55.871 INFO:tasks.workunit.client.0.smithi063.stdout:9/278: write d8/d19/d20/d22/d29/d31/f51 [736816,113575] 0 2024-03-22T07:15:55.871 INFO:tasks.workunit.client.0.smithi063.stdout:9/279: truncate d8/f13 695991 0 2024-03-22T07:15:55.872 INFO:tasks.workunit.client.0.smithi063.stdout:9/280: chown d8/d17/d1e 416289149 1 2024-03-22T07:15:55.872 INFO:tasks.workunit.client.0.smithi063.stdout:9/281: write d8/d15/f46 [883040,40059] 0 2024-03-22T07:15:55.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:55 smithi063 ceph-mon[29200]: pgmap v250: 65 pgs: 65 active+clean; 501 MiB data, 3.2 GiB used, 533 GiB / 536 GiB avail; 36 MiB/s rd, 44 MiB/s wr, 639 op/s 2024-03-22T07:15:55.873 INFO:tasks.workunit.client.0.smithi063.stdout:0/469: symlink da/de/d17/d25/d3c/d43/d76/d92/l94 0 2024-03-22T07:15:55.875 INFO:tasks.workunit.client.1.smithi069.stdout:0/131: mknod d3/d7/d15/d29/c31 0 2024-03-22T07:15:55.875 INFO:tasks.workunit.client.1.smithi069.stdout:0/132: fdatasync d3/d7/f1e 0 2024-03-22T07:15:55.877 INFO:tasks.workunit.client.1.smithi069.stdout:6/99: symlink d4/dd/d12/d13/l21 0 2024-03-22T07:15:55.877 INFO:tasks.workunit.client.0.smithi063.stdout:7/291: dread d5/f9 [0,4194304] 0 2024-03-22T07:15:55.878 INFO:tasks.workunit.client.0.smithi063.stdout:7/292: write d5/da/f11 [443432,107591] 0 2024-03-22T07:15:55.879 INFO:tasks.workunit.client.1.smithi069.stdout:7/199: link d9/db/f17 d9/d2b/f43 0 2024-03-22T07:15:55.881 INFO:tasks.workunit.client.0.smithi063.stdout:7/293: dread d5/d16/f2b [4194304,4194304] 0 2024-03-22T07:15:55.881 INFO:tasks.workunit.client.0.smithi063.stdout:7/294: chown d5/d14/d1d/d29/f36 5931 1 2024-03-22T07:15:55.882 INFO:tasks.workunit.client.1.smithi069.stdout:0/133: rmdir d3/d7 39 2024-03-22T07:15:55.883 INFO:tasks.workunit.client.0.smithi063.stdout:7/295: rename d5/d14/f1a to d5/d14/d1d/d29/f5d 0 2024-03-22T07:15:55.884 INFO:tasks.workunit.client.0.smithi063.stdout:7/296: symlink d5/d14/d1d/d40/l5e 0 2024-03-22T07:15:55.885 INFO:tasks.workunit.client.1.smithi069.stdout:4/126: rename l2 to d6/l26 0 2024-03-22T07:15:55.886 INFO:tasks.workunit.client.1.smithi069.stdout:7/200: rmdir d9/d2b/d1d/d2a 39 2024-03-22T07:15:55.888 INFO:tasks.workunit.client.1.smithi069.stdout:6/100: dread f3 [0,4194304] 0 2024-03-22T07:15:55.888 INFO:tasks.workunit.client.1.smithi069.stdout:0/134: creat d3/d7/d15/f32 x:0 0 0 2024-03-22T07:15:55.891 INFO:tasks.workunit.client.1.smithi069.stdout:4/127: link d6/dc/d1d/l22 d6/dc/d23/l27 0 2024-03-22T07:15:55.891 INFO:tasks.workunit.client.1.smithi069.stdout:4/128: readlink d6/dc/d1d/l22 0 2024-03-22T07:15:55.891 INFO:tasks.workunit.client.1.smithi069.stdout:4/129: chown d6/l8 4586856 1 2024-03-22T07:15:55.892 INFO:tasks.workunit.client.1.smithi069.stdout:4/130: getdents d6/dc/d10/d1f 0 2024-03-22T07:15:55.893 INFO:tasks.workunit.client.1.smithi069.stdout:4/131: mknod d6/d12/d18/c28 0 2024-03-22T07:15:55.895 INFO:tasks.workunit.client.1.smithi069.stdout:4/132: symlink d6/dc/d1d/l29 0 2024-03-22T07:15:55.897 INFO:tasks.workunit.client.1.smithi069.stdout:4/133: dread d6/dc/d10/f14 [0,4194304] 0 2024-03-22T07:15:55.898 INFO:tasks.workunit.client.1.smithi069.stdout:7/201: write d9/d2b/f43 [84488,92444] 0 2024-03-22T07:15:55.899 INFO:tasks.workunit.client.1.smithi069.stdout:7/202: readlink d9/d2b/l2d 0 2024-03-22T07:15:55.900 INFO:tasks.workunit.client.1.smithi069.stdout:7/203: mkdir d9/d41/d44 0 2024-03-22T07:15:55.900 INFO:tasks.workunit.client.1.smithi069.stdout:4/134: dread f5 [0,4194304] 0 2024-03-22T07:15:55.902 INFO:tasks.workunit.client.1.smithi069.stdout:7/204: link d9/d2b/l2d d9/d2b/d1d/d2a/d30/d3d/l45 0 2024-03-22T07:15:55.902 INFO:tasks.workunit.client.1.smithi069.stdout:7/205: write d9/db/d1a/f39 [411014,63568] 0 2024-03-22T07:15:55.902 INFO:tasks.workunit.client.1.smithi069.stdout:7/206: chown d9/d2b/d1d/d2a 196055545 1 2024-03-22T07:15:55.903 INFO:tasks.workunit.client.1.smithi069.stdout:4/135: mknod d6/d12/d18/c2a 0 2024-03-22T07:15:55.903 INFO:tasks.workunit.client.1.smithi069.stdout:4/136: write d6/dc/ff [1635991,20228] 0 2024-03-22T07:15:55.904 INFO:tasks.workunit.client.1.smithi069.stdout:7/207: rename f6 to d9/f46 0 2024-03-22T07:15:55.906 INFO:tasks.workunit.client.0.smithi063.stdout:9/282: dwrite d8/d19/d20/d22/f27 [0,4194304] 0 2024-03-22T07:15:55.913 INFO:tasks.workunit.client.0.smithi063.stdout:0/470: dwrite da/de/d17/d25/d3c/d5f/f52 [0,4194304] 0 2024-03-22T07:15:55.913 INFO:tasks.workunit.client.1.smithi069.stdout:0/135: dwrite d3/d7/d17/f1a [0,4194304] 0 2024-03-22T07:15:55.916 INFO:tasks.workunit.client.0.smithi063.stdout:9/283: fsync d8/d19/d20/d22/d29/d31/d4b/f4d 0 2024-03-22T07:15:55.919 INFO:tasks.workunit.client.0.smithi063.stdout:0/471: mkdir da/de/d17/d25/d95 0 2024-03-22T07:15:55.920 INFO:tasks.workunit.client.1.smithi069.stdout:6/101: dwrite d4/dd/d12/f1f [0,4194304] 0 2024-03-22T07:15:55.921 INFO:tasks.workunit.client.1.smithi069.stdout:7/208: dwrite d9/d2b/d1d/d2a/d30/f3a [0,4194304] 0 2024-03-22T07:15:55.922 INFO:tasks.workunit.client.0.smithi063.stdout:7/297: rename d5/da/d28/d31 to d5/da/d5f 0 2024-03-22T07:15:55.922 INFO:tasks.workunit.client.0.smithi063.stdout:7/298: write d5/d14/d1d/d29/d4a/f55 [211588,41785] 0 2024-03-22T07:15:55.922 INFO:tasks.workunit.client.0.smithi063.stdout:9/284: symlink d8/d19/d41/l55 0 2024-03-22T07:15:55.923 INFO:tasks.workunit.client.0.smithi063.stdout:0/472: rename da/de/d17/d25/d3c/d43/c7b to da/de/d17/d25/d3c/d43/d80/c96 0 2024-03-22T07:15:55.923 INFO:tasks.workunit.client.0.smithi063.stdout:0/473: chown da/de/d17/d25/d3c/l7e 14808051 1 2024-03-22T07:15:55.924 INFO:tasks.workunit.client.0.smithi063.stdout:9/285: mkdir d8/d19/d20/d22/d56 0 2024-03-22T07:15:55.925 INFO:tasks.workunit.client.1.smithi069.stdout:0/136: dread d3/da/f14 [0,4194304] 0 2024-03-22T07:15:55.925 INFO:tasks.workunit.client.1.smithi069.stdout:0/137: creat d3/d7/d15/d29/f33 x:0 0 0 2024-03-22T07:15:55.925 INFO:tasks.workunit.client.1.smithi069.stdout:0/138: dread - d3/d7/f12 zero size 2024-03-22T07:15:55.926 INFO:tasks.workunit.client.1.smithi069.stdout:7/209: creat d9/d41/d44/f47 x:0 0 0 2024-03-22T07:15:55.926 INFO:tasks.workunit.client.1.smithi069.stdout:6/102: symlink d4/l22 0 2024-03-22T07:15:55.926 INFO:tasks.workunit.client.1.smithi069.stdout:6/103: chown d4 121665777 1 2024-03-22T07:15:55.926 INFO:tasks.workunit.client.0.smithi063.stdout:0/474: mkdir da/de/d17/d25/d3c/d5f/d97 0 2024-03-22T07:15:55.928 INFO:tasks.workunit.client.1.smithi069.stdout:9/88: sync 2024-03-22T07:15:55.928 INFO:tasks.workunit.client.1.smithi069.stdout:5/68: sync 2024-03-22T07:15:55.928 INFO:tasks.workunit.client.1.smithi069.stdout:5/69: fdatasync d1/f3 0 2024-03-22T07:15:55.928 INFO:tasks.workunit.client.1.smithi069.stdout:3/114: sync 2024-03-22T07:15:55.931 INFO:tasks.workunit.client.1.smithi069.stdout:5/70: symlink d1/d10/l13 0 2024-03-22T07:15:55.934 INFO:tasks.workunit.client.1.smithi069.stdout:3/115: rename f2 to d5/d9/d26/f28 0 2024-03-22T07:15:55.935 INFO:tasks.workunit.client.0.smithi063.stdout:9/286: dread d8/d19/d20/d22/f27 [0,4194304] 0 2024-03-22T07:15:55.935 INFO:tasks.workunit.client.0.smithi063.stdout:9/287: chown d8/d19/d20/d22/d29/d31/f51 139184 1 2024-03-22T07:15:55.937 INFO:tasks.workunit.client.1.smithi069.stdout:6/104: rmdir d4/dd/d12 39 2024-03-22T07:15:55.937 INFO:tasks.workunit.client.1.smithi069.stdout:0/139: dread d3/d7/d15/d1f/f2b [0,4194304] 0 2024-03-22T07:15:55.938 INFO:tasks.workunit.client.0.smithi063.stdout:0/475: dread da/de/d17/f3a [0,4194304] 0 2024-03-22T07:15:55.940 INFO:tasks.workunit.client.0.smithi063.stdout:0/476: unlink da/de/f18 0 2024-03-22T07:15:55.940 INFO:tasks.workunit.client.0.smithi063.stdout:0/477: getdents da/de/d60/d56/d77 0 2024-03-22T07:15:55.940 INFO:tasks.workunit.client.0.smithi063.stdout:0/478: chown da/de/d60 269281 1 2024-03-22T07:15:55.940 INFO:tasks.workunit.client.0.smithi063.stdout:0/479: write da/de/f5c [1245086,90843] 0 2024-03-22T07:15:55.940 INFO:tasks.workunit.client.0.smithi063.stdout:0/480: readlink da/de/d39/l42 0 2024-03-22T07:15:55.940 INFO:tasks.workunit.client.0.smithi063.stdout:9/288: dread d8/d19/d20/d22/d29/f36 [0,4194304] 0 2024-03-22T07:15:55.942 INFO:tasks.workunit.client.1.smithi069.stdout:6/105: link d4/dd/d12/f1f d4/dd/d12/d13/f23 0 2024-03-22T07:15:55.942 INFO:tasks.workunit.client.1.smithi069.stdout:9/89: write d2/dd/f10 [135735,127555] 0 2024-03-22T07:15:55.942 INFO:tasks.workunit.client.1.smithi069.stdout:4/137: dwrite d6/dc/ff [4194304,4194304] 0 2024-03-22T07:15:55.942 INFO:tasks.workunit.client.1.smithi069.stdout:9/90: write d2/dd/f10 [2150448,85504] 0 2024-03-22T07:15:55.942 INFO:tasks.workunit.client.1.smithi069.stdout:9/91: write d2/dd/f10 [2003056,25819] 0 2024-03-22T07:15:55.942 INFO:tasks.workunit.client.0.smithi063.stdout:9/289: symlink d8/d19/d20/d22/d29/d31/d53/l57 0 2024-03-22T07:15:55.944 INFO:tasks.workunit.client.0.smithi063.stdout:9/290: chown d8/f14 28 1 2024-03-22T07:15:55.945 INFO:tasks.workunit.client.1.smithi069.stdout:3/116: getdents d5/d9/d1e 0 2024-03-22T07:15:55.945 INFO:tasks.workunit.client.1.smithi069.stdout:3/117: chown l4 924880 1 2024-03-22T07:15:55.945 INFO:tasks.workunit.client.1.smithi069.stdout:5/71: read d1/f9 [2461332,78272] 0 2024-03-22T07:15:55.945 INFO:tasks.workunit.client.1.smithi069.stdout:5/72: write d1/f9 [2370527,35455] 0 2024-03-22T07:15:55.946 INFO:tasks.workunit.client.1.smithi069.stdout:7/210: dread d9/db/f17 [0,4194304] 0 2024-03-22T07:15:55.946 INFO:tasks.workunit.client.1.smithi069.stdout:7/211: chown f8 244 1 2024-03-22T07:15:55.946 INFO:tasks.workunit.client.0.smithi063.stdout:0/481: getdents da/de 0 2024-03-22T07:15:55.947 INFO:tasks.workunit.client.0.smithi063.stdout:0/482: chown da/de/d17/d25/f27 4462475 1 2024-03-22T07:15:55.947 INFO:tasks.workunit.client.0.smithi063.stdout:0/483: mkdir da/de/d17/d25/d95/d98 0 2024-03-22T07:15:55.947 INFO:tasks.workunit.client.1.smithi069.stdout:4/138: dread d6/dc/d10/f14 [0,4194304] 0 2024-03-22T07:15:55.949 INFO:tasks.workunit.client.0.smithi063.stdout:0/484: dread da/de/d17/d25/f27 [0,4194304] 0 2024-03-22T07:15:55.950 INFO:tasks.workunit.client.0.smithi063.stdout:0/485: mkdir da/de/d17/d25/d99 0 2024-03-22T07:15:55.950 INFO:tasks.workunit.client.1.smithi069.stdout:9/92: unlink d2/f16 0 2024-03-22T07:15:55.950 INFO:tasks.workunit.client.1.smithi069.stdout:9/93: read d2/dd/f10 [2498692,117413] 0 2024-03-22T07:15:55.950 INFO:tasks.workunit.client.1.smithi069.stdout:9/94: stat d2/dd/l15 0 2024-03-22T07:15:55.951 INFO:tasks.workunit.client.0.smithi063.stdout:0/486: mknod da/de/d67/c9a 0 2024-03-22T07:15:55.952 INFO:tasks.workunit.client.0.smithi063.stdout:0/487: mkdir da/de/d60/d9b 0 2024-03-22T07:15:55.954 INFO:tasks.workunit.client.1.smithi069.stdout:3/118: rename l4 to d5/d9/d26/l29 0 2024-03-22T07:15:55.957 INFO:tasks.workunit.client.1.smithi069.stdout:5/73: creat d1/d10/f14 x:0 0 0 2024-03-22T07:15:55.957 INFO:tasks.workunit.client.1.smithi069.stdout:5/74: readlink d1/d7/d8/lc 0 2024-03-22T07:15:55.961 INFO:tasks.workunit.client.1.smithi069.stdout:9/95: unlink d2/dd/c1d 0 2024-03-22T07:15:55.963 INFO:tasks.workunit.client.1.smithi069.stdout:3/119: link d5/c8 d5/d9/d23/c2a 0 2024-03-22T07:15:55.963 INFO:tasks.workunit.client.1.smithi069.stdout:0/140: rmdir d3/d7/d15/d29 39 2024-03-22T07:15:55.965 INFO:tasks.workunit.client.1.smithi069.stdout:5/75: rename d1/d10/f14 to d1/d7/d8/f15 0 2024-03-22T07:15:55.967 INFO:tasks.workunit.client.1.smithi069.stdout:9/96: mkdir d2/d21 0 2024-03-22T07:15:55.967 INFO:tasks.workunit.client.1.smithi069.stdout:9/97: stat d2/dd 0 2024-03-22T07:15:55.970 INFO:tasks.workunit.client.1.smithi069.stdout:3/120: rename d5/d9/f15 to d5/d9/d23/f2b 0 2024-03-22T07:15:55.970 INFO:tasks.workunit.client.1.smithi069.stdout:3/121: readlink d5/d9/l1d 0 2024-03-22T07:15:55.970 INFO:tasks.workunit.client.1.smithi069.stdout:3/122: chown d5/d9/db/f1c 146112 1 2024-03-22T07:15:55.972 INFO:tasks.workunit.client.1.smithi069.stdout:3/123: unlink d5/d9/fa 0 2024-03-22T07:15:55.973 INFO:tasks.workunit.client.1.smithi069.stdout:3/124: mknod d5/c2c 0 2024-03-22T07:15:55.974 INFO:tasks.workunit.client.1.smithi069.stdout:3/125: mkdir d5/d9/db/d2d 0 2024-03-22T07:15:55.976 INFO:tasks.workunit.client.1.smithi069.stdout:3/126: rename d5/d9/f19 to d5/d9/d26/f2e 0 2024-03-22T07:15:55.976 INFO:tasks.workunit.client.1.smithi069.stdout:3/127: chown d5/d9/db/c27 14687 1 2024-03-22T07:15:55.977 INFO:tasks.workunit.client.1.smithi069.stdout:3/128: symlink d5/l2f 0 2024-03-22T07:15:55.978 INFO:tasks.workunit.client.1.smithi069.stdout:3/129: fsync d5/d9/f13 0 2024-03-22T07:15:55.983 INFO:tasks.workunit.client.0.smithi063.stdout:1/432: sync 2024-03-22T07:15:55.983 INFO:tasks.workunit.client.0.smithi063.stdout:1/433: fsync d4/df/d13/d2b/d6c/f89 0 2024-03-22T07:15:55.983 INFO:tasks.workunit.client.0.smithi063.stdout:8/417: sync 2024-03-22T07:15:55.983 INFO:tasks.workunit.client.0.smithi063.stdout:4/331: sync 2024-03-22T07:15:55.983 INFO:tasks.workunit.client.0.smithi063.stdout:5/369: sync 2024-03-22T07:15:55.983 INFO:tasks.workunit.client.0.smithi063.stdout:2/354: sync 2024-03-22T07:15:55.984 INFO:tasks.workunit.client.1.smithi069.stdout:9/98: read d2/dd/d18/f20 [2164746,63933] 0 2024-03-22T07:15:55.985 INFO:tasks.workunit.client.0.smithi063.stdout:5/370: rmdir d8 39 2024-03-22T07:15:55.985 INFO:tasks.workunit.client.1.smithi069.stdout:9/99: unlink d2/dd/c1a 0 2024-03-22T07:15:55.985 INFO:tasks.workunit.client.1.smithi069.stdout:9/100: rmdir d2/dd/d18 39 2024-03-22T07:15:55.986 INFO:tasks.workunit.client.0.smithi063.stdout:4/332: mknod d1/d5/d43/d27/d2c/c7a 0 2024-03-22T07:15:55.987 INFO:tasks.workunit.client.1.smithi069.stdout:6/106: dwrite f3 [0,4194304] 0 2024-03-22T07:15:55.987 INFO:tasks.workunit.client.1.smithi069.stdout:6/107: chown d4/dd/d12/f1f 40114191 1 2024-03-22T07:15:55.987 INFO:tasks.workunit.client.1.smithi069.stdout:9/101: mknod d2/dd/c22 0 2024-03-22T07:15:55.987 INFO:tasks.workunit.client.1.smithi069.stdout:9/102: chown d2/dd/c17 476 1 2024-03-22T07:15:55.987 INFO:tasks.workunit.client.0.smithi063.stdout:1/434: getdents d4/df/d2c 0 2024-03-22T07:15:55.989 INFO:tasks.workunit.client.1.smithi069.stdout:6/108: rename d4/dd/d12/c19 to d4/d7/c24 0 2024-03-22T07:15:55.989 INFO:tasks.workunit.client.1.smithi069.stdout:6/109: truncate d4/dd/d12/f1d 53951 0 2024-03-22T07:15:55.989 INFO:tasks.workunit.client.1.smithi069.stdout:6/110: creat d4/dd/f25 x:0 0 0 2024-03-22T07:15:55.990 INFO:tasks.workunit.client.0.smithi063.stdout:8/418: rename d0/d1b/d28/d7b/l23 to d0/l7e 0 2024-03-22T07:15:55.990 INFO:tasks.workunit.client.1.smithi069.stdout:9/103: mknod d2/dd/c23 0 2024-03-22T07:15:55.991 INFO:tasks.workunit.client.0.smithi063.stdout:1/435: creat d4/f93 x:0 0 0 2024-03-22T07:15:55.992 INFO:tasks.workunit.client.0.smithi063.stdout:8/419: getdents d0/d1b/d28 0 2024-03-22T07:15:55.992 INFO:tasks.workunit.client.0.smithi063.stdout:8/420: dread - d0/d1f/f5c zero size 2024-03-22T07:15:55.992 INFO:tasks.workunit.client.0.smithi063.stdout:8/421: stat d0/ca 0 2024-03-22T07:15:55.992 INFO:tasks.workunit.client.0.smithi063.stdout:8/422: write d0/d24/d33/f5e [4619509,11394] 0 2024-03-22T07:15:55.994 INFO:tasks.workunit.client.0.smithi063.stdout:1/436: rename d4/f73 to d4/d14/f94 0 2024-03-22T07:15:55.995 INFO:tasks.workunit.client.0.smithi063.stdout:1/437: fdatasync d4/df/d13/d39/f5f 0 2024-03-22T07:15:55.995 INFO:tasks.workunit.client.0.smithi063.stdout:1/438: chown d4/d71/d88 5463 1 2024-03-22T07:15:55.995 INFO:tasks.workunit.client.1.smithi069.stdout:9/104: creat d2/dd/d18/f24 x:0 0 0 2024-03-22T07:15:55.996 INFO:tasks.workunit.client.1.smithi069.stdout:9/105: getdents d2/d21 0 2024-03-22T07:15:55.996 INFO:tasks.workunit.client.0.smithi063.stdout:8/423: creat d0/d1b/d28/d7b/f7f x:0 0 0 2024-03-22T07:15:55.997 INFO:tasks.workunit.client.0.smithi063.stdout:1/439: symlink d4/d5a/l95 0 2024-03-22T07:15:55.998 INFO:tasks.workunit.client.0.smithi063.stdout:1/440: readlink d4/lc 0 2024-03-22T07:15:56.000 INFO:tasks.workunit.client.0.smithi063.stdout:1/441: unlink d4/d14/f50 0 2024-03-22T07:15:56.001 INFO:tasks.workunit.client.0.smithi063.stdout:4/333: write d1/d5/d38/f44 [3385633,122533] 0 2024-03-22T07:15:56.002 INFO:tasks.workunit.client.0.smithi063.stdout:4/334: write d1/f26 [948912,10904] 0 2024-03-22T07:15:56.003 INFO:tasks.workunit.client.0.smithi063.stdout:1/442: symlink d4/df/l96 0 2024-03-22T07:15:56.005 INFO:tasks.workunit.client.0.smithi063.stdout:9/291: dwrite d8/d19/f45 [0,4194304] 0 2024-03-22T07:15:56.005 INFO:tasks.workunit.client.1.smithi069.stdout:4/139: dwrite d6/dc/d10/f14 [0,4194304] 0 2024-03-22T07:15:56.006 INFO:tasks.workunit.client.1.smithi069.stdout:4/140: read d6/dc/f1a [624949,20369] 0 2024-03-22T07:15:56.006 INFO:tasks.workunit.client.0.smithi063.stdout:1/443: unlink d4/df/c6a 0 2024-03-22T07:15:56.006 INFO:tasks.workunit.client.0.smithi063.stdout:1/444: fdatasync d4/df/d13/d39/f8e 0 2024-03-22T07:15:56.007 INFO:tasks.workunit.client.0.smithi063.stdout:9/292: write d8/fa [2070845,46032] 0 2024-03-22T07:15:56.007 INFO:tasks.workunit.client.0.smithi063.stdout:4/335: dread d1/d5/d38/f49 [0,4194304] 0 2024-03-22T07:15:56.008 INFO:tasks.workunit.client.0.smithi063.stdout:1/445: creat d4/d8b/f97 x:0 0 0 2024-03-22T07:15:56.010 INFO:tasks.workunit.client.0.smithi063.stdout:1/446: unlink d4/d14/f47 0 2024-03-22T07:15:56.017 INFO:tasks.workunit.client.0.smithi063.stdout:8/424: write d0/d24/f29 [4989892,121261] 0 2024-03-22T07:15:56.019 INFO:tasks.workunit.client.0.smithi063.stdout:8/425: symlink d0/d1b/d28/d3e/l80 0 2024-03-22T07:15:56.022 INFO:tasks.workunit.client.0.smithi063.stdout:8/426: rename d0/c14 to d0/d1b/d28/d3e/c81 0 2024-03-22T07:15:56.022 INFO:tasks.workunit.client.0.smithi063.stdout:8/427: chown d0/d1b/d28/d7b/d4a/c51 2 1 2024-03-22T07:15:56.022 INFO:tasks.workunit.client.0.smithi063.stdout:8/428: write d0/fc [3190475,102505] 0 2024-03-22T07:15:56.023 INFO:tasks.workunit.client.0.smithi063.stdout:8/429: symlink d0/d1f/l82 0 2024-03-22T07:15:56.033 INFO:tasks.workunit.client.1.smithi069.stdout:7/212: dwrite d9/f3b [0,4194304] 0 2024-03-22T07:15:56.034 INFO:tasks.workunit.client.1.smithi069.stdout:5/76: dwrite d1/d7/d8/f12 [0,4194304] 0 2024-03-22T07:15:56.034 INFO:tasks.workunit.client.1.smithi069.stdout:0/141: dwrite d3/d7/d15/d29/f33 [0,4194304] 0 2024-03-22T07:15:56.034 INFO:tasks.workunit.client.1.smithi069.stdout:7/213: mkdir d9/db/d48 0 2024-03-22T07:15:56.037 INFO:tasks.workunit.client.1.smithi069.stdout:0/142: creat d3/d7/d24/f34 x:0 0 0 2024-03-22T07:15:56.038 INFO:tasks.workunit.client.1.smithi069.stdout:0/143: symlink d3/d7/d15/d1f/l35 0 2024-03-22T07:15:56.038 INFO:tasks.workunit.client.1.smithi069.stdout:0/144: chown d3/d7/cc 408663072 1 2024-03-22T07:15:56.038 INFO:tasks.workunit.client.1.smithi069.stdout:0/145: chown d3/da/c23 15464 1 2024-03-22T07:15:56.039 INFO:tasks.workunit.client.1.smithi069.stdout:0/146: symlink d3/d7/d15/l36 0 2024-03-22T07:15:56.039 INFO:tasks.workunit.client.1.smithi069.stdout:7/214: dread d9/db/f17 [0,4194304] 0 2024-03-22T07:15:56.040 INFO:tasks.workunit.client.1.smithi069.stdout:0/147: mknod d3/d7/d17/c37 0 2024-03-22T07:15:56.041 INFO:tasks.workunit.client.1.smithi069.stdout:0/148: creat d3/da/f38 x:0 0 0 2024-03-22T07:15:56.041 INFO:tasks.workunit.client.1.smithi069.stdout:0/149: stat d3/da/l25 0 2024-03-22T07:15:56.047 INFO:tasks.workunit.client.1.smithi069.stdout:3/130: dwrite d5/d9/fe [0,4194304] 0 2024-03-22T07:15:56.047 INFO:tasks.workunit.client.1.smithi069.stdout:3/131: readlink d5/d9/l1f 0 2024-03-22T07:15:56.047 INFO:tasks.workunit.client.1.smithi069.stdout:3/132: readlink d5/l2f 0 2024-03-22T07:15:56.048 INFO:tasks.workunit.client.0.smithi063.stdout:2/355: dwrite d0/de/d12/d20/f3f [0,4194304] 0 2024-03-22T07:15:56.048 INFO:tasks.workunit.client.0.smithi063.stdout:2/356: chown d0/de/l5b 4318 1 2024-03-22T07:15:56.049 INFO:tasks.workunit.client.0.smithi063.stdout:2/357: chown d0/de/d12/d20/c45 1636826 1 2024-03-22T07:15:56.049 INFO:tasks.workunit.client.0.smithi063.stdout:2/358: dread - d0/de/d35/f44 zero size 2024-03-22T07:15:56.049 INFO:tasks.workunit.client.0.smithi063.stdout:2/359: chown d0/de/d12/f1f 1 1 2024-03-22T07:15:56.049 INFO:tasks.workunit.client.0.smithi063.stdout:5/371: dwrite d8/f1c [0,4194304] 0 2024-03-22T07:15:56.049 INFO:tasks.workunit.client.0.smithi063.stdout:2/360: truncate d0/f13 489684 0 2024-03-22T07:15:56.050 INFO:tasks.workunit.client.1.smithi069.stdout:7/215: dread d9/db/d1a/f2c [0,4194304] 0 2024-03-22T07:15:56.050 INFO:tasks.workunit.client.1.smithi069.stdout:3/133: creat d5/d9/db/f30 x:0 0 0 2024-03-22T07:15:56.050 INFO:tasks.workunit.client.1.smithi069.stdout:7/216: readlink d9/d2b/d1d/d2a/d30/d3d/l45 0 2024-03-22T07:15:56.050 INFO:tasks.workunit.client.1.smithi069.stdout:3/134: read d5/d9/db/f1c [45671,106862] 0 2024-03-22T07:15:56.051 INFO:tasks.workunit.client.1.smithi069.stdout:7/217: creat d9/db/d1a/f49 x:0 0 0 2024-03-22T07:15:56.052 INFO:tasks.workunit.client.1.smithi069.stdout:3/135: truncate d5/d9/d26/f28 460353 0 2024-03-22T07:15:56.053 INFO:tasks.workunit.client.1.smithi069.stdout:7/218: readlink d9/db/d1a/l1b 0 2024-03-22T07:15:56.054 INFO:tasks.workunit.client.1.smithi069.stdout:6/111: dwrite d4/d7/da/f1b [0,4194304] 0 2024-03-22T07:15:56.054 INFO:tasks.workunit.client.1.smithi069.stdout:4/141: dwrite d6/dc/ff [4194304,4194304] 0 2024-03-22T07:15:56.054 INFO:tasks.workunit.client.1.smithi069.stdout:9/106: dwrite d2/dd/d18/f24 [0,4194304] 0 2024-03-22T07:15:56.054 INFO:tasks.workunit.client.1.smithi069.stdout:9/107: fsync d2/dd/d18/f20 0 2024-03-22T07:15:56.055 INFO:tasks.workunit.client.1.smithi069.stdout:3/136: creat d5/f31 x:0 0 0 2024-03-22T07:15:56.055 INFO:tasks.workunit.client.0.smithi063.stdout:4/336: fsync d1/d5/d43/f17 0 2024-03-22T07:15:56.055 INFO:tasks.workunit.client.0.smithi063.stdout:4/337: truncate d1/d5/dc/f41 614081 0 2024-03-22T07:15:56.056 INFO:tasks.workunit.client.0.smithi063.stdout:9/293: dwrite f2 [0,4194304] 0 2024-03-22T07:15:56.056 INFO:tasks.workunit.client.0.smithi063.stdout:4/338: creat d1/d5/d43/d27/d2c/f7b x:0 0 0 2024-03-22T07:15:56.057 INFO:tasks.workunit.client.0.smithi063.stdout:4/339: write d1/d5/d43/d27/f4b [454864,23953] 0 2024-03-22T07:15:56.058 INFO:tasks.workunit.client.0.smithi063.stdout:4/340: creat d1/d5/d43/d27/d2c/f7c x:0 0 0 2024-03-22T07:15:56.058 INFO:tasks.workunit.client.0.smithi063.stdout:4/341: fsync d1/d5/d43/f31 0 2024-03-22T07:15:56.058 INFO:tasks.workunit.client.0.smithi063.stdout:1/447: dwrite d4/df/d13/f34 [0,4194304] 0 2024-03-22T07:15:56.058 INFO:tasks.workunit.client.0.smithi063.stdout:1/448: chown d4/d5a/f65 62242 1 2024-03-22T07:15:56.058 INFO:tasks.workunit.client.0.smithi063.stdout:1/449: write d4/f76 [2230382,121894] 0 2024-03-22T07:15:56.063 INFO:tasks.workunit.client.1.smithi069.stdout:3/137: dread d5/d9/db/f1c [0,4194304] 0 2024-03-22T07:15:56.064 INFO:tasks.workunit.client.1.smithi069.stdout:4/142: unlink f3 0 2024-03-22T07:15:56.066 INFO:tasks.workunit.client.1.smithi069.stdout:7/219: dread d9/db/d1a/f2c [0,4194304] 0 2024-03-22T07:15:56.066 INFO:tasks.workunit.client.1.smithi069.stdout:7/220: fdatasync d9/f18 0 2024-03-22T07:15:56.067 INFO:tasks.workunit.client.0.smithi063.stdout:1/450: symlink d4/df/d13/d2b/l98 0 2024-03-22T07:15:56.069 INFO:tasks.workunit.client.0.smithi063.stdout:9/294: creat d8/d15/f58 x:0 0 0 2024-03-22T07:15:56.070 INFO:tasks.workunit.client.0.smithi063.stdout:1/451: mknod d4/df/d13/d39/c99 0 2024-03-22T07:15:56.072 INFO:tasks.workunit.client.0.smithi063.stdout:1/452: truncate d4/df/d2c/f4b 3573666 0 2024-03-22T07:15:56.072 INFO:tasks.workunit.client.0.smithi063.stdout:1/453: write d4/df/f1a [2506536,50301] 0 2024-03-22T07:15:56.074 INFO:tasks.workunit.client.0.smithi063.stdout:1/454: creat d4/d5a/f9a x:0 0 0 2024-03-22T07:15:56.075 INFO:tasks.workunit.client.0.smithi063.stdout:1/455: symlink d4/d5a/l9b 0 2024-03-22T07:15:56.078 INFO:tasks.workunit.client.1.smithi069.stdout:4/143: symlink d6/d12/l2b 0 2024-03-22T07:15:56.080 INFO:tasks.workunit.client.0.smithi063.stdout:2/361: dwrite d0/f53 [0,4194304] 0 2024-03-22T07:15:56.080 INFO:tasks.workunit.client.1.smithi069.stdout:7/221: symlink d9/d2b/d1d/d2a/l4a 0 2024-03-22T07:15:56.081 INFO:tasks.workunit.client.1.smithi069.stdout:9/108: getdents d2/dd 0 2024-03-22T07:15:56.083 INFO:tasks.workunit.client.0.smithi063.stdout:5/372: dwrite d8/d13/f18 [4194304,4194304] 0 2024-03-22T07:15:56.084 INFO:tasks.workunit.client.1.smithi069.stdout:7/222: dread d9/d2b/f20 [0,4194304] 0 2024-03-22T07:15:56.084 INFO:tasks.workunit.client.1.smithi069.stdout:7/223: read - d9/db/f24 zero size 2024-03-22T07:15:56.084 INFO:tasks.workunit.client.1.smithi069.stdout:4/144: unlink d6/d12/d18/c2a 0 2024-03-22T07:15:56.084 INFO:tasks.workunit.client.1.smithi069.stdout:4/145: chown d6/dc/ff 1051421 1 2024-03-22T07:15:56.084 INFO:tasks.workunit.client.1.smithi069.stdout:4/146: stat d6/f9 0 2024-03-22T07:15:56.085 INFO:tasks.workunit.client.1.smithi069.stdout:7/224: mkdir d9/db/d48/d4b 0 2024-03-22T07:15:56.089 INFO:tasks.workunit.client.1.smithi069.stdout:7/225: creat d9/d2b/d2e/f4c x:0 0 0 2024-03-22T07:15:56.089 INFO:tasks.workunit.client.1.smithi069.stdout:7/226: creat d9/d2b/d2e/f4d x:0 0 0 2024-03-22T07:15:56.091 INFO:tasks.workunit.client.1.smithi069.stdout:4/147: mknod d6/c2c 0 2024-03-22T07:15:56.093 INFO:tasks.workunit.client.1.smithi069.stdout:9/109: dread d2/dd/d18/f24 [0,4194304] 0 2024-03-22T07:15:56.093 INFO:tasks.workunit.client.1.smithi069.stdout:9/110: fdatasync d2/f5 0 2024-03-22T07:15:56.093 INFO:tasks.workunit.client.1.smithi069.stdout:7/227: symlink d9/d2b/d1d/d2a/d30/l4e 0 2024-03-22T07:15:56.094 INFO:tasks.workunit.client.1.smithi069.stdout:4/148: truncate d6/dc/d10/f17 180677 0 2024-03-22T07:15:56.096 INFO:tasks.workunit.client.1.smithi069.stdout:9/111: rename d2/c8 to d2/dd/d18/c25 0 2024-03-22T07:15:56.097 INFO:tasks.workunit.client.1.smithi069.stdout:7/228: truncate d9/db/f16 2229138 0 2024-03-22T07:15:56.097 INFO:tasks.workunit.client.1.smithi069.stdout:0/150: fsync d3/d7/d24/f34 0 2024-03-22T07:15:56.099 INFO:tasks.workunit.client.0.smithi063.stdout:2/362: dwrite d0/de/f65 [0,4194304] 0 2024-03-22T07:15:56.099 INFO:tasks.workunit.client.0.smithi063.stdout:2/363: chown d0/de 0 1 2024-03-22T07:15:56.100 INFO:tasks.workunit.client.1.smithi069.stdout:7/229: creat d9/db/d48/d4b/f4f x:0 0 0 2024-03-22T07:15:56.100 INFO:tasks.workunit.client.1.smithi069.stdout:7/230: write d9/d2b/f38 [1182185,63438] 0 2024-03-22T07:15:56.102 INFO:tasks.workunit.client.1.smithi069.stdout:9/112: dread d2/f5 [0,4194304] 0 2024-03-22T07:15:56.103 INFO:tasks.workunit.client.1.smithi069.stdout:7/231: mkdir d9/db/d48/d50 0 2024-03-22T07:15:56.113 INFO:tasks.workunit.client.1.smithi069.stdout:4/149: rmdir d6/dc/d10 39 2024-03-22T07:15:56.117 INFO:tasks.workunit.client.1.smithi069.stdout:9/113: getdents d2 0 2024-03-22T07:15:56.119 INFO:tasks.workunit.client.1.smithi069.stdout:9/114: creat d2/dd/f26 x:0 0 0 2024-03-22T07:15:56.122 INFO:tasks.workunit.client.0.smithi063.stdout:2/364: dwrite d0/de/d12/d39/f76 [0,4194304] 0 2024-03-22T07:15:56.122 INFO:tasks.workunit.client.0.smithi063.stdout:1/456: read d4/d14/f1b [2314527,86000] 0 2024-03-22T07:15:56.122 INFO:tasks.workunit.client.0.smithi063.stdout:1/457: fsync d4/d5a/f65 0 2024-03-22T07:15:56.122 INFO:tasks.workunit.client.0.smithi063.stdout:1/458: write d4/d14/d57/f5d [696923,114944] 0 2024-03-22T07:15:56.122 INFO:tasks.workunit.client.0.smithi063.stdout:9/295: dwrite d8/d19/f3a [0,4194304] 0 2024-03-22T07:15:56.122 INFO:tasks.workunit.client.0.smithi063.stdout:9/296: chown d8/d19/d20/d22/d29/d31/d37 40782 1 2024-03-22T07:15:56.123 INFO:tasks.workunit.client.1.smithi069.stdout:9/115: creat d2/f27 x:0 0 0 2024-03-22T07:15:56.123 INFO:tasks.workunit.client.1.smithi069.stdout:9/116: chown d2/lc 8423 1 2024-03-22T07:15:56.123 INFO:tasks.workunit.client.1.smithi069.stdout:9/117: chown d2/dd/d18 179855 1 2024-03-22T07:15:56.123 INFO:tasks.workunit.client.1.smithi069.stdout:0/151: dwrite d3/d7/d24/f34 [0,4194304] 0 2024-03-22T07:15:56.125 INFO:tasks.workunit.client.1.smithi069.stdout:9/118: dread d2/dd/d18/f20 [0,4194304] 0 2024-03-22T07:15:56.127 INFO:tasks.workunit.client.1.smithi069.stdout:9/119: rename d2/c9 to d2/c28 0 2024-03-22T07:15:56.127 INFO:tasks.workunit.client.0.smithi063.stdout:1/459: rename d4/df/d13/d2b/d3a/l53 to d4/df/d13/d39/d63/l9c 0 2024-03-22T07:15:56.127 INFO:tasks.workunit.client.0.smithi063.stdout:1/460: write d4/df/d13/d2b/d3a/f81 [631610,95261] 0 2024-03-22T07:15:56.128 INFO:tasks.workunit.client.0.smithi063.stdout:6/434: sync 2024-03-22T07:15:56.128 INFO:tasks.workunit.client.0.smithi063.stdout:3/388: sync 2024-03-22T07:15:56.128 INFO:tasks.workunit.client.0.smithi063.stdout:1/461: fsync d4/df/f48 0 2024-03-22T07:15:56.128 INFO:tasks.workunit.client.1.smithi069.stdout:9/120: mknod d2/c29 0 2024-03-22T07:15:56.129 INFO:tasks.workunit.client.0.smithi063.stdout:2/365: mknod d0/de/d35/d36/c7a 0 2024-03-22T07:15:56.130 INFO:tasks.workunit.client.0.smithi063.stdout:2/366: chown d0/de/d1b/d32/c77 229970832 1 2024-03-22T07:15:56.130 INFO:tasks.workunit.client.0.smithi063.stdout:2/367: creat d0/de/d58/f7b x:0 0 0 2024-03-22T07:15:56.130 INFO:tasks.workunit.client.0.smithi063.stdout:2/368: chown d0/de/d12/d5a 17012 1 2024-03-22T07:15:56.130 INFO:tasks.workunit.client.0.smithi063.stdout:3/389: symlink da/d63/l82 0 2024-03-22T07:15:56.132 INFO:tasks.workunit.client.0.smithi063.stdout:2/369: creat d0/de/d35/d36/f7c x:0 0 0 2024-03-22T07:15:56.133 INFO:tasks.workunit.client.1.smithi069.stdout:9/121: getdents d2/dd 0 2024-03-22T07:15:56.133 INFO:tasks.workunit.client.1.smithi069.stdout:9/122: truncate d2/f27 326952 0 2024-03-22T07:15:56.133 INFO:tasks.workunit.client.1.smithi069.stdout:1/146: sync 2024-03-22T07:15:56.133 INFO:tasks.workunit.client.1.smithi069.stdout:2/127: sync 2024-03-22T07:15:56.133 INFO:tasks.workunit.client.1.smithi069.stdout:2/128: fdatasync f1 0 2024-03-22T07:15:56.134 INFO:tasks.workunit.client.0.smithi063.stdout:2/370: unlink d0/de/d1b/c57 0 2024-03-22T07:15:56.134 INFO:tasks.workunit.client.0.smithi063.stdout:2/371: fsync d0/f53 0 2024-03-22T07:15:56.134 INFO:tasks.workunit.client.1.smithi069.stdout:9/123: symlink d2/dd/l2a 0 2024-03-22T07:15:56.136 INFO:tasks.workunit.client.1.smithi069.stdout:2/129: mkdir d18/d1c/d27/d33 0 2024-03-22T07:15:56.136 INFO:tasks.workunit.client.1.smithi069.stdout:2/130: read f12 [4076924,28653] 0 2024-03-22T07:15:56.136 INFO:tasks.workunit.client.0.smithi063.stdout:2/372: rename d0/de/d12/f6a to d0/de/d58/f7d 0 2024-03-22T07:15:56.137 INFO:tasks.workunit.client.1.smithi069.stdout:2/131: mkdir d18/d1c/d34 0 2024-03-22T07:15:56.138 INFO:tasks.workunit.client.0.smithi063.stdout:2/373: creat d0/de/d74/f7e x:0 0 0 2024-03-22T07:15:56.138 INFO:tasks.workunit.client.0.smithi063.stdout:2/374: write d0/de/d74/f7e [810895,71407] 0 2024-03-22T07:15:56.138 INFO:tasks.workunit.client.1.smithi069.stdout:9/124: rename d2/c14 to d2/dd/d18/c2b 0 2024-03-22T07:15:56.139 INFO:tasks.workunit.client.0.smithi063.stdout:6/435: dread d3/d76/f72 [0,4194304] 0 2024-03-22T07:15:56.139 INFO:tasks.workunit.client.0.smithi063.stdout:6/436: chown d3/d1c/c2d 381686312 1 2024-03-22T07:15:56.139 INFO:tasks.workunit.client.0.smithi063.stdout:3/390: read da/f52 [1349521,56746] 0 2024-03-22T07:15:56.139 INFO:tasks.workunit.client.0.smithi063.stdout:2/375: mkdir d0/de/d58/d7f 0 2024-03-22T07:15:56.140 INFO:tasks.workunit.client.1.smithi069.stdout:1/147: rename d1/l9 to d1/l2c 0 2024-03-22T07:15:56.140 INFO:tasks.workunit.client.1.smithi069.stdout:1/148: write d1/f22 [1764164,946] 0 2024-03-22T07:15:56.141 INFO:tasks.workunit.client.0.smithi063.stdout:3/391: mkdir da/d63/d65/d83 0 2024-03-22T07:15:56.141 INFO:tasks.workunit.client.0.smithi063.stdout:3/392: fdatasync da/d16/d3f/f44 0 2024-03-22T07:15:56.141 INFO:tasks.workunit.client.0.smithi063.stdout:1/462: dread d4/d5a/f65 [0,4194304] 0 2024-03-22T07:15:56.142 INFO:tasks.workunit.client.0.smithi063.stdout:3/393: mknod da/d63/d65/d7b/c84 0 2024-03-22T07:15:56.142 INFO:tasks.workunit.client.0.smithi063.stdout:3/394: fdatasync da/d16/d26/d69/f6b 0 2024-03-22T07:15:56.143 INFO:tasks.workunit.client.1.smithi069.stdout:9/125: write d2/dd/d18/f24 [1045404,67419] 0 2024-03-22T07:15:56.143 INFO:tasks.workunit.client.1.smithi069.stdout:1/149: write d1/d13/f17 [177208,125150] 0 2024-03-22T07:15:56.143 INFO:tasks.workunit.client.0.smithi063.stdout:1/463: symlink d4/df/d13/d39/l9d 0 2024-03-22T07:15:56.143 INFO:tasks.workunit.client.1.smithi069.stdout:2/132: dread d18/f1e [0,4194304] 0 2024-03-22T07:15:56.144 INFO:tasks.workunit.client.0.smithi063.stdout:1/464: mkdir d4/df/d13/d2b/d6c/d85/d9e 0 2024-03-22T07:15:56.145 INFO:tasks.workunit.client.1.smithi069.stdout:9/126: write d2/dd/f26 [657115,46651] 0 2024-03-22T07:15:56.145 INFO:tasks.workunit.client.1.smithi069.stdout:4/150: dwrite f5 [0,4194304] 0 2024-03-22T07:15:56.145 INFO:tasks.workunit.client.1.smithi069.stdout:4/151: readlink d6/dc/d1d/l29 0 2024-03-22T07:15:56.145 INFO:tasks.workunit.client.1.smithi069.stdout:4/152: stat d6/d12/d18/d21 0 2024-03-22T07:15:56.146 INFO:tasks.workunit.client.0.smithi063.stdout:1/465: rmdir d4/d71 39 2024-03-22T07:15:56.146 INFO:tasks.workunit.client.0.smithi063.stdout:6/437: dread d3/d6/fa [0,4194304] 0 2024-03-22T07:15:56.146 INFO:tasks.workunit.client.0.smithi063.stdout:1/466: creat d4/d71/d88/f9f x:0 0 0 2024-03-22T07:15:56.146 INFO:tasks.workunit.client.1.smithi069.stdout:2/133: rename d18/d1c/d25/f2c to d18/d1c/d27/d33/f35 0 2024-03-22T07:15:56.147 INFO:tasks.workunit.client.1.smithi069.stdout:2/134: creat d18/d1c/d25/f36 x:0 0 0 2024-03-22T07:15:56.147 INFO:tasks.workunit.client.1.smithi069.stdout:9/127: fsync d2/dd/f10 0 2024-03-22T07:15:56.149 INFO:tasks.workunit.client.0.smithi063.stdout:6/438: rename d3/d15/d16/l31 to d3/d15/d16/d60/d6d/d4e/l92 0 2024-03-22T07:15:56.149 INFO:tasks.workunit.client.1.smithi069.stdout:1/150: dread d1/d25/f28 [0,4194304] 0 2024-03-22T07:15:56.151 INFO:tasks.workunit.client.0.smithi063.stdout:9/297: dwrite d8/d19/d20/d22/f27 [0,4194304] 0 2024-03-22T07:15:56.152 INFO:tasks.workunit.client.1.smithi069.stdout:2/135: mknod d18/d1c/d27/d33/c37 0 2024-03-22T07:15:56.153 INFO:tasks.workunit.client.1.smithi069.stdout:9/128: truncate d2/dd/f10 3521302 0 2024-03-22T07:15:56.153 INFO:tasks.workunit.client.1.smithi069.stdout:9/129: write d2/f27 [541904,88356] 0 2024-03-22T07:15:56.154 INFO:tasks.workunit.client.0.smithi063.stdout:9/298: rmdir d8/d3e 0 2024-03-22T07:15:56.155 INFO:tasks.workunit.client.0.smithi063.stdout:3/395: write da/d16/d3f/f56 [499845,3672] 0 2024-03-22T07:15:56.155 INFO:tasks.workunit.client.0.smithi063.stdout:3/396: write da/fb [183239,61531] 0 2024-03-22T07:15:56.155 INFO:tasks.workunit.client.1.smithi069.stdout:1/151: creat d1/d13/f2d x:0 0 0 2024-03-22T07:15:56.156 INFO:tasks.workunit.client.0.smithi063.stdout:9/299: creat d8/d19/d20/d22/d29/d31/d37/f59 x:0 0 0 2024-03-22T07:15:56.156 INFO:tasks.workunit.client.0.smithi063.stdout:9/300: stat d8/d19/d39 0 2024-03-22T07:15:56.164 INFO:tasks.workunit.client.0.smithi063.stdout:2/376: fsync d0/de/d74/f7e 0 2024-03-22T07:15:56.164 INFO:tasks.workunit.client.1.smithi069.stdout:4/153: dread d6/dc/ff [4194304,4194304] 0 2024-03-22T07:15:56.170 INFO:tasks.workunit.client.1.smithi069.stdout:2/136: dread d18/f1e [0,4194304] 0 2024-03-22T07:15:56.171 INFO:tasks.workunit.client.1.smithi069.stdout:2/137: symlink d18/l38 0 2024-03-22T07:15:56.171 INFO:tasks.workunit.client.1.smithi069.stdout:2/138: creat d18/d1c/d27/d33/f39 x:0 0 0 2024-03-22T07:15:56.172 INFO:tasks.workunit.client.1.smithi069.stdout:2/139: truncate d18/f19 1414396 0 2024-03-22T07:15:56.174 INFO:tasks.workunit.client.1.smithi069.stdout:2/140: dread d18/d1c/f30 [0,4194304] 0 2024-03-22T07:15:56.174 INFO:tasks.workunit.client.1.smithi069.stdout:2/141: creat d18/f3a x:0 0 0 2024-03-22T07:15:56.175 INFO:tasks.workunit.client.1.smithi069.stdout:2/142: symlink d18/d1c/d27/d33/l3b 0 2024-03-22T07:15:56.177 INFO:tasks.workunit.client.1.smithi069.stdout:2/143: link d18/d1c/c31 d18/d1c/d27/c3c 0 2024-03-22T07:15:56.185 INFO:tasks.workunit.client.1.smithi069.stdout:4/154: dread f5 [0,4194304] 0 2024-03-22T07:15:56.186 INFO:tasks.workunit.client.1.smithi069.stdout:4/155: symlink d6/dc/d10/l2d 0 2024-03-22T07:15:56.187 INFO:tasks.workunit.client.1.smithi069.stdout:4/156: mknod d6/d12/c2e 0 2024-03-22T07:15:56.191 INFO:tasks.workunit.client.1.smithi069.stdout:4/157: rmdir d6/dc 39 2024-03-22T07:15:56.192 INFO:tasks.workunit.client.1.smithi069.stdout:4/158: creat d6/d12/d18/d21/f2f x:0 0 0 2024-03-22T07:15:56.193 INFO:tasks.workunit.client.1.smithi069.stdout:4/159: truncate d6/dc/ff 6597834 0 2024-03-22T07:15:56.193 INFO:tasks.workunit.client.1.smithi069.stdout:4/160: write d6/f9 [11670786,111634] 0 2024-03-22T07:15:56.194 INFO:tasks.workunit.client.1.smithi069.stdout:4/161: creat d6/f30 x:0 0 0 2024-03-22T07:15:56.194 INFO:tasks.workunit.client.1.smithi069.stdout:4/162: chown d6/dc/d23/l27 14833020 1 2024-03-22T07:15:56.196 INFO:tasks.workunit.client.1.smithi069.stdout:4/163: rename d6/dc/d10 to d6/dc/d1d/d31 0 2024-03-22T07:15:56.196 INFO:tasks.workunit.client.1.smithi069.stdout:4/164: write d6/dc/f1a [2154441,83188] 0 2024-03-22T07:15:56.196 INFO:tasks.workunit.client.1.smithi069.stdout:4/165: dread - d6/d12/d18/d21/f2f zero size 2024-03-22T07:15:56.197 INFO:tasks.workunit.client.1.smithi069.stdout:4/166: symlink d6/d12/l32 0 2024-03-22T07:15:56.197 INFO:tasks.workunit.client.1.smithi069.stdout:4/167: readlink d6/l7 0 2024-03-22T07:15:56.197 INFO:tasks.workunit.client.1.smithi069.stdout:4/168: stat d6/d12/d18/d21 0 2024-03-22T07:15:56.198 INFO:tasks.workunit.client.1.smithi069.stdout:4/169: creat d6/d12/f33 x:0 0 0 2024-03-22T07:15:56.207 INFO:tasks.workunit.client.1.smithi069.stdout:1/152: dwrite d1/d25/f28 [0,4194304] 0 2024-03-22T07:15:56.208 INFO:tasks.workunit.client.0.smithi063.stdout:3/397: dwrite da/d16/d3f/f64 [0,4194304] 0 2024-03-22T07:15:56.208 INFO:tasks.workunit.client.0.smithi063.stdout:3/398: write da/d16/f6d [508579,37668] 0 2024-03-22T07:15:56.208 INFO:tasks.workunit.client.0.smithi063.stdout:3/399: readlink da/l15 0 2024-03-22T07:15:56.208 INFO:tasks.workunit.client.0.smithi063.stdout:3/400: truncate da/d16/d3f/f67 429265 0 2024-03-22T07:15:56.208 INFO:tasks.workunit.client.0.smithi063.stdout:1/467: dwrite d4/f1e [0,4194304] 0 2024-03-22T07:15:56.210 INFO:tasks.workunit.client.0.smithi063.stdout:6/439: dwrite d3/d1c/f63 [0,4194304] 0 2024-03-22T07:15:56.210 INFO:tasks.workunit.client.0.smithi063.stdout:6/440: chown d3/d6/d2a/d38/f49 49906 1 2024-03-22T07:15:56.211 INFO:tasks.workunit.client.0.smithi063.stdout:9/301: dwrite d8/d19/d41/f4a [0,4194304] 0 2024-03-22T07:15:56.212 INFO:tasks.workunit.client.1.smithi069.stdout:1/153: mknod d1/d25/c2e 0 2024-03-22T07:15:56.212 INFO:tasks.workunit.client.0.smithi063.stdout:3/401: creat da/d63/d65/d4b/f85 x:0 0 0 2024-03-22T07:15:56.212 INFO:tasks.workunit.client.0.smithi063.stdout:3/402: write da/d16/d3f/f4c [939852,40792] 0 2024-03-22T07:15:56.212 INFO:tasks.workunit.client.1.smithi069.stdout:2/144: dwrite d18/f3a [0,4194304] 0 2024-03-22T07:15:56.214 INFO:tasks.workunit.client.1.smithi069.stdout:1/154: truncate d1/fb 207073 0 2024-03-22T07:15:56.214 INFO:tasks.workunit.client.1.smithi069.stdout:1/155: fdatasync d1/d25/f28 0 2024-03-22T07:15:56.216 INFO:tasks.workunit.client.0.smithi063.stdout:6/441: creat d3/d15/d16/d60/d6d/d4e/d5f/f93 x:0 0 0 2024-03-22T07:15:56.218 INFO:tasks.workunit.client.0.smithi063.stdout:6/442: creat d3/d6/d35/f94 x:0 0 0 2024-03-22T07:15:56.219 INFO:tasks.workunit.client.1.smithi069.stdout:1/156: symlink d1/d25/l2f 0 2024-03-22T07:15:56.221 INFO:tasks.workunit.client.1.smithi069.stdout:1/157: mknod d1/d25/c30 0 2024-03-22T07:15:56.222 INFO:tasks.workunit.client.1.smithi069.stdout:9/130: dread d2/dd/f10 [0,4194304] 0 2024-03-22T07:15:56.225 INFO:tasks.workunit.client.1.smithi069.stdout:4/170: dwrite d6/f30 [0,4194304] 0 2024-03-22T07:15:56.226 INFO:tasks.workunit.client.1.smithi069.stdout:1/158: creat d1/d25/f31 x:0 0 0 2024-03-22T07:15:56.226 INFO:tasks.workunit.client.1.smithi069.stdout:1/159: creat d1/d2a/f32 x:0 0 0 2024-03-22T07:15:56.227 INFO:tasks.workunit.client.1.smithi069.stdout:4/171: truncate d6/dc/d1d/d31/f14 3082540 0 2024-03-22T07:15:56.229 INFO:tasks.workunit.client.1.smithi069.stdout:2/145: dread d18/d1c/f22 [0,4194304] 0 2024-03-22T07:15:56.229 INFO:tasks.workunit.client.1.smithi069.stdout:2/146: chown d18/l1f 2 1 2024-03-22T07:15:56.229 INFO:tasks.workunit.client.1.smithi069.stdout:1/160: getdents d1/d13/d16/d1e 0 2024-03-22T07:15:56.229 INFO:tasks.workunit.client.1.smithi069.stdout:1/161: stat d1/l29 0 2024-03-22T07:15:56.230 INFO:tasks.workunit.client.1.smithi069.stdout:1/162: dread - d1/d2a/f32 zero size 2024-03-22T07:15:56.230 INFO:tasks.workunit.client.0.smithi063.stdout:9/302: dread f2 [0,4194304] 0 2024-03-22T07:15:56.230 INFO:tasks.workunit.client.0.smithi063.stdout:9/303: fsync d8/d19/f35 0 2024-03-22T07:15:56.232 INFO:tasks.workunit.client.1.smithi069.stdout:4/172: chown d6/lb 243492 1 2024-03-22T07:15:56.233 INFO:tasks.workunit.client.1.smithi069.stdout:1/163: getdents d1 0 2024-03-22T07:15:56.235 INFO:tasks.workunit.client.0.smithi063.stdout:9/304: dread d8/d19/d20/d22/d29/f36 [0,4194304] 0 2024-03-22T07:15:56.235 INFO:tasks.workunit.client.0.smithi063.stdout:9/305: fsync d8/d19/d20/f21 0 2024-03-22T07:15:56.235 INFO:tasks.workunit.client.0.smithi063.stdout:9/306: write d8/d19/d20/d22/d29/d31/f51 [1835097,3754] 0 2024-03-22T07:15:56.235 INFO:tasks.workunit.client.0.smithi063.stdout:9/307: fdatasync d8/d15/f58 0 2024-03-22T07:15:56.238 INFO:tasks.workunit.client.0.smithi063.stdout:6/443: dread d3/f19 [0,4194304] 0 2024-03-22T07:15:56.240 INFO:tasks.workunit.client.0.smithi063.stdout:6/444: truncate d3/d15/d16/d60/d6d/d4c/f75 649826 0 2024-03-22T07:15:56.245 INFO:tasks.workunit.client.1.smithi069.stdout:1/164: dread d1/d13/f27 [0,4194304] 0 2024-03-22T07:15:56.245 INFO:tasks.workunit.client.1.smithi069.stdout:1/165: dread - d1/d13/f2d zero size 2024-03-22T07:15:56.246 INFO:tasks.workunit.client.1.smithi069.stdout:1/166: rmdir d1/d13/d16/d1e 39 2024-03-22T07:15:56.246 INFO:tasks.workunit.client.0.smithi063.stdout:1/468: dwrite d4/df/d13/d39/d63/f77 [0,4194304] 0 2024-03-22T07:15:56.247 INFO:tasks.workunit.client.0.smithi063.stdout:1/469: fdatasync d4/df/d13/d39/f82 0 2024-03-22T07:15:56.247 INFO:tasks.workunit.client.0.smithi063.stdout:1/470: chown d4/d14/f18 79389538 1 2024-03-22T07:15:56.247 INFO:tasks.workunit.client.0.smithi063.stdout:1/471: chown d4/df/d13/d39/l9d 994105587 1 2024-03-22T07:15:56.247 INFO:tasks.workunit.client.0.smithi063.stdout:1/472: fsync d4/df/d13/d39/d63/f77 0 2024-03-22T07:15:56.247 INFO:tasks.workunit.client.0.smithi063.stdout:1/473: chown d4/df 97 1 2024-03-22T07:15:56.247 INFO:tasks.workunit.client.0.smithi063.stdout:1/474: truncate d4/df/d13/d2b/d3a/f81 1641036 0 2024-03-22T07:15:56.250 INFO:tasks.workunit.client.1.smithi069.stdout:1/167: symlink d1/d13/d16/d1e/l33 0 2024-03-22T07:15:56.250 INFO:tasks.workunit.client.1.smithi069.stdout:1/168: creat d1/d13/f34 x:0 0 0 2024-03-22T07:15:56.251 INFO:tasks.workunit.client.1.smithi069.stdout:9/131: truncate d2/dd/f26 591636 0 2024-03-22T07:15:56.253 INFO:tasks.workunit.client.0.smithi063.stdout:3/403: dwrite da/d63/d65/f77 [0,4194304] 0 2024-03-22T07:15:56.253 INFO:tasks.workunit.client.0.smithi063.stdout:3/404: getdents da/d63/d65/d83 0 2024-03-22T07:15:56.254 INFO:tasks.workunit.client.0.smithi063.stdout:3/405: chown da/d16/d26/f2a 9783 1 2024-03-22T07:15:56.255 INFO:tasks.workunit.client.1.smithi069.stdout:9/132: symlink d2/dd/l2c 0 2024-03-22T07:15:56.255 INFO:tasks.workunit.client.1.smithi069.stdout:9/133: read d2/dd/f10 [1227347,75292] 0 2024-03-22T07:15:56.257 INFO:tasks.workunit.client.0.smithi063.stdout:0/488: sync 2024-03-22T07:15:56.257 INFO:tasks.workunit.client.0.smithi063.stdout:7/299: sync 2024-03-22T07:15:56.257 INFO:tasks.workunit.client.0.smithi063.stdout:7/300: fdatasync d5/da/ff 0 2024-03-22T07:15:56.258 INFO:tasks.workunit.client.1.smithi069.stdout:1/169: write d1/d13/f27 [3303781,84680] 0 2024-03-22T07:15:56.259 INFO:tasks.workunit.client.0.smithi063.stdout:7/301: mknod d5/da/d5f/d33/d57/c60 0 2024-03-22T07:15:56.260 INFO:tasks.workunit.client.1.smithi069.stdout:9/134: link d2/dd/l11 d2/dd/l2d 0 2024-03-22T07:15:56.260 INFO:tasks.workunit.client.1.smithi069.stdout:9/135: read d2/dd/d18/f20 [2632645,34882] 0 2024-03-22T07:15:56.260 INFO:tasks.workunit.client.0.smithi063.stdout:0/489: mknod da/de/d17/d25/d3c/d43/d76/c9c 0 2024-03-22T07:15:56.262 INFO:tasks.workunit.client.1.smithi069.stdout:1/170: creat d1/f35 x:0 0 0 2024-03-22T07:15:56.262 INFO:tasks.workunit.client.0.smithi063.stdout:7/302: link d5/f18 d5/d14/f61 0 2024-03-22T07:15:56.263 INFO:tasks.workunit.client.1.smithi069.stdout:9/136: mkdir d2/dd/d18/d2e 0 2024-03-22T07:15:56.263 INFO:tasks.workunit.client.1.smithi069.stdout:1/171: write d1/d13/f26 [423389,15398] 0 2024-03-22T07:15:56.264 INFO:tasks.workunit.client.1.smithi069.stdout:9/137: mknod d2/c2f 0 2024-03-22T07:15:56.264 INFO:tasks.workunit.client.0.smithi063.stdout:0/490: creat da/de/d17/d25/d3c/d43/d76/f9d x:0 0 0 2024-03-22T07:15:56.264 INFO:tasks.workunit.client.1.smithi069.stdout:1/172: mkdir d1/d13/d36 0 2024-03-22T07:15:56.265 INFO:tasks.workunit.client.1.smithi069.stdout:9/138: creat d2/dd/d18/d2e/f30 x:0 0 0 2024-03-22T07:15:56.266 INFO:tasks.workunit.client.0.smithi063.stdout:0/491: creat da/de/d17/d4e/f9e x:0 0 0 2024-03-22T07:15:56.267 INFO:tasks.workunit.client.0.smithi063.stdout:0/492: fsync da/de/f62 0 2024-03-22T07:15:56.267 INFO:tasks.workunit.client.0.smithi063.stdout:9/308: dwrite d8/d19/f23 [0,4194304] 0 2024-03-22T07:15:56.268 INFO:tasks.workunit.client.0.smithi063.stdout:9/309: write d8/fa [4179540,130680] 0 2024-03-22T07:15:56.268 INFO:tasks.workunit.client.0.smithi063.stdout:7/303: rmdir d5/d14/d1d 39 2024-03-22T07:15:56.273 INFO:tasks.workunit.client.0.smithi063.stdout:9/310: unlink d8/d19/d41/l55 0 2024-03-22T07:15:56.273 INFO:tasks.workunit.client.0.smithi063.stdout:7/304: rmdir d5/da/d5f 39 2024-03-22T07:15:56.273 INFO:tasks.workunit.client.0.smithi063.stdout:7/305: stat d5/d16/f2b 0 2024-03-22T07:15:56.274 INFO:tasks.workunit.client.0.smithi063.stdout:0/493: rmdir da/d49 39 2024-03-22T07:15:56.275 INFO:tasks.workunit.client.0.smithi063.stdout:9/311: symlink d8/d19/d2b/l5a 0 2024-03-22T07:15:56.277 INFO:tasks.workunit.client.1.smithi069.stdout:4/173: dwrite d6/dc/d1d/d31/f14 [0,4194304] 0 2024-03-22T07:15:56.277 INFO:tasks.workunit.client.0.smithi063.stdout:9/312: unlink d8/d19/d20/d22/d29/f36 0 2024-03-22T07:15:56.277 INFO:tasks.workunit.client.0.smithi063.stdout:9/313: creat d8/d19/d20/d22/d29/d31/d4b/f5b x:0 0 0 2024-03-22T07:15:56.278 INFO:tasks.workunit.client.0.smithi063.stdout:6/445: dwrite d3/d6/f3e [0,4194304] 0 2024-03-22T07:15:56.279 INFO:tasks.workunit.client.1.smithi069.stdout:4/174: symlink d6/dc/d23/l34 0 2024-03-22T07:15:56.279 INFO:tasks.workunit.client.1.smithi069.stdout:4/175: readlink d6/l8 0 2024-03-22T07:15:56.281 INFO:tasks.workunit.client.0.smithi063.stdout:6/446: mkdir d3/d15/d95 0 2024-03-22T07:15:56.281 INFO:tasks.workunit.client.0.smithi063.stdout:6/447: truncate d3/d15/d16/d60/d6d/d4c/f6f 301685 0 2024-03-22T07:15:56.281 INFO:tasks.workunit.client.0.smithi063.stdout:6/448: chown d3/d15/d16/d60/d6d/c2b 0 1 2024-03-22T07:15:56.281 INFO:tasks.workunit.client.0.smithi063.stdout:6/449: write d3/d6/f28 [4803690,7436] 0 2024-03-22T07:15:56.281 INFO:tasks.workunit.client.1.smithi069.stdout:9/139: truncate d2/dd/f26 1588134 0 2024-03-22T07:15:56.284 INFO:tasks.workunit.client.1.smithi069.stdout:9/140: mkdir d2/d31 0 2024-03-22T07:15:56.286 INFO:tasks.workunit.client.0.smithi063.stdout:7/306: write d5/d16/d1b/d21/f2a [1298606,107506] 0 2024-03-22T07:15:56.288 INFO:tasks.workunit.client.1.smithi069.stdout:1/173: fsync d1/f35 0 2024-03-22T07:15:56.289 INFO:tasks.workunit.client.0.smithi063.stdout:0/494: write da/de/d17/d25/d3c/d64/d78/f85 [2922865,45259] 0 2024-03-22T07:15:56.289 INFO:tasks.workunit.client.0.smithi063.stdout:3/406: dwrite da/d16/f6d [0,4194304] 0 2024-03-22T07:15:56.290 INFO:tasks.workunit.client.1.smithi069.stdout:1/174: mkdir d1/d13/d37 0 2024-03-22T07:15:56.290 INFO:tasks.workunit.client.1.smithi069.stdout:1/175: truncate d1/f35 801721 0 2024-03-22T07:15:56.290 INFO:tasks.workunit.client.1.smithi069.stdout:1/176: creat d1/d25/f38 x:0 0 0 2024-03-22T07:15:56.294 INFO:tasks.workunit.client.0.smithi063.stdout:7/307: mkdir d5/d16/d1b/d21/d62 0 2024-03-22T07:15:56.294 INFO:tasks.workunit.client.0.smithi063.stdout:7/308: readlink d5/d16/l4c 0 2024-03-22T07:15:56.295 INFO:tasks.workunit.client.1.smithi069.stdout:1/177: dread d1/d13/f17 [0,4194304] 0 2024-03-22T07:15:56.299 INFO:tasks.workunit.client.0.smithi063.stdout:3/407: mkdir da/d63/d65/d86 0 2024-03-22T07:15:56.300 INFO:tasks.workunit.client.0.smithi063.stdout:0/495: write da/de/f23 [5781352,97128] 0 2024-03-22T07:15:56.300 INFO:tasks.workunit.client.0.smithi063.stdout:9/314: write d8/d19/d41/f4a [2632181,76774] 0 2024-03-22T07:15:56.302 INFO:tasks.workunit.client.0.smithi063.stdout:3/408: link da/d16/l45 da/d16/d3f/l87 0 2024-03-22T07:15:56.304 INFO:tasks.workunit.client.1.smithi069.stdout:1/178: dread d1/fa [0,4194304] 0 2024-03-22T07:15:56.307 INFO:tasks.workunit.client.1.smithi069.stdout:1/179: write d1/f22 [977008,115048] 0 2024-03-22T07:15:56.307 INFO:tasks.workunit.client.1.smithi069.stdout:1/180: creat d1/d13/f39 x:0 0 0 2024-03-22T07:15:56.308 INFO:tasks.workunit.client.0.smithi063.stdout:9/315: rename d8/d19/d20/d22/d29/d31/d4b/f5b to d8/d17/d1e/f5c 0 2024-03-22T07:15:56.308 INFO:tasks.workunit.client.0.smithi063.stdout:9/316: write d8/d19/d20/d22/d29/d31/f4e [532708,72747] 0 2024-03-22T07:15:56.308 INFO:tasks.workunit.client.0.smithi063.stdout:9/317: dread - d8/d17/d1e/f5c zero size 2024-03-22T07:15:56.308 INFO:tasks.workunit.client.0.smithi063.stdout:9/318: write d8/d15/f1c [3186433,95979] 0 2024-03-22T07:15:56.308 INFO:tasks.workunit.client.0.smithi063.stdout:7/309: dread d5/d14/d1d/d29/f36 [0,4194304] 0 2024-03-22T07:15:56.309 INFO:tasks.workunit.client.0.smithi063.stdout:9/319: stat d8/d17/d1e/l1f 0 2024-03-22T07:15:56.310 INFO:tasks.workunit.client.1.smithi069.stdout:1/181: rename d1/l29 to d1/d2a/l3a 0 2024-03-22T07:15:56.310 INFO:tasks.workunit.client.1.smithi069.stdout:1/182: creat d1/f3b x:0 0 0 2024-03-22T07:15:56.311 INFO:tasks.workunit.client.0.smithi063.stdout:7/310: creat d5/d16/f63 x:0 0 0 2024-03-22T07:15:56.313 INFO:tasks.workunit.client.0.smithi063.stdout:7/311: creat d5/d14/d1d/d29/d3b/f64 x:0 0 0 2024-03-22T07:15:56.314 INFO:tasks.workunit.client.0.smithi063.stdout:9/320: getdents d8/d19/d20/d22/d29 0 2024-03-22T07:15:56.314 INFO:tasks.workunit.client.0.smithi063.stdout:9/321: chown d8/d15/f46 8632757 1 2024-03-22T07:15:56.315 INFO:tasks.workunit.client.0.smithi063.stdout:7/312: rmdir d5/d14/d1d/d40 39 2024-03-22T07:15:56.316 INFO:tasks.workunit.client.0.smithi063.stdout:7/313: truncate d5/da/d5f/f5c 326965 0 2024-03-22T07:15:56.316 INFO:tasks.workunit.client.0.smithi063.stdout:7/314: chown d5/d16/d1b/c1c 923957 1 2024-03-22T07:15:56.317 INFO:tasks.workunit.client.0.smithi063.stdout:7/315: link d5/d14/d1d/l20 d5/d16/d43/l65 0 2024-03-22T07:15:56.318 INFO:tasks.workunit.client.1.smithi069.stdout:9/141: dwrite d2/f5 [0,4194304] 0 2024-03-22T07:15:56.318 INFO:tasks.workunit.client.1.smithi069.stdout:9/142: creat d2/dd/d18/d2e/f32 x:0 0 0 2024-03-22T07:15:56.320 INFO:tasks.workunit.client.1.smithi069.stdout:1/183: fdatasync d1/f22 0 2024-03-22T07:15:56.321 INFO:tasks.workunit.client.1.smithi069.stdout:1/184: unlink d1/d25/f31 0 2024-03-22T07:15:56.322 INFO:tasks.workunit.client.1.smithi069.stdout:4/176: dwrite d6/f30 [0,4194304] 0 2024-03-22T07:15:56.325 INFO:tasks.workunit.client.1.smithi069.stdout:1/185: mknod d1/c3c 0 2024-03-22T07:15:56.326 INFO:tasks.workunit.client.1.smithi069.stdout:9/143: dread d2/dd/f10 [0,4194304] 0 2024-03-22T07:15:56.328 INFO:tasks.workunit.client.1.smithi069.stdout:1/186: symlink d1/d25/l3d 0 2024-03-22T07:15:56.328 INFO:tasks.workunit.client.1.smithi069.stdout:9/144: write d2/dd/f26 [849737,91544] 0 2024-03-22T07:15:56.332 INFO:tasks.workunit.client.1.smithi069.stdout:1/187: rename d1/d13/d16/d1e/f20 to d1/d13/d36/f3e 0 2024-03-22T07:15:56.332 INFO:tasks.workunit.client.1.smithi069.stdout:1/188: truncate d1/d13/f1b 845737 0 2024-03-22T07:15:56.333 INFO:tasks.workunit.client.1.smithi069.stdout:1/189: link d1/d25/c30 d1/d25/c3f 0 2024-03-22T07:15:56.333 INFO:tasks.workunit.client.1.smithi069.stdout:1/190: fsync d1/d2a/f32 0 2024-03-22T07:15:56.335 INFO:tasks.workunit.client.0.smithi063.stdout:0/496: dwrite da/f54 [0,4194304] 0 2024-03-22T07:15:56.335 INFO:tasks.workunit.client.0.smithi063.stdout:9/322: dread d8/d17/f26 [0,4194304] 0 2024-03-22T07:15:56.335 INFO:tasks.workunit.client.0.smithi063.stdout:0/497: chown da/ff 6 1 2024-03-22T07:15:56.339 INFO:tasks.workunit.client.0.smithi063.stdout:3/409: dwrite da/f71 [0,4194304] 0 2024-03-22T07:15:56.340 INFO:tasks.workunit.client.0.smithi063.stdout:9/323: mkdir d8/d19/d20/d22/d5d 0 2024-03-22T07:15:56.342 INFO:tasks.workunit.client.0.smithi063.stdout:9/324: truncate d8/f14 3154065 0 2024-03-22T07:15:56.344 INFO:tasks.workunit.client.0.smithi063.stdout:9/325: symlink d8/d15/l5e 0 2024-03-22T07:15:56.346 INFO:tasks.workunit.client.0.smithi063.stdout:9/326: creat d8/d19/d2b/f5f x:0 0 0 2024-03-22T07:15:56.346 INFO:tasks.workunit.client.0.smithi063.stdout:3/410: dread da/fb [0,4194304] 0 2024-03-22T07:15:56.346 INFO:tasks.workunit.client.0.smithi063.stdout:3/411: dread - da/d16/f2d zero size 2024-03-22T07:15:56.347 INFO:tasks.workunit.client.0.smithi063.stdout:9/327: stat d8/f54 0 2024-03-22T07:15:56.349 INFO:tasks.workunit.client.0.smithi063.stdout:3/412: unlink da/f3a 0 2024-03-22T07:15:56.349 INFO:tasks.workunit.client.0.smithi063.stdout:3/413: fsync da/d16/f5c 0 2024-03-22T07:15:56.352 INFO:tasks.workunit.client.0.smithi063.stdout:9/328: link d8/c12 d8/d19/d20/d22/d29/d31/d37/c60 0 2024-03-22T07:15:56.353 INFO:tasks.workunit.client.0.smithi063.stdout:9/329: mknod d8/d17/d1e/c61 0 2024-03-22T07:15:56.353 INFO:tasks.workunit.client.0.smithi063.stdout:9/330: write d8/d19/d41/f4a [5136661,14309] 0 2024-03-22T07:15:56.354 INFO:tasks.workunit.client.0.smithi063.stdout:9/331: dread - d8/d19/d20/d22/d29/d31/d37/f59 zero size 2024-03-22T07:15:56.364 INFO:tasks.workunit.client.0.smithi063.stdout:3/414: dread da/d16/f6d [0,4194304] 0 2024-03-22T07:15:56.364 INFO:tasks.workunit.client.0.smithi063.stdout:3/415: readlink da/d16/d3f/l87 0 2024-03-22T07:15:56.365 INFO:tasks.workunit.client.0.smithi063.stdout:7/316: dwrite d5/f9 [4194304,4194304] 0 2024-03-22T07:15:56.366 INFO:tasks.workunit.client.0.smithi063.stdout:3/416: unlink da/d16/d3f/f56 0 2024-03-22T07:15:56.367 INFO:tasks.workunit.client.0.smithi063.stdout:0/498: dwrite da/de/d17/d25/f6a [0,4194304] 0 2024-03-22T07:15:56.367 INFO:tasks.workunit.client.1.smithi069.stdout:1/191: dwrite d1/d13/f34 [0,4194304] 0 2024-03-22T07:15:56.368 INFO:tasks.workunit.client.1.smithi069.stdout:4/177: dwrite d6/dc/f1a [0,4194304] 0 2024-03-22T07:15:56.370 INFO:tasks.workunit.client.1.smithi069.stdout:9/145: dwrite d2/dd/d18/f20 [4194304,4194304] 0 2024-03-22T07:15:56.370 INFO:tasks.workunit.client.1.smithi069.stdout:9/146: dread - d2/dd/d18/d2e/f32 zero size 2024-03-22T07:15:56.371 INFO:tasks.workunit.client.1.smithi069.stdout:1/192: unlink d1/l10 0 2024-03-22T07:15:56.374 INFO:tasks.workunit.client.1.smithi069.stdout:4/178: dread f5 [0,4194304] 0 2024-03-22T07:15:56.374 INFO:tasks.workunit.client.1.smithi069.stdout:4/179: chown d6/l8 6 1 2024-03-22T07:15:56.374 INFO:tasks.workunit.client.1.smithi069.stdout:4/180: chown d6/d12/d18 324 1 2024-03-22T07:15:56.379 INFO:tasks.workunit.client.1.smithi069.stdout:4/181: link d6/d12/c2e d6/dc/d1d/c35 0 2024-03-22T07:15:56.379 INFO:tasks.workunit.client.1.smithi069.stdout:4/182: dread - d6/d12/d18/d21/f2f zero size 2024-03-22T07:15:56.380 INFO:tasks.workunit.client.1.smithi069.stdout:4/183: dread d6/dc/d1d/d31/f17 [0,4194304] 0 2024-03-22T07:15:56.380 INFO:tasks.workunit.client.1.smithi069.stdout:4/184: write d6/f9 [1646158,128536] 0 2024-03-22T07:15:56.380 INFO:tasks.workunit.client.1.smithi069.stdout:9/147: getdents d2/dd/d18/d2e 0 2024-03-22T07:15:56.381 INFO:tasks.workunit.client.0.smithi063.stdout:0/499: dread da/de/f23 [0,4194304] 0 2024-03-22T07:15:56.381 INFO:tasks.workunit.client.0.smithi063.stdout:0/500: write da/f82 [127568,29195] 0 2024-03-22T07:15:56.383 INFO:tasks.workunit.client.1.smithi069.stdout:4/185: rename d6/dc/d23/l27 to d6/dc/d1d/d31/d1f/l36 0 2024-03-22T07:15:56.384 INFO:tasks.workunit.client.1.smithi069.stdout:4/186: readlink d6/d12/d18/l19 0 2024-03-22T07:15:56.384 INFO:tasks.workunit.client.1.smithi069.stdout:4/187: dread - d6/d12/f33 zero size 2024-03-22T07:15:56.387 INFO:tasks.workunit.client.1.smithi069.stdout:9/148: truncate d2/f5 3350335 0 2024-03-22T07:15:56.388 INFO:tasks.workunit.client.1.smithi069.stdout:1/193: dwrite d1/d25/f38 [0,4194304] 0 2024-03-22T07:15:56.388 INFO:tasks.workunit.client.1.smithi069.stdout:9/149: unlink d2/dd/d18/c19 0 2024-03-22T07:15:56.392 INFO:tasks.workunit.client.0.smithi063.stdout:3/417: mkdir da/d16/d26/d69/d88 0 2024-03-22T07:15:56.394 INFO:tasks.workunit.client.1.smithi069.stdout:1/194: rename d1/d13/f34 to d1/d13/d37/f40 0 2024-03-22T07:15:56.396 INFO:tasks.workunit.client.0.smithi063.stdout:0/501: creat da/de/d60/d9b/f9f x:0 0 0 2024-03-22T07:15:56.406 INFO:tasks.workunit.client.0.smithi063.stdout:8/430: sync 2024-03-22T07:15:56.406 INFO:tasks.workunit.client.0.smithi063.stdout:5/373: sync 2024-03-22T07:15:56.406 INFO:tasks.workunit.client.0.smithi063.stdout:4/342: sync 2024-03-22T07:15:56.406 INFO:tasks.workunit.client.0.smithi063.stdout:2/377: sync 2024-03-22T07:15:56.406 INFO:tasks.workunit.client.0.smithi063.stdout:4/343: fsync d1/d5/d43/f35 0 2024-03-22T07:15:56.415 INFO:tasks.workunit.client.0.smithi063.stdout:0/502: write da/de/f31 [7681322,43735] 0 2024-03-22T07:15:56.415 INFO:tasks.workunit.client.0.smithi063.stdout:0/503: fdatasync da/de/d17/d25/d3c/d43/d80/f86 0 2024-03-22T07:15:56.419 INFO:tasks.workunit.client.0.smithi063.stdout:9/332: dread d8/f14 [0,4194304] 0 2024-03-22T07:15:56.420 INFO:tasks.workunit.client.0.smithi063.stdout:9/333: symlink d8/d17/d1e/l62 0 2024-03-22T07:15:56.421 INFO:tasks.workunit.client.1.smithi069.stdout:8/131: sync 2024-03-22T07:15:56.421 INFO:tasks.workunit.client.1.smithi069.stdout:5/77: sync 2024-03-22T07:15:56.423 INFO:tasks.workunit.client.1.smithi069.stdout:5/78: mknod d1/d10/c16 0 2024-03-22T07:15:56.425 INFO:tasks.workunit.client.1.smithi069.stdout:5/79: creat d1/d7/f17 x:0 0 0 2024-03-22T07:15:56.425 INFO:tasks.workunit.client.1.smithi069.stdout:5/80: write d1/d7/f17 [783590,113149] 0 2024-03-22T07:15:56.425 INFO:tasks.workunit.client.1.smithi069.stdout:5/81: creat d1/d7/d8/f18 x:0 0 0 2024-03-22T07:15:56.428 INFO:tasks.workunit.client.0.smithi063.stdout:2/378: dread d0/de/d12/d20/f3f [0,4194304] 0 2024-03-22T07:15:56.428 INFO:tasks.workunit.client.0.smithi063.stdout:2/379: read - d0/f51 zero size 2024-03-22T07:15:56.428 INFO:tasks.workunit.client.0.smithi063.stdout:2/380: write d0/f51 [1008728,96687] 0 2024-03-22T07:15:56.428 INFO:tasks.workunit.client.0.smithi063.stdout:2/381: readlink d0/de/d35/l3c 0 2024-03-22T07:15:56.436 INFO:tasks.workunit.client.1.smithi069.stdout:8/132: dread d1/de/d12/f19 [0,4194304] 0 2024-03-22T07:15:56.436 INFO:tasks.workunit.client.1.smithi069.stdout:8/133: fsync d1/de/d20/f26 0 2024-03-22T07:15:56.436 INFO:tasks.workunit.client.1.smithi069.stdout:8/134: read - d1/d1c/f22 zero size 2024-03-22T07:15:56.438 INFO:tasks.workunit.client.1.smithi069.stdout:8/135: dread d1/fc [0,4194304] 0 2024-03-22T07:15:56.439 INFO:tasks.workunit.client.1.smithi069.stdout:8/136: link d1/de/f1a d1/d1c/f27 0 2024-03-22T07:15:56.441 INFO:tasks.workunit.client.1.smithi069.stdout:8/137: symlink d1/de/d20/l28 0 2024-03-22T07:15:56.441 INFO:tasks.workunit.client.1.smithi069.stdout:9/150: dwrite d2/dd/d18/d2e/f30 [0,4194304] 0 2024-03-22T07:15:56.441 INFO:tasks.workunit.client.1.smithi069.stdout:9/151: fsync d2/dd/d18/d2e/f30 0 2024-03-22T07:15:56.441 INFO:tasks.workunit.client.1.smithi069.stdout:9/152: fsync d2/dd/d18/d2e/f30 0 2024-03-22T07:15:56.442 INFO:tasks.workunit.client.1.smithi069.stdout:4/188: dwrite d6/dc/f1a [0,4194304] 0 2024-03-22T07:15:56.442 INFO:tasks.workunit.client.1.smithi069.stdout:4/189: read d6/dc/d1d/d31/f17 [153604,45705] 0 2024-03-22T07:15:56.444 INFO:tasks.workunit.client.1.smithi069.stdout:4/190: link d6/d12/d18/d21/f2f d6/dc/f37 0 2024-03-22T07:15:56.444 INFO:tasks.workunit.client.1.smithi069.stdout:4/191: chown d6/d12/d18/c28 219607612 1 2024-03-22T07:15:56.445 INFO:tasks.workunit.client.1.smithi069.stdout:4/192: mknod d6/d12/d18/d21/c38 0 2024-03-22T07:15:56.446 INFO:tasks.workunit.client.1.smithi069.stdout:4/193: stat d6/dc/d1d/c35 0 2024-03-22T07:15:56.446 INFO:tasks.workunit.client.1.smithi069.stdout:4/194: truncate f5 5084454 0 2024-03-22T07:15:56.449 INFO:tasks.workunit.client.1.smithi069.stdout:8/138: dread d1/de/f23 [0,4194304] 0 2024-03-22T07:15:56.450 INFO:tasks.workunit.client.1.smithi069.stdout:8/139: link d1/d1c/f27 d1/f29 0 2024-03-22T07:15:56.466 INFO:tasks.workunit.client.1.smithi069.stdout:4/195: write d6/f9 [4680644,114081] 0 2024-03-22T07:15:56.466 INFO:tasks.workunit.client.1.smithi069.stdout:1/195: dwrite d1/d13/d37/f40 [4194304,4194304] 0 2024-03-22T07:15:56.473 INFO:tasks.workunit.client.1.smithi069.stdout:1/196: dread d1/d13/f27 [0,4194304] 0 2024-03-22T07:15:56.476 INFO:tasks.workunit.client.1.smithi069.stdout:1/197: rename d1/d13/d16 to d1/d25/d41 0 2024-03-22T07:15:56.476 INFO:tasks.workunit.client.1.smithi069.stdout:1/198: fsync d1/f22 0 2024-03-22T07:15:56.476 INFO:tasks.workunit.client.1.smithi069.stdout:1/199: write d1/d13/f2d [784898,44337] 0 2024-03-22T07:15:56.476 INFO:tasks.workunit.client.1.smithi069.stdout:1/200: write d1/d13/f2d [1035477,56070] 0 2024-03-22T07:15:56.476 INFO:tasks.workunit.client.1.smithi069.stdout:1/201: read d1/fa [2870545,55608] 0 2024-03-22T07:15:56.477 INFO:tasks.workunit.client.1.smithi069.stdout:1/202: creat d1/f42 x:0 0 0 2024-03-22T07:15:56.477 INFO:tasks.workunit.client.1.smithi069.stdout:1/203: unlink d1/d25/f28 0 2024-03-22T07:15:56.478 INFO:tasks.workunit.client.1.smithi069.stdout:1/204: read d1/f35 [284499,31346] 0 2024-03-22T07:15:56.479 INFO:tasks.workunit.client.0.smithi063.stdout:5/374: dwrite d8/d13/d1e/d3c/f49 [0,4194304] 0 2024-03-22T07:15:56.479 INFO:tasks.workunit.client.0.smithi063.stdout:8/431: dwrite d0/d1b/f6e [0,4194304] 0 2024-03-22T07:15:56.479 INFO:tasks.workunit.client.0.smithi063.stdout:8/432: truncate d0/d1b/f69 516558 0 2024-03-22T07:15:56.479 INFO:tasks.workunit.client.1.smithi069.stdout:1/205: creat d1/d13/d36/f43 x:0 0 0 2024-03-22T07:15:56.480 INFO:tasks.workunit.client.0.smithi063.stdout:2/382: dwrite d0/de/d12/d20/f70 [0,4194304] 0 2024-03-22T07:15:56.480 INFO:tasks.workunit.client.0.smithi063.stdout:0/504: dwrite da/de/d17/d4e/f9e [0,4194304] 0 2024-03-22T07:15:56.480 INFO:tasks.workunit.client.0.smithi063.stdout:9/334: dwrite d8/d19/d2b/f5f [0,4194304] 0 2024-03-22T07:15:56.481 INFO:tasks.workunit.client.0.smithi063.stdout:5/375: creat d8/d13/d1e/d31/d64/d65/f6c x:0 0 0 2024-03-22T07:15:56.481 INFO:tasks.workunit.client.0.smithi063.stdout:5/376: fdatasync d8/d13/d1e/d31/f4e 0 2024-03-22T07:15:56.481 INFO:tasks.workunit.client.1.smithi069.stdout:1/206: link d1/d25/d41/d1e/l33 d1/d13/d37/l44 0 2024-03-22T07:15:56.481 INFO:tasks.workunit.client.1.smithi069.stdout:1/207: write d1/d13/f2d [1275518,14695] 0 2024-03-22T07:15:56.483 INFO:tasks.workunit.client.0.smithi063.stdout:8/433: mkdir d0/d26/d2d/d83 0 2024-03-22T07:15:56.484 INFO:tasks.workunit.client.1.smithi069.stdout:8/140: dwrite d1/d1c/f22 [0,4194304] 0 2024-03-22T07:15:56.484 INFO:tasks.workunit.client.1.smithi069.stdout:8/141: dread - d1/de/f17 zero size 2024-03-22T07:15:56.484 INFO:tasks.workunit.client.1.smithi069.stdout:8/142: write d1/fd [4729574,33170] 0 2024-03-22T07:15:56.484 INFO:tasks.workunit.client.1.smithi069.stdout:1/208: truncate d1/fb 762915 0 2024-03-22T07:15:56.484 INFO:tasks.workunit.client.1.smithi069.stdout:1/209: creat d1/d13/f45 x:0 0 0 2024-03-22T07:15:56.486 INFO:tasks.workunit.client.1.smithi069.stdout:4/196: dwrite d6/dc/f37 [0,4194304] 0 2024-03-22T07:15:56.486 INFO:tasks.workunit.client.0.smithi063.stdout:0/505: link da/c1e da/de/d17/d4e/ca0 0 2024-03-22T07:15:56.488 INFO:tasks.workunit.client.1.smithi069.stdout:8/143: stat d1/de/l15 0 2024-03-22T07:15:56.489 INFO:tasks.workunit.client.0.smithi063.stdout:9/335: mkdir d8/d15/d63 0 2024-03-22T07:15:56.489 INFO:tasks.workunit.client.0.smithi063.stdout:9/336: fsync d8/d15/f46 0 2024-03-22T07:15:56.489 INFO:tasks.workunit.client.0.smithi063.stdout:9/337: write d8/d15/d2f/f47 [1563267,55199] 0 2024-03-22T07:15:56.489 INFO:tasks.workunit.client.0.smithi063.stdout:9/338: chown d8/d19/d20/f21 5155 1 2024-03-22T07:15:56.490 INFO:tasks.workunit.client.0.smithi063.stdout:5/377: mknod d8/d13/d1e/d31/c6d 0 2024-03-22T07:15:56.490 INFO:tasks.workunit.client.0.smithi063.stdout:8/434: rmdir d0/d26/d2d 39 2024-03-22T07:15:56.495 INFO:tasks.workunit.client.0.smithi063.stdout:9/339: truncate d8/d19/d20/f34 3917955 0 2024-03-22T07:15:56.496 INFO:tasks.workunit.client.0.smithi063.stdout:8/435: creat d0/d1b/f84 x:0 0 0 2024-03-22T07:15:56.496 INFO:tasks.workunit.client.0.smithi063.stdout:8/436: write d0/d24/d5b/f60 [875178,63098] 0 2024-03-22T07:15:56.498 INFO:tasks.workunit.client.0.smithi063.stdout:5/378: rename d8/lf to d8/l6e 0 2024-03-22T07:15:56.500 INFO:tasks.workunit.client.0.smithi063.stdout:8/437: mknod d0/c85 0 2024-03-22T07:15:56.500 INFO:tasks.workunit.client.0.smithi063.stdout:8/438: chown d0/d1b/c73 23585635 1 2024-03-22T07:15:56.501 INFO:tasks.workunit.client.1.smithi069.stdout:8/144: link d1/de/l1b d1/de/l2a 0 2024-03-22T07:15:56.501 INFO:tasks.workunit.client.1.smithi069.stdout:8/145: write d1/f29 [377305,13306] 0 2024-03-22T07:15:56.503 INFO:tasks.workunit.client.0.smithi063.stdout:8/439: rename d0/l47 to d0/d26/l86 0 2024-03-22T07:15:56.503 INFO:tasks.workunit.client.0.smithi063.stdout:8/440: creat d0/d26/d2d/d59/f87 x:0 0 0 2024-03-22T07:15:56.503 INFO:tasks.workunit.client.1.smithi069.stdout:8/146: rename d1/de/lf to d1/d1c/l2b 0 2024-03-22T07:15:56.504 INFO:tasks.workunit.client.1.smithi069.stdout:8/147: truncate d1/de/d20/f26 203988 0 2024-03-22T07:15:56.504 INFO:tasks.workunit.client.1.smithi069.stdout:8/148: write d1/de/f23 [3382930,73930] 0 2024-03-22T07:15:56.505 INFO:tasks.workunit.client.1.smithi069.stdout:8/149: mknod d1/d1c/c2c 0 2024-03-22T07:15:56.507 INFO:tasks.workunit.client.1.smithi069.stdout:8/150: dread d1/de/d20/f26 [0,4194304] 0 2024-03-22T07:15:56.511 INFO:tasks.workunit.client.1.smithi069.stdout:8/151: rename d1/d1c/f27 to d1/de/f2d 0 2024-03-22T07:15:56.511 INFO:tasks.workunit.client.1.smithi069.stdout:8/152: truncate d1/de/d20/f26 1188719 0 2024-03-22T07:15:56.515 INFO:tasks.workunit.client.0.smithi063.stdout:1/475: sync 2024-03-22T07:15:56.516 INFO:tasks.workunit.client.1.smithi069.stdout:1/210: getdents d1/d25 0 2024-03-22T07:15:56.516 INFO:tasks.workunit.client.0.smithi063.stdout:1/476: link d4/df/d13/d39/f5f d4/df/d13/d39/fa0 0 2024-03-22T07:15:56.518 INFO:tasks.workunit.client.0.smithi063.stdout:1/477: mkdir d4/d71/d88/da1 0 2024-03-22T07:15:56.519 INFO:tasks.workunit.client.0.smithi063.stdout:1/478: symlink d4/d8b/la2 0 2024-03-22T07:15:56.519 INFO:tasks.workunit.client.0.smithi063.stdout:1/479: write d4/d14/f1b [3543484,57528] 0 2024-03-22T07:15:56.520 INFO:tasks.workunit.client.0.smithi063.stdout:1/480: creat d4/df/d13/d2b/d86/fa3 x:0 0 0 2024-03-22T07:15:56.520 INFO:tasks.workunit.client.0.smithi063.stdout:1/481: creat d4/d5a/fa4 x:0 0 0 2024-03-22T07:15:56.522 INFO:tasks.workunit.client.1.smithi069.stdout:4/197: dwrite d6/dc/f1a [0,4194304] 0 2024-03-22T07:15:56.522 INFO:tasks.workunit.client.0.smithi063.stdout:2/383: dwrite d0/de/d58/f7b [0,4194304] 0 2024-03-22T07:15:56.525 INFO:tasks.workunit.client.0.smithi063.stdout:2/384: truncate d0/fb 2712597 0 2024-03-22T07:15:56.525 INFO:tasks.workunit.client.0.smithi063.stdout:2/385: chown d0/c79 1 1 2024-03-22T07:15:56.526 INFO:tasks.workunit.client.1.smithi069.stdout:6/112: sync 2024-03-22T07:15:56.527 INFO:tasks.workunit.client.0.smithi063.stdout:0/506: dwrite da/de/f5c [0,4194304] 0 2024-03-22T07:15:56.529 INFO:tasks.workunit.client.1.smithi069.stdout:6/113: creat d4/d7/f26 x:0 0 0 2024-03-22T07:15:56.529 INFO:tasks.workunit.client.1.smithi069.stdout:6/114: fdatasync d4/fb 0 2024-03-22T07:15:56.531 INFO:tasks.workunit.client.0.smithi063.stdout:5/379: dwrite d8/d13/d1e/f2f [0,4194304] 0 2024-03-22T07:15:56.531 INFO:tasks.workunit.client.0.smithi063.stdout:5/380: dread - d8/d13/f5d zero size 2024-03-22T07:15:56.531 INFO:tasks.workunit.client.0.smithi063.stdout:5/381: read d8/d13/d24/f33 [5905162,100297] 0 2024-03-22T07:15:56.533 INFO:tasks.workunit.client.0.smithi063.stdout:5/382: mkdir d8/d13/d1e/d31/d64/d65/d6f 0 2024-03-22T07:15:56.534 INFO:tasks.workunit.client.0.smithi063.stdout:5/383: dread d8/d13/d1e/d31/f4e [0,4194304] 0 2024-03-22T07:15:56.534 INFO:tasks.workunit.client.0.smithi063.stdout:5/384: chown d8/d13/d28/c3f 252389 1 2024-03-22T07:15:56.536 INFO:tasks.workunit.client.0.smithi063.stdout:5/385: unlink d8/cd 0 2024-03-22T07:15:56.536 INFO:tasks.workunit.client.0.smithi063.stdout:5/386: write d8/d13/d1e/f27 [2300503,42975] 0 2024-03-22T07:15:56.537 INFO:tasks.workunit.client.0.smithi063.stdout:5/387: mknod d8/d13/d1e/d3c/c70 0 2024-03-22T07:15:56.537 INFO:tasks.workunit.client.0.smithi063.stdout:5/388: readlink d8/l2c 0 2024-03-22T07:15:56.537 INFO:tasks.workunit.client.0.smithi063.stdout:5/389: fdatasync d8/d13/d1e/d3c/f49 0 2024-03-22T07:15:56.537 INFO:tasks.workunit.client.0.smithi063.stdout:5/390: creat d8/d13/d1e/d31/d64/d65/f71 x:0 0 0 2024-03-22T07:15:56.537 INFO:tasks.workunit.client.0.smithi063.stdout:5/391: readlink d8/l32 0 2024-03-22T07:15:56.538 INFO:tasks.workunit.client.0.smithi063.stdout:5/392: truncate d8/d13/d1e/f38 2039254 0 2024-03-22T07:15:56.539 INFO:tasks.workunit.client.0.smithi063.stdout:5/393: mknod d8/d13/d28/d57/c72 0 2024-03-22T07:15:56.539 INFO:tasks.workunit.client.0.smithi063.stdout:5/394: fdatasync d8/d13/d1e/d3c/f55 0 2024-03-22T07:15:56.540 INFO:tasks.workunit.client.0.smithi063.stdout:2/386: write d0/de/d58/f7b [929348,125863] 0 2024-03-22T07:15:56.540 INFO:tasks.workunit.client.0.smithi063.stdout:2/387: chown d0/de/d1b/d63/f6c 464079192 1 2024-03-22T07:15:56.540 INFO:tasks.workunit.client.0.smithi063.stdout:5/395: dread d8/d13/d1e/d31/f4e [0,4194304] 0 2024-03-22T07:15:56.541 INFO:tasks.workunit.client.0.smithi063.stdout:2/388: rename d0/l17 to d0/de/d1b/d63/l80 0 2024-03-22T07:15:56.542 INFO:tasks.workunit.client.0.smithi063.stdout:2/389: chown d0/f9 8021347 1 2024-03-22T07:15:56.542 INFO:tasks.workunit.client.0.smithi063.stdout:2/390: fdatasync d0/de/f65 0 2024-03-22T07:15:56.542 INFO:tasks.workunit.client.0.smithi063.stdout:2/391: write d0/de/d12/f27 [2605722,84118] 0 2024-03-22T07:15:56.542 INFO:tasks.workunit.client.0.smithi063.stdout:2/392: dread - d0/de/f38 zero size 2024-03-22T07:15:56.544 INFO:tasks.workunit.client.1.smithi069.stdout:6/115: dread d4/dd/d12/d13/f23 [0,4194304] 0 2024-03-22T07:15:56.544 INFO:tasks.workunit.client.1.smithi069.stdout:6/116: read d4/dd/d12/f1d [45554,86121] 0 2024-03-22T07:15:56.544 INFO:tasks.workunit.client.1.smithi069.stdout:6/117: fsync d4/dd/f25 0 2024-03-22T07:15:56.546 INFO:tasks.workunit.client.0.smithi063.stdout:8/441: dwrite d0/d1f/f5c [0,4194304] 0 2024-03-22T07:15:56.546 INFO:tasks.workunit.client.0.smithi063.stdout:8/442: write d0/d26/d2d/f4d [295662,82370] 0 2024-03-22T07:15:56.547 INFO:tasks.workunit.client.1.smithi069.stdout:6/118: getdents d4/dd 0 2024-03-22T07:15:56.548 INFO:tasks.workunit.client.0.smithi063.stdout:0/507: write da/de/d17/d25/d3c/d5f/f52 [104373,69659] 0 2024-03-22T07:15:56.551 INFO:tasks.workunit.client.0.smithi063.stdout:0/508: getdents da/de/d60 0 2024-03-22T07:15:56.552 INFO:tasks.workunit.client.0.smithi063.stdout:9/340: dwrite d8/fa [0,4194304] 0 2024-03-22T07:15:56.553 INFO:tasks.workunit.client.1.smithi069.stdout:8/153: dwrite d1/fc [0,4194304] 0 2024-03-22T07:15:56.553 INFO:tasks.workunit.client.1.smithi069.stdout:8/154: write d1/de/f17 [867507,42873] 0 2024-03-22T07:15:56.553 INFO:tasks.workunit.client.0.smithi063.stdout:0/509: link da/de/d67/c6b da/de/d17/d25/d95/d98/ca1 0 2024-03-22T07:15:56.553 INFO:tasks.workunit.client.0.smithi063.stdout:0/510: chown da/de/d60/f69 123 1 2024-03-22T07:15:56.553 INFO:tasks.workunit.client.0.smithi063.stdout:0/511: readlink da/de/d17/d25/d3c/d64/d78/l8f 0 2024-03-22T07:15:56.555 INFO:tasks.workunit.client.0.smithi063.stdout:5/396: write d8/f14 [986042,83239] 0 2024-03-22T07:15:56.555 INFO:tasks.workunit.client.0.smithi063.stdout:5/397: stat d8/d13 0 2024-03-22T07:15:56.555 INFO:tasks.workunit.client.0.smithi063.stdout:1/482: dwrite d4/df/d13/d2b/d86/f87 [0,4194304] 0 2024-03-22T07:15:56.555 INFO:tasks.workunit.client.0.smithi063.stdout:9/341: link d8/fa d8/d19/d20/d22/d5d/f64 0 2024-03-22T07:15:56.556 INFO:tasks.workunit.client.1.smithi069.stdout:8/155: symlink d1/de/l2e 0 2024-03-22T07:15:56.556 INFO:tasks.workunit.client.1.smithi069.stdout:8/156: truncate d1/de/d20/f26 2112642 0 2024-03-22T07:15:56.556 INFO:tasks.workunit.client.1.smithi069.stdout:8/157: fdatasync d1/de/d20/f26 0 2024-03-22T07:15:56.556 INFO:tasks.workunit.client.1.smithi069.stdout:8/158: chown d1/de/l2e 26 1 2024-03-22T07:15:56.558 INFO:tasks.workunit.client.1.smithi069.stdout:6/119: dread d4/fb [4194304,4194304] 0 2024-03-22T07:15:56.558 INFO:tasks.workunit.client.0.smithi063.stdout:1/483: creat d4/df/d13/d2b/d6c/d85/d9e/fa5 x:0 0 0 2024-03-22T07:15:56.560 INFO:tasks.workunit.client.1.smithi069.stdout:8/159: unlink d1/de/f17 0 2024-03-22T07:15:56.560 INFO:tasks.workunit.client.0.smithi063.stdout:0/512: dread da/de/d17/d25/d3c/f3f [0,4194304] 0 2024-03-22T07:15:56.560 INFO:tasks.workunit.client.0.smithi063.stdout:0/513: chown da/de/d17/d25/d3c/d64/d78/l8f 3 1 2024-03-22T07:15:56.561 INFO:tasks.workunit.client.0.smithi063.stdout:0/514: creat da/de/d67/fa2 x:0 0 0 2024-03-22T07:15:56.565 INFO:tasks.workunit.client.1.smithi069.stdout:8/160: unlink d1/de/d12/l24 0 2024-03-22T07:15:56.565 INFO:tasks.workunit.client.1.smithi069.stdout:8/161: chown d1/l7 248959 1 2024-03-22T07:15:56.565 INFO:tasks.workunit.client.1.smithi069.stdout:8/162: chown d1/de/f23 1937 1 2024-03-22T07:15:56.566 INFO:tasks.workunit.client.1.smithi069.stdout:6/120: symlink d4/l27 0 2024-03-22T07:15:56.567 INFO:tasks.workunit.client.1.smithi069.stdout:8/163: dread d1/de/f1a [0,4194304] 0 2024-03-22T07:15:56.567 INFO:tasks.workunit.client.1.smithi069.stdout:4/198: truncate d6/dc/f1a 3031039 0 2024-03-22T07:15:56.568 INFO:tasks.workunit.client.0.smithi063.stdout:9/342: dread d8/d19/d20/d22/f27 [0,4194304] 0 2024-03-22T07:15:56.568 INFO:tasks.workunit.client.0.smithi063.stdout:9/343: creat d8/d19/d20/d22/f65 x:0 0 0 2024-03-22T07:15:56.569 INFO:tasks.workunit.client.1.smithi069.stdout:4/199: link d6/f9 d6/dc/d1d/d31/d1f/f39 0 2024-03-22T07:15:56.569 INFO:tasks.workunit.client.1.smithi069.stdout:4/200: write d6/f9 [419952,93261] 0 2024-03-22T07:15:56.569 INFO:tasks.workunit.client.0.smithi063.stdout:1/484: dread f2 [0,4194304] 0 2024-03-22T07:15:56.574 INFO:tasks.workunit.client.1.smithi069.stdout:6/121: read d4/d7/da/f1b [2353851,96816] 0 2024-03-22T07:15:56.574 INFO:tasks.workunit.client.1.smithi069.stdout:6/122: stat d4/dd/d12/c1e 0 2024-03-22T07:15:56.574 INFO:tasks.workunit.client.1.smithi069.stdout:6/123: stat d4/dd/c18 0 2024-03-22T07:15:56.576 INFO:tasks.workunit.client.1.smithi069.stdout:8/164: write d1/d1c/f22 [4190491,67106] 0 2024-03-22T07:15:56.579 INFO:tasks.workunit.client.1.smithi069.stdout:4/201: dread d6/d12/d18/d21/f2f [0,4194304] 0 2024-03-22T07:15:56.580 INFO:tasks.workunit.client.1.smithi069.stdout:4/202: mkdir d6/d16/d3a 0 2024-03-22T07:15:56.580 INFO:tasks.workunit.client.1.smithi069.stdout:4/203: link d6/dc/d1d/d31/c11 d6/d12/d18/c3b 0 2024-03-22T07:15:56.581 INFO:tasks.workunit.client.1.smithi069.stdout:4/204: creat d6/dc/d23/f3c x:0 0 0 2024-03-22T07:15:56.582 INFO:tasks.workunit.client.1.smithi069.stdout:4/205: write d6/dc/d1d/d31/d1f/f39 [4731632,68787] 0 2024-03-22T07:15:56.583 INFO:tasks.workunit.client.1.smithi069.stdout:4/206: symlink d6/d12/d18/l3d 0 2024-03-22T07:15:56.583 INFO:tasks.workunit.client.1.smithi069.stdout:4/207: write d6/dc/d23/f3c [948691,60202] 0 2024-03-22T07:15:56.583 INFO:tasks.workunit.client.1.smithi069.stdout:4/208: mknod d6/dc/c3e 0 2024-03-22T07:15:56.583 INFO:tasks.workunit.client.1.smithi069.stdout:4/209: readlink d6/lb 0 2024-03-22T07:15:56.584 INFO:tasks.workunit.client.1.smithi069.stdout:4/210: mkdir d6/dc/d1d/d31/d1f/d3f 0 2024-03-22T07:15:56.591 INFO:tasks.workunit.client.1.smithi069.stdout:4/211: dread f5 [0,4194304] 0 2024-03-22T07:15:56.591 INFO:tasks.workunit.client.1.smithi069.stdout:4/212: truncate f5 5241395 0 2024-03-22T07:15:56.593 INFO:tasks.workunit.client.1.smithi069.stdout:4/213: truncate d6/dc/f1a 3172829 0 2024-03-22T07:15:56.593 INFO:tasks.workunit.client.1.smithi069.stdout:4/214: rename d6/dc/d1d/d31/l2d to d6/d12/d18/d21/l40 0 2024-03-22T07:15:56.594 INFO:tasks.workunit.client.0.smithi063.stdout:9/344: unlink d8/d17/c38 0 2024-03-22T07:15:56.594 INFO:tasks.workunit.client.0.smithi063.stdout:9/345: stat d8/d17/d1e/l50 0 2024-03-22T07:15:56.594 INFO:tasks.workunit.client.0.smithi063.stdout:2/393: rename d0/fb to d0/de/d12/d5a/f81 0 2024-03-22T07:15:56.594 INFO:tasks.workunit.client.0.smithi063.stdout:2/394: fsync d0/de/d12/d20/f70 0 2024-03-22T07:15:56.594 INFO:tasks.workunit.client.0.smithi063.stdout:2/395: fdatasync d0/de/d12/d20/f3f 0 2024-03-22T07:15:56.594 INFO:tasks.workunit.client.0.smithi063.stdout:9/346: dread d8/d19/d20/d22/d29/d31/f4e [0,4194304] 0 2024-03-22T07:15:56.595 INFO:tasks.workunit.client.0.smithi063.stdout:2/396: creat d0/de/d12/d20/f82 x:0 0 0 2024-03-22T07:15:56.595 INFO:tasks.workunit.client.0.smithi063.stdout:9/347: creat d8/d19/d41/d48/f66 x:0 0 0 2024-03-22T07:15:56.595 INFO:tasks.workunit.client.0.smithi063.stdout:8/443: rename d0/d1b/c37 to d0/c88 0 2024-03-22T07:15:56.595 INFO:tasks.workunit.client.0.smithi063.stdout:8/444: stat d0/d1b/d28/d7b/f7f 0 2024-03-22T07:15:56.595 INFO:tasks.workunit.client.0.smithi063.stdout:6/450: sync 2024-03-22T07:15:56.595 INFO:tasks.workunit.client.0.smithi063.stdout:9/348: mknod d8/d17/c67 0 2024-03-22T07:15:56.596 INFO:tasks.workunit.client.1.smithi069.stdout:4/215: unlink d6/le 0 2024-03-22T07:15:56.596 INFO:tasks.workunit.client.1.smithi069.stdout:4/216: readlink d6/d12/l32 0 2024-03-22T07:15:56.596 INFO:tasks.workunit.client.1.smithi069.stdout:4/217: creat d6/d12/f41 x:0 0 0 2024-03-22T07:15:56.596 INFO:tasks.workunit.client.1.smithi069.stdout:4/218: creat d6/dc/d1d/d31/d1f/f42 x:0 0 0 2024-03-22T07:15:56.596 INFO:tasks.workunit.client.1.smithi069.stdout:8/165: dwrite d1/de/f1a [0,4194304] 0 2024-03-22T07:15:56.597 INFO:tasks.workunit.client.1.smithi069.stdout:4/219: chown d6/dc/d1d/c35 7 1 2024-03-22T07:15:56.597 INFO:tasks.workunit.client.1.smithi069.stdout:8/166: chown d1/de/l1b 22352394 1 2024-03-22T07:15:56.597 INFO:tasks.workunit.client.1.smithi069.stdout:4/220: readlink d6/d12/l32 0 2024-03-22T07:15:56.597 INFO:tasks.workunit.client.1.smithi069.stdout:8/167: write d1/de/d20/f26 [1968078,124523] 0 2024-03-22T07:15:56.597 INFO:tasks.workunit.client.1.smithi069.stdout:8/168: read d1/d1c/f22 [4156678,55234] 0 2024-03-22T07:15:56.597 INFO:tasks.workunit.client.1.smithi069.stdout:4/221: symlink d6/d12/d18/d21/l43 0 2024-03-22T07:15:56.598 INFO:tasks.workunit.client.0.smithi063.stdout:0/515: rename da/de/d17/d25/d3c/d5f/f52 to da/de/d60/d56/d77/fa3 0 2024-03-22T07:15:56.598 INFO:tasks.workunit.client.0.smithi063.stdout:9/349: mknod d8/d17/d1e/c68 0 2024-03-22T07:15:56.598 INFO:tasks.workunit.client.0.smithi063.stdout:9/350: creat d8/d17/f69 x:0 0 0 2024-03-22T07:15:56.599 INFO:tasks.workunit.client.1.smithi069.stdout:8/169: unlink d1/de/d12/c18 0 2024-03-22T07:15:56.601 INFO:tasks.workunit.client.0.smithi063.stdout:6/451: dread d3/d6/fa [0,4194304] 0 2024-03-22T07:15:56.601 INFO:tasks.workunit.client.0.smithi063.stdout:6/452: write d3/f7c [810123,72425] 0 2024-03-22T07:15:56.603 INFO:tasks.workunit.client.0.smithi063.stdout:9/351: mkdir d8/d19/d20/d22/d29/d31/d53/d6a 0 2024-03-22T07:15:56.603 INFO:tasks.workunit.client.1.smithi069.stdout:8/170: rename d1/fc to d1/de/d20/f2f 0 2024-03-22T07:15:56.604 INFO:tasks.workunit.client.0.smithi063.stdout:0/516: write da/de/d17/d25/f68 [485119,46234] 0 2024-03-22T07:15:56.605 INFO:tasks.workunit.client.0.smithi063.stdout:8/445: rename d0/d1b/f69 to d0/f89 0 2024-03-22T07:15:56.605 INFO:tasks.workunit.client.0.smithi063.stdout:8/446: readlink d0/d26/l86 0 2024-03-22T07:15:56.606 INFO:tasks.workunit.client.0.smithi063.stdout:8/447: chown d0/c52 624 1 2024-03-22T07:15:56.606 INFO:tasks.workunit.client.0.smithi063.stdout:6/453: creat d3/f96 x:0 0 0 2024-03-22T07:15:56.612 INFO:tasks.workunit.client.0.smithi063.stdout:8/448: mknod d0/d1b/d28/d7b/d4a/c8a 0 2024-03-22T07:15:56.616 INFO:tasks.workunit.client.0.smithi063.stdout:8/449: mknod d0/d1b/d28/d3e/c8b 0 2024-03-22T07:15:56.617 INFO:tasks.workunit.client.0.smithi063.stdout:8/450: chown d0/l7e 1 1 2024-03-22T07:15:56.617 INFO:tasks.workunit.client.0.smithi063.stdout:6/454: dread d3/d6/d2a/d38/f49 [0,4194304] 0 2024-03-22T07:15:56.619 INFO:tasks.workunit.client.0.smithi063.stdout:8/451: unlink d0/d26/l70 0 2024-03-22T07:15:56.625 INFO:tasks.workunit.client.0.smithi063.stdout:0/517: dwrite da/de/d17/d25/d3c/d5f/f4b [0,4194304] 0 2024-03-22T07:15:56.625 INFO:tasks.workunit.client.0.smithi063.stdout:4/344: sync 2024-03-22T07:15:56.625 INFO:tasks.workunit.client.0.smithi063.stdout:3/418: sync 2024-03-22T07:15:56.625 INFO:tasks.workunit.client.0.smithi063.stdout:7/317: sync 2024-03-22T07:15:56.627 INFO:tasks.workunit.client.0.smithi063.stdout:9/352: dwrite d8/d17/f1d [0,4194304] 0 2024-03-22T07:15:56.630 INFO:tasks.workunit.client.0.smithi063.stdout:0/518: rename da/de/d39/f41 to da/de/d17/d25/d3c/d5f/d97/fa4 0 2024-03-22T07:15:56.632 INFO:tasks.workunit.client.0.smithi063.stdout:3/419: mkdir da/d16/d89 0 2024-03-22T07:15:56.633 INFO:tasks.workunit.client.0.smithi063.stdout:7/318: getdents d5/d16/d43 0 2024-03-22T07:15:56.633 INFO:tasks.workunit.client.0.smithi063.stdout:7/319: write d5/da/f25 [1331820,73401] 0 2024-03-22T07:15:56.641 INFO:tasks.workunit.client.0.smithi063.stdout:0/519: unlink da/de/d60/d56/c7f 0 2024-03-22T07:15:56.641 INFO:tasks.workunit.client.0.smithi063.stdout:0/520: creat da/de/d60/d9b/fa5 x:0 0 0 2024-03-22T07:15:56.642 INFO:tasks.workunit.client.0.smithi063.stdout:7/320: symlink d5/da/d4e/l66 0 2024-03-22T07:15:56.642 INFO:tasks.workunit.client.0.smithi063.stdout:7/321: chown d5/f17 40921 1 2024-03-22T07:15:56.648 INFO:tasks.workunit.client.0.smithi063.stdout:9/353: dread d8/fd [0,4194304] 0 2024-03-22T07:15:56.648 INFO:tasks.workunit.client.0.smithi063.stdout:0/521: rename da/de/d67/fa2 to da/de/d60/d9b/fa6 0 2024-03-22T07:15:56.651 INFO:tasks.workunit.client.0.smithi063.stdout:0/522: symlink da/de/d60/d56/la7 0 2024-03-22T07:15:56.652 INFO:tasks.workunit.client.0.smithi063.stdout:8/452: dwrite d0/d1b/d28/f43 [0,4194304] 0 2024-03-22T07:15:56.653 INFO:tasks.workunit.client.0.smithi063.stdout:0/523: symlink da/de/d60/la8 0 2024-03-22T07:15:56.655 INFO:tasks.workunit.client.0.smithi063.stdout:4/345: dwrite d1/d5/d43/f14 [0,4194304] 0 2024-03-22T07:15:56.655 INFO:tasks.workunit.client.0.smithi063.stdout:3/420: dwrite da/d63/d65/f77 [4194304,4194304] 0 2024-03-22T07:15:56.663 INFO:tasks.workunit.client.0.smithi063.stdout:3/421: truncate da/d16/d3f/f4c 689118 0 2024-03-22T07:15:56.665 INFO:tasks.workunit.client.0.smithi063.stdout:3/422: symlink da/d63/d65/d83/l8a 0 2024-03-22T07:15:56.671 INFO:tasks.workunit.client.0.smithi063.stdout:0/524: read da/de/d60/f69 [2988901,11447] 0 2024-03-22T07:15:56.671 INFO:tasks.workunit.client.0.smithi063.stdout:7/322: dwrite d5/da/f11 [0,4194304] 0 2024-03-22T07:15:56.674 INFO:tasks.workunit.client.0.smithi063.stdout:7/323: creat d5/da/d5f/d33/d57/f67 x:0 0 0 2024-03-22T07:15:56.680 INFO:tasks.workunit.client.0.smithi063.stdout:0/525: dread da/de/f24 [0,4194304] 0 2024-03-22T07:15:56.685 INFO:tasks.workunit.client.0.smithi063.stdout:9/354: dwrite d8/d19/d20/f34 [0,4194304] 0 2024-03-22T07:15:56.685 INFO:tasks.workunit.client.0.smithi063.stdout:9/355: creat d8/d19/d41/f6b x:0 0 0 2024-03-22T07:15:56.688 INFO:tasks.workunit.client.0.smithi063.stdout:0/526: getdents da/de/d60 0 2024-03-22T07:15:56.689 INFO:tasks.workunit.client.0.smithi063.stdout:0/527: truncate da/f54 226927 0 2024-03-22T07:15:56.690 INFO:tasks.workunit.client.0.smithi063.stdout:0/528: mknod da/de/d17/d25/d3c/d43/d76/d92/ca9 0 2024-03-22T07:15:56.691 INFO:tasks.workunit.client.0.smithi063.stdout:8/453: dwrite d0/d1b/d28/d7b/d45/f46 [0,4194304] 0 2024-03-22T07:15:56.692 INFO:tasks.workunit.client.0.smithi063.stdout:8/454: chown d0/f22 487 1 2024-03-22T07:15:56.692 INFO:tasks.workunit.client.0.smithi063.stdout:4/346: dwrite d1/d5/d43/d27/d2c/d4a/f4d [0,4194304] 0 2024-03-22T07:15:56.692 INFO:tasks.workunit.client.0.smithi063.stdout:4/347: fdatasync d1/d5/d43/d27/d2c/d3d/d54/f6b 0 2024-03-22T07:15:56.694 INFO:tasks.workunit.client.0.smithi063.stdout:8/455: symlink d0/d24/d33/d4c/l8c 0 2024-03-22T07:15:56.694 INFO:tasks.workunit.client.0.smithi063.stdout:7/324: dwrite d5/d16/f2e [0,4194304] 0 2024-03-22T07:15:56.694 INFO:tasks.workunit.client.0.smithi063.stdout:3/423: dwrite da/d16/f42 [4194304,4194304] 0 2024-03-22T07:15:56.695 INFO:tasks.workunit.client.0.smithi063.stdout:3/424: write da/d63/d65/f78 [204018,74681] 0 2024-03-22T07:15:56.695 INFO:tasks.workunit.client.0.smithi063.stdout:3/425: chown da/c1d 3122 1 2024-03-22T07:15:56.695 INFO:tasks.workunit.client.0.smithi063.stdout:3/426: chown da/d63/d65/d81 401046 1 2024-03-22T07:15:56.695 INFO:tasks.workunit.client.1.smithi069.stdout:3/138: sync 2024-03-22T07:15:56.697 INFO:tasks.workunit.client.0.smithi063.stdout:7/325: mknod d5/d14/d1d/d29/d4a/c68 0 2024-03-22T07:15:56.699 INFO:tasks.workunit.client.0.smithi063.stdout:7/326: getdents d5/da 0 2024-03-22T07:15:56.701 INFO:tasks.workunit.client.0.smithi063.stdout:3/427: rename da/d16/d26/f58 to da/d16/d26/f8b 0 2024-03-22T07:15:56.704 INFO:tasks.workunit.client.0.smithi063.stdout:5/398: sync 2024-03-22T07:15:56.716 INFO:tasks.workunit.client.0.smithi063.stdout:3/428: dread da/d16/d26/f8b [0,4194304] 0 2024-03-22T07:15:56.716 INFO:tasks.workunit.client.0.smithi063.stdout:3/429: chown da/d16/d3f/l87 188990009 1 2024-03-22T07:15:56.716 INFO:tasks.workunit.client.0.smithi063.stdout:3/430: mkdir da/d16/d89/d8c 0 2024-03-22T07:15:56.718 INFO:tasks.workunit.client.0.smithi063.stdout:3/431: unlink da/d16/f5c 0 2024-03-22T07:15:56.719 INFO:tasks.workunit.client.0.smithi063.stdout:3/432: fsync da/d16/f38 0 2024-03-22T07:15:56.719 INFO:tasks.workunit.client.0.smithi063.stdout:3/433: truncate da/d16/f38 4779426 0 2024-03-22T07:15:56.719 INFO:tasks.workunit.client.0.smithi063.stdout:3/434: stat da/c1b 0 2024-03-22T07:15:56.719 INFO:tasks.workunit.client.0.smithi063.stdout:3/435: symlink da/d16/d89/l8d 0 2024-03-22T07:15:56.720 INFO:tasks.workunit.client.0.smithi063.stdout:3/436: link da/c53 da/d63/d65/d86/c8e 0 2024-03-22T07:15:56.720 INFO:tasks.workunit.client.0.smithi063.stdout:3/437: write da/f6a [860858,87566] 0 2024-03-22T07:15:56.720 INFO:tasks.workunit.client.0.smithi063.stdout:3/438: rmdir da/d63/d65/d83 39 2024-03-22T07:15:56.721 INFO:tasks.workunit.client.0.smithi063.stdout:3/439: link da/d63/d65/f78 da/d16/d26/d69/f8f 0 2024-03-22T07:15:56.725 INFO:tasks.workunit.client.0.smithi063.stdout:4/348: dwrite d1/d5/d43/d27/d2c/f7b [0,4194304] 0 2024-03-22T07:15:56.726 INFO:tasks.workunit.client.1.smithi069.stdout:3/139: dwrite d5/f18 [0,4194304] 0 2024-03-22T07:15:56.726 INFO:tasks.workunit.client.0.smithi063.stdout:7/327: dwrite d5/d14/d1d/d29/f47 [0,4194304] 0 2024-03-22T07:15:56.729 INFO:tasks.workunit.client.1.smithi069.stdout:3/140: rename d5/d9/db/f1c to d5/d9/d1e/f32 0 2024-03-22T07:15:56.730 INFO:tasks.workunit.client.0.smithi063.stdout:4/349: link d1/d5/dc/f41 d1/d5/d43/d27/d2c/d4a/d78/f7d 0 2024-03-22T07:15:56.731 INFO:tasks.workunit.client.1.smithi069.stdout:3/141: rmdir d5/d9/db/d2d 0 2024-03-22T07:15:56.731 INFO:tasks.workunit.client.1.smithi069.stdout:3/142: fdatasync d5/d9/d23/f2b 0 2024-03-22T07:15:56.731 INFO:tasks.workunit.client.1.smithi069.stdout:3/143: fdatasync d5/d9/f13 0 2024-03-22T07:15:56.734 INFO:tasks.workunit.client.1.smithi069.stdout:3/144: dread d5/f6 [0,4194304] 0 2024-03-22T07:15:56.734 INFO:tasks.workunit.client.1.smithi069.stdout:3/145: truncate d5/d9/f13 75406 0 2024-03-22T07:15:56.734 INFO:tasks.workunit.client.1.smithi069.stdout:3/146: chown d5/f31 776 1 2024-03-22T07:15:56.738 INFO:tasks.workunit.client.0.smithi063.stdout:7/328: rename d5/d14/d1d/d40/l5e to d5/d14/d1d/l69 0 2024-03-22T07:15:56.739 INFO:tasks.workunit.client.0.smithi063.stdout:4/350: link d1/d5/d43/d27/d2c/d4a/l5e d1/d5/d43/d27/d2c/d3d/l7e 0 2024-03-22T07:15:56.740 INFO:tasks.workunit.client.0.smithi063.stdout:7/329: creat d5/d14/d1d/d29/f6a x:0 0 0 2024-03-22T07:15:56.741 INFO:tasks.workunit.client.0.smithi063.stdout:3/440: dwrite da/f52 [0,4194304] 0 2024-03-22T07:15:56.741 INFO:tasks.workunit.client.0.smithi063.stdout:3/441: chown da 273 1 2024-03-22T07:15:56.741 INFO:tasks.workunit.client.0.smithi063.stdout:4/351: unlink d1/d5/f23 0 2024-03-22T07:15:56.743 INFO:tasks.workunit.client.0.smithi063.stdout:7/330: unlink d5/d14/f37 0 2024-03-22T07:15:56.745 INFO:tasks.workunit.client.1.smithi069.stdout:3/147: write d5/f18 [3666795,79216] 0 2024-03-22T07:15:56.745 INFO:tasks.workunit.client.0.smithi063.stdout:3/442: creat da/d16/d26/d69/d88/f90 x:0 0 0 2024-03-22T07:15:56.746 INFO:tasks.workunit.client.0.smithi063.stdout:3/443: symlink da/d16/d26/l91 0 2024-03-22T07:15:56.756 INFO:tasks.workunit.client.0.smithi063.stdout:3/444: write da/d16/d26/f27 [3777105,67994] 0 2024-03-22T07:15:56.758 INFO:tasks.workunit.client.0.smithi063.stdout:7/331: dwrite d5/d14/d1d/d29/f5d [8388608,4194304] 0 2024-03-22T07:15:56.758 INFO:tasks.workunit.client.0.smithi063.stdout:3/445: mkdir da/d63/d65/d92 0 2024-03-22T07:15:56.759 INFO:tasks.workunit.client.0.smithi063.stdout:7/332: write d5/da/d5f/f5c [1271695,10838] 0 2024-03-22T07:15:56.760 INFO:tasks.workunit.client.1.smithi069.stdout:3/148: mkdir d5/d33 0 2024-03-22T07:15:56.761 INFO:tasks.workunit.client.0.smithi063.stdout:3/446: unlink da/d63/c7c 0 2024-03-22T07:15:56.762 INFO:tasks.workunit.client.0.smithi063.stdout:3/447: creat da/d63/d65/d86/f93 x:0 0 0 2024-03-22T07:15:56.762 INFO:tasks.workunit.client.0.smithi063.stdout:3/448: truncate da/d16/d3f/f4c 1137688 0 2024-03-22T07:15:56.764 INFO:tasks.workunit.client.0.smithi063.stdout:3/449: getdents da/d63/d65/d4b 0 2024-03-22T07:15:56.764 INFO:tasks.workunit.client.0.smithi063.stdout:3/450: fsync da/f1c 0 2024-03-22T07:15:56.765 INFO:tasks.workunit.client.0.smithi063.stdout:3/451: chown da/d16/l45 709976 1 2024-03-22T07:15:56.766 INFO:tasks.workunit.client.0.smithi063.stdout:3/452: mknod da/d63/d65/d83/c94 0 2024-03-22T07:15:56.768 INFO:tasks.workunit.client.0.smithi063.stdout:3/453: getdents da 0 2024-03-22T07:15:56.768 INFO:tasks.workunit.client.0.smithi063.stdout:3/454: chown da/d63/d65/d83 4069004 1 2024-03-22T07:15:56.770 INFO:tasks.workunit.client.0.smithi063.stdout:3/455: truncate da/f21 727659 0 2024-03-22T07:15:56.770 INFO:tasks.workunit.client.0.smithi063.stdout:3/456: symlink da/d16/d89/l95 0 2024-03-22T07:15:56.770 INFO:tasks.workunit.client.1.smithi069.stdout:3/149: dread d5/d9/d26/f2e [0,4194304] 0 2024-03-22T07:15:56.772 INFO:tasks.workunit.client.1.smithi069.stdout:3/150: rename d5/d9/d1e/c1b to d5/d9/d23/c34 0 2024-03-22T07:15:56.772 INFO:tasks.workunit.client.1.smithi069.stdout:3/151: write d5/d9/d1e/f21 [1177717,7953] 0 2024-03-22T07:15:56.792 INFO:tasks.workunit.client.0.smithi063.stdout:3/457: dwrite da/f57 [0,4194304] 0 2024-03-22T07:15:56.793 INFO:tasks.workunit.client.1.smithi069.stdout:3/152: dwrite d5/d9/d1e/f32 [0,4194304] 0 2024-03-22T07:15:56.793 INFO:tasks.workunit.client.1.smithi069.stdout:3/153: dread d5/d9/f13 [0,4194304] 0 2024-03-22T07:15:56.795 INFO:tasks.workunit.client.1.smithi069.stdout:5/82: sync 2024-03-22T07:15:56.795 INFO:tasks.workunit.client.1.smithi069.stdout:0/152: sync 2024-03-22T07:15:56.795 INFO:tasks.workunit.client.1.smithi069.stdout:5/83: chown d1/d7/cd 38955389 1 2024-03-22T07:15:56.795 INFO:tasks.workunit.client.1.smithi069.stdout:5/84: fsync d1/d7/d8/f12 0 2024-03-22T07:15:56.796 INFO:tasks.workunit.client.1.smithi069.stdout:0/153: read d3/d7/d15/f1b [324156,113774] 0 2024-03-22T07:15:56.796 INFO:tasks.workunit.client.1.smithi069.stdout:0/154: readlink d3/d7/d15/d1f/l2f 0 2024-03-22T07:15:56.796 INFO:tasks.workunit.client.1.smithi069.stdout:7/232: sync 2024-03-22T07:15:56.796 INFO:tasks.workunit.client.1.smithi069.stdout:2/147: sync 2024-03-22T07:15:56.796 INFO:tasks.workunit.client.1.smithi069.stdout:2/148: fdatasync f6 0 2024-03-22T07:15:56.796 INFO:tasks.workunit.client.1.smithi069.stdout:2/149: chown d18/d1c 3706576 1 2024-03-22T07:15:56.797 INFO:tasks.workunit.client.1.smithi069.stdout:5/85: symlink d1/d6/l19 0 2024-03-22T07:15:56.799 INFO:tasks.workunit.client.1.smithi069.stdout:5/86: link d1/c5 d1/d7/d8/c1a 0 2024-03-22T07:15:56.801 INFO:tasks.workunit.client.1.smithi069.stdout:7/233: link d9/c15 d9/d2b/c51 0 2024-03-22T07:15:56.802 INFO:tasks.workunit.client.1.smithi069.stdout:5/87: dread d1/d7/f17 [0,4194304] 0 2024-03-22T07:15:56.805 INFO:tasks.workunit.client.1.smithi069.stdout:7/234: mknod d9/db/d1a/c52 0 2024-03-22T07:15:56.805 INFO:tasks.workunit.client.1.smithi069.stdout:7/235: write d9/db/d1a/f21 [948227,34988] 0 2024-03-22T07:15:56.806 INFO:tasks.workunit.client.0.smithi063.stdout:3/458: dread da/d16/f38 [0,4194304] 0 2024-03-22T07:15:56.807 INFO:tasks.workunit.client.1.smithi069.stdout:2/150: dread d18/d1c/d25/f2e [0,4194304] 0 2024-03-22T07:15:56.807 INFO:tasks.workunit.client.1.smithi069.stdout:2/151: fdatasync f1 0 2024-03-22T07:15:56.816 INFO:tasks.workunit.client.1.smithi069.stdout:3/154: dwrite d5/d9/d1e/f22 [4194304,4194304] 0 2024-03-22T07:15:56.816 INFO:tasks.workunit.client.1.smithi069.stdout:3/155: write d5/f18 [2451498,71285] 0 2024-03-22T07:15:56.839 INFO:tasks.workunit.client.1.smithi069.stdout:5/88: dwrite d1/d7/d8/f12 [0,4194304] 0 2024-03-22T07:15:56.839 INFO:tasks.workunit.client.1.smithi069.stdout:7/236: dwrite d9/f18 [0,4194304] 0 2024-03-22T07:15:56.839 INFO:tasks.workunit.client.1.smithi069.stdout:0/155: dwrite d3/d7/d15/f20 [0,4194304] 0 2024-03-22T07:15:56.839 INFO:tasks.workunit.client.1.smithi069.stdout:0/156: creat d3/f39 x:0 0 0 2024-03-22T07:15:56.840 INFO:tasks.workunit.client.1.smithi069.stdout:0/157: chown d3/ld 23096756 1 2024-03-22T07:15:56.841 INFO:tasks.workunit.client.0.smithi063.stdout:3/459: dwrite da/f3d [4194304,4194304] 0 2024-03-22T07:15:56.841 INFO:tasks.workunit.client.0.smithi063.stdout:3/460: fsync da/d63/f75 0 2024-03-22T07:15:56.843 INFO:tasks.workunit.client.1.smithi069.stdout:5/89: rename d1/d10/l13 to d1/d7/d8/l1b 0 2024-03-22T07:15:56.843 INFO:tasks.workunit.client.1.smithi069.stdout:7/237: dread f4 [0,4194304] 0 2024-03-22T07:15:56.846 INFO:tasks.workunit.client.1.smithi069.stdout:0/158: creat d3/da/f3a x:0 0 0 2024-03-22T07:15:56.846 INFO:tasks.workunit.client.1.smithi069.stdout:0/159: write d3/da/f3a [103275,22696] 0 2024-03-22T07:15:56.850 INFO:tasks.workunit.client.1.smithi069.stdout:7/238: rename d9/d2b to d9/d41/d44/d53 0 2024-03-22T07:15:56.852 INFO:tasks.workunit.client.1.smithi069.stdout:5/90: write d1/d7/d8/f12 [1529655,106308] 0 2024-03-22T07:15:56.852 INFO:tasks.workunit.client.1.smithi069.stdout:7/239: creat d9/d41/d44/d53/d1d/d2a/d30/f54 x:0 0 0 2024-03-22T07:15:56.852 INFO:tasks.workunit.client.1.smithi069.stdout:7/240: chown d9/c32 12334 1 2024-03-22T07:15:56.856 INFO:tasks.workunit.client.1.smithi069.stdout:0/160: write d3/d7/d15/f20 [1714244,70875] 0 2024-03-22T07:15:56.857 INFO:tasks.workunit.client.0.smithi063.stdout:3/461: dwrite da/d16/d3f/f70 [0,4194304] 0 2024-03-22T07:15:56.864 INFO:tasks.workunit.client.0.smithi063.stdout:3/462: write da/d16/f25 [183069,79259] 0 2024-03-22T07:15:56.864 INFO:tasks.workunit.client.1.smithi069.stdout:5/91: mknod d1/d6/c1c 0 2024-03-22T07:15:56.865 INFO:tasks.workunit.client.1.smithi069.stdout:5/92: readlink d1/d7/d8/lc 0 2024-03-22T07:15:56.865 INFO:tasks.workunit.client.1.smithi069.stdout:5/93: stat d1/d7/d8 0 2024-03-22T07:15:56.865 INFO:tasks.workunit.client.0.smithi063.stdout:3/463: getdents da/d16/d26/d69/d88 0 2024-03-22T07:15:56.866 INFO:tasks.workunit.client.0.smithi063.stdout:3/464: mkdir da/d63/d65/d92/d96 0 2024-03-22T07:15:56.867 INFO:tasks.workunit.client.0.smithi063.stdout:3/465: creat da/d16/f97 x:0 0 0 2024-03-22T07:15:56.867 INFO:tasks.workunit.client.0.smithi063.stdout:3/466: symlink da/d63/l98 0 2024-03-22T07:15:56.868 INFO:tasks.workunit.client.0.smithi063.stdout:3/467: chown da/d16/d26/c28 916 1 2024-03-22T07:15:56.872 INFO:tasks.workunit.client.0.smithi063.stdout:2/397: sync 2024-03-22T07:15:56.872 INFO:tasks.workunit.client.0.smithi063.stdout:1/485: sync 2024-03-22T07:15:56.872 INFO:tasks.workunit.client.0.smithi063.stdout:2/398: readlink d0/de/d1b/d63/l80 0 2024-03-22T07:15:56.872 INFO:tasks.workunit.client.0.smithi063.stdout:1/486: truncate d4/df/f11 3188635 0 2024-03-22T07:15:56.873 INFO:tasks.workunit.client.1.smithi069.stdout:0/161: dwrite d3/da/f11 [0,4194304] 0 2024-03-22T07:15:56.873 INFO:tasks.workunit.client.0.smithi063.stdout:2/399: mknod d0/de/d1b/d32/c83 0 2024-03-22T07:15:56.874 INFO:tasks.workunit.client.0.smithi063.stdout:2/400: mknod d0/de/d58/c84 0 2024-03-22T07:15:56.878 INFO:tasks.workunit.client.0.smithi063.stdout:3/468: dread da/d16/f38 [0,4194304] 0 2024-03-22T07:15:56.878 INFO:tasks.workunit.client.0.smithi063.stdout:3/469: fsync da/d16/f42 0 2024-03-22T07:15:56.882 INFO:tasks.workunit.client.0.smithi063.stdout:3/470: rmdir da/d16/d3f 39 2024-03-22T07:15:56.883 INFO:tasks.workunit.client.1.smithi069.stdout:0/162: dread d3/f8 [0,4194304] 0 2024-03-22T07:15:56.884 INFO:tasks.workunit.client.0.smithi063.stdout:3/471: rename da/f2f to da/d63/d65/f99 0 2024-03-22T07:15:56.886 INFO:tasks.workunit.client.0.smithi063.stdout:3/472: creat da/d63/d65/d92/d96/f9a x:0 0 0 2024-03-22T07:15:56.896 INFO:tasks.workunit.client.0.smithi063.stdout:2/401: dwrite d0/de/d35/d36/f7c [0,4194304] 0 2024-03-22T07:15:56.899 INFO:tasks.workunit.client.0.smithi063.stdout:2/402: mknod d0/de/d35/c85 0 2024-03-22T07:15:56.899 INFO:tasks.workunit.client.0.smithi063.stdout:2/403: creat d0/de/d1b/d63/f86 x:0 0 0 2024-03-22T07:15:56.901 INFO:tasks.workunit.client.0.smithi063.stdout:2/404: mknod d0/de/d58/d7f/c87 0 2024-03-22T07:15:56.901 INFO:tasks.workunit.client.0.smithi063.stdout:2/405: fdatasync d0/f41 0 2024-03-22T07:15:56.902 INFO:tasks.workunit.client.0.smithi063.stdout:2/406: dread - d0/de/f4e zero size 2024-03-22T07:15:56.902 INFO:tasks.workunit.client.0.smithi063.stdout:3/473: rmdir da/d63/d65 39 2024-03-22T07:15:56.902 INFO:tasks.workunit.client.0.smithi063.stdout:3/474: chown da/d16/d3f/f79 296390 1 2024-03-22T07:15:56.902 INFO:tasks.workunit.client.0.smithi063.stdout:3/475: chown da/d63/d65/d4b/c4f 734616654 1 2024-03-22T07:15:56.903 INFO:tasks.workunit.client.0.smithi063.stdout:3/476: mkdir da/d63/d65/d81/d9b 0 2024-03-22T07:15:56.903 INFO:tasks.workunit.client.0.smithi063.stdout:3/477: chown da/c1b 0 1 2024-03-22T07:15:56.903 INFO:tasks.workunit.client.0.smithi063.stdout:3/478: chown da/d16/l7d 49794 1 2024-03-22T07:15:56.903 INFO:tasks.workunit.client.0.smithi063.stdout:3/479: stat da/d63/d65/c74 0 2024-03-22T07:15:56.909 INFO:tasks.workunit.client.1.smithi069.stdout:1/211: sync 2024-03-22T07:15:56.909 INFO:tasks.workunit.client.1.smithi069.stdout:9/153: sync 2024-03-22T07:15:56.909 INFO:tasks.workunit.client.1.smithi069.stdout:0/163: dwrite d3/d7/d15/d1f/f2b [0,4194304] 0 2024-03-22T07:15:56.909 INFO:tasks.workunit.client.1.smithi069.stdout:0/164: fdatasync d3/d7/d15/d29/f30 0 2024-03-22T07:15:56.910 INFO:tasks.workunit.client.1.smithi069.stdout:1/212: write d1/d13/f2d [97140,73978] 0 2024-03-22T07:15:56.920 INFO:tasks.workunit.client.1.smithi069.stdout:5/94: unlink d1/d7/f17 0 2024-03-22T07:15:56.923 INFO:tasks.workunit.client.1.smithi069.stdout:7/241: rename d9/c1c to d9/d41/d44/d53/c55 0 2024-03-22T07:15:56.927 INFO:tasks.workunit.client.1.smithi069.stdout:9/154: symlink d2/dd/d18/l33 0 2024-03-22T07:15:56.928 INFO:tasks.workunit.client.1.smithi069.stdout:0/165: creat d3/da/f3b x:0 0 0 2024-03-22T07:15:56.929 INFO:tasks.workunit.client.1.smithi069.stdout:7/242: unlink d9/f14 0 2024-03-22T07:15:56.930 INFO:tasks.workunit.client.1.smithi069.stdout:0/166: mknod d3/d7/d15/c3c 0 2024-03-22T07:15:56.931 INFO:tasks.workunit.client.1.smithi069.stdout:0/167: dread - d3/d7/f12 zero size 2024-03-22T07:15:56.938 INFO:tasks.workunit.client.1.smithi069.stdout:5/95: dwrite d1/f3 [0,4194304] 0 2024-03-22T07:15:56.939 INFO:tasks.workunit.client.1.smithi069.stdout:5/96: rename d1/c5 to d1/d7/c1d 0 2024-03-22T07:15:56.970 INFO:tasks.workunit.client.1.smithi069.stdout:9/155: dwrite d2/dd/f10 [0,4194304] 0 2024-03-22T07:15:56.972 INFO:tasks.workunit.client.1.smithi069.stdout:7/243: dwrite d9/db/d48/d4b/f4f [0,4194304] 0 2024-03-22T07:15:56.972 INFO:tasks.workunit.client.1.smithi069.stdout:7/244: chown d9/db/d48 1 1 2024-03-22T07:15:56.975 INFO:tasks.workunit.client.1.smithi069.stdout:7/245: creat d9/db/d1a/f56 x:0 0 0 2024-03-22T07:15:56.975 INFO:tasks.workunit.client.1.smithi069.stdout:7/246: read f8 [99445,46319] 0 2024-03-22T07:15:56.981 INFO:tasks.workunit.client.1.smithi069.stdout:7/247: dread d9/db/f16 [0,4194304] 0 2024-03-22T07:15:56.983 INFO:tasks.workunit.client.0.smithi063.stdout:3/480: dwrite da/d16/d3f/f44 [0,4194304] 0 2024-03-22T07:15:56.983 INFO:tasks.workunit.client.0.smithi063.stdout:3/481: fdatasync da/d16/d3f/f4c 0 2024-03-22T07:15:56.983 INFO:tasks.workunit.client.1.smithi069.stdout:9/156: write d2/dd/d18/d2e/f30 [1491225,26194] 0 2024-03-22T07:15:56.983 INFO:tasks.workunit.client.1.smithi069.stdout:0/168: dwrite d3/d7/d15/d29/f30 [0,4194304] 0 2024-03-22T07:15:56.983 INFO:tasks.workunit.client.1.smithi069.stdout:0/169: stat d3/d7/d15/d1f/f26 0 2024-03-22T07:15:56.986 INFO:tasks.workunit.client.1.smithi069.stdout:5/97: dwrite d1/f9 [4194304,4194304] 0 2024-03-22T07:15:56.986 INFO:tasks.workunit.client.1.smithi069.stdout:5/98: creat d1/d6/f1e x:0 0 0 2024-03-22T07:15:56.986 INFO:tasks.workunit.client.1.smithi069.stdout:5/99: write d1/d7/d8/f12 [1878219,9357] 0 2024-03-22T07:15:56.987 INFO:tasks.workunit.client.0.smithi063.stdout:3/482: symlink da/d63/d65/d92/d96/l9c 0 2024-03-22T07:15:56.987 INFO:tasks.workunit.client.1.smithi069.stdout:9/157: fsync d2/dd/d18/d2e/f30 0 2024-03-22T07:15:56.993 INFO:tasks.workunit.client.1.smithi069.stdout:9/158: dread d2/dd/d18/d2e/f30 [0,4194304] 0 2024-03-22T07:15:56.994 INFO:tasks.workunit.client.1.smithi069.stdout:9/159: chown d2 329934217 1 2024-03-22T07:15:56.994 INFO:tasks.workunit.client.0.smithi063.stdout:6/455: sync 2024-03-22T07:15:56.994 INFO:tasks.workunit.client.0.smithi063.stdout:6/456: truncate d3/d15/d16/d60/d6d/d4e/d5a/d88/f8d 1676506 0 2024-03-22T07:15:56.995 INFO:tasks.workunit.client.0.smithi063.stdout:9/356: sync 2024-03-22T07:15:56.995 INFO:tasks.workunit.client.0.smithi063.stdout:0/529: sync 2024-03-22T07:15:56.997 INFO:tasks.workunit.client.0.smithi063.stdout:9/357: fsync d8/f10 0 2024-03-22T07:15:56.998 INFO:tasks.workunit.client.0.smithi063.stdout:3/483: dread da/d16/f42 [4194304,4194304] 0 2024-03-22T07:15:56.998 INFO:tasks.workunit.client.0.smithi063.stdout:3/484: write da/d16/d3f/f4c [414640,98347] 0 2024-03-22T07:15:56.999 INFO:tasks.workunit.client.0.smithi063.stdout:0/530: rename da/de/d67/c9a to da/de/d17/d25/d3c/d64/caa 0 2024-03-22T07:15:57.001 INFO:tasks.workunit.client.0.smithi063.stdout:9/358: symlink d8/d19/d20/d22/l6c 0 2024-03-22T07:15:57.002 INFO:tasks.workunit.client.0.smithi063.stdout:3/485: symlink da/d63/d65/d81/d9b/l9d 0 2024-03-22T07:15:57.003 INFO:tasks.workunit.client.0.smithi063.stdout:9/359: symlink d8/d19/d2b/l6d 0 2024-03-22T07:15:57.003 INFO:tasks.workunit.client.0.smithi063.stdout:3/486: write da/d16/d26/f29 [943095,870] 0 2024-03-22T07:15:57.004 INFO:tasks.workunit.client.1.smithi069.stdout:9/160: rmdir d2/dd/d18 39 2024-03-22T07:15:57.005 INFO:tasks.workunit.client.0.smithi063.stdout:9/360: mknod d8/d19/d20/d22/d29/d31/d4b/c6e 0 2024-03-22T07:15:57.005 INFO:tasks.workunit.client.0.smithi063.stdout:9/361: write d8/d17/d1e/f5c [1044255,113815] 0 2024-03-22T07:15:57.007 INFO:tasks.workunit.client.0.smithi063.stdout:3/487: creat da/d16/d89/d8c/f9e x:0 0 0 2024-03-22T07:15:57.008 INFO:tasks.workunit.client.0.smithi063.stdout:4/352: sync 2024-03-22T07:15:57.008 INFO:tasks.workunit.client.0.smithi063.stdout:2/407: sync 2024-03-22T07:15:57.008 INFO:tasks.workunit.client.0.smithi063.stdout:3/488: rmdir da/d63/d65/d92/d96 39 2024-03-22T07:15:57.008 INFO:tasks.workunit.client.0.smithi063.stdout:2/408: creat d0/de/d12/d20/f88 x:0 0 0 2024-03-22T07:15:57.008 INFO:tasks.workunit.client.0.smithi063.stdout:7/333: sync 2024-03-22T07:15:57.009 INFO:tasks.workunit.client.0.smithi063.stdout:8/456: sync 2024-03-22T07:15:57.009 INFO:tasks.workunit.client.0.smithi063.stdout:5/399: sync 2024-03-22T07:15:57.009 INFO:tasks.workunit.client.0.smithi063.stdout:1/487: sync 2024-03-22T07:15:57.009 INFO:tasks.workunit.client.0.smithi063.stdout:7/334: write d5/d16/d1b/d21/f2a [588621,118821] 0 2024-03-22T07:15:57.009 INFO:tasks.workunit.client.0.smithi063.stdout:1/488: write d4/df/d13/d2b/d6c/d85/d9e/fa5 [374464,10521] 0 2024-03-22T07:15:57.009 INFO:tasks.workunit.client.0.smithi063.stdout:7/335: write d5/d14/d1d/d29/d3b/f64 [125147,70206] 0 2024-03-22T07:15:57.009 INFO:tasks.workunit.client.0.smithi063.stdout:4/353: mknod d1/d5/d43/d27/d2c/d3d/d54/c7f 0 2024-03-22T07:15:57.010 INFO:tasks.workunit.client.1.smithi069.stdout:5/100: fsync d1/d6/f1e 0 2024-03-22T07:15:57.010 INFO:tasks.workunit.client.1.smithi069.stdout:5/101: write d1/d6/f1e [504656,75272] 0 2024-03-22T07:15:57.010 INFO:tasks.workunit.client.1.smithi069.stdout:5/102: creat d1/f1f x:0 0 0 2024-03-22T07:15:57.011 INFO:tasks.workunit.client.1.smithi069.stdout:6/124: sync 2024-03-22T07:15:57.011 INFO:tasks.workunit.client.1.smithi069.stdout:4/222: sync 2024-03-22T07:15:57.011 INFO:tasks.workunit.client.1.smithi069.stdout:4/223: getdents d6/d16/d3a 0 2024-03-22T07:15:57.011 INFO:tasks.workunit.client.0.smithi063.stdout:0/531: write da/de/d39/f72 [2806776,108374] 0 2024-03-22T07:15:57.012 INFO:tasks.workunit.client.1.smithi069.stdout:6/125: read d4/f6 [1274876,28183] 0 2024-03-22T07:15:57.012 INFO:tasks.workunit.client.1.smithi069.stdout:9/161: rename d2/c2f to d2/dd/d18/c34 0 2024-03-22T07:15:57.012 INFO:tasks.workunit.client.1.smithi069.stdout:6/126: rename d4 to d4/d7/d28 22 2024-03-22T07:15:57.013 INFO:tasks.workunit.client.0.smithi063.stdout:3/489: rename da/d16/d26/d69/d88 to da/d63/d65/d81/d9b/d9f 0 2024-03-22T07:15:57.014 INFO:tasks.workunit.client.0.smithi063.stdout:2/409: mknod d0/de/c89 0 2024-03-22T07:15:57.014 INFO:tasks.workunit.client.0.smithi063.stdout:2/410: chown d0/de/d35 178666 1 2024-03-22T07:15:57.015 INFO:tasks.workunit.client.1.smithi069.stdout:4/224: link d6/d12/d18/d21/l40 d6/dc/l44 0 2024-03-22T07:15:57.015 INFO:tasks.workunit.client.1.smithi069.stdout:6/127: symlink d4/dd/l29 0 2024-03-22T07:15:57.015 INFO:tasks.workunit.client.0.smithi063.stdout:5/400: write d8/d13/d1e/d31/f4e [609547,11940] 0 2024-03-22T07:15:57.016 INFO:tasks.workunit.client.0.smithi063.stdout:8/457: getdents d0 0 2024-03-22T07:15:57.017 INFO:tasks.workunit.client.0.smithi063.stdout:1/489: truncate d4/d14/f18 3522191 0 2024-03-22T07:15:57.019 INFO:tasks.workunit.client.0.smithi063.stdout:7/336: getdents d5/da/d5f 0 2024-03-22T07:15:57.022 INFO:tasks.workunit.client.0.smithi063.stdout:4/354: mkdir d1/d5/d43/d27/d2c/d4a/d80 0 2024-03-22T07:15:57.022 INFO:tasks.workunit.client.0.smithi063.stdout:4/355: chown d1/f21 167847438 1 2024-03-22T07:15:57.024 INFO:tasks.workunit.client.1.smithi069.stdout:0/170: dwrite d3/f8 [0,4194304] 0 2024-03-22T07:15:57.024 INFO:tasks.workunit.client.1.smithi069.stdout:5/103: fsync d1/d6/f1e 0 2024-03-22T07:15:57.024 INFO:tasks.workunit.client.1.smithi069.stdout:5/104: fdatasync d1/d6/f1e 0 2024-03-22T07:15:57.025 INFO:tasks.workunit.client.0.smithi063.stdout:0/532: truncate da/de/d17/d25/f68 1502522 0 2024-03-22T07:15:57.025 INFO:tasks.workunit.client.0.smithi063.stdout:9/362: dread f2 [0,4194304] 0 2024-03-22T07:15:57.025 INFO:tasks.workunit.client.0.smithi063.stdout:6/457: dwrite d3/d6/fc [0,4194304] 0 2024-03-22T07:15:57.026 INFO:tasks.workunit.client.1.smithi069.stdout:7/248: dwrite f8 [0,4194304] 0 2024-03-22T07:15:57.026 INFO:tasks.workunit.client.1.smithi069.stdout:9/162: dread d2/f5 [0,4194304] 0 2024-03-22T07:15:57.026 INFO:tasks.workunit.client.1.smithi069.stdout:9/163: chown d2/ca 12562 1 2024-03-22T07:15:57.026 INFO:tasks.workunit.client.1.smithi069.stdout:9/164: chown d2/dd/d18/c25 32303 1 2024-03-22T07:15:57.026 INFO:tasks.workunit.client.0.smithi063.stdout:3/490: mknod da/d63/d65/d83/ca0 0 2024-03-22T07:15:57.027 INFO:tasks.workunit.client.0.smithi063.stdout:3/491: chown da/l14 5532 1 2024-03-22T07:15:57.027 INFO:tasks.workunit.client.0.smithi063.stdout:2/411: mkdir d0/de/d74/d8a 0 2024-03-22T07:15:57.028 INFO:tasks.workunit.client.1.smithi069.stdout:0/171: rename d3/d7/d15/c3c to d3/d7/d17/c3d 0 2024-03-22T07:15:57.028 INFO:tasks.workunit.client.1.smithi069.stdout:0/172: read - d3/f39 zero size 2024-03-22T07:15:57.029 INFO:tasks.workunit.client.1.smithi069.stdout:5/105: symlink d1/d6/l20 0 2024-03-22T07:15:57.029 INFO:tasks.workunit.client.0.smithi063.stdout:8/458: rmdir d0/d1b/d28/d7b/d4a 39 2024-03-22T07:15:57.029 INFO:tasks.workunit.client.0.smithi063.stdout:8/459: chown d0/d24/f35 151547923 1 2024-03-22T07:15:57.029 INFO:tasks.workunit.client.0.smithi063.stdout:8/460: stat d0/d1b/d28/d5f/c63 0 2024-03-22T07:15:57.030 INFO:tasks.workunit.client.0.smithi063.stdout:1/490: mkdir d4/df/d13/d2b/d6c/da6 0 2024-03-22T07:15:57.030 INFO:tasks.workunit.client.0.smithi063.stdout:1/491: readlink d4/l35 0 2024-03-22T07:15:57.031 INFO:tasks.workunit.client.0.smithi063.stdout:1/492: truncate d4/df/d13/d39/d63/f66 2194852 0 2024-03-22T07:15:57.031 INFO:tasks.workunit.client.1.smithi069.stdout:7/249: truncate d9/db/d48/d4b/f4f 3123778 0 2024-03-22T07:15:57.031 INFO:tasks.workunit.client.1.smithi069.stdout:9/165: creat d2/dd/d18/f35 x:0 0 0 2024-03-22T07:15:57.032 INFO:tasks.workunit.client.0.smithi063.stdout:4/356: unlink d1/d5/f2d 0 2024-03-22T07:15:57.033 INFO:tasks.workunit.client.1.smithi069.stdout:0/173: link d3/da/l25 d3/d7/d17/l3e 0 2024-03-22T07:15:57.034 INFO:tasks.workunit.client.1.smithi069.stdout:7/250: readlink d9/d41/d44/d53/l25 0 2024-03-22T07:15:57.034 INFO:tasks.workunit.client.1.smithi069.stdout:7/251: stat d9/d41/d44/d53 0 2024-03-22T07:15:57.034 INFO:tasks.workunit.client.1.smithi069.stdout:7/252: truncate d9/d41/d44/d53/d2e/f4d 246612 0 2024-03-22T07:15:57.034 INFO:tasks.workunit.client.0.smithi063.stdout:0/533: mknod da/de/d17/d25/d95/d98/cab 0 2024-03-22T07:15:57.035 INFO:tasks.workunit.client.0.smithi063.stdout:0/534: chown da/de/d17/d4e/ca0 43 1 2024-03-22T07:15:57.035 INFO:tasks.workunit.client.0.smithi063.stdout:0/535: write da/de/d39/f8c [40883,123906] 0 2024-03-22T07:15:57.035 INFO:tasks.workunit.client.1.smithi069.stdout:9/166: mknod d2/dd/d18/c36 0 2024-03-22T07:15:57.036 INFO:tasks.workunit.client.1.smithi069.stdout:0/174: creat d3/d7/d17/f3f x:0 0 0 2024-03-22T07:15:57.036 INFO:tasks.workunit.client.1.smithi069.stdout:0/175: fsync d3/f8 0 2024-03-22T07:15:57.036 INFO:tasks.workunit.client.1.smithi069.stdout:0/176: creat d3/f40 x:0 0 0 2024-03-22T07:15:57.036 INFO:tasks.workunit.client.1.smithi069.stdout:0/177: truncate d3/d7/d17/f3f 780010 0 2024-03-22T07:15:57.036 INFO:tasks.workunit.client.1.smithi069.stdout:0/178: fsync d3/f39 0 2024-03-22T07:15:57.036 INFO:tasks.workunit.client.0.smithi063.stdout:9/363: creat d8/d19/d39/f6f x:0 0 0 2024-03-22T07:15:57.037 INFO:tasks.workunit.client.0.smithi063.stdout:9/364: write d8/d19/d20/d22/d29/d31/d37/f59 [715911,5074] 0 2024-03-22T07:15:57.037 INFO:tasks.workunit.client.1.smithi069.stdout:9/167: unlink d2/c28 0 2024-03-22T07:15:57.037 INFO:tasks.workunit.client.0.smithi063.stdout:6/458: link d3/d1c/f5d d3/d15/d16/d57/f97 0 2024-03-22T07:15:57.038 INFO:tasks.workunit.client.0.smithi063.stdout:2/412: mkdir d0/de/d1b/d8b 0 2024-03-22T07:15:57.039 INFO:tasks.workunit.client.0.smithi063.stdout:3/492: dread da/f71 [0,4194304] 0 2024-03-22T07:15:57.039 INFO:tasks.workunit.client.0.smithi063.stdout:3/493: fdatasync da/d16/d26/f27 0 2024-03-22T07:15:57.039 INFO:tasks.workunit.client.1.smithi069.stdout:0/179: creat d3/d7/d15/d1f/f41 x:0 0 0 2024-03-22T07:15:57.040 INFO:tasks.workunit.client.1.smithi069.stdout:9/168: creat d2/dd/f37 x:0 0 0 2024-03-22T07:15:57.040 INFO:tasks.workunit.client.1.smithi069.stdout:9/169: truncate d2/dd/f37 916081 0 2024-03-22T07:15:57.040 INFO:tasks.workunit.client.1.smithi069.stdout:9/170: chown d2/dd/d18/d2e/f32 244513 1 2024-03-22T07:15:57.041 INFO:tasks.workunit.client.0.smithi063.stdout:1/493: mkdir d4/d8b/da7 0 2024-03-22T07:15:57.041 INFO:tasks.workunit.client.0.smithi063.stdout:1/494: readlink d4/df/l1c 0 2024-03-22T07:15:57.042 INFO:tasks.workunit.client.1.smithi069.stdout:5/106: fsync d1/d7/d8/f12 0 2024-03-22T07:15:57.042 INFO:tasks.workunit.client.1.smithi069.stdout:4/225: dwrite d6/dc/d23/f3c [0,4194304] 0 2024-03-22T07:15:57.042 INFO:tasks.workunit.client.1.smithi069.stdout:4/226: chown f5 1 1 2024-03-22T07:15:57.048 INFO:tasks.workunit.client.1.smithi069.stdout:4/227: dread d6/dc/ff [4194304,4194304] 0 2024-03-22T07:15:57.048 INFO:tasks.workunit.client.0.smithi063.stdout:0/536: link f1 da/de/d17/d25/d3c/d64/fac 0 2024-03-22T07:15:57.050 INFO:tasks.workunit.client.0.smithi063.stdout:8/461: dwrite d0/d1b/d28/d7b/d45/f58 [0,4194304] 0 2024-03-22T07:15:57.051 INFO:tasks.workunit.client.1.smithi069.stdout:6/128: dwrite d4/dd/f25 [0,4194304] 0 2024-03-22T07:15:57.054 INFO:tasks.workunit.client.1.smithi069.stdout:5/107: unlink d1/d6/c1c 0 2024-03-22T07:15:57.054 INFO:tasks.workunit.client.0.smithi063.stdout:9/365: mknod d8/d19/d20/d22/c70 0 2024-03-22T07:15:57.054 INFO:tasks.workunit.client.0.smithi063.stdout:2/413: write d0/de/d12/d20/f52 [3223468,80792] 0 2024-03-22T07:15:57.054 INFO:tasks.workunit.client.0.smithi063.stdout:5/401: link d8/d13/f18 d8/d13/d28/f73 0 2024-03-22T07:15:57.055 INFO:tasks.workunit.client.1.smithi069.stdout:0/180: dread d3/d7/d15/d29/f30 [0,4194304] 0 2024-03-22T07:15:57.055 INFO:tasks.workunit.client.1.smithi069.stdout:0/181: stat d3/d7/f1e 0 2024-03-22T07:15:57.055 INFO:tasks.workunit.client.1.smithi069.stdout:0/182: read - d3/d7/d15/d1f/f41 zero size 2024-03-22T07:15:57.055 INFO:tasks.workunit.client.1.smithi069.stdout:4/228: truncate d6/dc/f37 2716964 0 2024-03-22T07:15:57.060 INFO:tasks.workunit.client.1.smithi069.stdout:9/171: write d2/dd/f10 [104348,78792] 0 2024-03-22T07:15:57.061 INFO:tasks.workunit.client.1.smithi069.stdout:9/172: readlink d2/l4 0 2024-03-22T07:15:57.061 INFO:tasks.workunit.client.1.smithi069.stdout:6/129: symlink d4/d7/da/l2a 0 2024-03-22T07:15:57.061 INFO:tasks.workunit.client.1.smithi069.stdout:6/130: chown d4/d7/da/lc 18866 1 2024-03-22T07:15:57.061 INFO:tasks.workunit.client.1.smithi069.stdout:5/108: mkdir d1/d6/d21 0 2024-03-22T07:15:57.061 INFO:tasks.workunit.client.1.smithi069.stdout:5/109: creat d1/d7/f22 x:0 0 0 2024-03-22T07:15:57.061 INFO:tasks.workunit.client.1.smithi069.stdout:5/110: write d1/d7/d8/f12 [2934831,104900] 0 2024-03-22T07:15:57.062 INFO:tasks.workunit.client.1.smithi069.stdout:4/229: creat d6/d12/d18/f45 x:0 0 0 2024-03-22T07:15:57.063 INFO:tasks.workunit.client.0.smithi063.stdout:1/495: dwrite d4/df/d13/f40 [0,4194304] 0 2024-03-22T07:15:57.063 INFO:tasks.workunit.client.1.smithi069.stdout:6/131: mkdir d4/dd/d12/d2b 0 2024-03-22T07:15:57.064 INFO:tasks.workunit.client.0.smithi063.stdout:4/357: rmdir d1/d5/d43/d27 39 2024-03-22T07:15:57.066 INFO:tasks.workunit.client.1.smithi069.stdout:5/111: mkdir d1/d23 0 2024-03-22T07:15:57.067 INFO:tasks.workunit.client.1.smithi069.stdout:0/183: getdents d3/d7/d17 0 2024-03-22T07:15:57.067 INFO:tasks.workunit.client.1.smithi069.stdout:0/184: dread - d3/d7/d15/f32 zero size 2024-03-22T07:15:57.067 INFO:tasks.workunit.client.1.smithi069.stdout:0/185: write d3/d7/d15/f20 [3345600,91642] 0 2024-03-22T07:15:57.069 INFO:tasks.workunit.client.1.smithi069.stdout:4/230: truncate d6/dc/f1a 1998054 0 2024-03-22T07:15:57.069 INFO:tasks.workunit.client.1.smithi069.stdout:4/231: fsync d6/dc/d1d/d31/f14 0 2024-03-22T07:15:57.071 INFO:tasks.workunit.client.1.smithi069.stdout:9/173: dread d2/f5 [0,4194304] 0 2024-03-22T07:15:57.071 INFO:tasks.workunit.client.1.smithi069.stdout:9/174: read d2/dd/d18/d2e/f30 [1222217,28747] 0 2024-03-22T07:15:57.072 INFO:tasks.workunit.client.1.smithi069.stdout:0/186: dread d3/d7/fb [0,4194304] 0 2024-03-22T07:15:57.074 INFO:tasks.workunit.client.1.smithi069.stdout:5/112: rmdir d1/d23 0 2024-03-22T07:15:57.080 INFO:tasks.workunit.client.0.smithi063.stdout:0/537: rename da/l30 to da/de/d17/d25/d3c/d5f/d97/lad 0 2024-03-22T07:15:57.081 INFO:tasks.workunit.client.0.smithi063.stdout:6/459: fsync d3/d1c/f5d 0 2024-03-22T07:15:57.081 INFO:tasks.workunit.client.0.smithi063.stdout:6/460: creat d3/d15/d16/d60/d6d/d4e/d5f/f98 x:0 0 0 2024-03-22T07:15:57.085 INFO:tasks.workunit.client.0.smithi063.stdout:8/462: symlink d0/l8d 0 2024-03-22T07:15:57.088 INFO:tasks.workunit.client.0.smithi063.stdout:9/366: fsync d8/f1a 0 2024-03-22T07:15:57.091 INFO:tasks.workunit.client.0.smithi063.stdout:0/538: dread da/de/d17/f3a [0,4194304] 0 2024-03-22T07:15:57.091 INFO:tasks.workunit.client.0.smithi063.stdout:2/414: creat d0/de/d1b/f8c x:0 0 0 2024-03-22T07:15:57.091 INFO:tasks.workunit.client.0.smithi063.stdout:2/415: readlink d0/de/d1b/d63/l80 0 2024-03-22T07:15:57.094 INFO:tasks.workunit.client.0.smithi063.stdout:5/402: symlink d8/d13/d1e/d31/d58/l74 0 2024-03-22T07:15:57.094 INFO:tasks.workunit.client.0.smithi063.stdout:6/461: dread d3/f9 [4194304,4194304] 0 2024-03-22T07:15:57.096 INFO:tasks.workunit.client.0.smithi063.stdout:4/358: symlink d1/d5/d43/d27/d2c/d3d/d54/d5f/d79/l81 0 2024-03-22T07:15:57.098 INFO:tasks.workunit.client.0.smithi063.stdout:8/463: rmdir d0/d24/d33/d4c 39 2024-03-22T07:15:57.099 INFO:tasks.workunit.client.0.smithi063.stdout:9/367: creat d8/d19/d20/d22/d56/f71 x:0 0 0 2024-03-22T07:15:57.102 INFO:tasks.workunit.client.0.smithi063.stdout:6/462: mknod d3/d1c/d32/d51/c99 0 2024-03-22T07:15:57.102 INFO:tasks.workunit.client.0.smithi063.stdout:6/463: write d3/d15/f22 [4778122,38038] 0 2024-03-22T07:15:57.102 INFO:tasks.workunit.client.0.smithi063.stdout:6/464: creat d3/d15/d16/f9a x:0 0 0 2024-03-22T07:15:57.104 INFO:tasks.workunit.client.0.smithi063.stdout:4/359: unlink d1/f26 0 2024-03-22T07:15:57.105 INFO:tasks.workunit.client.0.smithi063.stdout:8/464: rename d0/l15 to d0/d1b/d28/d3e/l8e 0 2024-03-22T07:15:57.110 INFO:tasks.workunit.client.0.smithi063.stdout:9/368: symlink d8/l72 0 2024-03-22T07:15:57.122 INFO:tasks.workunit.client.1.smithi069.stdout:8/171: sync 2024-03-22T07:15:57.122 INFO:tasks.workunit.client.1.smithi069.stdout:2/152: sync 2024-03-22T07:15:57.122 INFO:tasks.workunit.client.1.smithi069.stdout:8/172: write d1/f14 [3334558,104041] 0 2024-03-22T07:15:57.123 INFO:tasks.workunit.client.1.smithi069.stdout:8/173: stat d1/de/c25 0 2024-03-22T07:15:57.125 INFO:tasks.workunit.client.1.smithi069.stdout:3/156: sync 2024-03-22T07:15:57.126 INFO:tasks.workunit.client.1.smithi069.stdout:8/174: dread d1/de/d12/f19 [0,4194304] 0 2024-03-22T07:15:57.126 INFO:tasks.workunit.client.1.smithi069.stdout:3/157: mkdir d5/d9/d1e/d35 0 2024-03-22T07:15:57.127 INFO:tasks.workunit.client.1.smithi069.stdout:3/158: truncate d5/d9/d1e/f22 5405733 0 2024-03-22T07:15:57.128 INFO:tasks.workunit.client.1.smithi069.stdout:8/175: getdents d1/de/d12 0 2024-03-22T07:15:57.128 INFO:tasks.workunit.client.1.smithi069.stdout:8/176: creat d1/de/f30 x:0 0 0 2024-03-22T07:15:57.128 INFO:tasks.workunit.client.1.smithi069.stdout:2/153: dread f5 [0,4194304] 0 2024-03-22T07:15:57.129 INFO:tasks.workunit.client.1.smithi069.stdout:8/177: mknod d1/c31 0 2024-03-22T07:15:57.130 INFO:tasks.workunit.client.1.smithi069.stdout:2/154: mkdir d18/d1c/d27/d33/d3d 0 2024-03-22T07:15:57.131 INFO:tasks.workunit.client.1.smithi069.stdout:8/178: creat d1/de/d12/f32 x:0 0 0 2024-03-22T07:15:57.131 INFO:tasks.workunit.client.1.smithi069.stdout:8/179: fsync d1/de/d12/f16 0 2024-03-22T07:15:57.131 INFO:tasks.workunit.client.1.smithi069.stdout:2/155: mkdir d18/d1c/d3e 0 2024-03-22T07:15:57.132 INFO:tasks.workunit.client.1.smithi069.stdout:2/156: getdents d18/d1c/d27 0 2024-03-22T07:15:57.133 INFO:tasks.workunit.client.1.smithi069.stdout:2/157: truncate d18/d1c/f24 1300046 0 2024-03-22T07:15:57.133 INFO:tasks.workunit.client.1.smithi069.stdout:2/158: dread - d18/d1c/d25/f36 zero size 2024-03-22T07:15:57.133 INFO:tasks.workunit.client.1.smithi069.stdout:2/159: chown d18/l23 11659482 1 2024-03-22T07:15:57.133 INFO:tasks.workunit.client.1.smithi069.stdout:2/160: stat d18/d1c/d27 0 2024-03-22T07:15:57.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.133+0000 7fa1e7d0a700 1 -- 172.21.15.63:0/2054574170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e0066b70 msgr2=0x7fa1e00ff9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:57.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.133+0000 7fa1e7d0a700 1 --2- 172.21.15.63:0/2054574170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e0066b70 0x7fa1e00ff9e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc004940 tx=0x7fa1dc009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:57.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.134+0000 7fa1e7d0a700 1 -- 172.21.15.63:0/2054574170 shutdown_connections 2024-03-22T07:15:57.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.134+0000 7fa1e7d0a700 1 --2- 172.21.15.63:0/2054574170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1e00fff20 0x7fa1e0102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:57.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.134+0000 7fa1e7d0a700 1 --2- 172.21.15.63:0/2054574170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e0066b70 0x7fa1e00ff9e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:57.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.134+0000 7fa1e7d0a700 1 -- 172.21.15.63:0/2054574170 >> 172.21.15.63:0/2054574170 conn(0x7fa1e00f7450 msgr2=0x7fa1e00f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:57.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.134+0000 7fa1e7d0a700 1 -- 172.21.15.63:0/2054574170 shutdown_connections 2024-03-22T07:15:57.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.134+0000 7fa1e7d0a700 1 -- 172.21.15.63:0/2054574170 wait complete. 2024-03-22T07:15:57.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.134+0000 7fa1e7d0a700 1 Processor -- start 2024-03-22T07:15:57.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.134+0000 7fa1e7d0a700 1 -- start start 2024-03-22T07:15:57.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.134+0000 7fa1e7d0a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1e0066b70 0x7fa1e010da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:57.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.134+0000 7fa1e7d0a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00fff20 0x7fa1e010dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:57.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.134+0000 7fa1e7d0a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e010e5b0 con 0x7fa1e00fff20 2024-03-22T07:15:57.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.134+0000 7fa1e7d0a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e010ed60 con 0x7fa1e0066b70 2024-03-22T07:15:57.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.134+0000 7fa1e52a5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00fff20 0x7fa1e010dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:57.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.134+0000 7fa1e52a5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00fff20 0x7fa1e010dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39778/0 (socket says 172.21.15.63:39778) 2024-03-22T07:15:57.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.134+0000 7fa1e52a5700 1 -- 172.21.15.63:0/778168914 learned_addr learned my addr 172.21.15.63:0/778168914 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:15:57.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.135+0000 7fa1e5aa6700 1 --2- 172.21.15.63:0/778168914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1e0066b70 0x7fa1e010da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:57.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.135+0000 7fa1e52a5700 1 -- 172.21.15.63:0/778168914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1e0066b70 msgr2=0x7fa1e010da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:57.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.135+0000 7fa1e52a5700 1 --2- 172.21.15.63:0/778168914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1e0066b70 0x7fa1e010da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:57.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.135+0000 7fa1e52a5700 1 -- 172.21.15.63:0/778168914 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc005040 con 0x7fa1e00fff20 2024-03-22T07:15:57.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.135+0000 7fa1e52a5700 1 --2- 172.21.15.63:0/778168914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00fff20 0x7fa1e010dfb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000eb40 tx=0x7fa1d000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:57.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.135+0000 7fa1d67fc700 1 -- 172.21.15.63:0/778168914 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1d0007cc0 con 0x7fa1e00fff20 2024-03-22T07:15:57.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.135+0000 7fa1d67fc700 1 -- 172.21.15.63:0/778168914 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa1d000f040 con 0x7fa1e00fff20 2024-03-22T07:15:57.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.135+0000 7fa1e7d0a700 1 -- 172.21.15.63:0/778168914 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e010efe0 con 0x7fa1e00fff20 2024-03-22T07:15:57.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.135+0000 7fa1d67fc700 1 -- 172.21.15.63:0/778168914 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1d000ca10 con 0x7fa1e00fff20 2024-03-22T07:15:57.137 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.135+0000 7fa1e7d0a700 1 -- 172.21.15.63:0/778168914 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e010f520 con 0x7fa1e00fff20 2024-03-22T07:15:57.139 INFO:tasks.workunit.client.1.smithi069.stdout:2/161: link f12 d18/d1c/d34/f3f 0 2024-03-22T07:15:57.141 INFO:tasks.workunit.client.1.smithi069.stdout:2/162: fdatasync d18/d1c/f26 0 2024-03-22T07:15:57.141 INFO:tasks.workunit.client.1.smithi069.stdout:2/163: write d18/d1c/d34/f3f [623167,44988] 0 2024-03-22T07:15:57.141 INFO:tasks.workunit.client.1.smithi069.stdout:2/164: truncate f1 939337 0 2024-03-22T07:15:57.141 INFO:tasks.workunit.client.1.smithi069.stdout:3/159: dread d5/f18 [0,4194304] 0 2024-03-22T07:15:57.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.136+0000 7fa1e7d0a700 1 -- 172.21.15.63:0/778168914 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e005f5f0 con 0x7fa1e00fff20 2024-03-22T07:15:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.137+0000 7fa1d67fc700 1 -- 172.21.15.63:0/778168914 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 20) v1 ==== 88751+0+0 (secure 0 0 0) 0x7fa1d000cc80 con 0x7fa1e00fff20 2024-03-22T07:15:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.137+0000 7fa1d67fc700 1 --2- 172.21.15.63:0/778168914 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa1cc06af90 0x7fa1cc06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.137+0000 7fa1d67fc700 1 -- 172.21.15.63:0/778168914 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa1d0089360 con 0x7fa1e00fff20 2024-03-22T07:15:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.138+0000 7fa1e5aa6700 1 --2- 172.21.15.63:0/778168914 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa1cc06af90 0x7fa1cc06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:57.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.138+0000 7fa1e5aa6700 1 --2- 172.21.15.63:0/778168914 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa1cc06af90 0x7fa1cc06d440 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc004890 tx=0x7fa1dc009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:57.142 INFO:tasks.workunit.client.0.smithi063.stdout:8/465: write d0/d1b/d28/f3b [3976133,93714] 0 2024-03-22T07:15:57.143 INFO:tasks.workunit.client.1.smithi069.stdout:4/232: dwrite d6/d12/f41 [0,4194304] 0 2024-03-22T07:15:57.143 INFO:tasks.workunit.client.1.smithi069.stdout:9/175: dwrite d2/dd/d18/d2e/f32 [0,4194304] 0 2024-03-22T07:15:57.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.144+0000 7fa1d67fc700 1 -- 172.21.15.63:0/778168914 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa1d0059100 con 0x7fa1e00fff20 2024-03-22T07:15:57.145 INFO:tasks.workunit.client.0.smithi063.stdout:9/369: dread d8/d17/f1d [0,4194304] 0 2024-03-22T07:15:57.147 INFO:tasks.workunit.client.1.smithi069.stdout:8/180: dread d1/de/d20/f2f [0,4194304] 0 2024-03-22T07:15:57.147 INFO:tasks.workunit.client.1.smithi069.stdout:8/181: write d1/de/d20/f26 [416359,4798] 0 2024-03-22T07:15:57.147 INFO:tasks.workunit.client.1.smithi069.stdout:9/176: creat d2/dd/f38 x:0 0 0 2024-03-22T07:15:57.147 INFO:tasks.workunit.client.1.smithi069.stdout:9/177: dread - d2/dd/f38 zero size 2024-03-22T07:15:57.149 INFO:tasks.workunit.client.1.smithi069.stdout:8/182: stat d1/de/l1b 0 2024-03-22T07:15:57.149 INFO:tasks.workunit.client.1.smithi069.stdout:8/183: fdatasync d1/de/f30 0 2024-03-22T07:15:57.149 INFO:tasks.workunit.client.1.smithi069.stdout:8/184: chown d1/de/d12/f19 941864686 1 2024-03-22T07:15:57.149 INFO:tasks.workunit.client.1.smithi069.stdout:8/185: chown d1/de/l1e 3 1 2024-03-22T07:15:57.149 INFO:tasks.workunit.client.1.smithi069.stdout:9/178: mknod d2/dd/d18/d2e/c39 0 2024-03-22T07:15:57.150 INFO:tasks.workunit.client.0.smithi063.stdout:1/496: dwrite d4/df/d13/d2b/d3a/f42 [0,4194304] 0 2024-03-22T07:15:57.150 INFO:tasks.workunit.client.0.smithi063.stdout:8/466: dread d0/d1b/d28/d7b/d45/f58 [0,4194304] 0 2024-03-22T07:15:57.150 INFO:tasks.workunit.client.0.smithi063.stdout:8/467: fsync d0/f22 0 2024-03-22T07:15:57.151 INFO:tasks.workunit.client.0.smithi063.stdout:1/497: mkdir d4/df/d13/d2b/d6c/da8 0 2024-03-22T07:15:57.151 INFO:tasks.workunit.client.0.smithi063.stdout:1/498: stat d4/d14/f62 0 2024-03-22T07:15:57.152 INFO:tasks.workunit.client.1.smithi069.stdout:6/132: dwrite d4/dd/d12/f1d [0,4194304] 0 2024-03-22T07:15:57.152 INFO:tasks.workunit.client.1.smithi069.stdout:0/187: dwrite d3/d7/d15/d1f/f21 [4194304,4194304] 0 2024-03-22T07:15:57.152 INFO:tasks.workunit.client.0.smithi063.stdout:8/468: rmdir d0/d1f/d7a 0 2024-03-22T07:15:57.153 INFO:tasks.workunit.client.1.smithi069.stdout:8/186: dread d1/de/d12/f19 [0,4194304] 0 2024-03-22T07:15:57.154 INFO:tasks.workunit.client.0.smithi063.stdout:8/469: mkdir d0/d26/d2d/d8f 0 2024-03-22T07:15:57.154 INFO:tasks.workunit.client.0.smithi063.stdout:8/470: chown d0/d26/d2d/d8f 30 1 2024-03-22T07:15:57.154 INFO:tasks.workunit.client.0.smithi063.stdout:2/416: dwrite d0/de/d12/d5a/f81 [0,4194304] 0 2024-03-22T07:15:57.154 INFO:tasks.workunit.client.0.smithi063.stdout:5/403: dwrite d8/d13/f5d [0,4194304] 0 2024-03-22T07:15:57.156 INFO:tasks.workunit.client.0.smithi063.stdout:0/539: dwrite da/de/d17/d25/d3c/d43/d76/f9d [0,4194304] 0 2024-03-22T07:15:57.156 INFO:tasks.workunit.client.0.smithi063.stdout:2/417: unlink d0/de/f4e 0 2024-03-22T07:15:57.158 INFO:tasks.workunit.client.0.smithi063.stdout:4/360: dwrite d1/d5/d38/d59/f71 [0,4194304] 0 2024-03-22T07:15:57.158 INFO:tasks.workunit.client.0.smithi063.stdout:4/361: write d1/d5/d38/f44 [4130937,109720] 0 2024-03-22T07:15:57.159 INFO:tasks.workunit.client.0.smithi063.stdout:4/362: fdatasync d1/d5/dc/f41 0 2024-03-22T07:15:57.159 INFO:tasks.workunit.client.1.smithi069.stdout:6/133: dread d4/fb [4194304,4194304] 0 2024-03-22T07:15:57.159 INFO:tasks.workunit.client.1.smithi069.stdout:6/134: chown d4/d7/ce 481209473 1 2024-03-22T07:15:57.161 INFO:tasks.workunit.client.1.smithi069.stdout:4/233: write d6/dc/d23/f3c [1944560,126248] 0 2024-03-22T07:15:57.162 INFO:tasks.workunit.client.0.smithi063.stdout:2/418: symlink d0/de/d1b/d63/d78/l8d 0 2024-03-22T07:15:57.162 INFO:tasks.workunit.client.0.smithi063.stdout:2/419: write d0/de/d12/d20/f5d [1173774,72793] 0 2024-03-22T07:15:57.162 INFO:tasks.workunit.client.0.smithi063.stdout:2/420: write d0/de/d12/d20/f2a [3519731,105465] 0 2024-03-22T07:15:57.162 INFO:tasks.workunit.client.0.smithi063.stdout:8/471: dread d0/d24/f35 [0,4194304] 0 2024-03-22T07:15:57.163 INFO:tasks.workunit.client.1.smithi069.stdout:8/187: unlink d1/de/c13 0 2024-03-22T07:15:57.163 INFO:tasks.workunit.client.1.smithi069.stdout:5/113: dwrite d1/d7/d8/f12 [4194304,4194304] 0 2024-03-22T07:15:57.163 INFO:tasks.workunit.client.1.smithi069.stdout:5/114: creat d1/d7/d8/f24 x:0 0 0 2024-03-22T07:15:57.163 INFO:tasks.workunit.client.1.smithi069.stdout:5/115: creat d1/d7/f25 x:0 0 0 2024-03-22T07:15:57.170 INFO:tasks.workunit.client.1.smithi069.stdout:5/116: mkdir d1/d6/d26 0 2024-03-22T07:15:57.171 INFO:tasks.workunit.client.1.smithi069.stdout:5/117: write d1/d7/f25 [847262,104502] 0 2024-03-22T07:15:57.171 INFO:tasks.workunit.client.1.smithi069.stdout:5/118: creat d1/d7/d8/f27 x:0 0 0 2024-03-22T07:15:57.174 INFO:tasks.workunit.client.1.smithi069.stdout:5/119: mkdir d1/d28 0 2024-03-22T07:15:57.174 INFO:tasks.workunit.client.1.smithi069.stdout:5/120: write d1/d7/f22 [751600,92772] 0 2024-03-22T07:15:57.178 INFO:tasks.workunit.client.0.smithi063.stdout:9/370: dwrite f7 [0,4194304] 0 2024-03-22T07:15:57.178 INFO:tasks.workunit.client.0.smithi063.stdout:9/371: fdatasync d8/f54 0 2024-03-22T07:15:57.178 INFO:tasks.workunit.client.0.smithi063.stdout:9/372: chown d8/d19/f23 1744103 1 2024-03-22T07:15:57.178 INFO:tasks.workunit.client.0.smithi063.stdout:0/540: dread da/de/d17/d25/d3c/d64/fac [0,4194304] 0 2024-03-22T07:15:57.179 INFO:tasks.workunit.client.0.smithi063.stdout:2/421: link d0/de/d1b/d32/c83 d0/de/d1b/d8b/c8e 0 2024-03-22T07:15:57.179 INFO:tasks.workunit.client.0.smithi063.stdout:8/472: mknod d0/d24/d33/c90 0 2024-03-22T07:15:57.179 INFO:tasks.workunit.client.0.smithi063.stdout:8/473: chown d0/d1b/d28/d7b/c1d 402376 1 2024-03-22T07:15:57.180 INFO:tasks.workunit.client.1.smithi069.stdout:9/179: truncate d2/dd/f10 2498815 0 2024-03-22T07:15:57.182 INFO:tasks.workunit.client.0.smithi063.stdout:9/373: readlink d8/l4c 0 2024-03-22T07:15:57.182 INFO:tasks.workunit.client.0.smithi063.stdout:9/374: write d8/d17/f69 [633564,18741] 0 2024-03-22T07:15:57.182 INFO:tasks.workunit.client.0.smithi063.stdout:9/375: creat d8/d17/f73 x:0 0 0 2024-03-22T07:15:57.182 INFO:tasks.workunit.client.0.smithi063.stdout:0/541: mknod da/de/d17/d4e/cae 0 2024-03-22T07:15:57.187 INFO:tasks.workunit.client.1.smithi069.stdout:9/180: creat d2/d21/f3a x:0 0 0 2024-03-22T07:15:57.188 INFO:tasks.workunit.client.1.smithi069.stdout:2/165: getdents d18/d1c/d27/d33 0 2024-03-22T07:15:57.188 INFO:tasks.workunit.client.1.smithi069.stdout:9/181: creat d2/dd/d18/f3b x:0 0 0 2024-03-22T07:15:57.188 INFO:tasks.workunit.client.1.smithi069.stdout:9/182: readlink d2/dd/l2a 0 2024-03-22T07:15:57.188 INFO:tasks.workunit.client.1.smithi069.stdout:2/166: write d18/f1e [5123204,89381] 0 2024-03-22T07:15:57.188 INFO:tasks.workunit.client.0.smithi063.stdout:8/474: unlink d0/d1b/d28/f3b 0 2024-03-22T07:15:57.191 INFO:tasks.workunit.client.0.smithi063.stdout:9/376: rename f7 to d8/d19/d41/f74 0 2024-03-22T07:15:57.191 INFO:tasks.workunit.client.0.smithi063.stdout:9/377: fsync d8/d19/d20/d22/d29/d31/f4e 0 2024-03-22T07:15:57.191 INFO:tasks.workunit.client.0.smithi063.stdout:9/378: creat d8/d19/d20/d22/d56/f75 x:0 0 0 2024-03-22T07:15:57.191 INFO:tasks.workunit.client.0.smithi063.stdout:9/379: chown d8/d19/d2b/l6d 194 1 2024-03-22T07:15:57.193 INFO:tasks.workunit.client.0.smithi063.stdout:0/542: creat da/de/d60/faf x:0 0 0 2024-03-22T07:15:57.196 INFO:tasks.workunit.client.0.smithi063.stdout:8/475: creat d0/d24/d33/d4c/d78/f91 x:0 0 0 2024-03-22T07:15:57.196 INFO:tasks.workunit.client.0.smithi063.stdout:6/465: fsync d3/d15/d16/f9a 0 2024-03-22T07:15:57.202 INFO:tasks.workunit.client.0.smithi063.stdout:0/543: mknod da/de/d17/cb0 0 2024-03-22T07:15:57.203 INFO:tasks.workunit.client.0.smithi063.stdout:8/476: unlink d0/d24/d5b/c6d 0 2024-03-22T07:15:57.205 INFO:tasks.workunit.client.0.smithi063.stdout:0/544: mknod da/de/d17/d25/d95/d98/cb1 0 2024-03-22T07:15:57.206 INFO:tasks.workunit.client.0.smithi063.stdout:6/466: symlink d3/l9b 0 2024-03-22T07:15:57.207 INFO:tasks.workunit.client.0.smithi063.stdout:6/467: truncate d3/d15/d16/f59 309597 0 2024-03-22T07:15:57.238 INFO:tasks.workunit.client.0.smithi063.stdout:1/499: dwrite d4/df/d2c/f54 [0,4194304] 0 2024-03-22T07:15:57.239 INFO:tasks.workunit.client.1.smithi069.stdout:4/234: dwrite d6/dc/d1d/d31/d1f/f42 [0,4194304] 0 2024-03-22T07:15:57.240 INFO:tasks.workunit.client.0.smithi063.stdout:1/500: rmdir d4/df/d13/d2b/d6c/d85/d9e 39 2024-03-22T07:15:57.240 INFO:tasks.workunit.client.0.smithi063.stdout:1/501: dread - d4/d8b/f97 zero size 2024-03-22T07:15:57.243 INFO:tasks.workunit.client.0.smithi063.stdout:8/477: write d0/d24/f5d [2922588,118872] 0 2024-03-22T07:15:57.243 INFO:tasks.workunit.client.0.smithi063.stdout:8/478: write d0/d1b/f84 [340243,28786] 0 2024-03-22T07:15:57.243 INFO:tasks.workunit.client.0.smithi063.stdout:1/502: rename d4/df/d2c/f4b to d4/d5a/fa9 0 2024-03-22T07:15:57.243 INFO:tasks.workunit.client.0.smithi063.stdout:1/503: read - d4/d14/d57/f80 zero size 2024-03-22T07:15:57.243 INFO:tasks.workunit.client.0.smithi063.stdout:1/504: readlink d4/d5a/l95 0 2024-03-22T07:15:57.246 INFO:tasks.workunit.client.0.smithi063.stdout:8/479: creat d0/d26/f92 x:0 0 0 2024-03-22T07:15:57.246 INFO:tasks.workunit.client.0.smithi063.stdout:1/505: rmdir d4/df/d13/d2b/d6c/d85/d9e 39 2024-03-22T07:15:57.247 INFO:tasks.workunit.client.0.smithi063.stdout:1/506: readlink d4/df/d13/d2b/d3a/l7b 0 2024-03-22T07:15:57.247 INFO:tasks.workunit.client.0.smithi063.stdout:5/404: dwrite d8/d13/d1e/f38 [0,4194304] 0 2024-03-22T07:15:57.248 INFO:tasks.workunit.client.1.smithi069.stdout:8/188: dwrite d1/de/d20/f2f [0,4194304] 0 2024-03-22T07:15:57.249 INFO:tasks.workunit.client.1.smithi069.stdout:5/121: dwrite d1/f1f [0,4194304] 0 2024-03-22T07:15:57.249 INFO:tasks.workunit.client.1.smithi069.stdout:5/122: chown d1/f3 2 1 2024-03-22T07:15:57.249 INFO:tasks.workunit.client.1.smithi069.stdout:3/160: dwrite d5/d9/d1e/f22 [0,4194304] 0 2024-03-22T07:15:57.250 INFO:tasks.workunit.client.0.smithi063.stdout:6/468: dwrite d3/d6/d2a/d38/f4d [0,4194304] 0 2024-03-22T07:15:57.250 INFO:tasks.workunit.client.0.smithi063.stdout:2/422: dwrite d0/de/d1b/d32/f54 [0,4194304] 0 2024-03-22T07:15:57.250 INFO:tasks.workunit.client.0.smithi063.stdout:4/363: dwrite d1/d5/d43/d27/d2c/d3d/f62 [0,4194304] 0 2024-03-22T07:15:57.250 INFO:tasks.workunit.client.0.smithi063.stdout:4/364: fdatasync d1/d5/f2f 0 2024-03-22T07:15:57.250 INFO:tasks.workunit.client.0.smithi063.stdout:4/365: creat d1/d5/d3c/f82 x:0 0 0 2024-03-22T07:15:57.250 INFO:tasks.workunit.client.0.smithi063.stdout:4/366: chown d1/d5/f12 1545576 1 2024-03-22T07:15:57.250 INFO:tasks.workunit.client.0.smithi063.stdout:4/367: creat d1/d5/d3c/f83 x:0 0 0 2024-03-22T07:15:57.250 INFO:tasks.workunit.client.0.smithi063.stdout:8/480: getdents d0/d24/d33/d4c 0 2024-03-22T07:15:57.251 INFO:tasks.workunit.client.1.smithi069.stdout:8/189: symlink d1/d1c/l33 0 2024-03-22T07:15:57.251 INFO:tasks.workunit.client.1.smithi069.stdout:8/190: readlink d1/de/l1b 0 2024-03-22T07:15:57.251 INFO:tasks.workunit.client.1.smithi069.stdout:8/191: read d1/f9 [1796581,5883] 0 2024-03-22T07:15:57.251 INFO:tasks.workunit.client.0.smithi063.stdout:7/337: sync 2024-03-22T07:15:57.251 INFO:tasks.workunit.client.0.smithi063.stdout:0/545: dwrite da/de/d60/faf [0,4194304] 0 2024-03-22T07:15:57.253 INFO:tasks.workunit.client.0.smithi063.stdout:3/494: sync 2024-03-22T07:15:57.255 INFO:tasks.workunit.client.1.smithi069.stdout:5/123: truncate d1/f3 370879 0 2024-03-22T07:15:57.257 INFO:tasks.workunit.client.1.smithi069.stdout:4/235: dread d6/f9 [0,4194304] 0 2024-03-22T07:15:57.257 INFO:tasks.workunit.client.1.smithi069.stdout:2/167: dwrite d18/d1c/f22 [4194304,4194304] 0 2024-03-22T07:15:57.259 INFO:tasks.workunit.client.1.smithi069.stdout:8/192: truncate d1/de/f2d 1594610 0 2024-03-22T07:15:57.259 INFO:tasks.workunit.client.1.smithi069.stdout:8/193: write d1/de/f30 [123642,41282] 0 2024-03-22T07:15:57.260 INFO:tasks.workunit.client.0.smithi063.stdout:6/469: symlink d3/d1c/d32/d51/l9c 0 2024-03-22T07:15:57.260 INFO:tasks.workunit.client.0.smithi063.stdout:6/470: chown d3/d15/d16/d60/d6d/d4e/d5f/l80 1 1 2024-03-22T07:15:57.264 INFO:tasks.workunit.client.1.smithi069.stdout:4/236: dread d6/d12/d18/d21/f2f [0,4194304] 0 2024-03-22T07:15:57.265 INFO:tasks.workunit.client.0.smithi063.stdout:8/481: dread d0/fc [0,4194304] 0 2024-03-22T07:15:57.267 INFO:tasks.workunit.client.1.smithi069.stdout:1/213: sync 2024-03-22T07:15:57.268 INFO:tasks.workunit.client.0.smithi063.stdout:9/380: getdents d8/d17 0 2024-03-22T07:15:57.271 INFO:tasks.workunit.client.1.smithi069.stdout:4/237: unlink d6/dc/d1d/d31/f17 0 2024-03-22T07:15:57.271 INFO:tasks.workunit.client.0.smithi063.stdout:7/338: mknod d5/da/d5f/c6b 0 2024-03-22T07:15:57.273 INFO:tasks.workunit.client.1.smithi069.stdout:1/214: mknod d1/c46 0 2024-03-22T07:15:57.273 INFO:tasks.workunit.client.0.smithi063.stdout:5/405: dread d8/d13/f5d [0,4194304] 0 2024-03-22T07:15:57.275 INFO:tasks.workunit.client.1.smithi069.stdout:9/183: rename d2/dd/d18 to d2/dd/d3c 0 2024-03-22T07:15:57.276 INFO:tasks.workunit.client.0.smithi063.stdout:3/495: mknod da/d63/d65/d92/d96/ca1 0 2024-03-22T07:15:57.278 INFO:tasks.workunit.client.1.smithi069.stdout:4/238: truncate d6/f9 3428837 0 2024-03-22T07:15:57.278 INFO:tasks.workunit.client.0.smithi063.stdout:6/471: unlink d3/d6/c91 0 2024-03-22T07:15:57.280 INFO:tasks.workunit.client.1.smithi069.stdout:1/215: mknod d1/d13/c47 0 2024-03-22T07:15:57.280 INFO:tasks.workunit.client.1.smithi069.stdout:1/216: creat d1/f48 x:0 0 0 2024-03-22T07:15:57.283 INFO:tasks.workunit.client.0.smithi063.stdout:8/482: mknod d0/d1b/c93 0 2024-03-22T07:15:57.283 INFO:tasks.workunit.client.0.smithi063.stdout:8/483: chown d0/d24/f42 24399954 1 2024-03-22T07:15:57.283 INFO:tasks.workunit.client.0.smithi063.stdout:7/339: getdents d5/d14/d1d/d29/d4a 0 2024-03-22T07:15:57.283 INFO:tasks.workunit.client.0.smithi063.stdout:7/340: truncate d5/da/f25 2087125 0 2024-03-22T07:15:57.284 INFO:tasks.workunit.client.1.smithi069.stdout:9/184: rename d2/lf to d2/l3d 0 2024-03-22T07:15:57.284 INFO:tasks.workunit.client.1.smithi069.stdout:9/185: read - d2/dd/f38 zero size 2024-03-22T07:15:57.284 INFO:tasks.workunit.client.1.smithi069.stdout:4/239: creat d6/dc/d1d/d31/d1f/d3f/f46 x:0 0 0 2024-03-22T07:15:57.284 INFO:tasks.workunit.client.1.smithi069.stdout:9/186: truncate d2/dd/f26 528056 0 2024-03-22T07:15:57.285 INFO:tasks.workunit.client.1.smithi069.stdout:4/240: dread d6/d12/d18/d21/f2f [0,4194304] 0 2024-03-22T07:15:57.285 INFO:tasks.workunit.client.1.smithi069.stdout:4/241: readlink d6/dc/l44 0 2024-03-22T07:15:57.286 INFO:tasks.workunit.client.0.smithi063.stdout:6/472: unlink d3/d15/d16/f26 0 2024-03-22T07:15:57.286 INFO:tasks.workunit.client.1.smithi069.stdout:9/187: unlink d2/dd/l2d 0 2024-03-22T07:15:57.286 INFO:tasks.workunit.client.1.smithi069.stdout:4/242: mknod d6/d12/d18/c47 0 2024-03-22T07:15:57.287 INFO:tasks.workunit.client.0.smithi063.stdout:8/484: creat d0/d24/d33/d4c/f94 x:0 0 0 2024-03-22T07:15:57.288 INFO:tasks.workunit.client.1.smithi069.stdout:4/243: mknod d6/c48 0 2024-03-22T07:15:57.288 INFO:tasks.workunit.client.1.smithi069.stdout:4/244: chown d6/l7 0 1 2024-03-22T07:15:57.288 INFO:tasks.workunit.client.1.smithi069.stdout:4/245: chown d6/d12/f41 449 1 2024-03-22T07:15:57.288 INFO:tasks.workunit.client.1.smithi069.stdout:4/246: creat d6/d12/d18/d21/f49 x:0 0 0 2024-03-22T07:15:57.288 INFO:tasks.workunit.client.1.smithi069.stdout:4/247: dread - d6/d12/f33 zero size 2024-03-22T07:15:57.289 INFO:tasks.workunit.client.1.smithi069.stdout:4/248: stat d6/dc/c3e 0 2024-03-22T07:15:57.290 INFO:tasks.workunit.client.0.smithi063.stdout:9/381: rmdir d8/d19/d41/d48 39 2024-03-22T07:15:57.291 INFO:tasks.workunit.client.0.smithi063.stdout:9/382: chown d8/d19/d20/d22/d29/d31/d37 0 1 2024-03-22T07:15:57.291 INFO:tasks.workunit.client.0.smithi063.stdout:5/406: mknod d8/d13/d24/d62/c75 0 2024-03-22T07:15:57.291 INFO:tasks.workunit.client.0.smithi063.stdout:5/407: fsync d8/d13/d1e/d31/f52 0 2024-03-22T07:15:57.291 INFO:tasks.workunit.client.0.smithi063.stdout:6/473: mknod d3/d6/d2a/d38/c9d 0 2024-03-22T07:15:57.291 INFO:tasks.workunit.client.0.smithi063.stdout:6/474: write d3/d15/d16/d60/d6d/d4e/d5f/f98 [148159,36000] 0 2024-03-22T07:15:57.291 INFO:tasks.workunit.client.0.smithi063.stdout:6/475: chown d3/d1c/d32 86734258 1 2024-03-22T07:15:57.291 INFO:tasks.workunit.client.0.smithi063.stdout:8/485: creat d0/d24/d5b/f95 x:0 0 0 2024-03-22T07:15:57.292 INFO:tasks.workunit.client.0.smithi063.stdout:9/383: mkdir d8/d19/d20/d22/d5d/d76 0 2024-03-22T07:15:57.292 INFO:tasks.workunit.client.0.smithi063.stdout:9/384: readlink d8/d17/d1e/l1f 0 2024-03-22T07:15:57.293 INFO:tasks.workunit.client.0.smithi063.stdout:5/408: symlink d8/d13/d1e/d3c/l76 0 2024-03-22T07:15:57.293 INFO:tasks.workunit.client.0.smithi063.stdout:5/409: readlink d8/d13/d28/l40 0 2024-03-22T07:15:57.293 INFO:tasks.workunit.client.0.smithi063.stdout:8/486: symlink d0/d24/d33/l96 0 2024-03-22T07:15:57.294 INFO:tasks.workunit.client.0.smithi063.stdout:9/385: symlink d8/d19/d20/d22/d56/l77 0 2024-03-22T07:15:57.294 INFO:tasks.workunit.client.0.smithi063.stdout:9/386: truncate d8/d17/f43 623782 0 2024-03-22T07:15:57.295 INFO:tasks.workunit.client.1.smithi069.stdout:4/249: dread d6/dc/d23/f3c [0,4194304] 0 2024-03-22T07:15:57.296 INFO:tasks.workunit.client.0.smithi063.stdout:6/476: dread d3/d15/f85 [0,4194304] 0 2024-03-22T07:15:57.296 INFO:tasks.workunit.client.0.smithi063.stdout:6/477: truncate d3/d15/f7f 720184 0 2024-03-22T07:15:57.297 INFO:tasks.workunit.client.0.smithi063.stdout:9/387: getdents d8/d19/d20/d22/d29/d31 0 2024-03-22T07:15:57.297 INFO:tasks.workunit.client.0.smithi063.stdout:9/388: write d8/d19/d41/f6b [251887,64901] 0 2024-03-22T07:15:57.297 INFO:tasks.workunit.client.0.smithi063.stdout:9/389: creat d8/d19/d20/d22/d29/d31/d4b/f78 x:0 0 0 2024-03-22T07:15:57.297 INFO:tasks.workunit.client.0.smithi063.stdout:9/390: dread - d8/d19/d39/f6f zero size 2024-03-22T07:15:57.298 INFO:tasks.workunit.client.1.smithi069.stdout:4/250: mkdir d6/dc/d1d/d31/d4a 0 2024-03-22T07:15:57.299 INFO:tasks.workunit.client.0.smithi063.stdout:6/478: unlink d3/d6/d2a/d38/f61 0 2024-03-22T07:15:57.299 INFO:tasks.workunit.client.0.smithi063.stdout:6/479: fdatasync d3/d6/d35/f94 0 2024-03-22T07:15:57.300 INFO:tasks.workunit.client.0.smithi063.stdout:9/391: symlink d8/d15/d63/l79 0 2024-03-22T07:15:57.301 INFO:tasks.workunit.client.0.smithi063.stdout:9/392: creat d8/d15/d2f/f7a x:0 0 0 2024-03-22T07:15:57.302 INFO:tasks.workunit.client.1.smithi069.stdout:4/251: dread d6/dc/f37 [0,4194304] 0 2024-03-22T07:15:57.303 INFO:tasks.workunit.client.0.smithi063.stdout:3/496: read da/d63/d65/f77 [111404,65571] 0 2024-03-22T07:15:57.303 INFO:tasks.workunit.client.0.smithi063.stdout:3/497: creat da/d63/d65/d81/d9b/d9f/fa2 x:0 0 0 2024-03-22T07:15:57.305 INFO:tasks.workunit.client.0.smithi063.stdout:3/498: symlink da/d63/d65/d86/la3 0 2024-03-22T07:15:57.305 INFO:tasks.workunit.client.1.smithi069.stdout:4/252: mkdir d6/d4b 0 2024-03-22T07:15:57.306 INFO:tasks.workunit.client.1.smithi069.stdout:4/253: mknod d6/d4b/c4c 0 2024-03-22T07:15:57.306 INFO:tasks.workunit.client.1.smithi069.stdout:4/254: dread - d6/d12/d18/f45 zero size 2024-03-22T07:15:57.309 INFO:tasks.workunit.client.1.smithi069.stdout:3/161: dwrite d5/d9/d26/f28 [0,4194304] 0 2024-03-22T07:15:57.319 INFO:tasks.workunit.client.0.smithi063.stdout:0/546: dwrite f5 [4194304,4194304] 0 2024-03-22T07:15:57.321 INFO:tasks.workunit.client.0.smithi063.stdout:1/507: dwrite d4/df/d13/d2b/d86/f87 [0,4194304] 0 2024-03-22T07:15:57.321 INFO:tasks.workunit.client.0.smithi063.stdout:1/508: dread - d4/df/d13/d2b/f84 zero size 2024-03-22T07:15:57.322 INFO:tasks.workunit.client.0.smithi063.stdout:3/499: write da/d16/d26/f8b [2495575,62970] 0 2024-03-22T07:15:57.325 INFO:tasks.workunit.client.0.smithi063.stdout:3/500: creat da/d63/d65/d81/fa4 x:0 0 0 2024-03-22T07:15:57.326 INFO:tasks.workunit.client.0.smithi063.stdout:3/501: mkdir da/d63/d65/d81/da5 0 2024-03-22T07:15:57.328 INFO:tasks.workunit.client.0.smithi063.stdout:3/502: link da/d63/d65/d86/c8e da/d16/d26/d69/ca6 0 2024-03-22T07:15:57.328 INFO:tasks.workunit.client.0.smithi063.stdout:3/503: write da/d16/d89/d8c/f9e [561672,94612] 0 2024-03-22T07:15:57.330 INFO:tasks.workunit.client.0.smithi063.stdout:3/504: creat da/d16/d26/fa7 x:0 0 0 2024-03-22T07:15:57.330 INFO:tasks.workunit.client.0.smithi063.stdout:3/505: rmdir da/d63/d65/d7b 39 2024-03-22T07:15:57.331 INFO:tasks.workunit.client.0.smithi063.stdout:3/506: rmdir da/d63/d65 39 2024-03-22T07:15:57.332 INFO:tasks.workunit.client.0.smithi063.stdout:3/507: unlink da/f21 0 2024-03-22T07:15:57.333 INFO:tasks.workunit.client.1.smithi069.stdout:2/168: dwrite d18/f2b [0,4194304] 0 2024-03-22T07:15:57.335 INFO:tasks.workunit.client.0.smithi063.stdout:1/509: write d4/df/d13/f3c [4058216,41617] 0 2024-03-22T07:15:57.337 INFO:tasks.workunit.client.1.smithi069.stdout:2/169: rmdir d18/d1c/d34 39 2024-03-22T07:15:57.337 INFO:tasks.workunit.client.1.smithi069.stdout:2/170: chown d18/f2b 6361580 1 2024-03-22T07:15:57.338 INFO:tasks.workunit.client.0.smithi063.stdout:1/510: creat d4/d8b/da7/faa x:0 0 0 2024-03-22T07:15:57.338 INFO:tasks.workunit.client.0.smithi063.stdout:1/511: chown d4/d5a/f65 0 1 2024-03-22T07:15:57.340 INFO:tasks.workunit.client.1.smithi069.stdout:2/171: rmdir d18 39 2024-03-22T07:15:57.340 INFO:tasks.workunit.client.1.smithi069.stdout:2/172: truncate d18/d1c/f30 1436994 0 2024-03-22T07:15:57.341 INFO:tasks.workunit.client.1.smithi069.stdout:2/173: mkdir d18/d1c/d3e/d40 0 2024-03-22T07:15:57.341 INFO:tasks.workunit.client.0.smithi063.stdout:1/512: rename d4/df/d13/d2b/d3a/f6e to d4/df/d13/d2b/d6c/d85/fab 0 2024-03-22T07:15:57.346 INFO:tasks.workunit.client.0.smithi063.stdout:1/513: rename d4/df/l19 to d4/df/d13/d2b/d6c/d85/d9e/lac 0 2024-03-22T07:15:57.346 INFO:tasks.workunit.client.0.smithi063.stdout:1/514: truncate d4/f1f 4970602 0 2024-03-22T07:15:57.352 INFO:tasks.workunit.client.0.smithi063.stdout:2/423: dwrite d0/f51 [0,4194304] 0 2024-03-22T07:15:57.356 INFO:tasks.workunit.client.1.smithi069.stdout:2/174: write d18/d1c/f2d [1732140,105549] 0 2024-03-22T07:15:57.360 INFO:tasks.workunit.client.0.smithi063.stdout:4/368: dwrite d1/d5/d43/f29 [0,4194304] 0 2024-03-22T07:15:57.366 INFO:tasks.workunit.client.0.smithi063.stdout:9/393: dwrite d8/d19/d20/d22/d29/d31/f32 [0,4194304] 0 2024-03-22T07:15:57.366 INFO:tasks.workunit.client.0.smithi063.stdout:9/394: write d8/d17/f43 [684864,115731] 0 2024-03-22T07:15:57.366 INFO:tasks.workunit.client.0.smithi063.stdout:9/395: dread - d8/d19/d39/f6f zero size 2024-03-22T07:15:57.366 INFO:tasks.workunit.client.0.smithi063.stdout:5/410: dwrite d8/d13/f5d [0,4194304] 0 2024-03-22T07:15:57.366 INFO:tasks.workunit.client.0.smithi063.stdout:1/515: dwrite d4/df/d2c/f90 [4194304,4194304] 0 2024-03-22T07:15:57.366 INFO:tasks.workunit.client.0.smithi063.stdout:3/508: dwrite da/d16/f6d [0,4194304] 0 2024-03-22T07:15:57.367 INFO:tasks.workunit.client.0.smithi063.stdout:0/547: dwrite da/de/d17/f2f [4194304,4194304] 0 2024-03-22T07:15:57.367 INFO:tasks.workunit.client.0.smithi063.stdout:0/548: fsync da/de/d17/d25/d3c/d43/d76/f9d 0 2024-03-22T07:15:57.367 INFO:tasks.workunit.client.0.smithi063.stdout:0/549: write da/de/d17/d25/f70 [205472,106209] 0 2024-03-22T07:15:57.367 INFO:tasks.workunit.client.0.smithi063.stdout:0/550: fsync da/d49/f6d 0 2024-03-22T07:15:57.367 INFO:tasks.workunit.client.1.smithi069.stdout:4/255: dwrite d6/dc/f37 [0,4194304] 0 2024-03-22T07:15:57.367 INFO:tasks.workunit.client.1.smithi069.stdout:5/124: dwrite d1/f1f [4194304,4194304] 0 2024-03-22T07:15:57.367 INFO:tasks.workunit.client.1.smithi069.stdout:3/162: dwrite d5/d9/f13 [0,4194304] 0 2024-03-22T07:15:57.368 INFO:tasks.workunit.client.1.smithi069.stdout:4/256: creat d6/d4b/f4d x:0 0 0 2024-03-22T07:15:57.368 INFO:tasks.workunit.client.1.smithi069.stdout:4/257: fdatasync d6/dc/d23/f3c 0 2024-03-22T07:15:57.370 INFO:tasks.workunit.client.0.smithi063.stdout:2/424: write d0/de/d12/d20/f3f [688456,82181] 0 2024-03-22T07:15:57.370 INFO:tasks.workunit.client.0.smithi063.stdout:8/487: dwrite d0/fc [0,4194304] 0 2024-03-22T07:15:57.370 INFO:tasks.workunit.client.0.smithi063.stdout:8/488: dread - d0/d1b/d28/d7b/f7f zero size 2024-03-22T07:15:57.371 INFO:tasks.workunit.client.0.smithi063.stdout:9/396: mkdir d8/d19/d41/d48/d7b 0 2024-03-22T07:15:57.371 INFO:tasks.workunit.client.0.smithi063.stdout:9/397: write d8/d19/d20/d22/d29/d31/f4e [1437425,26604] 0 2024-03-22T07:15:57.371 INFO:tasks.workunit.client.1.smithi069.stdout:5/125: symlink d1/d10/l29 0 2024-03-22T07:15:57.375 INFO:tasks.workunit.client.1.smithi069.stdout:8/194: read d1/de/f1a [518833,91733] 0 2024-03-22T07:15:57.380 INFO:tasks.workunit.client.1.smithi069.stdout:3/163: mkdir d5/d9/d26/d36 0 2024-03-22T07:15:57.381 INFO:tasks.workunit.client.1.smithi069.stdout:5/126: stat d1/d7/d8/c1a 0 2024-03-22T07:15:57.382 INFO:tasks.workunit.client.1.smithi069.stdout:5/127: chown d1/d7/c1d 153943 1 2024-03-22T07:15:57.385 INFO:tasks.workunit.client.0.smithi063.stdout:3/509: rmdir da/d16/d89 39 2024-03-22T07:15:57.385 INFO:tasks.workunit.client.0.smithi063.stdout:3/510: readlink da/l41 0 2024-03-22T07:15:57.385 INFO:tasks.workunit.client.0.smithi063.stdout:0/551: dread da/de/d17/f1a [0,4194304] 0 2024-03-22T07:15:57.385 INFO:tasks.workunit.client.0.smithi063.stdout:0/552: truncate da/de/d17/f45 126148 0 2024-03-22T07:15:57.385 INFO:tasks.workunit.client.1.smithi069.stdout:3/164: creat d5/d9/d1e/d35/f37 x:0 0 0 2024-03-22T07:15:57.386 INFO:tasks.workunit.client.1.smithi069.stdout:8/195: getdents d1 0 2024-03-22T07:15:57.388 INFO:tasks.workunit.client.1.smithi069.stdout:5/128: dread d1/d6/f1e [0,4194304] 0 2024-03-22T07:15:57.388 INFO:tasks.workunit.client.1.smithi069.stdout:5/129: truncate d1/d7/d8/f24 896718 0 2024-03-22T07:15:57.390 INFO:tasks.workunit.client.1.smithi069.stdout:3/165: mknod d5/d9/d26/c38 0 2024-03-22T07:15:57.390 INFO:tasks.workunit.client.1.smithi069.stdout:8/196: mknod d1/d1c/c34 0 2024-03-22T07:15:57.391 INFO:tasks.workunit.client.1.smithi069.stdout:8/197: symlink d1/de/l35 0 2024-03-22T07:15:57.391 INFO:tasks.workunit.client.1.smithi069.stdout:0/188: sync 2024-03-22T07:15:57.391 INFO:tasks.workunit.client.1.smithi069.stdout:7/253: sync 2024-03-22T07:15:57.392 INFO:tasks.workunit.client.1.smithi069.stdout:3/166: dread d5/f6 [0,4194304] 0 2024-03-22T07:15:57.393 INFO:tasks.workunit.client.1.smithi069.stdout:0/189: truncate d3/f13 509121 0 2024-03-22T07:15:57.396 INFO:tasks.workunit.client.1.smithi069.stdout:3/167: getdents d5/d9/d23 0 2024-03-22T07:15:57.399 INFO:tasks.workunit.client.1.smithi069.stdout:8/198: dread d1/d1c/f22 [0,4194304] 0 2024-03-22T07:15:57.399 INFO:tasks.workunit.client.1.smithi069.stdout:8/199: creat d1/de/d12/f36 x:0 0 0 2024-03-22T07:15:57.399 INFO:tasks.workunit.client.0.smithi063.stdout:4/369: write d1/d5/d43/d27/d2c/f7b [1269453,58070] 0 2024-03-22T07:15:57.400 INFO:tasks.workunit.client.0.smithi063.stdout:4/370: fsync d1/d5/d38/d59/f64 0 2024-03-22T07:15:57.401 INFO:tasks.workunit.client.1.smithi069.stdout:7/254: rename d9/d41 to d9/db/d48/d50/d57 0 2024-03-22T07:15:57.402 INFO:tasks.workunit.client.0.smithi063.stdout:1/516: dwrite d4/d5a/fa9 [0,4194304] 0 2024-03-22T07:15:57.403 INFO:tasks.workunit.client.0.smithi063.stdout:9/398: dread d8/d17/f26 [0,4194304] 0 2024-03-22T07:15:57.403 INFO:tasks.workunit.client.0.smithi063.stdout:2/425: readlink d0/de/d35/d36/l4f 0 2024-03-22T07:15:57.404 INFO:tasks.workunit.client.1.smithi069.stdout:0/190: getdents d3/da 0 2024-03-22T07:15:57.404 INFO:tasks.workunit.client.1.smithi069.stdout:0/191: dread - d3/da/f38 zero size 2024-03-22T07:15:57.404 INFO:tasks.workunit.client.0.smithi063.stdout:5/411: dwrite d8/d13/f4b [0,4194304] 0 2024-03-22T07:15:57.408 INFO:tasks.workunit.client.0.smithi063.stdout:8/489: mkdir d0/d1b/d28/d3e/d97 0 2024-03-22T07:15:57.410 INFO:tasks.workunit.client.0.smithi063.stdout:8/490: write d0/d24/f74 [80909,68741] 0 2024-03-22T07:15:57.410 INFO:tasks.workunit.client.0.smithi063.stdout:8/491: creat d0/d1f/f98 x:0 0 0 2024-03-22T07:15:57.410 INFO:tasks.workunit.client.1.smithi069.stdout:7/255: link d9/db/d48/d50/d57/d44/d53/c55 d9/db/d48/d50/d57/c58 0 2024-03-22T07:15:57.410 INFO:tasks.workunit.client.0.smithi063.stdout:3/511: creat da/d63/d65/fa8 x:0 0 0 2024-03-22T07:15:57.411 INFO:tasks.workunit.client.1.smithi069.stdout:7/256: creat d9/db/d48/d50/d57/f59 x:0 0 0 2024-03-22T07:15:57.411 INFO:tasks.workunit.client.1.smithi069.stdout:7/257: fsync d9/db/d1a/f21 0 2024-03-22T07:15:57.412 INFO:tasks.workunit.client.1.smithi069.stdout:7/258: mknod d9/db/d48/d50/d57/d44/d53/d1d/d2a/c5a 0 2024-03-22T07:15:57.413 INFO:tasks.workunit.client.0.smithi063.stdout:0/553: unlink da/de/d17/d25/d3c/d43/c44 0 2024-03-22T07:15:57.414 INFO:tasks.workunit.client.1.smithi069.stdout:7/259: creat d9/db/d48/f5b x:0 0 0 2024-03-22T07:15:57.416 INFO:tasks.workunit.client.1.smithi069.stdout:0/192: write d3/d7/d15/d1f/f2b [3655053,116419] 0 2024-03-22T07:15:57.416 INFO:tasks.workunit.client.1.smithi069.stdout:4/258: dwrite f5 [4194304,4194304] 0 2024-03-22T07:15:57.416 INFO:tasks.workunit.client.1.smithi069.stdout:0/193: truncate d3/da/f3a 667789 0 2024-03-22T07:15:57.416 INFO:tasks.workunit.client.1.smithi069.stdout:0/194: stat d3/d7 0 2024-03-22T07:15:57.417 INFO:tasks.workunit.client.1.smithi069.stdout:4/259: truncate d6/dc/f1a 256094 0 2024-03-22T07:15:57.418 INFO:tasks.workunit.client.1.smithi069.stdout:4/260: chown d6/d12/d18/d21/f49 102860516 1 2024-03-22T07:15:57.418 INFO:tasks.workunit.client.1.smithi069.stdout:4/261: write d6/dc/d1d/d31/d1f/f39 [2534616,35159] 0 2024-03-22T07:15:57.418 INFO:tasks.workunit.client.0.smithi063.stdout:4/371: unlink d1/d5/d43/d27/d2c/f7b 0 2024-03-22T07:15:57.419 INFO:tasks.workunit.client.1.smithi069.stdout:0/195: unlink d3/d7/d15/d29/f33 0 2024-03-22T07:15:57.420 INFO:tasks.workunit.client.1.smithi069.stdout:0/196: write d3/d7/fb [291835,33079] 0 2024-03-22T07:15:57.420 INFO:tasks.workunit.client.0.smithi063.stdout:9/399: write d8/f13 [170617,20675] 0 2024-03-22T07:15:57.420 INFO:tasks.workunit.client.0.smithi063.stdout:2/426: truncate d0/de/f31 348007 0 2024-03-22T07:15:57.424 INFO:tasks.workunit.client.0.smithi063.stdout:5/412: rmdir d8/d13/d1e/d31/d64/d65/d6f 0 2024-03-22T07:15:57.424 INFO:tasks.workunit.client.0.smithi063.stdout:5/413: chown d8/d13/f18 0 1 2024-03-22T07:15:57.427 INFO:tasks.workunit.client.1.smithi069.stdout:7/260: write d9/f18 [1325574,107433] 0 2024-03-22T07:15:57.427 INFO:tasks.workunit.client.1.smithi069.stdout:5/130: dwrite d1/d7/d8/f24 [0,4194304] 0 2024-03-22T07:15:57.427 INFO:tasks.workunit.client.1.smithi069.stdout:5/131: fsync d1/d6/ff 0 2024-03-22T07:15:57.427 INFO:tasks.workunit.client.1.smithi069.stdout:5/132: stat d1/d6/f1e 0 2024-03-22T07:15:57.427 INFO:tasks.workunit.client.1.smithi069.stdout:5/133: truncate d1/f3 569024 0 2024-03-22T07:15:57.427 INFO:tasks.workunit.client.1.smithi069.stdout:5/134: readlink d1/d6/l19 0 2024-03-22T07:15:57.429 INFO:tasks.workunit.client.1.smithi069.stdout:7/261: rename d9/db/d48/d50/d57/d44/d53/d2e/c34 to d9/db/d48/c5c 0 2024-03-22T07:15:57.430 INFO:tasks.workunit.client.0.smithi063.stdout:8/492: mkdir d0/d1b/d28/d5f/d99 0 2024-03-22T07:15:57.430 INFO:tasks.workunit.client.1.smithi069.stdout:8/200: dwrite d1/de/d20/f2f [4194304,4194304] 0 2024-03-22T07:15:57.431 INFO:tasks.workunit.client.1.smithi069.stdout:3/168: dwrite d5/f18 [0,4194304] 0 2024-03-22T07:15:57.431 INFO:tasks.workunit.client.0.smithi063.stdout:4/372: write d1/f5b [517499,33309] 0 2024-03-22T07:15:57.431 INFO:tasks.workunit.client.0.smithi063.stdout:3/512: mkdir da/d16/d89/da9 0 2024-03-22T07:15:57.431 INFO:tasks.workunit.client.0.smithi063.stdout:3/513: fdatasync da/d16/d3f/f79 0 2024-03-22T07:15:57.432 INFO:tasks.workunit.client.0.smithi063.stdout:3/514: chown da/d63/d65/d92/d96/f9a 15457 1 2024-03-22T07:15:57.433 INFO:tasks.workunit.client.0.smithi063.stdout:0/554: getdents da/de/d17/d25 0 2024-03-22T07:15:57.433 INFO:tasks.workunit.client.0.smithi063.stdout:0/555: write da/f2b [1094509,129944] 0 2024-03-22T07:15:57.434 INFO:tasks.workunit.client.1.smithi069.stdout:5/135: rename d1/c4 to d1/d28/c2a 0 2024-03-22T07:15:57.434 INFO:tasks.workunit.client.1.smithi069.stdout:0/197: dread d3/f8 [0,4194304] 0 2024-03-22T07:15:57.435 INFO:tasks.workunit.client.1.smithi069.stdout:0/198: chown d3/da/f11 3656 1 2024-03-22T07:15:57.436 INFO:tasks.workunit.client.0.smithi063.stdout:1/517: dwrite d4/d8b/f97 [0,4194304] 0 2024-03-22T07:15:57.437 INFO:tasks.workunit.client.0.smithi063.stdout:1/518: chown d4/df/l7e 1860 1 2024-03-22T07:15:57.437 INFO:tasks.workunit.client.0.smithi063.stdout:1/519: write d4/df/d13/d39/f5e [415267,76709] 0 2024-03-22T07:15:57.437 INFO:tasks.workunit.client.0.smithi063.stdout:9/400: creat d8/d19/d41/d48/f7c x:0 0 0 2024-03-22T07:15:57.437 INFO:tasks.workunit.client.1.smithi069.stdout:7/262: dread d9/db/f17 [0,4194304] 0 2024-03-22T07:15:57.437 INFO:tasks.workunit.client.1.smithi069.stdout:8/201: mknod d1/de/d12/c37 0 2024-03-22T07:15:57.438 INFO:tasks.workunit.client.0.smithi063.stdout:2/427: creat d0/de/d35/d36/f8f x:0 0 0 2024-03-22T07:15:57.438 INFO:tasks.workunit.client.0.smithi063.stdout:9/401: write d8/d15/f46 [106561,70609] 0 2024-03-22T07:15:57.440 INFO:tasks.workunit.client.1.smithi069.stdout:3/169: creat d5/d33/f39 x:0 0 0 2024-03-22T07:15:57.440 INFO:tasks.workunit.client.1.smithi069.stdout:3/170: write d5/d33/f39 [575859,10428] 0 2024-03-22T07:15:57.440 INFO:tasks.workunit.client.1.smithi069.stdout:3/171: fdatasync d5/d9/d1e/d35/f37 0 2024-03-22T07:15:57.441 INFO:tasks.workunit.client.0.smithi063.stdout:8/493: rename d0/ca to d0/d1b/c9a 0 2024-03-22T07:15:57.441 INFO:tasks.workunit.client.0.smithi063.stdout:8/494: creat d0/d1b/d28/d7b/f9b x:0 0 0 2024-03-22T07:15:57.441 INFO:tasks.workunit.client.0.smithi063.stdout:8/495: creat d0/d24/d33/d4c/f9c x:0 0 0 2024-03-22T07:15:57.444 INFO:tasks.workunit.client.1.smithi069.stdout:4/262: dread d6/f9 [0,4194304] 0 2024-03-22T07:15:57.444 INFO:tasks.workunit.client.0.smithi063.stdout:0/556: truncate da/de/d17/d25/d3c/d64/d78/f85 489980 0 2024-03-22T07:15:57.446 INFO:tasks.workunit.client.0.smithi063.stdout:1/520: mknod d4/d8b/da7/cad 0 2024-03-22T07:15:57.448 INFO:tasks.workunit.client.0.smithi063.stdout:9/402: mknod d8/d19/d20/d22/d29/d31/d53/d6a/c7d 0 2024-03-22T07:15:57.449 INFO:tasks.workunit.client.0.smithi063.stdout:5/414: dwrite d8/f10 [0,4194304] 0 2024-03-22T07:15:57.449 INFO:tasks.workunit.client.0.smithi063.stdout:5/415: creat d8/d13/d28/d57/f77 x:0 0 0 2024-03-22T07:15:57.450 INFO:tasks.workunit.client.0.smithi063.stdout:3/515: rename da/d16/l31 to da/d63/d65/d81/da5/laa 0 2024-03-22T07:15:57.450 INFO:tasks.workunit.client.1.smithi069.stdout:7/263: fdatasync d9/db/f19 0 2024-03-22T07:15:57.454 INFO:tasks.workunit.client.1.smithi069.stdout:0/199: creat d3/d7/d15/f42 x:0 0 0 2024-03-22T07:15:57.455 INFO:tasks.workunit.client.0.smithi063.stdout:0/557: link da/d49/f6d da/de/d17/d25/d3c/d64/fb2 0 2024-03-22T07:15:57.456 INFO:tasks.workunit.client.0.smithi063.stdout:0/558: truncate da/de/d17/d25/d3c/d5f/d97/fa4 4758655 0 2024-03-22T07:15:57.456 INFO:tasks.workunit.client.0.smithi063.stdout:0/559: read - da/d49/f74 zero size 2024-03-22T07:15:57.456 INFO:tasks.workunit.client.0.smithi063.stdout:0/560: truncate da/de/d17/d25/f65 197732 0 2024-03-22T07:15:57.456 INFO:tasks.workunit.client.0.smithi063.stdout:0/561: chown da/de/f5c 3 1 2024-03-22T07:15:57.456 INFO:tasks.workunit.client.1.smithi069.stdout:8/202: creat d1/d1c/f38 x:0 0 0 2024-03-22T07:15:57.457 INFO:tasks.workunit.client.0.smithi063.stdout:2/428: write d0/de/d12/d20/f70 [2688598,116320] 0 2024-03-22T07:15:57.457 INFO:tasks.workunit.client.1.smithi069.stdout:3/172: creat d5/d9/f3a x:0 0 0 2024-03-22T07:15:57.461 INFO:tasks.workunit.client.1.smithi069.stdout:5/136: getdents d1/d28 0 2024-03-22T07:15:57.463 INFO:tasks.workunit.client.1.smithi069.stdout:5/137: mknod d1/d7/c2b 0 2024-03-22T07:15:57.465 INFO:tasks.workunit.client.1.smithi069.stdout:5/138: mknod d1/d6/c2c 0 2024-03-22T07:15:57.465 INFO:tasks.workunit.client.0.smithi063.stdout:9/403: dread d8/d15/d2f/f47 [0,4194304] 0 2024-03-22T07:15:57.467 INFO:tasks.workunit.client.1.smithi069.stdout:0/200: dread d3/f13 [0,4194304] 0 2024-03-22T07:15:57.468 INFO:tasks.workunit.client.1.smithi069.stdout:5/139: rename d1/d6/l19 to d1/d6/d26/l2d 0 2024-03-22T07:15:57.469 INFO:tasks.workunit.client.0.smithi063.stdout:8/496: dread d0/d26/f27 [0,4194304] 0 2024-03-22T07:15:57.475 INFO:tasks.workunit.client.1.smithi069.stdout:4/263: dwrite d6/d12/d18/f45 [0,4194304] 0 2024-03-22T07:15:57.476 INFO:tasks.workunit.client.0.smithi063.stdout:5/416: chown d8/d13/f48 524 1 2024-03-22T07:15:57.477 INFO:tasks.workunit.client.0.smithi063.stdout:5/417: readlink d8/l2c 0 2024-03-22T07:15:57.478 INFO:tasks.workunit.client.0.smithi063.stdout:3/516: symlink da/lab 0 2024-03-22T07:15:57.478 INFO:tasks.workunit.client.1.smithi069.stdout:3/173: dread d5/d9/d1e/f22 [0,4194304] 0 2024-03-22T07:15:57.478 INFO:tasks.workunit.client.1.smithi069.stdout:3/174: readlink d5/d9/l1d 0 2024-03-22T07:15:57.479 INFO:tasks.workunit.client.1.smithi069.stdout:0/201: read d3/d7/d15/d1f/f21 [5155253,44624] 0 2024-03-22T07:15:57.480 INFO:tasks.workunit.client.1.smithi069.stdout:0/202: write d3/d7/f1e [290038,86440] 0 2024-03-22T07:15:57.480 INFO:tasks.workunit.client.0.smithi063.stdout:1/521: rename d4/df/d13/d2b/c55 to d4/df/d13/d39/d63/cae 0 2024-03-22T07:15:57.480 INFO:tasks.workunit.client.0.smithi063.stdout:1/522: fdatasync d4/d14/d57/f5d 0 2024-03-22T07:15:57.481 INFO:tasks.workunit.client.1.smithi069.stdout:4/264: getdents d6/dc/d1d/d31/d1f 0 2024-03-22T07:15:57.481 INFO:tasks.workunit.client.1.smithi069.stdout:4/265: readlink d6/dc/d1d/d31/d1f/l36 0 2024-03-22T07:15:57.481 INFO:tasks.workunit.client.0.smithi063.stdout:9/404: creat d8/d17/d1e/f7e x:0 0 0 2024-03-22T07:15:57.481 INFO:tasks.workunit.client.0.smithi063.stdout:9/405: write d8/d19/d20/d22/d29/d31/f32 [4703671,28864] 0 2024-03-22T07:15:57.482 INFO:tasks.workunit.client.0.smithi063.stdout:8/497: symlink d0/d1b/d28/d7b/d4a/l9d 0 2024-03-22T07:15:57.482 INFO:tasks.workunit.client.0.smithi063.stdout:8/498: chown d0/d1b/d28/l31 26526870 1 2024-03-22T07:15:57.483 INFO:tasks.workunit.client.1.smithi069.stdout:5/140: dread d1/f1f [4194304,4194304] 0 2024-03-22T07:15:57.483 INFO:tasks.workunit.client.1.smithi069.stdout:5/141: write d1/f9 [2468915,83999] 0 2024-03-22T07:15:57.483 INFO:tasks.workunit.client.1.smithi069.stdout:4/266: mknod d6/dc/d1d/c4e 0 2024-03-22T07:15:57.483 INFO:tasks.workunit.client.1.smithi069.stdout:5/142: write d1/d6/ff [649305,57675] 0 2024-03-22T07:15:57.483 INFO:tasks.workunit.client.1.smithi069.stdout:5/143: truncate d1/d6/ff 714778 0 2024-03-22T07:15:57.483 INFO:tasks.workunit.client.1.smithi069.stdout:4/267: rmdir d6/dc/d1d 39 2024-03-22T07:15:57.484 INFO:tasks.workunit.client.1.smithi069.stdout:4/268: truncate d6/d12/f33 809806 0 2024-03-22T07:15:57.484 INFO:tasks.workunit.client.1.smithi069.stdout:4/269: write d6/f9 [859682,15768] 0 2024-03-22T07:15:57.484 INFO:tasks.workunit.client.0.smithi063.stdout:4/373: dwrite d1/d5/d43/f2e [0,4194304] 0 2024-03-22T07:15:57.485 INFO:tasks.workunit.client.0.smithi063.stdout:5/418: creat d8/d13/d24/f78 x:0 0 0 2024-03-22T07:15:57.487 INFO:tasks.workunit.client.0.smithi063.stdout:3/517: link da/d63/d65/d86/f93 da/d63/d65/fac 0 2024-03-22T07:15:57.487 INFO:tasks.workunit.client.0.smithi063.stdout:3/518: dread - da/d63/d65/fa8 zero size 2024-03-22T07:15:57.487 INFO:tasks.workunit.client.0.smithi063.stdout:3/519: write da/d63/d65/f99 [2856,40567] 0 2024-03-22T07:15:57.489 INFO:tasks.workunit.client.1.smithi069.stdout:0/203: write d3/d7/d17/f1a [4134339,82209] 0 2024-03-22T07:15:57.489 INFO:tasks.workunit.client.0.smithi063.stdout:1/523: mknod d4/d14/caf 0 2024-03-22T07:15:57.490 INFO:tasks.workunit.client.1.smithi069.stdout:0/204: symlink d3/l43 0 2024-03-22T07:15:57.491 INFO:tasks.workunit.client.1.smithi069.stdout:0/205: symlink d3/d7/d24/l44 0 2024-03-22T07:15:57.491 INFO:tasks.workunit.client.0.smithi063.stdout:9/406: mkdir d8/d19/d20/d22/d29/d31/d7f 0 2024-03-22T07:15:57.493 INFO:tasks.workunit.client.1.smithi069.stdout:3/175: write d5/d9/fe [3238687,84053] 0 2024-03-22T07:15:57.493 INFO:tasks.workunit.client.1.smithi069.stdout:3/176: chown d5/d9/fe 27385 1 2024-03-22T07:15:57.493 INFO:tasks.workunit.client.1.smithi069.stdout:0/206: link d3/f8 d3/d7/d15/f45 0 2024-03-22T07:15:57.494 INFO:tasks.workunit.client.0.smithi063.stdout:8/499: mkdir d0/d9e 0 2024-03-22T07:15:57.494 INFO:tasks.workunit.client.0.smithi063.stdout:8/500: chown d0/d26/d2d/d8f 27708 1 2024-03-22T07:15:57.499 INFO:tasks.workunit.client.0.smithi063.stdout:5/419: creat d8/d13/d1e/f79 x:0 0 0 2024-03-22T07:15:57.499 INFO:tasks.workunit.client.0.smithi063.stdout:5/420: readlink d8/lb 0 2024-03-22T07:15:57.499 INFO:tasks.workunit.client.0.smithi063.stdout:5/421: dread - d8/d13/d24/f78 zero size 2024-03-22T07:15:57.499 INFO:tasks.workunit.client.0.smithi063.stdout:5/422: write d8/f25 [1450391,72650] 0 2024-03-22T07:15:57.501 INFO:tasks.workunit.client.0.smithi063.stdout:3/520: creat da/d63/d65/d81/da5/fad x:0 0 0 2024-03-22T07:15:57.502 INFO:tasks.workunit.client.1.smithi069.stdout:6/135: sync 2024-03-22T07:15:57.502 INFO:tasks.workunit.client.1.smithi069.stdout:6/136: creat d4/d7/f2c x:0 0 0 2024-03-22T07:15:57.504 INFO:tasks.workunit.client.0.smithi063.stdout:1/524: rename d4/df/d13/d2b/d6c/f89 to d4/d5a/fb0 0 2024-03-22T07:15:57.504 INFO:tasks.workunit.client.0.smithi063.stdout:1/525: stat d4/df/f11 0 2024-03-22T07:15:57.505 INFO:tasks.workunit.client.1.smithi069.stdout:5/144: write d1/d7/d8/f24 [4015637,35797] 0 2024-03-22T07:15:57.505 INFO:tasks.workunit.client.1.smithi069.stdout:5/145: write d1/d7/d8/f27 [601274,75164] 0 2024-03-22T07:15:57.505 INFO:tasks.workunit.client.1.smithi069.stdout:5/146: creat d1/d6/f2e x:0 0 0 2024-03-22T07:15:57.507 INFO:tasks.workunit.client.0.smithi063.stdout:8/501: rename d0/d24/c30 to d0/d24/d33/d4c/d78/c9f 0 2024-03-22T07:15:57.509 INFO:tasks.workunit.client.0.smithi063.stdout:1/526: symlink d4/df/d13/d2b/lb1 0 2024-03-22T07:15:57.509 INFO:tasks.workunit.client.0.smithi063.stdout:1/527: chown d4/df/d13/d2b/d86/fa3 3962828 1 2024-03-22T07:15:57.509 INFO:tasks.workunit.client.0.smithi063.stdout:1/528: fsync d4/df/d2c/f90 0 2024-03-22T07:15:57.509 INFO:tasks.workunit.client.0.smithi063.stdout:1/529: chown d4/d14/d57/c5b 1672 1 2024-03-22T07:15:57.510 INFO:tasks.workunit.client.0.smithi063.stdout:8/502: symlink d0/d1b/d28/la0 0 2024-03-22T07:15:57.510 INFO:tasks.workunit.client.0.smithi063.stdout:8/503: fsync d0/d24/d5b/f44 0 2024-03-22T07:15:57.510 INFO:tasks.workunit.client.0.smithi063.stdout:8/504: fdatasync d0/d1b/f1e 0 2024-03-22T07:15:57.510 INFO:tasks.workunit.client.0.smithi063.stdout:8/505: write d0/d24/f74 [320305,46428] 0 2024-03-22T07:15:57.510 INFO:tasks.workunit.client.0.smithi063.stdout:8/506: chown d0/f89 4 1 2024-03-22T07:15:57.512 INFO:tasks.workunit.client.0.smithi063.stdout:8/507: mkdir d0/d1b/d28/d5f/d99/da1 0 2024-03-22T07:15:57.513 INFO:tasks.workunit.client.0.smithi063.stdout:8/508: mknod d0/d26/d2d/d8f/ca2 0 2024-03-22T07:15:57.515 INFO:tasks.workunit.client.1.smithi069.stdout:8/203: dwrite d1/d1c/f38 [0,4194304] 0 2024-03-22T07:15:57.516 INFO:tasks.workunit.client.0.smithi063.stdout:8/509: link d0/d1f/f53 d0/d1b/d28/fa3 0 2024-03-22T07:15:57.517 INFO:tasks.workunit.client.1.smithi069.stdout:7/264: dwrite d9/db/d48/d50/d57/d44/d53/d1d/d2a/d30/f54 [0,4194304] 0 2024-03-22T07:15:57.517 INFO:tasks.workunit.client.1.smithi069.stdout:7/265: truncate d9/db/f24 576516 0 2024-03-22T07:15:57.517 INFO:tasks.workunit.client.0.smithi063.stdout:2/429: dwrite d0/de/d1b/d63/f6c [0,4194304] 0 2024-03-22T07:15:57.517 INFO:tasks.workunit.client.0.smithi063.stdout:1/530: dread d4/d14/d57/f5d [0,4194304] 0 2024-03-22T07:15:57.517 INFO:tasks.workunit.client.0.smithi063.stdout:2/430: fdatasync d0/de/d1b/f50 0 2024-03-22T07:15:57.521 INFO:tasks.workunit.client.0.smithi063.stdout:8/510: write d0/d1b/f1e [8845757,111547] 0 2024-03-22T07:15:57.524 INFO:tasks.workunit.client.0.smithi063.stdout:8/511: link d0/d26/d2d/d59/f87 d0/d1b/d28/d7b/fa4 0 2024-03-22T07:15:57.524 INFO:tasks.workunit.client.0.smithi063.stdout:8/512: dread d0/f64 [0,4194304] 0 2024-03-22T07:15:57.529 INFO:tasks.workunit.client.1.smithi069.stdout:7/266: link d9/db/d48/d50/d57/d44/d53/c23 d9/db/d48/d4b/c5d 0 2024-03-22T07:15:57.529 INFO:tasks.workunit.client.1.smithi069.stdout:7/267: chown d9/db/d48/d50/d57/d44/f47 2405 1 2024-03-22T07:15:57.529 INFO:tasks.workunit.client.1.smithi069.stdout:7/268: chown d9/db/d48/d50/d57/d44/d53/f38 171723 1 2024-03-22T07:15:57.535 INFO:tasks.workunit.client.0.smithi063.stdout:4/374: dwrite d1/d5/d3c/f83 [0,4194304] 0 2024-03-22T07:15:57.536 INFO:tasks.workunit.client.0.smithi063.stdout:7/341: sync 2024-03-22T07:15:57.536 INFO:tasks.workunit.client.0.smithi063.stdout:1/531: write d4/df/d2c/f54 [3653935,92105] 0 2024-03-22T07:15:57.537 INFO:tasks.workunit.client.0.smithi063.stdout:4/375: creat d1/d5/f84 x:0 0 0 2024-03-22T07:15:57.538 INFO:tasks.workunit.client.0.smithi063.stdout:1/532: truncate d4/df/d13/d39/f5f 3278238 0 2024-03-22T07:15:57.539 INFO:tasks.workunit.client.0.smithi063.stdout:1/533: truncate d4/d14/d57/f80 251679 0 2024-03-22T07:15:57.539 INFO:tasks.workunit.client.0.smithi063.stdout:0/562: dwrite da/d49/f63 [4194304,4194304] 0 2024-03-22T07:15:57.540 INFO:tasks.workunit.client.0.smithi063.stdout:4/376: mkdir d1/d5/d85 0 2024-03-22T07:15:57.540 INFO:tasks.workunit.client.0.smithi063.stdout:1/534: mkdir d4/df/d13/d2b/d6c/da8/db2 0 2024-03-22T07:15:57.540 INFO:tasks.workunit.client.0.smithi063.stdout:2/431: dread d0/de/d12/f27 [0,4194304] 0 2024-03-22T07:15:57.542 INFO:tasks.workunit.client.0.smithi063.stdout:1/535: rmdir d4/df/d13/d2b/d6c/da8 39 2024-03-22T07:15:57.543 INFO:tasks.workunit.client.0.smithi063.stdout:4/377: rmdir d1/d5/dc 39 2024-03-22T07:15:57.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.543+0000 7fa1e7d0a700 1 -- 172.21.15.63:0/778168914 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1e010f790 con 0x7fa1cc06af90 2024-03-22T07:15:57.544 INFO:tasks.workunit.client.0.smithi063.stdout:1/536: rmdir d4/d14/d57 39 2024-03-22T07:15:57.544 INFO:tasks.workunit.client.0.smithi063.stdout:1/537: fdatasync d4/d14/d57/f5d 0 2024-03-22T07:15:57.544 INFO:tasks.workunit.client.0.smithi063.stdout:1/538: write d4/df/d13/d2b/d86/fa3 [270793,16958] 0 2024-03-22T07:15:57.544 INFO:tasks.workunit.client.0.smithi063.stdout:1/539: read - d4/df/f4e zero size 2024-03-22T07:15:57.544 INFO:tasks.workunit.client.0.smithi063.stdout:1/540: creat d4/d71/d88/fb3 x:0 0 0 2024-03-22T07:15:57.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.545+0000 7fa1d67fc700 1 -- 172.21.15.63:0/778168914 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fa1e010f790 con 0x7fa1cc06af90 2024-03-22T07:15:57.546 INFO:tasks.workunit.client.0.smithi063.stdout:1/541: symlink d4/d5a/d91/lb4 0 2024-03-22T07:15:57.546 INFO:tasks.workunit.client.0.smithi063.stdout:1/542: dread - d4/f93 zero size 2024-03-22T07:15:57.547 INFO:tasks.workunit.client.0.smithi063.stdout:1/543: creat d4/df/d13/d2b/d86/fb5 x:0 0 0 2024-03-22T07:15:57.547 INFO:tasks.workunit.client.0.smithi063.stdout:2/432: dread d0/de/d1b/d32/f49 [0,4194304] 0 2024-03-22T07:15:57.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.547+0000 7fa1e7d0a700 1 -- 172.21.15.63:0/778168914 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa1cc06af90 msgr2=0x7fa1cc06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:57.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.547+0000 7fa1e7d0a700 1 --2- 172.21.15.63:0/778168914 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa1cc06af90 0x7fa1cc06d440 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc004890 tx=0x7fa1dc009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:57.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.547+0000 7fa1e7d0a700 1 -- 172.21.15.63:0/778168914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00fff20 msgr2=0x7fa1e010dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:57.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.547+0000 7fa1e7d0a700 1 --2- 172.21.15.63:0/778168914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00fff20 0x7fa1e010dfb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000eb40 tx=0x7fa1d000b040 comp rx=0 tx=0).stop 2024-03-22T07:15:57.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.547+0000 7fa1e7d0a700 1 -- 172.21.15.63:0/778168914 shutdown_connections 2024-03-22T07:15:57.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.547+0000 7fa1e7d0a700 1 --2- 172.21.15.63:0/778168914 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa1cc06af90 0x7fa1cc06d440 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:57.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.547+0000 7fa1e7d0a700 1 --2- 172.21.15.63:0/778168914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1e0066b70 0x7fa1e010da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:57.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.547+0000 7fa1e7d0a700 1 --2- 172.21.15.63:0/778168914 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa1e00fff20 0x7fa1e010dfb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:57.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.547+0000 7fa1e7d0a700 1 -- 172.21.15.63:0/778168914 >> 172.21.15.63:0/778168914 conn(0x7fa1e00f7450 msgr2=0x7fa1e00f7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:57.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.548+0000 7fa1e7d0a700 1 -- 172.21.15.63:0/778168914 shutdown_connections 2024-03-22T07:15:57.551 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.548+0000 7fa1e7d0a700 1 -- 172.21.15.63:0/778168914 wait complete. 2024-03-22T07:15:57.551 INFO:tasks.workunit.client.0.smithi063.stdout:1/544: unlink d4/d5a/l75 0 2024-03-22T07:15:57.551 INFO:tasks.workunit.client.0.smithi063.stdout:1/545: chown d4/d5a/l95 142 1 2024-03-22T07:15:57.551 INFO:tasks.workunit.client.0.smithi063.stdout:0/563: write da/de/d17/d25/f6a [3261187,7342] 0 2024-03-22T07:15:57.552 INFO:tasks.workunit.client.0.smithi063.stdout:0/564: truncate da/de/d17/f45 236125 0 2024-03-22T07:15:57.552 INFO:tasks.workunit.client.0.smithi063.stdout:0/565: write da/de/d17/f28 [1093931,70066] 0 2024-03-22T07:15:57.553 INFO:tasks.workunit.client.0.smithi063.stdout:1/546: mkdir d4/d5a/d91/db6 0 2024-03-22T07:15:57.555 INFO:tasks.workunit.client.0.smithi063.stdout:0/566: mkdir da/de/d17/d25/d95/d98/db3 0 2024-03-22T07:15:57.557 INFO:tasks.workunit.client.0.smithi063.stdout:4/378: dread d1/d5/d43/f29 [0,4194304] 0 2024-03-22T07:15:57.558 INFO:tasks.workunit.client.0.smithi063.stdout:1/547: rename d4/df/d2c/f54 to d4/d5a/d91/db6/fb7 0 2024-03-22T07:15:57.559 INFO:tasks.workunit.client.1.smithi069.stdout:7/269: write d9/f3b [2684623,75341] 0 2024-03-22T07:15:57.559 INFO:tasks.workunit.client.1.smithi069.stdout:4/270: dwrite d6/d12/f33 [0,4194304] 0 2024-03-22T07:15:57.567 INFO:tasks.workunit.client.1.smithi069.stdout:0/207: dwrite d3/da/f11 [4194304,4194304] 0 2024-03-22T07:15:57.567 INFO:tasks.workunit.client.0.smithi063.stdout:6/480: sync 2024-03-22T07:15:57.567 INFO:tasks.workunit.client.0.smithi063.stdout:6/481: truncate d3/d1c/f5d 340026 0 2024-03-22T07:15:57.568 INFO:tasks.workunit.client.0.smithi063.stdout:9/407: dwrite d8/d17/f26 [0,4194304] 0 2024-03-22T07:15:57.568 INFO:tasks.workunit.client.0.smithi063.stdout:9/408: creat d8/d19/d41/f80 x:0 0 0 2024-03-22T07:15:57.568 INFO:tasks.workunit.client.0.smithi063.stdout:9/409: stat d8/d17/d1e 0 2024-03-22T07:15:57.568 INFO:tasks.workunit.client.0.smithi063.stdout:9/410: read d8/d17/f69 [651260,65804] 0 2024-03-22T07:15:57.568 INFO:tasks.workunit.client.0.smithi063.stdout:9/411: write d8/d19/d20/d22/d29/d31/f51 [2408542,8805] 0 2024-03-22T07:15:57.569 INFO:tasks.workunit.client.0.smithi063.stdout:5/423: dwrite d8/d13/d1e/d31/d64/d65/f6c [0,4194304] 0 2024-03-22T07:15:57.570 INFO:tasks.workunit.client.1.smithi069.stdout:0/208: dread d3/d7/d17/f3f [0,4194304] 0 2024-03-22T07:15:57.571 INFO:tasks.workunit.client.0.smithi063.stdout:8/513: dwrite d0/d24/d33/d4c/d78/f91 [0,4194304] 0 2024-03-22T07:15:57.573 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:15:57.574 INFO:tasks.workunit.client.1.smithi069.stdout:6/137: dwrite d4/d7/f2c [0,4194304] 0 2024-03-22T07:15:57.574 INFO:tasks.workunit.client.0.smithi063.stdout:4/379: dread d1/d5/f20 [0,4194304] 0 2024-03-22T07:15:57.574 INFO:tasks.workunit.client.0.smithi063.stdout:4/380: creat d1/d5/f86 x:0 0 0 2024-03-22T07:15:57.575 INFO:tasks.workunit.client.1.smithi069.stdout:8/204: dwrite d1/de/d12/f19 [0,4194304] 0 2024-03-22T07:15:57.576 INFO:tasks.workunit.client.1.smithi069.stdout:6/138: getdents d4/dd/d12/d13 0 2024-03-22T07:15:57.577 INFO:tasks.workunit.client.0.smithi063.stdout:5/424: symlink d8/d13/d28/d66/l7a 0 2024-03-22T07:15:57.578 INFO:tasks.workunit.client.1.smithi069.stdout:8/205: mkdir d1/d1c/d39 0 2024-03-22T07:15:57.579 INFO:tasks.workunit.client.1.smithi069.stdout:8/206: rmdir d1/d1c/d39 0 2024-03-22T07:15:57.582 INFO:tasks.workunit.client.0.smithi063.stdout:0/567: dread da/de/f31 [4194304,4194304] 0 2024-03-22T07:15:57.582 INFO:tasks.workunit.client.0.smithi063.stdout:3/521: dwrite da/d16/f38 [0,4194304] 0 2024-03-22T07:15:57.585 INFO:tasks.workunit.client.0.smithi063.stdout:5/425: getdents d8/d13/d1e/d31/d64 0 2024-03-22T07:15:57.585 INFO:tasks.workunit.client.0.smithi063.stdout:5/426: write d8/d13/d1e/d3c/f4c [1035811,28503] 0 2024-03-22T07:15:57.586 INFO:tasks.workunit.client.1.smithi069.stdout:8/207: dread d1/d1c/f22 [0,4194304] 0 2024-03-22T07:15:57.586 INFO:tasks.workunit.client.0.smithi063.stdout:6/482: write d3/d6/d2a/f5e [2166732,107275] 0 2024-03-22T07:15:57.586 INFO:tasks.workunit.client.0.smithi063.stdout:6/483: stat d3/d15/d16/f25 0 2024-03-22T07:15:57.587 INFO:tasks.workunit.client.0.smithi063.stdout:8/514: dread d0/d1b/d28/f43 [0,4194304] 0 2024-03-22T07:15:57.587 INFO:tasks.workunit.client.0.smithi063.stdout:8/515: getdents d0/d26/d2d/d83 0 2024-03-22T07:15:57.587 INFO:tasks.workunit.client.0.smithi063.stdout:8/516: write d0/d24/f35 [3661886,36357] 0 2024-03-22T07:15:57.589 INFO:tasks.workunit.client.0.smithi063.stdout:2/433: dwrite d0/de/d12/d20/f88 [0,4194304] 0 2024-03-22T07:15:57.592 INFO:tasks.workunit.client.0.smithi063.stdout:4/381: dread d1/f3 [0,4194304] 0 2024-03-22T07:15:57.592 INFO:tasks.workunit.client.0.smithi063.stdout:4/382: creat d1/d5/d38/d59/f87 x:0 0 0 2024-03-22T07:15:57.592 INFO:tasks.workunit.client.0.smithi063.stdout:3/522: mkdir da/d16/d26/d69/dae 0 2024-03-22T07:15:57.594 INFO:tasks.workunit.client.0.smithi063.stdout:6/484: link d3/d15/d16/d60/d6d/l81 d3/d6/l9e 0 2024-03-22T07:15:57.594 INFO:tasks.workunit.client.0.smithi063.stdout:6/485: fdatasync d3/d1c/f63 0 2024-03-22T07:15:57.594 INFO:tasks.workunit.client.0.smithi063.stdout:6/486: fsync d3/d15/d16/d60/d6d/d4c/f7d 0 2024-03-22T07:15:57.594 INFO:tasks.workunit.client.0.smithi063.stdout:6/487: chown d3/d15/f17 230182032 1 2024-03-22T07:15:57.596 INFO:tasks.workunit.client.0.smithi063.stdout:8/517: mknod d0/d24/d5b/ca5 0 2024-03-22T07:15:57.599 INFO:tasks.workunit.client.0.smithi063.stdout:5/427: dread d8/d13/d28/f73 [4194304,4194304] 0 2024-03-22T07:15:57.600 INFO:tasks.workunit.client.0.smithi063.stdout:5/428: creat d8/d13/d1e/f7b x:0 0 0 2024-03-22T07:15:57.603 INFO:tasks.workunit.client.0.smithi063.stdout:2/434: symlink d0/de/l90 0 2024-03-22T07:15:57.609 INFO:tasks.workunit.client.0.smithi063.stdout:4/383: rename d1/d5/dc/l34 to d1/d5/d38/l88 0 2024-03-22T07:15:57.610 INFO:tasks.workunit.client.0.smithi063.stdout:3/523: symlink da/d16/d89/d8c/laf 0 2024-03-22T07:15:57.615 INFO:tasks.workunit.client.0.smithi063.stdout:6/488: rename d3/d15/d16/d60/d6d/d4e/d5f/f98 to d3/d6/d2a/d38/f9f 0 2024-03-22T07:15:57.618 INFO:tasks.workunit.client.0.smithi063.stdout:6/489: mknod d3/d1c/ca0 0 2024-03-22T07:15:57.619 INFO:tasks.workunit.client.0.smithi063.stdout:7/342: dwrite d5/da/f11 [0,4194304] 0 2024-03-22T07:15:57.620 INFO:tasks.workunit.client.0.smithi063.stdout:5/429: write d8/d13/d24/f50 [3375951,15487] 0 2024-03-22T07:15:57.621 INFO:tasks.workunit.client.0.smithi063.stdout:6/490: rename d3/d15/f68 to d3/d76/fa1 0 2024-03-22T07:15:57.622 INFO:tasks.workunit.client.0.smithi063.stdout:7/343: symlink d5/da/d5f/d33/d57/l6c 0 2024-03-22T07:15:57.623 INFO:tasks.workunit.client.0.smithi063.stdout:7/344: getdents d5/da/d5f/d33 0 2024-03-22T07:15:57.631 INFO:tasks.workunit.client.1.smithi069.stdout:0/209: dwrite d3/d7/d15/d1f/f2b [0,4194304] 0 2024-03-22T07:15:57.632 INFO:tasks.workunit.client.1.smithi069.stdout:0/210: creat d3/d7/d15/f46 x:0 0 0 2024-03-22T07:15:57.634 INFO:tasks.workunit.client.0.smithi063.stdout:5/430: write d8/d13/f4b [2577906,54782] 0 2024-03-22T07:15:57.635 INFO:tasks.workunit.client.0.smithi063.stdout:5/431: write d8/d13/d24/f78 [335837,102219] 0 2024-03-22T07:15:57.635 INFO:tasks.workunit.client.0.smithi063.stdout:5/432: creat d8/d13/d1e/d31/d64/d65/f7c x:0 0 0 2024-03-22T07:15:57.635 INFO:tasks.workunit.client.0.smithi063.stdout:5/433: readlink d8/d13/d28/d66/l7a 0 2024-03-22T07:15:57.635 INFO:tasks.workunit.client.0.smithi063.stdout:5/434: stat d8/d13/d1e/d31/d58/f68 0 2024-03-22T07:15:57.641 INFO:tasks.workunit.client.0.smithi063.stdout:2/435: dwrite d0/de/d1b/d63/f6c [0,4194304] 0 2024-03-22T07:15:57.641 INFO:tasks.workunit.client.0.smithi063.stdout:2/436: chown d0/de/d35/d36/c62 132380625 1 2024-03-22T07:15:57.641 INFO:tasks.workunit.client.0.smithi063.stdout:2/437: write d0/de/d1b/d63/f86 [475348,70347] 0 2024-03-22T07:15:57.641 INFO:tasks.workunit.client.0.smithi063.stdout:2/438: creat d0/de/d58/f91 x:0 0 0 2024-03-22T07:15:57.644 INFO:tasks.workunit.client.0.smithi063.stdout:2/439: unlink d0/de/d12/d20/l61 0 2024-03-22T07:15:57.644 INFO:tasks.workunit.client.0.smithi063.stdout:2/440: truncate d0/de/f4c 1333650 0 2024-03-22T07:15:57.645 INFO:tasks.workunit.client.0.smithi063.stdout:2/441: truncate d0/de/d12/f1f 1995172 0 2024-03-22T07:15:57.646 INFO:tasks.workunit.client.0.smithi063.stdout:2/442: mkdir d0/de/d1b/d92 0 2024-03-22T07:15:57.647 INFO:tasks.workunit.client.0.smithi063.stdout:4/384: dwrite d1/d5/f86 [0,4194304] 0 2024-03-22T07:15:57.649 INFO:tasks.workunit.client.0.smithi063.stdout:1/548: dwrite d4/df/d2c/f90 [0,4194304] 0 2024-03-22T07:15:57.649 INFO:tasks.workunit.client.0.smithi063.stdout:9/412: dwrite d8/d19/d2b/f5f [0,4194304] 0 2024-03-22T07:15:57.649 INFO:tasks.workunit.client.1.smithi069.stdout:4/271: dwrite d6/d12/d18/f45 [0,4194304] 0 2024-03-22T07:15:57.649 INFO:tasks.workunit.client.1.smithi069.stdout:4/272: creat d6/d12/d18/f4f x:0 0 0 2024-03-22T07:15:57.653 INFO:tasks.workunit.client.0.smithi063.stdout:1/549: mknod d4/cb8 0 2024-03-22T07:15:57.653 INFO:tasks.workunit.client.0.smithi063.stdout:1/550: creat d4/d71/d88/fb9 x:0 0 0 2024-03-22T07:15:57.653 INFO:tasks.workunit.client.0.smithi063.stdout:1/551: write d4/f3b [1016290,118632] 0 2024-03-22T07:15:57.654 INFO:tasks.workunit.client.1.smithi069.stdout:4/273: symlink d6/d16/d3a/l50 0 2024-03-22T07:15:57.656 INFO:tasks.workunit.client.0.smithi063.stdout:1/552: symlink d4/df/d13/d39/d6f/lba 0 2024-03-22T07:15:57.658 INFO:tasks.workunit.client.1.smithi069.stdout:4/274: mkdir d6/dc/d1d/d31/d1f/d3f/d51 0 2024-03-22T07:15:57.658 INFO:tasks.workunit.client.1.smithi069.stdout:4/275: chown d6/d12/d18/d21/l40 77263971 1 2024-03-22T07:15:57.659 INFO:tasks.workunit.client.0.smithi063.stdout:6/491: dwrite d3/d15/f23 [0,4194304] 0 2024-03-22T07:15:57.660 INFO:tasks.workunit.client.0.smithi063.stdout:0/568: truncate da/de/d17/d25/f6a 2638322 0 2024-03-22T07:15:57.661 INFO:tasks.workunit.client.0.smithi063.stdout:2/443: dread d0/de/d12/d5a/f81 [0,4194304] 0 2024-03-22T07:15:57.661 INFO:tasks.workunit.client.0.smithi063.stdout:7/345: dwrite d5/d16/d43/f56 [0,4194304] 0 2024-03-22T07:15:57.661 INFO:tasks.workunit.client.0.smithi063.stdout:7/346: fsync d5/da/f35 0 2024-03-22T07:15:57.661 INFO:tasks.workunit.client.0.smithi063.stdout:7/347: fsync d5/d14/d1d/d29/f47 0 2024-03-22T07:15:57.662 INFO:tasks.workunit.client.1.smithi069.stdout:4/276: link d6/d12/c25 d6/d12/d18/d21/c52 0 2024-03-22T07:15:57.664 INFO:tasks.workunit.client.0.smithi063.stdout:2/444: dread d0/de/d1b/d32/f49 [0,4194304] 0 2024-03-22T07:15:57.665 INFO:tasks.workunit.client.0.smithi063.stdout:1/553: getdents d4/df/d2c 0 2024-03-22T07:15:57.665 INFO:tasks.workunit.client.0.smithi063.stdout:6/492: dread d3/d15/d16/d60/d6d/d4c/f75 [0,4194304] 0 2024-03-22T07:15:57.665 INFO:tasks.workunit.client.0.smithi063.stdout:6/493: chown d3/d6/f10 3756 1 2024-03-22T07:15:57.666 INFO:tasks.workunit.client.0.smithi063.stdout:0/569: creat da/de/fb4 x:0 0 0 2024-03-22T07:15:57.668 INFO:tasks.workunit.client.0.smithi063.stdout:5/435: dwrite d8/d13/d1e/d3c/f55 [0,4194304] 0 2024-03-22T07:15:57.669 INFO:tasks.workunit.client.0.smithi063.stdout:5/436: write d8/d13/d24/f50 [412788,103165] 0 2024-03-22T07:15:57.670 INFO:tasks.workunit.client.0.smithi063.stdout:7/348: mknod d5/da/d5f/c6d 0 2024-03-22T07:15:57.671 INFO:tasks.workunit.client.0.smithi063.stdout:2/445: rmdir d0/de/d1b/d63/d78 39 2024-03-22T07:15:57.671 INFO:tasks.workunit.client.0.smithi063.stdout:2/446: chown d0/de/d1b/d32/f3b 175113867 1 2024-03-22T07:15:57.673 INFO:tasks.workunit.client.0.smithi063.stdout:1/554: mkdir d4/d71/dbb 0 2024-03-22T07:15:57.674 INFO:tasks.workunit.client.0.smithi063.stdout:6/494: symlink d3/d1c/d32/d51/la2 0 2024-03-22T07:15:57.674 INFO:tasks.workunit.client.0.smithi063.stdout:6/495: write d3/f19 [5200172,56592] 0 2024-03-22T07:15:57.674 INFO:tasks.workunit.client.0.smithi063.stdout:6/496: write d3/d15/f62 [1181644,120] 0 2024-03-22T07:15:57.675 INFO:tasks.workunit.client.0.smithi063.stdout:6/497: chown d3/d15/d16/d60/d6d/d4e 17235420 1 2024-03-22T07:15:57.675 INFO:tasks.workunit.client.0.smithi063.stdout:3/524: dwrite da/f71 [0,4194304] 0 2024-03-22T07:15:57.676 INFO:tasks.workunit.client.1.smithi069.stdout:0/211: dwrite d3/d7/d15/f32 [0,4194304] 0 2024-03-22T07:15:57.676 INFO:tasks.workunit.client.1.smithi069.stdout:0/212: write d3/f13 [1181392,98183] 0 2024-03-22T07:15:57.676 INFO:tasks.workunit.client.1.smithi069.stdout:4/277: dread d6/dc/ff [0,4194304] 0 2024-03-22T07:15:57.677 INFO:tasks.workunit.client.0.smithi063.stdout:5/437: mkdir d8/d13/d28/d7d 0 2024-03-22T07:15:57.677 INFO:tasks.workunit.client.0.smithi063.stdout:4/385: dwrite d1/d5/f20 [0,4194304] 0 2024-03-22T07:15:57.677 INFO:tasks.workunit.client.0.smithi063.stdout:4/386: write d1/f45 [585159,55344] 0 2024-03-22T07:15:57.679 INFO:tasks.workunit.client.0.smithi063.stdout:7/349: mknod d5/da/d5f/c6e 0 2024-03-22T07:15:57.679 INFO:tasks.workunit.client.0.smithi063.stdout:7/350: dread - d5/d16/f63 zero size 2024-03-22T07:15:57.681 INFO:tasks.workunit.client.0.smithi063.stdout:1/555: mknod d4/df/d13/d39/cbc 0 2024-03-22T07:15:57.684 INFO:tasks.workunit.client.0.smithi063.stdout:3/525: creat da/d63/d65/d92/fb0 x:0 0 0 2024-03-22T07:15:57.686 INFO:tasks.workunit.client.0.smithi063.stdout:6/498: dread d3/d15/f23 [0,4194304] 0 2024-03-22T07:15:57.686 INFO:tasks.workunit.client.0.smithi063.stdout:6/499: write d3/d76/f72 [4424719,86973] 0 2024-03-22T07:15:57.688 INFO:tasks.workunit.client.0.smithi063.stdout:5/438: mkdir d8/d13/d1e/d31/d58/d7e 0 2024-03-22T07:15:57.688 INFO:tasks.workunit.client.0.smithi063.stdout:2/447: dread d0/de/d12/d20/f88 [0,4194304] 0 2024-03-22T07:15:57.692 INFO:tasks.workunit.client.0.smithi063.stdout:4/387: unlink d1/d5/c3f 0 2024-03-22T07:15:57.693 INFO:tasks.workunit.client.0.smithi063.stdout:0/570: dwrite da/de/d60/d56/d77/fa3 [0,4194304] 0 2024-03-22T07:15:57.694 INFO:tasks.workunit.client.0.smithi063.stdout:7/351: symlink d5/da/d5f/d33/l6f 0 2024-03-22T07:15:57.694 INFO:tasks.workunit.client.0.smithi063.stdout:7/352: creat d5/d16/d1b/d21/f70 x:0 0 0 2024-03-22T07:15:57.694 INFO:tasks.workunit.client.0.smithi063.stdout:7/353: truncate d5/f39 4381916 0 2024-03-22T07:15:57.694 INFO:tasks.workunit.client.0.smithi063.stdout:1/556: symlink d4/d71/lbd 0 2024-03-22T07:15:57.695 INFO:tasks.workunit.client.0.smithi063.stdout:3/526: rmdir da/d16/d89 39 2024-03-22T07:15:57.696 INFO:tasks.workunit.client.0.smithi063.stdout:3/527: stat da/d16/d3f/f67 0 2024-03-22T07:15:57.697 INFO:tasks.workunit.client.0.smithi063.stdout:5/439: link d8/d13/d1e/l29 d8/d13/d1e/d31/d64/d65/d6a/l7f 0 2024-03-22T07:15:57.697 INFO:tasks.workunit.client.0.smithi063.stdout:5/440: write d8/d13/d1e/d31/d58/f68 [817968,34881] 0 2024-03-22T07:15:57.700 INFO:tasks.workunit.client.1.smithi069.stdout:1/217: sync 2024-03-22T07:15:57.700 INFO:tasks.workunit.client.1.smithi069.stdout:1/218: write d1/f35 [1232165,2827] 0 2024-03-22T07:15:57.701 INFO:tasks.workunit.client.1.smithi069.stdout:1/219: symlink d1/d13/l49 0 2024-03-22T07:15:57.701 INFO:tasks.workunit.client.1.smithi069.stdout:1/220: chown d1/d13/l49 464 1 2024-03-22T07:15:57.702 INFO:tasks.workunit.client.0.smithi063.stdout:6/500: dread d3/d6/f3e [0,4194304] 0 2024-03-22T07:15:57.703 INFO:tasks.workunit.client.1.smithi069.stdout:1/221: unlink d1/l11 0 2024-03-22T07:15:57.703 INFO:tasks.workunit.client.1.smithi069.stdout:1/222: stat d1/d13/f2d 0 2024-03-22T07:15:57.703 INFO:tasks.workunit.client.0.smithi063.stdout:4/388: creat d1/d5/dc/f89 x:0 0 0 2024-03-22T07:15:57.704 INFO:tasks.workunit.client.1.smithi069.stdout:1/223: rename d1/d13/c47 to d1/d25/d41/c4a 0 2024-03-22T07:15:57.704 INFO:tasks.workunit.client.1.smithi069.stdout:1/224: write d1/d13/f39 [620572,16965] 0 2024-03-22T07:15:57.706 INFO:tasks.workunit.client.1.smithi069.stdout:4/278: dwrite d6/d12/d18/f4f [0,4194304] 0 2024-03-22T07:15:57.706 INFO:tasks.workunit.client.1.smithi069.stdout:4/279: chown d6/ca 87374667 1 2024-03-22T07:15:57.706 INFO:tasks.workunit.client.1.smithi069.stdout:4/280: truncate d6/dc/d1d/d31/d1f/d3f/f46 650579 0 2024-03-22T07:15:57.706 INFO:tasks.workunit.client.1.smithi069.stdout:4/281: chown d6/d12/d18/f4f 61873777 1 2024-03-22T07:15:57.707 INFO:tasks.workunit.client.1.smithi069.stdout:0/213: dwrite d3/d7/d15/d1f/f21 [4194304,4194304] 0 2024-03-22T07:15:57.707 INFO:tasks.workunit.client.1.smithi069.stdout:0/214: getdents d3/d7/d17 0 2024-03-22T07:15:57.708 INFO:tasks.workunit.client.0.smithi063.stdout:0/571: mkdir da/de/d17/d25/d3c/d43/db5 0 2024-03-22T07:15:57.708 INFO:tasks.workunit.client.0.smithi063.stdout:0/572: truncate da/de/d17/d25/d3c/f66 1046251 0 2024-03-22T07:15:57.710 INFO:tasks.workunit.client.1.smithi069.stdout:4/282: creat d6/f53 x:0 0 0 2024-03-22T07:15:57.711 INFO:tasks.workunit.client.0.smithi063.stdout:7/354: mknod d5/da/d5f/c71 0 2024-03-22T07:15:57.712 INFO:tasks.workunit.client.0.smithi063.stdout:3/528: symlink da/d63/lb1 0 2024-03-22T07:15:57.712 INFO:tasks.workunit.client.0.smithi063.stdout:3/529: fsync da/d63/d65/d92/d96/f9a 0 2024-03-22T07:15:57.712 INFO:tasks.workunit.client.0.smithi063.stdout:3/530: fsync da/d63/d65/d81/da5/fad 0 2024-03-22T07:15:57.712 INFO:tasks.workunit.client.0.smithi063.stdout:3/531: read da/fb [283416,105308] 0 2024-03-22T07:15:57.713 INFO:tasks.workunit.client.1.smithi069.stdout:4/283: symlink d6/l54 0 2024-03-22T07:15:57.715 INFO:tasks.workunit.client.1.smithi069.stdout:1/225: dread d1/d13/d37/f40 [0,4194304] 0 2024-03-22T07:15:57.716 INFO:tasks.workunit.client.1.smithi069.stdout:4/284: dread d6/dc/d23/f3c [0,4194304] 0 2024-03-22T07:15:57.717 INFO:tasks.workunit.client.1.smithi069.stdout:4/285: unlink d6/f53 0 2024-03-22T07:15:57.718 INFO:tasks.workunit.client.1.smithi069.stdout:4/286: mkdir d6/dc/d1d/d55 0 2024-03-22T07:15:57.718 INFO:tasks.workunit.client.1.smithi069.stdout:4/287: write d6/d4b/f4d [530792,29501] 0 2024-03-22T07:15:57.719 INFO:tasks.workunit.client.0.smithi063.stdout:1/557: dwrite d4/f93 [0,4194304] 0 2024-03-22T07:15:57.719 INFO:tasks.workunit.client.0.smithi063.stdout:1/558: write d4/f9 [506581,15122] 0 2024-03-22T07:15:57.719 INFO:tasks.workunit.client.0.smithi063.stdout:6/501: mkdir d3/d15/da3 0 2024-03-22T07:15:57.721 INFO:tasks.workunit.client.1.smithi069.stdout:4/288: mkdir d6/dc/d1d/d31/d56 0 2024-03-22T07:15:57.721 INFO:tasks.workunit.client.1.smithi069.stdout:4/289: chown f5 7003 1 2024-03-22T07:15:57.721 INFO:tasks.workunit.client.0.smithi063.stdout:6/502: dread d3/d15/f85 [0,4194304] 0 2024-03-22T07:15:57.722 INFO:tasks.workunit.client.0.smithi063.stdout:4/389: creat d1/d5/d85/f8a x:0 0 0 2024-03-22T07:15:57.723 INFO:tasks.workunit.client.1.smithi069.stdout:4/290: creat d6/d16/f57 x:0 0 0 2024-03-22T07:15:57.723 INFO:tasks.workunit.client.1.smithi069.stdout:4/291: chown d6/dc/d1d/l22 4269856 1 2024-03-22T07:15:57.724 INFO:tasks.workunit.client.1.smithi069.stdout:4/292: mkdir d6/dc/d1d/d31/d1f/d58 0 2024-03-22T07:15:57.726 INFO:tasks.workunit.client.1.smithi069.stdout:4/293: mkdir d6/dc/d23/d59 0 2024-03-22T07:15:57.726 INFO:tasks.workunit.client.1.smithi069.stdout:4/294: write d6/f9 [4186267,76491] 0 2024-03-22T07:15:57.726 INFO:tasks.workunit.client.0.smithi063.stdout:5/441: dwrite d8/d13/d1e/f79 [0,4194304] 0 2024-03-22T07:15:57.728 INFO:tasks.workunit.client.1.smithi069.stdout:4/295: symlink d6/dc/d23/d59/l5a 0 2024-03-22T07:15:57.728 INFO:tasks.workunit.client.1.smithi069.stdout:4/296: write d6/d12/d18/f4f [4382864,19341] 0 2024-03-22T07:15:57.728 INFO:tasks.workunit.client.1.smithi069.stdout:4/297: stat d6/d12/d18/f4f 0 2024-03-22T07:15:57.728 INFO:tasks.workunit.client.1.smithi069.stdout:4/298: truncate d6/d4b/f4d 1239513 0 2024-03-22T07:15:57.728 INFO:tasks.workunit.client.1.smithi069.stdout:4/299: chown d6/dc 0 1 2024-03-22T07:15:57.732 INFO:tasks.workunit.client.1.smithi069.stdout:4/300: read d6/dc/f1a [123754,15354] 0 2024-03-22T07:15:57.739 INFO:tasks.workunit.client.0.smithi063.stdout:3/532: dwrite da/d63/d65/d81/d9b/d9f/fa2 [0,4194304] 0 2024-03-22T07:15:57.740 INFO:tasks.workunit.client.0.smithi063.stdout:3/533: dread - da/d16/f97 zero size 2024-03-22T07:15:57.740 INFO:tasks.workunit.client.1.smithi069.stdout:0/215: dwrite d3/d7/d15/f1b [0,4194304] 0 2024-03-22T07:15:57.741 INFO:tasks.workunit.client.0.smithi063.stdout:7/355: dwrite d5/d16/d1b/f52 [0,4194304] 0 2024-03-22T07:15:57.741 INFO:tasks.workunit.client.0.smithi063.stdout:7/356: creat d5/d14/d1d/d29/d3b/f72 x:0 0 0 2024-03-22T07:15:57.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.744+0000 7f346c285700 1 -- 172.21.15.63:0/2862734893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34640662e0 msgr2=0x7f3464101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:57.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.744+0000 7f346c285700 1 --2- 172.21.15.63:0/2862734893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34640662e0 0x7f3464101dc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3460004940 tx=0x7f3460009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:57.744 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:57 smithi063 ceph-mon[29200]: pgmap v251: 65 pgs: 65 active+clean; 1.0 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 105 MiB/s rd, 129 MiB/s wr, 791 op/s 2024-03-22T07:15:57.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.744+0000 7f346c285700 1 -- 172.21.15.63:0/2862734893 shutdown_connections 2024-03-22T07:15:57.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.744+0000 7f346c285700 1 --2- 172.21.15.63:0/2862734893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3464102300 0x7f34641046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:57.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.744+0000 7f346c285700 1 --2- 172.21.15.63:0/2862734893 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f34640662e0 0x7f3464101dc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:57.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.744+0000 7f346c285700 1 -- 172.21.15.63:0/2862734893 >> 172.21.15.63:0/2862734893 conn(0x7f34640f7430 msgr2=0x7f34640f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:57.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.744+0000 7f346c285700 1 -- 172.21.15.63:0/2862734893 shutdown_connections 2024-03-22T07:15:57.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.744+0000 7f346c285700 1 -- 172.21.15.63:0/2862734893 wait complete. 2024-03-22T07:15:57.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.745+0000 7f346c285700 1 Processor -- start 2024-03-22T07:15:57.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.745+0000 7f346c285700 1 -- start start 2024-03-22T07:15:57.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.746+0000 7f346c285700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34640662e0 0x7f34640fdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:57.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.746+0000 7f346c285700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3464102300 0x7f34640fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:57.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.746+0000 7f346c285700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34640fe8f0 con 0x7f3464102300 2024-03-22T07:15:57.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.746+0000 7f346c285700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34640ff140 con 0x7f34640662e0 2024-03-22T07:15:57.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.746+0000 7f346a021700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34640662e0 0x7f34640fdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:57.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.746+0000 7f3469820700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3464102300 0x7f34640fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:57.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.746+0000 7f346a021700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34640662e0 0x7f34640fdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.63:58656/0 (socket says 172.21.15.63:58656) 2024-03-22T07:15:57.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.746+0000 7f346a021700 1 -- 172.21.15.63:0/2697159806 learned_addr learned my addr 172.21.15.63:0/2697159806 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:15:57.748 INFO:tasks.workunit.client.1.smithi069.stdout:2/175: sync 2024-03-22T07:15:57.748 INFO:tasks.workunit.client.1.smithi069.stdout:5/147: sync 2024-03-22T07:15:57.748 INFO:tasks.workunit.client.1.smithi069.stdout:9/188: sync 2024-03-22T07:15:57.748 INFO:tasks.workunit.client.1.smithi069.stdout:3/177: sync 2024-03-22T07:15:57.748 INFO:tasks.workunit.client.1.smithi069.stdout:5/148: fdatasync d1/d7/d8/f15 0 2024-03-22T07:15:57.748 INFO:tasks.workunit.client.1.smithi069.stdout:2/176: creat d18/d1c/f41 x:0 0 0 2024-03-22T07:15:57.748 INFO:tasks.workunit.client.1.smithi069.stdout:2/177: fdatasync d18/d1c/d25/f2e 0 2024-03-22T07:15:57.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.747+0000 7f3469820700 1 -- 172.21.15.63:0/2697159806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34640662e0 msgr2=0x7f34640fdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:57.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.747+0000 7f3469820700 1 --2- 172.21.15.63:0/2697159806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34640662e0 0x7f34640fdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:57.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.747+0000 7f3469820700 1 -- 172.21.15.63:0/2697159806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3460005040 con 0x7f3464102300 2024-03-22T07:15:57.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.747+0000 7f346a021700 1 --2- 172.21.15.63:0/2697159806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34640662e0 0x7f34640fdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:15:57.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.747+0000 7f3469820700 1 --2- 172.21.15.63:0/2697159806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3464102300 0x7f34640fe2a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3454002870 tx=0x7f3454004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:57.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.747+0000 7f345affd700 1 -- 172.21.15.63:0/2697159806 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3454007cc0 con 0x7f3464102300 2024-03-22T07:15:57.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.748+0000 7f345affd700 1 -- 172.21.15.63:0/2697159806 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f345400f040 con 0x7f3464102300 2024-03-22T07:15:57.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.748+0000 7f345affd700 1 -- 172.21.15.63:0/2697159806 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34540139a0 con 0x7f3464102300 2024-03-22T07:15:57.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.748+0000 7f346c285700 1 -- 172.21.15.63:0/2697159806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34640ff420 con 0x7f3464102300 2024-03-22T07:15:57.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.748+0000 7f346c285700 1 -- 172.21.15.63:0/2697159806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34640ff9e0 con 0x7f3464102300 2024-03-22T07:15:57.751 INFO:tasks.workunit.client.1.smithi069.stdout:9/189: creat d2/dd/d3c/f3e x:0 0 0 2024-03-22T07:15:57.751 INFO:tasks.workunit.client.1.smithi069.stdout:9/190: dread - d2/dd/d3c/f3e zero size 2024-03-22T07:15:57.751 INFO:tasks.workunit.client.1.smithi069.stdout:9/191: rename d2/dd/f38 to d2/dd/d3c/d2e/f3f 0 2024-03-22T07:15:57.751 INFO:tasks.workunit.client.1.smithi069.stdout:9/192: chown d2/dd/d3c/f35 7 1 2024-03-22T07:15:57.751 INFO:tasks.workunit.client.1.smithi069.stdout:9/193: fdatasync d2/dd/d3c/d2e/f32 0 2024-03-22T07:15:57.751 INFO:tasks.workunit.client.1.smithi069.stdout:9/194: fsync d2/f5 0 2024-03-22T07:15:57.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.750+0000 7f346c285700 1 -- 172.21.15.63:0/2697159806 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f346405f5f0 con 0x7f3464102300 2024-03-22T07:15:57.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.750+0000 7f345affd700 1 -- 172.21.15.63:0/2697159806 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 20) v1 ==== 88751+0+0 (secure 0 0 0) 0x7f345401c400 con 0x7f3464102300 2024-03-22T07:15:57.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.751+0000 7f345affd700 1 --2- 172.21.15.63:0/2697159806 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f345006af90 0x7f345006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:57.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.751+0000 7f345affd700 1 -- 172.21.15.63:0/2697159806 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f3454088d90 con 0x7f3464102300 2024-03-22T07:15:57.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.751+0000 7f346a021700 1 --2- 172.21.15.63:0/2697159806 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f345006af90 0x7f345006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:57.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.752+0000 7f346a021700 1 --2- 172.21.15.63:0/2697159806 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f345006af90 0x7f345006d440 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3460004f90 tx=0x7f3460009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:57.752 INFO:tasks.workunit.client.0.smithi063.stdout:7/357: write d5/d14/f61 [2373369,125696] 0 2024-03-22T07:15:57.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:57.757+0000 7f345affd700 1 -- 172.21.15.63:0/2697159806 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f3454055070 con 0x7f3464102300 2024-03-22T07:15:57.758 INFO:tasks.workunit.client.1.smithi069.stdout:2/178: dread d18/f2b [0,4194304] 0 2024-03-22T07:15:57.758 INFO:tasks.workunit.client.1.smithi069.stdout:2/179: dread - d18/f1b zero size 2024-03-22T07:15:57.758 INFO:tasks.workunit.client.1.smithi069.stdout:2/180: symlink d18/d1c/d34/l42 0 2024-03-22T07:15:57.759 INFO:tasks.workunit.client.1.smithi069.stdout:2/181: write d18/f1b [313506,130988] 0 2024-03-22T07:15:57.759 INFO:tasks.workunit.client.1.smithi069.stdout:2/182: mknod d18/d1c/c43 0 2024-03-22T07:15:57.759 INFO:tasks.workunit.client.1.smithi069.stdout:2/183: read d18/d1c/f22 [2514441,28453] 0 2024-03-22T07:15:57.760 INFO:tasks.workunit.client.1.smithi069.stdout:2/184: truncate d18/f1e 524936 0 2024-03-22T07:15:57.761 INFO:tasks.workunit.client.1.smithi069.stdout:2/185: creat d18/d1c/d3e/d40/f44 x:0 0 0 2024-03-22T07:15:57.762 INFO:tasks.workunit.client.1.smithi069.stdout:2/186: symlink d18/d1c/l45 0 2024-03-22T07:15:57.764 INFO:tasks.workunit.client.1.smithi069.stdout:3/178: dwrite d5/d9/d1e/f21 [0,4194304] 0 2024-03-22T07:15:57.764 INFO:tasks.workunit.client.1.smithi069.stdout:0/216: dwrite d3/d7/d15/f20 [0,4194304] 0 2024-03-22T07:15:57.767 INFO:tasks.workunit.client.1.smithi069.stdout:3/179: mkdir d5/d9/db/d3b 0 2024-03-22T07:15:57.769 INFO:tasks.workunit.client.1.smithi069.stdout:3/180: mkdir d5/d33/d3c 0 2024-03-22T07:15:57.770 INFO:tasks.workunit.client.1.smithi069.stdout:3/181: creat d5/d9/db/f3d x:0 0 0 2024-03-22T07:15:57.770 INFO:tasks.workunit.client.1.smithi069.stdout:2/187: write d18/f1e [902665,71906] 0 2024-03-22T07:15:57.771 INFO:tasks.workunit.client.1.smithi069.stdout:0/217: mknod d3/d7/d15/c47 0 2024-03-22T07:15:57.773 INFO:tasks.workunit.client.1.smithi069.stdout:9/195: dwrite d2/dd/d3c/f20 [4194304,4194304] 0 2024-03-22T07:15:57.774 INFO:tasks.workunit.client.1.smithi069.stdout:9/196: write d2/dd/d3c/f3e [77693,64199] 0 2024-03-22T07:15:57.775 INFO:tasks.workunit.client.1.smithi069.stdout:2/188: getdents d18 0 2024-03-22T07:15:57.775 INFO:tasks.workunit.client.1.smithi069.stdout:2/189: chown d18/d1c/d27/d33 7 1 2024-03-22T07:15:57.776 INFO:tasks.workunit.client.1.smithi069.stdout:9/197: link d2/dd/d3c/d2e/f32 d2/d31/f40 0 2024-03-22T07:15:57.776 INFO:tasks.workunit.client.1.smithi069.stdout:9/198: write d2/dd/d3c/f20 [353815,11039] 0 2024-03-22T07:15:57.777 INFO:tasks.workunit.client.1.smithi069.stdout:2/190: creat d18/d1c/d27/d33/d3d/f46 x:0 0 0 2024-03-22T07:15:57.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:57 smithi069 ceph-mon[36441]: pgmap v251: 65 pgs: 65 active+clean; 1.0 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 105 MiB/s rd, 129 MiB/s wr, 791 op/s 2024-03-22T07:15:57.806 INFO:tasks.workunit.client.1.smithi069.stdout:3/182: dwrite d5/d9/d1e/f32 [0,4194304] 0 2024-03-22T07:15:57.807 INFO:tasks.workunit.client.1.smithi069.stdout:0/218: dwrite d3/f13 [0,4194304] 0 2024-03-22T07:15:57.807 INFO:tasks.workunit.client.1.smithi069.stdout:0/219: write d3/d7/d17/f1a [4379399,128870] 0 2024-03-22T07:15:57.807 INFO:tasks.workunit.client.1.smithi069.stdout:2/191: dwrite d18/f19 [0,4194304] 0 2024-03-22T07:15:57.807 INFO:tasks.workunit.client.1.smithi069.stdout:2/192: creat d18/d1c/d25/f47 x:0 0 0 2024-03-22T07:15:57.810 INFO:tasks.workunit.client.1.smithi069.stdout:0/220: dread d3/d7/d15/d29/f30 [0,4194304] 0 2024-03-22T07:15:57.813 INFO:tasks.workunit.client.0.smithi063.stdout:0/573: mknod da/de/d17/d25/d3c/d43/d80/cb6 0 2024-03-22T07:15:57.817 INFO:tasks.workunit.client.0.smithi063.stdout:1/559: link d4/f76 d4/df/fbe 0 2024-03-22T07:15:57.822 INFO:tasks.workunit.client.0.smithi063.stdout:4/390: creat d1/d5/d43/d27/d2c/d4a/f8b x:0 0 0 2024-03-22T07:15:57.822 INFO:tasks.workunit.client.0.smithi063.stdout:4/391: readlink d1/d5/d43/d27/d2c/d3d/l77 0 2024-03-22T07:15:57.823 INFO:tasks.workunit.client.0.smithi063.stdout:4/392: dread d1/d5/dc/f41 [0,4194304] 0 2024-03-22T07:15:57.823 INFO:tasks.workunit.client.0.smithi063.stdout:4/393: readlink d1/d5/dc/l37 0 2024-03-22T07:15:57.828 INFO:tasks.workunit.client.0.smithi063.stdout:5/442: unlink d8/d13/d1e/d31/d64/d65/d6a/l7f 0 2024-03-22T07:15:57.830 INFO:tasks.workunit.client.1.smithi069.stdout:0/221: dwrite d3/f39 [0,4194304] 0 2024-03-22T07:15:57.830 INFO:tasks.workunit.client.1.smithi069.stdout:0/222: stat d3/f40 0 2024-03-22T07:15:57.831 INFO:tasks.workunit.client.0.smithi063.stdout:3/534: fdatasync da/d16/d26/f76 0 2024-03-22T07:15:57.832 INFO:tasks.workunit.client.1.smithi069.stdout:0/223: mknod d3/d7/d17/c48 0 2024-03-22T07:15:57.843 INFO:tasks.workunit.client.0.smithi063.stdout:5/443: read d8/d13/d1e/d3c/f49 [1441634,8314] 0 2024-03-22T07:15:57.843 INFO:tasks.workunit.client.0.smithi063.stdout:1/560: dwrite d4/d5a/f65 [4194304,4194304] 0 2024-03-22T07:15:57.843 INFO:tasks.workunit.client.0.smithi063.stdout:1/561: creat d4/df/d13/d2b/d86/fbf x:0 0 0 2024-03-22T07:15:57.843 INFO:tasks.workunit.client.0.smithi063.stdout:0/574: dwrite f5 [0,4194304] 0 2024-03-22T07:15:57.844 INFO:tasks.workunit.client.1.smithi069.stdout:0/224: write d3/f39 [580034,83807] 0 2024-03-22T07:15:57.846 INFO:tasks.workunit.client.0.smithi063.stdout:4/394: creat d1/d5/d43/f8c x:0 0 0 2024-03-22T07:15:57.846 INFO:tasks.workunit.client.0.smithi063.stdout:4/395: dread - d1/d5/d3c/d65/f73 zero size 2024-03-22T07:15:57.855 INFO:tasks.workunit.client.1.smithi069.stdout:0/225: write d3/d7/d24/f34 [3566847,61484] 0 2024-03-22T07:15:57.856 INFO:tasks.workunit.client.1.smithi069.stdout:0/226: creat d3/d7/f49 x:0 0 0 2024-03-22T07:15:57.857 INFO:tasks.workunit.client.0.smithi063.stdout:1/562: mkdir d4/d71/dbb/dc0 0 2024-03-22T07:15:57.857 INFO:tasks.workunit.client.0.smithi063.stdout:1/563: getdents d4/df/d13/d2b/d6c/da8/db2 0 2024-03-22T07:15:57.857 INFO:tasks.workunit.client.0.smithi063.stdout:3/535: dwrite da/d63/d65/f77 [8388608,4194304] 0 2024-03-22T07:15:57.858 INFO:tasks.workunit.client.1.smithi069.stdout:0/227: rmdir d3/d7/d24 39 2024-03-22T07:15:57.858 INFO:tasks.workunit.client.0.smithi063.stdout:0/575: mknod da/de/d60/cb7 0 2024-03-22T07:15:57.858 INFO:tasks.workunit.client.0.smithi063.stdout:5/444: dwrite d8/f44 [0,4194304] 0 2024-03-22T07:15:57.859 INFO:tasks.workunit.client.0.smithi063.stdout:5/445: creat d8/d13/d1e/f80 x:0 0 0 2024-03-22T07:15:57.859 INFO:tasks.workunit.client.1.smithi069.stdout:0/228: rename d3/f40 to d3/d7/f4a 0 2024-03-22T07:15:57.860 INFO:tasks.workunit.client.0.smithi063.stdout:4/396: creat d1/d5/d43/d27/d2c/d3d/d54/f8d x:0 0 0 2024-03-22T07:15:57.860 INFO:tasks.workunit.client.1.smithi069.stdout:0/229: mknod d3/d7/d15/d29/c4b 0 2024-03-22T07:15:57.862 INFO:tasks.workunit.client.0.smithi063.stdout:0/576: read da/f54 [203447,102650] 0 2024-03-22T07:15:57.862 INFO:tasks.workunit.client.0.smithi063.stdout:0/577: read da/de/d17/d25/f68 [431067,61488] 0 2024-03-22T07:15:57.866 INFO:tasks.workunit.client.0.smithi063.stdout:1/564: symlink d4/d5a/lc1 0 2024-03-22T07:15:57.866 INFO:tasks.workunit.client.0.smithi063.stdout:1/565: chown d4/d71/d88/da1 1284223 1 2024-03-22T07:15:57.871 INFO:tasks.workunit.client.1.smithi069.stdout:0/230: write d3/f13 [3189506,50784] 0 2024-03-22T07:15:57.871 INFO:tasks.workunit.client.1.smithi069.stdout:0/231: stat d3/d7/d15/d1f/f41 0 2024-03-22T07:15:57.872 INFO:tasks.workunit.client.1.smithi069.stdout:0/232: creat d3/d7/d17/f4c x:0 0 0 2024-03-22T07:15:57.872 INFO:tasks.workunit.client.0.smithi063.stdout:3/536: read da/d16/d26/f76 [6588082,93278] 0 2024-03-22T07:15:57.876 INFO:tasks.workunit.client.0.smithi063.stdout:0/578: link da/de/d39/f90 da/de/d17/d25/d3c/d5f/fb8 0 2024-03-22T07:15:57.876 INFO:tasks.workunit.client.0.smithi063.stdout:0/579: creat da/de/d60/d56/d8b/fb9 x:0 0 0 2024-03-22T07:15:57.876 INFO:tasks.workunit.client.0.smithi063.stdout:0/580: read da/de/f23 [3914545,129350] 0 2024-03-22T07:15:57.878 INFO:tasks.workunit.client.0.smithi063.stdout:3/537: creat da/d63/d65/d4b/fb2 x:0 0 0 2024-03-22T07:15:57.880 INFO:tasks.workunit.client.0.smithi063.stdout:4/397: mknod d1/d5/d43/d27/d2c/d3d/d54/d5f/c8e 0 2024-03-22T07:15:57.882 INFO:tasks.workunit.client.0.smithi063.stdout:0/581: symlink da/de/d67/lba 0 2024-03-22T07:15:57.884 INFO:tasks.workunit.client.0.smithi063.stdout:3/538: mknod da/d16/d3f/cb3 0 2024-03-22T07:15:57.885 INFO:tasks.workunit.client.0.smithi063.stdout:4/398: rmdir d1/d5/d43/d27/d2c/d4a/d78 39 2024-03-22T07:15:57.887 INFO:tasks.workunit.client.0.smithi063.stdout:4/399: write d1/d5/d43/f35 [435406,85307] 0 2024-03-22T07:15:57.889 INFO:tasks.workunit.client.0.smithi063.stdout:5/446: dwrite d8/d13/d1e/d3c/f4c [0,4194304] 0 2024-03-22T07:15:57.889 INFO:tasks.workunit.client.0.smithi063.stdout:3/539: dread da/d16/d26/f2a [0,4194304] 0 2024-03-22T07:15:57.893 INFO:tasks.workunit.client.0.smithi063.stdout:1/566: dwrite d4/df/d13/d2b/d3a/f81 [0,4194304] 0 2024-03-22T07:15:57.893 INFO:tasks.workunit.client.0.smithi063.stdout:1/567: write d4/df/f1a [3951990,119922] 0 2024-03-22T07:15:57.893 INFO:tasks.workunit.client.0.smithi063.stdout:1/568: dread - d4/d5a/fa4 zero size 2024-03-22T07:15:57.893 INFO:tasks.workunit.client.1.smithi069.stdout:0/233: dwrite d3/d7/d15/d1f/f21 [0,4194304] 0 2024-03-22T07:15:57.895 INFO:tasks.workunit.client.0.smithi063.stdout:0/582: dread da/de/d60/f69 [0,4194304] 0 2024-03-22T07:15:57.898 INFO:tasks.workunit.client.0.smithi063.stdout:3/540: dread da/d16/f25 [0,4194304] 0 2024-03-22T07:15:57.898 INFO:tasks.workunit.client.0.smithi063.stdout:3/541: chown da/d63 45965122 1 2024-03-22T07:15:57.899 INFO:tasks.workunit.client.1.smithi069.stdout:0/234: rename d3/d7/d15/d1f/l35 to d3/d7/d24/l4d 0 2024-03-22T07:15:57.904 INFO:tasks.workunit.client.0.smithi063.stdout:0/583: mknod da/de/d17/d25/d95/d98/db3/cbb 0 2024-03-22T07:15:57.905 INFO:tasks.workunit.client.0.smithi063.stdout:7/358: rename d5/d14 to d5/d16/d1b/d21/d62/d73 0 2024-03-22T07:15:57.907 INFO:tasks.workunit.client.0.smithi063.stdout:0/584: mkdir da/de/d17/d25/d99/dbc 0 2024-03-22T07:15:57.912 INFO:tasks.workunit.client.0.smithi063.stdout:1/569: read d4/d5a/f65 [5074214,12253] 0 2024-03-22T07:15:57.912 INFO:tasks.workunit.client.0.smithi063.stdout:7/359: link d5/da/d5f/c71 d5/d16/d1b/d21/d62/d73/d1d/d4f/c74 0 2024-03-22T07:15:57.914 INFO:tasks.workunit.client.1.smithi069.stdout:6/139: sync 2024-03-22T07:15:57.915 INFO:tasks.workunit.client.1.smithi069.stdout:7/270: sync 2024-03-22T07:15:57.915 INFO:tasks.workunit.client.1.smithi069.stdout:8/208: sync 2024-03-22T07:15:57.915 INFO:tasks.workunit.client.0.smithi063.stdout:0/585: dread da/de/d17/d25/d3c/d64/fac [0,4194304] 0 2024-03-22T07:15:57.915 INFO:tasks.workunit.client.0.smithi063.stdout:0/586: chown da/d49/f63 230960995 1 2024-03-22T07:15:57.917 INFO:tasks.workunit.client.1.smithi069.stdout:6/140: rename d4/dd/l29 to d4/d7/l2d 0 2024-03-22T07:15:57.918 INFO:tasks.workunit.client.0.smithi063.stdout:1/570: dread d4/df/fbe [0,4194304] 0 2024-03-22T07:15:57.918 INFO:tasks.workunit.client.1.smithi069.stdout:8/209: mknod d1/de/d20/c3a 0 2024-03-22T07:15:57.918 INFO:tasks.workunit.client.1.smithi069.stdout:8/210: truncate d1/de/d20/f26 2930443 0 2024-03-22T07:15:57.919 INFO:tasks.workunit.client.0.smithi063.stdout:7/360: symlink d5/da/d4e/l75 0 2024-03-22T07:15:57.919 INFO:tasks.workunit.client.1.smithi069.stdout:7/271: symlink d9/db/d48/d50/d57/d44/d53/d1d/l5e 0 2024-03-22T07:15:57.920 INFO:tasks.workunit.client.0.smithi063.stdout:1/571: unlink d4/df/l7e 0 2024-03-22T07:15:57.921 INFO:tasks.workunit.client.1.smithi069.stdout:8/211: rmdir d1/d1c 39 2024-03-22T07:15:57.921 INFO:tasks.workunit.client.1.smithi069.stdout:8/212: stat d1/de/d20/c3a 0 2024-03-22T07:15:57.921 INFO:tasks.workunit.client.1.smithi069.stdout:8/213: chown d1/d1c/c21 3 1 2024-03-22T07:15:57.921 INFO:tasks.workunit.client.1.smithi069.stdout:8/214: fsync d1/de/f1a 0 2024-03-22T07:15:57.921 INFO:tasks.workunit.client.1.smithi069.stdout:8/215: creat d1/de/d20/f3b x:0 0 0 2024-03-22T07:15:57.921 INFO:tasks.workunit.client.1.smithi069.stdout:8/216: creat d1/de/d20/f3c x:0 0 0 2024-03-22T07:15:57.921 INFO:tasks.workunit.client.1.smithi069.stdout:8/217: readlink d1/de/l2a 0 2024-03-22T07:15:57.922 INFO:tasks.workunit.client.0.smithi063.stdout:1/572: symlink d4/df/d13/d2b/d6c/da8/lc2 0 2024-03-22T07:15:57.922 INFO:tasks.workunit.client.0.smithi063.stdout:1/573: symlink d4/df/d13/d2b/d3a/lc3 0 2024-03-22T07:15:57.923 INFO:tasks.workunit.client.0.smithi063.stdout:1/574: chown d4/d14/d57/f80 162 1 2024-03-22T07:15:57.923 INFO:tasks.workunit.client.1.smithi069.stdout:8/218: mknod d1/de/d20/c3d 0 2024-03-22T07:15:57.924 INFO:tasks.workunit.client.0.smithi063.stdout:1/575: read d4/f3b [839678,86451] 0 2024-03-22T07:15:57.924 INFO:tasks.workunit.client.0.smithi063.stdout:1/576: fdatasync d4/df/d13/d2b/d86/f87 0 2024-03-22T07:15:57.924 INFO:tasks.workunit.client.1.smithi069.stdout:8/219: write d1/f14 [4094805,107235] 0 2024-03-22T07:15:57.925 INFO:tasks.workunit.client.0.smithi063.stdout:5/447: dwrite d8/d13/d1e/f38 [4194304,4194304] 0 2024-03-22T07:15:57.925 INFO:tasks.workunit.client.0.smithi063.stdout:5/448: write d8/d13/d1e/f80 [729908,101895] 0 2024-03-22T07:15:57.925 INFO:tasks.workunit.client.0.smithi063.stdout:1/577: mknod d4/df/d13/d2b/d3a/cc4 0 2024-03-22T07:15:57.926 INFO:tasks.workunit.client.0.smithi063.stdout:4/400: dwrite d1/d5/d3c/f47 [4194304,4194304] 0 2024-03-22T07:15:57.929 INFO:tasks.workunit.client.0.smithi063.stdout:5/449: rmdir d8/d13/d28/d66 39 2024-03-22T07:15:57.929 INFO:tasks.workunit.client.1.smithi069.stdout:6/141: dread d4/dd/f25 [0,4194304] 0 2024-03-22T07:15:57.931 INFO:tasks.workunit.client.0.smithi063.stdout:1/578: link d4/d8b/da7/faa d4/df/d13/d2b/d6c/da8/fc5 0 2024-03-22T07:15:57.931 INFO:tasks.workunit.client.0.smithi063.stdout:1/579: creat d4/d8b/da7/fc6 x:0 0 0 2024-03-22T07:15:57.931 INFO:tasks.workunit.client.0.smithi063.stdout:1/580: fdatasync d4/df/f4e 0 2024-03-22T07:15:57.931 INFO:tasks.workunit.client.0.smithi063.stdout:7/361: dread d5/d16/f32 [0,4194304] 0 2024-03-22T07:15:57.932 INFO:tasks.workunit.client.0.smithi063.stdout:3/542: dwrite da/d16/f97 [0,4194304] 0 2024-03-22T07:15:57.932 INFO:tasks.workunit.client.1.smithi069.stdout:0/235: dwrite d3/d7/f12 [0,4194304] 0 2024-03-22T07:15:57.933 INFO:tasks.workunit.client.1.smithi069.stdout:0/236: creat d3/d7/d15/d1f/f4e x:0 0 0 2024-03-22T07:15:57.933 INFO:tasks.workunit.client.1.smithi069.stdout:8/220: dread d1/de/f1a [0,4194304] 0 2024-03-22T07:15:57.933 INFO:tasks.workunit.client.0.smithi063.stdout:4/401: mkdir d1/d5/d3c/d8f 0 2024-03-22T07:15:57.935 INFO:tasks.workunit.client.1.smithi069.stdout:0/237: getdents d3/d7/d24 0 2024-03-22T07:15:57.935 INFO:tasks.workunit.client.0.smithi063.stdout:1/581: creat d4/df/d13/d2b/d6c/d85/fc7 x:0 0 0 2024-03-22T07:15:57.936 INFO:tasks.workunit.client.0.smithi063.stdout:7/362: link d5/da/f11 d5/da/f76 0 2024-03-22T07:15:57.937 INFO:tasks.workunit.client.1.smithi069.stdout:8/221: unlink d1/c11 0 2024-03-22T07:15:57.938 INFO:tasks.workunit.client.0.smithi063.stdout:3/543: mknod da/d16/d26/d69/cb4 0 2024-03-22T07:15:57.938 INFO:tasks.workunit.client.0.smithi063.stdout:3/544: chown da/d16/l54 181417220 1 2024-03-22T07:15:57.940 INFO:tasks.workunit.client.0.smithi063.stdout:7/363: symlink d5/d16/l77 0 2024-03-22T07:15:57.940 INFO:tasks.workunit.client.0.smithi063.stdout:0/587: dwrite da/de/f31 [4194304,4194304] 0 2024-03-22T07:15:57.940 INFO:tasks.workunit.client.0.smithi063.stdout:0/588: write da/de/fb4 [161058,105555] 0 2024-03-22T07:15:57.942 INFO:tasks.workunit.client.1.smithi069.stdout:6/142: dread f3 [0,4194304] 0 2024-03-22T07:15:57.944 INFO:tasks.workunit.client.0.smithi063.stdout:0/589: rename da/de/d60/d56/la7 to da/de/d17/d25/d3c/d43/d76/lbd 0 2024-03-22T07:15:57.944 INFO:tasks.workunit.client.0.smithi063.stdout:0/590: write da/de/d17/d25/d3c/d43/d80/f86 [946370,80906] 0 2024-03-22T07:15:57.944 INFO:tasks.workunit.client.0.smithi063.stdout:0/591: fdatasync da/de/d17/d4e/f57 0 2024-03-22T07:15:57.944 INFO:tasks.workunit.client.0.smithi063.stdout:0/592: fsync da/de/f62 0 2024-03-22T07:15:57.944 INFO:tasks.workunit.client.0.smithi063.stdout:9/413: sync 2024-03-22T07:15:57.944 INFO:tasks.workunit.client.0.smithi063.stdout:8/518: sync 2024-03-22T07:15:57.945 INFO:tasks.workunit.client.0.smithi063.stdout:6/503: sync 2024-03-22T07:15:57.945 INFO:tasks.workunit.client.0.smithi063.stdout:6/504: chown d3/d15/d16/d69/f83 996 1 2024-03-22T07:15:57.945 INFO:tasks.workunit.client.0.smithi063.stdout:8/519: write d0/d26/f27 [1713914,19940] 0 2024-03-22T07:15:57.945 INFO:tasks.workunit.client.0.smithi063.stdout:6/505: dread - d3/d15/d16/d60/d6d/d4e/d5f/f93 zero size 2024-03-22T07:15:57.945 INFO:tasks.workunit.client.0.smithi063.stdout:8/520: chown d0/d24/d5b/ca5 6434294 1 2024-03-22T07:15:57.945 INFO:tasks.workunit.client.0.smithi063.stdout:6/506: fdatasync d3/f9 0 2024-03-22T07:15:57.945 INFO:tasks.workunit.client.0.smithi063.stdout:2/448: sync 2024-03-22T07:15:57.946 INFO:tasks.workunit.client.1.smithi069.stdout:8/222: rename d1/de/d12 to d1/d1c/d3e 0 2024-03-22T07:15:57.946 INFO:tasks.workunit.client.1.smithi069.stdout:8/223: fdatasync d1/de/f2d 0 2024-03-22T07:15:57.948 INFO:tasks.workunit.client.1.smithi069.stdout:8/224: link d1/d1c/l2b d1/d1c/d3e/l3f 0 2024-03-22T07:15:57.950 INFO:tasks.workunit.client.0.smithi063.stdout:4/402: rename d1/d5/dc/c50 to d1/d5/d38/c90 0 2024-03-22T07:15:57.950 INFO:tasks.workunit.client.0.smithi063.stdout:4/403: chown d1/d5/d3c/f82 1350 1 2024-03-22T07:15:57.950 INFO:tasks.workunit.client.0.smithi063.stdout:4/404: chown d1/d5/d43/d27/d2c/d3d/d54/d5f/c76 69987287 1 2024-03-22T07:15:57.951 INFO:tasks.workunit.client.0.smithi063.stdout:9/414: getdents d8/d19/d20/d22/d29/d31 0 2024-03-22T07:15:57.952 INFO:tasks.workunit.client.0.smithi063.stdout:3/545: write da/f71 [3762599,13929] 0 2024-03-22T07:15:57.957 INFO:tasks.workunit.client.0.smithi063.stdout:0/593: dread da/de/d17/d4e/f9e [0,4194304] 0 2024-03-22T07:15:57.959 INFO:tasks.workunit.client.0.smithi063.stdout:8/521: symlink d0/d1f/la6 0 2024-03-22T07:15:57.959 INFO:tasks.workunit.client.0.smithi063.stdout:8/522: creat d0/d1f/fa7 x:0 0 0 2024-03-22T07:15:57.959 INFO:tasks.workunit.client.0.smithi063.stdout:8/523: readlink d0/d1b/d28/la0 0 2024-03-22T07:15:57.959 INFO:tasks.workunit.client.0.smithi063.stdout:8/524: write d0/d24/f29 [1323270,68937] 0 2024-03-22T07:15:57.960 INFO:tasks.workunit.client.1.smithi069.stdout:6/143: write d4/dd/d12/f1d [1494302,76201] 0 2024-03-22T07:15:57.962 INFO:tasks.workunit.client.1.smithi069.stdout:6/144: getdents d4 0 2024-03-22T07:15:57.963 INFO:tasks.workunit.client.0.smithi063.stdout:2/449: dread d0/de/d1b/f50 [0,4194304] 0 2024-03-22T07:15:57.963 INFO:tasks.workunit.client.0.smithi063.stdout:2/450: fdatasync d0/de/d12/d20/f47 0 2024-03-22T07:15:57.963 INFO:tasks.workunit.client.0.smithi063.stdout:2/451: chown d0/f51 27682722 1 2024-03-22T07:15:57.963 INFO:tasks.workunit.client.0.smithi063.stdout:5/450: dwrite d8/fe [4194304,4194304] 0 2024-03-22T07:15:57.963 INFO:tasks.workunit.client.0.smithi063.stdout:9/415: mknod d8/d19/d20/d22/d29/d31/d4b/c81 0 2024-03-22T07:15:57.965 INFO:tasks.workunit.client.0.smithi063.stdout:9/416: creat d8/d19/d20/d22/d56/f82 x:0 0 0 2024-03-22T07:15:57.965 INFO:tasks.workunit.client.0.smithi063.stdout:9/417: truncate d8/d19/d20/d22/d56/f75 184829 0 2024-03-22T07:15:57.965 INFO:tasks.workunit.client.0.smithi063.stdout:9/418: fdatasync d8/d19/d20/d22/d56/f75 0 2024-03-22T07:15:57.965 INFO:tasks.workunit.client.0.smithi063.stdout:9/419: stat d8/d19/d20/d22/d56/f82 0 2024-03-22T07:15:57.966 INFO:tasks.workunit.client.0.smithi063.stdout:3/546: creat da/d16/d89/fb5 x:0 0 0 2024-03-22T07:15:57.967 INFO:tasks.workunit.client.0.smithi063.stdout:3/547: chown da/l14 60441478 1 2024-03-22T07:15:57.967 INFO:tasks.workunit.client.0.smithi063.stdout:3/548: write da/f57 [4285992,103796] 0 2024-03-22T07:15:57.970 INFO:tasks.workunit.client.1.smithi069.stdout:8/225: write d1/d1c/f38 [302143,16385] 0 2024-03-22T07:15:57.970 INFO:tasks.workunit.client.1.smithi069.stdout:8/226: truncate d1/d1c/d3e/f16 2538600 0 2024-03-22T07:15:57.973 INFO:tasks.workunit.client.0.smithi063.stdout:8/525: link d0/d1b/d28/d7b/f9b d0/d26/d2d/d83/fa8 0 2024-03-22T07:15:57.973 INFO:tasks.workunit.client.0.smithi063.stdout:8/526: fdatasync d0/d1b/d28/d7b/d45/f58 0 2024-03-22T07:15:57.973 INFO:tasks.workunit.client.0.smithi063.stdout:8/527: write d0/d1f/f98 [520211,88297] 0 2024-03-22T07:15:57.973 INFO:tasks.workunit.client.0.smithi063.stdout:8/528: chown d0/l3 3 1 2024-03-22T07:15:57.977 INFO:tasks.workunit.client.1.smithi069.stdout:6/145: write d4/d7/f2c [340464,102969] 0 2024-03-22T07:15:57.978 INFO:tasks.workunit.client.0.smithi063.stdout:5/451: unlink d8/d13/d1e/d31/d64/d65/f6c 0 2024-03-22T07:15:57.979 INFO:tasks.workunit.client.0.smithi063.stdout:2/452: dread d0/de/d35/d36/f3a [0,4194304] 0 2024-03-22T07:15:57.980 INFO:tasks.workunit.client.0.smithi063.stdout:2/453: write d0/de/d12/d20/f88 [4771821,92634] 0 2024-03-22T07:15:57.980 INFO:tasks.workunit.client.0.smithi063.stdout:2/454: getdents d0/de/d74/d8a 0 2024-03-22T07:15:57.981 INFO:tasks.workunit.client.0.smithi063.stdout:9/420: truncate d8/d19/d20/d22/d29/d31/d37/f59 143315 0 2024-03-22T07:15:57.981 INFO:tasks.workunit.client.0.smithi063.stdout:9/421: readlink d8/d19/l49 0 2024-03-22T07:15:57.981 INFO:tasks.workunit.client.0.smithi063.stdout:9/422: dread - d8/d19/d41/d48/f66 zero size 2024-03-22T07:15:57.982 INFO:tasks.workunit.client.0.smithi063.stdout:3/549: mknod da/d16/d89/d8c/cb6 0 2024-03-22T07:15:57.985 INFO:tasks.workunit.client.0.smithi063.stdout:2/455: mknod d0/de/d12/d5a/c93 0 2024-03-22T07:15:57.989 INFO:tasks.workunit.client.0.smithi063.stdout:3/550: creat da/d63/d65/d86/fb7 x:0 0 0 2024-03-22T07:15:57.989 INFO:tasks.workunit.client.0.smithi063.stdout:7/364: dwrite d5/d16/f1e [0,4194304] 0 2024-03-22T07:15:57.989 INFO:tasks.workunit.client.1.smithi069.stdout:6/146: dread d4/d7/da/f1b [0,4194304] 0 2024-03-22T07:15:57.991 INFO:tasks.workunit.client.0.smithi063.stdout:7/365: mkdir d5/d16/d43/d78 0 2024-03-22T07:15:57.993 INFO:tasks.workunit.client.0.smithi063.stdout:7/366: symlink d5/d16/d1b/d21/d62/d73/d1d/d29/l79 0 2024-03-22T07:15:57.996 INFO:tasks.workunit.client.0.smithi063.stdout:7/367: rename d5/da/ff to d5/d16/d1b/d21/d62/d73/f7a 0 2024-03-22T07:15:57.997 INFO:tasks.workunit.client.0.smithi063.stdout:7/368: creat d5/d38/f7b x:0 0 0 2024-03-22T07:15:57.997 INFO:tasks.workunit.client.1.smithi069.stdout:0/238: dwrite d3/da/f38 [0,4194304] 0 2024-03-22T07:15:57.998 INFO:tasks.workunit.client.0.smithi063.stdout:7/369: creat d5/d16/d1b/f7c x:0 0 0 2024-03-22T07:15:57.999 INFO:tasks.workunit.client.0.smithi063.stdout:4/405: dwrite d1/d5/d43/f2e [0,4194304] 0 2024-03-22T07:15:58.000 INFO:tasks.workunit.client.1.smithi069.stdout:0/239: rmdir d3/da 39 2024-03-22T07:15:58.000 INFO:tasks.workunit.client.1.smithi069.stdout:0/240: fdatasync d3/d7/d15/d1f/f41 0 2024-03-22T07:15:58.000 INFO:tasks.workunit.client.1.smithi069.stdout:0/241: write d3/d7/d15/f45 [5093564,95275] 0 2024-03-22T07:15:58.001 INFO:tasks.workunit.client.1.smithi069.stdout:0/242: rmdir d3/d7/d24 39 2024-03-22T07:15:58.001 INFO:tasks.workunit.client.1.smithi069.stdout:0/243: chown d3/d7/d17/c37 1073652491 1 2024-03-22T07:15:58.001 INFO:tasks.workunit.client.1.smithi069.stdout:0/244: chown d3/d7/f4a 6142984 1 2024-03-22T07:15:58.002 INFO:tasks.workunit.client.1.smithi069.stdout:0/245: write d3/d7/d15/d1f/f41 [706816,88468] 0 2024-03-22T07:15:58.002 INFO:tasks.workunit.client.0.smithi063.stdout:4/406: unlink d1/d5/c5d 0 2024-03-22T07:15:58.002 INFO:tasks.workunit.client.0.smithi063.stdout:4/407: chown d1/d5/d38/f49 372 1 2024-03-22T07:15:58.002 INFO:tasks.workunit.client.0.smithi063.stdout:5/452: read d8/f12 [1337870,107137] 0 2024-03-22T07:15:58.002 INFO:tasks.workunit.client.0.smithi063.stdout:0/594: dwrite da/de/d17/d25/d3c/d43/d76/f9d [0,4194304] 0 2024-03-22T07:15:58.003 INFO:tasks.workunit.client.0.smithi063.stdout:0/595: mknod da/de/d17/d25/d95/d98/cbe 0 2024-03-22T07:15:58.004 INFO:tasks.workunit.client.1.smithi069.stdout:0/246: creat d3/d7/d15/f4f x:0 0 0 2024-03-22T07:15:58.005 INFO:tasks.workunit.client.0.smithi063.stdout:8/529: dwrite d0/fc [0,4194304] 0 2024-03-22T07:15:58.005 INFO:tasks.workunit.client.0.smithi063.stdout:0/596: fsync da/de/d17/d25/d3c/d43/d80/f86 0 2024-03-22T07:15:58.005 INFO:tasks.workunit.client.0.smithi063.stdout:5/453: getdents d8/d13/d1e 0 2024-03-22T07:15:58.005 INFO:tasks.workunit.client.0.smithi063.stdout:6/507: dwrite d3/d1c/f5d [0,4194304] 0 2024-03-22T07:15:58.006 INFO:tasks.workunit.client.1.smithi069.stdout:0/247: rmdir d3/d7/d17/d1c 39 2024-03-22T07:15:58.006 INFO:tasks.workunit.client.1.smithi069.stdout:6/147: write d4/dd/d12/d13/f23 [2085229,24714] 0 2024-03-22T07:15:58.010 INFO:tasks.workunit.client.0.smithi063.stdout:8/530: link d0/c6b d0/d26/d2d/d8f/ca9 0 2024-03-22T07:15:58.010 INFO:tasks.workunit.client.0.smithi063.stdout:8/531: dread - d0/d24/d5b/f95 zero size 2024-03-22T07:15:58.010 INFO:tasks.workunit.client.1.smithi069.stdout:0/248: symlink d3/d7/l50 0 2024-03-22T07:15:58.011 INFO:tasks.workunit.client.1.smithi069.stdout:8/227: dread d1/d1c/f38 [0,4194304] 0 2024-03-22T07:15:58.015 INFO:tasks.workunit.client.1.smithi069.stdout:8/228: rename d1/de/f23 to d1/de/f40 0 2024-03-22T07:15:58.015 INFO:tasks.workunit.client.1.smithi069.stdout:8/229: readlink d1/de/l15 0 2024-03-22T07:15:58.016 INFO:tasks.workunit.client.1.smithi069.stdout:0/249: link d3/d7/d15/d1f/f41 d3/d7/d15/f51 0 2024-03-22T07:15:58.016 INFO:tasks.workunit.client.1.smithi069.stdout:0/250: write d3/da/f3b [54871,123663] 0 2024-03-22T07:15:58.017 INFO:tasks.workunit.client.0.smithi063.stdout:6/508: creat d3/d15/d16/d60/d6d/d4c/d8b/fa4 x:0 0 0 2024-03-22T07:15:58.018 INFO:tasks.workunit.client.0.smithi063.stdout:8/532: creat d0/d1b/d28/d3e/d97/faa x:0 0 0 2024-03-22T07:15:58.018 INFO:tasks.workunit.client.1.smithi069.stdout:0/251: rename d3/f8 to d3/da/f52 0 2024-03-22T07:15:58.019 INFO:tasks.workunit.client.1.smithi069.stdout:0/252: rmdir d3/d7/d24 39 2024-03-22T07:15:58.021 INFO:tasks.workunit.client.1.smithi069.stdout:0/253: rename c2 to d3/c53 0 2024-03-22T07:15:58.021 INFO:tasks.workunit.client.1.smithi069.stdout:0/254: readlink d3/d7/lf 0 2024-03-22T07:15:58.022 INFO:tasks.workunit.client.1.smithi069.stdout:0/255: symlink d3/d7/d15/d29/l54 0 2024-03-22T07:15:58.023 INFO:tasks.workunit.client.1.smithi069.stdout:0/256: mkdir d3/d7/d15/d1f/d55 0 2024-03-22T07:15:58.024 INFO:tasks.workunit.client.1.smithi069.stdout:0/257: creat d3/d7/d17/f56 x:0 0 0 2024-03-22T07:15:58.026 INFO:tasks.workunit.client.1.smithi069.stdout:0/258: mknod d3/d7/d15/d29/c57 0 2024-03-22T07:15:58.031 INFO:tasks.workunit.client.0.smithi063.stdout:8/533: mknod d0/d26/d2d/cab 0 2024-03-22T07:15:58.034 INFO:tasks.workunit.client.0.smithi063.stdout:2/456: dwrite d0/f10 [4194304,4194304] 0 2024-03-22T07:15:58.034 INFO:tasks.workunit.client.0.smithi063.stdout:2/457: write d0/de/d1b/f8c [876246,13729] 0 2024-03-22T07:15:58.035 INFO:tasks.workunit.client.0.smithi063.stdout:6/509: rename d3/d1c to d3/d6/d2a/d38/da5 0 2024-03-22T07:15:58.038 INFO:tasks.workunit.client.0.smithi063.stdout:2/458: mknod d0/de/d58/c94 0 2024-03-22T07:15:58.038 INFO:tasks.workunit.client.0.smithi063.stdout:9/423: rmdir d8/d19/d20/d22/d56 39 2024-03-22T07:15:58.039 INFO:tasks.workunit.client.0.smithi063.stdout:9/424: fsync d8/d17/f73 0 2024-03-22T07:15:58.042 INFO:tasks.workunit.client.0.smithi063.stdout:8/534: rename d0/d26/l86 to d0/d1b/d28/d5f/d99/da1/lac 0 2024-03-22T07:15:58.042 INFO:tasks.workunit.client.0.smithi063.stdout:8/535: creat d0/d26/fad x:0 0 0 2024-03-22T07:15:58.044 INFO:tasks.workunit.client.0.smithi063.stdout:6/510: symlink d3/d15/d16/d69/d8c/la6 0 2024-03-22T07:15:58.047 INFO:tasks.workunit.client.0.smithi063.stdout:9/425: unlink d8/d19/d20/d22/d29/d31/f32 0 2024-03-22T07:15:58.050 INFO:tasks.workunit.client.0.smithi063.stdout:6/511: creat d3/d15/d16/d60/d6d/d4e/fa7 x:0 0 0 2024-03-22T07:15:58.050 INFO:tasks.workunit.client.0.smithi063.stdout:6/512: stat d3/d15/d16/d57/f97 0 2024-03-22T07:15:58.050 INFO:tasks.workunit.client.0.smithi063.stdout:6/513: fdatasync d3/d6/d2a/d38/f4d 0 2024-03-22T07:15:58.055 INFO:tasks.workunit.client.0.smithi063.stdout:8/536: dread d0/d26/f34 [0,4194304] 0 2024-03-22T07:15:58.055 INFO:tasks.workunit.client.0.smithi063.stdout:8/537: read d0/d1b/f1e [8487214,89373] 0 2024-03-22T07:15:58.055 INFO:tasks.workunit.client.0.smithi063.stdout:8/538: write d0/d1f/f53 [1473873,2967] 0 2024-03-22T07:15:58.059 INFO:tasks.workunit.client.0.smithi063.stdout:6/514: creat d3/d6/d2a/fa8 x:0 0 0 2024-03-22T07:15:58.060 INFO:tasks.workunit.client.0.smithi063.stdout:6/515: creat d3/d15/d16/d57/fa9 x:0 0 0 2024-03-22T07:15:58.061 INFO:tasks.workunit.client.0.smithi063.stdout:6/516: mknod d3/d15/d16/d47/caa 0 2024-03-22T07:15:58.061 INFO:tasks.workunit.client.0.smithi063.stdout:6/517: chown d3/d76/fa1 15567 1 2024-03-22T07:15:58.062 INFO:tasks.workunit.client.0.smithi063.stdout:9/426: write d8/f1a [6222130,103593] 0 2024-03-22T07:15:58.063 INFO:tasks.workunit.client.0.smithi063.stdout:3/551: dwrite da/d16/d26/f8b [0,4194304] 0 2024-03-22T07:15:58.063 INFO:tasks.workunit.client.0.smithi063.stdout:7/370: dwrite d5/f39 [0,4194304] 0 2024-03-22T07:15:58.063 INFO:tasks.workunit.client.0.smithi063.stdout:3/552: chown da/d16/d26/d69/ca6 33 1 2024-03-22T07:15:58.063 INFO:tasks.workunit.client.0.smithi063.stdout:5/454: dwrite d8/d13/d24/f50 [0,4194304] 0 2024-03-22T07:15:58.064 INFO:tasks.workunit.client.0.smithi063.stdout:6/518: rename d3/l6a to d3/d15/d16/d60/d7b/lab 0 2024-03-22T07:15:58.064 INFO:tasks.workunit.client.0.smithi063.stdout:0/597: dwrite da/de/d17/f2f [0,4194304] 0 2024-03-22T07:15:58.065 INFO:tasks.workunit.client.1.smithi069.stdout:6/148: dwrite d4/d7/f2c [0,4194304] 0 2024-03-22T07:15:58.065 INFO:tasks.workunit.client.1.smithi069.stdout:6/149: write d4/dd/d12/f1f [4608063,67482] 0 2024-03-22T07:15:58.065 INFO:tasks.workunit.client.1.smithi069.stdout:8/230: dwrite d1/f29 [0,4194304] 0 2024-03-22T07:15:58.066 INFO:tasks.workunit.client.0.smithi063.stdout:9/427: symlink d8/d15/d2f/l83 0 2024-03-22T07:15:58.067 INFO:tasks.workunit.client.0.smithi063.stdout:7/371: symlink d5/da/d5f/d33/d57/l7d 0 2024-03-22T07:15:58.067 INFO:tasks.workunit.client.1.smithi069.stdout:8/231: mknod d1/de/d20/c41 0 2024-03-22T07:15:58.068 INFO:tasks.workunit.client.0.smithi063.stdout:3/553: mkdir da/d63/d65/d81/da5/db8 0 2024-03-22T07:15:58.072 INFO:tasks.workunit.client.1.smithi069.stdout:8/232: getdents d1/de 0 2024-03-22T07:15:58.072 INFO:tasks.workunit.client.1.smithi069.stdout:8/233: fsync d1/f14 0 2024-03-22T07:15:58.073 INFO:tasks.workunit.client.1.smithi069.stdout:6/150: dread d4/dd/d12/f1f [4194304,4194304] 0 2024-03-22T07:15:58.073 INFO:tasks.workunit.client.0.smithi063.stdout:6/519: symlink d3/d15/d16/d60/d6d/d4e/lac 0 2024-03-22T07:15:58.074 INFO:tasks.workunit.client.1.smithi069.stdout:8/234: symlink d1/de/d20/l42 0 2024-03-22T07:15:58.074 INFO:tasks.workunit.client.1.smithi069.stdout:8/235: fsync d1/de/d20/f2f 0 2024-03-22T07:15:58.075 INFO:tasks.workunit.client.1.smithi069.stdout:8/236: chown d1/de/d20/f2f 163 1 2024-03-22T07:15:58.076 INFO:tasks.workunit.client.0.smithi063.stdout:0/598: creat da/de/d17/d25/d3c/d43/d76/d92/fbf x:0 0 0 2024-03-22T07:15:58.076 INFO:tasks.workunit.client.0.smithi063.stdout:0/599: fsync da/de/d17/d25/d3c/d43/d76/f9d 0 2024-03-22T07:15:58.078 INFO:tasks.workunit.client.1.smithi069.stdout:8/237: mkdir d1/de/d20/d43 0 2024-03-22T07:15:58.078 INFO:tasks.workunit.client.1.smithi069.stdout:8/238: fsync d1/d1c/d3e/f36 0 2024-03-22T07:15:58.078 INFO:tasks.workunit.client.1.smithi069.stdout:6/151: dread d4/d7/da/f1b [0,4194304] 0 2024-03-22T07:15:58.078 INFO:tasks.workunit.client.0.smithi063.stdout:5/455: dread d8/d13/d24/f50 [0,4194304] 0 2024-03-22T07:15:58.078 INFO:tasks.workunit.client.0.smithi063.stdout:3/554: mkdir da/d63/d65/db9 0 2024-03-22T07:15:58.078 INFO:tasks.workunit.client.0.smithi063.stdout:3/555: fdatasync da/fb 0 2024-03-22T07:15:58.078 INFO:tasks.workunit.client.0.smithi063.stdout:6/520: mknod d3/d6/d2a/d38/da5/cad 0 2024-03-22T07:15:58.079 INFO:tasks.workunit.client.0.smithi063.stdout:6/521: write d3/d6/fc [305511,81209] 0 2024-03-22T07:15:58.079 INFO:tasks.workunit.client.1.smithi069.stdout:8/239: symlink d1/d1c/l44 0 2024-03-22T07:15:58.080 INFO:tasks.workunit.client.0.smithi063.stdout:0/600: unlink da/de/d60/c3d 0 2024-03-22T07:15:58.080 INFO:tasks.workunit.client.0.smithi063.stdout:0/601: write da/de/d17/d25/d3c/d5f/d97/fa4 [4928997,5703] 0 2024-03-22T07:15:58.080 INFO:tasks.workunit.client.0.smithi063.stdout:0/602: readlink da/de/d17/d25/d3c/d43/l6f 0 2024-03-22T07:15:58.080 INFO:tasks.workunit.client.0.smithi063.stdout:0/603: read da/f54 [161924,9372] 0 2024-03-22T07:15:58.080 INFO:tasks.workunit.client.0.smithi063.stdout:7/372: write d5/da/d5f/d33/f53 [12555327,5229] 0 2024-03-22T07:15:58.080 INFO:tasks.workunit.client.1.smithi069.stdout:8/240: symlink d1/de/l45 0 2024-03-22T07:15:58.081 INFO:tasks.workunit.client.0.smithi063.stdout:5/456: chown d8/d13/f63 1 1 2024-03-22T07:15:58.083 INFO:tasks.workunit.client.0.smithi063.stdout:3/556: creat da/d63/d65/d81/d9b/fba x:0 0 0 2024-03-22T07:15:58.083 INFO:tasks.workunit.client.1.smithi069.stdout:1/226: sync 2024-03-22T07:15:58.083 INFO:tasks.workunit.client.1.smithi069.stdout:1/227: write d1/d13/f45 [702750,110428] 0 2024-03-22T07:15:58.084 INFO:tasks.workunit.client.0.smithi063.stdout:9/428: dwrite d8/d19/d41/f80 [0,4194304] 0 2024-03-22T07:15:58.084 INFO:tasks.workunit.client.0.smithi063.stdout:0/604: dread da/f82 [0,4194304] 0 2024-03-22T07:15:58.086 INFO:tasks.workunit.client.1.smithi069.stdout:8/241: dread d1/d1c/f22 [0,4194304] 0 2024-03-22T07:15:58.087 INFO:tasks.workunit.client.0.smithi063.stdout:7/373: rmdir d5/d16/d1b/d21/d62/d73/d34 0 2024-03-22T07:15:58.089 INFO:tasks.workunit.client.1.smithi069.stdout:8/242: creat d1/d1c/d3e/f46 x:0 0 0 2024-03-22T07:15:58.089 INFO:tasks.workunit.client.0.smithi063.stdout:2/459: rmdir d0/de 39 2024-03-22T07:15:58.091 INFO:tasks.workunit.client.1.smithi069.stdout:8/243: creat d1/d1c/f47 x:0 0 0 2024-03-22T07:15:58.093 INFO:tasks.workunit.client.1.smithi069.stdout:8/244: mkdir d1/d1c/d48 0 2024-03-22T07:15:58.093 INFO:tasks.workunit.client.0.smithi063.stdout:7/374: creat d5/d16/d1b/d21/d62/d73/d1d/d29/d4a/f7e x:0 0 0 2024-03-22T07:15:58.095 INFO:tasks.workunit.client.0.smithi063.stdout:3/557: mknod da/d16/d26/cbb 0 2024-03-22T07:15:58.096 INFO:tasks.workunit.client.0.smithi063.stdout:9/429: write d8/fa [3681514,3391] 0 2024-03-22T07:15:58.096 INFO:tasks.workunit.client.0.smithi063.stdout:9/430: truncate d8/d17/f73 795274 0 2024-03-22T07:15:58.096 INFO:tasks.workunit.client.0.smithi063.stdout:9/431: chown d8/l4c 13 1 2024-03-22T07:15:58.096 INFO:tasks.workunit.client.0.smithi063.stdout:9/432: write d8/d19/d41/f4a [3087104,52064] 0 2024-03-22T07:15:58.096 INFO:tasks.workunit.client.0.smithi063.stdout:7/375: truncate d5/d16/d1b/d21/d62/d73/f61 2771603 0 2024-03-22T07:15:58.097 INFO:tasks.workunit.client.0.smithi063.stdout:2/460: creat d0/de/d1b/d63/d6e/f95 x:0 0 0 2024-03-22T07:15:58.098 INFO:tasks.workunit.client.0.smithi063.stdout:9/433: truncate d8/d19/f3a 3290743 0 2024-03-22T07:15:58.098 INFO:tasks.workunit.client.0.smithi063.stdout:9/434: stat d8/d19/d41/f80 0 2024-03-22T07:15:58.098 INFO:tasks.workunit.client.0.smithi063.stdout:7/376: mknod d5/d16/d1b/d21/d62/d73/d1d/d29/d3b/c7f 0 2024-03-22T07:15:58.099 INFO:tasks.workunit.client.0.smithi063.stdout:7/377: creat d5/d16/d1b/f80 x:0 0 0 2024-03-22T07:15:58.100 INFO:tasks.workunit.client.0.smithi063.stdout:2/461: creat d0/de/d74/f96 x:0 0 0 2024-03-22T07:15:58.102 INFO:tasks.workunit.client.0.smithi063.stdout:7/378: truncate d5/d16/d43/f56 1929232 0 2024-03-22T07:15:58.103 INFO:tasks.workunit.client.0.smithi063.stdout:5/457: dread d8/d13/d1e/d3c/f49 [0,4194304] 0 2024-03-22T07:15:58.107 INFO:tasks.workunit.client.0.smithi063.stdout:0/605: read da/de/d17/d25/d3c/f3f [7402896,89843] 0 2024-03-22T07:15:58.108 INFO:tasks.workunit.client.0.smithi063.stdout:0/606: dread da/f82 [0,4194304] 0 2024-03-22T07:15:58.110 INFO:tasks.workunit.client.1.smithi069.stdout:8/245: dread d1/de/f40 [0,4194304] 0 2024-03-22T07:15:58.110 INFO:tasks.workunit.client.0.smithi063.stdout:0/607: symlink da/de/d17/d25/d3c/d43/d76/lc0 0 2024-03-22T07:15:58.110 INFO:tasks.workunit.client.0.smithi063.stdout:0/608: chown da/de/d39/f90 1403 1 2024-03-22T07:15:58.111 INFO:tasks.workunit.client.0.smithi063.stdout:0/609: mknod da/de/d17/d25/d95/cc1 0 2024-03-22T07:15:58.111 INFO:tasks.workunit.client.0.smithi063.stdout:0/610: chown da/de/d17/d25/d3c/d5f/d97/lad 19415 1 2024-03-22T07:15:58.113 INFO:tasks.workunit.client.0.smithi063.stdout:5/458: dread d8/d13/d1e/f80 [0,4194304] 0 2024-03-22T07:15:58.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.113+0000 7f346c285700 1 -- 172.21.15.63:0/2697159806 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f34640ffc50 con 0x7f345006af90 2024-03-22T07:15:58.114 INFO:tasks.workunit.client.0.smithi063.stdout:5/459: mkdir d8/d13/d24/d81 0 2024-03-22T07:15:58.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.114+0000 7f345affd700 1 -- 172.21.15.63:0/2697159806 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f34640ffc50 con 0x7f345006af90 2024-03-22T07:15:58.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.116+0000 7f346c285700 1 -- 172.21.15.63:0/2697159806 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f345006af90 msgr2=0x7f345006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:58.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.116+0000 7f346c285700 1 --2- 172.21.15.63:0/2697159806 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f345006af90 0x7f345006d440 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3460004f90 tx=0x7f3460009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:58.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.116+0000 7f346c285700 1 -- 172.21.15.63:0/2697159806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3464102300 msgr2=0x7f34640fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:58.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.116+0000 7f346c285700 1 --2- 172.21.15.63:0/2697159806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3464102300 0x7f34640fe2a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3454002870 tx=0x7f3454004570 comp rx=0 tx=0).stop 2024-03-22T07:15:58.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.117+0000 7f346c285700 1 -- 172.21.15.63:0/2697159806 shutdown_connections 2024-03-22T07:15:58.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.117+0000 7f346c285700 1 --2- 172.21.15.63:0/2697159806 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f345006af90 0x7f345006d440 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:58.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.117+0000 7f346c285700 1 --2- 172.21.15.63:0/2697159806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34640662e0 0x7f34640fdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:58.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.117+0000 7f346c285700 1 --2- 172.21.15.63:0/2697159806 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3464102300 0x7f34640fe2a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:58.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.117+0000 7f346c285700 1 -- 172.21.15.63:0/2697159806 >> 172.21.15.63:0/2697159806 conn(0x7f34640f7430 msgr2=0x7f34640f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:58.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.117+0000 7f346c285700 1 -- 172.21.15.63:0/2697159806 shutdown_connections 2024-03-22T07:15:58.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.117+0000 7f346c285700 1 -- 172.21.15.63:0/2697159806 wait complete. 2024-03-22T07:15:58.122 INFO:tasks.workunit.client.0.smithi063.stdout:0/611: dread da/de/d17/d25/d3c/d43/f83 [0,4194304] 0 2024-03-22T07:15:58.124 INFO:tasks.workunit.client.0.smithi063.stdout:0/612: creat da/fc2 x:0 0 0 2024-03-22T07:15:58.131 INFO:tasks.workunit.client.0.smithi063.stdout:2/462: dread d0/de/d58/f7b [0,4194304] 0 2024-03-22T07:15:58.132 INFO:tasks.workunit.client.0.smithi063.stdout:2/463: rmdir d0/de/d1b/d63 39 2024-03-22T07:15:58.134 INFO:tasks.workunit.client.0.smithi063.stdout:2/464: getdents d0/de/d1b/d63 0 2024-03-22T07:15:58.134 INFO:tasks.workunit.client.0.smithi063.stdout:2/465: write d0/de/d12/d20/f3f [2295594,94291] 0 2024-03-22T07:15:58.134 INFO:tasks.workunit.client.0.smithi063.stdout:2/466: write d0/de/d74/f96 [800033,127822] 0 2024-03-22T07:15:58.135 INFO:tasks.workunit.client.0.smithi063.stdout:2/467: truncate d0/f9 3015372 0 2024-03-22T07:15:58.136 INFO:tasks.workunit.client.0.smithi063.stdout:2/468: truncate d0/de/d74/f7e 10776 0 2024-03-22T07:15:58.138 INFO:tasks.workunit.client.0.smithi063.stdout:7/379: dread d5/da/f2c [0,4194304] 0 2024-03-22T07:15:58.138 INFO:tasks.workunit.client.0.smithi063.stdout:7/380: stat d5/d16/d1b/d21/d62/d73/d1d 0 2024-03-22T07:15:58.138 INFO:tasks.workunit.client.0.smithi063.stdout:6/522: dwrite d3/d15/d16/d69/f83 [4194304,4194304] 0 2024-03-22T07:15:58.140 INFO:tasks.workunit.client.0.smithi063.stdout:9/435: dwrite d8/d19/d41/f80 [0,4194304] 0 2024-03-22T07:15:58.140 INFO:tasks.workunit.client.0.smithi063.stdout:7/381: unlink d5/da/d5f/d33/d57/l6c 0 2024-03-22T07:15:58.140 INFO:tasks.workunit.client.1.smithi069.stdout:6/152: dwrite f3 [0,4194304] 0 2024-03-22T07:15:58.140 INFO:tasks.workunit.client.1.smithi069.stdout:6/153: readlink d4/dd/d12/l17 0 2024-03-22T07:15:58.140 INFO:tasks.workunit.client.1.smithi069.stdout:6/154: fdatasync d4/dd/d12/d13/f23 0 2024-03-22T07:15:58.140 INFO:tasks.workunit.client.1.smithi069.stdout:6/155: rename d4/l27 to d4/l2e 0 2024-03-22T07:15:58.140 INFO:tasks.workunit.client.1.smithi069.stdout:6/156: stat d4/dd/d12/f1f 0 2024-03-22T07:15:58.141 INFO:tasks.workunit.client.0.smithi063.stdout:5/460: write d8/d13/d1e/d3c/f4c [3289499,77] 0 2024-03-22T07:15:58.141 INFO:tasks.workunit.client.0.smithi063.stdout:9/436: mknod d8/d19/c84 0 2024-03-22T07:15:58.141 INFO:tasks.workunit.client.0.smithi063.stdout:9/437: getdents d8/d19/d20/d22/d29/d31/d7f 0 2024-03-22T07:15:58.142 INFO:tasks.workunit.client.0.smithi063.stdout:5/461: mkdir d8/d13/d1e/d31/d58/d7e/d82 0 2024-03-22T07:15:58.144 INFO:tasks.workunit.client.0.smithi063.stdout:5/462: write d8/d13/d1e/d31/d58/f68 [553205,109607] 0 2024-03-22T07:15:58.144 INFO:tasks.workunit.client.0.smithi063.stdout:5/463: read d8/f25 [7360,24353] 0 2024-03-22T07:15:58.144 INFO:tasks.workunit.client.0.smithi063.stdout:9/438: rmdir d8/d19/d41 39 2024-03-22T07:15:58.144 INFO:tasks.workunit.client.0.smithi063.stdout:9/439: fdatasync d8/d19/d20/f21 0 2024-03-22T07:15:58.144 INFO:tasks.workunit.client.0.smithi063.stdout:9/440: dread - d8/d19/d41/d48/f66 zero size 2024-03-22T07:15:58.146 INFO:tasks.workunit.client.0.smithi063.stdout:5/464: symlink d8/d13/d24/d81/l83 0 2024-03-22T07:15:58.147 INFO:tasks.workunit.client.0.smithi063.stdout:9/441: link d8/d19/f35 d8/d19/d41/d48/f85 0 2024-03-22T07:15:58.149 INFO:tasks.workunit.client.0.smithi063.stdout:9/442: getdents d8/d19/d20/d22/d29/d31/d37 0 2024-03-22T07:15:58.149 INFO:tasks.workunit.client.0.smithi063.stdout:6/523: dread d3/d6/d35/d3a/f40 [0,4194304] 0 2024-03-22T07:15:58.149 INFO:tasks.workunit.client.0.smithi063.stdout:6/524: truncate d3/d6/f3e 5103712 0 2024-03-22T07:15:58.150 INFO:tasks.workunit.client.1.smithi069.stdout:1/228: dwrite d1/fa [0,4194304] 0 2024-03-22T07:15:58.150 INFO:tasks.workunit.client.1.smithi069.stdout:6/157: dread d4/f14 [0,4194304] 0 2024-03-22T07:15:58.151 INFO:tasks.workunit.client.0.smithi063.stdout:3/558: dwrite da/d63/d65/d81/d9b/d9f/f90 [0,4194304] 0 2024-03-22T07:15:58.151 INFO:tasks.workunit.client.0.smithi063.stdout:3/559: write da/d16/d89/fb5 [249320,67225] 0 2024-03-22T07:15:58.151 INFO:tasks.workunit.client.0.smithi063.stdout:9/443: creat d8/d19/d20/d22/d29/d31/f86 x:0 0 0 2024-03-22T07:15:58.151 INFO:tasks.workunit.client.0.smithi063.stdout:2/469: fdatasync d0/de/d74/f96 0 2024-03-22T07:15:58.151 INFO:tasks.workunit.client.0.smithi063.stdout:0/613: dwrite da/de/d60/faf [0,4194304] 0 2024-03-22T07:15:58.152 INFO:tasks.workunit.client.1.smithi069.stdout:8/246: rmdir d1/de/d20 39 2024-03-22T07:15:58.152 INFO:tasks.workunit.client.1.smithi069.stdout:8/247: fdatasync d1/de/f2d 0 2024-03-22T07:15:58.152 INFO:tasks.workunit.client.1.smithi069.stdout:8/248: chown d1/d1c/d3e/f32 1 1 2024-03-22T07:15:58.152 INFO:tasks.workunit.client.1.smithi069.stdout:8/249: write d1/d1c/d3e/f46 [745065,115643] 0 2024-03-22T07:15:58.152 INFO:tasks.workunit.client.1.smithi069.stdout:8/250: readlink d1/de/l15 0 2024-03-22T07:15:58.152 INFO:tasks.workunit.client.1.smithi069.stdout:8/251: chown d1/d1c/f22 3108 1 2024-03-22T07:15:58.153 INFO:tasks.workunit.client.1.smithi069.stdout:6/158: symlink d4/dd/d12/l2f 0 2024-03-22T07:15:58.153 INFO:tasks.workunit.client.1.smithi069.stdout:6/159: write d4/dd/d12/f1f [1041037,54645] 0 2024-03-22T07:15:58.155 INFO:tasks.workunit.client.0.smithi063.stdout:5/465: dread d8/d13/d1e/f79 [0,4194304] 0 2024-03-22T07:15:58.155 INFO:tasks.workunit.client.0.smithi063.stdout:5/466: write d8/d13/d1e/f80 [1619550,2029] 0 2024-03-22T07:15:58.155 INFO:tasks.workunit.client.0.smithi063.stdout:3/560: symlink da/d63/d65/d4b/lbc 0 2024-03-22T07:15:58.156 INFO:tasks.workunit.client.1.smithi069.stdout:8/252: rename d1/de/d20/c41 to d1/c49 0 2024-03-22T07:15:58.158 INFO:tasks.workunit.client.1.smithi069.stdout:8/253: symlink d1/l4a 0 2024-03-22T07:15:58.161 INFO:tasks.workunit.client.0.smithi063.stdout:9/444: mknod d8/d19/d20/d22/d56/c87 0 2024-03-22T07:15:58.161 INFO:tasks.workunit.client.0.smithi063.stdout:9/445: truncate d8/d19/d20/d22/d29/d31/d37/f59 1030108 0 2024-03-22T07:15:58.161 INFO:tasks.workunit.client.0.smithi063.stdout:9/446: chown d8/d17/d1e/l2e 0 1 2024-03-22T07:15:58.161 INFO:tasks.workunit.client.0.smithi063.stdout:0/614: truncate da/f2b 228060 0 2024-03-22T07:15:58.162 INFO:tasks.workunit.client.1.smithi069.stdout:4/301: sync 2024-03-22T07:15:58.166 INFO:tasks.workunit.client.0.smithi063.stdout:9/447: rename d8/d19/d20/d22/d29/d31/d53/l57 to d8/d15/l88 0 2024-03-22T07:15:58.167 INFO:tasks.workunit.client.1.smithi069.stdout:1/229: read d1/fa [480102,99199] 0 2024-03-22T07:15:58.167 INFO:tasks.workunit.client.1.smithi069.stdout:6/160: getdents d4/dd/d12 0 2024-03-22T07:15:58.169 INFO:tasks.workunit.client.0.smithi063.stdout:6/525: write d3/d6/f2c [1612630,3745] 0 2024-03-22T07:15:58.170 INFO:tasks.workunit.client.0.smithi063.stdout:0/615: link da/de/d60/faf da/de/d17/d25/d3c/d53/d89/fc3 0 2024-03-22T07:15:58.170 INFO:tasks.workunit.client.0.smithi063.stdout:0/616: creat da/de/d17/d25/d3c/d43/d80/fc4 x:0 0 0 2024-03-22T07:15:58.186 INFO:tasks.workunit.client.0.smithi063.stdout:9/448: getdents d8/d19/d2b 0 2024-03-22T07:15:58.187 INFO:tasks.workunit.client.0.smithi063.stdout:0/617: write da/de/d17/f2f [4257700,78090] 0 2024-03-22T07:15:58.192 INFO:tasks.workunit.client.0.smithi063.stdout:0/618: dread da/de/d60/f33 [0,4194304] 0 2024-03-22T07:15:58.192 INFO:tasks.workunit.client.0.smithi063.stdout:7/382: getdents d5/d16/d1b/d21/d62/d73 0 2024-03-22T07:15:58.192 INFO:tasks.workunit.client.0.smithi063.stdout:0/619: chown da/de/d17/d25/l48 6543422 1 2024-03-22T07:15:58.192 INFO:tasks.workunit.client.0.smithi063.stdout:0/620: write da/de/d17/f28 [819451,49196] 0 2024-03-22T07:15:58.206 INFO:tasks.workunit.client.0.smithi063.stdout:9/449: rename d8/d19/d20/f21 to d8/d19/d2b/f89 0 2024-03-22T07:15:58.206 INFO:tasks.workunit.client.0.smithi063.stdout:9/450: fdatasync d8/d17/f1d 0 2024-03-22T07:15:58.209 INFO:tasks.workunit.client.0.smithi063.stdout:2/470: dwrite d0/de/d12/d39/f76 [0,4194304] 0 2024-03-22T07:15:58.210 INFO:tasks.workunit.client.0.smithi063.stdout:7/383: dread d5/d16/d1b/d21/d62/d73/d1d/f51 [0,4194304] 0 2024-03-22T07:15:58.210 INFO:tasks.workunit.client.0.smithi063.stdout:7/384: fdatasync d5/da/d5f/d33/d57/f67 0 2024-03-22T07:15:58.213 INFO:tasks.workunit.client.1.smithi069.stdout:4/302: dwrite d6/dc/f37 [0,4194304] 0 2024-03-22T07:15:58.213 INFO:tasks.workunit.client.1.smithi069.stdout:4/303: chown d6/dc/d1d/d31/c11 50 1 2024-03-22T07:15:58.214 INFO:tasks.workunit.client.0.smithi063.stdout:2/471: rename d0/de/d1b/d63/d78 to d0/de/d58/d7f/d97 0 2024-03-22T07:15:58.214 INFO:tasks.workunit.client.0.smithi063.stdout:2/472: write d0/de/d1b/d63/f86 [952564,12735] 0 2024-03-22T07:15:58.214 INFO:tasks.workunit.client.0.smithi063.stdout:2/473: truncate d0/f8 2352555 0 2024-03-22T07:15:58.214 INFO:tasks.workunit.client.0.smithi063.stdout:0/621: dwrite da/de/d17/d25/f3e [0,4194304] 0 2024-03-22T07:15:58.215 INFO:tasks.workunit.client.1.smithi069.stdout:8/254: dwrite d1/de/f40 [0,4194304] 0 2024-03-22T07:15:58.215 INFO:tasks.workunit.client.1.smithi069.stdout:8/255: read d1/d1c/f22 [2546062,106508] 0 2024-03-22T07:15:58.215 INFO:tasks.workunit.client.1.smithi069.stdout:4/304: unlink d6/dc/l13 0 2024-03-22T07:15:58.215 INFO:tasks.workunit.client.1.smithi069.stdout:1/230: dwrite d1/f35 [0,4194304] 0 2024-03-22T07:15:58.216 INFO:tasks.workunit.client.1.smithi069.stdout:8/256: creat d1/d1c/d48/f4b x:0 0 0 2024-03-22T07:15:58.216 INFO:tasks.workunit.client.1.smithi069.stdout:8/257: chown d1/d1c/l44 286243 1 2024-03-22T07:15:58.216 INFO:tasks.workunit.client.1.smithi069.stdout:8/258: creat d1/d1c/d3e/f4c x:0 0 0 2024-03-22T07:15:58.216 INFO:tasks.workunit.client.1.smithi069.stdout:8/259: write d1/fd [474506,36035] 0 2024-03-22T07:15:58.217 INFO:tasks.workunit.client.0.smithi063.stdout:5/467: dwrite d8/f44 [0,4194304] 0 2024-03-22T07:15:58.217 INFO:tasks.workunit.client.0.smithi063.stdout:3/561: dwrite da/d16/d3f/f44 [0,4194304] 0 2024-03-22T07:15:58.217 INFO:tasks.workunit.client.0.smithi063.stdout:7/385: getdents d5/d16/d43 0 2024-03-22T07:15:58.217 INFO:tasks.workunit.client.0.smithi063.stdout:7/386: creat d5/d38/f81 x:0 0 0 2024-03-22T07:15:58.217 INFO:tasks.workunit.client.0.smithi063.stdout:7/387: readlink d5/d16/d1b/d21/d62/d73/d1d/d29/d3b/l3d 0 2024-03-22T07:15:58.217 INFO:tasks.workunit.client.0.smithi063.stdout:2/474: truncate d0/de/d12/d20/f29 1178414 0 2024-03-22T07:15:58.219 INFO:tasks.workunit.client.1.smithi069.stdout:1/231: creat d1/d25/d41/f4b x:0 0 0 2024-03-22T07:15:58.220 INFO:tasks.workunit.client.0.smithi063.stdout:3/562: truncate da/d63/d65/f99 4192234 0 2024-03-22T07:15:58.222 INFO:tasks.workunit.client.0.smithi063.stdout:5/468: creat d8/d13/d1e/d31/d58/d7e/f84 x:0 0 0 2024-03-22T07:15:58.222 INFO:tasks.workunit.client.0.smithi063.stdout:5/469: write d8/d13/d28/f73 [8846453,18806] 0 2024-03-22T07:15:58.223 INFO:tasks.workunit.client.1.smithi069.stdout:8/260: link d1/de/d20/l28 d1/l4d 0 2024-03-22T07:15:58.224 INFO:tasks.workunit.client.1.smithi069.stdout:8/261: mkdir d1/de/d20/d43/d4e 0 2024-03-22T07:15:58.226 INFO:tasks.workunit.client.0.smithi063.stdout:2/475: getdents d0/de/d12/d39 0 2024-03-22T07:15:58.227 INFO:tasks.workunit.client.0.smithi063.stdout:0/622: read da/de/d60/faf [430282,41285] 0 2024-03-22T07:15:58.228 INFO:tasks.workunit.client.0.smithi063.stdout:5/470: chown d8/d13/l36 1252809 1 2024-03-22T07:15:58.228 INFO:tasks.workunit.client.0.smithi063.stdout:5/471: readlink d8/d13/d28/l56 0 2024-03-22T07:15:58.231 INFO:tasks.workunit.client.0.smithi063.stdout:2/476: rename d0/de/f31 to d0/de/d12/d39/f98 0 2024-03-22T07:15:58.234 INFO:tasks.workunit.client.0.smithi063.stdout:0/623: rename da/de/fb4 to da/de/d17/d25/d3c/d5f/d97/fc5 0 2024-03-22T07:15:58.234 INFO:tasks.workunit.client.0.smithi063.stdout:0/624: fdatasync da/de/d39/f8c 0 2024-03-22T07:15:58.234 INFO:tasks.workunit.client.0.smithi063.stdout:5/472: dread d8/d13/d1e/d3c/f4c [0,4194304] 0 2024-03-22T07:15:58.235 INFO:tasks.workunit.client.0.smithi063.stdout:0/625: mkdir da/de/d17/d25/d3c/dc6 0 2024-03-22T07:15:58.236 INFO:tasks.workunit.client.0.smithi063.stdout:5/473: mknod d8/d13/d24/d62/c85 0 2024-03-22T07:15:58.237 INFO:tasks.workunit.client.0.smithi063.stdout:0/626: getdents da/d49 0 2024-03-22T07:15:58.238 INFO:tasks.workunit.client.0.smithi063.stdout:0/627: dread da/de/d17/d25/d3c/f66 [0,4194304] 0 2024-03-22T07:15:58.238 INFO:tasks.workunit.client.0.smithi063.stdout:2/477: write d0/de/d1b/d32/f49 [10770,44292] 0 2024-03-22T07:15:58.238 INFO:tasks.workunit.client.0.smithi063.stdout:2/478: write d0/f53 [4222062,79284] 0 2024-03-22T07:15:58.242 INFO:tasks.workunit.client.0.smithi063.stdout:2/479: rename d0/f10 to d0/de/d12/d20/f99 0 2024-03-22T07:15:58.242 INFO:tasks.workunit.client.0.smithi063.stdout:2/480: dread - d0/de/d1b/d63/d6e/f95 zero size 2024-03-22T07:15:58.243 INFO:tasks.workunit.client.0.smithi063.stdout:0/628: getdents da/de/d17/d25/d3c/d43/d76 0 2024-03-22T07:15:58.245 INFO:tasks.workunit.client.0.smithi063.stdout:2/481: symlink d0/de/d35/l9a 0 2024-03-22T07:15:58.252 INFO:tasks.workunit.client.0.smithi063.stdout:0/629: dread da/de/d17/d25/d3c/d5f/f4b [0,4194304] 0 2024-03-22T07:15:58.252 INFO:tasks.workunit.client.0.smithi063.stdout:0/630: chown da/de/f31 297 1 2024-03-22T07:15:58.254 INFO:tasks.workunit.client.0.smithi063.stdout:0/631: mknod da/de/d17/d25/d3c/d43/d80/cc7 0 2024-03-22T07:15:58.256 INFO:tasks.workunit.client.0.smithi063.stdout:0/632: symlink da/de/d17/d25/d3c/d43/d76/lc8 0 2024-03-22T07:15:58.257 INFO:tasks.workunit.client.0.smithi063.stdout:1/582: sync 2024-03-22T07:15:58.257 INFO:tasks.workunit.client.0.smithi063.stdout:1/583: chown d4/l16 5584594 1 2024-03-22T07:15:58.257 INFO:tasks.workunit.client.0.smithi063.stdout:1/584: chown d4/d5a/d91/lb4 116 1 2024-03-22T07:15:58.258 INFO:tasks.workunit.client.0.smithi063.stdout:0/633: mkdir da/de/d17/d25/d3c/d43/dc9 0 2024-03-22T07:15:58.258 INFO:tasks.workunit.client.0.smithi063.stdout:0/634: write da/de/f24 [8288945,118345] 0 2024-03-22T07:15:58.262 INFO:tasks.workunit.client.1.smithi069.stdout:8/262: dwrite d1/de/d20/f26 [0,4194304] 0 2024-03-22T07:15:58.263 INFO:tasks.workunit.client.0.smithi063.stdout:5/474: dwrite d8/d13/d1e/f80 [0,4194304] 0 2024-03-22T07:15:58.263 INFO:tasks.workunit.client.1.smithi069.stdout:4/305: dwrite d6/f9 [0,4194304] 0 2024-03-22T07:15:58.263 INFO:tasks.workunit.client.0.smithi063.stdout:3/563: write da/d63/d65/f99 [3014089,89816] 0 2024-03-22T07:15:58.267 INFO:tasks.workunit.client.0.smithi063.stdout:0/635: write da/ff [7669401,36161] 0 2024-03-22T07:15:58.268 INFO:tasks.workunit.client.1.smithi069.stdout:8/263: rename d1/de/d20/f26 to d1/de/d20/f4f 0 2024-03-22T07:15:58.268 INFO:tasks.workunit.client.1.smithi069.stdout:8/264: write d1/d1c/d3e/f4c [401857,10885] 0 2024-03-22T07:15:58.268 INFO:tasks.workunit.client.1.smithi069.stdout:8/265: fsync d1/f9 0 2024-03-22T07:15:58.268 INFO:tasks.workunit.client.1.smithi069.stdout:8/266: dread - d1/de/d20/f3c zero size 2024-03-22T07:15:58.268 INFO:tasks.workunit.client.0.smithi063.stdout:5/475: mkdir d8/d13/d1e/d31/d64/d65/d86 0 2024-03-22T07:15:58.270 INFO:tasks.workunit.client.0.smithi063.stdout:1/585: dread d4/d5a/fa9 [0,4194304] 0 2024-03-22T07:15:58.270 INFO:tasks.workunit.client.0.smithi063.stdout:1/586: fdatasync d4/df/d13/d39/d63/f77 0 2024-03-22T07:15:58.270 INFO:tasks.workunit.client.0.smithi063.stdout:1/587: write d4/df/d13/d2b/f84 [185652,15956] 0 2024-03-22T07:15:58.270 INFO:tasks.workunit.client.0.smithi063.stdout:3/564: dread da/d16/d26/d69/f8f [0,4194304] 0 2024-03-22T07:15:58.271 INFO:tasks.workunit.client.0.smithi063.stdout:5/476: mkdir d8/d13/d28/d57/d5f/d87 0 2024-03-22T07:15:58.272 INFO:tasks.workunit.client.0.smithi063.stdout:2/482: dwrite d0/de/d12/d20/f3f [0,4194304] 0 2024-03-22T07:15:58.273 INFO:tasks.workunit.client.1.smithi069.stdout:4/306: read d6/dc/d1d/d31/f14 [738716,61256] 0 2024-03-22T07:15:58.273 INFO:tasks.workunit.client.0.smithi063.stdout:2/483: creat d0/de/d12/d20/f9b x:0 0 0 2024-03-22T07:15:58.274 INFO:tasks.workunit.client.0.smithi063.stdout:2/484: chown d0/de/d35/f6f 7861759 1 2024-03-22T07:15:58.276 INFO:tasks.workunit.client.1.smithi069.stdout:4/307: dread d6/dc/f1a [0,4194304] 0 2024-03-22T07:15:58.277 INFO:tasks.workunit.client.0.smithi063.stdout:3/565: link da/d63/d65/d86/fb7 da/d16/d26/fbd 0 2024-03-22T07:15:58.277 INFO:tasks.workunit.client.1.smithi069.stdout:4/308: creat d6/dc/d1d/d31/d1f/f5b x:0 0 0 2024-03-22T07:15:58.281 INFO:tasks.workunit.client.0.smithi063.stdout:2/485: unlink d0/l67 0 2024-03-22T07:15:58.282 INFO:tasks.workunit.client.0.smithi063.stdout:5/477: dread d8/d13/d24/f50 [0,4194304] 0 2024-03-22T07:15:58.282 INFO:tasks.workunit.client.1.smithi069.stdout:5/149: sync 2024-03-22T07:15:58.282 INFO:tasks.workunit.client.1.smithi069.stdout:2/193: sync 2024-03-22T07:15:58.282 INFO:tasks.workunit.client.1.smithi069.stdout:7/272: sync 2024-03-22T07:15:58.282 INFO:tasks.workunit.client.1.smithi069.stdout:9/199: sync 2024-03-22T07:15:58.282 INFO:tasks.workunit.client.1.smithi069.stdout:2/194: creat d18/d1c/d27/d33/d3d/f48 x:0 0 0 2024-03-22T07:15:58.282 INFO:tasks.workunit.client.1.smithi069.stdout:9/200: fdatasync d2/d21/f3a 0 2024-03-22T07:15:58.282 INFO:tasks.workunit.client.1.smithi069.stdout:2/195: stat d18/f2b 0 2024-03-22T07:15:58.283 INFO:tasks.workunit.client.1.smithi069.stdout:6/161: sync 2024-03-22T07:15:58.283 INFO:tasks.workunit.client.1.smithi069.stdout:0/259: sync 2024-03-22T07:15:58.283 INFO:tasks.workunit.client.1.smithi069.stdout:3/183: sync 2024-03-22T07:15:58.283 INFO:tasks.workunit.client.1.smithi069.stdout:8/267: read d1/f1d [3838350,47922] 0 2024-03-22T07:15:58.284 INFO:tasks.workunit.client.0.smithi063.stdout:2/486: creat d0/de/d1b/d63/f9c x:0 0 0 2024-03-22T07:15:58.284 INFO:tasks.workunit.client.1.smithi069.stdout:5/150: mknod d1/d6/c2f 0 2024-03-22T07:15:58.284 INFO:tasks.workunit.client.1.smithi069.stdout:5/151: fsync d1/d6/f1e 0 2024-03-22T07:15:58.284 INFO:tasks.workunit.client.1.smithi069.stdout:5/152: write d1/d7/f22 [1736171,30879] 0 2024-03-22T07:15:58.284 INFO:tasks.workunit.client.1.smithi069.stdout:5/153: fsync d1/f3 0 2024-03-22T07:15:58.284 INFO:tasks.workunit.client.1.smithi069.stdout:7/273: symlink d9/db/l5f 0 2024-03-22T07:15:58.285 INFO:tasks.workunit.client.0.smithi063.stdout:1/588: dread d4/df/d13/d2b/d3a/f81 [0,4194304] 0 2024-03-22T07:15:58.286 INFO:tasks.workunit.client.1.smithi069.stdout:2/196: link fa d18/d1c/d27/f49 0 2024-03-22T07:15:58.287 INFO:tasks.workunit.client.0.smithi063.stdout:5/478: unlink d8/d13/d24/d62/c75 0 2024-03-22T07:15:58.287 INFO:tasks.workunit.client.0.smithi063.stdout:5/479: chown d8/d13/d28/c42 0 1 2024-03-22T07:15:58.287 INFO:tasks.workunit.client.1.smithi069.stdout:6/162: rmdir d4 39 2024-03-22T07:15:58.289 INFO:tasks.workunit.client.0.smithi063.stdout:0/636: dwrite da/de/d17/d25/d3c/d43/d76/f9d [0,4194304] 0 2024-03-22T07:15:58.290 INFO:tasks.workunit.client.1.smithi069.stdout:0/260: unlink d3/d7/d15/f51 0 2024-03-22T07:15:58.291 INFO:tasks.workunit.client.0.smithi063.stdout:2/487: creat d0/de/d12/d5a/f9d x:0 0 0 2024-03-22T07:15:58.291 INFO:tasks.workunit.client.0.smithi063.stdout:2/488: fsync d0/de/d12/d39/f98 0 2024-03-22T07:15:58.292 INFO:tasks.workunit.client.1.smithi069.stdout:8/268: rename d1/de/l45 to d1/d1c/l50 0 2024-03-22T07:15:58.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.294+0000 7fac61ffb700 1 -- 172.21.15.63:0/4147606132 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac6402b590 con 0x7fac5c09b1d0 2024-03-22T07:15:58.294 INFO:tasks.workunit.client.0.smithi063.stdout:5/480: chown d8/d13/l19 40 1 2024-03-22T07:15:58.295 INFO:tasks.workunit.client.1.smithi069.stdout:2/197: link d18/d1c/f41 d18/d1c/d34/f4a 0 2024-03-22T07:15:58.295 INFO:tasks.workunit.client.1.smithi069.stdout:2/198: truncate d18/d1c/d3e/d40/f44 429677 0 2024-03-22T07:15:58.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.295+0000 7fac63fff700 1 -- 172.21.15.63:0/4147606132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac5c09b1d0 msgr2=0x7fac5c09b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:58.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.295+0000 7fac63fff700 1 --2- 172.21.15.63:0/4147606132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac5c09b1d0 0x7fac5c09b5e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fac64063880 tx=0x7fac640636e0 comp rx=0 tx=0).stop 2024-03-22T07:15:58.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.295+0000 7fac63fff700 1 -- 172.21.15.63:0/4147606132 shutdown_connections 2024-03-22T07:15:58.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.295+0000 7fac63fff700 1 --2- 172.21.15.63:0/4147606132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac5c09c480 0x7fac5c09c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:58.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.295+0000 7fac63fff700 1 --2- 172.21.15.63:0/4147606132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac5c09b1d0 0x7fac5c09b5e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:58.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.295+0000 7fac63fff700 1 -- 172.21.15.63:0/4147606132 >> 172.21.15.63:0/4147606132 conn(0x7fac5c00a360 msgr2=0x7fac5c00c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:58.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.295+0000 7fac63fff700 1 -- 172.21.15.63:0/4147606132 shutdown_connections 2024-03-22T07:15:58.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.295+0000 7fac63fff700 1 -- 172.21.15.63:0/4147606132 wait complete. 2024-03-22T07:15:58.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.296+0000 7fac63fff700 1 Processor -- start 2024-03-22T07:15:58.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.296+0000 7fac63fff700 1 -- start start 2024-03-22T07:15:58.297 INFO:tasks.workunit.client.1.smithi069.stdout:6/163: creat d4/dd/d12/d2b/f30 x:0 0 0 2024-03-22T07:15:58.298 INFO:tasks.workunit.client.1.smithi069.stdout:6/164: write d4/d7/f26 [1037605,42502] 0 2024-03-22T07:15:58.298 INFO:tasks.workunit.client.1.smithi069.stdout:6/165: fsync d4/f14 0 2024-03-22T07:15:58.298 INFO:tasks.workunit.client.1.smithi069.stdout:6/166: readlink d4/dd/d12/l17 0 2024-03-22T07:15:58.298 INFO:tasks.workunit.client.1.smithi069.stdout:3/184: rmdir d5/d9/db 39 2024-03-22T07:15:58.298 INFO:tasks.workunit.client.1.smithi069.stdout:3/185: chown d5/d33 1 1 2024-03-22T07:15:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.296+0000 7fac63fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac5c09b1d0 0x7fac5c0abd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.296+0000 7fac63fff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac5c09c480 0x7fac5c0ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.296+0000 7fac63fff700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c0ac960 con 0x7fac5c09b1d0 2024-03-22T07:15:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.296+0000 7fac63fff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c0b1300 con 0x7fac5c09c480 2024-03-22T07:15:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.296+0000 7fac62ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac5c09b1d0 0x7fac5c0abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.296+0000 7fac62ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac5c09b1d0 0x7fac5c0abd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39828/0 (socket says 172.21.15.63:39828) 2024-03-22T07:15:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.297+0000 7fac62ffd700 1 -- 172.21.15.63:0/3046674262 learned_addr learned my addr 172.21.15.63:0/3046674262 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:15:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.297+0000 7fac627fc700 1 --2- 172.21.15.63:0/3046674262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac5c09c480 0x7fac5c0ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.297+0000 7fac62ffd700 1 -- 172.21.15.63:0/3046674262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac5c09c480 msgr2=0x7fac5c0ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.297+0000 7fac62ffd700 1 --2- 172.21.15.63:0/3046674262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac5c09c480 0x7fac5c0ac2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.297+0000 7fac62ffd700 1 -- 172.21.15.63:0/3046674262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6404bf10 con 0x7fac5c09b1d0 2024-03-22T07:15:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.297+0000 7fac627fc700 1 --2- 172.21.15.63:0/3046674262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac5c09c480 0x7fac5c0ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:15:58.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.297+0000 7fac62ffd700 1 --2- 172.21.15.63:0/3046674262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac5c09b1d0 0x7fac5c0abd90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fac64061610 tx=0x7fac640636e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:58.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.297+0000 7fac5b7fe700 1 -- 172.21.15.63:0/3046674262 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac64083070 con 0x7fac5c09b1d0 2024-03-22T07:15:58.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.297+0000 7fac63fff700 1 -- 172.21.15.63:0/3046674262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac5c0b1500 con 0x7fac5c09b1d0 2024-03-22T07:15:58.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.297+0000 7fac63fff700 1 -- 172.21.15.63:0/3046674262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac5c0b1a40 con 0x7fac5c09b1d0 2024-03-22T07:15:58.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.298+0000 7fac5b7fe700 1 -- 172.21.15.63:0/3046674262 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac64065050 con 0x7fac5c09b1d0 2024-03-22T07:15:58.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.298+0000 7fac5b7fe700 1 -- 172.21.15.63:0/3046674262 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac640736b0 con 0x7fac5c09b1d0 2024-03-22T07:15:58.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.299+0000 7fac63fff700 1 -- 172.21.15.63:0/3046674262 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac440052e0 con 0x7fac5c09b1d0 2024-03-22T07:15:58.301 INFO:tasks.workunit.client.1.smithi069.stdout:0/261: mknod d3/c58 0 2024-03-22T07:15:58.301 INFO:tasks.workunit.client.1.smithi069.stdout:8/269: mknod d1/d1c/d3e/c51 0 2024-03-22T07:15:58.301 INFO:tasks.workunit.client.1.smithi069.stdout:8/270: creat d1/d1c/f52 x:0 0 0 2024-03-22T07:15:58.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.299+0000 7fac5b7fe700 1 -- 172.21.15.63:0/3046674262 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 20) v1 ==== 88751+0+0 (secure 0 0 0) 0x7fac64064ac0 con 0x7fac5c09b1d0 2024-03-22T07:15:58.301 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.300+0000 7fac5b7fe700 1 --2- 172.21.15.63:0/3046674262 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fac5006af90 0x7fac5006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:58.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.300+0000 7fac627fc700 1 --2- 172.21.15.63:0/3046674262 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fac5006af90 0x7fac5006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:58.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.300+0000 7fac5b7fe700 1 -- 172.21.15.63:0/3046674262 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fac640f3550 con 0x7fac5c09b1d0 2024-03-22T07:15:58.302 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.301+0000 7fac627fc700 1 --2- 172.21.15.63:0/3046674262 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fac5006af90 0x7fac5006d440 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fac4c009b10 tx=0x7fac4c008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:58.302 INFO:tasks.workunit.client.0.smithi063.stdout:2/489: dread d0/de/f14 [0,4194304] 0 2024-03-22T07:15:58.303 INFO:tasks.workunit.client.1.smithi069.stdout:6/167: creat d4/dd/d12/d13/f31 x:0 0 0 2024-03-22T07:15:58.303 INFO:tasks.workunit.client.1.smithi069.stdout:4/309: rmdir d6/dc/d1d/d31 39 2024-03-22T07:15:58.303 INFO:tasks.workunit.client.1.smithi069.stdout:5/154: dread d1/d7/f22 [0,4194304] 0 2024-03-22T07:15:58.303 INFO:tasks.workunit.client.1.smithi069.stdout:0/262: dread d3/d7/f1e [0,4194304] 0 2024-03-22T07:15:58.303 INFO:tasks.workunit.client.1.smithi069.stdout:5/155: readlink d1/d6/l20 0 2024-03-22T07:15:58.303 INFO:tasks.workunit.client.1.smithi069.stdout:0/263: readlink d3/da/l25 0 2024-03-22T07:15:58.303 INFO:tasks.workunit.client.1.smithi069.stdout:5/156: truncate d1/d7/f22 2163582 0 2024-03-22T07:15:58.303 INFO:tasks.workunit.client.1.smithi069.stdout:5/157: creat d1/d7/d8/f30 x:0 0 0 2024-03-22T07:15:58.303 INFO:tasks.workunit.client.1.smithi069.stdout:4/310: dread d6/dc/d1d/d31/d1f/d3f/f46 [0,4194304] 0 2024-03-22T07:15:58.305 INFO:tasks.workunit.client.1.smithi069.stdout:2/199: dread d18/f1b [0,4194304] 0 2024-03-22T07:15:58.306 INFO:tasks.workunit.client.1.smithi069.stdout:2/200: chown f6 2021308841 1 2024-03-22T07:15:58.306 INFO:tasks.workunit.client.1.smithi069.stdout:2/201: fdatasync d18/d1c/d27/d33/f39 0 2024-03-22T07:15:58.306 INFO:tasks.workunit.client.1.smithi069.stdout:2/202: creat d18/d1c/d25/f4b x:0 0 0 2024-03-22T07:15:58.306 INFO:tasks.workunit.client.0.smithi063.stdout:5/481: creat d8/d13/d28/d57/d5f/d87/f88 x:0 0 0 2024-03-22T07:15:58.306 INFO:tasks.workunit.client.0.smithi063.stdout:5/482: readlink d8/d13/l19 0 2024-03-22T07:15:58.306 INFO:tasks.workunit.client.0.smithi063.stdout:0/637: dread da/de/d17/d25/d3c/d5f/d97/fa4 [0,4194304] 0 2024-03-22T07:15:58.307 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.307+0000 7fac5b7fe700 1 -- 172.21.15.63:0/3046674262 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fac640c32c0 con 0x7fac5c09b1d0 2024-03-22T07:15:58.307 INFO:tasks.workunit.client.1.smithi069.stdout:3/186: rename d5/d9/fe to d5/d9/d1e/f3e 0 2024-03-22T07:15:58.308 INFO:tasks.workunit.client.0.smithi063.stdout:0/638: mkdir da/dca 0 2024-03-22T07:15:58.311 INFO:tasks.workunit.client.0.smithi063.stdout:0/639: dread da/de/d39/f8c [0,4194304] 0 2024-03-22T07:15:58.312 INFO:tasks.workunit.client.1.smithi069.stdout:6/168: creat d4/dd/d12/f32 x:0 0 0 2024-03-22T07:15:58.317 INFO:tasks.workunit.client.1.smithi069.stdout:4/311: write d6/d12/d18/f4f [247718,80521] 0 2024-03-22T07:15:58.318 INFO:tasks.workunit.client.1.smithi069.stdout:7/274: dwrite d9/db/d48/d50/d57/f59 [0,4194304] 0 2024-03-22T07:15:58.318 INFO:tasks.workunit.client.1.smithi069.stdout:7/275: chown d9/db/d1a/f56 0 1 2024-03-22T07:15:58.319 INFO:tasks.workunit.client.1.smithi069.stdout:6/169: creat d4/dd/f33 x:0 0 0 2024-03-22T07:15:58.319 INFO:tasks.workunit.client.1.smithi069.stdout:6/170: creat d4/dd/d12/f34 x:0 0 0 2024-03-22T07:15:58.321 INFO:tasks.workunit.client.1.smithi069.stdout:4/312: link d6/dc/d1d/d31/d1f/f5b d6/dc/d23/f5c 0 2024-03-22T07:15:58.324 INFO:tasks.workunit.client.1.smithi069.stdout:7/276: creat d9/db/d48/d50/d57/d44/d53/f60 x:0 0 0 2024-03-22T07:15:58.326 INFO:tasks.workunit.client.1.smithi069.stdout:4/313: mkdir d6/d12/d5d 0 2024-03-22T07:15:58.327 INFO:tasks.workunit.client.0.smithi063.stdout:0/640: dread da/de/d60/f69 [0,4194304] 0 2024-03-22T07:15:58.327 INFO:tasks.workunit.client.1.smithi069.stdout:7/277: mkdir d9/db/d48/d50/d57/d44/d61 0 2024-03-22T07:15:58.328 INFO:tasks.workunit.client.1.smithi069.stdout:4/314: symlink d6/dc/d1d/d31/d1f/d3f/d51/l5e 0 2024-03-22T07:15:58.328 INFO:tasks.workunit.client.1.smithi069.stdout:4/315: fdatasync d6/f30 0 2024-03-22T07:15:58.329 INFO:tasks.workunit.client.0.smithi063.stdout:0/641: unlink da/de/d17/d25/d3c/d43/l8a 0 2024-03-22T07:15:58.329 INFO:tasks.workunit.client.1.smithi069.stdout:7/278: creat d9/db/d48/d50/d57/d44/f62 x:0 0 0 2024-03-22T07:15:58.330 INFO:tasks.workunit.client.1.smithi069.stdout:4/316: creat d6/dc/d1d/d55/f5f x:0 0 0 2024-03-22T07:15:58.331 INFO:tasks.workunit.client.1.smithi069.stdout:7/279: rmdir d9/db/d48/d50/d57/d44/d61 0 2024-03-22T07:15:58.350 INFO:tasks.workunit.client.1.smithi069.stdout:4/317: dread d6/d12/d18/d21/f2f [0,4194304] 0 2024-03-22T07:15:58.352 INFO:tasks.workunit.client.0.smithi063.stdout:1/589: dwrite d4/d14/f94 [0,4194304] 0 2024-03-22T07:15:58.352 INFO:tasks.workunit.client.0.smithi063.stdout:1/590: chown d4/df/d13/l2d 1906 1 2024-03-22T07:15:58.352 INFO:tasks.workunit.client.1.smithi069.stdout:9/201: dwrite d2/f5 [0,4194304] 0 2024-03-22T07:15:58.355 INFO:tasks.workunit.client.0.smithi063.stdout:1/591: getdents d4/df/d13/d39 0 2024-03-22T07:15:58.357 INFO:tasks.workunit.client.0.smithi063.stdout:1/592: symlink d4/d71/dbb/lc8 0 2024-03-22T07:15:58.359 INFO:tasks.workunit.client.0.smithi063.stdout:1/593: rename d4/d5a/l7f to d4/d5a/lc9 0 2024-03-22T07:15:58.360 INFO:tasks.workunit.client.0.smithi063.stdout:1/594: dread d4/d14/d57/f80 [0,4194304] 0 2024-03-22T07:15:58.361 INFO:tasks.workunit.client.1.smithi069.stdout:8/271: dwrite d1/d1c/d3e/f32 [0,4194304] 0 2024-03-22T07:15:58.361 INFO:tasks.workunit.client.1.smithi069.stdout:8/272: chown d1/d1c/d3e/f36 2 1 2024-03-22T07:15:58.361 INFO:tasks.workunit.client.0.smithi063.stdout:1/595: rename d4/d14/l3e to d4/d14/lca 0 2024-03-22T07:15:58.362 INFO:tasks.workunit.client.0.smithi063.stdout:1/596: stat d4/d5a/c61 0 2024-03-22T07:15:58.364 INFO:tasks.workunit.client.1.smithi069.stdout:4/318: write d6/dc/d1d/d31/d1f/f42 [661734,19183] 0 2024-03-22T07:15:58.365 INFO:tasks.workunit.client.1.smithi069.stdout:4/319: mknod d6/d12/c60 0 2024-03-22T07:15:58.367 INFO:tasks.workunit.client.1.smithi069.stdout:4/320: mknod d6/dc/d1d/c61 0 2024-03-22T07:15:58.367 INFO:tasks.workunit.client.1.smithi069.stdout:4/321: fsync d6/d16/f57 0 2024-03-22T07:15:58.369 INFO:tasks.workunit.client.1.smithi069.stdout:9/202: dread d2/dd/d3c/f24 [0,4194304] 0 2024-03-22T07:15:58.369 INFO:tasks.workunit.client.1.smithi069.stdout:9/203: chown d2/dd/d3c/l33 5693115 1 2024-03-22T07:15:58.370 INFO:tasks.workunit.client.1.smithi069.stdout:4/322: creat d6/dc/d1d/d31/d1f/f62 x:0 0 0 2024-03-22T07:15:58.370 INFO:tasks.workunit.client.1.smithi069.stdout:4/323: write f5 [8790491,2153] 0 2024-03-22T07:15:58.370 INFO:tasks.workunit.client.1.smithi069.stdout:4/324: fdatasync d6/d12/d18/f45 0 2024-03-22T07:15:58.370 INFO:tasks.workunit.client.1.smithi069.stdout:0/264: dwrite d3/d7/d15/d1f/f26 [0,4194304] 0 2024-03-22T07:15:58.370 INFO:tasks.workunit.client.1.smithi069.stdout:5/158: dwrite d1/d7/f25 [0,4194304] 0 2024-03-22T07:15:58.371 INFO:tasks.workunit.client.1.smithi069.stdout:5/159: chown d1/d7/f25 34655314 1 2024-03-22T07:15:58.373 INFO:tasks.workunit.client.1.smithi069.stdout:9/204: getdents d2/dd/d3c/d2e 0 2024-03-22T07:15:58.373 INFO:tasks.workunit.client.1.smithi069.stdout:6/171: dwrite d4/dd/f25 [4194304,4194304] 0 2024-03-22T07:15:58.374 INFO:tasks.workunit.client.1.smithi069.stdout:2/203: dwrite d18/d1c/d27/d33/f39 [0,4194304] 0 2024-03-22T07:15:58.374 INFO:tasks.workunit.client.0.smithi063.stdout:0/642: dwrite da/f40 [0,4194304] 0 2024-03-22T07:15:58.377 INFO:tasks.workunit.client.0.smithi063.stdout:0/643: rename da/de/d17/d4e/cae to da/de/ccb 0 2024-03-22T07:15:58.377 INFO:tasks.workunit.client.0.smithi063.stdout:0/644: creat da/de/d67/fcc x:0 0 0 2024-03-22T07:15:58.379 INFO:tasks.workunit.client.1.smithi069.stdout:5/160: mknod d1/d28/c31 0 2024-03-22T07:15:58.383 INFO:tasks.workunit.client.1.smithi069.stdout:9/205: symlink d2/d21/l41 0 2024-03-22T07:15:58.383 INFO:tasks.workunit.client.0.smithi063.stdout:1/597: dwrite d4/f1f [0,4194304] 0 2024-03-22T07:15:58.383 INFO:tasks.workunit.client.0.smithi063.stdout:0/645: mknod da/de/d67/ccd 0 2024-03-22T07:15:58.384 INFO:tasks.workunit.client.0.smithi063.stdout:1/598: truncate d4/d14/f18 401962 0 2024-03-22T07:15:58.385 INFO:tasks.workunit.client.1.smithi069.stdout:3/187: rename d5/d9/d1e to d5/d9/d26/d3f 0 2024-03-22T07:15:58.386 INFO:tasks.workunit.client.1.smithi069.stdout:7/280: rename d9/db/d48 to d9/db/d48/d50/d57/d44/d63 22 2024-03-22T07:15:58.386 INFO:tasks.workunit.client.1.smithi069.stdout:7/281: readlink d9/db/d48/d50/d57/d44/d53/l2d 0 2024-03-22T07:15:58.386 INFO:tasks.workunit.client.1.smithi069.stdout:7/282: chown d9/db/d48/d50/d57/d44/d53/d2e/c35 1 1 2024-03-22T07:15:58.388 INFO:tasks.workunit.client.0.smithi063.stdout:4/408: sync 2024-03-22T07:15:58.394 INFO:tasks.workunit.client.1.smithi069.stdout:9/206: write d2/d31/f40 [2117395,11647] 0 2024-03-22T07:15:58.394 INFO:tasks.workunit.client.1.smithi069.stdout:9/207: chown d2 0 1 2024-03-22T07:15:58.398 INFO:tasks.workunit.client.1.smithi069.stdout:8/273: dwrite d1/de/d20/f3c [0,4194304] 0 2024-03-22T07:15:58.400 INFO:tasks.workunit.client.0.smithi063.stdout:0/646: write da/de/d17/f2f [1813689,122430] 0 2024-03-22T07:15:58.401 INFO:tasks.workunit.client.1.smithi069.stdout:2/204: link d18/d1c/d27/d33/c37 d18/d1c/d34/c4c 0 2024-03-22T07:15:58.401 INFO:tasks.workunit.client.1.smithi069.stdout:0/265: dwrite d3/d7/f4a [0,4194304] 0 2024-03-22T07:15:58.402 INFO:tasks.workunit.client.1.smithi069.stdout:5/161: mknod d1/d7/d8/c32 0 2024-03-22T07:15:58.404 INFO:tasks.workunit.client.1.smithi069.stdout:3/188: rmdir d5/d33/d3c 0 2024-03-22T07:15:58.404 INFO:tasks.workunit.client.1.smithi069.stdout:3/189: stat d5/c2c 0 2024-03-22T07:15:58.405 INFO:tasks.workunit.client.1.smithi069.stdout:7/283: creat d9/d42/f64 x:0 0 0 2024-03-22T07:15:58.405 INFO:tasks.workunit.client.1.smithi069.stdout:7/284: fdatasync d9/db/d48/d50/d57/d44/d53/f43 0 2024-03-22T07:15:58.411 INFO:tasks.workunit.client.1.smithi069.stdout:6/172: link d4/dd/d12/d2b/f30 d4/d7/f35 0 2024-03-22T07:15:58.411 INFO:tasks.workunit.client.0.smithi063.stdout:8/539: sync 2024-03-22T07:15:58.412 INFO:tasks.workunit.client.0.smithi063.stdout:8/540: fsync d0/d24/d5b/f95 0 2024-03-22T07:15:58.412 INFO:tasks.workunit.client.1.smithi069.stdout:8/274: dread d1/d1c/d3e/f19 [0,4194304] 0 2024-03-22T07:15:58.416 INFO:tasks.workunit.client.0.smithi063.stdout:0/647: write da/de/d17/d4e/f9e [2404411,21318] 0 2024-03-22T07:15:58.416 INFO:tasks.workunit.client.0.smithi063.stdout:0/648: creat da/de/d17/d4e/fce x:0 0 0 2024-03-22T07:15:58.416 INFO:tasks.workunit.client.0.smithi063.stdout:0/649: creat da/de/fcf x:0 0 0 2024-03-22T07:15:58.417 INFO:tasks.workunit.client.1.smithi069.stdout:0/266: mknod d3/d7/d17/d1c/c59 0 2024-03-22T07:15:58.419 INFO:tasks.workunit.client.0.smithi063.stdout:0/650: symlink da/ld0 0 2024-03-22T07:15:58.419 INFO:tasks.workunit.client.0.smithi063.stdout:0/651: fdatasync da/de/d17/d25/d3c/d43/d76/f9d 0 2024-03-22T07:15:58.420 INFO:tasks.workunit.client.1.smithi069.stdout:5/162: mkdir d1/d6/d26/d33 0 2024-03-22T07:15:58.421 INFO:tasks.workunit.client.1.smithi069.stdout:3/190: symlink d5/d9/db/d3b/l40 0 2024-03-22T07:15:58.429 INFO:tasks.workunit.client.1.smithi069.stdout:8/275: symlink d1/de/d20/d43/d4e/l53 0 2024-03-22T07:15:58.429 INFO:tasks.workunit.client.1.smithi069.stdout:8/276: fsync d1/f29 0 2024-03-22T07:15:58.431 INFO:tasks.workunit.client.1.smithi069.stdout:0/267: mkdir d3/d7/d15/d29/d5a 0 2024-03-22T07:15:58.432 INFO:tasks.workunit.client.1.smithi069.stdout:3/191: mkdir d5/d9/d26/d36/d41 0 2024-03-22T07:15:58.433 INFO:tasks.workunit.client.1.smithi069.stdout:7/285: getdents d9/db/d48/d50/d57/d44/d53 0 2024-03-22T07:15:58.433 INFO:tasks.workunit.client.1.smithi069.stdout:7/286: creat d9/db/d48/d4b/f65 x:0 0 0 2024-03-22T07:15:58.434 INFO:tasks.workunit.client.0.smithi063.stdout:0/652: dread da/de/d17/f1a [0,4194304] 0 2024-03-22T07:15:58.434 INFO:tasks.workunit.client.0.smithi063.stdout:0/653: read da/de/d17/d25/f65 [151417,20439] 0 2024-03-22T07:15:58.436 INFO:tasks.workunit.client.0.smithi063.stdout:0/654: getdents da/de/d60 0 2024-03-22T07:15:58.436 INFO:tasks.workunit.client.0.smithi063.stdout:0/655: fdatasync da/de/d17/d4e/f57 0 2024-03-22T07:15:58.439 INFO:tasks.workunit.client.1.smithi069.stdout:7/287: symlink d9/db/d48/d50/d57/d44/d53/d2e/l66 0 2024-03-22T07:15:58.441 INFO:tasks.workunit.client.1.smithi069.stdout:7/288: symlink d9/db/l67 0 2024-03-22T07:15:58.441 INFO:tasks.workunit.client.1.smithi069.stdout:7/289: truncate d9/db/d1a/f49 287795 0 2024-03-22T07:15:58.443 INFO:tasks.workunit.client.1.smithi069.stdout:7/290: rmdir d9/db/d48/d50/d57/d44/d53/d1d/d2a 39 2024-03-22T07:15:58.443 INFO:tasks.workunit.client.1.smithi069.stdout:8/277: read d1/d1c/d3e/f32 [1101351,87739] 0 2024-03-22T07:15:58.444 INFO:tasks.workunit.client.1.smithi069.stdout:8/278: readlink d1/de/l35 0 2024-03-22T07:15:58.446 INFO:tasks.workunit.client.1.smithi069.stdout:7/291: rename d9/db/d48/c5c to d9/db/d48/d50/d57/d44/c68 0 2024-03-22T07:15:58.446 INFO:tasks.workunit.client.1.smithi069.stdout:7/292: creat d9/db/f69 x:0 0 0 2024-03-22T07:15:58.448 INFO:tasks.workunit.client.1.smithi069.stdout:8/279: dread d1/d1c/d3e/f4c [0,4194304] 0 2024-03-22T07:15:58.449 INFO:tasks.workunit.client.1.smithi069.stdout:2/205: rename d18/d1c to d18/d4d 0 2024-03-22T07:15:58.449 INFO:tasks.workunit.client.1.smithi069.stdout:2/206: truncate d18/d4d/f41 220395 0 2024-03-22T07:15:58.449 INFO:tasks.workunit.client.0.smithi063.stdout:0/656: dread da/de/f62 [0,4194304] 0 2024-03-22T07:15:58.449 INFO:tasks.workunit.client.0.smithi063.stdout:0/657: fdatasync da/de/fcf 0 2024-03-22T07:15:58.451 INFO:tasks.workunit.client.1.smithi069.stdout:8/280: link d1/de/d20/l42 d1/de/d20/l54 0 2024-03-22T07:15:58.451 INFO:tasks.workunit.client.0.smithi063.stdout:8/541: dwrite d0/f62 [0,4194304] 0 2024-03-22T07:15:58.453 INFO:tasks.workunit.client.0.smithi063.stdout:8/542: mknod d0/d26/d2d/d83/cae 0 2024-03-22T07:15:58.454 INFO:tasks.workunit.client.1.smithi069.stdout:2/207: rename d18/d4d/c31 to d18/d4d/d27/c4e 0 2024-03-22T07:15:58.458 INFO:tasks.workunit.client.1.smithi069.stdout:2/208: truncate f5 1243095 0 2024-03-22T07:15:58.466 INFO:tasks.workunit.client.1.smithi069.stdout:8/281: dread d1/de/d20/f4f [0,4194304] 0 2024-03-22T07:15:58.466 INFO:tasks.workunit.client.1.smithi069.stdout:8/282: write d1/d1c/f47 [151153,54199] 0 2024-03-22T07:15:58.468 INFO:tasks.workunit.client.1.smithi069.stdout:5/163: dwrite d1/d6/f1e [0,4194304] 0 2024-03-22T07:15:58.470 INFO:tasks.workunit.client.1.smithi069.stdout:5/164: stat d1/d7/cb 0 2024-03-22T07:15:58.470 INFO:tasks.workunit.client.1.smithi069.stdout:5/165: fdatasync d1/d6/f1e 0 2024-03-22T07:15:58.470 INFO:tasks.workunit.client.1.smithi069.stdout:4/325: dwrite d6/dc/d1d/d31/d1f/f42 [4194304,4194304] 0 2024-03-22T07:15:58.471 INFO:tasks.workunit.client.1.smithi069.stdout:8/283: dread d1/de/d20/f4f [0,4194304] 0 2024-03-22T07:15:58.471 INFO:tasks.workunit.client.1.smithi069.stdout:5/166: symlink d1/d6/l34 0 2024-03-22T07:15:58.472 INFO:tasks.workunit.client.1.smithi069.stdout:8/284: symlink d1/de/d20/d43/d4e/l55 0 2024-03-22T07:15:58.473 INFO:tasks.workunit.client.1.smithi069.stdout:0/268: dwrite d3/d7/d15/d1f/f41 [0,4194304] 0 2024-03-22T07:15:58.474 INFO:tasks.workunit.client.1.smithi069.stdout:8/285: mkdir d1/d1c/d48/d56 0 2024-03-22T07:15:58.475 INFO:tasks.workunit.client.1.smithi069.stdout:8/286: readlink d1/d1c/d3e/l3f 0 2024-03-22T07:15:58.475 INFO:tasks.workunit.client.1.smithi069.stdout:8/287: readlink d1/de/l2e 0 2024-03-22T07:15:58.475 INFO:tasks.workunit.client.1.smithi069.stdout:8/288: fdatasync d1/d1c/d3e/f36 0 2024-03-22T07:15:58.476 INFO:tasks.workunit.client.0.smithi063.stdout:4/409: dwrite d1/d5/d43/d27/d2c/d4a/f4d [4194304,4194304] 0 2024-03-22T07:15:58.487 INFO:tasks.workunit.client.0.smithi063.stdout:4/410: dread d1/f45 [0,4194304] 0 2024-03-22T07:15:58.488 INFO:tasks.workunit.client.0.smithi063.stdout:4/411: readlink d1/d5/dc/l3b 0 2024-03-22T07:15:58.488 INFO:tasks.workunit.client.0.smithi063.stdout:4/412: mknod d1/d5/d43/d27/d2c/d4a/d78/c91 0 2024-03-22T07:15:58.490 INFO:tasks.workunit.client.1.smithi069.stdout:6/173: dwrite d4/f6 [0,4194304] 0 2024-03-22T07:15:58.490 INFO:tasks.workunit.client.1.smithi069.stdout:6/174: creat d4/dd/d12/d2b/f36 x:0 0 0 2024-03-22T07:15:58.491 INFO:tasks.workunit.client.1.smithi069.stdout:3/192: dwrite d5/d9/d26/f28 [0,4194304] 0 2024-03-22T07:15:58.491 INFO:tasks.workunit.client.1.smithi069.stdout:3/193: fdatasync d5/f18 0 2024-03-22T07:15:58.491 INFO:tasks.workunit.client.1.smithi069.stdout:3/194: write d5/d9/d26/f2e [1657616,130191] 0 2024-03-22T07:15:58.493 INFO:tasks.workunit.client.1.smithi069.stdout:3/195: symlink d5/d9/d26/d3f/d35/l42 0 2024-03-22T07:15:58.494 INFO:tasks.workunit.client.1.smithi069.stdout:3/196: fsync d5/f18 0 2024-03-22T07:15:58.494 INFO:tasks.workunit.client.1.smithi069.stdout:3/197: fsync d5/f7 0 2024-03-22T07:15:58.494 INFO:tasks.workunit.client.1.smithi069.stdout:3/198: creat d5/d9/d26/d36/d41/f43 x:0 0 0 2024-03-22T07:15:58.494 INFO:tasks.workunit.client.1.smithi069.stdout:3/199: write d5/d33/f39 [1289478,63519] 0 2024-03-22T07:15:58.496 INFO:tasks.workunit.client.1.smithi069.stdout:3/200: rename d5/l16 to d5/d9/d26/l44 0 2024-03-22T07:15:58.496 INFO:tasks.workunit.client.1.smithi069.stdout:3/201: write d5/d9/d26/d3f/f3e [2429895,88399] 0 2024-03-22T07:15:58.496 INFO:tasks.workunit.client.1.smithi069.stdout:1/232: sync 2024-03-22T07:15:58.499 INFO:tasks.workunit.client.1.smithi069.stdout:1/233: unlink d1/d25/d41/l1d 0 2024-03-22T07:15:58.501 INFO:tasks.workunit.client.1.smithi069.stdout:1/234: mkdir d1/d13/d37/d4c 0 2024-03-22T07:15:58.501 INFO:tasks.workunit.client.1.smithi069.stdout:1/235: read - d1/f48 zero size 2024-03-22T07:15:58.502 INFO:tasks.workunit.client.1.smithi069.stdout:1/236: creat d1/d25/f4d x:0 0 0 2024-03-22T07:15:58.512 INFO:tasks.workunit.client.1.smithi069.stdout:7/293: dwrite d9/f18 [0,4194304] 0 2024-03-22T07:15:58.512 INFO:tasks.workunit.client.1.smithi069.stdout:7/294: chown d9/db/d48/d50/d57/c58 103189635 1 2024-03-22T07:15:58.513 INFO:tasks.workunit.client.1.smithi069.stdout:7/295: readlink d9/db/d48/d50/d57/d44/d53/d1d/d2a/l4a 0 2024-03-22T07:15:58.513 INFO:tasks.workunit.client.1.smithi069.stdout:7/296: read d9/db/d48/d50/d57/d44/d53/f38 [1009247,31002] 0 2024-03-22T07:15:58.526 INFO:tasks.workunit.client.0.smithi063.stdout:0/658: dwrite da/d49/f6d [4194304,4194304] 0 2024-03-22T07:15:58.527 INFO:tasks.workunit.client.0.smithi063.stdout:0/659: dread da/de/d17/d25/d3c/d43/f83 [0,4194304] 0 2024-03-22T07:15:58.528 INFO:tasks.workunit.client.0.smithi063.stdout:0/660: creat da/de/d17/d25/d3c/d53/d89/fd1 x:0 0 0 2024-03-22T07:15:58.528 INFO:tasks.workunit.client.0.smithi063.stdout:0/661: readlink da/de/l1d 0 2024-03-22T07:15:58.529 INFO:tasks.workunit.client.1.smithi069.stdout:2/209: dwrite d18/d4d/f41 [0,4194304] 0 2024-03-22T07:15:58.529 INFO:tasks.workunit.client.0.smithi063.stdout:8/543: dwrite d0/d24/f5d [0,4194304] 0 2024-03-22T07:15:58.529 INFO:tasks.workunit.client.0.smithi063.stdout:8/544: creat d0/d26/faf x:0 0 0 2024-03-22T07:15:58.532 INFO:tasks.workunit.client.1.smithi069.stdout:8/289: dwrite d1/f9 [4194304,4194304] 0 2024-03-22T07:15:58.532 INFO:tasks.workunit.client.1.smithi069.stdout:8/290: fsync d1/de/f2d 0 2024-03-22T07:15:58.534 INFO:tasks.workunit.client.0.smithi063.stdout:8/545: getdents d0/d1b/d28 0 2024-03-22T07:15:58.534 INFO:tasks.workunit.client.1.smithi069.stdout:2/210: mkdir d18/d4d/d27/d33/d4f 0 2024-03-22T07:15:58.535 INFO:tasks.workunit.client.0.smithi063.stdout:8/546: symlink d0/d26/d2d/d83/lb0 0 2024-03-22T07:15:58.536 INFO:tasks.workunit.client.1.smithi069.stdout:5/167: dwrite d1/d6/f2e [0,4194304] 0 2024-03-22T07:15:58.536 INFO:tasks.workunit.client.1.smithi069.stdout:5/168: fdatasync d1/d7/d8/f30 0 2024-03-22T07:15:58.537 INFO:tasks.workunit.client.0.smithi063.stdout:4/413: dwrite d1/d5/d43/d27/d2c/d4a/f8b [0,4194304] 0 2024-03-22T07:15:58.537 INFO:tasks.workunit.client.0.smithi063.stdout:4/414: write d1/d5/d3c/f82 [157162,85291] 0 2024-03-22T07:15:58.538 INFO:tasks.workunit.client.0.smithi063.stdout:8/547: creat d0/d26/d2d/fb1 x:0 0 0 2024-03-22T07:15:58.538 INFO:tasks.workunit.client.1.smithi069.stdout:0/269: dwrite d3/d7/d15/d1f/f41 [0,4194304] 0 2024-03-22T07:15:58.538 INFO:tasks.workunit.client.1.smithi069.stdout:0/270: fdatasync d3/d7/d17/f4c 0 2024-03-22T07:15:58.538 INFO:tasks.workunit.client.1.smithi069.stdout:0/271: fsync d3/d7/d15/f32 0 2024-03-22T07:15:58.538 INFO:tasks.workunit.client.1.smithi069.stdout:0/272: chown d3/d7/d17/d1c/c28 19 1 2024-03-22T07:15:58.538 INFO:tasks.workunit.client.1.smithi069.stdout:0/273: read - d3/d7/d15/d1f/f4e zero size 2024-03-22T07:15:58.538 INFO:tasks.workunit.client.1.smithi069.stdout:0/274: fdatasync d3/d7/fb 0 2024-03-22T07:15:58.538 INFO:tasks.workunit.client.1.smithi069.stdout:3/202: dwrite d5/d9/db/f3d [0,4194304] 0 2024-03-22T07:15:58.539 INFO:tasks.workunit.client.0.smithi063.stdout:4/415: symlink d1/d5/d43/d27/d2c/d3d/l92 0 2024-03-22T07:15:58.540 INFO:tasks.workunit.client.0.smithi063.stdout:8/548: mkdir d0/d1b/d28/d3e/db2 0 2024-03-22T07:15:58.540 INFO:tasks.workunit.client.0.smithi063.stdout:8/549: chown d0/d24/d33/c49 109858859 1 2024-03-22T07:15:58.540 INFO:tasks.workunit.client.0.smithi063.stdout:8/550: chown d0/d1b/d28/d3e/db2 8 1 2024-03-22T07:15:58.540 INFO:tasks.workunit.client.1.smithi069.stdout:4/326: dwrite d6/d12/d18/d21/f49 [0,4194304] 0 2024-03-22T07:15:58.541 INFO:tasks.workunit.client.1.smithi069.stdout:6/175: dwrite d4/f6 [0,4194304] 0 2024-03-22T07:15:58.541 INFO:tasks.workunit.client.1.smithi069.stdout:1/237: write d1/d13/d37/f40 [6378853,105909] 0 2024-03-22T07:15:58.541 INFO:tasks.workunit.client.1.smithi069.stdout:4/327: chown d6/d12/f33 14357 1 2024-03-22T07:15:58.541 INFO:tasks.workunit.client.1.smithi069.stdout:1/238: stat d1/d25/d41/f19 0 2024-03-22T07:15:58.541 INFO:tasks.workunit.client.1.smithi069.stdout:7/297: write d9/db/d48/d50/d57/d44/d53/d1d/d2a/d30/f54 [1688711,11484] 0 2024-03-22T07:15:58.542 INFO:tasks.workunit.client.0.smithi063.stdout:8/551: rename d0/d1f/fa7 to d0/d1b/d28/d5f/d99/da1/fb3 0 2024-03-22T07:15:58.543 INFO:tasks.workunit.client.1.smithi069.stdout:2/211: getdents d18/d4d/d3e 0 2024-03-22T07:15:58.543 INFO:tasks.workunit.client.1.smithi069.stdout:7/298: read d9/db/d48/d50/d57/d44/d53/d2e/f4d [28683,74952] 0 2024-03-22T07:15:58.544 INFO:tasks.workunit.client.0.smithi063.stdout:4/416: mkdir d1/d5/d93 0 2024-03-22T07:15:58.544 INFO:tasks.workunit.client.0.smithi063.stdout:4/417: write d1/f6a [391758,120646] 0 2024-03-22T07:15:58.544 INFO:tasks.workunit.client.0.smithi063.stdout:4/418: dread - d1/d5/dc/f5a zero size 2024-03-22T07:15:58.545 INFO:tasks.workunit.client.0.smithi063.stdout:8/552: symlink d0/d26/d2d/d59/lb4 0 2024-03-22T07:15:58.546 INFO:tasks.workunit.client.0.smithi063.stdout:4/419: mknod d1/d5/d43/d27/d2c/d3d/d54/d5f/c94 0 2024-03-22T07:15:58.546 INFO:tasks.workunit.client.0.smithi063.stdout:4/420: chown d1/d5/d43/d27/c2b 44697586 1 2024-03-22T07:15:58.549 INFO:tasks.workunit.client.1.smithi069.stdout:8/291: read d1/de/d20/f2f [7473208,13721] 0 2024-03-22T07:15:58.550 INFO:tasks.workunit.client.0.smithi063.stdout:8/553: symlink d0/d1b/d28/lb5 0 2024-03-22T07:15:58.551 INFO:tasks.workunit.client.1.smithi069.stdout:5/169: dread d1/d7/d8/f24 [0,4194304] 0 2024-03-22T07:15:58.556 INFO:tasks.workunit.client.1.smithi069.stdout:0/275: write d3/d7/d15/f1b [273865,8263] 0 2024-03-22T07:15:58.556 INFO:tasks.workunit.client.1.smithi069.stdout:1/239: truncate d1/f22 1417778 0 2024-03-22T07:15:58.556 INFO:tasks.workunit.client.1.smithi069.stdout:1/240: creat d1/d2a/f4e x:0 0 0 2024-03-22T07:15:58.560 INFO:tasks.workunit.client.0.smithi063.stdout:8/554: mknod d0/d26/d2d/d8f/cb6 0 2024-03-22T07:15:58.561 INFO:tasks.workunit.client.1.smithi069.stdout:8/292: rename d1/d1c/d3e/f4c to d1/d1c/d48/d56/f57 0 2024-03-22T07:15:58.561 INFO:tasks.workunit.client.1.smithi069.stdout:8/293: write d1/f1d [3138365,9971] 0 2024-03-22T07:15:58.561 INFO:tasks.workunit.client.1.smithi069.stdout:7/299: write f8 [2569981,71444] 0 2024-03-22T07:15:58.565 INFO:tasks.workunit.client.1.smithi069.stdout:3/203: dread d5/d9/db/f3d [0,4194304] 0 2024-03-22T07:15:58.565 INFO:tasks.workunit.client.1.smithi069.stdout:1/241: unlink d1/d25/c3f 0 2024-03-22T07:15:58.566 INFO:tasks.workunit.client.1.smithi069.stdout:1/242: write d1/d13/f2d [770962,79665] 0 2024-03-22T07:15:58.566 INFO:tasks.workunit.client.0.smithi063.stdout:4/421: rmdir d1/d5/d38/d59 39 2024-03-22T07:15:58.568 INFO:tasks.workunit.client.0.smithi063.stdout:8/555: mkdir d0/d24/d5b/db7 0 2024-03-22T07:15:58.570 INFO:tasks.workunit.client.0.smithi063.stdout:8/556: read d0/d1b/d28/d7b/d4a/f5a [10028,98836] 0 2024-03-22T07:15:58.570 INFO:tasks.workunit.client.0.smithi063.stdout:8/557: write d0/f62 [4976331,52517] 0 2024-03-22T07:15:58.572 INFO:tasks.workunit.client.0.smithi063.stdout:8/558: read d0/d1b/f84 [36552,67703] 0 2024-03-22T07:15:58.575 INFO:tasks.workunit.client.1.smithi069.stdout:1/243: mknod d1/c4f 0 2024-03-22T07:15:58.575 INFO:tasks.workunit.client.1.smithi069.stdout:7/300: write d9/f18 [640662,106286] 0 2024-03-22T07:15:58.575 INFO:tasks.workunit.client.0.smithi063.stdout:4/422: rename d1/d5/d38 to d1/d5/dc/d95 0 2024-03-22T07:15:58.577 INFO:tasks.workunit.client.1.smithi069.stdout:4/328: dwrite d6/d4b/f4d [0,4194304] 0 2024-03-22T07:15:58.577 INFO:tasks.workunit.client.1.smithi069.stdout:4/329: readlink d6/d12/d18/d21/l43 0 2024-03-22T07:15:58.577 INFO:tasks.workunit.client.1.smithi069.stdout:2/212: dwrite f12 [0,4194304] 0 2024-03-22T07:15:58.577 INFO:tasks.workunit.client.1.smithi069.stdout:2/213: write d18/d4d/f26 [242070,124917] 0 2024-03-22T07:15:58.577 INFO:tasks.workunit.client.1.smithi069.stdout:2/214: fdatasync f1 0 2024-03-22T07:15:58.578 INFO:tasks.workunit.client.0.smithi063.stdout:8/559: creat d0/d24/fb8 x:0 0 0 2024-03-22T07:15:58.578 INFO:tasks.workunit.client.0.smithi063.stdout:8/560: creat d0/d24/fb9 x:0 0 0 2024-03-22T07:15:58.578 INFO:tasks.workunit.client.0.smithi063.stdout:0/662: dwrite da/de/d17/d25/d3c/d5f/d97/fc5 [0,4194304] 0 2024-03-22T07:15:58.578 INFO:tasks.workunit.client.1.smithi069.stdout:1/244: dread d1/fb [0,4194304] 0 2024-03-22T07:15:58.579 INFO:tasks.workunit.client.1.smithi069.stdout:7/301: rename d9/f18 to d9/db/d48/d50/d57/d44/d53/d2e/f6a 0 2024-03-22T07:15:58.581 INFO:tasks.workunit.client.0.smithi063.stdout:4/423: symlink d1/d5/d43/d27/d2c/d4a/d78/l96 0 2024-03-22T07:15:58.584 INFO:tasks.workunit.client.1.smithi069.stdout:4/330: truncate d6/f30 614021 0 2024-03-22T07:15:58.584 INFO:tasks.workunit.client.1.smithi069.stdout:4/331: stat d6/dc/d1d/d31/d1f 0 2024-03-22T07:15:58.584 INFO:tasks.workunit.client.1.smithi069.stdout:4/332: fsync d6/dc/d1d/d31/d1f/f42 0 2024-03-22T07:15:58.584 INFO:tasks.workunit.client.1.smithi069.stdout:4/333: write d6/dc/d1d/d31/d1f/f62 [743408,10877] 0 2024-03-22T07:15:58.584 INFO:tasks.workunit.client.1.smithi069.stdout:4/334: readlink d6/d12/d18/d21/l43 0 2024-03-22T07:15:58.587 INFO:tasks.workunit.client.1.smithi069.stdout:2/215: mkdir d18/d4d/d3e/d50 0 2024-03-22T07:15:58.587 INFO:tasks.workunit.client.1.smithi069.stdout:2/216: read - d18/d4d/d27/d33/d3d/f46 zero size 2024-03-22T07:15:58.590 INFO:tasks.workunit.client.0.smithi063.stdout:0/663: creat da/de/d60/fd2 x:0 0 0 2024-03-22T07:15:58.590 INFO:tasks.workunit.client.0.smithi063.stdout:8/561: write d0/d24/d33/d4c/d78/f91 [722169,68293] 0 2024-03-22T07:15:58.591 INFO:tasks.workunit.client.0.smithi063.stdout:8/562: chown d0/f1a 3 1 2024-03-22T07:15:58.591 INFO:tasks.workunit.client.0.smithi063.stdout:8/563: write d0/f19 [3089198,114614] 0 2024-03-22T07:15:58.591 INFO:tasks.workunit.client.0.smithi063.stdout:4/424: fsync d1/d5/d43/f14 0 2024-03-22T07:15:58.591 INFO:tasks.workunit.client.0.smithi063.stdout:4/425: readlink d1/d5/d43/d27/d2c/d3d/l92 0 2024-03-22T07:15:58.591 INFO:tasks.workunit.client.1.smithi069.stdout:1/245: rename d1/d13/d37 to d1/d2a/d50 0 2024-03-22T07:15:58.592 INFO:tasks.workunit.client.1.smithi069.stdout:4/335: truncate d6/dc/d1d/d31/d1f/f39 402820 0 2024-03-22T07:15:58.594 INFO:tasks.workunit.client.0.smithi063.stdout:8/564: mkdir d0/d26/d2d/d59/dba 0 2024-03-22T07:15:58.595 INFO:tasks.workunit.client.1.smithi069.stdout:2/217: link d18/d4d/f2d d18/d4d/d3e/d40/f51 0 2024-03-22T07:15:58.595 INFO:tasks.workunit.client.1.smithi069.stdout:5/170: truncate d1/d6/f2e 1569737 0 2024-03-22T07:15:58.596 INFO:tasks.workunit.client.0.smithi063.stdout:8/565: mknod d0/d24/d33/d4c/d78/cbb 0 2024-03-22T07:15:58.596 INFO:tasks.workunit.client.1.smithi069.stdout:0/276: dwrite d3/d7/d15/f46 [0,4194304] 0 2024-03-22T07:15:58.600 INFO:tasks.workunit.client.1.smithi069.stdout:1/246: rename d1/d2a/f2b to d1/d25/d41/d1e/f51 0 2024-03-22T07:15:58.601 INFO:tasks.workunit.client.1.smithi069.stdout:3/204: dwrite d5/d9/d26/d3f/f22 [4194304,4194304] 0 2024-03-22T07:15:58.603 INFO:tasks.workunit.client.1.smithi069.stdout:0/277: mkdir d3/d7/d15/d29/d5a/d5b 0 2024-03-22T07:15:58.604 INFO:tasks.workunit.client.1.smithi069.stdout:0/278: dread - d3/d7/d17/f56 zero size 2024-03-22T07:15:58.604 INFO:tasks.workunit.client.1.smithi069.stdout:0/279: readlink d3/l43 0 2024-03-22T07:15:58.604 INFO:tasks.workunit.client.1.smithi069.stdout:0/280: creat d3/d7/f5c x:0 0 0 2024-03-22T07:15:58.608 INFO:tasks.workunit.client.0.smithi063.stdout:8/566: dread d0/d24/f29 [0,4194304] 0 2024-03-22T07:15:58.608 INFO:tasks.workunit.client.1.smithi069.stdout:8/294: dwrite d1/fd [0,4194304] 0 2024-03-22T07:15:58.609 INFO:tasks.workunit.client.1.smithi069.stdout:4/336: dread d6/dc/d1d/d31/d1f/f62 [0,4194304] 0 2024-03-22T07:15:58.609 INFO:tasks.workunit.client.0.smithi063.stdout:0/664: write da/de/d60/d56/d77/fa3 [3176861,24957] 0 2024-03-22T07:15:58.612 INFO:tasks.workunit.client.0.smithi063.stdout:0/665: write da/de/d60/f33 [773665,111574] 0 2024-03-22T07:15:58.613 INFO:tasks.workunit.client.1.smithi069.stdout:1/247: write d1/d25/d41/f19 [5575898,119980] 0 2024-03-22T07:15:58.613 INFO:tasks.workunit.client.1.smithi069.stdout:1/248: write d1/d25/d41/d1e/f51 [884560,81679] 0 2024-03-22T07:15:58.613 INFO:tasks.workunit.client.1.smithi069.stdout:1/249: dread - d1/d2a/f4e zero size 2024-03-22T07:15:58.613 INFO:tasks.workunit.client.0.smithi063.stdout:0/666: rmdir da/de/d17/d25/d3c/d43/d76 39 2024-03-22T07:15:58.613 INFO:tasks.workunit.client.0.smithi063.stdout:0/667: stat da/de/d39 0 2024-03-22T07:15:58.614 INFO:tasks.workunit.client.1.smithi069.stdout:5/171: rename d1/d6/c2c to d1/d10/c35 0 2024-03-22T07:15:58.615 INFO:tasks.workunit.client.1.smithi069.stdout:5/172: chown d1/d7/d8/f24 415 1 2024-03-22T07:15:58.615 INFO:tasks.workunit.client.0.smithi063.stdout:0/668: link da/d49/f63 da/de/d17/d25/d3c/fd3 0 2024-03-22T07:15:58.618 INFO:tasks.workunit.client.1.smithi069.stdout:3/205: write d5/d9/d26/d3f/f21 [1169588,20038] 0 2024-03-22T07:15:58.618 INFO:tasks.workunit.client.1.smithi069.stdout:0/281: mkdir d3/d7/d15/d29/d5a/d5d 0 2024-03-22T07:15:58.621 INFO:tasks.workunit.client.0.smithi063.stdout:4/426: dwrite d1/d5/d3c/f47 [4194304,4194304] 0 2024-03-22T07:15:58.622 INFO:tasks.workunit.client.0.smithi063.stdout:4/427: unlink d1/d5/d43/d27/c2b 0 2024-03-22T07:15:58.623 INFO:tasks.workunit.client.1.smithi069.stdout:8/295: mknod d1/de/d20/d43/d4e/c58 0 2024-03-22T07:15:58.623 INFO:tasks.workunit.client.1.smithi069.stdout:8/296: truncate d1/d1c/d3e/f36 1012474 0 2024-03-22T07:15:58.623 INFO:tasks.workunit.client.1.smithi069.stdout:8/297: creat d1/d1c/d48/f59 x:0 0 0 2024-03-22T07:15:58.623 INFO:tasks.workunit.client.1.smithi069.stdout:8/298: chown d1/de/d20/d43/d4e/l55 1321158 1 2024-03-22T07:15:58.624 INFO:tasks.workunit.client.0.smithi063.stdout:4/428: creat d1/d5/f97 x:0 0 0 2024-03-22T07:15:58.626 INFO:tasks.workunit.client.1.smithi069.stdout:2/218: dwrite d18/d4d/d25/f2e [0,4194304] 0 2024-03-22T07:15:58.626 INFO:tasks.workunit.client.0.smithi063.stdout:4/429: symlink d1/d5/l98 0 2024-03-22T07:15:58.626 INFO:tasks.workunit.client.0.smithi063.stdout:4/430: dread - d1/d5/dc/d95/d59/f87 zero size 2024-03-22T07:15:58.627 INFO:tasks.workunit.client.0.smithi063.stdout:4/431: creat d1/d5/d43/d27/d2c/d3d/f99 x:0 0 0 2024-03-22T07:15:58.627 INFO:tasks.workunit.client.1.smithi069.stdout:4/337: rename d6/d12/d18/l3d to d6/dc/l63 0 2024-03-22T07:15:58.628 INFO:tasks.workunit.client.0.smithi063.stdout:4/432: creat d1/d5/dc/d95/d59/f9a x:0 0 0 2024-03-22T07:15:58.630 INFO:tasks.workunit.client.1.smithi069.stdout:5/173: link d1/d7/d8/c32 d1/d6/c36 0 2024-03-22T07:15:58.631 INFO:tasks.workunit.client.0.smithi063.stdout:8/567: dwrite d0/f89 [0,4194304] 0 2024-03-22T07:15:58.631 INFO:tasks.workunit.client.0.smithi063.stdout:4/433: rename d1/d5/d43/d27/d2c/d3d/d54/d5f/d79/l81 to d1/d5/d43/d27/d2c/d4a/d78/l9b 0 2024-03-22T07:15:58.632 INFO:tasks.workunit.client.1.smithi069.stdout:3/206: unlink d5/d9/d26/f28 0 2024-03-22T07:15:58.632 INFO:tasks.workunit.client.1.smithi069.stdout:3/207: chown d5/d9/d26/d36/d41 942417 1 2024-03-22T07:15:58.634 INFO:tasks.workunit.client.1.smithi069.stdout:1/250: rmdir d1/d2a/d50 39 2024-03-22T07:15:58.634 INFO:tasks.workunit.client.1.smithi069.stdout:1/251: chown d1/d2a/d50/f40 11050 1 2024-03-22T07:15:58.636 INFO:tasks.workunit.client.0.smithi063.stdout:8/568: dread d0/d24/f74 [0,4194304] 0 2024-03-22T07:15:58.637 INFO:tasks.workunit.client.0.smithi063.stdout:8/569: chown d0/d1b/d28/d7b/d4a/c51 34 1 2024-03-22T07:15:58.638 INFO:tasks.workunit.client.1.smithi069.stdout:2/219: link d18/c32 d18/d4d/d34/c52 0 2024-03-22T07:15:58.638 INFO:tasks.workunit.client.1.smithi069.stdout:2/220: write d18/d4d/d25/f47 [249207,124003] 0 2024-03-22T07:15:58.642 INFO:tasks.workunit.client.1.smithi069.stdout:4/338: creat d6/dc/d1d/d31/d1f/d3f/f64 x:0 0 0 2024-03-22T07:15:58.644 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.644+0000 7fac63fff700 1 -- 172.21.15.63:0/3046674262 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fac44000bf0 con 0x7fac5006af90 2024-03-22T07:15:58.646 INFO:tasks.workunit.client.1.smithi069.stdout:0/282: rename d3/d7/d15/d1f/d55 to d3/d7/d5e 0 2024-03-22T07:15:58.647 INFO:tasks.workunit.client.1.smithi069.stdout:5/174: mknod d1/d7/c37 0 2024-03-22T07:15:58.647 INFO:tasks.workunit.client.1.smithi069.stdout:5/175: truncate d1/d7/f25 4411288 0 2024-03-22T07:15:58.648 INFO:tasks.workunit.client.1.smithi069.stdout:5/176: chown d1/d10/c16 184198 1 2024-03-22T07:15:58.648 INFO:tasks.workunit.client.1.smithi069.stdout:5/177: write d1/d7/d8/f27 [1385670,125165] 0 2024-03-22T07:15:58.649 INFO:tasks.workunit.client.1.smithi069.stdout:3/208: creat d5/d9/d26/d3f/d35/f45 x:0 0 0 2024-03-22T07:15:58.649 INFO:tasks.workunit.client.1.smithi069.stdout:3/209: fdatasync d5/d9/d26/d3f/d35/f37 0 2024-03-22T07:15:58.649 INFO:tasks.workunit.client.1.smithi069.stdout:3/210: chown d5/d9/d26/d36 6414 1 2024-03-22T07:15:58.650 INFO:tasks.workunit.client.0.smithi063.stdout:0/669: dwrite da/de/d60/d9b/f9f [0,4194304] 0 2024-03-22T07:15:58.651 INFO:tasks.workunit.client.0.smithi063.stdout:6/526: sync 2024-03-22T07:15:58.652 INFO:tasks.workunit.client.1.smithi069.stdout:1/252: mknod d1/d25/d41/d1e/c52 0 2024-03-22T07:15:58.652 INFO:tasks.workunit.client.1.smithi069.stdout:1/253: fsync d1/d13/f27 0 2024-03-22T07:15:58.654 INFO:tasks.workunit.client.1.smithi069.stdout:4/339: mkdir d6/dc/d1d/d31/d4a/d65 0 2024-03-22T07:15:58.654 INFO:tasks.workunit.client.1.smithi069.stdout:4/340: truncate d6/dc/d1d/d31/d1f/f5b 640450 0 2024-03-22T07:15:58.657 INFO:tasks.workunit.client.1.smithi069.stdout:4/341: fdatasync d6/d12/d18/f4f 0 2024-03-22T07:15:58.658 INFO:tasks.workunit.client.0.smithi063.stdout:6/527: creat d3/d15/d16/d60/d6d/fae x:0 0 0 2024-03-22T07:15:58.659 INFO:tasks.workunit.client.1.smithi069.stdout:0/283: rmdir d3/d7/d15/d29/d5a/d5b 0 2024-03-22T07:15:58.661 INFO:tasks.workunit.client.0.smithi063.stdout:4/434: dwrite d1/d5/d3c/f52 [0,4194304] 0 2024-03-22T07:15:58.661 INFO:tasks.workunit.client.1.smithi069.stdout:5/178: creat d1/d6/d26/d33/f38 x:0 0 0 2024-03-22T07:15:58.661 INFO:tasks.workunit.client.1.smithi069.stdout:5/179: chown d1/d10/c35 4 1 2024-03-22T07:15:58.662 INFO:tasks.workunit.client.1.smithi069.stdout:4/342: symlink d6/d12/l66 0 2024-03-22T07:15:58.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.663+0000 7fac5b7fe700 1 -- 172.21.15.63:0/3046674262 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fac44000bf0 con 0x7fac5006af90 2024-03-22T07:15:58.663 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:15:58.663 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (6m) 2m ago 7m 24.4M - 0.25.0 c8568f914cd2 2f8544438b7c 2024-03-22T07:15:58.663 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (8m) 2m ago 8m 6660k - 17.2.7-726-g9c4e0c34 837124da804f d6c6bbeb6904 2024-03-22T07:15:58.663 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (7m) 13s ago 7m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 30992f8aa88c 2024-03-22T07:15:58.664 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (6m) 2m ago 7m 85.4M - 9.4.7 954c08fa6188 f0b1d2a54439 2024-03-22T07:15:58.664 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (2m) 2m ago 2m 11.2M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:15:58.664 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (2m) 2m ago 2m 13.8M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:15:58.664 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (2m) 13s ago 2m 18.7M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:15:58.664 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (2m) 13s ago 2m 18.5M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:15:58.664 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:9283 running (9m) 2m ago 9m 480M - 17.2.7-726-g9c4e0c34 837124da804f 422b5e836153 2024-03-22T07:15:58.664 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283 running (33s) 13s ago 6m 54.2M - 19.0.0-1667-gdb0330b1 4f325f8440e9 5ad05e49e512 2024-03-22T07:15:58.664 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (9m) 2m ago 9m 49.2M 2048M 17.2.7-726-g9c4e0c34 837124da804f f92a5bb6caee 2024-03-22T07:15:58.664 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (6m) 13s ago 6m 34.3M 2048M 17.2.7-726-g9c4e0c34 837124da804f 95f6103f7603 2024-03-22T07:15:58.664 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (7m) 2m ago 7m 16.3M - 1.5.0 0da6a335fe13 47501603d3e8 2024-03-22T07:15:58.664 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (6m) 13s ago 6m 16.4M - 1.5.0 0da6a335fe13 49f4ca105fb9 2024-03-22T07:15:58.664 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (6m) 2m ago 6m 46.2M 4288M 17.2.7-726-g9c4e0c34 837124da804f 580f22188a10 2024-03-22T07:15:58.664 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (5m) 2m ago 5m 49.1M 4288M 17.2.7-726-g9c4e0c34 837124da804f fe1ace35c88a 2024-03-22T07:15:58.664 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (5m) 2m ago 5m 45.9M 4288M 17.2.7-726-g9c4e0c34 837124da804f 4cd995265dcf 2024-03-22T07:15:58.665 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (4m) 13s ago 4m 92.4M 5312M 17.2.7-726-g9c4e0c34 837124da804f 45c9a759b76d 2024-03-22T07:15:58.665 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (4m) 13s ago 4m 83.7M 5312M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:15:58.665 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (3m) 13s ago 3m 80.9M 5312M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:15:58.665 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (6m) 2m ago 7m 53.7M - 2.43.0 a07b618ecd1d 827638a5870a 2024-03-22T07:15:58.665 INFO:tasks.workunit.client.1.smithi069.stdout:5/180: mknod d1/d28/c39 0 2024-03-22T07:15:58.665 INFO:tasks.workunit.client.1.smithi069.stdout:8/299: dwrite d1/d1c/f52 [0,4194304] 0 2024-03-22T07:15:58.666 INFO:tasks.workunit.client.1.smithi069.stdout:4/343: symlink d6/dc/d23/d59/l67 0 2024-03-22T07:15:58.667 INFO:tasks.workunit.client.0.smithi063.stdout:4/435: mkdir d1/d5/dc/d95/d59/d9c 0 2024-03-22T07:15:58.667 INFO:tasks.workunit.client.0.smithi063.stdout:4/436: read d1/f21 [3210178,60130] 0 2024-03-22T07:15:58.667 INFO:tasks.workunit.client.0.smithi063.stdout:4/437: creat d1/d5/d43/d27/d2c/d4a/f9d x:0 0 0 2024-03-22T07:15:58.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.667+0000 7fac63fff700 1 -- 172.21.15.63:0/3046674262 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fac5006af90 msgr2=0x7fac5006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:58.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.667+0000 7fac63fff700 1 --2- 172.21.15.63:0/3046674262 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fac5006af90 0x7fac5006d440 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fac4c009b10 tx=0x7fac4c008040 comp rx=0 tx=0).stop 2024-03-22T07:15:58.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.667+0000 7fac63fff700 1 -- 172.21.15.63:0/3046674262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac5c09b1d0 msgr2=0x7fac5c0abd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:58.668 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.667+0000 7fac63fff700 1 --2- 172.21.15.63:0/3046674262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac5c09b1d0 0x7fac5c0abd90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fac64061610 tx=0x7fac640636e0 comp rx=0 tx=0).stop 2024-03-22T07:15:58.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.668+0000 7fac63fff700 1 -- 172.21.15.63:0/3046674262 shutdown_connections 2024-03-22T07:15:58.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.668+0000 7fac63fff700 1 --2- 172.21.15.63:0/3046674262 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fac5006af90 0x7fac5006d440 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:58.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.668+0000 7fac63fff700 1 --2- 172.21.15.63:0/3046674262 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fac5c09b1d0 0x7fac5c0abd90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:58.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.668+0000 7fac63fff700 1 --2- 172.21.15.63:0/3046674262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac5c09c480 0x7fac5c0ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:58.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.668+0000 7fac63fff700 1 -- 172.21.15.63:0/3046674262 >> 172.21.15.63:0/3046674262 conn(0x7fac5c00a360 msgr2=0x7fac5c00b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:58.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.668+0000 7fac63fff700 1 -- 172.21.15.63:0/3046674262 shutdown_connections 2024-03-22T07:15:58.669 INFO:tasks.workunit.client.0.smithi063.stdout:4/438: truncate d1/d5/d43/f2e 72935 0 2024-03-22T07:15:58.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.669+0000 7fac63fff700 1 -- 172.21.15.63:0/3046674262 wait complete. 2024-03-22T07:15:58.670 INFO:tasks.workunit.client.1.smithi069.stdout:8/300: unlink d1/d1c/d48/f59 0 2024-03-22T07:15:58.670 INFO:tasks.workunit.client.0.smithi063.stdout:4/439: symlink d1/d5/d3c/d8f/l9e 0 2024-03-22T07:15:58.672 INFO:tasks.workunit.client.0.smithi063.stdout:4/440: link d1/d5/dc/d95/d59/f87 d1/d5/d43/d27/d2c/f9f 0 2024-03-22T07:15:58.672 INFO:tasks.workunit.client.1.smithi069.stdout:8/301: creat d1/de/f5a x:0 0 0 2024-03-22T07:15:58.683 INFO:tasks.workunit.client.1.smithi069.stdout:3/211: write d5/d9/d26/d3f/f21 [1088547,121764] 0 2024-03-22T07:15:58.683 INFO:tasks.workunit.client.0.smithi063.stdout:8/570: dwrite d0/d1b/d28/d7b/d4a/f5a [0,4194304] 0 2024-03-22T07:15:58.687 INFO:tasks.workunit.client.1.smithi069.stdout:3/212: mkdir d5/d9/d23/d46 0 2024-03-22T07:15:58.687 INFO:tasks.workunit.client.1.smithi069.stdout:3/213: creat d5/d33/f47 x:0 0 0 2024-03-22T07:15:58.687 INFO:tasks.workunit.client.1.smithi069.stdout:3/214: dread - d5/d9/d26/d3f/d35/f37 zero size 2024-03-22T07:15:58.687 INFO:tasks.workunit.client.1.smithi069.stdout:3/215: creat d5/d9/f48 x:0 0 0 2024-03-22T07:15:58.687 INFO:tasks.workunit.client.1.smithi069.stdout:3/216: stat d5/d9/d26/d3f/f21 0 2024-03-22T07:15:58.689 INFO:tasks.workunit.client.0.smithi063.stdout:4/441: dread d1/d5/f9 [0,4194304] 0 2024-03-22T07:15:58.689 INFO:tasks.workunit.client.1.smithi069.stdout:8/302: write d1/de/f1a [1060164,129460] 0 2024-03-22T07:15:58.690 INFO:tasks.workunit.client.0.smithi063.stdout:4/442: link d1/d5/d43/d27/d2c/c7a d1/d5/d43/d27/ca0 0 2024-03-22T07:15:58.690 INFO:tasks.workunit.client.0.smithi063.stdout:4/443: write d1/d5/dc/f1d [8225038,71330] 0 2024-03-22T07:15:58.691 INFO:tasks.workunit.client.0.smithi063.stdout:4/444: mknod d1/d5/d43/d27/d2c/d3d/d54/d5f/d79/ca1 0 2024-03-22T07:15:58.697 INFO:tasks.workunit.client.1.smithi069.stdout:2/221: dwrite d18/d4d/f22 [0,4194304] 0 2024-03-22T07:15:58.700 INFO:tasks.workunit.client.1.smithi069.stdout:1/254: dwrite d1/d13/d36/f43 [0,4194304] 0 2024-03-22T07:15:58.700 INFO:tasks.workunit.client.1.smithi069.stdout:1/255: fdatasync d1/f3b 0 2024-03-22T07:15:58.700 INFO:tasks.workunit.client.1.smithi069.stdout:1/256: fsync d1/d13/f26 0 2024-03-22T07:15:58.701 INFO:tasks.workunit.client.1.smithi069.stdout:3/217: dread d5/d9/d26/d3f/f22 [4194304,4194304] 0 2024-03-22T07:15:58.701 INFO:tasks.workunit.client.1.smithi069.stdout:1/257: rename d1/d13/l1c to d1/d2a/l53 0 2024-03-22T07:15:58.701 INFO:tasks.workunit.client.1.smithi069.stdout:1/258: chown d1/d25/d41/d1e 0 1 2024-03-22T07:15:58.702 INFO:tasks.workunit.client.1.smithi069.stdout:1/259: creat d1/d13/d36/f54 x:0 0 0 2024-03-22T07:15:58.704 INFO:tasks.workunit.client.0.smithi063.stdout:0/670: dwrite da/de/d17/f2f [4194304,4194304] 0 2024-03-22T07:15:58.705 INFO:tasks.workunit.client.1.smithi069.stdout:5/181: dwrite d1/d6/f2e [0,4194304] 0 2024-03-22T07:15:58.705 INFO:tasks.workunit.client.1.smithi069.stdout:0/284: dwrite d3/d7/f49 [0,4194304] 0 2024-03-22T07:15:58.705 INFO:tasks.workunit.client.1.smithi069.stdout:0/285: chown d3/f39 1559 1 2024-03-22T07:15:58.705 INFO:tasks.workunit.client.0.smithi063.stdout:8/571: dread d0/f1a [0,4194304] 0 2024-03-22T07:15:58.706 INFO:tasks.workunit.client.0.smithi063.stdout:0/671: symlink da/de/d60/ld4 0 2024-03-22T07:15:58.707 INFO:tasks.workunit.client.1.smithi069.stdout:5/182: mkdir d1/d6/d26/d3a 0 2024-03-22T07:15:58.707 INFO:tasks.workunit.client.1.smithi069.stdout:3/218: dread d5/f6 [0,4194304] 0 2024-03-22T07:15:58.707 INFO:tasks.workunit.client.1.smithi069.stdout:3/219: readlink d5/d9/d26/l44 0 2024-03-22T07:15:58.707 INFO:tasks.workunit.client.1.smithi069.stdout:3/220: stat d5/l2f 0 2024-03-22T07:15:58.707 INFO:tasks.workunit.client.1.smithi069.stdout:3/221: dread - d5/f31 zero size 2024-03-22T07:15:58.708 INFO:tasks.workunit.client.1.smithi069.stdout:3/222: chown d5 254494 1 2024-03-22T07:15:58.710 INFO:tasks.workunit.client.1.smithi069.stdout:4/344: dwrite d6/dc/d1d/d31/d1f/f42 [0,4194304] 0 2024-03-22T07:15:58.710 INFO:tasks.workunit.client.1.smithi069.stdout:5/183: symlink d1/d7/l3b 0 2024-03-22T07:15:58.710 INFO:tasks.workunit.client.1.smithi069.stdout:5/184: read d1/d7/d8/f12 [3451597,117878] 0 2024-03-22T07:15:58.713 INFO:tasks.workunit.client.0.smithi063.stdout:8/572: rename d0/d1b/d28/d5f/d99 to d0/d26/d2d/d83/dbc 0 2024-03-22T07:15:58.713 INFO:tasks.workunit.client.1.smithi069.stdout:3/223: mkdir d5/d9/d26/d36/d41/d49 0 2024-03-22T07:15:58.714 INFO:tasks.workunit.client.0.smithi063.stdout:6/528: rmdir d3/d15/d16 39 2024-03-22T07:15:58.716 INFO:tasks.workunit.client.0.smithi063.stdout:4/445: dwrite d1/d5/d43/d27/d2c/f7c [0,4194304] 0 2024-03-22T07:15:58.717 INFO:tasks.workunit.client.1.smithi069.stdout:4/345: mkdir d6/d16/d68 0 2024-03-22T07:15:58.719 INFO:tasks.workunit.client.1.smithi069.stdout:5/185: mknod d1/d6/d26/d3a/c3c 0 2024-03-22T07:15:58.719 INFO:tasks.workunit.client.1.smithi069.stdout:5/186: write d1/d7/d8/f18 [988470,16704] 0 2024-03-22T07:15:58.719 INFO:tasks.workunit.client.1.smithi069.stdout:5/187: dread - d1/d7/d8/f15 zero size 2024-03-22T07:15:58.720 INFO:tasks.workunit.client.1.smithi069.stdout:3/224: creat d5/f4a x:0 0 0 2024-03-22T07:15:58.721 INFO:tasks.workunit.client.0.smithi063.stdout:4/446: dread d1/d5/dc/d95/f49 [0,4194304] 0 2024-03-22T07:15:58.721 INFO:tasks.workunit.client.0.smithi063.stdout:0/672: creat da/de/d17/d25/d95/d98/db3/fd5 x:0 0 0 2024-03-22T07:15:58.721 INFO:tasks.workunit.client.0.smithi063.stdout:0/673: write da/d49/f51 [902819,105225] 0 2024-03-22T07:15:58.722 INFO:tasks.workunit.client.1.smithi069.stdout:4/346: creat d6/dc/d1d/d31/d1f/d58/f69 x:0 0 0 2024-03-22T07:15:58.722 INFO:tasks.workunit.client.1.smithi069.stdout:4/347: creat d6/d12/d18/d21/f6a x:0 0 0 2024-03-22T07:15:58.723 INFO:tasks.workunit.client.0.smithi063.stdout:8/573: mkdir d0/d1b/d28/d3e/db2/dbd 0 2024-03-22T07:15:58.723 INFO:tasks.workunit.client.0.smithi063.stdout:8/574: dread - d0/d24/fb8 zero size 2024-03-22T07:15:58.723 INFO:tasks.workunit.client.0.smithi063.stdout:8/575: truncate d0/f19 4491185 0 2024-03-22T07:15:58.723 INFO:tasks.workunit.client.1.smithi069.stdout:5/188: truncate d1/d6/ff 297651 0 2024-03-22T07:15:58.723 INFO:tasks.workunit.client.1.smithi069.stdout:5/189: fdatasync d1/d7/f25 0 2024-03-22T07:15:58.724 INFO:tasks.workunit.client.0.smithi063.stdout:6/529: unlink d3/d6/l39 0 2024-03-22T07:15:58.725 INFO:tasks.workunit.client.0.smithi063.stdout:6/530: write d3/d15/d16/f9a [1013450,68327] 0 2024-03-22T07:15:58.725 INFO:tasks.workunit.client.0.smithi063.stdout:6/531: truncate d3/d15/d16/d60/d6d/d4c/d8b/fa4 908493 0 2024-03-22T07:15:58.725 INFO:tasks.workunit.client.0.smithi063.stdout:6/532: write d3/d76/f5b [202601,3058] 0 2024-03-22T07:15:58.729 INFO:tasks.workunit.client.0.smithi063.stdout:4/447: truncate d1/d5/d43/d27/d2c/d4a/f8b 2241478 0 2024-03-22T07:15:58.729 INFO:tasks.workunit.client.1.smithi069.stdout:8/303: dwrite d1/f14 [0,4194304] 0 2024-03-22T07:15:58.729 INFO:tasks.workunit.client.1.smithi069.stdout:8/304: readlink d1/de/d20/d43/d4e/l53 0 2024-03-22T07:15:58.730 INFO:tasks.workunit.client.0.smithi063.stdout:0/674: rmdir da/de/d17/d25/d95 39 2024-03-22T07:15:58.731 INFO:tasks.workunit.client.0.smithi063.stdout:8/576: mknod d0/d1b/d4f/cbe 0 2024-03-22T07:15:58.732 INFO:tasks.workunit.client.1.smithi069.stdout:0/286: dwrite d3/da/f52 [0,4194304] 0 2024-03-22T07:15:58.732 INFO:tasks.workunit.client.1.smithi069.stdout:1/260: dwrite d1/d2a/f32 [0,4194304] 0 2024-03-22T07:15:58.736 INFO:tasks.workunit.client.1.smithi069.stdout:8/305: mkdir d1/de/d20/d43/d4e/d5b 0 2024-03-22T07:15:58.736 INFO:tasks.workunit.client.1.smithi069.stdout:8/306: chown d1/d1c/f52 1694209 1 2024-03-22T07:15:58.737 INFO:tasks.workunit.client.1.smithi069.stdout:5/190: dread d1/d7/d8/f12 [0,4194304] 0 2024-03-22T07:15:58.738 INFO:tasks.workunit.client.1.smithi069.stdout:0/287: symlink d3/d7/d24/l5f 0 2024-03-22T07:15:58.740 INFO:tasks.workunit.client.0.smithi063.stdout:0/675: rmdir da/de/d17/d25/d3c/d5f 39 2024-03-22T07:15:58.741 INFO:tasks.workunit.client.0.smithi063.stdout:0/676: getdents da/d49 0 2024-03-22T07:15:58.741 INFO:tasks.workunit.client.0.smithi063.stdout:0/677: fsync da/de/d17/d25/d3c/d43/d76/d92/fbf 0 2024-03-22T07:15:58.741 INFO:tasks.workunit.client.0.smithi063.stdout:8/577: mkdir d0/d1b/d28/d3e/db2/dbf 0 2024-03-22T07:15:58.741 INFO:tasks.workunit.client.0.smithi063.stdout:4/448: getdents d1/d5/d43/d27 0 2024-03-22T07:15:58.742 INFO:tasks.workunit.client.1.smithi069.stdout:8/307: link d1/l4d d1/d1c/d48/l5c 0 2024-03-22T07:15:58.742 INFO:tasks.workunit.client.0.smithi063.stdout:0/678: fsync da/d49/f51 0 2024-03-22T07:15:58.742 INFO:tasks.workunit.client.0.smithi063.stdout:0/679: chown da/d49 92180157 1 2024-03-22T07:15:58.743 INFO:tasks.workunit.client.0.smithi063.stdout:8/578: mkdir d0/d26/d2d/d59/dc0 0 2024-03-22T07:15:58.743 INFO:tasks.workunit.client.0.smithi063.stdout:8/579: chown d0/d24/fb8 110 1 2024-03-22T07:15:58.743 INFO:tasks.workunit.client.0.smithi063.stdout:8/580: readlink d0/d26/d2d/d83/dbc/da1/lac 0 2024-03-22T07:15:58.746 INFO:tasks.workunit.client.0.smithi063.stdout:9/451: sync 2024-03-22T07:15:58.746 INFO:tasks.workunit.client.0.smithi063.stdout:0/680: rename da/d49/f63 to da/de/d17/d25/d95/fd6 0 2024-03-22T07:15:58.747 INFO:tasks.workunit.client.1.smithi069.stdout:0/288: creat d3/da/f60 x:0 0 0 2024-03-22T07:15:58.747 INFO:tasks.workunit.client.0.smithi063.stdout:4/449: mknod d1/d5/ca2 0 2024-03-22T07:15:58.749 INFO:tasks.workunit.client.1.smithi069.stdout:8/308: rename d1/d1c/d3e/f19 to d1/de/d20/d43/d4e/f5d 0 2024-03-22T07:15:58.749 INFO:tasks.workunit.client.1.smithi069.stdout:8/309: write d1/d1c/f38 [772084,82089] 0 2024-03-22T07:15:58.749 INFO:tasks.workunit.client.1.smithi069.stdout:8/310: write d1/de/f2d [672621,1777] 0 2024-03-22T07:15:58.749 INFO:tasks.workunit.client.0.smithi063.stdout:9/452: dread d8/d19/f35 [0,4194304] 0 2024-03-22T07:15:58.749 INFO:tasks.workunit.client.0.smithi063.stdout:9/453: chown d8/d17/f26 113 1 2024-03-22T07:15:58.749 INFO:tasks.workunit.client.0.smithi063.stdout:9/454: chown d8/d19/d20/d22/d56/f71 417216178 1 2024-03-22T07:15:58.749 INFO:tasks.workunit.client.0.smithi063.stdout:0/681: link da/de/d17/d25/d3c/d53/l6e da/de/d17/d25/d3c/d43/dc9/ld7 0 2024-03-22T07:15:58.750 INFO:tasks.workunit.client.0.smithi063.stdout:0/682: write da/de/d17/d25/f65 [1153200,111005] 0 2024-03-22T07:15:58.750 INFO:tasks.workunit.client.1.smithi069.stdout:5/191: rmdir d1/d6/d26/d33 39 2024-03-22T07:15:58.750 INFO:tasks.workunit.client.0.smithi063.stdout:4/450: mknod d1/d5/d43/d27/d2c/d4a/ca3 0 2024-03-22T07:15:58.751 INFO:tasks.workunit.client.0.smithi063.stdout:4/451: dread - d1/d5/d3c/d65/f73 zero size 2024-03-22T07:15:58.753 INFO:tasks.workunit.client.0.smithi063.stdout:9/455: symlink d8/d19/l8a 0 2024-03-22T07:15:58.753 INFO:tasks.workunit.client.1.smithi069.stdout:8/311: mkdir d1/d1c/d48/d5e 0 2024-03-22T07:15:58.755 INFO:tasks.workunit.client.0.smithi063.stdout:1/599: sync 2024-03-22T07:15:58.755 INFO:tasks.workunit.client.0.smithi063.stdout:3/566: sync 2024-03-22T07:15:58.755 INFO:tasks.workunit.client.0.smithi063.stdout:2/490: sync 2024-03-22T07:15:58.756 INFO:tasks.workunit.client.0.smithi063.stdout:5/483: sync 2024-03-22T07:15:58.756 INFO:tasks.workunit.client.0.smithi063.stdout:2/491: chown d0/de/d1b/d32/f49 1216 1 2024-03-22T07:15:58.756 INFO:tasks.workunit.client.0.smithi063.stdout:3/567: write da/d16/d26/f2a [3635148,34429] 0 2024-03-22T07:15:58.756 INFO:tasks.workunit.client.0.smithi063.stdout:3/568: chown da/c43 3259751 1 2024-03-22T07:15:58.756 INFO:tasks.workunit.client.0.smithi063.stdout:7/388: sync 2024-03-22T07:15:58.756 INFO:tasks.workunit.client.0.smithi063.stdout:7/389: write d5/d16/d1b/f80 [287774,55533] 0 2024-03-22T07:15:58.756 INFO:tasks.workunit.client.0.smithi063.stdout:7/390: rename d5/d16/d1b to d5/d16/d1b/d21/d62/d82 22 2024-03-22T07:15:58.756 INFO:tasks.workunit.client.1.smithi069.stdout:9/208: sync 2024-03-22T07:15:58.757 INFO:tasks.workunit.client.1.smithi069.stdout:3/225: getdents d5 0 2024-03-22T07:15:58.757 INFO:tasks.workunit.client.1.smithi069.stdout:3/226: fdatasync d5/d9/db/f30 0 2024-03-22T07:15:58.758 INFO:tasks.workunit.client.0.smithi063.stdout:9/456: creat d8/d19/d39/f8b x:0 0 0 2024-03-22T07:15:58.758 INFO:tasks.workunit.client.0.smithi063.stdout:0/683: getdents da 0 2024-03-22T07:15:58.759 INFO:tasks.workunit.client.0.smithi063.stdout:0/684: fdatasync da/d49/f6d 0 2024-03-22T07:15:58.759 INFO:tasks.workunit.client.0.smithi063.stdout:0/685: write da/de/f62 [4561147,70335] 0 2024-03-22T07:15:58.759 INFO:tasks.workunit.client.0.smithi063.stdout:0/686: fsync da/f54 0 2024-03-22T07:15:58.759 INFO:tasks.workunit.client.1.smithi069.stdout:0/289: dread d3/d7/d15/d1f/f26 [0,4194304] 0 2024-03-22T07:15:58.760 INFO:tasks.workunit.client.1.smithi069.stdout:0/290: readlink d3/d7/d17/l2a 0 2024-03-22T07:15:58.760 INFO:tasks.workunit.client.1.smithi069.stdout:0/291: chown d3/da/f3b 1999 1 2024-03-22T07:15:58.763 INFO:tasks.workunit.client.0.smithi063.stdout:1/600: truncate d4/df/d13/d39/f5f 1391446 0 2024-03-22T07:15:58.763 INFO:tasks.workunit.client.0.smithi063.stdout:7/391: truncate d5/d16/d1b/d21/d62/d73/d1d/f51 1242423 0 2024-03-22T07:15:58.764 INFO:tasks.workunit.client.1.smithi069.stdout:9/209: getdents d2 0 2024-03-22T07:15:58.764 INFO:tasks.workunit.client.0.smithi063.stdout:9/457: mknod d8/d15/d63/c8c 0 2024-03-22T07:15:58.765 INFO:tasks.workunit.client.1.smithi069.stdout:9/210: rmdir d2 39 2024-03-22T07:15:58.769 INFO:tasks.workunit.client.0.smithi063.stdout:1/601: link d4/l35 d4/d5a/d91/lcb 0 2024-03-22T07:15:58.770 INFO:tasks.workunit.client.1.smithi069.stdout:9/211: rename d2/dd/d3c/f24 to d2/dd/d3c/d2e/f42 0 2024-03-22T07:15:58.770 INFO:tasks.workunit.client.1.smithi069.stdout:9/212: read d2/d31/f40 [1764299,129384] 0 2024-03-22T07:15:58.770 INFO:tasks.workunit.client.1.smithi069.stdout:9/213: creat d2/dd/d3c/f43 x:0 0 0 2024-03-22T07:15:58.770 INFO:tasks.workunit.client.1.smithi069.stdout:9/214: chown d2/dd/d3c/d2e/f32 129 1 2024-03-22T07:15:58.770 INFO:tasks.workunit.client.1.smithi069.stdout:1/261: dwrite d1/fa [0,4194304] 0 2024-03-22T07:15:58.772 INFO:tasks.workunit.client.1.smithi069.stdout:6/176: sync 2024-03-22T07:15:58.772 INFO:tasks.workunit.client.1.smithi069.stdout:6/177: fdatasync d4/dd/d12/f32 0 2024-03-22T07:15:58.773 INFO:tasks.workunit.client.0.smithi063.stdout:0/687: getdents da/de/d17/d25/d3c/d53/d89 0 2024-03-22T07:15:58.773 INFO:tasks.workunit.client.0.smithi063.stdout:0/688: creat da/de/d17/d25/d3c/d43/d80/fd8 x:0 0 0 2024-03-22T07:15:58.773 INFO:tasks.workunit.client.0.smithi063.stdout:1/602: creat d4/df/d13/d2b/d6c/da8/fcc x:0 0 0 2024-03-22T07:15:58.773 INFO:tasks.workunit.client.0.smithi063.stdout:1/603: creat d4/d5a/fcd x:0 0 0 2024-03-22T07:15:58.773 INFO:tasks.workunit.client.0.smithi063.stdout:1/604: creat d4/df/d13/d2b/d3a/fce x:0 0 0 2024-03-22T07:15:58.774 INFO:tasks.workunit.client.1.smithi069.stdout:1/262: mkdir d1/d2a/d50/d4c/d55 0 2024-03-22T07:15:58.775 INFO:tasks.workunit.client.1.smithi069.stdout:0/292: write d3/d7/d15/d1f/f21 [5061770,129420] 0 2024-03-22T07:15:58.775 INFO:tasks.workunit.client.1.smithi069.stdout:0/293: fdatasync d3/d7/d15/f20 0 2024-03-22T07:15:58.775 INFO:tasks.workunit.client.1.smithi069.stdout:0/294: chown d3/da/f52 843225 1 2024-03-22T07:15:58.775 INFO:tasks.workunit.client.1.smithi069.stdout:8/312: write d1/fd [2391040,95831] 0 2024-03-22T07:15:58.777 INFO:tasks.workunit.client.0.smithi063.stdout:9/458: write d8/d19/d20/f34 [1421511,81647] 0 2024-03-22T07:15:58.778 INFO:tasks.workunit.client.0.smithi063.stdout:1/605: symlink d4/df/d13/d2b/d6c/d85/d9e/lcf 0 2024-03-22T07:15:58.779 INFO:tasks.workunit.client.0.smithi063.stdout:1/606: dread - d4/f8c zero size 2024-03-22T07:15:58.779 INFO:tasks.workunit.client.1.smithi069.stdout:8/313: mknod d1/de/c5f 0 2024-03-22T07:15:58.779 INFO:tasks.workunit.client.1.smithi069.stdout:8/314: fsync d1/d1c/d3e/f32 0 2024-03-22T07:15:58.779 INFO:tasks.workunit.client.1.smithi069.stdout:0/295: getdents d3/d7/d15/d1f 0 2024-03-22T07:15:58.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:58 smithi069 ceph-mon[36441]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:58.782 INFO:tasks.workunit.client.1.smithi069.stdout:8/315: mknod d1/de/d20/d43/c60 0 2024-03-22T07:15:58.782 INFO:tasks.workunit.client.1.smithi069.stdout:8/316: write d1/f29 [1440947,108476] 0 2024-03-22T07:15:58.782 INFO:tasks.workunit.client.1.smithi069.stdout:8/317: creat d1/d1c/d3e/f61 x:0 0 0 2024-03-22T07:15:58.782 INFO:tasks.workunit.client.1.smithi069.stdout:0/296: truncate d3/d7/d15/d1f/f41 3751605 0 2024-03-22T07:15:58.783 INFO:tasks.workunit.client.0.smithi063.stdout:9/459: mknod d8/d19/d20/d22/d29/d31/d4b/c8d 0 2024-03-22T07:15:58.783 INFO:tasks.workunit.client.0.smithi063.stdout:1/607: creat d4/df/d13/d2b/d6c/d85/d9e/fd0 x:0 0 0 2024-03-22T07:15:58.784 INFO:tasks.workunit.client.0.smithi063.stdout:6/533: dwrite d3/d15/f85 [0,4194304] 0 2024-03-22T07:15:58.786 INFO:tasks.workunit.client.0.smithi063.stdout:1/608: link d4/d5a/lc9 d4/df/d13/ld1 0 2024-03-22T07:15:58.788 INFO:tasks.workunit.client.1.smithi069.stdout:0/297: rename d3/d7/d15/d1f/f21 to d3/d7/d5e/f61 0 2024-03-22T07:15:58.788 INFO:tasks.workunit.client.1.smithi069.stdout:0/298: write d3/d7/d5e/f61 [4870242,53885] 0 2024-03-22T07:15:58.789 INFO:tasks.workunit.client.0.smithi063.stdout:1/609: write d4/f1f [4314776,98089] 0 2024-03-22T07:15:58.789 INFO:tasks.workunit.client.1.smithi069.stdout:0/299: read d3/d7/d17/f3f [702053,106810] 0 2024-03-22T07:15:58.790 INFO:tasks.workunit.client.0.smithi063.stdout:4/452: dwrite d1/d5/dc/ff [0,4194304] 0 2024-03-22T07:15:58.790 INFO:tasks.workunit.client.0.smithi063.stdout:1/610: rename d4/f76 to d4/d71/d88/da1/fd2 0 2024-03-22T07:15:58.791 INFO:tasks.workunit.client.1.smithi069.stdout:9/215: dread d2/dd/f10 [0,4194304] 0 2024-03-22T07:15:58.791 INFO:tasks.workunit.client.1.smithi069.stdout:9/216: dread - d2/dd/d3c/f35 zero size 2024-03-22T07:15:58.791 INFO:tasks.workunit.client.1.smithi069.stdout:0/300: rmdir d3/d7/d15/d29 39 2024-03-22T07:15:58.793 INFO:tasks.workunit.client.1.smithi069.stdout:9/217: symlink d2/dd/d3c/l44 0 2024-03-22T07:15:58.793 INFO:tasks.workunit.client.0.smithi063.stdout:4/453: rename d1/d5/d93 to d1/d5/dc/d95/da4 0 2024-03-22T07:15:58.793 INFO:tasks.workunit.client.1.smithi069.stdout:8/318: dread d1/f14 [0,4194304] 0 2024-03-22T07:15:58.794 INFO:tasks.workunit.client.1.smithi069.stdout:8/319: chown d1/de/d20/d43/d4e 130738916 1 2024-03-22T07:15:58.794 INFO:tasks.workunit.client.1.smithi069.stdout:8/320: chown d1/d1c/f47 93115535 1 2024-03-22T07:15:58.795 INFO:tasks.workunit.client.0.smithi063.stdout:4/454: unlink d1/d5/d43/d27/d2c/f7c 0 2024-03-22T07:15:58.795 INFO:tasks.workunit.client.0.smithi063.stdout:4/455: chown d1/f45 2 1 2024-03-22T07:15:58.795 INFO:tasks.workunit.client.0.smithi063.stdout:1/611: dread d4/df/d13/d2b/f84 [0,4194304] 0 2024-03-22T07:15:58.796 INFO:tasks.workunit.client.1.smithi069.stdout:0/301: mknod d3/da/c62 0 2024-03-22T07:15:58.796 INFO:tasks.workunit.client.1.smithi069.stdout:0/302: chown d3/d7/d15/d1f/l2f 8976633 1 2024-03-22T07:15:58.799 INFO:tasks.workunit.client.1.smithi069.stdout:8/321: symlink d1/d1c/d48/l62 0 2024-03-22T07:15:58.802 INFO:tasks.workunit.client.1.smithi069.stdout:6/178: read d4/f6 [982542,88985] 0 2024-03-22T07:15:58.802 INFO:tasks.workunit.client.1.smithi069.stdout:0/303: unlink d3/d7/d15/f4f 0 2024-03-22T07:15:58.802 INFO:tasks.workunit.client.1.smithi069.stdout:0/304: truncate d3/d7/d15/d29/f30 2916703 0 2024-03-22T07:15:58.803 INFO:tasks.workunit.client.1.smithi069.stdout:0/305: unlink d3/d7/d15/f42 0 2024-03-22T07:15:58.805 INFO:tasks.workunit.client.1.smithi069.stdout:6/179: dread d4/f14 [0,4194304] 0 2024-03-22T07:15:58.815 INFO:tasks.workunit.client.1.smithi069.stdout:5/192: dwrite d1/d6/d26/d33/f38 [0,4194304] 0 2024-03-22T07:15:58.817 INFO:tasks.workunit.client.0.smithi063.stdout:2/492: dwrite d0/f9 [0,4194304] 0 2024-03-22T07:15:58.818 INFO:tasks.workunit.client.0.smithi063.stdout:4/456: write d1/d5/dc/f10 [112431,52272] 0 2024-03-22T07:15:58.818 INFO:tasks.workunit.client.0.smithi063.stdout:8/581: dwrite d0/d24/d33/d4c/f94 [0,4194304] 0 2024-03-22T07:15:58.818 INFO:tasks.workunit.client.0.smithi063.stdout:8/582: chown d0/d1f/f40 411232 1 2024-03-22T07:15:58.818 INFO:tasks.workunit.client.0.smithi063.stdout:8/583: dread - d0/d24/d5b/f95 zero size 2024-03-22T07:15:58.818 INFO:tasks.workunit.client.0.smithi063.stdout:8/584: write d0/d1b/f84 [1185303,155] 0 2024-03-22T07:15:58.819 INFO:tasks.workunit.client.1.smithi069.stdout:3/227: dwrite d5/f31 [0,4194304] 0 2024-03-22T07:15:58.824 INFO:tasks.workunit.client.0.smithi063.stdout:8/585: truncate d0/d1b/d4f/f79 3437460 0 2024-03-22T07:15:58.827 INFO:tasks.workunit.client.0.smithi063.stdout:8/586: rename d0/d1b/d28/c2f to d0/d24/cc1 0 2024-03-22T07:15:58.836 INFO:tasks.workunit.client.0.smithi063.stdout:2/493: read d0/de/d12/d20/f47 [3634854,4019] 0 2024-03-22T07:15:58.836 INFO:tasks.workunit.client.0.smithi063.stdout:4/457: write d1/d5/d43/f14 [1239420,118068] 0 2024-03-22T07:15:58.836 INFO:tasks.workunit.client.0.smithi063.stdout:4/458: truncate d1/f45 1438638 0 2024-03-22T07:15:58.836 INFO:tasks.workunit.client.0.smithi063.stdout:5/484: dwrite d8/d13/d28/f73 [4194304,4194304] 0 2024-03-22T07:15:58.836 INFO:tasks.workunit.client.0.smithi063.stdout:5/485: stat d8/f25 0 2024-03-22T07:15:58.836 INFO:tasks.workunit.client.0.smithi063.stdout:5/486: truncate d8/d13/d1e/d31/d64/d65/f7c 1017539 0 2024-03-22T07:15:58.837 INFO:tasks.workunit.client.0.smithi063.stdout:0/689: dwrite da/de/d60/d56/d8b/fb9 [0,4194304] 0 2024-03-22T07:15:58.838 INFO:tasks.workunit.client.0.smithi063.stdout:4/459: symlink d1/d5/dc/d95/d59/la5 0 2024-03-22T07:15:58.838 INFO:tasks.workunit.client.0.smithi063.stdout:4/460: creat d1/d5/d43/d27/d2c/d3d/d54/fa6 x:0 0 0 2024-03-22T07:15:58.838 INFO:tasks.workunit.client.0.smithi063.stdout:4/461: creat d1/d5/dc/fa7 x:0 0 0 2024-03-22T07:15:58.844 INFO:tasks.workunit.client.0.smithi063.stdout:4/462: creat d1/d5/d43/d27/d2c/d4a/d80/fa8 x:0 0 0 2024-03-22T07:15:58.844 INFO:tasks.workunit.client.0.smithi063.stdout:4/463: write d1/d5/dc/d95/d59/f87 [939830,122081] 0 2024-03-22T07:15:58.844 INFO:tasks.workunit.client.0.smithi063.stdout:4/464: truncate d1/d5/dc/f41 1611864 0 2024-03-22T07:15:58.844 INFO:tasks.workunit.client.0.smithi063.stdout:0/690: rmdir da/de/d67 39 2024-03-22T07:15:58.846 INFO:tasks.workunit.client.0.smithi063.stdout:0/691: mknod da/de/d17/d25/d3c/d64/cd9 0 2024-03-22T07:15:58.846 INFO:tasks.workunit.client.0.smithi063.stdout:0/692: creat da/de/d17/d25/d3c/d43/d80/fda x:0 0 0 2024-03-22T07:15:58.849 INFO:tasks.workunit.client.0.smithi063.stdout:0/693: unlink da/de/l35 0 2024-03-22T07:15:58.849 INFO:tasks.workunit.client.0.smithi063.stdout:0/694: dread - da/de/d60/fd2 zero size 2024-03-22T07:15:58.850 INFO:tasks.workunit.client.0.smithi063.stdout:0/695: chown da/de/d60/d9b 63 1 2024-03-22T07:15:58.850 INFO:tasks.workunit.client.1.smithi069.stdout:1/263: dwrite d1/f48 [0,4194304] 0 2024-03-22T07:15:58.853 INFO:tasks.workunit.client.0.smithi063.stdout:6/534: dwrite d3/d6/d35/d3a/f40 [0,4194304] 0 2024-03-22T07:15:58.853 INFO:tasks.workunit.client.0.smithi063.stdout:0/696: rmdir da/dca 0 2024-03-22T07:15:58.853 INFO:tasks.workunit.client.0.smithi063.stdout:8/587: dread d0/d24/d5b/f44 [0,4194304] 0 2024-03-22T07:15:58.854 INFO:tasks.workunit.client.0.smithi063.stdout:8/588: chown d0/d1b/d28/d7b/d4a/l9d 0 1 2024-03-22T07:15:58.854 INFO:tasks.workunit.client.0.smithi063.stdout:0/697: write da/de/d17/d25/d3c/d43/d80/fda [453435,127166] 0 2024-03-22T07:15:58.854 INFO:tasks.workunit.client.0.smithi063.stdout:8/589: fdatasync d0/f64 0 2024-03-22T07:15:58.854 INFO:tasks.workunit.client.0.smithi063.stdout:8/590: read d0/f62 [5016950,25998] 0 2024-03-22T07:15:58.854 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:58 smithi063 ceph-mon[29200]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:58.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.854+0000 7f89f5d94700 1 -- 172.21.15.63:0/3134995845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89f00fcf80 msgr2=0x7f89f00fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:58.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.854+0000 7f89f5d94700 1 --2- 172.21.15.63:0/3134995845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89f00fcf80 0x7f89f00fd3f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f89e4004940 tx=0x7f89e4009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:58.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.854+0000 7f89f5d94700 1 -- 172.21.15.63:0/3134995845 shutdown_connections 2024-03-22T07:15:58.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.854+0000 7f89f5d94700 1 --2- 172.21.15.63:0/3134995845 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89f00fcf80 0x7f89f00fd3f0 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f89e4004940 tx=0x7f89e4009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:58.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.854+0000 7f89f5d94700 1 --2- 172.21.15.63:0/3134995845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89f00fbcd0 0x7f89f00fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:58.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.854+0000 7f89f5d94700 1 -- 172.21.15.63:0/3134995845 >> 172.21.15.63:0/3134995845 conn(0x7f89f00f7430 msgr2=0x7f89f00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:58.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.855+0000 7f89f5d94700 1 -- 172.21.15.63:0/3134995845 shutdown_connections 2024-03-22T07:15:58.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.855+0000 7f89f5d94700 1 -- 172.21.15.63:0/3134995845 wait complete. 2024-03-22T07:15:58.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.855+0000 7f89f5d94700 1 Processor -- start 2024-03-22T07:15:58.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.856+0000 7f89f5d94700 1 -- start start 2024-03-22T07:15:58.856 INFO:tasks.workunit.client.0.smithi063.stdout:6/535: readlink d3/l30 0 2024-03-22T07:15:58.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.856+0000 7f89f5d94700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89f00fbcd0 0x7f89f010fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:58.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.856+0000 7f89f5d94700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89f01103a0 0x7f89f0111110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:58.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.856+0000 7f89f5d94700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f0110860 con 0x7f89f00fbcd0 2024-03-22T07:15:58.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.857+0000 7f89f5d94700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f01109d0 con 0x7f89f01103a0 2024-03-22T07:15:58.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.857+0000 7f89ef7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89f00fbcd0 0x7f89f010fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:58.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.857+0000 7f89ef7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89f00fbcd0 0x7f89f010fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39844/0 (socket says 172.21.15.63:39844) 2024-03-22T07:15:58.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.857+0000 7f89ef7fe700 1 -- 172.21.15.63:0/950638700 learned_addr learned my addr 172.21.15.63:0/950638700 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:15:58.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.857+0000 7f89ef7fe700 1 -- 172.21.15.63:0/950638700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89f01103a0 msgr2=0x7f89f0111110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:58.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.857+0000 7f89ef7fe700 1 --2- 172.21.15.63:0/950638700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89f01103a0 0x7f89f0111110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:58.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.857+0000 7f89ef7fe700 1 -- 172.21.15.63:0/950638700 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4005040 con 0x7f89f00fbcd0 2024-03-22T07:15:58.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.858+0000 7f89ef7fe700 1 --2- 172.21.15.63:0/950638700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89f00fbcd0 0x7f89f010fe60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f89e000c8c0 tx=0x7f89e00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:58.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.859+0000 7f89d7fff700 1 -- 172.21.15.63:0/950638700 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89e000a980 con 0x7f89f00fbcd0 2024-03-22T07:15:58.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.859+0000 7f89f5d94700 1 -- 172.21.15.63:0/950638700 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f01117f0 con 0x7f89f00fbcd0 2024-03-22T07:15:58.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.859+0000 7f89d7fff700 1 -- 172.21.15.63:0/950638700 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89e000ce60 con 0x7f89f00fbcd0 2024-03-22T07:15:58.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.859+0000 7f89d7fff700 1 -- 172.21.15.63:0/950638700 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89e00115e0 con 0x7f89f00fbcd0 2024-03-22T07:15:58.859 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.859+0000 7f89f5d94700 1 -- 172.21.15.63:0/950638700 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f0111db0 con 0x7f89f00fbcd0 2024-03-22T07:15:58.862 INFO:tasks.workunit.client.1.smithi069.stdout:1/264: creat d1/d2a/d50/d4c/d55/f56 x:0 0 0 2024-03-22T07:15:58.862 INFO:tasks.workunit.client.1.smithi069.stdout:1/265: creat d1/d2a/f57 x:0 0 0 2024-03-22T07:15:58.862 INFO:tasks.workunit.client.1.smithi069.stdout:1/266: creat d1/d13/f58 x:0 0 0 2024-03-22T07:15:58.862 INFO:tasks.workunit.client.1.smithi069.stdout:1/267: truncate d1/d13/f2d 2090709 0 2024-03-22T07:15:58.862 INFO:tasks.workunit.client.1.smithi069.stdout:1/268: symlink d1/d25/d41/l59 0 2024-03-22T07:15:58.863 INFO:tasks.workunit.client.0.smithi063.stdout:8/591: unlink d0/d26/d2d/d59/lb4 0 2024-03-22T07:15:58.863 INFO:tasks.workunit.client.0.smithi063.stdout:8/592: chown d0/d1b/c9a 26425 1 2024-03-22T07:15:58.863 INFO:tasks.workunit.client.0.smithi063.stdout:8/593: fsync d0/d24/d33/d4c/f7d 0 2024-03-22T07:15:58.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.860+0000 7f89f5d94700 1 -- 172.21.15.63:0/950638700 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f005f5f0 con 0x7f89f00fbcd0 2024-03-22T07:15:58.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.861+0000 7f89d7fff700 1 -- 172.21.15.63:0/950638700 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 20) v1 ==== 88751+0+0 (secure 0 0 0) 0x7f89e0012400 con 0x7f89f00fbcd0 2024-03-22T07:15:58.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.862+0000 7f89d7fff700 1 --2- 172.21.15.63:0/950638700 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f89d806b060 0x7f89d806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:58.864 INFO:tasks.workunit.client.0.smithi063.stdout:8/594: creat d0/d1b/d28/d3e/fc2 x:0 0 0 2024-03-22T07:15:58.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.862+0000 7f89d7fff700 1 -- 172.21.15.63:0/950638700 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f89e0089860 con 0x7f89f00fbcd0 2024-03-22T07:15:58.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.862+0000 7f89eeffd700 1 --2- 172.21.15.63:0/950638700 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f89d806b060 0x7f89d806d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:58.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.863+0000 7f89eeffd700 1 --2- 172.21.15.63:0/950638700 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f89d806b060 0x7f89d806d510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009fd0 tx=0x7f89e4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:58.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:58.868+0000 7f89d7fff700 1 -- 172.21.15.63:0/950638700 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7f89e0055b20 con 0x7f89f00fbcd0 2024-03-22T07:15:58.875 INFO:tasks.workunit.client.1.smithi069.stdout:3/228: dwrite d5/d9/db/f3d [0,4194304] 0 2024-03-22T07:15:58.875 INFO:tasks.workunit.client.1.smithi069.stdout:3/229: creat d5/d9/f4b x:0 0 0 2024-03-22T07:15:58.875 INFO:tasks.workunit.client.1.smithi069.stdout:3/230: write d5/d9/f4b [42178,52510] 0 2024-03-22T07:15:58.876 INFO:tasks.workunit.client.0.smithi063.stdout:4/465: dwrite d1/f21 [0,4194304] 0 2024-03-22T07:15:58.876 INFO:tasks.workunit.client.0.smithi063.stdout:4/466: truncate d1/d5/dc/d95/f60 290676 0 2024-03-22T07:15:58.876 INFO:tasks.workunit.client.1.smithi069.stdout:9/218: dwrite d2/dd/d3c/f20 [0,4194304] 0 2024-03-22T07:15:58.877 INFO:tasks.workunit.client.1.smithi069.stdout:8/322: dwrite d1/de/d20/f4f [0,4194304] 0 2024-03-22T07:15:58.877 INFO:tasks.workunit.client.1.smithi069.stdout:5/193: dwrite d1/d6/f1e [0,4194304] 0 2024-03-22T07:15:58.877 INFO:tasks.workunit.client.0.smithi063.stdout:6/536: read d3/d76/fa1 [99240,23553] 0 2024-03-22T07:15:58.879 INFO:tasks.workunit.client.1.smithi069.stdout:3/231: creat d5/d9/d26/f4c x:0 0 0 2024-03-22T07:15:58.879 INFO:tasks.workunit.client.1.smithi069.stdout:3/232: dread - d5/d9/f48 zero size 2024-03-22T07:15:58.880 INFO:tasks.workunit.client.0.smithi063.stdout:1/612: dwrite d4/df/fbe [0,4194304] 0 2024-03-22T07:15:58.880 INFO:tasks.workunit.client.0.smithi063.stdout:5/487: dwrite d8/d13/d1e/d3c/f49 [0,4194304] 0 2024-03-22T07:15:58.880 INFO:tasks.workunit.client.0.smithi063.stdout:2/494: dwrite d0/de/d58/f7b [4194304,4194304] 0 2024-03-22T07:15:58.880 INFO:tasks.workunit.client.0.smithi063.stdout:2/495: dread - d0/de/d58/f7d zero size 2024-03-22T07:15:58.880 INFO:tasks.workunit.client.0.smithi063.stdout:8/595: dread d0/d26/f27 [0,4194304] 0 2024-03-22T07:15:58.882 INFO:tasks.workunit.client.0.smithi063.stdout:4/467: dread d1/d5/d43/f31 [0,4194304] 0 2024-03-22T07:15:58.883 INFO:tasks.workunit.client.0.smithi063.stdout:1/613: dread d4/df/d13/d2b/d6c/d85/d9e/fa5 [0,4194304] 0 2024-03-22T07:15:58.883 INFO:tasks.workunit.client.0.smithi063.stdout:8/596: dread d0/d24/d5b/f60 [0,4194304] 0 2024-03-22T07:15:58.883 INFO:tasks.workunit.client.0.smithi063.stdout:8/597: stat d0/d24/d33/d4c/f7d 0 2024-03-22T07:15:58.887 INFO:tasks.workunit.client.1.smithi069.stdout:8/323: creat d1/d1c/d48/d5e/f63 x:0 0 0 2024-03-22T07:15:58.887 INFO:tasks.workunit.client.1.smithi069.stdout:8/324: readlink d1/l10 0 2024-03-22T07:15:58.887 INFO:tasks.workunit.client.1.smithi069.stdout:8/325: creat d1/d1c/d48/f64 x:0 0 0 2024-03-22T07:15:58.889 INFO:tasks.workunit.client.1.smithi069.stdout:5/194: mkdir d1/d7/d3d 0 2024-03-22T07:15:58.889 INFO:tasks.workunit.client.1.smithi069.stdout:5/195: chown d1/d7/c37 30 1 2024-03-22T07:15:58.890 INFO:tasks.workunit.client.1.smithi069.stdout:5/196: fsync d1/d7/d8/f24 0 2024-03-22T07:15:58.890 INFO:tasks.workunit.client.1.smithi069.stdout:5/197: write d1/d7/d8/f30 [249680,15918] 0 2024-03-22T07:15:58.890 INFO:tasks.workunit.client.1.smithi069.stdout:8/326: write d1/d1c/d48/d56/f57 [77859,81727] 0 2024-03-22T07:15:58.890 INFO:tasks.workunit.client.1.smithi069.stdout:8/327: write d1/d1c/d48/d56/f57 [970387,69414] 0 2024-03-22T07:15:58.894 INFO:tasks.workunit.client.0.smithi063.stdout:2/496: read d0/f51 [3026610,64072] 0 2024-03-22T07:15:58.894 INFO:tasks.workunit.client.1.smithi069.stdout:3/233: link d5/d9/db/d3b/l40 d5/l4d 0 2024-03-22T07:15:58.896 INFO:tasks.workunit.client.1.smithi069.stdout:9/219: getdents d2/dd 0 2024-03-22T07:15:58.896 INFO:tasks.workunit.client.1.smithi069.stdout:9/220: truncate d2/dd/d3c/f43 442739 0 2024-03-22T07:15:58.896 INFO:tasks.workunit.client.1.smithi069.stdout:3/234: write d5/d33/f39 [143171,80459] 0 2024-03-22T07:15:58.897 INFO:tasks.workunit.client.1.smithi069.stdout:3/235: truncate d5/d9/f3a 628323 0 2024-03-22T07:15:58.897 INFO:tasks.workunit.client.1.smithi069.stdout:3/236: write d5/f31 [4416391,112338] 0 2024-03-22T07:15:58.897 INFO:tasks.workunit.client.1.smithi069.stdout:3/237: chown d5/d9/d23/d46 0 1 2024-03-22T07:15:58.897 INFO:tasks.workunit.client.0.smithi063.stdout:1/614: dread d4/d14/f21 [0,4194304] 0 2024-03-22T07:15:58.898 INFO:tasks.workunit.client.1.smithi069.stdout:5/198: mkdir d1/d6/d26/d3e 0 2024-03-22T07:15:58.901 INFO:tasks.workunit.client.1.smithi069.stdout:8/328: creat d1/de/d20/f65 x:0 0 0 2024-03-22T07:15:58.902 INFO:tasks.workunit.client.1.smithi069.stdout:9/221: creat d2/d31/f45 x:0 0 0 2024-03-22T07:15:58.902 INFO:tasks.workunit.client.1.smithi069.stdout:9/222: write d2/d31/f45 [227269,127649] 0 2024-03-22T07:15:58.903 INFO:tasks.workunit.client.1.smithi069.stdout:9/223: stat d2/f5 0 2024-03-22T07:15:58.905 INFO:tasks.workunit.client.1.smithi069.stdout:3/238: rename d5/d9/d26/d36/d41/f43 to d5/f4e 0 2024-03-22T07:15:58.907 INFO:tasks.workunit.client.1.smithi069.stdout:5/199: mkdir d1/d7/d3d/d3f 0 2024-03-22T07:15:58.908 INFO:tasks.workunit.client.1.smithi069.stdout:5/200: creat d1/d7/f40 x:0 0 0 2024-03-22T07:15:58.908 INFO:tasks.workunit.client.0.smithi063.stdout:2/497: write d0/f8 [148413,7584] 0 2024-03-22T07:15:58.909 INFO:tasks.workunit.client.1.smithi069.stdout:1/269: fsync d1/d2a/f57 0 2024-03-22T07:15:58.909 INFO:tasks.workunit.client.1.smithi069.stdout:1/270: write d1/d13/f2d [639057,65906] 0 2024-03-22T07:15:58.909 INFO:tasks.workunit.client.1.smithi069.stdout:1/271: write d1/d13/f27 [4707806,123901] 0 2024-03-22T07:15:58.912 INFO:tasks.workunit.client.0.smithi063.stdout:4/468: truncate d1/d5/f20 4943542 0 2024-03-22T07:15:58.912 INFO:tasks.workunit.client.1.smithi069.stdout:7/302: sync 2024-03-22T07:15:58.914 INFO:tasks.workunit.client.0.smithi063.stdout:0/698: getdents da 0 2024-03-22T07:15:58.914 INFO:tasks.workunit.client.0.smithi063.stdout:1/615: write d4/d8b/f97 [1484299,97020] 0 2024-03-22T07:15:58.914 INFO:tasks.workunit.client.1.smithi069.stdout:7/303: unlink d9/db/d48/d50/d57/d44/d53/c23 0 2024-03-22T07:15:58.915 INFO:tasks.workunit.client.0.smithi063.stdout:1/616: chown d4/df/l68 0 1 2024-03-22T07:15:58.915 INFO:tasks.workunit.client.0.smithi063.stdout:1/617: creat d4/fd3 x:0 0 0 2024-03-22T07:15:58.915 INFO:tasks.workunit.client.0.smithi063.stdout:1/618: readlink d4/df/d13/d2b/l2e 0 2024-03-22T07:15:58.915 INFO:tasks.workunit.client.0.smithi063.stdout:1/619: fsync d4/df/d13/d39/fa0 0 2024-03-22T07:15:58.916 INFO:tasks.workunit.client.1.smithi069.stdout:8/329: dread d1/de/d20/f3c [0,4194304] 0 2024-03-22T07:15:58.916 INFO:tasks.workunit.client.1.smithi069.stdout:8/330: read d1/d1c/d3e/f32 [3473906,75435] 0 2024-03-22T07:15:58.916 INFO:tasks.workunit.client.1.smithi069.stdout:8/331: chown d1/f14 0 1 2024-03-22T07:15:58.916 INFO:tasks.workunit.client.0.smithi063.stdout:2/498: getdents d0/de/d58/d7f 0 2024-03-22T07:15:58.917 INFO:tasks.workunit.client.0.smithi063.stdout:8/598: dwrite d0/d1b/d28/d7b/f7f [0,4194304] 0 2024-03-22T07:15:58.919 INFO:tasks.workunit.client.1.smithi069.stdout:7/304: unlink d9/db/d48/d50/d57/d44/f62 0 2024-03-22T07:15:58.919 INFO:tasks.workunit.client.0.smithi063.stdout:6/537: dwrite d3/f19 [0,4194304] 0 2024-03-22T07:15:58.919 INFO:tasks.workunit.client.0.smithi063.stdout:0/699: truncate da/de/d17/d25/d95/fd6 2944885 0 2024-03-22T07:15:58.920 INFO:tasks.workunit.client.0.smithi063.stdout:1/620: creat d4/d71/fd4 x:0 0 0 2024-03-22T07:15:58.921 INFO:tasks.workunit.client.1.smithi069.stdout:1/272: dread d1/fa [0,4194304] 0 2024-03-22T07:15:58.922 INFO:tasks.workunit.client.1.smithi069.stdout:8/332: mkdir d1/d1c/d3e/d66 0 2024-03-22T07:15:58.923 INFO:tasks.workunit.client.1.smithi069.stdout:8/333: write d1/d1c/d48/f4b [972548,102452] 0 2024-03-22T07:15:58.927 INFO:tasks.workunit.client.1.smithi069.stdout:8/334: creat d1/de/d20/d43/d4e/d5b/f67 x:0 0 0 2024-03-22T07:15:58.927 INFO:tasks.workunit.client.1.smithi069.stdout:8/335: write d1/de/d20/d43/d4e/d5b/f67 [264017,85572] 0 2024-03-22T07:15:58.927 INFO:tasks.workunit.client.1.smithi069.stdout:8/336: creat d1/d1c/f68 x:0 0 0 2024-03-22T07:15:58.927 INFO:tasks.workunit.client.1.smithi069.stdout:5/201: dread d1/d7/d8/f24 [0,4194304] 0 2024-03-22T07:15:58.927 INFO:tasks.workunit.client.1.smithi069.stdout:0/306: dwrite d3/d7/d15/d29/f30 [0,4194304] 0 2024-03-22T07:15:58.927 INFO:tasks.workunit.client.1.smithi069.stdout:7/305: dread d9/db/d1a/f39 [0,4194304] 0 2024-03-22T07:15:58.930 INFO:tasks.workunit.client.0.smithi063.stdout:0/700: symlink da/de/d17/d25/d3c/d64/ldb 0 2024-03-22T07:15:58.931 INFO:tasks.workunit.client.1.smithi069.stdout:7/306: symlink d9/db/d48/d50/d57/d44/d53/d1d/l6b 0 2024-03-22T07:15:58.933 INFO:tasks.workunit.client.0.smithi063.stdout:1/621: mknod d4/d71/dbb/cd5 0 2024-03-22T07:15:58.936 INFO:tasks.workunit.client.1.smithi069.stdout:7/307: rename d9/db/l67 to d9/l6c 0 2024-03-22T07:15:58.936 INFO:tasks.workunit.client.1.smithi069.stdout:7/308: write f8 [3722179,84834] 0 2024-03-22T07:15:58.936 INFO:tasks.workunit.client.1.smithi069.stdout:7/309: truncate d9/db/d48/d4b/f65 342811 0 2024-03-22T07:15:58.936 INFO:tasks.workunit.client.1.smithi069.stdout:7/310: fsync d9/db/d48/d4b/f4f 0 2024-03-22T07:15:58.940 INFO:tasks.workunit.client.0.smithi063.stdout:2/499: rmdir d0/de/d74 39 2024-03-22T07:15:58.940 INFO:tasks.workunit.client.0.smithi063.stdout:2/500: fsync d0/de/f65 0 2024-03-22T07:15:58.940 INFO:tasks.workunit.client.0.smithi063.stdout:2/501: readlink d0/de/d35/l9a 0 2024-03-22T07:15:58.941 INFO:tasks.workunit.client.1.smithi069.stdout:5/202: dread d1/f1f [0,4194304] 0 2024-03-22T07:15:58.941 INFO:tasks.workunit.client.0.smithi063.stdout:0/701: link da/de/d39/c3b da/de/d17/d25/d3c/cdc 0 2024-03-22T07:15:58.941 INFO:tasks.workunit.client.0.smithi063.stdout:0/702: chown da/f2b 10999 1 2024-03-22T07:15:58.942 INFO:tasks.workunit.client.1.smithi069.stdout:7/311: creat d9/db/d48/d50/d57/d44/d53/f6d x:0 0 0 2024-03-22T07:15:58.942 INFO:tasks.workunit.client.1.smithi069.stdout:1/273: dread d1/f35 [0,4194304] 0 2024-03-22T07:15:58.942 INFO:tasks.workunit.client.1.smithi069.stdout:1/274: write d1/f48 [5145736,84042] 0 2024-03-22T07:15:58.942 INFO:tasks.workunit.client.1.smithi069.stdout:1/275: write d1/f22 [678180,103171] 0 2024-03-22T07:15:58.942 INFO:tasks.workunit.client.1.smithi069.stdout:1/276: truncate d1/f42 491245 0 2024-03-22T07:15:58.944 INFO:tasks.workunit.client.1.smithi069.stdout:7/312: mkdir d9/db/d48/d50/d57/d44/d53/d1d/d2a/d6e 0 2024-03-22T07:15:58.944 INFO:tasks.workunit.client.0.smithi063.stdout:1/622: creat d4/df/d13/d39/fd6 x:0 0 0 2024-03-22T07:15:58.944 INFO:tasks.workunit.client.0.smithi063.stdout:2/502: truncate d0/de/d12/d20/f70 3319208 0 2024-03-22T07:15:58.946 INFO:tasks.workunit.client.1.smithi069.stdout:1/277: symlink d1/d13/l5a 0 2024-03-22T07:15:58.947 INFO:tasks.workunit.client.0.smithi063.stdout:0/703: rename da/de/d17/d25/d3c/dc6 to da/de/ddd 0 2024-03-22T07:15:58.949 INFO:tasks.workunit.client.1.smithi069.stdout:1/278: mkdir d1/d13/d36/d5b 0 2024-03-22T07:15:58.949 INFO:tasks.workunit.client.1.smithi069.stdout:1/279: dread - d1/d2a/d50/d4c/d55/f56 zero size 2024-03-22T07:15:58.949 INFO:tasks.workunit.client.1.smithi069.stdout:1/280: creat d1/d25/f5c x:0 0 0 2024-03-22T07:15:58.949 INFO:tasks.workunit.client.1.smithi069.stdout:1/281: chown d1/d25/l3d 7 1 2024-03-22T07:15:58.949 INFO:tasks.workunit.client.1.smithi069.stdout:1/282: write d1/d13/d36/f3e [890416,88760] 0 2024-03-22T07:15:58.949 INFO:tasks.workunit.client.1.smithi069.stdout:1/283: readlink d1/d13/l23 0 2024-03-22T07:15:58.949 INFO:tasks.workunit.client.0.smithi063.stdout:1/623: mknod d4/d14/d57/cd7 0 2024-03-22T07:15:58.950 INFO:tasks.workunit.client.1.smithi069.stdout:1/284: symlink d1/d2a/d50/d4c/l5d 0 2024-03-22T07:15:58.952 INFO:tasks.workunit.client.1.smithi069.stdout:1/285: mkdir d1/d2a/d5e 0 2024-03-22T07:15:58.953 INFO:tasks.workunit.client.1.smithi069.stdout:3/239: dwrite d5/d9/d26/d3f/f3e [0,4194304] 0 2024-03-22T07:15:58.954 INFO:tasks.workunit.client.0.smithi063.stdout:0/704: rename da/de/d17/d25/d95/d98/cab to da/de/d60/d56/d77/cde 0 2024-03-22T07:15:58.957 INFO:tasks.workunit.client.0.smithi063.stdout:1/624: mkdir d4/d71/dbb/dc0/dd8 0 2024-03-22T07:15:58.957 INFO:tasks.workunit.client.0.smithi063.stdout:1/625: chown d4/d14/c2f 56938119 1 2024-03-22T07:15:58.957 INFO:tasks.workunit.client.1.smithi069.stdout:3/240: getdents d5/d9 0 2024-03-22T07:15:58.957 INFO:tasks.workunit.client.1.smithi069.stdout:3/241: dread - d5/d9/db/f30 zero size 2024-03-22T07:15:58.958 INFO:tasks.workunit.client.1.smithi069.stdout:3/242: chown d5/l17 123 1 2024-03-22T07:15:58.959 INFO:tasks.workunit.client.0.smithi063.stdout:0/705: rename da/de/d17/d4e/f57 to da/de/d17/d4e/fdf 0 2024-03-22T07:15:58.960 INFO:tasks.workunit.client.0.smithi063.stdout:1/626: mkdir d4/df/d13/d39/d6f/dd9 0 2024-03-22T07:15:58.960 INFO:tasks.workunit.client.0.smithi063.stdout:1/627: creat d4/df/d13/d39/d63/fda x:0 0 0 2024-03-22T07:15:58.962 INFO:tasks.workunit.client.1.smithi069.stdout:5/203: dread d1/d6/f2e [0,4194304] 0 2024-03-22T07:15:58.963 INFO:tasks.workunit.client.0.smithi063.stdout:1/628: mkdir d4/d71/d88/da1/ddb 0 2024-03-22T07:15:58.963 INFO:tasks.workunit.client.1.smithi069.stdout:5/204: creat d1/d28/f41 x:0 0 0 2024-03-22T07:15:58.963 INFO:tasks.workunit.client.1.smithi069.stdout:5/205: creat d1/d6/d26/f42 x:0 0 0 2024-03-22T07:15:58.965 INFO:tasks.workunit.client.1.smithi069.stdout:5/206: mkdir d1/d7/d8/d43 0 2024-03-22T07:15:58.965 INFO:tasks.workunit.client.1.smithi069.stdout:5/207: truncate d1/d7/f40 1018238 0 2024-03-22T07:15:58.966 INFO:tasks.workunit.client.1.smithi069.stdout:5/208: truncate d1/f9 5461231 0 2024-03-22T07:15:58.967 INFO:tasks.workunit.client.1.smithi069.stdout:5/209: write d1/d7/d8/f18 [406163,110509] 0 2024-03-22T07:15:58.968 INFO:tasks.workunit.client.0.smithi063.stdout:4/469: dwrite d1/d5/d3c/f47 [0,4194304] 0 2024-03-22T07:15:58.969 INFO:tasks.workunit.client.0.smithi063.stdout:8/599: dwrite d0/d26/f27 [0,4194304] 0 2024-03-22T07:15:58.969 INFO:tasks.workunit.client.0.smithi063.stdout:8/600: write d0/d24/d33/d4c/f7d [544937,9547] 0 2024-03-22T07:15:58.970 INFO:tasks.workunit.client.0.smithi063.stdout:8/601: creat d0/d1b/fc3 x:0 0 0 2024-03-22T07:15:58.970 INFO:tasks.workunit.client.0.smithi063.stdout:4/470: write d1/d5/f67 [882400,93083] 0 2024-03-22T07:15:58.970 INFO:tasks.workunit.client.1.smithi069.stdout:5/210: mkdir d1/d10/d44 0 2024-03-22T07:15:58.972 INFO:tasks.workunit.client.1.smithi069.stdout:3/243: dread d5/d9/fd [0,4194304] 0 2024-03-22T07:15:58.972 INFO:tasks.workunit.client.1.smithi069.stdout:3/244: write d5/d9/f13 [4740904,72519] 0 2024-03-22T07:15:58.972 INFO:tasks.workunit.client.1.smithi069.stdout:3/245: chown d5/d9/d26/d3f/d35/f45 3167952 1 2024-03-22T07:15:58.972 INFO:tasks.workunit.client.1.smithi069.stdout:9/224: dwrite d2/f27 [0,4194304] 0 2024-03-22T07:15:58.972 INFO:tasks.workunit.client.1.smithi069.stdout:9/225: stat d2/dd/f37 0 2024-03-22T07:15:58.972 INFO:tasks.workunit.client.0.smithi063.stdout:8/602: unlink d0/d24/d33/c49 0 2024-03-22T07:15:58.974 INFO:tasks.workunit.client.1.smithi069.stdout:5/211: mknod d1/d7/d3d/d3f/c45 0 2024-03-22T07:15:58.974 INFO:tasks.workunit.client.0.smithi063.stdout:4/471: truncate d1/d5/d85/f8a 517365 0 2024-03-22T07:15:58.975 INFO:tasks.workunit.client.0.smithi063.stdout:4/472: read d1/d5/f9 [4736739,79877] 0 2024-03-22T07:15:58.975 INFO:tasks.workunit.client.0.smithi063.stdout:4/473: chown d1/d5/d43/d27/d2c/d3d/l7e 88319864 1 2024-03-22T07:15:58.975 INFO:tasks.workunit.client.0.smithi063.stdout:4/474: fsync d1/d5/f9 0 2024-03-22T07:15:58.975 INFO:tasks.workunit.client.0.smithi063.stdout:4/475: chown d1/d5/dc/f10 339275 1 2024-03-22T07:15:58.975 INFO:tasks.workunit.client.0.smithi063.stdout:8/603: mknod d0/d26/d2d/d83/cc4 0 2024-03-22T07:15:58.975 INFO:tasks.workunit.client.1.smithi069.stdout:5/212: mknod d1/d7/d8/c46 0 2024-03-22T07:15:58.977 INFO:tasks.workunit.client.1.smithi069.stdout:8/337: dwrite d1/de/f2d [4194304,4194304] 0 2024-03-22T07:15:58.977 INFO:tasks.workunit.client.0.smithi063.stdout:8/604: rename d0/d26/d2d/d59/c71 to d0/d1b/d28/cc5 0 2024-03-22T07:15:58.977 INFO:tasks.workunit.client.0.smithi063.stdout:8/605: write d0/d24/fb9 [513969,68778] 0 2024-03-22T07:15:58.978 INFO:tasks.workunit.client.1.smithi069.stdout:0/307: dwrite d3/da/f14 [4194304,4194304] 0 2024-03-22T07:15:58.979 INFO:tasks.workunit.client.0.smithi063.stdout:4/476: rmdir d1/d5/d85 39 2024-03-22T07:15:58.981 INFO:tasks.workunit.client.0.smithi063.stdout:2/503: dwrite d0/de/d12/d20/f82 [0,4194304] 0 2024-03-22T07:15:58.981 INFO:tasks.workunit.client.0.smithi063.stdout:8/606: mkdir d0/d1b/d28/d3e/d97/dc6 0 2024-03-22T07:15:58.981 INFO:tasks.workunit.client.0.smithi063.stdout:8/607: stat d0/d26/faf 0 2024-03-22T07:15:58.981 INFO:tasks.workunit.client.0.smithi063.stdout:6/538: dwrite d3/d15/f62 [0,4194304] 0 2024-03-22T07:15:58.986 INFO:tasks.workunit.client.0.smithi063.stdout:4/477: rename d1/d5/d43/d27/d2c/d4a/f8b to d1/d5/dc/d95/d59/fa9 0 2024-03-22T07:15:58.986 INFO:tasks.workunit.client.1.smithi069.stdout:9/226: dread d2/f27 [0,4194304] 0 2024-03-22T07:15:58.991 INFO:tasks.workunit.client.0.smithi063.stdout:4/478: dread d1/d5/f9 [0,4194304] 0 2024-03-22T07:15:58.994 INFO:tasks.workunit.client.0.smithi063.stdout:6/539: link d3/d15/d16/d60/d6d/d4e/d5a/d88/c8e d3/d6/d35/d3a/caf 0 2024-03-22T07:15:58.994 INFO:tasks.workunit.client.0.smithi063.stdout:6/540: chown d3/d6/d2a/d38/da5/d32/d51/l79 328938399 1 2024-03-22T07:15:58.994 INFO:tasks.workunit.client.0.smithi063.stdout:0/706: dread da/de/d17/d25/d3c/fd3 [0,4194304] 0 2024-03-22T07:15:58.994 INFO:tasks.workunit.client.1.smithi069.stdout:0/308: rmdir d3/d7/d17 39 2024-03-22T07:15:58.995 INFO:tasks.workunit.client.0.smithi063.stdout:6/541: read d3/d15/d16/f25 [583942,26654] 0 2024-03-22T07:15:58.995 INFO:tasks.workunit.client.0.smithi063.stdout:6/542: getdents d3/d6/d2a/d38/da5/d32/d51/d64 0 2024-03-22T07:15:58.996 INFO:tasks.workunit.client.0.smithi063.stdout:8/608: rename d0/d1b/c93 to d0/d26/d2d/d59/dc0/cc7 0 2024-03-22T07:15:58.997 INFO:tasks.workunit.client.0.smithi063.stdout:8/609: write d0/d1f/f53 [802167,117432] 0 2024-03-22T07:15:58.997 INFO:tasks.workunit.client.0.smithi063.stdout:8/610: stat d0/d26/fad 0 2024-03-22T07:15:58.997 INFO:tasks.workunit.client.1.smithi069.stdout:1/286: rmdir d1 39 2024-03-22T07:15:58.998 INFO:tasks.workunit.client.0.smithi063.stdout:2/504: symlink d0/l9e 0 2024-03-22T07:15:58.998 INFO:tasks.workunit.client.0.smithi063.stdout:2/505: dread - d0/de/d1b/f48 zero size 2024-03-22T07:15:58.998 INFO:tasks.workunit.client.0.smithi063.stdout:2/506: dread - d0/de/d58/f73 zero size 2024-03-22T07:15:58.998 INFO:tasks.workunit.client.0.smithi063.stdout:1/629: dwrite d4/df/d13/d39/f82 [0,4194304] 0 2024-03-22T07:15:59.005 INFO:tasks.workunit.client.0.smithi063.stdout:1/630: symlink d4/df/d13/ldc 0 2024-03-22T07:15:59.005 INFO:tasks.workunit.client.0.smithi063.stdout:8/611: rename d0/d1b/f84 to d0/d1b/fc8 0 2024-03-22T07:15:59.008 INFO:tasks.workunit.client.0.smithi063.stdout:1/631: mkdir d4/d14/ddd 0 2024-03-22T07:15:59.008 INFO:tasks.workunit.client.0.smithi063.stdout:1/632: truncate d4/f3b 1860060 0 2024-03-22T07:15:59.008 INFO:tasks.workunit.client.0.smithi063.stdout:1/633: write d4/d71/d88/fb9 [36220,100085] 0 2024-03-22T07:15:59.009 INFO:tasks.workunit.client.1.smithi069.stdout:5/213: dwrite d1/d6/f1e [0,4194304] 0 2024-03-22T07:15:59.009 INFO:tasks.workunit.client.1.smithi069.stdout:9/227: rename d2/dd/l11 to d2/l46 0 2024-03-22T07:15:59.011 INFO:tasks.workunit.client.0.smithi063.stdout:8/612: mkdir d0/d26/d2d/d59/dba/dc9 0 2024-03-22T07:15:59.012 INFO:tasks.workunit.client.0.smithi063.stdout:8/613: write d0/d26/d2d/d83/fa8 [746868,45628] 0 2024-03-22T07:15:59.012 INFO:tasks.workunit.client.0.smithi063.stdout:8/614: truncate d0/d26/d2d/d59/f87 1000407 0 2024-03-22T07:15:59.012 INFO:tasks.workunit.client.1.smithi069.stdout:0/309: write d3/d7/d5e/f61 [1352072,20850] 0 2024-03-22T07:15:59.012 INFO:tasks.workunit.client.1.smithi069.stdout:0/310: readlink d3/d7/d15/l36 0 2024-03-22T07:15:59.014 INFO:tasks.workunit.client.0.smithi063.stdout:1/634: mknod d4/d71/d88/cde 0 2024-03-22T07:15:59.019 INFO:tasks.workunit.client.1.smithi069.stdout:8/338: truncate d1/d1c/d48/f4b 340784 0 2024-03-22T07:15:59.023 INFO:tasks.workunit.client.1.smithi069.stdout:5/214: dread d1/d6/d26/d33/f38 [0,4194304] 0 2024-03-22T07:15:59.023 INFO:tasks.workunit.client.1.smithi069.stdout:5/215: truncate d1/f9 6214736 0 2024-03-22T07:15:59.028 INFO:tasks.workunit.client.1.smithi069.stdout:1/287: unlink d1/fa 0 2024-03-22T07:15:59.029 INFO:tasks.workunit.client.1.smithi069.stdout:9/228: truncate d2/d31/f40 3539288 0 2024-03-22T07:15:59.029 INFO:tasks.workunit.client.1.smithi069.stdout:9/229: write d2/dd/d3c/f3b [289290,63102] 0 2024-03-22T07:15:59.031 INFO:tasks.workunit.client.1.smithi069.stdout:0/311: creat d3/d7/d15/d29/f63 x:0 0 0 2024-03-22T07:15:59.031 INFO:tasks.workunit.client.1.smithi069.stdout:0/312: truncate d3/da/f3a 764944 0 2024-03-22T07:15:59.031 INFO:tasks.workunit.client.1.smithi069.stdout:0/313: fsync d3/d7/d17/f3f 0 2024-03-22T07:15:59.034 INFO:tasks.workunit.client.1.smithi069.stdout:8/339: symlink d1/l69 0 2024-03-22T07:15:59.036 INFO:tasks.workunit.client.1.smithi069.stdout:1/288: mkdir d1/d2a/d50/d4c/d55/d5f 0 2024-03-22T07:15:59.036 INFO:tasks.workunit.client.1.smithi069.stdout:1/289: write d1/d25/f38 [4410379,130223] 0 2024-03-22T07:15:59.036 INFO:tasks.workunit.client.1.smithi069.stdout:1/290: write d1/d13/f45 [1429055,95379] 0 2024-03-22T07:15:59.036 INFO:tasks.workunit.client.1.smithi069.stdout:9/230: symlink d2/d21/l47 0 2024-03-22T07:15:59.036 INFO:tasks.workunit.client.1.smithi069.stdout:1/291: write d1/d13/f39 [500010,35169] 0 2024-03-22T07:15:59.037 INFO:tasks.workunit.client.1.smithi069.stdout:8/340: mknod d1/d1c/d48/c6a 0 2024-03-22T07:15:59.037 INFO:tasks.workunit.client.1.smithi069.stdout:8/341: fsync d1/d1c/d48/d5e/f63 0 2024-03-22T07:15:59.040 INFO:tasks.workunit.client.1.smithi069.stdout:1/292: truncate d1/f48 174793 0 2024-03-22T07:15:59.040 INFO:tasks.workunit.client.1.smithi069.stdout:1/293: dread - d1/f3b zero size 2024-03-22T07:15:59.043 INFO:tasks.workunit.client.1.smithi069.stdout:8/342: rename d1/de/l2e to d1/d1c/d48/d5e/l6b 0 2024-03-22T07:15:59.044 INFO:tasks.workunit.client.1.smithi069.stdout:1/294: symlink d1/d25/d41/d1e/l60 0 2024-03-22T07:15:59.047 INFO:tasks.workunit.client.1.smithi069.stdout:1/295: mknod d1/d25/c61 0 2024-03-22T07:15:59.049 INFO:tasks.workunit.client.1.smithi069.stdout:8/343: rename d1/de/d20/d43 to d1/de/d6c 0 2024-03-22T07:15:59.050 INFO:tasks.workunit.client.0.smithi063.stdout:6/543: dwrite d3/f89 [0,4194304] 0 2024-03-22T07:15:59.050 INFO:tasks.workunit.client.0.smithi063.stdout:6/544: write d3/d6/f10 [4691142,77616] 0 2024-03-22T07:15:59.050 INFO:tasks.workunit.client.1.smithi069.stdout:8/344: truncate d1/d1c/d48/d56/f57 886865 0 2024-03-22T07:15:59.052 INFO:tasks.workunit.client.1.smithi069.stdout:8/345: creat d1/de/d6c/f6d x:0 0 0 2024-03-22T07:15:59.053 INFO:tasks.workunit.client.0.smithi063.stdout:6/545: getdents d3/d15/d16/d69/d8c 0 2024-03-22T07:15:59.053 INFO:tasks.workunit.client.0.smithi063.stdout:6/546: mknod d3/d15/d16/d60/d6d/d4e/d5a/d88/cb0 0 2024-03-22T07:15:59.054 INFO:tasks.workunit.client.1.smithi069.stdout:8/346: unlink d1/de/d6c/d4e/f5d 0 2024-03-22T07:15:59.054 INFO:tasks.workunit.client.0.smithi063.stdout:6/547: creat d3/d6/d2a/d38/da5/d32/fb1 x:0 0 0 2024-03-22T07:15:59.055 INFO:tasks.workunit.client.0.smithi063.stdout:6/548: creat d3/d15/d16/d60/d6d/d4e/d5a/d88/fb2 x:0 0 0 2024-03-22T07:15:59.059 INFO:tasks.workunit.client.0.smithi063.stdout:8/615: dwrite d0/f1a [0,4194304] 0 2024-03-22T07:15:59.059 INFO:tasks.workunit.client.0.smithi063.stdout:8/616: rename d0/d1b/d28/d7b to d0/d1b/d28/d7b/d45/dca 22 2024-03-22T07:15:59.059 INFO:tasks.workunit.client.0.smithi063.stdout:4/479: dwrite d1/d5/dc/d95/d59/f64 [0,4194304] 0 2024-03-22T07:15:59.059 INFO:tasks.workunit.client.0.smithi063.stdout:2/507: dwrite d0/de/d12/d20/f47 [0,4194304] 0 2024-03-22T07:15:59.059 INFO:tasks.workunit.client.0.smithi063.stdout:1/635: dwrite d4/df/d13/d2b/d86/fbf [0,4194304] 0 2024-03-22T07:15:59.059 INFO:tasks.workunit.client.0.smithi063.stdout:1/636: creat d4/df/d13/d2b/d6c/da8/fdf x:0 0 0 2024-03-22T07:15:59.059 INFO:tasks.workunit.client.0.smithi063.stdout:1/637: chown d4/df/d13/d2b/d6c/da6 35720 1 2024-03-22T07:15:59.060 INFO:tasks.workunit.client.1.smithi069.stdout:0/314: getdents d3/d7/d15/d29 0 2024-03-22T07:15:59.067 INFO:tasks.workunit.client.1.smithi069.stdout:9/231: dwrite d2/f27 [0,4194304] 0 2024-03-22T07:15:59.067 INFO:tasks.workunit.client.1.smithi069.stdout:9/232: readlink d2/d21/l41 0 2024-03-22T07:15:59.067 INFO:tasks.workunit.client.1.smithi069.stdout:9/233: stat d2/dd/d3c/f3b 0 2024-03-22T07:15:59.067 INFO:tasks.workunit.client.0.smithi063.stdout:8/617: mknod d0/d1b/d28/d7b/d4a/ccb 0 2024-03-22T07:15:59.067 INFO:tasks.workunit.client.0.smithi063.stdout:0/707: dwrite da/de/d17/d4e/fce [0,4194304] 0 2024-03-22T07:15:59.068 INFO:tasks.workunit.client.1.smithi069.stdout:0/315: dread d3/d7/fb [0,4194304] 0 2024-03-22T07:15:59.068 INFO:tasks.workunit.client.1.smithi069.stdout:0/316: creat d3/d7/f64 x:0 0 0 2024-03-22T07:15:59.071 INFO:tasks.workunit.client.0.smithi063.stdout:2/508: mknod d0/de/d1b/d63/c9f 0 2024-03-22T07:15:59.074 INFO:tasks.workunit.client.0.smithi063.stdout:8/618: creat d0/d26/d2d/d83/fcc x:0 0 0 2024-03-22T07:15:59.078 INFO:tasks.workunit.client.1.smithi069.stdout:0/317: mknod d3/d7/d24/c65 0 2024-03-22T07:15:59.079 INFO:tasks.workunit.client.1.smithi069.stdout:0/318: stat d3/d7/d17 0 2024-03-22T07:15:59.080 INFO:tasks.workunit.client.1.smithi069.stdout:0/319: rename d3/d7/d17/c48 to d3/d7/d15/d1f/c66 0 2024-03-22T07:15:59.081 INFO:tasks.workunit.client.1.smithi069.stdout:0/320: mknod d3/c67 0 2024-03-22T07:15:59.083 INFO:tasks.workunit.client.1.smithi069.stdout:0/321: link d3/c22 d3/d7/d17/d1c/c68 0 2024-03-22T07:15:59.085 INFO:tasks.workunit.client.1.smithi069.stdout:0/322: creat d3/d7/d15/d29/f69 x:0 0 0 2024-03-22T07:15:59.086 INFO:tasks.workunit.client.0.smithi063.stdout:6/549: dread d3/d76/f72 [0,4194304] 0 2024-03-22T07:15:59.089 INFO:tasks.workunit.client.0.smithi063.stdout:8/619: mknod d0/d1b/d28/d3e/db2/ccd 0 2024-03-22T07:15:59.091 INFO:tasks.workunit.client.0.smithi063.stdout:6/550: rename d3/d15/f78 to d3/d15/d16/d60/fb3 0 2024-03-22T07:15:59.091 INFO:tasks.workunit.client.0.smithi063.stdout:6/551: chown d3/d15/d16/d60/d6d/d4e/d5f/f93 1161 1 2024-03-22T07:15:59.091 INFO:tasks.workunit.client.0.smithi063.stdout:6/552: chown d3/d15/d16/d57/l70 5776812 1 2024-03-22T07:15:59.091 INFO:tasks.workunit.client.0.smithi063.stdout:8/620: write d0/d24/d5b/f60 [533898,71784] 0 2024-03-22T07:15:59.091 INFO:tasks.workunit.client.0.smithi063.stdout:8/621: creat d0/d1b/d28/d3e/d97/fce x:0 0 0 2024-03-22T07:15:59.093 INFO:tasks.workunit.client.0.smithi063.stdout:6/553: mknod d3/d15/d16/d60/d6d/d4e/cb4 0 2024-03-22T07:15:59.098 INFO:tasks.workunit.client.1.smithi069.stdout:2/222: sync 2024-03-22T07:15:59.098 INFO:tasks.workunit.client.0.smithi063.stdout:8/622: stat d0/d1b/d28/d3e/l8e 0 2024-03-22T07:15:59.099 INFO:tasks.workunit.client.0.smithi063.stdout:8/623: chown d0/d1b/c73 54509854 1 2024-03-22T07:15:59.099 INFO:tasks.workunit.client.0.smithi063.stdout:6/554: creat d3/d15/d16/d47/fb5 x:0 0 0 2024-03-22T07:15:59.099 INFO:tasks.workunit.client.0.smithi063.stdout:8/624: dread d0/d24/fb9 [0,4194304] 0 2024-03-22T07:15:59.100 INFO:tasks.workunit.client.0.smithi063.stdout:6/555: symlink d3/d15/da3/lb6 0 2024-03-22T07:15:59.127 INFO:tasks.workunit.client.0.smithi063.stdout:1/638: dwrite d4/f93 [0,4194304] 0 2024-03-22T07:15:59.127 INFO:tasks.workunit.client.0.smithi063.stdout:6/556: write d3/d15/d16/d57/f97 [279045,92454] 0 2024-03-22T07:15:59.133 INFO:tasks.workunit.client.0.smithi063.stdout:6/557: creat d3/d15/d16/d60/d6d/d4e/d5f/fb7 x:0 0 0 2024-03-22T07:15:59.139 INFO:tasks.workunit.client.1.smithi069.stdout:8/347: dwrite d1/d1c/d3e/f16 [0,4194304] 0 2024-03-22T07:15:59.139 INFO:tasks.workunit.client.1.smithi069.stdout:8/348: write d1/d1c/d3e/f32 [2009525,18631] 0 2024-03-22T07:15:59.139 INFO:tasks.workunit.client.1.smithi069.stdout:9/234: dwrite d2/dd/d3c/d2e/f3f [0,4194304] 0 2024-03-22T07:15:59.140 INFO:tasks.workunit.client.0.smithi063.stdout:0/708: dwrite da/de/d17/f5d [0,4194304] 0 2024-03-22T07:15:59.140 INFO:tasks.workunit.client.0.smithi063.stdout:0/709: readlink da/de/d67/lba 0 2024-03-22T07:15:59.140 INFO:tasks.workunit.client.1.smithi069.stdout:2/223: dwrite d18/d4d/d27/d33/f39 [0,4194304] 0 2024-03-22T07:15:59.140 INFO:tasks.workunit.client.0.smithi063.stdout:4/480: dwrite d1/d5/dc/d95/f60 [0,4194304] 0 2024-03-22T07:15:59.141 INFO:tasks.workunit.client.0.smithi063.stdout:4/481: read - d1/d5/d43/d27/d2c/d3d/d54/f8d zero size 2024-03-22T07:15:59.141 INFO:tasks.workunit.client.0.smithi063.stdout:2/509: dwrite d0/de/d12/d39/f76 [0,4194304] 0 2024-03-22T07:15:59.142 INFO:tasks.workunit.client.1.smithi069.stdout:0/323: dwrite d3/da/f60 [0,4194304] 0 2024-03-22T07:15:59.142 INFO:tasks.workunit.client.1.smithi069.stdout:0/324: chown d3/d7/d15/d29/f69 6 1 2024-03-22T07:15:59.144 INFO:tasks.workunit.client.0.smithi063.stdout:8/625: dwrite d0/d24/f5d [0,4194304] 0 2024-03-22T07:15:59.144 INFO:tasks.workunit.client.0.smithi063.stdout:0/710: write da/de/d17/d25/d3c/fd3 [1873573,110128] 0 2024-03-22T07:15:59.144 INFO:tasks.workunit.client.0.smithi063.stdout:0/711: chown da/de/d17/d25/d3c/d43/db5 1872 1 2024-03-22T07:15:59.145 INFO:tasks.workunit.client.1.smithi069.stdout:2/224: rename f12 to d18/d4d/d34/f53 0 2024-03-22T07:15:59.145 INFO:tasks.workunit.client.1.smithi069.stdout:8/349: getdents d1/de/d20 0 2024-03-22T07:15:59.146 INFO:tasks.workunit.client.1.smithi069.stdout:9/235: dread d2/d31/f40 [0,4194304] 0 2024-03-22T07:15:59.147 INFO:tasks.workunit.client.0.smithi063.stdout:2/510: link d0/de/l5b d0/de/d12/la0 0 2024-03-22T07:15:59.148 INFO:tasks.workunit.client.1.smithi069.stdout:2/225: creat d18/d4d/d27/d33/d3d/f54 x:0 0 0 2024-03-22T07:15:59.148 INFO:tasks.workunit.client.1.smithi069.stdout:2/226: readlink d18/d4d/l2a 0 2024-03-22T07:15:59.150 INFO:tasks.workunit.client.1.smithi069.stdout:2/227: creat d18/d4d/d27/d33/f55 x:0 0 0 2024-03-22T07:15:59.150 INFO:tasks.workunit.client.0.smithi063.stdout:1/639: dwrite d4/f12 [0,4194304] 0 2024-03-22T07:15:59.153 INFO:tasks.workunit.client.0.smithi063.stdout:8/626: mknod d0/d26/d2d/d59/ccf 0 2024-03-22T07:15:59.156 INFO:tasks.workunit.client.0.smithi063.stdout:0/712: creat da/fe0 x:0 0 0 2024-03-22T07:15:59.157 INFO:tasks.workunit.client.0.smithi063.stdout:8/627: mkdir d0/d1b/d28/d3e/db2/dbd/dd0 0 2024-03-22T07:15:59.157 INFO:tasks.workunit.client.0.smithi063.stdout:8/628: chown d0/d26/d2d/d59/f87 3 1 2024-03-22T07:15:59.157 INFO:tasks.workunit.client.1.smithi069.stdout:0/325: write d3/da/f60 [3082090,85553] 0 2024-03-22T07:15:59.157 INFO:tasks.workunit.client.1.smithi069.stdout:0/326: read d3/d7/d15/f1b [1108848,42936] 0 2024-03-22T07:15:59.157 INFO:tasks.workunit.client.1.smithi069.stdout:0/327: write d3/d7/d17/f56 [334770,44810] 0 2024-03-22T07:15:59.157 INFO:tasks.workunit.client.1.smithi069.stdout:0/328: stat d3/d7/d15/f46 0 2024-03-22T07:15:59.159 INFO:tasks.workunit.client.0.smithi063.stdout:0/713: symlink da/de/d17/d25/d3c/d64/d78/le1 0 2024-03-22T07:15:59.160 INFO:tasks.workunit.client.0.smithi063.stdout:0/714: chown da/d49/f51 2989186 1 2024-03-22T07:15:59.160 INFO:tasks.workunit.client.0.smithi063.stdout:0/715: chown da/de/d17/c34 36 1 2024-03-22T07:15:59.160 INFO:tasks.workunit.client.0.smithi063.stdout:0/716: readlink da/de/d17/d25/d3c/d43/dc9/ld7 0 2024-03-22T07:15:59.160 INFO:tasks.workunit.client.0.smithi063.stdout:0/717: write da/de/d17/d25/d3c/d43/d80/f86 [1974967,95133] 0 2024-03-22T07:15:59.160 INFO:tasks.workunit.client.0.smithi063.stdout:0/718: write da/de/fcf [912859,61308] 0 2024-03-22T07:15:59.161 INFO:tasks.workunit.client.0.smithi063.stdout:8/629: mkdir d0/d1b/d28/d3e/d97/dd1 0 2024-03-22T07:15:59.170 INFO:tasks.workunit.client.0.smithi063.stdout:1/640: write d4/df/d13/d2b/d86/fbf [2301933,117078] 0 2024-03-22T07:15:59.170 INFO:tasks.workunit.client.0.smithi063.stdout:1/641: write d4/df/d13/d2b/f84 [804627,65915] 0 2024-03-22T07:15:59.172 INFO:tasks.workunit.client.0.smithi063.stdout:3/569: sync 2024-03-22T07:15:59.172 INFO:tasks.workunit.client.0.smithi063.stdout:3/570: chown da/d16/d26/d69 17510909 1 2024-03-22T07:15:59.172 INFO:tasks.workunit.client.0.smithi063.stdout:3/571: stat da/d16/d26/d69/cb4 0 2024-03-22T07:15:59.172 INFO:tasks.workunit.client.0.smithi063.stdout:3/572: dread - da/d63/d65/d4b/fb2 zero size 2024-03-22T07:15:59.175 INFO:tasks.workunit.client.0.smithi063.stdout:3/573: truncate da/d16/d26/f76 7554203 0 2024-03-22T07:15:59.175 INFO:tasks.workunit.client.0.smithi063.stdout:3/574: chown da/d63/d65/d7b 1762293 1 2024-03-22T07:15:59.176 INFO:tasks.workunit.client.0.smithi063.stdout:1/642: getdents d4/d71 0 2024-03-22T07:15:59.176 INFO:tasks.workunit.client.0.smithi063.stdout:1/643: chown d4/df 97782 1 2024-03-22T07:15:59.176 INFO:tasks.workunit.client.0.smithi063.stdout:3/575: write da/d16/d26/f2a [2405639,12857] 0 2024-03-22T07:15:59.186 INFO:tasks.workunit.client.0.smithi063.stdout:7/392: sync 2024-03-22T07:15:59.190 INFO:tasks.workunit.client.1.smithi069.stdout:9/236: dwrite d2/dd/d3c/d2e/f32 [0,4194304] 0 2024-03-22T07:15:59.192 INFO:tasks.workunit.client.1.smithi069.stdout:9/237: mkdir d2/dd/d3c/d48 0 2024-03-22T07:15:59.192 INFO:tasks.workunit.client.0.smithi063.stdout:3/576: write da/d16/d26/d69/f6b [2560905,11376] 0 2024-03-22T07:15:59.193 INFO:tasks.workunit.client.0.smithi063.stdout:3/577: fsync da/d16/f42 0 2024-03-22T07:15:59.193 INFO:tasks.workunit.client.0.smithi063.stdout:9/460: sync 2024-03-22T07:15:59.193 INFO:tasks.workunit.client.0.smithi063.stdout:5/488: sync 2024-03-22T07:15:59.193 INFO:tasks.workunit.client.1.smithi069.stdout:9/238: write d2/dd/f37 [30389,83909] 0 2024-03-22T07:15:59.193 INFO:tasks.workunit.client.0.smithi063.stdout:3/578: mknod da/d63/cbe 0 2024-03-22T07:15:59.194 INFO:tasks.workunit.client.0.smithi063.stdout:9/461: symlink d8/d15/l8e 0 2024-03-22T07:15:59.195 INFO:tasks.workunit.client.1.smithi069.stdout:9/239: rename d2/lc to d2/dd/d3c/d2e/l49 0 2024-03-22T07:15:59.197 INFO:tasks.workunit.client.0.smithi063.stdout:2/511: dwrite d0/de/d12/d20/f88 [4194304,4194304] 0 2024-03-22T07:15:59.197 INFO:tasks.workunit.client.0.smithi063.stdout:2/512: truncate d0/de/d35/f6f 183086 0 2024-03-22T07:15:59.197 INFO:tasks.workunit.client.0.smithi063.stdout:2/513: read d0/de/d35/f6f [165440,28376] 0 2024-03-22T07:15:59.198 INFO:tasks.workunit.client.0.smithi063.stdout:9/462: rmdir d8/d19/d41/d48/d7b 0 2024-03-22T07:15:59.198 INFO:tasks.workunit.client.1.smithi069.stdout:0/329: truncate d3/da/f60 417441 0 2024-03-22T07:15:59.198 INFO:tasks.workunit.client.1.smithi069.stdout:0/330: chown d3/d7/d15/d1f/f4e 4724026 1 2024-03-22T07:15:59.198 INFO:tasks.workunit.client.1.smithi069.stdout:9/240: dread d2/dd/d3c/f3e [0,4194304] 0 2024-03-22T07:15:59.199 INFO:tasks.workunit.client.1.smithi069.stdout:9/241: dread - d2/dd/d3c/f35 zero size 2024-03-22T07:15:59.200 INFO:tasks.workunit.client.1.smithi069.stdout:9/242: creat d2/dd/d3c/d48/f4a x:0 0 0 2024-03-22T07:15:59.201 INFO:tasks.workunit.client.0.smithi063.stdout:3/579: rename da/d63/d65/f78 to da/d16/d26/fbf 0 2024-03-22T07:15:59.201 INFO:tasks.workunit.client.0.smithi063.stdout:0/719: dwrite da/f54 [0,4194304] 0 2024-03-22T07:15:59.201 INFO:tasks.workunit.client.0.smithi063.stdout:0/720: creat da/de/d17/d4e/fe2 x:0 0 0 2024-03-22T07:15:59.201 INFO:tasks.workunit.client.0.smithi063.stdout:0/721: write da/de/d17/f21 [5073030,13996] 0 2024-03-22T07:15:59.201 INFO:tasks.workunit.client.0.smithi063.stdout:0/722: creat da/de/d17/d25/d3c/fe3 x:0 0 0 2024-03-22T07:15:59.201 INFO:tasks.workunit.client.0.smithi063.stdout:0/723: read - da/fc2 zero size 2024-03-22T07:15:59.202 INFO:tasks.workunit.client.1.smithi069.stdout:8/350: dwrite d1/d1c/f68 [0,4194304] 0 2024-03-22T07:15:59.202 INFO:tasks.workunit.client.0.smithi063.stdout:8/630: dwrite d0/d26/f92 [0,4194304] 0 2024-03-22T07:15:59.202 INFO:tasks.workunit.client.1.smithi069.stdout:9/243: link d2/f5 d2/dd/d3c/d2e/f4b 0 2024-03-22T07:15:59.204 INFO:tasks.workunit.client.1.smithi069.stdout:8/351: dread d1/de/d6c/d4e/d5b/f67 [0,4194304] 0 2024-03-22T07:15:59.204 INFO:tasks.workunit.client.1.smithi069.stdout:8/352: write d1/de/d6c/f6d [1045162,112497] 0 2024-03-22T07:15:59.205 INFO:tasks.workunit.client.0.smithi063.stdout:9/463: mkdir d8/d19/d20/d22/d56/d8f 0 2024-03-22T07:15:59.205 INFO:tasks.workunit.client.0.smithi063.stdout:9/464: stat l5 0 2024-03-22T07:15:59.206 INFO:tasks.workunit.client.0.smithi063.stdout:9/465: read d8/d17/f69 [24569,123493] 0 2024-03-22T07:15:59.206 INFO:tasks.workunit.client.0.smithi063.stdout:9/466: write d8/d19/d39/f6f [312039,70830] 0 2024-03-22T07:15:59.207 INFO:tasks.workunit.client.0.smithi063.stdout:1/644: dwrite d4/d5a/fa9 [0,4194304] 0 2024-03-22T07:15:59.208 INFO:tasks.workunit.client.1.smithi069.stdout:9/244: mknod d2/dd/d3c/c4c 0 2024-03-22T07:15:59.210 INFO:tasks.workunit.client.1.smithi069.stdout:8/353: link d1/d1c/f22 d1/d1c/d3e/d66/f6e 0 2024-03-22T07:15:59.211 INFO:tasks.workunit.client.0.smithi063.stdout:2/514: write d0/f53 [3898286,123011] 0 2024-03-22T07:15:59.213 INFO:tasks.workunit.client.0.smithi063.stdout:3/580: mknod da/cc0 0 2024-03-22T07:15:59.213 INFO:tasks.workunit.client.0.smithi063.stdout:3/581: chown da/d63/d65/d81/d9b/d9f 42422 1 2024-03-22T07:15:59.217 INFO:tasks.workunit.client.0.smithi063.stdout:0/724: mknod da/de/d17/d25/d3c/d43/ce4 0 2024-03-22T07:15:59.217 INFO:tasks.workunit.client.0.smithi063.stdout:0/725: stat da/de/d17/d25/d3c/d64/fb2 0 2024-03-22T07:15:59.221 INFO:tasks.workunit.client.0.smithi063.stdout:1/645: dread d4/fb [0,4194304] 0 2024-03-22T07:15:59.221 INFO:tasks.workunit.client.0.smithi063.stdout:1/646: chown d4/f3b 832708340 1 2024-03-22T07:15:59.221 INFO:tasks.workunit.client.0.smithi063.stdout:8/631: symlink d0/d1b/d28/d3e/db2/ld2 0 2024-03-22T07:15:59.224 INFO:tasks.workunit.client.0.smithi063.stdout:1/647: write d4/df/f11 [3491979,43454] 0 2024-03-22T07:15:59.225 INFO:tasks.workunit.client.0.smithi063.stdout:5/489: dwrite d8/d13/d1e/d31/d64/d65/f7c [0,4194304] 0 2024-03-22T07:15:59.225 INFO:tasks.workunit.client.1.smithi069.stdout:0/331: dwrite d3/d7/d17/f1a [0,4194304] 0 2024-03-22T07:15:59.225 INFO:tasks.workunit.client.1.smithi069.stdout:0/332: write d3/d7/fb [906330,81439] 0 2024-03-22T07:15:59.225 INFO:tasks.workunit.client.1.smithi069.stdout:0/333: fdatasync d3/d7/d15/f32 0 2024-03-22T07:15:59.225 INFO:tasks.workunit.client.1.smithi069.stdout:0/334: chown d3/d7/d15 409488588 1 2024-03-22T07:15:59.226 INFO:tasks.workunit.client.0.smithi063.stdout:2/515: creat d0/de/d58/d7f/d97/fa1 x:0 0 0 2024-03-22T07:15:59.226 INFO:tasks.workunit.client.1.smithi069.stdout:0/335: mkdir d3/d7/d24/d6a 0 2024-03-22T07:15:59.226 INFO:tasks.workunit.client.1.smithi069.stdout:8/354: dread d1/d1c/d3e/d66/f6e [0,4194304] 0 2024-03-22T07:15:59.226 INFO:tasks.workunit.client.1.smithi069.stdout:8/355: chown d1/de/l2a 100459727 1 2024-03-22T07:15:59.226 INFO:tasks.workunit.client.1.smithi069.stdout:8/356: chown d1/d1c/c21 7769 1 2024-03-22T07:15:59.227 INFO:tasks.workunit.client.0.smithi063.stdout:0/726: mkdir da/de/d17/d25/d99/de5 0 2024-03-22T07:15:59.227 INFO:tasks.workunit.client.0.smithi063.stdout:0/727: chown da/f2b 212 1 2024-03-22T07:15:59.228 INFO:tasks.workunit.client.1.smithi069.stdout:8/357: mknod d1/de/d6c/d4e/d5b/c6f 0 2024-03-22T07:15:59.228 INFO:tasks.workunit.client.1.smithi069.stdout:8/358: fsync d1/de/f2d 0 2024-03-22T07:15:59.228 INFO:tasks.workunit.client.1.smithi069.stdout:8/359: readlink d1/d1c/d48/l62 0 2024-03-22T07:15:59.228 INFO:tasks.workunit.client.1.smithi069.stdout:8/360: chown d1/d1c/f68 37 1 2024-03-22T07:15:59.229 INFO:tasks.workunit.client.1.smithi069.stdout:8/361: mkdir d1/d1c/d70 0 2024-03-22T07:15:59.229 INFO:tasks.workunit.client.0.smithi063.stdout:8/632: creat d0/d1b/d28/d5f/fd3 x:0 0 0 2024-03-22T07:15:59.230 INFO:tasks.workunit.client.1.smithi069.stdout:8/362: mknod d1/c71 0 2024-03-22T07:15:59.230 INFO:tasks.workunit.client.1.smithi069.stdout:8/363: fsync d1/f1d 0 2024-03-22T07:15:59.230 INFO:tasks.workunit.client.1.smithi069.stdout:8/364: readlink d1/de/d6c/d4e/l53 0 2024-03-22T07:15:59.230 INFO:tasks.workunit.client.1.smithi069.stdout:9/245: rmdir d2/dd/d3c 39 2024-03-22T07:15:59.231 INFO:tasks.workunit.client.1.smithi069.stdout:9/246: creat d2/dd/d3c/d48/f4d x:0 0 0 2024-03-22T07:15:59.232 INFO:tasks.workunit.client.0.smithi063.stdout:9/467: rename d8/d19/f45 to d8/f90 0 2024-03-22T07:15:59.233 INFO:tasks.workunit.client.1.smithi069.stdout:9/247: unlink d2/dd/d3c/d2e/f42 0 2024-03-22T07:15:59.234 INFO:tasks.workunit.client.0.smithi063.stdout:1/648: mknod d4/d5a/d91/ce0 0 2024-03-22T07:15:59.239 INFO:tasks.workunit.client.0.smithi063.stdout:5/490: mknod d8/d13/d1e/d31/c89 0 2024-03-22T07:15:59.239 INFO:tasks.workunit.client.1.smithi069.stdout:8/365: write d1/de/f40 [4979376,33348] 0 2024-03-22T07:15:59.241 INFO:tasks.workunit.client.0.smithi063.stdout:2/516: mkdir d0/de/d58/d7f/d97/da2 0 2024-03-22T07:15:59.243 INFO:tasks.workunit.client.0.smithi063.stdout:0/728: link da/de/d17/d25/d3c/d43/d76/d92/fbf da/de/d39/fe6 0 2024-03-22T07:15:59.244 INFO:tasks.workunit.client.0.smithi063.stdout:8/633: stat d0/c7 0 2024-03-22T07:15:59.244 INFO:tasks.workunit.client.1.smithi069.stdout:8/366: fdatasync d1/de/f40 0 2024-03-22T07:15:59.245 INFO:tasks.workunit.client.0.smithi063.stdout:8/634: write d0/d1b/fc8 [444448,102607] 0 2024-03-22T07:15:59.248 INFO:tasks.workunit.client.0.smithi063.stdout:9/468: mknod d8/d19/d41/d48/c91 0 2024-03-22T07:15:59.248 INFO:tasks.workunit.client.0.smithi063.stdout:1/649: creat d4/d71/fe1 x:0 0 0 2024-03-22T07:15:59.249 INFO:tasks.workunit.client.1.smithi069.stdout:8/367: rmdir d1/d1c/d3e 39 2024-03-22T07:15:59.249 INFO:tasks.workunit.client.1.smithi069.stdout:8/368: write d1/d1c/f22 [922263,72380] 0 2024-03-22T07:15:59.249 INFO:tasks.workunit.client.1.smithi069.stdout:8/369: creat d1/de/d6c/f72 x:0 0 0 2024-03-22T07:15:59.249 INFO:tasks.workunit.client.1.smithi069.stdout:8/370: dread - d1/d1c/d48/f64 zero size 2024-03-22T07:15:59.249 INFO:tasks.workunit.client.1.smithi069.stdout:8/371: read d1/d1c/f38 [487998,111156] 0 2024-03-22T07:15:59.249 INFO:tasks.workunit.client.1.smithi069.stdout:8/372: mkdir d1/de/d6c/d73 0 2024-03-22T07:15:59.250 INFO:tasks.workunit.client.1.smithi069.stdout:8/373: creat d1/de/d6c/d73/f74 x:0 0 0 2024-03-22T07:15:59.251 INFO:tasks.workunit.client.1.smithi069.stdout:8/374: creat d1/d1c/d3e/d66/f75 x:0 0 0 2024-03-22T07:15:59.252 INFO:tasks.workunit.client.1.smithi069.stdout:8/375: truncate d1/d1c/d48/d5e/f63 132197 0 2024-03-22T07:15:59.252 INFO:tasks.workunit.client.1.smithi069.stdout:0/336: dwrite d3/d7/d15/f20 [0,4194304] 0 2024-03-22T07:15:59.252 INFO:tasks.workunit.client.0.smithi063.stdout:2/517: symlink d0/de/d1b/la3 0 2024-03-22T07:15:59.254 INFO:tasks.workunit.client.0.smithi063.stdout:0/729: symlink da/de/le7 0 2024-03-22T07:15:59.254 INFO:tasks.workunit.client.1.smithi069.stdout:8/376: link d1/f29 d1/de/d6c/d4e/d5b/f76 0 2024-03-22T07:15:59.255 INFO:tasks.workunit.client.1.smithi069.stdout:9/248: dwrite d2/dd/d3c/d2e/f3f [4194304,4194304] 0 2024-03-22T07:15:59.255 INFO:tasks.workunit.client.1.smithi069.stdout:6/180: sync 2024-03-22T07:15:59.255 INFO:tasks.workunit.client.1.smithi069.stdout:4/348: sync 2024-03-22T07:15:59.255 INFO:tasks.workunit.client.1.smithi069.stdout:7/313: sync 2024-03-22T07:15:59.255 INFO:tasks.workunit.client.1.smithi069.stdout:0/337: rename d3/da/f14 to d3/d7/d17/f6b 0 2024-03-22T07:15:59.258 INFO:tasks.workunit.client.1.smithi069.stdout:8/377: truncate d1/d1c/f47 29786 0 2024-03-22T07:15:59.258 INFO:tasks.workunit.client.1.smithi069.stdout:9/249: creat d2/dd/d3c/d2e/f4e x:0 0 0 2024-03-22T07:15:59.259 INFO:tasks.workunit.client.1.smithi069.stdout:6/181: dread d4/d7/f26 [0,4194304] 0 2024-03-22T07:15:59.259 INFO:tasks.workunit.client.1.smithi069.stdout:6/182: write d4/dd/d12/f34 [862225,46288] 0 2024-03-22T07:15:59.259 INFO:tasks.workunit.client.0.smithi063.stdout:1/650: mknod d4/df/ce2 0 2024-03-22T07:15:59.261 INFO:tasks.workunit.client.0.smithi063.stdout:2/518: symlink d0/de/d74/la4 0 2024-03-22T07:15:59.261 INFO:tasks.workunit.client.1.smithi069.stdout:9/250: symlink d2/dd/d3c/l4f 0 2024-03-22T07:15:59.264 INFO:tasks.workunit.client.1.smithi069.stdout:6/183: fdatasync d4/dd/d12/f34 0 2024-03-22T07:15:59.266 INFO:tasks.workunit.client.0.smithi063.stdout:3/582: rename da/d63/d65/d86 to da/d16/d89/d8c/dc1 0 2024-03-22T07:15:59.266 INFO:tasks.workunit.client.0.smithi063.stdout:5/491: rename d8/d13/d1e to d8/d13/d1e/d31/d58/d8a 22 2024-03-22T07:15:59.266 INFO:tasks.workunit.client.0.smithi063.stdout:5/492: write d8/d13/f4b [538376,70241] 0 2024-03-22T07:15:59.266 INFO:tasks.workunit.client.0.smithi063.stdout:5/493: fsync d8/d13/d1e/d31/d64/d65/f71 0 2024-03-22T07:15:59.267 INFO:tasks.workunit.client.1.smithi069.stdout:6/184: rmdir d4/dd/d12/d13 39 2024-03-22T07:15:59.267 INFO:tasks.workunit.client.1.smithi069.stdout:6/185: creat d4/dd/d12/d2b/f37 x:0 0 0 2024-03-22T07:15:59.267 INFO:tasks.workunit.client.1.smithi069.stdout:6/186: chown d4/dd 22431 1 2024-03-22T07:15:59.269 INFO:tasks.workunit.client.0.smithi063.stdout:8/635: dwrite d0/d1b/d28/d7b/f9b [0,4194304] 0 2024-03-22T07:15:59.269 INFO:tasks.workunit.client.0.smithi063.stdout:8/636: creat d0/d24/d33/d4c/fd4 x:0 0 0 2024-03-22T07:15:59.269 INFO:tasks.workunit.client.0.smithi063.stdout:8/637: write d0/d1b/d28/d3e/d97/fce [838115,91756] 0 2024-03-22T07:15:59.271 INFO:tasks.workunit.client.0.smithi063.stdout:1/651: unlink d4/df/d13/d2b/d86/fb5 0 2024-03-22T07:15:59.271 INFO:tasks.workunit.client.1.smithi069.stdout:6/187: dread d4/d7/f26 [0,4194304] 0 2024-03-22T07:15:59.274 INFO:tasks.workunit.client.0.smithi063.stdout:0/730: rmdir da/de/d17/d25/d3c/d64/d78 39 2024-03-22T07:15:59.275 INFO:tasks.workunit.client.0.smithi063.stdout:5/494: dread d8/d13/d24/f78 [0,4194304] 0 2024-03-22T07:15:59.280 INFO:tasks.workunit.client.0.smithi063.stdout:3/583: link da/d63/l82 da/d63/d65/d92/lc2 0 2024-03-22T07:15:59.281 INFO:tasks.workunit.client.1.smithi069.stdout:9/251: truncate d2/dd/d3c/d2e/f3f 5708118 0 2024-03-22T07:15:59.286 INFO:tasks.workunit.client.0.smithi063.stdout:1/652: mknod d4/d71/d88/da1/ddb/ce3 0 2024-03-22T07:15:59.286 INFO:tasks.workunit.client.0.smithi063.stdout:1/653: dread - d4/fd3 zero size 2024-03-22T07:15:59.289 INFO:tasks.workunit.client.0.smithi063.stdout:0/731: creat da/de/d17/d25/d3c/d5f/fe8 x:0 0 0 2024-03-22T07:15:59.291 INFO:tasks.workunit.client.0.smithi063.stdout:1/654: creat d4/df/d13/d2b/d6c/da6/fe4 x:0 0 0 2024-03-22T07:15:59.291 INFO:tasks.workunit.client.0.smithi063.stdout:1/655: write d4/df/d13/f3c [256120,966] 0 2024-03-22T07:15:59.291 INFO:tasks.workunit.client.0.smithi063.stdout:1/656: fdatasync d4/d14/f23 0 2024-03-22T07:15:59.291 INFO:tasks.workunit.client.0.smithi063.stdout:1/657: getdents d4/df/d13/d39/d6f/dd9 0 2024-03-22T07:15:59.291 INFO:tasks.workunit.client.0.smithi063.stdout:1/658: fdatasync d4/d71/d88/fb9 0 2024-03-22T07:15:59.292 INFO:tasks.workunit.client.0.smithi063.stdout:0/732: creat da/de/d60/d56/d77/fe9 x:0 0 0 2024-03-22T07:15:59.292 INFO:tasks.workunit.client.0.smithi063.stdout:0/733: stat da/c10 0 2024-03-22T07:15:59.296 INFO:tasks.workunit.client.0.smithi063.stdout:5/495: write d8/d13/d1e/d31/f46 [1801906,20426] 0 2024-03-22T07:15:59.296 INFO:tasks.workunit.client.0.smithi063.stdout:1/659: rmdir d4/d5a/d91/db6 39 2024-03-22T07:15:59.296 INFO:tasks.workunit.client.0.smithi063.stdout:1/660: write d4/f1f [4620360,74224] 0 2024-03-22T07:15:59.298 INFO:tasks.workunit.client.1.smithi069.stdout:4/349: dwrite d6/dc/d1d/d31/d1f/d58/f69 [0,4194304] 0 2024-03-22T07:15:59.298 INFO:tasks.workunit.client.1.smithi069.stdout:4/350: readlink d6/d16/d3a/l50 0 2024-03-22T07:15:59.299 INFO:tasks.workunit.client.1.smithi069.stdout:4/351: truncate d6/d12/f33 985484 0 2024-03-22T07:15:59.301 INFO:tasks.workunit.client.1.smithi069.stdout:4/352: rename d6/d12/c25 to d6/dc/d1d/d31/c6b 0 2024-03-22T07:15:59.301 INFO:tasks.workunit.client.0.smithi063.stdout:5/496: mkdir d8/d13/d28/d66/d8b 0 2024-03-22T07:15:59.302 INFO:tasks.workunit.client.1.smithi069.stdout:4/353: symlink d6/d12/d18/d21/l6c 0 2024-03-22T07:15:59.306 INFO:tasks.workunit.client.0.smithi063.stdout:2/519: dwrite d0/de/f38 [0,4194304] 0 2024-03-22T07:15:59.306 INFO:tasks.workunit.client.0.smithi063.stdout:5/497: rename d8/d13/d24/f78 to d8/d13/d1e/d3c/f8c 0 2024-03-22T07:15:59.306 INFO:tasks.workunit.client.0.smithi063.stdout:5/498: read d8/f12 [3327136,107680] 0 2024-03-22T07:15:59.309 INFO:tasks.workunit.client.1.smithi069.stdout:8/378: dwrite d1/de/d6c/d73/f74 [0,4194304] 0 2024-03-22T07:15:59.309 INFO:tasks.workunit.client.1.smithi069.stdout:8/379: fdatasync d1/de/d6c/d4e/d5b/f76 0 2024-03-22T07:15:59.311 INFO:tasks.workunit.client.0.smithi063.stdout:2/520: unlink d0/de/d35/d36/f7c 0 2024-03-22T07:15:59.311 INFO:tasks.workunit.client.0.smithi063.stdout:2/521: write d0/de/d12/f1f [2253213,79067] 0 2024-03-22T07:15:59.311 INFO:tasks.workunit.client.0.smithi063.stdout:2/522: readlink d0/de/d1b/l75 0 2024-03-22T07:15:59.311 INFO:tasks.workunit.client.0.smithi063.stdout:2/523: read - d0/de/d1b/d63/d6e/f72 zero size 2024-03-22T07:15:59.311 INFO:tasks.workunit.client.0.smithi063.stdout:2/524: write d0/de/d12/d20/f29 [1508622,35307] 0 2024-03-22T07:15:59.322 INFO:tasks.workunit.client.0.smithi063.stdout:2/525: dread d0/de/d1b/d32/f3b [0,4194304] 0 2024-03-22T07:15:59.323 INFO:tasks.workunit.client.0.smithi063.stdout:2/526: symlink d0/de/d58/la5 0 2024-03-22T07:15:59.323 INFO:tasks.workunit.client.0.smithi063.stdout:2/527: getdents d0/de/d58/d7f/d97/da2 0 2024-03-22T07:15:59.323 INFO:tasks.workunit.client.1.smithi069.stdout:4/354: write d6/d12/d18/d21/f2f [3595528,99027] 0 2024-03-22T07:15:59.331 INFO:tasks.workunit.client.1.smithi069.stdout:0/338: dwrite d3/d7/d15/f46 [0,4194304] 0 2024-03-22T07:15:59.331 INFO:tasks.workunit.client.0.smithi063.stdout:8/638: dwrite d0/d1f/f98 [0,4194304] 0 2024-03-22T07:15:59.331 INFO:tasks.workunit.client.0.smithi063.stdout:8/639: chown d0/d26/d2d/d83/lb0 1338863 1 2024-03-22T07:15:59.331 INFO:tasks.workunit.client.0.smithi063.stdout:3/584: dwrite da/d63/d65/f77 [8388608,4194304] 0 2024-03-22T07:15:59.332 INFO:tasks.workunit.client.1.smithi069.stdout:9/252: dwrite d2/dd/f26 [0,4194304] 0 2024-03-22T07:15:59.332 INFO:tasks.workunit.client.0.smithi063.stdout:1/661: dwrite d4/df/d13/d39/f5f [0,4194304] 0 2024-03-22T07:15:59.333 INFO:tasks.workunit.client.1.smithi069.stdout:6/188: dwrite d4/dd/d12/d13/f23 [0,4194304] 0 2024-03-22T07:15:59.333 INFO:tasks.workunit.client.1.smithi069.stdout:0/339: creat d3/d7/d15/d29/d5a/f6c x:0 0 0 2024-03-22T07:15:59.333 INFO:tasks.workunit.client.1.smithi069.stdout:0/340: chown d3/d7/d24/l44 3 1 2024-03-22T07:15:59.335 INFO:tasks.workunit.client.0.smithi063.stdout:8/640: rmdir d0/d1b/d28/d3e/db2/dbf 0 2024-03-22T07:15:59.336 INFO:tasks.workunit.client.1.smithi069.stdout:6/189: dread d4/dd/d12/f34 [0,4194304] 0 2024-03-22T07:15:59.339 INFO:tasks.workunit.client.1.smithi069.stdout:0/341: mkdir d3/d7/d17/d1c/d6d 0 2024-03-22T07:15:59.340 INFO:tasks.workunit.client.1.smithi069.stdout:6/190: getdents d4/dd/d12/d13 0 2024-03-22T07:15:59.340 INFO:tasks.workunit.client.1.smithi069.stdout:6/191: fsync d4/dd/d12/f32 0 2024-03-22T07:15:59.343 INFO:tasks.workunit.client.0.smithi063.stdout:1/662: symlink d4/d14/d57/le5 0 2024-03-22T07:15:59.344 INFO:tasks.workunit.client.1.smithi069.stdout:6/192: dread d4/f14 [0,4194304] 0 2024-03-22T07:15:59.346 INFO:tasks.workunit.client.0.smithi063.stdout:3/585: dread da/d16/d26/f36 [0,4194304] 0 2024-03-22T07:15:59.346 INFO:tasks.workunit.client.0.smithi063.stdout:3/586: write da/f71 [1920267,4191] 0 2024-03-22T07:15:59.347 INFO:tasks.workunit.client.1.smithi069.stdout:3/246: sync 2024-03-22T07:15:59.347 INFO:tasks.workunit.client.1.smithi069.stdout:3/247: dread - d5/d9/d26/f4c zero size 2024-03-22T07:15:59.348 INFO:tasks.workunit.client.0.smithi063.stdout:8/641: link d0/d26/l6c d0/d24/d33/d4c/ld5 0 2024-03-22T07:15:59.348 INFO:tasks.workunit.client.0.smithi063.stdout:8/642: dread - d0/d26/d2d/d83/dbc/da1/fb3 zero size 2024-03-22T07:15:59.353 INFO:tasks.workunit.client.0.smithi063.stdout:3/587: unlink da/d16/f25 0 2024-03-22T07:15:59.353 INFO:tasks.workunit.client.0.smithi063.stdout:3/588: write da/d16/d26/f76 [4766505,104313] 0 2024-03-22T07:15:59.354 INFO:tasks.workunit.client.0.smithi063.stdout:3/589: truncate da/d63/d65/f99 2805018 0 2024-03-22T07:15:59.354 INFO:tasks.workunit.client.0.smithi063.stdout:3/590: stat da/d63/d65/c74 0 2024-03-22T07:15:59.354 INFO:tasks.workunit.client.0.smithi063.stdout:1/663: read d4/df/d13/d39/fa0 [965520,89199] 0 2024-03-22T07:15:59.355 INFO:tasks.workunit.client.0.smithi063.stdout:1/664: chown d4/d5a/d91/lb4 56 1 2024-03-22T07:15:59.355 INFO:tasks.workunit.client.0.smithi063.stdout:3/591: write da/d16/d26/f48 [910504,114452] 0 2024-03-22T07:15:59.356 INFO:tasks.workunit.client.0.smithi063.stdout:1/665: mknod d4/df/d13/d39/d6f/ce6 0 2024-03-22T07:15:59.356 INFO:tasks.workunit.client.0.smithi063.stdout:1/666: chown d4/df/d13/f34 8337353 1 2024-03-22T07:15:59.356 INFO:tasks.workunit.client.0.smithi063.stdout:1/667: chown d4/df/d13/c28 788 1 2024-03-22T07:15:59.359 INFO:tasks.workunit.client.0.smithi063.stdout:3/592: rename da/d16/d3f/c6e to da/d63/d65/d4b/cc3 0 2024-03-22T07:15:59.360 INFO:tasks.workunit.client.0.smithi063.stdout:3/593: read da/d16/d26/d69/f8f [165468,46776] 0 2024-03-22T07:15:59.360 INFO:tasks.workunit.client.0.smithi063.stdout:3/594: write da/d63/d65/fac [117842,83696] 0 2024-03-22T07:15:59.361 INFO:tasks.workunit.client.0.smithi063.stdout:1/668: mkdir d4/df/de7 0 2024-03-22T07:15:59.365 INFO:tasks.workunit.client.1.smithi069.stdout:4/355: dwrite d6/dc/d23/f5c [0,4194304] 0 2024-03-22T07:15:59.365 INFO:tasks.workunit.client.1.smithi069.stdout:4/356: creat d6/d16/f6d x:0 0 0 2024-03-22T07:15:59.365 INFO:tasks.workunit.client.1.smithi069.stdout:4/357: write d6/d16/f6d [1032219,37771] 0 2024-03-22T07:15:59.365 INFO:tasks.workunit.client.0.smithi063.stdout:1/669: link d4/df/c8f d4/d71/d88/ce8 0 2024-03-22T07:15:59.366 INFO:tasks.workunit.client.0.smithi063.stdout:1/670: fdatasync d4/d8b/f97 0 2024-03-22T07:15:59.366 INFO:tasks.workunit.client.0.smithi063.stdout:8/643: write d0/d26/f92 [2734023,76010] 0 2024-03-22T07:15:59.366 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.366+0000 7f89f5d94700 1 -- 172.21.15.63:0/950638700 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f89f004bed0 con 0x7f89f00fbcd0 2024-03-22T07:15:59.367 INFO:tasks.workunit.client.0.smithi063.stdout:8/644: mknod d0/d24/d33/d4c/cd6 0 2024-03-22T07:15:59.367 INFO:tasks.workunit.client.0.smithi063.stdout:8/645: readlink d0/d1b/d28/la0 0 2024-03-22T07:15:59.367 INFO:tasks.workunit.client.0.smithi063.stdout:8/646: fdatasync d0/f1a 0 2024-03-22T07:15:59.367 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.367+0000 7f89d7fff700 1 -- 172.21.15.63:0/950638700 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f89e00126b0 con 0x7f89f00fbcd0 2024-03-22T07:15:59.368 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:15:59.368 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:15:59.368 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:15:59.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:15:59.368 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:15:59.368 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T07:15:59.368 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-22T07:15:59.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:15:59.368 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:15:59.368 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:15:59.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:15:59.368 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:15:59.368 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:15:59.368 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:15:59.369 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:15:59.369 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 13, 2024-03-22T07:15:59.369 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-22T07:15:59.369 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:15:59.369 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:15:59.370 INFO:tasks.workunit.client.0.smithi063.stdout:8/647: mknod d0/d24/d5b/cd7 0 2024-03-22T07:15:59.370 INFO:tasks.workunit.client.0.smithi063.stdout:8/648: truncate d0/d1b/d28/f43 950924 0 2024-03-22T07:15:59.370 INFO:tasks.workunit.client.0.smithi063.stdout:8/649: creat d0/d1b/d28/d5f/fd8 x:0 0 0 2024-03-22T07:15:59.370 INFO:tasks.workunit.client.0.smithi063.stdout:8/650: dread d0/d1b/d28/d7b/fa4 [0,4194304] 0 2024-03-22T07:15:59.370 INFO:tasks.workunit.client.0.smithi063.stdout:8/651: dread - d0/d1b/d28/d3e/d97/faa zero size 2024-03-22T07:15:59.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.370+0000 7f89f5d94700 1 -- 172.21.15.63:0/950638700 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f89d806b060 msgr2=0x7f89d806d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:59.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.370+0000 7f89f5d94700 1 --2- 172.21.15.63:0/950638700 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f89d806b060 0x7f89d806d510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009fd0 tx=0x7f89e4009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:59.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.370+0000 7f89f5d94700 1 -- 172.21.15.63:0/950638700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89f00fbcd0 msgr2=0x7f89f010fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:59.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.370+0000 7f89f5d94700 1 --2- 172.21.15.63:0/950638700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89f00fbcd0 0x7f89f010fe60 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f89e000c8c0 tx=0x7f89e00044c0 comp rx=0 tx=0).stop 2024-03-22T07:15:59.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.370+0000 7f89f5d94700 1 -- 172.21.15.63:0/950638700 shutdown_connections 2024-03-22T07:15:59.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.370+0000 7f89f5d94700 1 --2- 172.21.15.63:0/950638700 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7f89d806b060 0x7f89d806d510 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:59.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.370+0000 7f89f5d94700 1 --2- 172.21.15.63:0/950638700 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f89f00fbcd0 0x7f89f010fe60 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:59.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.370+0000 7f89f5d94700 1 --2- 172.21.15.63:0/950638700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f89f01103a0 0x7f89f0111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:59.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.370+0000 7f89f5d94700 1 -- 172.21.15.63:0/950638700 >> 172.21.15.63:0/950638700 conn(0x7f89f00f7430 msgr2=0x7f89f00f8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:59.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.370+0000 7f89f5d94700 1 -- 172.21.15.63:0/950638700 shutdown_connections 2024-03-22T07:15:59.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.370+0000 7f89f5d94700 1 -- 172.21.15.63:0/950638700 wait complete. 2024-03-22T07:15:59.372 INFO:tasks.workunit.client.0.smithi063.stdout:8/652: mkdir d0/d24/d33/dd9 0 2024-03-22T07:15:59.372 INFO:tasks.workunit.client.0.smithi063.stdout:8/653: truncate d0/d24/fb9 1126948 0 2024-03-22T07:15:59.372 INFO:tasks.workunit.client.0.smithi063.stdout:8/654: write d0/d1f/f53 [4785151,99138] 0 2024-03-22T07:15:59.372 INFO:tasks.workunit.client.0.smithi063.stdout:8/655: dread d0/d24/d33/d4c/f7d [0,4194304] 0 2024-03-22T07:15:59.373 INFO:tasks.workunit.client.1.smithi069.stdout:4/358: dread d6/dc/d1d/d31/d1f/f39 [0,4194304] 0 2024-03-22T07:15:59.373 INFO:tasks.workunit.client.1.smithi069.stdout:4/359: link d6/dc/d1d/c4e d6/d12/d18/c6e 0 2024-03-22T07:15:59.373 INFO:tasks.workunit.client.1.smithi069.stdout:4/360: creat d6/d12/f6f x:0 0 0 2024-03-22T07:15:59.374 INFO:tasks.workunit.client.0.smithi063.stdout:2/528: dwrite d0/f41 [0,4194304] 0 2024-03-22T07:15:59.374 INFO:tasks.workunit.client.0.smithi063.stdout:2/529: stat d0/de/d35/d36/c7a 0 2024-03-22T07:15:59.374 INFO:tasks.workunit.client.0.smithi063.stdout:2/530: write d0/de/d12/d20/f70 [619288,109219] 0 2024-03-22T07:15:59.374 INFO:tasks.workunit.client.0.smithi063.stdout:8/656: creat d0/d1b/d28/d7b/d45/fda x:0 0 0 2024-03-22T07:15:59.375 INFO:tasks.workunit.client.0.smithi063.stdout:1/671: dread d4/df/f11 [0,4194304] 0 2024-03-22T07:15:59.378 INFO:tasks.workunit.client.0.smithi063.stdout:1/672: rename d4/df/d13/f92 to d4/df/d13/d39/d6f/dd9/fe9 0 2024-03-22T07:15:59.378 INFO:tasks.workunit.client.0.smithi063.stdout:8/657: read d0/d26/d2d/f4d [145848,68017] 0 2024-03-22T07:15:59.380 INFO:tasks.workunit.client.0.smithi063.stdout:1/673: creat d4/d71/dbb/dc0/fea x:0 0 0 2024-03-22T07:15:59.380 INFO:tasks.workunit.client.0.smithi063.stdout:1/674: readlink d4/df/l68 0 2024-03-22T07:15:59.380 INFO:tasks.workunit.client.0.smithi063.stdout:1/675: chown d4/d71/dbb 1 1 2024-03-22T07:15:59.382 INFO:tasks.workunit.client.1.smithi069.stdout:9/253: dwrite d2/f27 [0,4194304] 0 2024-03-22T07:15:59.382 INFO:tasks.workunit.client.1.smithi069.stdout:9/254: readlink d2/l46 0 2024-03-22T07:15:59.385 INFO:tasks.workunit.client.1.smithi069.stdout:9/255: mknod d2/d31/c50 0 2024-03-22T07:15:59.387 INFO:tasks.workunit.client.1.smithi069.stdout:4/361: dwrite d6/d16/f57 [0,4194304] 0 2024-03-22T07:15:59.387 INFO:tasks.workunit.client.1.smithi069.stdout:4/362: creat d6/d12/f70 x:0 0 0 2024-03-22T07:15:59.389 INFO:tasks.workunit.client.1.smithi069.stdout:0/342: dwrite d3/d7/f4a [0,4194304] 0 2024-03-22T07:15:59.390 INFO:tasks.workunit.client.1.smithi069.stdout:3/248: dwrite d5/f18 [4194304,4194304] 0 2024-03-22T07:15:59.390 INFO:tasks.workunit.client.1.smithi069.stdout:6/193: dwrite d4/dd/d12/f32 [0,4194304] 0 2024-03-22T07:15:59.390 INFO:tasks.workunit.client.1.smithi069.stdout:6/194: read - d4/dd/d12/d2b/f30 zero size 2024-03-22T07:15:59.393 INFO:tasks.workunit.client.0.smithi063.stdout:2/531: write d0/de/d12/d20/f82 [794192,119935] 0 2024-03-22T07:15:59.394 INFO:tasks.workunit.client.1.smithi069.stdout:0/343: mkdir d3/d7/d15/d29/d6e 0 2024-03-22T07:15:59.394 INFO:tasks.workunit.client.1.smithi069.stdout:0/344: chown d3/f13 23455 1 2024-03-22T07:15:59.394 INFO:tasks.workunit.client.1.smithi069.stdout:0/345: stat d3/da/c23 0 2024-03-22T07:15:59.397 INFO:tasks.workunit.client.1.smithi069.stdout:3/249: mkdir d5/d9/d26/d3f/d35/d4f 0 2024-03-22T07:15:59.398 INFO:tasks.workunit.client.1.smithi069.stdout:9/256: write d2/dd/d3c/f20 [485106,26068] 0 2024-03-22T07:15:59.401 INFO:tasks.workunit.client.0.smithi063.stdout:3/595: write da/d63/d65/f99 [1258046,83195] 0 2024-03-22T07:15:59.401 INFO:tasks.workunit.client.1.smithi069.stdout:0/346: creat d3/d7/d24/d6a/f6f x:0 0 0 2024-03-22T07:15:59.401 INFO:tasks.workunit.client.1.smithi069.stdout:0/347: write d3/d7/d24/f34 [634021,111073] 0 2024-03-22T07:15:59.402 INFO:tasks.workunit.client.1.smithi069.stdout:0/348: fdatasync d3/d7/d15/f1b 0 2024-03-22T07:15:59.402 INFO:tasks.workunit.client.1.smithi069.stdout:0/349: chown d3/d7/d5e 83663 1 2024-03-22T07:15:59.402 INFO:tasks.workunit.client.1.smithi069.stdout:0/350: chown d3/c53 294548 1 2024-03-22T07:15:59.402 INFO:tasks.workunit.client.1.smithi069.stdout:0/351: fdatasync d3/d7/d15/d1f/f41 0 2024-03-22T07:15:59.402 INFO:tasks.workunit.client.1.smithi069.stdout:3/250: rename d5/d9/d26/f2e to d5/d9/d26/d3f/d35/f50 0 2024-03-22T07:15:59.403 INFO:tasks.workunit.client.1.smithi069.stdout:6/195: rmdir d4/d7 39 2024-03-22T07:15:59.404 INFO:tasks.workunit.client.1.smithi069.stdout:0/352: creat d3/d7/d15/d29/d6e/f70 x:0 0 0 2024-03-22T07:15:59.404 INFO:tasks.workunit.client.1.smithi069.stdout:0/353: fdatasync d3/da/f3b 0 2024-03-22T07:15:59.405 INFO:tasks.workunit.client.1.smithi069.stdout:3/251: mkdir d5/d9/d26/d36/d41/d51 0 2024-03-22T07:15:59.405 INFO:tasks.workunit.client.0.smithi063.stdout:3/596: mknod da/d16/cc4 0 2024-03-22T07:15:59.406 INFO:tasks.workunit.client.1.smithi069.stdout:9/257: getdents d2/dd/d3c/d48 0 2024-03-22T07:15:59.406 INFO:tasks.workunit.client.1.smithi069.stdout:0/354: mkdir d3/d71 0 2024-03-22T07:15:59.407 INFO:tasks.workunit.client.0.smithi063.stdout:3/597: rmdir da/d63/d65/d81/da5/db8 0 2024-03-22T07:15:59.407 INFO:tasks.workunit.client.1.smithi069.stdout:3/252: creat d5/d9/d26/d36/f52 x:0 0 0 2024-03-22T07:15:59.407 INFO:tasks.workunit.client.0.smithi063.stdout:2/532: dread d0/de/d1b/d63/f6c [0,4194304] 0 2024-03-22T07:15:59.408 INFO:tasks.workunit.client.1.smithi069.stdout:6/196: dread d4/f14 [0,4194304] 0 2024-03-22T07:15:59.408 INFO:tasks.workunit.client.1.smithi069.stdout:9/258: mkdir d2/d51 0 2024-03-22T07:15:59.409 INFO:tasks.workunit.client.0.smithi063.stdout:3/598: unlink da/d16/f6d 0 2024-03-22T07:15:59.409 INFO:tasks.workunit.client.0.smithi063.stdout:3/599: chown da/cc 21 1 2024-03-22T07:15:59.409 INFO:tasks.workunit.client.0.smithi063.stdout:3/600: write da/d63/d65/d92/d96/f9a [905112,76141] 0 2024-03-22T07:15:59.409 INFO:tasks.workunit.client.1.smithi069.stdout:0/355: rmdir d3/d7/d17/d1c/d6d 0 2024-03-22T07:15:59.410 INFO:tasks.workunit.client.0.smithi063.stdout:2/533: dread d0/de/d12/d20/f2c [0,4194304] 0 2024-03-22T07:15:59.410 INFO:tasks.workunit.client.0.smithi063.stdout:2/534: chown d0/de/d12/d20/c45 39676027 1 2024-03-22T07:15:59.411 INFO:tasks.workunit.client.0.smithi063.stdout:3/601: mkdir da/d63/d65/d92/dc5 0 2024-03-22T07:15:59.411 INFO:tasks.workunit.client.0.smithi063.stdout:3/602: write da/d63/d65/d92/fb0 [398255,55044] 0 2024-03-22T07:15:59.411 INFO:tasks.workunit.client.0.smithi063.stdout:3/603: write da/f71 [223107,89765] 0 2024-03-22T07:15:59.411 INFO:tasks.workunit.client.0.smithi063.stdout:3/604: getdents da/d63/d65/db9 0 2024-03-22T07:15:59.411 INFO:tasks.workunit.client.0.smithi063.stdout:3/605: truncate da/d16/d26/fbd 349304 0 2024-03-22T07:15:59.411 INFO:tasks.workunit.client.0.smithi063.stdout:3/606: write da/f71 [1668961,43857] 0 2024-03-22T07:15:59.412 INFO:tasks.workunit.client.1.smithi069.stdout:6/197: rename d4/dd/d12/d13/l21 to d4/d7/da/l38 0 2024-03-22T07:15:59.412 INFO:tasks.workunit.client.1.smithi069.stdout:6/198: mknod d4/dd/d12/c39 0 2024-03-22T07:15:59.412 INFO:tasks.workunit.client.1.smithi069.stdout:6/199: write d4/fb [8643637,117088] 0 2024-03-22T07:15:59.412 INFO:tasks.workunit.client.1.smithi069.stdout:6/200: truncate d4/d7/f2c 4594669 0 2024-03-22T07:15:59.413 INFO:tasks.workunit.client.0.smithi063.stdout:2/535: creat d0/de/d12/d5a/fa6 x:0 0 0 2024-03-22T07:15:59.413 INFO:tasks.workunit.client.0.smithi063.stdout:2/536: truncate d0/de/f14 1258835 0 2024-03-22T07:15:59.413 INFO:tasks.workunit.client.1.smithi069.stdout:6/201: unlink d4/d7/f26 0 2024-03-22T07:15:59.413 INFO:tasks.workunit.client.0.smithi063.stdout:8/658: dwrite d0/d1b/f6e [0,4194304] 0 2024-03-22T07:15:59.414 INFO:tasks.workunit.client.1.smithi069.stdout:4/363: dwrite d6/dc/d1d/d31/d1f/d3f/f64 [0,4194304] 0 2024-03-22T07:15:59.415 INFO:tasks.workunit.client.1.smithi069.stdout:4/364: creat d6/d16/f71 x:0 0 0 2024-03-22T07:15:59.415 INFO:tasks.workunit.client.1.smithi069.stdout:4/365: readlink d6/d12/d18/l19 0 2024-03-22T07:15:59.415 INFO:tasks.workunit.client.1.smithi069.stdout:4/366: fdatasync d6/dc/d1d/d31/d1f/d3f/f64 0 2024-03-22T07:15:59.415 INFO:tasks.workunit.client.0.smithi063.stdout:1/676: dwrite d4/d14/f62 [0,4194304] 0 2024-03-22T07:15:59.415 INFO:tasks.workunit.client.0.smithi063.stdout:1/677: write d4/df/d13/d2b/d3a/f70 [813281,77216] 0 2024-03-22T07:15:59.416 INFO:tasks.workunit.client.1.smithi069.stdout:0/356: dread d3/d7/d17/f56 [0,4194304] 0 2024-03-22T07:15:59.416 INFO:tasks.workunit.client.1.smithi069.stdout:0/357: truncate d3/d7/fb 1933506 0 2024-03-22T07:15:59.418 INFO:tasks.workunit.client.0.smithi063.stdout:3/607: mkdir da/d63/d65/d83/dc6 0 2024-03-22T07:15:59.418 INFO:tasks.workunit.client.0.smithi063.stdout:3/608: chown da/d16/d26/fbd 41557 1 2024-03-22T07:15:59.419 INFO:tasks.workunit.client.1.smithi069.stdout:4/367: symlink d6/d16/l72 0 2024-03-22T07:15:59.419 INFO:tasks.workunit.client.1.smithi069.stdout:4/368: chown d6/dc/d1d/d31/d1f/f5b 1 1 2024-03-22T07:15:59.419 INFO:tasks.workunit.client.1.smithi069.stdout:4/369: creat d6/d12/f73 x:0 0 0 2024-03-22T07:15:59.421 INFO:tasks.workunit.client.1.smithi069.stdout:3/253: dread d5/d9/d26/d3f/f21 [0,4194304] 0 2024-03-22T07:15:59.421 INFO:tasks.workunit.client.1.smithi069.stdout:3/254: readlink d5/l2f 0 2024-03-22T07:15:59.421 INFO:tasks.workunit.client.1.smithi069.stdout:3/255: chown d5/f4a 15 1 2024-03-22T07:15:59.423 INFO:tasks.workunit.client.1.smithi069.stdout:0/358: symlink d3/d7/d15/l72 0 2024-03-22T07:15:59.424 INFO:tasks.workunit.client.0.smithi063.stdout:8/659: dread d0/d26/f27 [0,4194304] 0 2024-03-22T07:15:59.425 INFO:tasks.workunit.client.0.smithi063.stdout:8/660: write d0/d26/d2d/d59/f87 [2003699,24988] 0 2024-03-22T07:15:59.425 INFO:tasks.workunit.client.1.smithi069.stdout:3/256: mknod d5/d9/db/c53 0 2024-03-22T07:15:59.425 INFO:tasks.workunit.client.1.smithi069.stdout:0/359: getdents d3/d7/d24 0 2024-03-22T07:15:59.427 INFO:tasks.workunit.client.1.smithi069.stdout:3/257: mkdir d5/d9/d26/d3f/d54 0 2024-03-22T07:15:59.427 INFO:tasks.workunit.client.1.smithi069.stdout:3/258: write d5/d9/f3a [618317,23307] 0 2024-03-22T07:15:59.428 INFO:tasks.workunit.client.1.smithi069.stdout:0/360: symlink d3/d7/d24/d6a/l73 0 2024-03-22T07:15:59.428 INFO:tasks.workunit.client.1.smithi069.stdout:4/370: write d6/dc/d1d/d31/d1f/d58/f69 [1459365,13248] 0 2024-03-22T07:15:59.428 INFO:tasks.workunit.client.1.smithi069.stdout:4/371: rename d6/dc/d1d/d31/d1f to d6/dc/d1d/d31/d1f/d58/d74 22 2024-03-22T07:15:59.429 INFO:tasks.workunit.client.0.smithi063.stdout:1/678: rmdir d4/df/d13/d2b/d6c 39 2024-03-22T07:15:59.429 INFO:tasks.workunit.client.0.smithi063.stdout:1/679: write d4/df/d13/d2b/d86/fa3 [866549,107970] 0 2024-03-22T07:15:59.429 INFO:tasks.workunit.client.0.smithi063.stdout:1/680: chown d4/d71/d88/ce8 68049 1 2024-03-22T07:15:59.429 INFO:tasks.workunit.client.0.smithi063.stdout:1/681: write d4/df/d13/d2b/d3a/f81 [4564017,54498] 0 2024-03-22T07:15:59.430 INFO:tasks.workunit.client.1.smithi069.stdout:3/259: symlink d5/d9/d26/d3f/l55 0 2024-03-22T07:15:59.430 INFO:tasks.workunit.client.1.smithi069.stdout:4/372: write d6/f9 [13248,100543] 0 2024-03-22T07:15:59.430 INFO:tasks.workunit.client.1.smithi069.stdout:0/361: mknod d3/d7/d24/d6a/c74 0 2024-03-22T07:15:59.430 INFO:tasks.workunit.client.1.smithi069.stdout:0/362: write d3/f13 [4055542,43769] 0 2024-03-22T07:15:59.433 INFO:tasks.workunit.client.0.smithi063.stdout:8/661: symlink d0/d24/d33/dd9/ldb 0 2024-03-22T07:15:59.433 INFO:tasks.workunit.client.0.smithi063.stdout:8/662: write d0/d24/d33/d4c/f9c [44476,106871] 0 2024-03-22T07:15:59.433 INFO:tasks.workunit.client.1.smithi069.stdout:4/373: symlink d6/dc/d1d/d31/l75 0 2024-03-22T07:15:59.435 INFO:tasks.workunit.client.1.smithi069.stdout:4/374: unlink d6/d12/d18/d21/c38 0 2024-03-22T07:15:59.435 INFO:tasks.workunit.client.1.smithi069.stdout:4/375: creat d6/d12/f76 x:0 0 0 2024-03-22T07:15:59.435 INFO:tasks.workunit.client.1.smithi069.stdout:4/376: dread - d6/dc/d1d/d55/f5f zero size 2024-03-22T07:15:59.435 INFO:tasks.workunit.client.0.smithi063.stdout:1/682: rename d4/df/d13/d2b/d86/f87 to d4/df/d13/d39/d6f/dd9/feb 0 2024-03-22T07:15:59.435 INFO:tasks.workunit.client.0.smithi063.stdout:1/683: write d4/df/d13/d2b/d6c/d85/fc7 [1005885,120022] 0 2024-03-22T07:15:59.436 INFO:tasks.workunit.client.0.smithi063.stdout:8/663: rmdir d0/d1b/d28/d5f 39 2024-03-22T07:15:59.440 INFO:tasks.workunit.client.1.smithi069.stdout:9/259: dwrite d2/dd/d3c/d48/f4d [0,4194304] 0 2024-03-22T07:15:59.441 INFO:tasks.workunit.client.1.smithi069.stdout:3/260: read d5/d9/db/f3d [2730593,16214] 0 2024-03-22T07:15:59.442 INFO:tasks.workunit.client.0.smithi063.stdout:2/537: dwrite d0/f23 [0,4194304] 0 2024-03-22T07:15:59.444 INFO:tasks.workunit.client.1.smithi069.stdout:6/202: getdents d4/d7 0 2024-03-22T07:15:59.444 INFO:tasks.workunit.client.1.smithi069.stdout:6/203: creat d4/dd/d12/d2b/f3a x:0 0 0 2024-03-22T07:15:59.446 INFO:tasks.workunit.client.0.smithi063.stdout:8/664: write d0/d24/d33/d4c/f7d [390261,54696] 0 2024-03-22T07:15:59.446 INFO:tasks.workunit.client.0.smithi063.stdout:1/684: dread d4/df/f44 [0,4194304] 0 2024-03-22T07:15:59.446 INFO:tasks.workunit.client.0.smithi063.stdout:1/685: fsync d4/d14/f46 0 2024-03-22T07:15:59.449 INFO:tasks.workunit.client.1.smithi069.stdout:4/377: fsync d6/d12/f73 0 2024-03-22T07:15:59.450 INFO:tasks.workunit.client.1.smithi069.stdout:4/378: creat d6/dc/d23/f77 x:0 0 0 2024-03-22T07:15:59.452 INFO:tasks.workunit.client.0.smithi063.stdout:3/609: rmdir da/d16 39 2024-03-22T07:15:59.455 INFO:tasks.workunit.client.1.smithi069.stdout:3/261: mkdir d5/d9/d26/d3f/d56 0 2024-03-22T07:15:59.456 INFO:tasks.workunit.client.1.smithi069.stdout:3/262: readlink d5/l17 0 2024-03-22T07:15:59.456 INFO:tasks.workunit.client.1.smithi069.stdout:3/263: dread - d5/d9/f48 zero size 2024-03-22T07:15:59.461 INFO:tasks.workunit.client.1.smithi069.stdout:4/379: creat d6/d16/d68/f78 x:0 0 0 2024-03-22T07:15:59.461 INFO:tasks.workunit.client.1.smithi069.stdout:4/380: write d6/dc/f37 [2875465,130928] 0 2024-03-22T07:15:59.461 INFO:tasks.workunit.client.1.smithi069.stdout:3/264: truncate d5/d9/d26/d3f/f3e 4119262 0 2024-03-22T07:15:59.463 INFO:tasks.workunit.client.1.smithi069.stdout:3/265: symlink d5/d9/l57 0 2024-03-22T07:15:59.464 INFO:tasks.workunit.client.1.smithi069.stdout:3/266: rmdir d5/d9/d23/d46 0 2024-03-22T07:15:59.465 INFO:tasks.workunit.client.1.smithi069.stdout:3/267: creat d5/d9/d26/d3f/d35/f58 x:0 0 0 2024-03-22T07:15:59.465 INFO:tasks.workunit.client.0.smithi063.stdout:2/538: creat d0/de/d58/d7f/fa7 x:0 0 0 2024-03-22T07:15:59.466 INFO:tasks.workunit.client.0.smithi063.stdout:8/665: mknod d0/d1b/d28/d3e/db2/dbd/dd0/cdc 0 2024-03-22T07:15:59.468 INFO:tasks.workunit.client.1.smithi069.stdout:3/268: symlink d5/d9/d23/l59 0 2024-03-22T07:15:59.469 INFO:tasks.workunit.client.0.smithi063.stdout:3/610: mkdir da/d16/d3f/dc7 0 2024-03-22T07:15:59.469 INFO:tasks.workunit.client.0.smithi063.stdout:3/611: chown da/d16/d89/l8d 158 1 2024-03-22T07:15:59.469 INFO:tasks.workunit.client.0.smithi063.stdout:2/539: symlink d0/de/d58/d7f/d97/da2/la8 0 2024-03-22T07:15:59.471 INFO:tasks.workunit.client.0.smithi063.stdout:3/612: creat da/d63/d65/d83/fc8 x:0 0 0 2024-03-22T07:15:59.473 INFO:tasks.workunit.client.1.smithi069.stdout:5/216: sync 2024-03-22T07:15:59.473 INFO:tasks.workunit.client.1.smithi069.stdout:1/296: sync 2024-03-22T07:15:59.473 INFO:tasks.workunit.client.1.smithi069.stdout:8/380: sync 2024-03-22T07:15:59.473 INFO:tasks.workunit.client.1.smithi069.stdout:2/228: sync 2024-03-22T07:15:59.473 INFO:tasks.workunit.client.1.smithi069.stdout:7/314: sync 2024-03-22T07:15:59.474 INFO:tasks.workunit.client.1.smithi069.stdout:8/381: creat d1/d1c/d48/d5e/f77 x:0 0 0 2024-03-22T07:15:59.474 INFO:tasks.workunit.client.1.smithi069.stdout:8/382: write d1/d1c/d48/d5e/f63 [796752,110339] 0 2024-03-22T07:15:59.474 INFO:tasks.workunit.client.1.smithi069.stdout:8/383: creat d1/d1c/d3e/d66/f78 x:0 0 0 2024-03-22T07:15:59.474 INFO:tasks.workunit.client.1.smithi069.stdout:8/384: creat d1/d1c/d48/d5e/f79 x:0 0 0 2024-03-22T07:15:59.477 INFO:tasks.workunit.client.1.smithi069.stdout:6/204: dwrite d4/dd/f33 [0,4194304] 0 2024-03-22T07:15:59.477 INFO:tasks.workunit.client.1.smithi069.stdout:6/205: chown d4/l2e 123495 1 2024-03-22T07:15:59.477 INFO:tasks.workunit.client.1.smithi069.stdout:6/206: rename d4/dd/d12/d13 to d4/dd/d12/d13/d3b 22 2024-03-22T07:15:59.477 INFO:tasks.workunit.client.1.smithi069.stdout:6/207: fsync d4/dd/d12/f1f 0 2024-03-22T07:15:59.477 INFO:tasks.workunit.client.1.smithi069.stdout:9/260: dwrite d2/dd/d3c/d2e/f3f [0,4194304] 0 2024-03-22T07:15:59.478 INFO:tasks.workunit.client.0.smithi063.stdout:2/540: rmdir d0/de/d1b/d92 0 2024-03-22T07:15:59.478 INFO:tasks.workunit.client.0.smithi063.stdout:2/541: chown d0/de/d1b/d32/c83 0 1 2024-03-22T07:15:59.478 INFO:tasks.workunit.client.0.smithi063.stdout:2/542: creat d0/de/d12/fa9 x:0 0 0 2024-03-22T07:15:59.479 INFO:tasks.workunit.client.0.smithi063.stdout:2/543: chown d0/de/d58/f73 41216089 1 2024-03-22T07:15:59.479 INFO:tasks.workunit.client.0.smithi063.stdout:2/544: fdatasync d0/de/d12/d20/f82 0 2024-03-22T07:15:59.479 INFO:tasks.workunit.client.0.smithi063.stdout:2/545: symlink d0/de/d35/d36/laa 0 2024-03-22T07:15:59.479 INFO:tasks.workunit.client.0.smithi063.stdout:8/666: write d0/d1b/d28/d7b/d45/f46 [965591,25277] 0 2024-03-22T07:15:59.480 INFO:tasks.workunit.client.0.smithi063.stdout:8/667: write d0/d24/f42 [4344133,63365] 0 2024-03-22T07:15:59.481 INFO:tasks.workunit.client.1.smithi069.stdout:5/217: rename d1/d7/d8/c32 to d1/d7/d8/c47 0 2024-03-22T07:15:59.482 INFO:tasks.workunit.client.1.smithi069.stdout:1/297: write d1/d13/f2d [1584377,128064] 0 2024-03-22T07:15:59.482 INFO:tasks.workunit.client.1.smithi069.stdout:1/298: fsync d1/d2a/f57 0 2024-03-22T07:15:59.482 INFO:tasks.workunit.client.1.smithi069.stdout:9/261: mknod d2/dd/d3c/d2e/c52 0 2024-03-22T07:15:59.482 INFO:tasks.workunit.client.1.smithi069.stdout:5/218: symlink d1/d7/d3d/l48 0 2024-03-22T07:15:59.482 INFO:tasks.workunit.client.1.smithi069.stdout:1/299: truncate d1/f18 2417793 0 2024-03-22T07:15:59.483 INFO:tasks.workunit.client.1.smithi069.stdout:5/219: mknod d1/d7/d3d/d3f/c49 0 2024-03-22T07:15:59.483 INFO:tasks.workunit.client.0.smithi063.stdout:8/668: rmdir d0/d24/d5b/db7 0 2024-03-22T07:15:59.483 INFO:tasks.workunit.client.0.smithi063.stdout:3/613: dread da/d16/d26/f2a [0,4194304] 0 2024-03-22T07:15:59.484 INFO:tasks.workunit.client.0.smithi063.stdout:8/669: symlink d0/d1b/d28/ldd 0 2024-03-22T07:15:59.485 INFO:tasks.workunit.client.1.smithi069.stdout:5/220: creat d1/d6/d26/d3e/f4a x:0 0 0 2024-03-22T07:15:59.485 INFO:tasks.workunit.client.1.smithi069.stdout:5/221: creat d1/d7/d8/f4b x:0 0 0 2024-03-22T07:15:59.485 INFO:tasks.workunit.client.1.smithi069.stdout:2/229: write d18/d4d/d25/f2e [74367,23175] 0 2024-03-22T07:15:59.485 INFO:tasks.workunit.client.1.smithi069.stdout:2/230: dread - d18/d4d/d27/d33/d3d/f46 zero size 2024-03-22T07:15:59.485 INFO:tasks.workunit.client.1.smithi069.stdout:5/222: rmdir d1/d7/d3d 39 2024-03-22T07:15:59.486 INFO:tasks.workunit.client.0.smithi063.stdout:8/670: mkdir d0/d1b/d4f/dde 0 2024-03-22T07:15:59.487 INFO:tasks.workunit.client.1.smithi069.stdout:5/223: creat d1/d6/d26/d33/f4c x:0 0 0 2024-03-22T07:15:59.487 INFO:tasks.workunit.client.1.smithi069.stdout:5/224: creat d1/d28/f4d x:0 0 0 2024-03-22T07:15:59.487 INFO:tasks.workunit.client.1.smithi069.stdout:5/225: creat d1/d6/d26/d33/f4e x:0 0 0 2024-03-22T07:15:59.489 INFO:tasks.workunit.client.0.smithi063.stdout:8/671: creat d0/d26/d2d/d59/dc0/fdf x:0 0 0 2024-03-22T07:15:59.490 INFO:tasks.workunit.client.0.smithi063.stdout:8/672: symlink d0/d24/d5b/le0 0 2024-03-22T07:15:59.492 INFO:tasks.workunit.client.1.smithi069.stdout:5/226: truncate d1/d7/f25 344113 0 2024-03-22T07:15:59.492 INFO:tasks.workunit.client.1.smithi069.stdout:5/227: stat d1/d7/d3d/d3f 0 2024-03-22T07:15:59.492 INFO:tasks.workunit.client.1.smithi069.stdout:5/228: stat d1/d7/d3d/d3f 0 2024-03-22T07:15:59.492 INFO:tasks.workunit.client.1.smithi069.stdout:5/229: readlink d1/d7/l3b 0 2024-03-22T07:15:59.492 INFO:tasks.workunit.client.1.smithi069.stdout:7/315: dread f8 [0,4194304] 0 2024-03-22T07:15:59.493 INFO:tasks.workunit.client.1.smithi069.stdout:8/385: dread d1/f29 [4194304,4194304] 0 2024-03-22T07:15:59.493 INFO:tasks.workunit.client.0.smithi063.stdout:8/673: link d0/c52 d0/d24/d5b/ce1 0 2024-03-22T07:15:59.493 INFO:tasks.workunit.client.0.smithi063.stdout:8/674: symlink d0/d1b/d28/d3e/d97/dd1/le2 0 2024-03-22T07:15:59.494 INFO:tasks.workunit.client.1.smithi069.stdout:7/316: truncate d9/db/d1a/f2c 2967046 0 2024-03-22T07:15:59.496 INFO:tasks.workunit.client.1.smithi069.stdout:3/269: dwrite d5/d9/f13 [0,4194304] 0 2024-03-22T07:15:59.502 INFO:tasks.workunit.client.1.smithi069.stdout:9/262: write d2/dd/d3c/d48/f4d [2503302,78321] 0 2024-03-22T07:15:59.502 INFO:tasks.workunit.client.1.smithi069.stdout:2/231: dread d18/d4d/d34/f3f [0,4194304] 0 2024-03-22T07:15:59.503 INFO:tasks.workunit.client.0.smithi063.stdout:3/614: dread da/d63/d65/f77 [0,4194304] 0 2024-03-22T07:15:59.503 INFO:tasks.workunit.client.0.smithi063.stdout:3/615: creat da/d63/d65/d81/d9b/fc9 x:0 0 0 2024-03-22T07:15:59.503 INFO:tasks.workunit.client.0.smithi063.stdout:8/675: dread d0/d24/d33/d4c/f9c [0,4194304] 0 2024-03-22T07:15:59.503 INFO:tasks.workunit.client.0.smithi063.stdout:3/616: rename da/d16/d26/f2a to da/fca 0 2024-03-22T07:15:59.504 INFO:tasks.workunit.client.1.smithi069.stdout:8/386: rmdir d1/de/d6c 39 2024-03-22T07:15:59.504 INFO:tasks.workunit.client.1.smithi069.stdout:8/387: chown d1/de/c5f 191983431 1 2024-03-22T07:15:59.504 INFO:tasks.workunit.client.1.smithi069.stdout:8/388: creat d1/d1c/d48/f7a x:0 0 0 2024-03-22T07:15:59.504 INFO:tasks.workunit.client.1.smithi069.stdout:8/389: write d1/f1d [4433200,96696] 0 2024-03-22T07:15:59.504 INFO:tasks.workunit.client.1.smithi069.stdout:3/270: dread d5/d33/f39 [0,4194304] 0 2024-03-22T07:15:59.505 INFO:tasks.workunit.client.0.smithi063.stdout:8/676: getdents d0/d26/d2d/d83/dbc/da1 0 2024-03-22T07:15:59.506 INFO:tasks.workunit.client.0.smithi063.stdout:3/617: creat da/d16/d89/d8c/dc1/fcb x:0 0 0 2024-03-22T07:15:59.506 INFO:tasks.workunit.client.0.smithi063.stdout:3/618: fdatasync da/d16/d26/f76 0 2024-03-22T07:15:59.507 INFO:tasks.workunit.client.0.smithi063.stdout:8/677: mknod d0/d24/d33/d4c/ce3 0 2024-03-22T07:15:59.507 INFO:tasks.workunit.client.0.smithi063.stdout:8/678: write d0/d1b/d28/d5f/fd3 [113729,999] 0 2024-03-22T07:15:59.508 INFO:tasks.workunit.client.0.smithi063.stdout:3/619: creat da/d16/d89/da9/fcc x:0 0 0 2024-03-22T07:15:59.508 INFO:tasks.workunit.client.0.smithi063.stdout:3/620: write da/d16/d26/f36 [1853273,79855] 0 2024-03-22T07:15:59.509 INFO:tasks.workunit.client.1.smithi069.stdout:2/232: write d18/d4d/d34/f3f [592505,59327] 0 2024-03-22T07:15:59.509 INFO:tasks.workunit.client.1.smithi069.stdout:3/271: rmdir d5/d9/d26/d36/d41/d49 0 2024-03-22T07:15:59.509 INFO:tasks.workunit.client.0.smithi063.stdout:8/679: rename d0/d24/d33/d4c/d78/c9f to d0/d26/d2d/ce4 0 2024-03-22T07:15:59.509 INFO:tasks.workunit.client.0.smithi063.stdout:8/680: write d0/d1b/fc3 [162918,54263] 0 2024-03-22T07:15:59.509 INFO:tasks.workunit.client.0.smithi063.stdout:8/681: truncate d0/d26/d2d/f4d 503633 0 2024-03-22T07:15:59.511 INFO:tasks.workunit.client.0.smithi063.stdout:2/546: dwrite d0/de/d1b/d63/f9c [0,4194304] 0 2024-03-22T07:15:59.512 INFO:tasks.workunit.client.1.smithi069.stdout:1/300: read d1/f18 [1168306,54234] 0 2024-03-22T07:15:59.512 INFO:tasks.workunit.client.0.smithi063.stdout:3/621: rename da/d16/d89/d8c/dc1/fb7 to da/d16/d3f/dc7/fcd 0 2024-03-22T07:15:59.513 INFO:tasks.workunit.client.1.smithi069.stdout:6/208: dwrite d4/f6 [0,4194304] 0 2024-03-22T07:15:59.513 INFO:tasks.workunit.client.1.smithi069.stdout:6/209: creat d4/dd/d12/d2b/f3c x:0 0 0 2024-03-22T07:15:59.513 INFO:tasks.workunit.client.1.smithi069.stdout:6/210: dread - d4/dd/d12/d2b/f3a zero size 2024-03-22T07:15:59.514 INFO:tasks.workunit.client.0.smithi063.stdout:8/682: symlink d0/d1b/d4f/le5 0 2024-03-22T07:15:59.514 INFO:tasks.workunit.client.0.smithi063.stdout:7/393: sync 2024-03-22T07:15:59.515 INFO:tasks.workunit.client.0.smithi063.stdout:7/394: getdents d5/d16/d1b/d21/d62/d73 0 2024-03-22T07:15:59.515 INFO:tasks.workunit.client.1.smithi069.stdout:6/211: mknod d4/dd/c3d 0 2024-03-22T07:15:59.517 INFO:tasks.workunit.client.1.smithi069.stdout:5/230: dwrite d1/d7/d8/f12 [8388608,4194304] 0 2024-03-22T07:15:59.517 INFO:tasks.workunit.client.1.smithi069.stdout:5/231: creat d1/f4f x:0 0 0 2024-03-22T07:15:59.517 INFO:tasks.workunit.client.1.smithi069.stdout:5/232: chown d1/d7/d8/f12 487701 1 2024-03-22T07:15:59.517 INFO:tasks.workunit.client.1.smithi069.stdout:5/233: write d1/d7/d8/f4b [711669,70972] 0 2024-03-22T07:15:59.519 INFO:tasks.workunit.client.1.smithi069.stdout:8/390: read d1/f9 [3157823,105865] 0 2024-03-22T07:15:59.520 INFO:tasks.workunit.client.0.smithi063.stdout:2/547: creat d0/de/d12/d39/fab x:0 0 0 2024-03-22T07:15:59.521 INFO:tasks.workunit.client.1.smithi069.stdout:5/234: mkdir d1/d10/d44/d50 0 2024-03-22T07:15:59.522 INFO:tasks.workunit.client.1.smithi069.stdout:5/235: chown d1/d10/d44/d50 9 1 2024-03-22T07:15:59.524 INFO:tasks.workunit.client.0.smithi063.stdout:3/622: symlink da/d63/d65/d92/d96/lce 0 2024-03-22T07:15:59.524 INFO:tasks.workunit.client.0.smithi063.stdout:3/623: stat da/d16/d3f/cb3 0 2024-03-22T07:15:59.524 INFO:tasks.workunit.client.0.smithi063.stdout:3/624: chown da/d63/l82 1300 1 2024-03-22T07:15:59.524 INFO:tasks.workunit.client.0.smithi063.stdout:8/683: creat d0/d24/d33/fe6 x:0 0 0 2024-03-22T07:15:59.524 INFO:tasks.workunit.client.0.smithi063.stdout:8/684: chown d0/d1b/d4f/le5 489 1 2024-03-22T07:15:59.524 INFO:tasks.workunit.client.0.smithi063.stdout:8/685: readlink d0/l7e 0 2024-03-22T07:15:59.527 INFO:tasks.workunit.client.0.smithi063.stdout:3/625: mkdir da/d63/d65/d81/d9b/d9f/dcf 0 2024-03-22T07:15:59.531 INFO:tasks.workunit.client.0.smithi063.stdout:3/626: symlink da/d63/d65/db9/ld0 0 2024-03-22T07:15:59.531 INFO:tasks.workunit.client.0.smithi063.stdout:8/686: truncate d0/d24/d33/d4c/f94 977566 0 2024-03-22T07:15:59.536 INFO:tasks.workunit.client.0.smithi063.stdout:3/627: getdents da/d63/d65/d81 0 2024-03-22T07:15:59.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.559+0000 7fa47ed99700 1 -- 172.21.15.63:0/2388574098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4780fcf80 msgr2=0x7fa4780fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:59.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.559+0000 7fa47ed99700 1 --2- 172.21.15.63:0/2388574098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4780fcf80 0x7fa4780fd3f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa464004940 tx=0x7fa464009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:59.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.559+0000 7fa47ed99700 1 -- 172.21.15.63:0/2388574098 shutdown_connections 2024-03-22T07:15:59.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.559+0000 7fa47ed99700 1 --2- 172.21.15.63:0/2388574098 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4780fcf80 0x7fa4780fd3f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:59.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.559+0000 7fa47ed99700 1 --2- 172.21.15.63:0/2388574098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4780fbcd0 0x7fa4780fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:59.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.559+0000 7fa47ed99700 1 -- 172.21.15.63:0/2388574098 >> 172.21.15.63:0/2388574098 conn(0x7fa4780f7430 msgr2=0x7fa4780f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:59.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.559+0000 7fa47ed99700 1 -- 172.21.15.63:0/2388574098 shutdown_connections 2024-03-22T07:15:59.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.559+0000 7fa47ed99700 1 -- 172.21.15.63:0/2388574098 wait complete. 2024-03-22T07:15:59.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.560+0000 7fa47ed99700 1 Processor -- start 2024-03-22T07:15:59.560 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.560+0000 7fa47ed99700 1 -- start start 2024-03-22T07:15:59.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.560+0000 7fa47ed99700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4780fbcd0 0x7fa47810da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:59.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.560+0000 7fa47ed99700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4780fcf80 0x7fa47810df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:59.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.560+0000 7fa47ed99700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47810e550 con 0x7fa4780fcf80 2024-03-22T07:15:59.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.560+0000 7fa47ed99700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47810eda0 con 0x7fa4780fbcd0 2024-03-22T07:15:59.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.560+0000 7fa477fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4780fcf80 0x7fa47810df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:59.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.560+0000 7fa477fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4780fcf80 0x7fa47810df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39850/0 (socket says 172.21.15.63:39850) 2024-03-22T07:15:59.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.560+0000 7fa477fff700 1 -- 172.21.15.63:0/847805565 learned_addr learned my addr 172.21.15.63:0/847805565 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:15:59.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.560+0000 7fa47cb35700 1 --2- 172.21.15.63:0/847805565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4780fbcd0 0x7fa47810da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:59.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.561+0000 7fa477fff700 1 -- 172.21.15.63:0/847805565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4780fbcd0 msgr2=0x7fa47810da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:59.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.561+0000 7fa477fff700 1 --2- 172.21.15.63:0/847805565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4780fbcd0 0x7fa47810da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:59.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.561+0000 7fa477fff700 1 -- 172.21.15.63:0/847805565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa464005040 con 0x7fa4780fcf80 2024-03-22T07:15:59.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.561+0000 7fa477fff700 1 --2- 172.21.15.63:0/847805565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4780fcf80 0x7fa47810df50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa464009150 tx=0x7fa464009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:59.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.561+0000 7fa4757fa700 1 -- 172.21.15.63:0/847805565 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa46400e040 con 0x7fa4780fcf80 2024-03-22T07:15:59.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.561+0000 7fa47ed99700 1 -- 172.21.15.63:0/847805565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47810f020 con 0x7fa4780fcf80 2024-03-22T07:15:59.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.561+0000 7fa4757fa700 1 -- 172.21.15.63:0/847805565 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa46400bbf0 con 0x7fa4780fcf80 2024-03-22T07:15:59.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.562+0000 7fa4757fa700 1 -- 172.21.15.63:0/847805565 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa46401c630 con 0x7fa4780fcf80 2024-03-22T07:15:59.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.562+0000 7fa47ed99700 1 -- 172.21.15.63:0/847805565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47810f560 con 0x7fa4780fcf80 2024-03-22T07:15:59.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.563+0000 7fa4757fa700 1 -- 172.21.15.63:0/847805565 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 20) v1 ==== 88751+0+0 (secure 0 0 0) 0x7fa46400bd60 con 0x7fa4780fcf80 2024-03-22T07:15:59.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.563+0000 7fa47ed99700 1 -- 172.21.15.63:0/847805565 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa478101b10 con 0x7fa4780fcf80 2024-03-22T07:15:59.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.563+0000 7fa4757fa700 1 --2- 172.21.15.63:0/847805565 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa46806ad90 0x7fa46806d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:15:59.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.563+0000 7fa47cb35700 1 --2- 172.21.15.63:0/847805565 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa46806ad90 0x7fa46806d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:15:59.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.563+0000 7fa4757fa700 1 -- 172.21.15.63:0/847805565 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa46408a6e0 con 0x7fa4780fcf80 2024-03-22T07:15:59.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.564+0000 7fa47cb35700 1 --2- 172.21.15.63:0/847805565 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa46806ad90 0x7fa46806d240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa46c004f40 tx=0x7fa46c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:15:59.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.568+0000 7fa4757fa700 1 -- 172.21.15.63:0/847805565 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176504 (secure 0 0 0) 0x7fa46405a4f0 con 0x7fa4780fcf80 2024-03-22T07:15:59.568 INFO:tasks.workunit.client.1.smithi069.stdout:2/233: dwrite d18/d4d/d25/f4b [0,4194304] 0 2024-03-22T07:15:59.568 INFO:tasks.workunit.client.1.smithi069.stdout:2/234: fdatasync d18/f2b 0 2024-03-22T07:15:59.569 INFO:tasks.workunit.client.0.smithi063.stdout:8/687: dwrite d0/d24/d5b/f44 [0,4194304] 0 2024-03-22T07:15:59.569 INFO:tasks.workunit.client.0.smithi063.stdout:7/395: dwrite d5/da/f11 [4194304,4194304] 0 2024-03-22T07:15:59.570 INFO:tasks.workunit.client.1.smithi069.stdout:9/263: dwrite d2/dd/d3c/f20 [4194304,4194304] 0 2024-03-22T07:15:59.570 INFO:tasks.workunit.client.1.smithi069.stdout:6/212: dwrite d4/dd/d12/d13/f23 [0,4194304] 0 2024-03-22T07:15:59.570 INFO:tasks.workunit.client.1.smithi069.stdout:6/213: write d4/dd/d12/d2b/f3a [489661,116992] 0 2024-03-22T07:15:59.570 INFO:tasks.workunit.client.1.smithi069.stdout:6/214: truncate d4/d7/f35 298463 0 2024-03-22T07:15:59.570 INFO:tasks.workunit.client.1.smithi069.stdout:6/215: readlink d4/d7/da/l38 0 2024-03-22T07:15:59.570 INFO:tasks.workunit.client.1.smithi069.stdout:6/216: dread - d4/dd/d12/d2b/f37 zero size 2024-03-22T07:15:59.570 INFO:tasks.workunit.client.1.smithi069.stdout:3/272: dwrite d5/d9/d26/d3f/d35/f37 [0,4194304] 0 2024-03-22T07:15:59.570 INFO:tasks.workunit.client.1.smithi069.stdout:5/236: dwrite d1/f9 [4194304,4194304] 0 2024-03-22T07:15:59.570 INFO:tasks.workunit.client.1.smithi069.stdout:3/273: dread - d5/d9/db/f30 zero size 2024-03-22T07:15:59.571 INFO:tasks.workunit.client.1.smithi069.stdout:3/274: creat d5/d9/d26/f5a x:0 0 0 2024-03-22T07:15:59.571 INFO:tasks.workunit.client.1.smithi069.stdout:3/275: stat d5/c2c 0 2024-03-22T07:15:59.571 INFO:tasks.workunit.client.1.smithi069.stdout:3/276: chown d5/d33/f39 3499802 1 2024-03-22T07:15:59.571 INFO:tasks.workunit.client.1.smithi069.stdout:3/277: chown d5/c14 21195546 1 2024-03-22T07:15:59.571 INFO:tasks.workunit.client.1.smithi069.stdout:3/278: getdents d5/d9/d26/d3f/d35/d4f 0 2024-03-22T07:15:59.571 INFO:tasks.workunit.client.1.smithi069.stdout:3/279: chown d5/d9/db/f3d 226921 1 2024-03-22T07:15:59.571 INFO:tasks.workunit.client.0.smithi063.stdout:3/628: dwrite da/d16/d3f/dc7/fcd [0,4194304] 0 2024-03-22T07:15:59.572 INFO:tasks.workunit.client.1.smithi069.stdout:2/235: mkdir d18/d4d/d27/d56 0 2024-03-22T07:15:59.573 INFO:tasks.workunit.client.0.smithi063.stdout:8/688: creat d0/d1b/d28/d5f/fe7 x:0 0 0 2024-03-22T07:15:59.574 INFO:tasks.workunit.client.0.smithi063.stdout:7/396: symlink d5/d16/d1b/d21/d62/d73/d1d/d40/l83 0 2024-03-22T07:15:59.575 INFO:tasks.workunit.client.1.smithi069.stdout:9/264: mkdir d2/dd/d3c/d53 0 2024-03-22T07:15:59.575 INFO:tasks.workunit.client.1.smithi069.stdout:9/265: readlink d2/d21/l41 0 2024-03-22T07:15:59.575 INFO:tasks.workunit.client.1.smithi069.stdout:9/266: readlink d2/dd/d3c/l44 0 2024-03-22T07:15:59.576 INFO:tasks.workunit.client.1.smithi069.stdout:9/267: dread d2/dd/f37 [0,4194304] 0 2024-03-22T07:15:59.576 INFO:tasks.workunit.client.1.smithi069.stdout:6/217: rename d4/d7/f2c to d4/d7/f3e 0 2024-03-22T07:15:59.576 INFO:tasks.workunit.client.1.smithi069.stdout:6/218: read d4/dd/d12/f1d [1123622,28136] 0 2024-03-22T07:15:59.577 INFO:tasks.workunit.client.1.smithi069.stdout:6/219: chown d4/f14 7482373 1 2024-03-22T07:15:59.577 INFO:tasks.workunit.client.0.smithi063.stdout:3/629: symlink da/d63/ld1 0 2024-03-22T07:15:59.577 INFO:tasks.workunit.client.0.smithi063.stdout:3/630: readlink da/d16/l22 0 2024-03-22T07:15:59.581 INFO:tasks.workunit.client.0.smithi063.stdout:7/397: unlink d5/d16/d1b/d21/d62/d73/f61 0 2024-03-22T07:15:59.581 INFO:tasks.workunit.client.0.smithi063.stdout:7/398: write d5/d16/f63 [359986,122768] 0 2024-03-22T07:15:59.581 INFO:tasks.workunit.client.0.smithi063.stdout:7/399: fdatasync d5/d16/f2e 0 2024-03-22T07:15:59.581 INFO:tasks.workunit.client.1.smithi069.stdout:5/237: rename d1/d7/l3b to d1/d6/d26/l51 0 2024-03-22T07:15:59.581 INFO:tasks.workunit.client.1.smithi069.stdout:9/268: rename d2/dd/d3c/d48 to d2/dd/d3c/d48/d54 22 2024-03-22T07:15:59.581 INFO:tasks.workunit.client.1.smithi069.stdout:9/269: chown d2/dd/d3c/c34 3 1 2024-03-22T07:15:59.582 INFO:tasks.workunit.client.0.smithi063.stdout:3/631: symlink da/d63/d65/d92/ld2 0 2024-03-22T07:15:59.585 INFO:tasks.workunit.client.1.smithi069.stdout:6/220: mkdir d4/d7/da/d3f 0 2024-03-22T07:15:59.585 INFO:tasks.workunit.client.0.smithi063.stdout:7/400: link d5/d16/d1b/d21/d62/d73/d1d/d29/f36 d5/da/d5f/f84 0 2024-03-22T07:15:59.585 INFO:tasks.workunit.client.1.smithi069.stdout:2/236: dread d18/f19 [0,4194304] 0 2024-03-22T07:15:59.586 INFO:tasks.workunit.client.1.smithi069.stdout:5/238: unlink d1/d6/d26/f42 0 2024-03-22T07:15:59.588 INFO:tasks.workunit.client.0.smithi063.stdout:7/401: getdents d5/d16/d1b/d21 0 2024-03-22T07:15:59.589 INFO:tasks.workunit.client.1.smithi069.stdout:6/221: creat d4/dd/d12/d13/f40 x:0 0 0 2024-03-22T07:15:59.591 INFO:tasks.workunit.client.0.smithi063.stdout:2/548: rename d0/de/d12/d39 to d0/de/d1b/d32/dac 0 2024-03-22T07:15:59.592 INFO:tasks.workunit.client.1.smithi069.stdout:5/239: rename d1/d7/d8/f27 to d1/d7/d8/f52 0 2024-03-22T07:15:59.592 INFO:tasks.workunit.client.1.smithi069.stdout:5/240: fsync d1/d7/d8/f15 0 2024-03-22T07:15:59.593 INFO:tasks.workunit.client.1.smithi069.stdout:5/241: stat d1/d6/ff 0 2024-03-22T07:15:59.595 INFO:tasks.workunit.client.1.smithi069.stdout:9/270: getdents d2/dd/d3c/d2e 0 2024-03-22T07:15:59.595 INFO:tasks.workunit.client.1.smithi069.stdout:3/280: dwrite d5/d9/f13 [0,4194304] 0 2024-03-22T07:15:59.596 INFO:tasks.workunit.client.0.smithi063.stdout:2/549: mknod d0/de/d35/d36/cad 0 2024-03-22T07:15:59.596 INFO:tasks.workunit.client.1.smithi069.stdout:6/222: truncate d4/dd/d12/f1d 2304374 0 2024-03-22T07:15:59.599 INFO:tasks.workunit.client.1.smithi069.stdout:5/242: rename d1/d10/c35 to d1/d7/d3d/c53 0 2024-03-22T07:15:59.599 INFO:tasks.workunit.client.1.smithi069.stdout:5/243: read - d1/d6/d26/d3e/f4a zero size 2024-03-22T07:15:59.599 INFO:tasks.workunit.client.0.smithi063.stdout:8/689: rename d0/d1b/d28/d3e/d97 to d0/d1b/d28/de8 0 2024-03-22T07:15:59.599 INFO:tasks.workunit.client.0.smithi063.stdout:8/690: fdatasync d0/d1b/d28/d5f/fe7 0 2024-03-22T07:15:59.601 INFO:tasks.workunit.client.0.smithi063.stdout:2/550: creat d0/de/d1b/fae x:0 0 0 2024-03-22T07:15:59.602 INFO:tasks.workunit.client.0.smithi063.stdout:7/402: dread d5/f39 [0,4194304] 0 2024-03-22T07:15:59.603 INFO:tasks.workunit.client.0.smithi063.stdout:7/403: stat d5/d16/d1b/d21/d62/d73/d1d/d29/l79 0 2024-03-22T07:15:59.603 INFO:tasks.workunit.client.0.smithi063.stdout:8/691: mkdir d0/d1b/d28/de8/dd1/de9 0 2024-03-22T07:15:59.607 INFO:tasks.workunit.client.0.smithi063.stdout:7/404: mkdir d5/d16/d1b/d21/d85 0 2024-03-22T07:15:59.609 INFO:tasks.workunit.client.1.smithi069.stdout:3/281: creat d5/d9/d23/f5b x:0 0 0 2024-03-22T07:15:59.609 INFO:tasks.workunit.client.1.smithi069.stdout:3/282: readlink d5/d9/l1f 0 2024-03-22T07:15:59.609 INFO:tasks.workunit.client.1.smithi069.stdout:3/283: write d5/d9/d23/f5b [584721,54309] 0 2024-03-22T07:15:59.609 INFO:tasks.workunit.client.1.smithi069.stdout:3/284: write d5/d9/d26/d3f/d35/f37 [4912621,109395] 0 2024-03-22T07:15:59.610 INFO:tasks.workunit.client.0.smithi063.stdout:8/692: link d0/d1b/d28/d3e/c8b d0/d1b/d28/d3e/db2/dbd/dd0/cea 0 2024-03-22T07:15:59.610 INFO:tasks.workunit.client.0.smithi063.stdout:8/693: chown d0/d1b/d4f 667382 1 2024-03-22T07:15:59.610 INFO:tasks.workunit.client.0.smithi063.stdout:8/694: chown d0 1013758813 1 2024-03-22T07:15:59.610 INFO:tasks.workunit.client.0.smithi063.stdout:8/695: chown d0/d24/d5b/f60 8319 1 2024-03-22T07:15:59.610 INFO:tasks.workunit.client.0.smithi063.stdout:8/696: chown d0/d24/d33/d4c/ld5 3352 1 2024-03-22T07:15:59.610 INFO:tasks.workunit.client.0.smithi063.stdout:8/697: chown d0/d1b/d4f 345830 1 2024-03-22T07:15:59.611 INFO:tasks.workunit.client.0.smithi063.stdout:2/551: link d0/de/d35/d36/f3a d0/de/d1b/d32/dac/faf 0 2024-03-22T07:15:59.612 INFO:tasks.workunit.client.1.smithi069.stdout:6/223: dread d4/dd/d12/f1f [0,4194304] 0 2024-03-22T07:15:59.612 INFO:tasks.workunit.client.1.smithi069.stdout:6/224: chown d4/dd/d12/l17 2303 1 2024-03-22T07:15:59.612 INFO:tasks.workunit.client.1.smithi069.stdout:6/225: fsync d4/fb 0 2024-03-22T07:15:59.612 INFO:tasks.workunit.client.1.smithi069.stdout:6/226: dread - d4/dd/d12/d13/f31 zero size 2024-03-22T07:15:59.612 INFO:tasks.workunit.client.1.smithi069.stdout:3/285: mknod d5/d9/d26/d3f/d54/c5c 0 2024-03-22T07:15:59.613 INFO:tasks.workunit.client.0.smithi063.stdout:7/405: creat d5/d16/d1b/d21/f86 x:0 0 0 2024-03-22T07:15:59.613 INFO:tasks.workunit.client.0.smithi063.stdout:7/406: write d5/da/f25 [3080938,119531] 0 2024-03-22T07:15:59.613 INFO:tasks.workunit.client.0.smithi063.stdout:2/552: rename d0/de/d74/d8a to d0/de/d74/db0 0 2024-03-22T07:15:59.616 INFO:tasks.workunit.client.1.smithi069.stdout:3/286: rename d5/d9/fd to d5/f5d 0 2024-03-22T07:15:59.616 INFO:tasks.workunit.client.0.smithi063.stdout:7/407: write d5/f18 [1144855,100713] 0 2024-03-22T07:15:59.617 INFO:tasks.workunit.client.1.smithi069.stdout:6/227: truncate d4/dd/d12/f1d 2585177 0 2024-03-22T07:15:59.619 INFO:tasks.workunit.client.0.smithi063.stdout:2/553: mknod d0/de/d1b/d63/d6e/cb1 0 2024-03-22T07:15:59.619 INFO:tasks.workunit.client.0.smithi063.stdout:8/698: dread d0/d1b/d4f/f79 [0,4194304] 0 2024-03-22T07:15:59.620 INFO:tasks.workunit.client.0.smithi063.stdout:7/408: truncate d5/d16/f22 7117175 0 2024-03-22T07:15:59.622 INFO:tasks.workunit.client.0.smithi063.stdout:2/554: symlink d0/de/d1b/d63/lb2 0 2024-03-22T07:15:59.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:59 smithi063 ceph-mon[29200]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:59.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:59 smithi063 ceph-mon[29200]: pgmap v252: 65 pgs: 65 active+clean; 1.1 GiB data, 4.9 GiB used, 531 GiB / 536 GiB avail; 122 MiB/s rd, 145 MiB/s wr, 707 op/s 2024-03-22T07:15:59.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:59 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:15:59.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:15:59 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/950638700' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:15:59.623 INFO:tasks.workunit.client.0.smithi063.stdout:3/632: dwrite da/d16/d26/f48 [0,4194304] 0 2024-03-22T07:15:59.623 INFO:tasks.workunit.client.0.smithi063.stdout:3/633: truncate da/d16/d3f/f50 73370 0 2024-03-22T07:15:59.624 INFO:tasks.workunit.client.1.smithi069.stdout:9/271: dwrite d2/dd/f26 [4194304,4194304] 0 2024-03-22T07:15:59.624 INFO:tasks.workunit.client.1.smithi069.stdout:9/272: write d2/dd/f37 [190170,52640] 0 2024-03-22T07:15:59.627 INFO:tasks.workunit.client.1.smithi069.stdout:2/237: dwrite d18/f3a [0,4194304] 0 2024-03-22T07:15:59.627 INFO:tasks.workunit.client.1.smithi069.stdout:9/273: rmdir d2/d31 39 2024-03-22T07:15:59.629 INFO:tasks.workunit.client.0.smithi063.stdout:8/699: rename d0/d26/d2d/d83/lb0 to d0/d1b/d28/d7b/leb 0 2024-03-22T07:15:59.629 INFO:tasks.workunit.client.0.smithi063.stdout:8/700: creat d0/d1b/d28/d3e/fec x:0 0 0 2024-03-22T07:15:59.629 INFO:tasks.workunit.client.0.smithi063.stdout:7/409: mknod d5/da/d28/d4b/c87 0 2024-03-22T07:15:59.629 INFO:tasks.workunit.client.0.smithi063.stdout:7/410: fsync d5/da/d5f/d33/d57/f67 0 2024-03-22T07:15:59.629 INFO:tasks.workunit.client.0.smithi063.stdout:7/411: write d5/f7 [1445838,85518] 0 2024-03-22T07:15:59.629 INFO:tasks.workunit.client.0.smithi063.stdout:7/412: readlink d5/d16/d1b/d21/d62/d73/d1d/d29/d3b/l3d 0 2024-03-22T07:15:59.629 INFO:tasks.workunit.client.0.smithi063.stdout:7/413: creat d5/d16/f88 x:0 0 0 2024-03-22T07:15:59.629 INFO:tasks.workunit.client.0.smithi063.stdout:7/414: fsync d5/da/f2c 0 2024-03-22T07:15:59.630 INFO:tasks.workunit.client.1.smithi069.stdout:2/238: creat d18/d4d/f57 x:0 0 0 2024-03-22T07:15:59.631 INFO:tasks.workunit.client.1.smithi069.stdout:9/274: link d2/dd/d3c/d2e/f30 d2/f55 0 2024-03-22T07:15:59.632 INFO:tasks.workunit.client.1.smithi069.stdout:6/228: dread d4/dd/d12/f32 [0,4194304] 0 2024-03-22T07:15:59.632 INFO:tasks.workunit.client.1.smithi069.stdout:6/229: readlink d4/d7/l2d 0 2024-03-22T07:15:59.632 INFO:tasks.workunit.client.0.smithi063.stdout:7/415: write d5/d16/f63 [395869,57587] 0 2024-03-22T07:15:59.633 INFO:tasks.workunit.client.1.smithi069.stdout:2/239: rename c13 to d18/d4d/c58 0 2024-03-22T07:15:59.633 INFO:tasks.workunit.client.1.smithi069.stdout:2/240: readlink d18/d4d/l2a 0 2024-03-22T07:15:59.635 INFO:tasks.workunit.client.1.smithi069.stdout:9/275: creat d2/d51/f56 x:0 0 0 2024-03-22T07:15:59.635 INFO:tasks.workunit.client.1.smithi069.stdout:9/276: creat d2/d51/f57 x:0 0 0 2024-03-22T07:15:59.637 INFO:tasks.workunit.client.0.smithi063.stdout:2/555: dread d0/f23 [0,4194304] 0 2024-03-22T07:15:59.637 INFO:tasks.workunit.client.0.smithi063.stdout:2/556: stat d0/de/d1b/d63/d6e/f95 0 2024-03-22T07:15:59.638 INFO:tasks.workunit.client.0.smithi063.stdout:7/416: write d5/f39 [1013132,57394] 0 2024-03-22T07:15:59.640 INFO:tasks.workunit.client.0.smithi063.stdout:8/701: rmdir d0/d26/d2d/d83/dbc/da1 39 2024-03-22T07:15:59.641 INFO:tasks.workunit.client.0.smithi063.stdout:8/702: write d0/d26/faf [68872,53063] 0 2024-03-22T07:15:59.641 INFO:tasks.workunit.client.0.smithi063.stdout:8/703: dread - d0/d1b/d28/d3e/fec zero size 2024-03-22T07:15:59.641 INFO:tasks.workunit.client.1.smithi069.stdout:3/287: dwrite d5/d9/d26/d3f/d35/f45 [0,4194304] 0 2024-03-22T07:15:59.647 INFO:tasks.workunit.client.1.smithi069.stdout:2/241: write d18/f3a [3973933,123219] 0 2024-03-22T07:15:59.648 INFO:tasks.workunit.client.1.smithi069.stdout:2/242: dread d18/f1b [0,4194304] 0 2024-03-22T07:15:59.649 INFO:tasks.workunit.client.0.smithi063.stdout:2/557: unlink d0/de/f38 0 2024-03-22T07:15:59.649 INFO:tasks.workunit.client.0.smithi063.stdout:2/558: fsync d0/de/d12/d20/f70 0 2024-03-22T07:15:59.649 INFO:tasks.workunit.client.0.smithi063.stdout:2/559: fsync d0/de/d12/f27 0 2024-03-22T07:15:59.650 INFO:tasks.workunit.client.1.smithi069.stdout:2/243: mknod d18/d4d/d27/d33/d4f/c59 0 2024-03-22T07:15:59.651 INFO:tasks.workunit.client.1.smithi069.stdout:6/230: dwrite d4/d7/da/f1b [0,4194304] 0 2024-03-22T07:15:59.658 INFO:tasks.workunit.client.0.smithi063.stdout:8/704: mkdir d0/d26/d2d/d59/dba/dc9/ded 0 2024-03-22T07:15:59.659 INFO:tasks.workunit.client.0.smithi063.stdout:7/417: rmdir d5/da/d5f/d33/d57 39 2024-03-22T07:15:59.661 INFO:tasks.workunit.client.0.smithi063.stdout:7/418: creat d5/f89 x:0 0 0 2024-03-22T07:15:59.662 INFO:tasks.workunit.client.0.smithi063.stdout:7/419: write d5/d16/d1b/d21/f59 [962177,72114] 0 2024-03-22T07:15:59.663 INFO:tasks.workunit.client.0.smithi063.stdout:9/469: sync 2024-03-22T07:15:59.663 INFO:tasks.workunit.client.0.smithi063.stdout:9/470: dread - d8/f54 zero size 2024-03-22T07:15:59.663 INFO:tasks.workunit.client.0.smithi063.stdout:5/499: sync 2024-03-22T07:15:59.663 INFO:tasks.workunit.client.0.smithi063.stdout:5/500: fsync d8/d13/d1e/d31/d58/d7e/f84 0 2024-03-22T07:15:59.663 INFO:tasks.workunit.client.0.smithi063.stdout:5/501: stat d8/d13/d1e/d31 0 2024-03-22T07:15:59.663 INFO:tasks.workunit.client.0.smithi063.stdout:5/502: chown d8/d13/f5d 43958 1 2024-03-22T07:15:59.663 INFO:tasks.workunit.client.0.smithi063.stdout:4/482: sync 2024-03-22T07:15:59.663 INFO:tasks.workunit.client.0.smithi063.stdout:4/483: fsync d1/d5/d43/d27/d2c/d4a/d80/fa8 0 2024-03-22T07:15:59.664 INFO:tasks.workunit.client.0.smithi063.stdout:0/734: sync 2024-03-22T07:15:59.664 INFO:tasks.workunit.client.0.smithi063.stdout:1/686: sync 2024-03-22T07:15:59.664 INFO:tasks.workunit.client.0.smithi063.stdout:6/558: sync 2024-03-22T07:15:59.666 INFO:tasks.workunit.client.0.smithi063.stdout:9/471: symlink d8/d19/d20/d22/d5d/l92 0 2024-03-22T07:15:59.666 INFO:tasks.workunit.client.0.smithi063.stdout:9/472: dread - d8/d19/d20/d22/d29/d31/f86 zero size 2024-03-22T07:15:59.666 INFO:tasks.workunit.client.0.smithi063.stdout:9/473: write d8/d19/d41/d48/f66 [324508,59652] 0 2024-03-22T07:15:59.666 INFO:tasks.workunit.client.0.smithi063.stdout:9/474: creat d8/d15/d2f/f93 x:0 0 0 2024-03-22T07:15:59.666 INFO:tasks.workunit.client.0.smithi063.stdout:9/475: chown d8/d19/f23 90263829 1 2024-03-22T07:15:59.668 INFO:tasks.workunit.client.0.smithi063.stdout:5/503: link d8/d13/d28/d66/l7a d8/d13/d28/d57/d5f/l8d 0 2024-03-22T07:15:59.668 INFO:tasks.workunit.client.0.smithi063.stdout:5/504: stat d8/d13/d1e/d3c 0 2024-03-22T07:15:59.669 INFO:tasks.workunit.client.0.smithi063.stdout:4/484: mkdir d1/d5/d43/d27/d2c/d3d/d54/d5f/d79/daa 0 2024-03-22T07:15:59.672 INFO:tasks.workunit.client.0.smithi063.stdout:2/560: read d0/f41 [2249479,30734] 0 2024-03-22T07:15:59.673 INFO:tasks.workunit.client.0.smithi063.stdout:6/559: truncate d3/d15/f85 239070 0 2024-03-22T07:15:59.674 INFO:tasks.workunit.client.1.smithi069.stdout:6/231: dread d4/f6 [0,4194304] 0 2024-03-22T07:15:59.675 INFO:tasks.workunit.client.1.smithi069.stdout:6/232: mkdir d4/dd/d41 0 2024-03-22T07:15:59.676 INFO:tasks.workunit.client.0.smithi063.stdout:6/560: link d3/d6/d2a/d38/f49 d3/d6/d2a/fb8 0 2024-03-22T07:15:59.677 INFO:tasks.workunit.client.0.smithi063.stdout:5/505: symlink d8/l8e 0 2024-03-22T07:15:59.677 INFO:tasks.workunit.client.0.smithi063.stdout:5/506: readlink d8/l32 0 2024-03-22T07:15:59.677 INFO:tasks.workunit.client.1.smithi069.stdout:6/233: rename d4/dd/d12/d13/f31 to d4/dd/f42 0 2024-03-22T07:15:59.681 INFO:tasks.workunit.client.0.smithi063.stdout:9/476: rmdir d8/d19/d20/d22/d29/d31/d7f 0 2024-03-22T07:15:59.681 INFO:tasks.workunit.client.0.smithi063.stdout:9/477: chown d8/l72 5629 1 2024-03-22T07:15:59.681 INFO:tasks.workunit.client.0.smithi063.stdout:6/561: mkdir d3/d6/d2a/d38/da5/d32/db9 0 2024-03-22T07:15:59.681 INFO:tasks.workunit.client.1.smithi069.stdout:3/288: dwrite d5/d9/db/f30 [0,4194304] 0 2024-03-22T07:15:59.683 INFO:tasks.workunit.client.0.smithi063.stdout:6/562: link d3/d6/d2a/d38/da5/d32/fb1 d3/d15/d16/d60/d6d/d4e/d5a/fba 0 2024-03-22T07:15:59.684 INFO:tasks.workunit.client.0.smithi063.stdout:5/507: symlink d8/l8f 0 2024-03-22T07:15:59.684 INFO:tasks.workunit.client.0.smithi063.stdout:5/508: fsync d8/d13/d1e/f80 0 2024-03-22T07:15:59.684 INFO:tasks.workunit.client.0.smithi063.stdout:9/478: dread d8/d19/d20/d22/d29/d31/f4e [0,4194304] 0 2024-03-22T07:15:59.685 INFO:tasks.workunit.client.0.smithi063.stdout:5/509: fsync d8/d13/d1e/f80 0 2024-03-22T07:15:59.685 INFO:tasks.workunit.client.0.smithi063.stdout:9/479: dread - d8/d19/d20/d22/f4f zero size 2024-03-22T07:15:59.685 INFO:tasks.workunit.client.1.smithi069.stdout:3/289: creat d5/d9/f5e x:0 0 0 2024-03-22T07:15:59.685 INFO:tasks.workunit.client.1.smithi069.stdout:6/234: write d4/d7/f3e [3303442,110013] 0 2024-03-22T07:15:59.687 INFO:tasks.workunit.client.1.smithi069.stdout:6/235: link d4/d7/ce d4/dd/d12/d13/c43 0 2024-03-22T07:15:59.687 INFO:tasks.workunit.client.1.smithi069.stdout:6/236: chown d4/l22 122940 1 2024-03-22T07:15:59.687 INFO:tasks.workunit.client.0.smithi063.stdout:6/563: rename d3/d76 to d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb 0 2024-03-22T07:15:59.688 INFO:tasks.workunit.client.1.smithi069.stdout:6/237: mkdir d4/dd/d12/d44 0 2024-03-22T07:15:59.689 INFO:tasks.workunit.client.1.smithi069.stdout:6/238: rename d4/d7/da/lc to d4/dd/l45 0 2024-03-22T07:15:59.689 INFO:tasks.workunit.client.1.smithi069.stdout:6/239: creat d4/f46 x:0 0 0 2024-03-22T07:15:59.702 INFO:tasks.workunit.client.0.smithi063.stdout:6/564: dread d3/d15/f22 [4194304,4194304] 0 2024-03-22T07:15:59.702 INFO:tasks.workunit.client.0.smithi063.stdout:6/565: read - d3/d15/d16/d60/f65 zero size 2024-03-22T07:15:59.702 INFO:tasks.workunit.client.0.smithi063.stdout:6/566: fsync d3/d15/d16/d60/f65 0 2024-03-22T07:15:59.704 INFO:tasks.workunit.client.0.smithi063.stdout:6/567: mknod d3/d15/d16/d60/d6d/d4e/d5f/cbc 0 2024-03-22T07:15:59.714 INFO:tasks.workunit.client.1.smithi069.stdout:6/240: dwrite d4/dd/f25 [0,4194304] 0 2024-03-22T07:15:59.715 INFO:tasks.workunit.client.1.smithi069.stdout:6/241: creat d4/f47 x:0 0 0 2024-03-22T07:15:59.715 INFO:tasks.workunit.client.0.smithi063.stdout:6/568: dread d3/f9 [0,4194304] 0 2024-03-22T07:15:59.715 INFO:tasks.workunit.client.0.smithi063.stdout:6/569: write d3/d6/d35/f67 [760767,57712] 0 2024-03-22T07:15:59.716 INFO:tasks.workunit.client.1.smithi069.stdout:6/242: mknod d4/dd/c48 0 2024-03-22T07:15:59.717 INFO:tasks.workunit.client.0.smithi063.stdout:6/570: rename d3/d15/d16/d60/d6d/d4c/l71 to d3/d15/d16/lbd 0 2024-03-22T07:15:59.717 INFO:tasks.workunit.client.1.smithi069.stdout:6/243: rename d4/dd/c48 to d4/dd/d12/c49 0 2024-03-22T07:15:59.719 INFO:tasks.workunit.client.0.smithi063.stdout:6/571: creat d3/d6/d35/d3a/fbe x:0 0 0 2024-03-22T07:15:59.719 INFO:tasks.workunit.client.0.smithi063.stdout:6/572: write d3/d6/d2a/d38/da5/d32/fb1 [935287,108666] 0 2024-03-22T07:15:59.719 INFO:tasks.workunit.client.0.smithi063.stdout:6/573: chown d3/d15/d16 243404 1 2024-03-22T07:15:59.719 INFO:tasks.workunit.client.0.smithi063.stdout:6/574: write d3/d15/f7f [1409623,91016] 0 2024-03-22T07:15:59.722 INFO:tasks.workunit.client.0.smithi063.stdout:6/575: write d3/f9 [1226283,63084] 0 2024-03-22T07:15:59.722 INFO:tasks.workunit.client.0.smithi063.stdout:6/576: fdatasync d3/d6/d35/d3a/f40 0 2024-03-22T07:15:59.755 INFO:tasks.workunit.client.0.smithi063.stdout:0/735: dread da/de/d17/f45 [0,4194304] 0 2024-03-22T07:15:59.755 INFO:tasks.workunit.client.0.smithi063.stdout:0/736: read da/de/f24 [7835351,77085] 0 2024-03-22T07:15:59.756 INFO:tasks.workunit.client.0.smithi063.stdout:0/737: mknod da/de/d60/d56/cea 0 2024-03-22T07:15:59.757 INFO:tasks.workunit.client.0.smithi063.stdout:8/705: dwrite d0/d24/d33/d4c/fd4 [0,4194304] 0 2024-03-22T07:15:59.757 INFO:tasks.workunit.client.0.smithi063.stdout:8/706: creat d0/d24/d33/fee x:0 0 0 2024-03-22T07:15:59.770 INFO:tasks.workunit.client.0.smithi063.stdout:7/420: write d5/da/d5f/f84 [3657414,20489] 0 2024-03-22T07:15:59.770 INFO:tasks.workunit.client.0.smithi063.stdout:7/421: rename d5/f18 to d5/da/d5f/f8a 0 2024-03-22T07:15:59.771 INFO:tasks.workunit.client.0.smithi063.stdout:7/422: symlink d5/d16/d43/d78/l8b 0 2024-03-22T07:15:59.772 INFO:tasks.workunit.client.0.smithi063.stdout:7/423: creat d5/da/d5f/f8c x:0 0 0 2024-03-22T07:15:59.773 INFO:tasks.workunit.client.0.smithi063.stdout:7/424: mkdir d5/d16/d1b/d21/d62/d8d 0 2024-03-22T07:15:59.773 INFO:tasks.workunit.client.0.smithi063.stdout:7/425: symlink d5/d16/d43/l8e 0 2024-03-22T07:15:59.775 INFO:tasks.workunit.client.0.smithi063.stdout:7/426: link d5/da/d5f/c71 d5/d16/d1b/d21/d85/c8f 0 2024-03-22T07:15:59.775 INFO:tasks.workunit.client.0.smithi063.stdout:7/427: creat d5/d16/f90 x:0 0 0 2024-03-22T07:15:59.776 INFO:tasks.workunit.client.0.smithi063.stdout:7/428: mknod d5/d16/d1b/d21/d85/c91 0 2024-03-22T07:15:59.777 INFO:tasks.workunit.client.0.smithi063.stdout:7/429: truncate d5/d16/d1b/d21/d62/d73/d1d/d29/f5d 9828373 0 2024-03-22T07:15:59.777 INFO:tasks.workunit.client.0.smithi063.stdout:7/430: stat d5/da/d5f/f5c 0 2024-03-22T07:15:59.777 INFO:tasks.workunit.client.0.smithi063.stdout:7/431: truncate d5/d16/d1b/d21/d62/d73/f7a 6710232 0 2024-03-22T07:15:59.778 INFO:tasks.workunit.client.0.smithi063.stdout:7/432: fdatasync d5/d16/f32 0 2024-03-22T07:15:59.778 INFO:tasks.workunit.client.0.smithi063.stdout:7/433: symlink d5/d16/d43/d78/l92 0 2024-03-22T07:15:59.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:59 smithi069 ceph-mon[36441]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:15:59.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:59 smithi069 ceph-mon[36441]: pgmap v252: 65 pgs: 65 active+clean; 1.1 GiB data, 4.9 GiB used, 531 GiB / 536 GiB avail; 122 MiB/s rd, 145 MiB/s wr, 707 op/s 2024-03-22T07:15:59.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:59 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:15:59.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:15:59 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/950638700' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:15:59.780 INFO:tasks.workunit.client.0.smithi063.stdout:1/687: dwrite d4/d71/dbb/dc0/fea [0,4194304] 0 2024-03-22T07:15:59.781 INFO:tasks.workunit.client.0.smithi063.stdout:1/688: chown d4/d14/f94 78018519 1 2024-03-22T07:15:59.782 INFO:tasks.workunit.client.0.smithi063.stdout:2/561: dwrite d0/de/f30 [0,4194304] 0 2024-03-22T07:15:59.782 INFO:tasks.workunit.client.0.smithi063.stdout:2/562: chown d0/de/d35/d36 70091 1 2024-03-22T07:15:59.783 INFO:tasks.workunit.client.0.smithi063.stdout:0/738: write da/de/d17/d4e/fce [1976274,76754] 0 2024-03-22T07:15:59.783 INFO:tasks.workunit.client.0.smithi063.stdout:0/739: truncate da/de/d60/f33 1456375 0 2024-03-22T07:15:59.783 INFO:tasks.workunit.client.0.smithi063.stdout:9/480: dwrite d8/d19/d41/f6b [0,4194304] 0 2024-03-22T07:15:59.783 INFO:tasks.workunit.client.1.smithi069.stdout:3/290: dwrite d5/d9/d26/d3f/d35/f50 [4194304,4194304] 0 2024-03-22T07:15:59.784 INFO:tasks.workunit.client.1.smithi069.stdout:6/244: dwrite d4/dd/d12/d2b/f37 [0,4194304] 0 2024-03-22T07:15:59.784 INFO:tasks.workunit.client.1.smithi069.stdout:6/245: chown d4/dd/d12/c49 6678 1 2024-03-22T07:15:59.784 INFO:tasks.workunit.client.1.smithi069.stdout:2/244: dwrite d18/d4d/d27/d33/d3d/f46 [0,4194304] 0 2024-03-22T07:15:59.784 INFO:tasks.workunit.client.1.smithi069.stdout:2/245: write d18/d4d/d3e/d40/f44 [818269,53954] 0 2024-03-22T07:15:59.784 INFO:tasks.workunit.client.1.smithi069.stdout:2/246: fsync d18/d4d/d25/f47 0 2024-03-22T07:15:59.786 INFO:tasks.workunit.client.0.smithi063.stdout:4/485: dwrite d1/d5/dc/d95/f44 [0,4194304] 0 2024-03-22T07:15:59.787 INFO:tasks.workunit.client.0.smithi063.stdout:4/486: creat d1/d5/dc/d95/d59/fab x:0 0 0 2024-03-22T07:15:59.787 INFO:tasks.workunit.client.0.smithi063.stdout:4/487: truncate d1/d5/dc/d95/d59/f87 1356443 0 2024-03-22T07:15:59.787 INFO:tasks.workunit.client.0.smithi063.stdout:1/689: dread d4/f38 [0,4194304] 0 2024-03-22T07:15:59.787 INFO:tasks.workunit.client.0.smithi063.stdout:1/690: stat d4/f8c 0 2024-03-22T07:15:59.788 INFO:tasks.workunit.client.0.smithi063.stdout:2/563: unlink d0/de/d35/d36/c7a 0 2024-03-22T07:15:59.788 INFO:tasks.workunit.client.1.smithi069.stdout:3/291: mknod d5/d9/d26/d36/d41/d51/c5f 0 2024-03-22T07:15:59.788 INFO:tasks.workunit.client.1.smithi069.stdout:3/292: write d5/d9/db/f3d [4486196,64895] 0 2024-03-22T07:15:59.788 INFO:tasks.workunit.client.1.smithi069.stdout:3/293: fdatasync d5/f4e 0 2024-03-22T07:15:59.788 INFO:tasks.workunit.client.1.smithi069.stdout:3/294: chown d5/d9/f5e 3221 1 2024-03-22T07:15:59.788 INFO:tasks.workunit.client.1.smithi069.stdout:3/295: write d5/d9/d26/d3f/f22 [9422660,61041] 0 2024-03-22T07:15:59.788 INFO:tasks.workunit.client.1.smithi069.stdout:3/296: write d5/d33/f47 [904256,10489] 0 2024-03-22T07:15:59.791 INFO:tasks.workunit.client.0.smithi063.stdout:0/740: unlink da/de/d17/d25/d3c/d5f/c46 0 2024-03-22T07:15:59.791 INFO:tasks.workunit.client.0.smithi063.stdout:0/741: write da/de/d17/d25/d3c/f66 [2029783,94997] 0 2024-03-22T07:15:59.792 INFO:tasks.workunit.client.1.smithi069.stdout:2/247: creat d18/d4d/d3e/d50/f5a x:0 0 0 2024-03-22T07:15:59.796 INFO:tasks.workunit.client.1.smithi069.stdout:2/248: creat d18/d4d/d25/f5b x:0 0 0 2024-03-22T07:15:59.797 INFO:tasks.workunit.client.1.smithi069.stdout:3/297: rmdir d5/d9/d23 39 2024-03-22T07:15:59.798 INFO:tasks.workunit.client.1.smithi069.stdout:3/298: fdatasync d5/d9/d26/d3f/d35/f58 0 2024-03-22T07:15:59.800 INFO:tasks.workunit.client.1.smithi069.stdout:3/299: rmdir d5/d9/d26/d3f 39 2024-03-22T07:15:59.806 INFO:tasks.workunit.client.1.smithi069.stdout:3/300: dread d5/d9/f3a [0,4194304] 0 2024-03-22T07:15:59.809 INFO:tasks.workunit.client.1.smithi069.stdout:3/301: rename d5/d9/db/d3b/l40 to d5/d9/d26/d36/d41/d51/l60 0 2024-03-22T07:15:59.809 INFO:tasks.workunit.client.1.smithi069.stdout:3/302: fdatasync d5/d9/d26/d3f/d35/f50 0 2024-03-22T07:15:59.810 INFO:tasks.workunit.client.1.smithi069.stdout:3/303: truncate d5/d9/d26/d3f/f3e 2065158 0 2024-03-22T07:15:59.810 INFO:tasks.workunit.client.1.smithi069.stdout:3/304: unlink d5/d9/d26/c38 0 2024-03-22T07:15:59.810 INFO:tasks.workunit.client.0.smithi063.stdout:1/691: rmdir d4/d14/ddd 0 2024-03-22T07:15:59.811 INFO:tasks.workunit.client.0.smithi063.stdout:2/564: link d0/de/d12/d20/f47 d0/de/d12/d20/fb3 0 2024-03-22T07:15:59.811 INFO:tasks.workunit.client.0.smithi063.stdout:0/742: symlink da/de/d17/d25/d3c/d53/d89/leb 0 2024-03-22T07:15:59.811 INFO:tasks.workunit.client.0.smithi063.stdout:0/743: chown da/de/d60/d9b/fa5 2663761 1 2024-03-22T07:15:59.811 INFO:tasks.workunit.client.0.smithi063.stdout:0/744: fsync f1 0 2024-03-22T07:15:59.812 INFO:tasks.workunit.client.0.smithi063.stdout:0/745: symlink da/lec 0 2024-03-22T07:15:59.813 INFO:tasks.workunit.client.1.smithi069.stdout:6/246: dwrite d4/f46 [0,4194304] 0 2024-03-22T07:15:59.813 INFO:tasks.workunit.client.1.smithi069.stdout:6/247: creat d4/dd/f4a x:0 0 0 2024-03-22T07:15:59.814 INFO:tasks.workunit.client.0.smithi063.stdout:9/481: dwrite d8/d19/d20/d22/d29/d31/f86 [0,4194304] 0 2024-03-22T07:15:59.814 INFO:tasks.workunit.client.0.smithi063.stdout:7/434: dwrite d5/d16/f2e [0,4194304] 0 2024-03-22T07:15:59.818 INFO:tasks.workunit.client.0.smithi063.stdout:1/692: getdents d4/df/d13/d39/d63 0 2024-03-22T07:15:59.818 INFO:tasks.workunit.client.0.smithi063.stdout:1/693: creat d4/d71/fec x:0 0 0 2024-03-22T07:15:59.820 INFO:tasks.workunit.client.0.smithi063.stdout:2/565: symlink d0/de/d12/d5a/lb4 0 2024-03-22T07:15:59.820 INFO:tasks.workunit.client.0.smithi063.stdout:4/488: dread d1/d5/d43/f17 [0,4194304] 0 2024-03-22T07:15:59.824 INFO:tasks.workunit.client.1.smithi069.stdout:2/249: dwrite d18/d4d/d34/f53 [0,4194304] 0 2024-03-22T07:15:59.825 INFO:tasks.workunit.client.1.smithi069.stdout:2/250: stat d18/d4d/c43 0 2024-03-22T07:15:59.825 INFO:tasks.workunit.client.0.smithi063.stdout:1/694: link d4/d5a/d91/lb4 d4/df/d13/d2b/d6c/d85/d9e/led 0 2024-03-22T07:15:59.826 INFO:tasks.workunit.client.1.smithi069.stdout:3/305: dread d5/f18 [0,4194304] 0 2024-03-22T07:15:59.827 INFO:tasks.workunit.client.0.smithi063.stdout:0/746: write da/de/d60/d9b/f9f [544560,30712] 0 2024-03-22T07:15:59.828 INFO:tasks.workunit.client.0.smithi063.stdout:4/489: mkdir d1/d5/dc/d95/d59/d9c/dac 0 2024-03-22T07:15:59.831 INFO:tasks.workunit.client.0.smithi063.stdout:9/482: dread d8/f90 [0,4194304] 0 2024-03-22T07:15:59.831 INFO:tasks.workunit.client.0.smithi063.stdout:0/747: symlink da/de/d17/d25/d3c/d43/d76/d92/led 0 2024-03-22T07:15:59.835 INFO:tasks.workunit.client.0.smithi063.stdout:2/566: rmdir d0/de/d1b/d63 39 2024-03-22T07:15:59.835 INFO:tasks.workunit.client.0.smithi063.stdout:1/695: dread d4/d14/f1b [0,4194304] 0 2024-03-22T07:15:59.835 INFO:tasks.workunit.client.0.smithi063.stdout:1/696: write d4/fd3 [95624,56905] 0 2024-03-22T07:15:59.835 INFO:tasks.workunit.client.0.smithi063.stdout:1/697: stat d4/df/d13/l20 0 2024-03-22T07:15:59.836 INFO:tasks.workunit.client.1.smithi069.stdout:2/251: dread d18/d4d/d27/d33/f39 [0,4194304] 0 2024-03-22T07:15:59.836 INFO:tasks.workunit.client.0.smithi063.stdout:4/490: unlink d1/f6a 0 2024-03-22T07:15:59.838 INFO:tasks.workunit.client.0.smithi063.stdout:9/483: creat d8/d17/f94 x:0 0 0 2024-03-22T07:15:59.838 INFO:tasks.workunit.client.0.smithi063.stdout:9/484: write d8/d15/f58 [1030018,40666] 0 2024-03-22T07:15:59.838 INFO:tasks.workunit.client.0.smithi063.stdout:2/567: readlink d0/l16 0 2024-03-22T07:15:59.839 INFO:tasks.workunit.client.1.smithi069.stdout:2/252: getdents d18/d4d/d27/d33/d3d 0 2024-03-22T07:15:59.840 INFO:tasks.workunit.client.0.smithi063.stdout:1/698: mkdir d4/d71/dbb/dc0/dd8/dee 0 2024-03-22T07:15:59.840 INFO:tasks.workunit.client.0.smithi063.stdout:1/699: dread - d4/df/f4e zero size 2024-03-22T07:15:59.840 INFO:tasks.workunit.client.1.smithi069.stdout:3/306: write d5/f5d [5319074,10104] 0 2024-03-22T07:15:59.842 INFO:tasks.workunit.client.1.smithi069.stdout:3/307: getdents d5/d9/d26/d36/d41/d51 0 2024-03-22T07:15:59.842 INFO:tasks.workunit.client.1.smithi069.stdout:3/308: stat d5/d9/d26/d3f/d54/c5c 0 2024-03-22T07:15:59.842 INFO:tasks.workunit.client.1.smithi069.stdout:3/309: write d5/d9/db/f3d [3646350,65540] 0 2024-03-22T07:15:59.843 INFO:tasks.workunit.client.0.smithi063.stdout:9/485: unlink d8/d15/d2f/f7a 0 2024-03-22T07:15:59.843 INFO:tasks.workunit.client.0.smithi063.stdout:9/486: write d8/d15/d2f/f47 [4821007,114649] 0 2024-03-22T07:15:59.843 INFO:tasks.workunit.client.0.smithi063.stdout:7/435: dwrite d5/d38/f81 [0,4194304] 0 2024-03-22T07:15:59.843 INFO:tasks.workunit.client.0.smithi063.stdout:7/436: stat d5/da/d5f/d33/d57/f67 0 2024-03-22T07:15:59.843 INFO:tasks.workunit.client.0.smithi063.stdout:7/437: write d5/d16/d1b/d21/d62/d73/d1d/d29/f47 [4803518,109042] 0 2024-03-22T07:15:59.843 INFO:tasks.workunit.client.0.smithi063.stdout:7/438: dread - d5/da/d5f/f8c zero size 2024-03-22T07:15:59.844 INFO:tasks.workunit.client.1.smithi069.stdout:3/310: rename d5/d9/d26/d3f/d35/d4f to d5/d9/d61 0 2024-03-22T07:15:59.847 INFO:tasks.workunit.client.1.smithi069.stdout:3/311: mknod d5/c62 0 2024-03-22T07:15:59.847 INFO:tasks.workunit.client.1.smithi069.stdout:3/312: chown d5/f7 71105497 1 2024-03-22T07:15:59.848 INFO:tasks.workunit.client.1.smithi069.stdout:3/313: write d5/d9/d23/f2b [940200,102452] 0 2024-03-22T07:15:59.849 INFO:tasks.workunit.client.0.smithi063.stdout:2/568: dread d0/de/d12/f1f [0,4194304] 0 2024-03-22T07:15:59.849 INFO:tasks.workunit.client.1.smithi069.stdout:6/248: dwrite d4/dd/d12/f1f [0,4194304] 0 2024-03-22T07:15:59.849 INFO:tasks.workunit.client.1.smithi069.stdout:3/314: rename d5/d9/f3a to d5/d9/db/f63 0 2024-03-22T07:15:59.850 INFO:tasks.workunit.client.1.smithi069.stdout:2/253: write d18/d4d/f22 [6348652,106741] 0 2024-03-22T07:15:59.852 INFO:tasks.workunit.client.1.smithi069.stdout:6/249: truncate d4/d7/f35 11135 0 2024-03-22T07:15:59.852 INFO:tasks.workunit.client.1.smithi069.stdout:6/250: fsync d4/dd/d12/d2b/f3a 0 2024-03-22T07:15:59.854 INFO:tasks.workunit.client.1.smithi069.stdout:3/315: rename d5/f31 to d5/f64 0 2024-03-22T07:15:59.855 INFO:tasks.workunit.client.0.smithi063.stdout:9/487: mkdir d8/d19/d20/d22/d5d/d76/d95 0 2024-03-22T07:15:59.855 INFO:tasks.workunit.client.1.smithi069.stdout:2/254: creat d18/d4d/d3e/d40/f5c x:0 0 0 2024-03-22T07:15:59.856 INFO:tasks.workunit.client.0.smithi063.stdout:7/439: mknod d5/d16/d1b/d21/d62/d73/d1d/d29/c93 0 2024-03-22T07:15:59.856 INFO:tasks.workunit.client.1.smithi069.stdout:6/251: symlink d4/dd/d12/d44/l4b 0 2024-03-22T07:15:59.862 INFO:tasks.workunit.client.1.smithi069.stdout:3/316: dread d5/d9/d26/d3f/f21 [0,4194304] 0 2024-03-22T07:15:59.863 INFO:tasks.workunit.client.0.smithi063.stdout:0/748: dwrite da/de/d17/d4e/f9e [0,4194304] 0 2024-03-22T07:15:59.863 INFO:tasks.workunit.client.0.smithi063.stdout:0/749: chown da/de/d60/f33 799435119 1 2024-03-22T07:15:59.864 INFO:tasks.workunit.client.1.smithi069.stdout:3/317: creat d5/d9/db/f65 x:0 0 0 2024-03-22T07:15:59.868 INFO:tasks.workunit.client.0.smithi063.stdout:0/750: creat da/de/d17/d25/d3c/d43/fee x:0 0 0 2024-03-22T07:15:59.873 INFO:tasks.workunit.client.0.smithi063.stdout:0/751: mknod da/de/d17/d25/d95/d98/db3/cef 0 2024-03-22T07:15:59.875 INFO:tasks.workunit.client.0.smithi063.stdout:4/491: rmdir d1 39 2024-03-22T07:15:59.875 INFO:tasks.workunit.client.1.smithi069.stdout:4/381: sync 2024-03-22T07:15:59.875 INFO:tasks.workunit.client.1.smithi069.stdout:7/317: sync 2024-03-22T07:15:59.875 INFO:tasks.workunit.client.1.smithi069.stdout:0/363: sync 2024-03-22T07:15:59.875 INFO:tasks.workunit.client.1.smithi069.stdout:7/318: read d9/db/f19 [1772147,106466] 0 2024-03-22T07:15:59.876 INFO:tasks.workunit.client.1.smithi069.stdout:4/382: stat d6/dc/d1d/l22 0 2024-03-22T07:15:59.876 INFO:tasks.workunit.client.0.smithi063.stdout:1/700: getdents d4/d71/dbb/dc0/dd8 0 2024-03-22T07:15:59.877 INFO:tasks.workunit.client.1.smithi069.stdout:7/319: rename d9/f3b to d9/f6f 0 2024-03-22T07:15:59.878 INFO:tasks.workunit.client.1.smithi069.stdout:7/320: chown d9/db/d48/d4b 22 1 2024-03-22T07:15:59.886 INFO:tasks.workunit.client.1.smithi069.stdout:3/318: dread d5/d9/d26/d3f/f22 [0,4194304] 0 2024-03-22T07:15:59.891 INFO:tasks.workunit.client.1.smithi069.stdout:7/321: dread d9/db/f19 [0,4194304] 0 2024-03-22T07:15:59.891 INFO:tasks.workunit.client.1.smithi069.stdout:4/383: dread d6/d12/d18/f4f [0,4194304] 0 2024-03-22T07:15:59.891 INFO:tasks.workunit.client.1.smithi069.stdout:4/384: fsync d6/dc/d1d/d31/d1f/d3f/f64 0 2024-03-22T07:15:59.892 INFO:tasks.workunit.client.1.smithi069.stdout:7/322: mknod d9/db/d48/d50/d57/d44/d53/d1d/d2a/d6e/c70 0 2024-03-22T07:15:59.892 INFO:tasks.workunit.client.1.smithi069.stdout:7/323: readlink d9/db/d1a/l1b 0 2024-03-22T07:15:59.892 INFO:tasks.workunit.client.1.smithi069.stdout:4/385: rmdir d6/dc/d23/d59 39 2024-03-22T07:15:59.908 INFO:tasks.workunit.client.1.smithi069.stdout:7/324: write d9/db/d48/d4b/f4f [2940350,121562] 0 2024-03-22T07:15:59.909 INFO:tasks.workunit.client.1.smithi069.stdout:7/325: write d9/db/d1a/f3c [270044,129786] 0 2024-03-22T07:15:59.909 INFO:tasks.workunit.client.1.smithi069.stdout:7/326: creat d9/d42/f71 x:0 0 0 2024-03-22T07:15:59.909 INFO:tasks.workunit.client.1.smithi069.stdout:7/327: chown d9/db/d1a/f56 241079003 1 2024-03-22T07:15:59.911 INFO:tasks.workunit.client.1.smithi069.stdout:7/328: symlink d9/db/d48/d50/d57/d44/l72 0 2024-03-22T07:15:59.911 INFO:tasks.workunit.client.1.smithi069.stdout:7/329: truncate d9/db/d1a/f3c 638175 0 2024-03-22T07:15:59.911 INFO:tasks.workunit.client.1.smithi069.stdout:4/386: dread d6/dc/d23/f5c [0,4194304] 0 2024-03-22T07:15:59.911 INFO:tasks.workunit.client.1.smithi069.stdout:4/387: readlink d6/l8 0 2024-03-22T07:15:59.913 INFO:tasks.workunit.client.1.smithi069.stdout:2/255: dwrite f6 [0,4194304] 0 2024-03-22T07:15:59.913 INFO:tasks.workunit.client.1.smithi069.stdout:7/330: unlink d9/db/d48/d50/d57/d44/d53/f20 0 2024-03-22T07:15:59.913 INFO:tasks.workunit.client.1.smithi069.stdout:6/252: dwrite d4/dd/d12/d2b/f36 [0,4194304] 0 2024-03-22T07:15:59.914 INFO:tasks.workunit.client.0.smithi063.stdout:9/488: dwrite d8/d19/d2b/f89 [0,4194304] 0 2024-03-22T07:15:59.914 INFO:tasks.workunit.client.0.smithi063.stdout:7/440: dwrite d5/da/f76 [4194304,4194304] 0 2024-03-22T07:15:59.915 INFO:tasks.workunit.client.0.smithi063.stdout:7/441: fsync d5/d16/f22 0 2024-03-22T07:15:59.915 INFO:tasks.workunit.client.0.smithi063.stdout:7/442: write d5/d16/d43/f56 [2067160,54716] 0 2024-03-22T07:15:59.915 INFO:tasks.workunit.client.0.smithi063.stdout:4/492: dwrite d1/d5/f67 [0,4194304] 0 2024-03-22T07:15:59.915 INFO:tasks.workunit.client.0.smithi063.stdout:1/701: dwrite f2 [0,4194304] 0 2024-03-22T07:15:59.917 INFO:tasks.workunit.client.0.smithi063.stdout:7/443: mknod d5/d16/d1b/d21/d62/d73/d1d/d29/d3b/c94 0 2024-03-22T07:15:59.918 INFO:tasks.workunit.client.0.smithi063.stdout:4/493: creat d1/d5/dc/d95/d59/d9c/fad x:0 0 0 2024-03-22T07:15:59.919 INFO:tasks.workunit.client.1.smithi069.stdout:0/364: dwrite d3/d7/d15/f32 [0,4194304] 0 2024-03-22T07:15:59.919 INFO:tasks.workunit.client.1.smithi069.stdout:2/256: link d18/d4d/d34/f4a d18/d4d/d3e/f5d 0 2024-03-22T07:15:59.919 INFO:tasks.workunit.client.1.smithi069.stdout:2/257: readlink l10 0 2024-03-22T07:15:59.921 INFO:tasks.workunit.client.0.smithi063.stdout:7/444: link d5/d16/d1b/f7c d5/d16/d1b/d21/d62/d73/d1d/d4f/f95 0 2024-03-22T07:15:59.922 INFO:tasks.workunit.client.1.smithi069.stdout:7/331: link d9/db/d48/d50/d57/d44/d53/c37 d9/db/d48/d50/d57/d44/d53/d1d/d2a/d6e/c73 0 2024-03-22T07:15:59.922 INFO:tasks.workunit.client.1.smithi069.stdout:7/332: readlink d9/db/d48/d50/d57/d44/d53/d1d/l6b 0 2024-03-22T07:15:59.924 INFO:tasks.workunit.client.1.smithi069.stdout:3/319: dread d5/d9/db/f3d [0,4194304] 0 2024-03-22T07:15:59.924 INFO:tasks.workunit.client.1.smithi069.stdout:3/320: chown d5/d33 3 1 2024-03-22T07:15:59.926 INFO:tasks.workunit.client.0.smithi063.stdout:4/494: write d1/d5/dc/d95/f44 [662672,55135] 0 2024-03-22T07:15:59.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.927+0000 7fa47ed99700 1 -- 172.21.15.63:0/847805565 --> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa478104910 con 0x7fa46806ad90 2024-03-22T07:15:59.928 INFO:tasks.workunit.client.0.smithi063.stdout:4/495: mknod d1/d5/dc/d95/da4/cae 0 2024-03-22T07:15:59.929 INFO:tasks.workunit.client.1.smithi069.stdout:6/253: symlink d4/d7/da/l4c 0 2024-03-22T07:15:59.929 INFO:tasks.workunit.client.0.smithi063.stdout:1/702: dread f2 [0,4194304] 0 2024-03-22T07:15:59.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.929+0000 7fa4757fa700 1 -- 172.21.15.63:0/847805565 <== mgr.14208 v2:172.21.15.63:6800/2001368263 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fa478104910 con 0x7fa46806ad90 2024-03-22T07:15:59.931 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:15:59.931 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:15:59.931 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:15:59.932 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-22T07:15:59.932 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-03-22T07:15:59.932 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "1/2 daemons upgraded", 2024-03-22T07:15:59.932 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mgr daemons", 2024-03-22T07:15:59.932 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:15:59.932 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:15:59.932 INFO:tasks.workunit.client.1.smithi069.stdout:3/321: unlink d5/l11 0 2024-03-22T07:15:59.933 INFO:tasks.workunit.client.0.smithi063.stdout:1/703: link d4/df/d13/d39/d63/c83 d4/df/d13/d2b/d6c/da8/cef 0 2024-03-22T07:15:59.933 INFO:tasks.workunit.client.0.smithi063.stdout:4/496: unlink d1/d5/f84 0 2024-03-22T07:15:59.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.934+0000 7fa47ed99700 1 -- 172.21.15.63:0/847805565 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa46806ad90 msgr2=0x7fa46806d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.934+0000 7fa47ed99700 1 --2- 172.21.15.63:0/847805565 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa46806ad90 0x7fa46806d240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa46c004f40 tx=0x7fa46c00b040 comp rx=0 tx=0).stop 2024-03-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.934+0000 7fa47ed99700 1 -- 172.21.15.63:0/847805565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4780fcf80 msgr2=0x7fa47810df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.934+0000 7fa47ed99700 1 --2- 172.21.15.63:0/847805565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4780fcf80 0x7fa47810df50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa464009150 tx=0x7fa464009e30 comp rx=0 tx=0).stop 2024-03-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.934+0000 7fa47ed99700 1 -- 172.21.15.63:0/847805565 shutdown_connections 2024-03-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.934+0000 7fa47ed99700 1 --2- 172.21.15.63:0/847805565 >> [v2:172.21.15.63:6800/2001368263,v1:172.21.15.63:6801/2001368263] conn(0x7fa46806ad90 0x7fa46806d240 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.934+0000 7fa47ed99700 1 --2- 172.21.15.63:0/847805565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4780fbcd0 0x7fa47810da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:59.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.934+0000 7fa47ed99700 1 --2- 172.21.15.63:0/847805565 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa4780fcf80 0x7fa47810df50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:15:59.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.934+0000 7fa47ed99700 1 -- 172.21.15.63:0/847805565 >> 172.21.15.63:0/847805565 conn(0x7fa4780f7430 msgr2=0x7fa478065b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:15:59.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.935+0000 7fa47ed99700 1 -- 172.21.15.63:0/847805565 shutdown_connections 2024-03-22T07:15:59.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:15:59.935+0000 7fa47ed99700 1 -- 172.21.15.63:0/847805565 wait complete. 2024-03-22T07:15:59.936 INFO:tasks.workunit.client.0.smithi063.stdout:1/704: rename d4/l16 to d4/d71/dbb/dc0/dd8/dee/lf0 0 2024-03-22T07:15:59.940 INFO:tasks.workunit.client.0.smithi063.stdout:7/445: dread d5/f7 [4194304,4194304] 0 2024-03-22T07:15:59.940 INFO:tasks.workunit.client.0.smithi063.stdout:7/446: chown d5/d16/d1b/c23 129695129 1 2024-03-22T07:15:59.940 INFO:tasks.workunit.client.0.smithi063.stdout:7/447: write d5/d16/d1b/d21/f86 [55869,80315] 0 2024-03-22T07:15:59.940 INFO:tasks.workunit.client.0.smithi063.stdout:7/448: chown d5/d16/d1b/f7c 6153059 1 2024-03-22T07:15:59.940 INFO:tasks.workunit.client.0.smithi063.stdout:4/497: rename d1/d5/f12 to d1/d5/dc/d95/d59/faf 0 2024-03-22T07:15:59.944 INFO:tasks.workunit.client.0.smithi063.stdout:4/498: dread d1/d5/dc/d95/d59/f87 [0,4194304] 0 2024-03-22T07:15:59.944 INFO:tasks.workunit.client.0.smithi063.stdout:7/449: getdents d5/d16/d1b/d21/d62/d73/d1d 0 2024-03-22T07:15:59.944 INFO:tasks.workunit.client.0.smithi063.stdout:7/450: chown d5/da/d4e/l75 107 1 2024-03-22T07:15:59.945 INFO:tasks.workunit.client.1.smithi069.stdout:7/333: rename d9/db/d48/d50/d57/d44/d53/d1d/d2a to d9/db/d48/d50/d57/d44/d53/d74 0 2024-03-22T07:15:59.945 INFO:tasks.workunit.client.1.smithi069.stdout:4/388: dwrite d6/f30 [0,4194304] 0 2024-03-22T07:15:59.945 INFO:tasks.workunit.client.1.smithi069.stdout:4/389: creat d6/d12/d18/d21/f79 x:0 0 0 2024-03-22T07:15:59.945 INFO:tasks.workunit.client.0.smithi063.stdout:9/489: dwrite d8/f10 [0,4194304] 0 2024-03-22T07:15:59.946 INFO:tasks.workunit.client.0.smithi063.stdout:4/499: creat d1/d5/d43/d27/d2c/d3d/d54/d5f/d79/fb0 x:0 0 0 2024-03-22T07:15:59.947 INFO:tasks.workunit.client.0.smithi063.stdout:7/451: rename d5/d16/d1b/d21/d62/d73/d1d/d29/l30 to d5/da/d28/d4b/l96 0 2024-03-22T07:15:59.947 INFO:tasks.workunit.client.0.smithi063.stdout:7/452: truncate d5/da/f25 3364965 0 2024-03-22T07:15:59.947 INFO:tasks.workunit.client.0.smithi063.stdout:7/453: creat d5/da/d5f/f97 x:0 0 0 2024-03-22T07:15:59.947 INFO:tasks.workunit.client.1.smithi069.stdout:7/334: mkdir d9/db/d48/d50/d57/d75 0 2024-03-22T07:15:59.949 INFO:tasks.workunit.client.1.smithi069.stdout:3/322: rename d5/d9/l20 to d5/d9/db/d3b/l66 0 2024-03-22T07:15:59.951 INFO:tasks.workunit.client.0.smithi063.stdout:4/500: dread d1/d5/f28 [0,4194304] 0 2024-03-22T07:15:59.951 INFO:tasks.workunit.client.0.smithi063.stdout:4/501: stat d1/d5/dc/d95/d59/la5 0 2024-03-22T07:15:59.951 INFO:tasks.workunit.client.0.smithi063.stdout:7/454: mkdir d5/d16/d1b/d21/d62/d8d/d98 0 2024-03-22T07:15:59.951 INFO:tasks.workunit.client.0.smithi063.stdout:7/455: chown d5/d16/d1b/d21/d62/d73/d1d/l20 0 1 2024-03-22T07:15:59.952 INFO:tasks.workunit.client.0.smithi063.stdout:4/502: write d1/d5/dc/d95/d59/f87 [1257538,87333] 0 2024-03-22T07:15:59.953 INFO:tasks.workunit.client.1.smithi069.stdout:4/390: getdents d6/dc/d23/d59 0 2024-03-22T07:15:59.954 INFO:tasks.workunit.client.1.smithi069.stdout:4/391: truncate d6/dc/d23/f77 753204 0 2024-03-22T07:15:59.954 INFO:tasks.workunit.client.1.smithi069.stdout:4/392: creat d6/dc/d1d/d31/d1f/f7a x:0 0 0 2024-03-22T07:15:59.954 INFO:tasks.workunit.client.1.smithi069.stdout:4/393: write d6/d12/f6f [476265,73523] 0 2024-03-22T07:15:59.954 INFO:tasks.workunit.client.1.smithi069.stdout:4/394: stat d6/d16/d3a/l50 0 2024-03-22T07:15:59.955 INFO:tasks.workunit.client.1.smithi069.stdout:7/335: dread - d9/db/d48/d50/d57/d44/d53/f60 zero size 2024-03-22T07:15:59.959 INFO:tasks.workunit.client.1.smithi069.stdout:6/254: dwrite d4/dd/f4a [0,4194304] 0 2024-03-22T07:15:59.959 INFO:tasks.workunit.client.1.smithi069.stdout:7/336: unlink d9/db/d48/d50/d57/d44/d53/d2e/f4c 0 2024-03-22T07:15:59.962 INFO:tasks.workunit.client.1.smithi069.stdout:6/255: link d4/dd/d12/d13/c20 d4/d7/da/d3f/c4d 0 2024-03-22T07:15:59.963 INFO:tasks.workunit.client.0.smithi063.stdout:3/634: sync 2024-03-22T07:15:59.963 INFO:tasks.workunit.client.0.smithi063.stdout:7/456: dread d5/d16/d43/f56 [0,4194304] 0 2024-03-22T07:15:59.965 INFO:tasks.workunit.client.0.smithi063.stdout:3/635: unlink da/d16/f2d 0 2024-03-22T07:15:59.966 INFO:tasks.workunit.client.0.smithi063.stdout:7/457: symlink d5/d16/d1b/d21/d62/d73/d1d/d4f/l99 0 2024-03-22T07:15:59.967 INFO:tasks.workunit.client.0.smithi063.stdout:7/458: stat d5/d16/d1b 0 2024-03-22T07:15:59.967 INFO:tasks.workunit.client.0.smithi063.stdout:7/459: dread - d5/d16/d1b/d21/d62/d73/d1d/d29/d3b/f72 zero size 2024-03-22T07:15:59.968 INFO:tasks.workunit.client.0.smithi063.stdout:3/636: mkdir da/d16/d89/da9/dd3 0 2024-03-22T07:15:59.968 INFO:tasks.workunit.client.0.smithi063.stdout:3/637: chown da/d16/l1f 8285 1 2024-03-22T07:15:59.968 INFO:tasks.workunit.client.0.smithi063.stdout:3/638: dread - da/d63/d65/d4b/f85 zero size 2024-03-22T07:15:59.968 INFO:tasks.workunit.client.0.smithi063.stdout:7/460: creat d5/da/d28/f9a x:0 0 0 2024-03-22T07:15:59.970 INFO:tasks.workunit.client.0.smithi063.stdout:3/639: truncate da/d63/d65/f5e 1295598 0 2024-03-22T07:15:59.970 INFO:tasks.workunit.client.0.smithi063.stdout:3/640: chown da/d63/f75 84 1 2024-03-22T07:15:59.970 INFO:tasks.workunit.client.1.smithi069.stdout:4/395: write f5 [3811955,126250] 0 2024-03-22T07:15:59.972 INFO:tasks.workunit.client.0.smithi063.stdout:7/461: rename d5/d16/d1b/d21/d62/d73/d1d/d29/d3b/c7f to d5/d16/d1b/d21/d62/d73/d1d/c9b 0 2024-03-22T07:15:59.973 INFO:tasks.workunit.client.1.smithi069.stdout:4/396: link d6/d16/d3a/l50 d6/dc/d1d/d31/l7b 0 2024-03-22T07:15:59.973 INFO:tasks.workunit.client.0.smithi063.stdout:3/641: rename da/l30 to da/d63/d65/d81/d9b/d9f/ld4 0 2024-03-22T07:15:59.973 INFO:tasks.workunit.client.0.smithi063.stdout:3/642: chown da/f6a 2 1 2024-03-22T07:15:59.973 INFO:tasks.workunit.client.0.smithi063.stdout:3/643: fdatasync da/d63/d65/f99 0 2024-03-22T07:15:59.974 INFO:tasks.workunit.client.1.smithi069.stdout:4/397: mkdir d6/dc/d1d/d31/d1f/d58/d7c 0 2024-03-22T07:15:59.974 INFO:tasks.workunit.client.1.smithi069.stdout:4/398: stat d6/dc/d1d/c61 0 2024-03-22T07:15:59.974 INFO:tasks.workunit.client.1.smithi069.stdout:4/399: fdatasync d6/d16/d68/f78 0 2024-03-22T07:15:59.975 INFO:tasks.workunit.client.0.smithi063.stdout:3/644: creat da/d16/d3f/dc7/fd5 x:0 0 0 2024-03-22T07:15:59.975 INFO:tasks.workunit.client.0.smithi063.stdout:3/645: symlink da/d16/d26/d69/dae/ld6 0 2024-03-22T07:15:59.977 INFO:tasks.workunit.client.1.smithi069.stdout:2/258: dwrite d18/d4d/d25/f47 [0,4194304] 0 2024-03-22T07:15:59.980 INFO:tasks.workunit.client.1.smithi069.stdout:2/259: creat d18/d4d/d27/f5e x:0 0 0 2024-03-22T07:15:59.982 INFO:tasks.workunit.client.1.smithi069.stdout:4/400: dread d6/d12/f33 [0,4194304] 0 2024-03-22T07:15:59.982 INFO:tasks.workunit.client.1.smithi069.stdout:4/401: stat d6/d4b/c4c 0 2024-03-22T07:15:59.983 INFO:tasks.workunit.client.1.smithi069.stdout:4/402: creat d6/dc/d23/d59/f7d x:0 0 0 2024-03-22T07:15:59.989 INFO:tasks.workunit.client.1.smithi069.stdout:4/403: dread d6/dc/d1d/d31/d1f/f5b [0,4194304] 0 2024-03-22T07:15:59.989 INFO:tasks.workunit.client.1.smithi069.stdout:4/404: creat d6/d16/f7e x:0 0 0 2024-03-22T07:15:59.989 INFO:tasks.workunit.client.1.smithi069.stdout:4/405: creat d6/dc/d1d/d31/d1f/f7f x:0 0 0 2024-03-22T07:15:59.989 INFO:tasks.workunit.client.1.smithi069.stdout:4/406: fdatasync d6/d12/f41 0 2024-03-22T07:16:00.000 INFO:tasks.workunit.client.0.smithi063.stdout:1/705: dwrite d4/df/d13/d39/d6f/dd9/fe9 [0,4194304] 0 2024-03-22T07:16:00.000 INFO:tasks.workunit.client.0.smithi063.stdout:1/706: chown d4/df/d13/d2b/c4a 9 1 2024-03-22T07:16:00.014 INFO:tasks.workunit.client.1.smithi069.stdout:3/323: dwrite d5/d9/d23/f2b [0,4194304] 0 2024-03-22T07:16:00.015 INFO:tasks.workunit.client.0.smithi063.stdout:9/490: dwrite d8/d19/d20/d22/f65 [0,4194304] 0 2024-03-22T07:16:00.015 INFO:tasks.workunit.client.0.smithi063.stdout:9/491: truncate d8/d19/d20/d22/d29/d31/d37/f59 2076640 0 2024-03-22T07:16:00.016 INFO:tasks.workunit.client.0.smithi063.stdout:4/503: dwrite d1/d5/d43/f35 [0,4194304] 0 2024-03-22T07:16:00.016 INFO:tasks.workunit.client.0.smithi063.stdout:4/504: dread - d1/d5/f97 zero size 2024-03-22T07:16:00.016 INFO:tasks.workunit.client.0.smithi063.stdout:4/505: fsync d1/f5b 0 2024-03-22T07:16:00.016 INFO:tasks.workunit.client.1.smithi069.stdout:6/256: dwrite d4/dd/d12/d2b/f37 [0,4194304] 0 2024-03-22T07:16:00.018 INFO:tasks.workunit.client.0.smithi063.stdout:9/492: mknod d8/d19/d20/c96 0 2024-03-22T07:16:00.018 INFO:tasks.workunit.client.0.smithi063.stdout:9/493: chown d8/d19/d20/d22/d29/d31/f86 21 1 2024-03-22T07:16:00.018 INFO:tasks.workunit.client.0.smithi063.stdout:7/462: dwrite d5/d16/d1b/d21/d62/d73/d1d/d29/d3b/f64 [0,4194304] 0 2024-03-22T07:16:00.018 INFO:tasks.workunit.client.1.smithi069.stdout:4/407: dwrite d6/dc/d1d/d31/f14 [0,4194304] 0 2024-03-22T07:16:00.018 INFO:tasks.workunit.client.1.smithi069.stdout:4/408: chown d6/c2c 0 1 2024-03-22T07:16:00.018 INFO:tasks.workunit.client.1.smithi069.stdout:4/409: creat d6/d12/f80 x:0 0 0 2024-03-22T07:16:00.019 INFO:tasks.workunit.client.0.smithi063.stdout:4/506: fdatasync d1/d5/f2f 0 2024-03-22T07:16:00.023 INFO:tasks.workunit.client.0.smithi063.stdout:7/463: write d5/f7 [8008663,63644] 0 2024-03-22T07:16:00.026 INFO:tasks.workunit.client.0.smithi063.stdout:9/494: rename d8/d17/f73 to d8/d19/d20/d22/d29/d31/d53/d6a/f97 0 2024-03-22T07:16:00.026 INFO:tasks.workunit.client.1.smithi069.stdout:6/257: creat d4/dd/d12/d44/f4e x:0 0 0 2024-03-22T07:16:00.028 INFO:tasks.workunit.client.0.smithi063.stdout:4/507: truncate d1/d5/d43/d27/d2c/d4a/f4d 4549855 0 2024-03-22T07:16:00.028 INFO:tasks.workunit.client.0.smithi063.stdout:4/508: chown d1/d5/dc/d95/l56 1 1 2024-03-22T07:16:00.031 INFO:tasks.workunit.client.1.smithi069.stdout:6/258: symlink d4/d7/l4f 0 2024-03-22T07:16:00.033 INFO:tasks.workunit.client.1.smithi069.stdout:6/259: creat d4/d7/f50 x:0 0 0 2024-03-22T07:16:00.037 INFO:tasks.workunit.client.1.smithi069.stdout:5/244: sync 2024-03-22T07:16:00.038 INFO:tasks.workunit.client.1.smithi069.stdout:1/301: sync 2024-03-22T07:16:00.038 INFO:tasks.workunit.client.1.smithi069.stdout:5/245: chown d1/d7/d8/d43 23256727 1 2024-03-22T07:16:00.038 INFO:tasks.workunit.client.1.smithi069.stdout:8/391: sync 2024-03-22T07:16:00.038 INFO:tasks.workunit.client.1.smithi069.stdout:8/392: stat d1/d1c/d3e/d66/f75 0 2024-03-22T07:16:00.039 INFO:tasks.workunit.client.0.smithi063.stdout:9/495: rename d8/d19/d2b/c30 to d8/d19/d20/d22/d29/d31/d53/d6a/c98 0 2024-03-22T07:16:00.039 INFO:tasks.workunit.client.0.smithi063.stdout:9/496: stat d8/d17/d1e/l1f 0 2024-03-22T07:16:00.039 INFO:tasks.workunit.client.1.smithi069.stdout:9/277: sync 2024-03-22T07:16:00.041 INFO:tasks.workunit.client.1.smithi069.stdout:3/324: dwrite d5/d33/f39 [0,4194304] 0 2024-03-22T07:16:00.041 INFO:tasks.workunit.client.1.smithi069.stdout:7/337: rename d9/db/d48/d50 to d9/d76 0 2024-03-22T07:16:00.045 INFO:tasks.workunit.client.1.smithi069.stdout:1/302: creat d1/d2a/d50/d4c/d55/f62 x:0 0 0 2024-03-22T07:16:00.045 INFO:tasks.workunit.client.1.smithi069.stdout:1/303: stat d1/fb 0 2024-03-22T07:16:00.047 INFO:tasks.workunit.client.1.smithi069.stdout:4/410: dwrite d6/f30 [0,4194304] 0 2024-03-22T07:16:00.049 INFO:tasks.workunit.client.1.smithi069.stdout:9/278: symlink d2/l58 0 2024-03-22T07:16:00.049 INFO:tasks.workunit.client.1.smithi069.stdout:9/279: creat d2/d51/f59 x:0 0 0 2024-03-22T07:16:00.053 INFO:tasks.workunit.client.1.smithi069.stdout:9/280: dread d2/d31/f45 [0,4194304] 0 2024-03-22T07:16:00.053 INFO:tasks.workunit.client.1.smithi069.stdout:9/281: chown d2/dd/f26 9966623 1 2024-03-22T07:16:00.055 INFO:tasks.workunit.client.1.smithi069.stdout:2/260: rename d18/d4d/d27/d33/f39 to d18/d4d/d34/f5f 0 2024-03-22T07:16:00.056 INFO:tasks.workunit.client.1.smithi069.stdout:1/304: mkdir d1/d2a/d50/d4c/d63 0 2024-03-22T07:16:00.056 INFO:tasks.workunit.client.1.smithi069.stdout:1/305: truncate d1/fb 1628116 0 2024-03-22T07:16:00.056 INFO:tasks.workunit.client.1.smithi069.stdout:1/306: fdatasync d1/d13/f1b 0 2024-03-22T07:16:00.056 INFO:tasks.workunit.client.1.smithi069.stdout:1/307: truncate d1/d13/d36/f3e 1193657 0 2024-03-22T07:16:00.056 INFO:tasks.workunit.client.1.smithi069.stdout:1/308: chown d1/d13/c15 2333 1 2024-03-22T07:16:00.057 INFO:tasks.workunit.client.1.smithi069.stdout:4/411: creat d6/dc/d1d/d31/d1f/d3f/d51/f81 x:0 0 0 2024-03-22T07:16:00.057 INFO:tasks.workunit.client.1.smithi069.stdout:1/309: dread d1/fb [0,4194304] 0 2024-03-22T07:16:00.058 INFO:tasks.workunit.client.1.smithi069.stdout:9/282: unlink d2/ca 0 2024-03-22T07:16:00.061 INFO:tasks.workunit.client.1.smithi069.stdout:6/260: rename d4/dd/d12/f34 to d4/dd/d41/f51 0 2024-03-22T07:16:00.062 INFO:tasks.workunit.client.1.smithi069.stdout:7/338: truncate d9/db/d48/d4b/f65 288418 0 2024-03-22T07:16:00.062 INFO:tasks.workunit.client.1.smithi069.stdout:7/339: write d9/db/d48/f5b [335908,81080] 0 2024-03-22T07:16:00.064 INFO:tasks.workunit.client.1.smithi069.stdout:1/310: mkdir d1/d25/d41/d64 0 2024-03-22T07:16:00.064 INFO:tasks.workunit.client.1.smithi069.stdout:1/311: write d1/d25/f4d [367580,47998] 0 2024-03-22T07:16:00.066 INFO:tasks.workunit.client.1.smithi069.stdout:6/261: symlink d4/l52 0 2024-03-22T07:16:00.066 INFO:tasks.workunit.client.1.smithi069.stdout:6/262: fdatasync f3 0 2024-03-22T07:16:00.067 INFO:tasks.workunit.client.1.smithi069.stdout:4/412: mkdir d6/d82 0 2024-03-22T07:16:00.068 INFO:tasks.workunit.client.0.smithi063.stdout:7/464: dwrite d5/d16/d1b/d21/d62/d73/d1d/d29/f5d [4194304,4194304] 0 2024-03-22T07:16:00.068 INFO:tasks.workunit.client.1.smithi069.stdout:1/312: symlink d1/d25/d41/d1e/l65 0 2024-03-22T07:16:00.068 INFO:tasks.workunit.client.1.smithi069.stdout:1/313: chown d1/d13/f26 205584 1 2024-03-22T07:16:00.069 INFO:tasks.workunit.client.1.smithi069.stdout:4/413: mkdir d6/d16/d83 0 2024-03-22T07:16:00.072 INFO:tasks.workunit.client.0.smithi063.stdout:4/509: dwrite d1/d5/dc/fa7 [0,4194304] 0 2024-03-22T07:16:00.073 INFO:tasks.workunit.client.1.smithi069.stdout:4/414: dread d6/dc/d1d/d31/d1f/f62 [0,4194304] 0 2024-03-22T07:16:00.078 INFO:tasks.workunit.client.0.smithi063.stdout:4/510: symlink d1/d5/d43/d27/d2c/d3d/d54/d5f/d79/daa/lb1 0 2024-03-22T07:16:00.079 INFO:tasks.workunit.client.0.smithi063.stdout:4/511: creat d1/d5/d43/d27/d2c/d4a/d78/fb2 x:0 0 0 2024-03-22T07:16:00.083 INFO:tasks.workunit.client.1.smithi069.stdout:9/283: dread d2/f27 [0,4194304] 0 2024-03-22T07:16:00.083 INFO:tasks.workunit.client.1.smithi069.stdout:9/284: write d2/dd/f10 [53631,3079] 0 2024-03-22T07:16:00.083 INFO:tasks.workunit.client.1.smithi069.stdout:9/285: read - d2/dd/d3c/d2e/f4e zero size 2024-03-22T07:16:00.083 INFO:tasks.workunit.client.1.smithi069.stdout:9/286: mknod d2/d51/c5a 0 2024-03-22T07:16:00.084 INFO:tasks.workunit.client.0.smithi063.stdout:4/512: dread - d1/d5/dc/f89 zero size 2024-03-22T07:16:00.084 INFO:tasks.workunit.client.0.smithi063.stdout:4/513: mknod d1/d5/d3c/cb3 0 2024-03-22T07:16:00.084 INFO:tasks.workunit.client.0.smithi063.stdout:4/514: truncate d1/d5/dc/d95/d59/f64 4092888 0 2024-03-22T07:16:00.086 INFO:tasks.workunit.client.0.smithi063.stdout:4/515: unlink d1/d5/d43/d27/d2c/c58 0 2024-03-22T07:16:00.087 INFO:tasks.workunit.client.0.smithi063.stdout:4/516: creat d1/fb4 x:0 0 0 2024-03-22T07:16:00.089 INFO:tasks.workunit.client.0.smithi063.stdout:4/517: rename d1/fb4 to d1/d5/dc/d95/fb5 0 2024-03-22T07:16:00.089 INFO:tasks.workunit.client.0.smithi063.stdout:4/518: write d1/d5/d43/f17 [4262448,73938] 0 2024-03-22T07:16:00.091 INFO:tasks.workunit.client.1.smithi069.stdout:8/393: dwrite d1/de/f40 [4194304,4194304] 0 2024-03-22T07:16:00.091 INFO:tasks.workunit.client.1.smithi069.stdout:8/394: chown d1/d1c/f22 846969 1 2024-03-22T07:16:00.091 INFO:tasks.workunit.client.1.smithi069.stdout:8/395: fdatasync d1/de/f30 0 2024-03-22T07:16:00.091 INFO:tasks.workunit.client.1.smithi069.stdout:9/287: read d2/f27 [1643498,102459] 0 2024-03-22T07:16:00.092 INFO:tasks.workunit.client.1.smithi069.stdout:5/246: dwrite d1/d7/f22 [0,4194304] 0 2024-03-22T07:16:00.092 INFO:tasks.workunit.client.1.smithi069.stdout:5/247: readlink d1/d7/d3d/l48 0 2024-03-22T07:16:00.093 INFO:tasks.workunit.client.0.smithi063.stdout:4/519: dread d1/d5/d3c/d65/f69 [0,4194304] 0 2024-03-22T07:16:00.093 INFO:tasks.workunit.client.1.smithi069.stdout:9/288: symlink d2/dd/d3c/d48/l5b 0 2024-03-22T07:16:00.093 INFO:tasks.workunit.client.1.smithi069.stdout:9/289: getdents d2/dd/d3c/d53 0 2024-03-22T07:16:00.094 INFO:tasks.workunit.client.1.smithi069.stdout:9/290: chown d2/dd/d3c/f35 21869 1 2024-03-22T07:16:00.094 INFO:tasks.workunit.client.1.smithi069.stdout:9/291: write d2/d51/f56 [110942,108275] 0 2024-03-22T07:16:00.095 INFO:tasks.workunit.client.1.smithi069.stdout:6/263: dread d4/dd/d12/d2b/f36 [0,4194304] 0 2024-03-22T07:16:00.095 INFO:tasks.workunit.client.0.smithi063.stdout:4/520: creat d1/d5/d43/d27/d2c/d3d/d54/d5f/d79/daa/fb6 x:0 0 0 2024-03-22T07:16:00.095 INFO:tasks.workunit.client.0.smithi063.stdout:4/521: creat d1/d5/d43/d27/d2c/d3d/d54/d5f/d79/daa/fb7 x:0 0 0 2024-03-22T07:16:00.096 INFO:tasks.workunit.client.1.smithi069.stdout:5/248: dread d1/d6/d26/d33/f38 [0,4194304] 0 2024-03-22T07:16:00.096 INFO:tasks.workunit.client.0.smithi063.stdout:4/522: creat d1/d5/dc/fb8 x:0 0 0 2024-03-22T07:16:00.097 INFO:tasks.workunit.client.1.smithi069.stdout:9/292: getdents d2/dd/d3c 0 2024-03-22T07:16:00.098 INFO:tasks.workunit.client.1.smithi069.stdout:9/293: creat d2/f5c x:0 0 0 2024-03-22T07:16:00.105 INFO:tasks.workunit.client.0.smithi063.stdout:5/510: sync 2024-03-22T07:16:00.105 INFO:tasks.workunit.client.1.smithi069.stdout:6/264: dread d4/dd/f4a [0,4194304] 0 2024-03-22T07:16:00.105 INFO:tasks.workunit.client.1.smithi069.stdout:6/265: fdatasync d4/dd/d12/f32 0 2024-03-22T07:16:00.106 INFO:tasks.workunit.client.0.smithi063.stdout:5/511: unlink d8/l8e 0 2024-03-22T07:16:00.106 INFO:tasks.workunit.client.0.smithi063.stdout:5/512: write d8/d13/f63 [4570438,9035] 0 2024-03-22T07:16:00.107 INFO:tasks.workunit.client.1.smithi069.stdout:6/266: truncate d4/dd/f33 3620590 0 2024-03-22T07:16:00.107 INFO:tasks.workunit.client.1.smithi069.stdout:6/267: truncate d4/dd/d12/d44/f4e 345270 0 2024-03-22T07:16:00.107 INFO:tasks.workunit.client.0.smithi063.stdout:5/513: mknod d8/d13/d1e/d31/d64/c90 0 2024-03-22T07:16:00.107 INFO:tasks.workunit.client.0.smithi063.stdout:5/514: creat d8/d13/d28/d57/d5f/d87/f91 x:0 0 0 2024-03-22T07:16:00.109 INFO:tasks.workunit.client.1.smithi069.stdout:9/294: dread d2/f55 [0,4194304] 0 2024-03-22T07:16:00.109 INFO:tasks.workunit.client.1.smithi069.stdout:9/295: chown d2/l4 244423372 1 2024-03-22T07:16:00.110 INFO:tasks.workunit.client.0.smithi063.stdout:9/497: unlink d8/d19/d20/d22/d29/d31/d53/d6a/c98 0 2024-03-22T07:16:00.110 INFO:tasks.workunit.client.0.smithi063.stdout:9/498: chown d8/d19/d20/d22/d29/d31 0 1 2024-03-22T07:16:00.110 INFO:tasks.workunit.client.0.smithi063.stdout:5/515: rmdir d8/d5e 39 2024-03-22T07:16:00.111 INFO:tasks.workunit.client.1.smithi069.stdout:8/396: dread d1/d1c/d3e/f16 [0,4194304] 0 2024-03-22T07:16:00.112 INFO:tasks.workunit.client.1.smithi069.stdout:8/397: symlink d1/de/d6c/l7b 0 2024-03-22T07:16:00.112 INFO:tasks.workunit.client.1.smithi069.stdout:8/398: mknod d1/de/d20/c7c 0 2024-03-22T07:16:00.114 INFO:tasks.workunit.client.0.smithi063.stdout:8/707: sync 2024-03-22T07:16:00.114 INFO:tasks.workunit.client.0.smithi063.stdout:6/577: sync 2024-03-22T07:16:00.114 INFO:tasks.workunit.client.0.smithi063.stdout:8/708: creat d0/d1b/d28/d7b/d45/fef x:0 0 0 2024-03-22T07:16:00.117 INFO:tasks.workunit.client.0.smithi063.stdout:7/465: dwrite d5/da/d5f/d33/f53 [0,4194304] 0 2024-03-22T07:16:00.117 INFO:tasks.workunit.client.0.smithi063.stdout:7/466: write d5/d16/d1b/d21/f2a [3545613,16986] 0 2024-03-22T07:16:00.118 INFO:tasks.workunit.client.1.smithi069.stdout:7/340: dwrite d9/f46 [0,4194304] 0 2024-03-22T07:16:00.118 INFO:tasks.workunit.client.1.smithi069.stdout:3/325: dwrite d5/d9/f48 [0,4194304] 0 2024-03-22T07:16:00.118 INFO:tasks.workunit.client.1.smithi069.stdout:3/326: dread - d5/d9/d26/d3f/d35/f58 zero size 2024-03-22T07:16:00.118 INFO:tasks.workunit.client.1.smithi069.stdout:8/399: dread d1/de/f30 [0,4194304] 0 2024-03-22T07:16:00.118 INFO:tasks.workunit.client.1.smithi069.stdout:8/400: write d1/d1c/d3e/f36 [1429972,127678] 0 2024-03-22T07:16:00.119 INFO:tasks.workunit.client.1.smithi069.stdout:4/415: dwrite d6/d12/d18/d21/f6a [0,4194304] 0 2024-03-22T07:16:00.119 INFO:tasks.workunit.client.1.smithi069.stdout:1/314: dwrite d1/f35 [0,4194304] 0 2024-03-22T07:16:00.119 INFO:tasks.workunit.client.1.smithi069.stdout:4/416: stat d6/d12/d5d 0 2024-03-22T07:16:00.119 INFO:tasks.workunit.client.1.smithi069.stdout:4/417: chown d6/dc/d1d 1065405974 1 2024-03-22T07:16:00.119 INFO:tasks.workunit.client.0.smithi063.stdout:6/578: mkdir d3/d6/d2a/d38/da5/d32/dbf 0 2024-03-22T07:16:00.119 INFO:tasks.workunit.client.0.smithi063.stdout:6/579: dread - d3/d6/d35/d3a/fbe zero size 2024-03-22T07:16:00.119 INFO:tasks.workunit.client.0.smithi063.stdout:8/709: rename d0/d1b/d28/de8/dd1/de9 to d0/d9e/df0 0 2024-03-22T07:16:00.119 INFO:tasks.workunit.client.1.smithi069.stdout:6/268: dread d4/f46 [0,4194304] 0 2024-03-22T07:16:00.123 INFO:tasks.workunit.client.1.smithi069.stdout:9/296: write d2/dd/d3c/d2e/f3f [2637060,6775] 0 2024-03-22T07:16:00.123 INFO:tasks.workunit.client.1.smithi069.stdout:7/341: chown d9/d76/d57/d44/d53/d74/d6e 38399 1 2024-03-22T07:16:00.123 INFO:tasks.workunit.client.1.smithi069.stdout:7/342: readlink l5 0 2024-03-22T07:16:00.124 INFO:tasks.workunit.client.0.smithi063.stdout:6/580: unlink d3/d6/d2a/d38/da5/ca0 0 2024-03-22T07:16:00.124 INFO:tasks.workunit.client.1.smithi069.stdout:3/327: truncate d5/d9/d26/d3f/d35/f50 7378202 0 2024-03-22T07:16:00.126 INFO:tasks.workunit.client.0.smithi063.stdout:8/710: creat d0/d26/d2d/d8f/ff1 x:0 0 0 2024-03-22T07:16:00.127 INFO:tasks.workunit.client.0.smithi063.stdout:5/516: write d8/d13/d28/d57/f5c [1455670,100643] 0 2024-03-22T07:16:00.127 INFO:tasks.workunit.client.0.smithi063.stdout:6/581: symlink d3/d6/d2a/d38/da5/d32/dbf/lc0 0 2024-03-22T07:16:00.127 INFO:tasks.workunit.client.0.smithi063.stdout:6/582: creat d3/d15/d16/d60/d6d/d4e/d5a/d88/fc1 x:0 0 0 2024-03-22T07:16:00.128 INFO:tasks.workunit.client.1.smithi069.stdout:1/315: mknod d1/d2a/d5e/c66 0 2024-03-22T07:16:00.131 INFO:tasks.workunit.client.0.smithi063.stdout:8/711: unlink d0/d1b/d28/d3e/l80 0 2024-03-22T07:16:00.131 INFO:tasks.workunit.client.0.smithi063.stdout:9/499: dwrite d8/f13 [0,4194304] 0 2024-03-22T07:16:00.131 INFO:tasks.workunit.client.0.smithi063.stdout:7/467: dread d5/da/d5f/d33/f53 [8388608,4194304] 0 2024-03-22T07:16:00.131 INFO:tasks.workunit.client.0.smithi063.stdout:7/468: stat d5/d16/d1b/d21/d62/d73/d1d/d29/l79 0 2024-03-22T07:16:00.131 INFO:tasks.workunit.client.0.smithi063.stdout:7/469: chown d5/d16/f88 22 1 2024-03-22T07:16:00.132 INFO:tasks.workunit.client.1.smithi069.stdout:6/269: symlink d4/dd/l53 0 2024-03-22T07:16:00.132 INFO:tasks.workunit.client.0.smithi063.stdout:5/517: creat d8/d13/d1e/d31/d64/d65/d86/f92 x:0 0 0 2024-03-22T07:16:00.133 INFO:tasks.workunit.client.1.smithi069.stdout:6/270: write d4/f14 [3940172,31783] 0 2024-03-22T07:16:00.133 INFO:tasks.workunit.client.0.smithi063.stdout:6/583: dread d3/f7c [0,4194304] 0 2024-03-22T07:16:00.135 INFO:tasks.workunit.client.0.smithi063.stdout:9/500: truncate d8/d19/d39/f6f 186075 0 2024-03-22T07:16:00.135 INFO:tasks.workunit.client.0.smithi063.stdout:9/501: readlink d8/d19/l49 0 2024-03-22T07:16:00.135 INFO:tasks.workunit.client.0.smithi063.stdout:9/502: write d8/d19/d20/d22/d29/d31/d4b/f4d [978288,84058] 0 2024-03-22T07:16:00.135 INFO:tasks.workunit.client.0.smithi063.stdout:5/518: mknod d8/d13/d28/c93 0 2024-03-22T07:16:00.136 INFO:tasks.workunit.client.0.smithi063.stdout:8/712: rmdir d0/d24/d33 39 2024-03-22T07:16:00.138 INFO:tasks.workunit.client.0.smithi063.stdout:5/519: creat d8/d13/d28/d57/d5f/f94 x:0 0 0 2024-03-22T07:16:00.142 INFO:tasks.workunit.client.0.smithi063.stdout:7/470: dread d5/f7 [4194304,4194304] 0 2024-03-22T07:16:00.142 INFO:tasks.workunit.client.0.smithi063.stdout:7/471: write d5/d16/d1b/d21/d62/d73/d1d/d4f/f95 [444082,104459] 0 2024-03-22T07:16:00.143 INFO:tasks.workunit.client.1.smithi069.stdout:7/343: creat d9/db/d48/d4b/f77 x:0 0 0 2024-03-22T07:16:00.145 INFO:tasks.workunit.client.1.smithi069.stdout:3/328: link d5/d9/d26/d3f/l24 d5/d33/l67 0 2024-03-22T07:16:00.145 INFO:tasks.workunit.client.1.smithi069.stdout:6/271: write d4/dd/d12/f1f [642175,54622] 0 2024-03-22T07:16:00.146 INFO:tasks.workunit.client.1.smithi069.stdout:8/401: getdents d1/d1c/d48 0 2024-03-22T07:16:00.146 INFO:tasks.workunit.client.1.smithi069.stdout:8/402: chown d1/de/d6c/f6d 8503065 1 2024-03-22T07:16:00.146 INFO:tasks.workunit.client.1.smithi069.stdout:6/272: read d4/dd/d12/d2b/f30 [3225,72121] 0 2024-03-22T07:16:00.146 INFO:tasks.workunit.client.1.smithi069.stdout:6/273: creat d4/dd/f54 x:0 0 0 2024-03-22T07:16:00.146 INFO:tasks.workunit.client.1.smithi069.stdout:6/274: stat d4/dd/f25 0 2024-03-22T07:16:00.147 INFO:tasks.workunit.client.1.smithi069.stdout:6/275: write d4/dd/d12/f1d [2708441,86602] 0 2024-03-22T07:16:00.147 INFO:tasks.workunit.client.1.smithi069.stdout:6/276: write d4/f47 [738977,126797] 0 2024-03-22T07:16:00.147 INFO:tasks.workunit.client.1.smithi069.stdout:3/329: read d5/d9/d26/d3f/f22 [8937311,69378] 0 2024-03-22T07:16:00.147 INFO:tasks.workunit.client.1.smithi069.stdout:1/316: creat d1/d13/d36/f67 x:0 0 0 2024-03-22T07:16:00.150 INFO:tasks.workunit.client.1.smithi069.stdout:7/344: symlink d9/d76/d57/d44/d53/d74/d30/d3d/l78 0 2024-03-22T07:16:00.151 INFO:tasks.workunit.client.1.smithi069.stdout:6/277: read d4/f46 [159121,15121] 0 2024-03-22T07:16:00.152 INFO:tasks.workunit.client.1.smithi069.stdout:8/403: mknod d1/d1c/d48/d56/c7d 0 2024-03-22T07:16:00.154 INFO:tasks.workunit.client.0.smithi063.stdout:9/503: dwrite f2 [0,4194304] 0 2024-03-22T07:16:00.154 INFO:tasks.workunit.client.0.smithi063.stdout:9/504: write d8/d19/d20/d22/f4f [408440,10883] 0 2024-03-22T07:16:00.156 INFO:tasks.workunit.client.1.smithi069.stdout:1/317: truncate d1/d25/f38 559143 0 2024-03-22T07:16:00.158 INFO:tasks.workunit.client.1.smithi069.stdout:7/345: rename d9/l12 to d9/d76/l79 0 2024-03-22T07:16:00.159 INFO:tasks.workunit.client.1.smithi069.stdout:1/318: dread d1/f22 [0,4194304] 0 2024-03-22T07:16:00.160 INFO:tasks.workunit.client.1.smithi069.stdout:1/319: creat d1/d13/d36/f68 x:0 0 0 2024-03-22T07:16:00.160 INFO:tasks.workunit.client.1.smithi069.stdout:3/330: dread d5/d9/d23/f2b [0,4194304] 0 2024-03-22T07:16:00.160 INFO:tasks.workunit.client.1.smithi069.stdout:3/331: dread - d5/d9/f5e zero size 2024-03-22T07:16:00.161 INFO:tasks.workunit.client.1.smithi069.stdout:6/278: mknod d4/dd/d12/c55 0 2024-03-22T07:16:00.161 INFO:tasks.workunit.client.1.smithi069.stdout:6/279: truncate d4/dd/d12/f1f 5056111 0 2024-03-22T07:16:00.165 INFO:tasks.workunit.client.1.smithi069.stdout:3/332: truncate d5/d9/d26/d3f/d35/f37 14448 0 2024-03-22T07:16:00.165 INFO:tasks.workunit.client.1.smithi069.stdout:3/333: write d5/d9/d26/d3f/d35/f50 [2886749,99334] 0 2024-03-22T07:16:00.165 INFO:tasks.workunit.client.1.smithi069.stdout:3/334: write d5/d9/d26/d3f/f21 [77150,106810] 0 2024-03-22T07:16:00.165 INFO:tasks.workunit.client.1.smithi069.stdout:0/365: sync 2024-03-22T07:16:00.167 INFO:tasks.workunit.client.1.smithi069.stdout:6/280: mkdir d4/dd/d12/d13/d56 0 2024-03-22T07:16:00.172 INFO:tasks.workunit.client.1.smithi069.stdout:8/404: dwrite d1/de/d20/f3c [0,4194304] 0 2024-03-22T07:16:00.173 INFO:tasks.workunit.client.1.smithi069.stdout:3/335: rename d5/d9/d23/f5b to d5/d9/f68 0 2024-03-22T07:16:00.173 INFO:tasks.workunit.client.0.smithi063.stdout:8/713: link d0/d1b/d28/d3e/db2/ccd d0/d24/d33/dd9/cf2 0 2024-03-22T07:16:00.176 INFO:tasks.workunit.client.0.smithi063.stdout:5/520: rmdir d8/d5e 39 2024-03-22T07:16:00.176 INFO:tasks.workunit.client.0.smithi063.stdout:5/521: creat d8/d13/d1e/d31/d64/d65/f95 x:0 0 0 2024-03-22T07:16:00.177 INFO:tasks.workunit.client.1.smithi069.stdout:6/281: rmdir d4/dd/d12/d13/d56 0 2024-03-22T07:16:00.178 INFO:tasks.workunit.client.0.smithi063.stdout:7/472: rename d5/da/f11 to d5/da/d28/d4b/f9c 0 2024-03-22T07:16:00.180 INFO:tasks.workunit.client.0.smithi063.stdout:9/505: dwrite d8/d19/f23 [0,4194304] 0 2024-03-22T07:16:00.180 INFO:tasks.workunit.client.1.smithi069.stdout:8/405: symlink d1/d1c/d3e/d66/l7e 0 2024-03-22T07:16:00.181 INFO:tasks.workunit.client.1.smithi069.stdout:3/336: mknod d5/d9/d26/d36/d41/c69 0 2024-03-22T07:16:00.182 INFO:tasks.workunit.client.0.smithi063.stdout:8/714: mknod d0/d1b/d28/de8/cf3 0 2024-03-22T07:16:00.184 INFO:tasks.workunit.client.1.smithi069.stdout:6/282: mknod d4/dd/d41/c57 0 2024-03-22T07:16:00.184 INFO:tasks.workunit.client.1.smithi069.stdout:6/283: write d4/dd/d41/f51 [1175666,56262] 0 2024-03-22T07:16:00.185 INFO:tasks.workunit.client.1.smithi069.stdout:8/406: mknod d1/de/d20/c7f 0 2024-03-22T07:16:00.187 INFO:tasks.workunit.client.1.smithi069.stdout:7/346: dwrite d9/d76/d57/d44/d53/f6d [0,4194304] 0 2024-03-22T07:16:00.187 INFO:tasks.workunit.client.1.smithi069.stdout:7/347: fsync d9/d76/d57/d44/d53/d74/d30/f54 0 2024-03-22T07:16:00.187 INFO:tasks.workunit.client.1.smithi069.stdout:3/337: symlink d5/d33/l6a 0 2024-03-22T07:16:00.187 INFO:tasks.workunit.client.0.smithi063.stdout:5/522: chown d8/d13/c2d 246258 1 2024-03-22T07:16:00.189 INFO:tasks.workunit.client.0.smithi063.stdout:7/473: mkdir d5/d16/d43/d9d 0 2024-03-22T07:16:00.190 INFO:tasks.workunit.client.1.smithi069.stdout:0/366: dwrite d3/d7/d15/f20 [0,4194304] 0 2024-03-22T07:16:00.192 INFO:tasks.workunit.client.1.smithi069.stdout:8/407: mknod d1/de/c80 0 2024-03-22T07:16:00.192 INFO:tasks.workunit.client.1.smithi069.stdout:8/408: truncate d1/d1c/d48/f7a 354500 0 2024-03-22T07:16:00.192 INFO:tasks.workunit.client.1.smithi069.stdout:8/409: truncate d1/d1c/d3e/f16 4291897 0 2024-03-22T07:16:00.192 INFO:tasks.workunit.client.1.smithi069.stdout:8/410: truncate d1/d1c/d3e/f46 1066711 0 2024-03-22T07:16:00.194 INFO:tasks.workunit.client.1.smithi069.stdout:7/348: mkdir d9/d76/d57/d75/d7a 0 2024-03-22T07:16:00.195 INFO:tasks.workunit.client.1.smithi069.stdout:3/338: getdents d5/d9/d26/d3f/d54 0 2024-03-22T07:16:00.195 INFO:tasks.workunit.client.1.smithi069.stdout:3/339: read d5/d9/db/f3d [371405,126883] 0 2024-03-22T07:16:00.195 INFO:tasks.workunit.client.1.smithi069.stdout:3/340: creat d5/d9/d26/d3f/d35/f6b x:0 0 0 2024-03-22T07:16:00.195 INFO:tasks.workunit.client.1.smithi069.stdout:3/341: creat d5/d9/d26/f6c x:0 0 0 2024-03-22T07:16:00.195 INFO:tasks.workunit.client.1.smithi069.stdout:3/342: dread - d5/d9/d26/f5a zero size 2024-03-22T07:16:00.196 INFO:tasks.workunit.client.1.smithi069.stdout:8/411: mknod d1/de/d6c/d4e/d5b/c81 0 2024-03-22T07:16:00.196 INFO:tasks.workunit.client.1.smithi069.stdout:8/412: write d1/d1c/d48/d5e/f79 [582028,24242] 0 2024-03-22T07:16:00.197 INFO:tasks.workunit.client.1.smithi069.stdout:7/349: creat d9/d76/d57/d44/d53/d74/d30/d3d/f7b x:0 0 0 2024-03-22T07:16:00.199 INFO:tasks.workunit.client.0.smithi063.stdout:8/715: mkdir d0/d26/d2d/d59/dba/df4 0 2024-03-22T07:16:00.199 INFO:tasks.workunit.client.1.smithi069.stdout:3/343: creat d5/d9/d23/f6d x:0 0 0 2024-03-22T07:16:00.201 INFO:tasks.workunit.client.1.smithi069.stdout:8/413: mkdir d1/de/d6c/d4e/d5b/d82 0 2024-03-22T07:16:00.201 INFO:tasks.workunit.client.0.smithi063.stdout:7/474: creat d5/da/d5f/d33/d57/f9e x:0 0 0 2024-03-22T07:16:00.201 INFO:tasks.workunit.client.0.smithi063.stdout:7/475: chown d5/d16/d1b/d21/d62/d73/d1d/l69 44024 1 2024-03-22T07:16:00.202 INFO:tasks.workunit.client.1.smithi069.stdout:7/350: creat d9/d76/d57/d44/d53/d2e/f7c x:0 0 0 2024-03-22T07:16:00.203 INFO:tasks.workunit.client.0.smithi063.stdout:8/716: rmdir d0/d24/d33 39 2024-03-22T07:16:00.203 INFO:tasks.workunit.client.0.smithi063.stdout:8/717: truncate d0/d24/d33/fee 948339 0 2024-03-22T07:16:00.203 INFO:tasks.workunit.client.0.smithi063.stdout:8/718: fdatasync d0/d1b/d28/f43 0 2024-03-22T07:16:00.204 INFO:tasks.workunit.client.0.smithi063.stdout:7/476: truncate d5/d16/d1b/d21/f2a 3841306 0 2024-03-22T07:16:00.205 INFO:tasks.workunit.client.0.smithi063.stdout:7/477: stat d5/l15 0 2024-03-22T07:16:00.208 INFO:tasks.workunit.client.1.smithi069.stdout:6/284: truncate d4/dd/d12/f1d 2194962 0 2024-03-22T07:16:00.209 INFO:tasks.workunit.client.1.smithi069.stdout:7/351: read d9/d76/d57/d44/d53/f38 [95474,26900] 0 2024-03-22T07:16:00.210 INFO:tasks.workunit.client.0.smithi063.stdout:8/719: rename d0/d1f to d0/d1b/d4f/dde/df5 0 2024-03-22T07:16:00.210 INFO:tasks.workunit.client.0.smithi063.stdout:8/720: getdents d0/d1b/d28/de8/dc6 0 2024-03-22T07:16:00.213 INFO:tasks.workunit.client.0.smithi063.stdout:9/506: dwrite d8/d17/f26 [4194304,4194304] 0 2024-03-22T07:16:00.215 INFO:tasks.workunit.client.0.smithi063.stdout:8/721: link d0/d1b/d4f/dde/df5/l76 d0/d24/lf6 0 2024-03-22T07:16:00.215 INFO:tasks.workunit.client.0.smithi063.stdout:8/722: fdatasync d0/d24/d33/d4c/fd4 0 2024-03-22T07:16:00.215 INFO:tasks.workunit.client.0.smithi063.stdout:8/723: read d0/d26/d2d/f4d [118879,44899] 0 2024-03-22T07:16:00.217 INFO:tasks.workunit.client.1.smithi069.stdout:7/352: truncate f4 3613090 0 2024-03-22T07:16:00.218 INFO:tasks.workunit.client.1.smithi069.stdout:6/285: getdents d4/dd/d41 0 2024-03-22T07:16:00.218 INFO:tasks.workunit.client.1.smithi069.stdout:6/286: chown d4/d7/da/f1b 30288 1 2024-03-22T07:16:00.219 INFO:tasks.workunit.client.0.smithi063.stdout:9/507: symlink d8/d17/d1e/l99 0 2024-03-22T07:16:00.219 INFO:tasks.workunit.client.1.smithi069.stdout:6/287: mkdir d4/dd/d12/d44/d58 0 2024-03-22T07:16:00.221 INFO:tasks.workunit.client.1.smithi069.stdout:6/288: creat d4/dd/d12/d13/f59 x:0 0 0 2024-03-22T07:16:00.222 INFO:tasks.workunit.client.0.smithi063.stdout:8/724: creat d0/d26/d2d/d59/dba/dc9/ded/ff7 x:0 0 0 2024-03-22T07:16:00.222 INFO:tasks.workunit.client.0.smithi063.stdout:9/508: creat d8/f9a x:0 0 0 2024-03-22T07:16:00.222 INFO:tasks.workunit.client.1.smithi069.stdout:7/353: dread d9/d76/d57/d44/d53/f38 [0,4194304] 0 2024-03-22T07:16:00.222 INFO:tasks.workunit.client.1.smithi069.stdout:6/289: creat d4/d7/da/f5a x:0 0 0 2024-03-22T07:16:00.227 INFO:tasks.workunit.client.0.smithi063.stdout:0/752: sync 2024-03-22T07:16:00.227 INFO:tasks.workunit.client.0.smithi063.stdout:2/569: sync 2024-03-22T07:16:00.227 INFO:tasks.workunit.client.0.smithi063.stdout:0/753: read da/de/d60/d9b/f9f [2976626,114904] 0 2024-03-22T07:16:00.228 INFO:tasks.workunit.client.0.smithi063.stdout:2/570: mknod d0/de/d74/cb5 0 2024-03-22T07:16:00.231 INFO:tasks.workunit.client.0.smithi063.stdout:0/754: rename da/de/f62 to da/de/d17/d25/d95/ff0 0 2024-03-22T07:16:00.234 INFO:tasks.workunit.client.1.smithi069.stdout:3/344: dwrite d5/d9/d26/f6c [0,4194304] 0 2024-03-22T07:16:00.237 INFO:tasks.workunit.client.1.smithi069.stdout:0/367: dwrite d3/d7/d15/d1f/f41 [0,4194304] 0 2024-03-22T07:16:00.238 INFO:tasks.workunit.client.1.smithi069.stdout:6/290: dread d4/f6 [0,4194304] 0 2024-03-22T07:16:00.239 INFO:tasks.workunit.client.1.smithi069.stdout:3/345: mkdir d5/d9/d26/d3f/d35/d6e 0 2024-03-22T07:16:00.241 INFO:tasks.workunit.client.1.smithi069.stdout:6/291: mkdir d4/d7/d5b 0 2024-03-22T07:16:00.241 INFO:tasks.workunit.client.1.smithi069.stdout:6/292: readlink d4/d7/l2d 0 2024-03-22T07:16:00.242 INFO:tasks.workunit.client.1.smithi069.stdout:3/346: creat d5/d9/d61/f6f x:0 0 0 2024-03-22T07:16:00.242 INFO:tasks.workunit.client.1.smithi069.stdout:3/347: rename d5 to d5/d9/d26/d3f/d35/d70 22 2024-03-22T07:16:00.243 INFO:tasks.workunit.client.1.smithi069.stdout:8/414: dwrite d1/d1c/f38 [0,4194304] 0 2024-03-22T07:16:00.243 INFO:tasks.workunit.client.1.smithi069.stdout:8/415: readlink d1/d1c/d48/l5c 0 2024-03-22T07:16:00.245 INFO:tasks.workunit.client.1.smithi069.stdout:3/348: unlink d5/f6 0 2024-03-22T07:16:00.245 INFO:tasks.workunit.client.1.smithi069.stdout:3/349: read - d5/d9/f5e zero size 2024-03-22T07:16:00.245 INFO:tasks.workunit.client.1.smithi069.stdout:3/350: write d5/f4a [88103,52487] 0 2024-03-22T07:16:00.245 INFO:tasks.workunit.client.1.smithi069.stdout:3/351: fsync d5/d33/f39 0 2024-03-22T07:16:00.247 INFO:tasks.workunit.client.1.smithi069.stdout:8/416: rename d1/d1c/d48/l62 to d1/d1c/l83 0 2024-03-22T07:16:00.250 INFO:tasks.workunit.client.1.smithi069.stdout:3/352: rmdir d5/d9/d26/d3f/d35/d6e 0 2024-03-22T07:16:00.251 INFO:tasks.workunit.client.1.smithi069.stdout:0/368: write d3/da/f52 [2946253,122215] 0 2024-03-22T07:16:00.252 INFO:tasks.workunit.client.1.smithi069.stdout:3/353: symlink d5/l71 0 2024-03-22T07:16:00.252 INFO:tasks.workunit.client.1.smithi069.stdout:3/354: creat d5/d9/d26/d36/d41/f72 x:0 0 0 2024-03-22T07:16:00.252 INFO:tasks.workunit.client.1.smithi069.stdout:3/355: write d5/d9/db/f3d [4009495,118844] 0 2024-03-22T07:16:00.253 INFO:tasks.workunit.client.1.smithi069.stdout:0/369: rename d3/d7/d15/f1b to d3/d7/d15/d29/d5a/f75 0 2024-03-22T07:16:00.254 INFO:tasks.workunit.client.0.smithi063.stdout:2/571: write d0/de/d12/d20/f88 [54386,23528] 0 2024-03-22T07:16:00.254 INFO:tasks.workunit.client.0.smithi063.stdout:7/478: dwrite d5/f9 [4194304,4194304] 0 2024-03-22T07:16:00.254 INFO:tasks.workunit.client.0.smithi063.stdout:2/572: write d0/de/d35/d36/f3a [4439733,40787] 0 2024-03-22T07:16:00.255 INFO:tasks.workunit.client.0.smithi063.stdout:7/479: creat d5/d16/d1b/d21/d85/f9f x:0 0 0 2024-03-22T07:16:00.259 INFO:tasks.workunit.client.1.smithi069.stdout:8/417: dread d1/d1c/d3e/f16 [0,4194304] 0 2024-03-22T07:16:00.262 INFO:tasks.workunit.client.1.smithi069.stdout:7/354: dwrite d9/db/f19 [4194304,4194304] 0 2024-03-22T07:16:00.263 INFO:tasks.workunit.client.1.smithi069.stdout:8/418: rmdir d1/de/d6c/d4e/d5b 39 2024-03-22T07:16:00.263 INFO:tasks.workunit.client.1.smithi069.stdout:8/419: readlink d1/d1c/d48/d5e/l6b 0 2024-03-22T07:16:00.264 INFO:tasks.workunit.client.1.smithi069.stdout:7/355: write d9/db/f24 [290358,86696] 0 2024-03-22T07:16:00.264 INFO:tasks.workunit.client.0.smithi063.stdout:8/725: dwrite d0/d24/fb9 [0,4194304] 0 2024-03-22T07:16:00.266 INFO:tasks.workunit.client.0.smithi063.stdout:8/726: creat d0/d1b/d4f/dde/ff8 x:0 0 0 2024-03-22T07:16:00.269 INFO:tasks.workunit.client.1.smithi069.stdout:0/370: dread d3/d7/d17/f6b [0,4194304] 0 2024-03-22T07:16:00.271 INFO:tasks.workunit.client.1.smithi069.stdout:7/356: symlink d9/d76/d57/d75/d7a/l7d 0 2024-03-22T07:16:00.273 INFO:tasks.workunit.client.0.smithi063.stdout:0/755: dwrite da/fc2 [0,4194304] 0 2024-03-22T07:16:00.273 INFO:tasks.workunit.client.0.smithi063.stdout:0/756: fdatasync da/de/d17/d25/d3c/d43/d80/fd8 0 2024-03-22T07:16:00.273 INFO:tasks.workunit.client.0.smithi063.stdout:9/509: dwrite d8/d15/f2a [0,4194304] 0 2024-03-22T07:16:00.274 INFO:tasks.workunit.client.1.smithi069.stdout:6/293: rmdir d4/dd/d12/d44 39 2024-03-22T07:16:00.276 INFO:tasks.workunit.client.1.smithi069.stdout:6/294: stat d4/dd/l45 0 2024-03-22T07:16:00.277 INFO:tasks.workunit.client.1.smithi069.stdout:6/295: symlink d4/d7/d5b/l5c 0 2024-03-22T07:16:00.278 INFO:tasks.workunit.client.0.smithi063.stdout:0/757: mkdir da/de/d17/d25/d3c/d53/d89/df1 0 2024-03-22T07:16:00.280 INFO:tasks.workunit.client.0.smithi063.stdout:9/510: truncate d8/d19/d20/d22/d29/d31/f86 2370329 0 2024-03-22T07:16:00.281 INFO:tasks.workunit.client.0.smithi063.stdout:0/758: mknod da/de/d60/d9b/cf2 0 2024-03-22T07:16:00.282 INFO:tasks.workunit.client.0.smithi063.stdout:9/511: mkdir d8/d19/d20/d22/d9b 0 2024-03-22T07:16:00.283 INFO:tasks.workunit.client.1.smithi069.stdout:7/357: write d9/f46 [1430540,72861] 0 2024-03-22T07:16:00.283 INFO:tasks.workunit.client.0.smithi063.stdout:0/759: truncate da/de/d17/d25/d3c/d43/d80/fda 387622 0 2024-03-22T07:16:00.285 INFO:tasks.workunit.client.0.smithi063.stdout:9/512: symlink d8/d17/d1e/l9c 0 2024-03-22T07:16:00.286 INFO:tasks.workunit.client.0.smithi063.stdout:9/513: creat d8/f9d x:0 0 0 2024-03-22T07:16:00.286 INFO:tasks.workunit.client.1.smithi069.stdout:7/358: unlink d9/db/d1a/f3c 0 2024-03-22T07:16:00.286 INFO:tasks.workunit.client.1.smithi069.stdout:7/359: fdatasync d9/db/d1a/f56 0 2024-03-22T07:16:00.287 INFO:tasks.workunit.client.1.smithi069.stdout:7/360: symlink d9/d76/d57/d44/d53/d74/d30/l7e 0 2024-03-22T07:16:00.287 INFO:tasks.workunit.client.1.smithi069.stdout:7/361: write d9/d76/d57/d44/f47 [565921,47083] 0 2024-03-22T07:16:00.291 INFO:tasks.workunit.client.1.smithi069.stdout:6/296: dread d4/dd/d12/d13/f23 [0,4194304] 0 2024-03-22T07:16:00.291 INFO:tasks.workunit.client.1.smithi069.stdout:6/297: readlink d4/d7/da/l38 0 2024-03-22T07:16:00.292 INFO:tasks.workunit.client.1.smithi069.stdout:7/362: dread d9/db/d1a/f21 [0,4194304] 0 2024-03-22T07:16:00.293 INFO:tasks.workunit.client.1.smithi069.stdout:6/298: symlink d4/dd/d12/d2b/l5d 0 2024-03-22T07:16:00.303 INFO:tasks.workunit.client.0.smithi063.stdout:9/514: write d8/d19/d2b/f5f [2561267,31026] 0 2024-03-22T07:16:00.304 INFO:tasks.workunit.client.0.smithi063.stdout:9/515: creat d8/d19/d20/d22/d9b/f9e x:0 0 0 2024-03-22T07:16:00.307 INFO:tasks.workunit.client.1.smithi069.stdout:3/356: dwrite d5/d9/f13 [0,4194304] 0 2024-03-22T07:16:00.307 INFO:tasks.workunit.client.1.smithi069.stdout:3/357: write d5/d9/f5e [993184,102068] 0 2024-03-22T07:16:00.307 INFO:tasks.workunit.client.1.smithi069.stdout:3/358: creat d5/d9/d23/f73 x:0 0 0 2024-03-22T07:16:00.308 INFO:tasks.workunit.client.0.smithi063.stdout:7/480: dwrite d5/d16/d1b/d21/d62/d73/d1d/d29/d4a/f7e [0,4194304] 0 2024-03-22T07:16:00.308 INFO:tasks.workunit.client.0.smithi063.stdout:7/481: dread - d5/d16/d1b/d21/d85/f9f zero size 2024-03-22T07:16:00.308 INFO:tasks.workunit.client.0.smithi063.stdout:0/760: dwrite da/de/d17/d25/d3c/d5f/d97/fa4 [0,4194304] 0 2024-03-22T07:16:00.308 INFO:tasks.workunit.client.0.smithi063.stdout:0/761: creat da/de/d60/d9b/ff3 x:0 0 0 2024-03-22T07:16:00.308 INFO:tasks.workunit.client.0.smithi063.stdout:0/762: write da/de/d60/d9b/fa5 [382645,75644] 0 2024-03-22T07:16:00.308 INFO:tasks.workunit.client.0.smithi063.stdout:2/573: dwrite d0/de/d74/f7e [0,4194304] 0 2024-03-22T07:16:00.309 INFO:tasks.workunit.client.1.smithi069.stdout:0/371: dwrite d3/d7/d15/d1f/f41 [4194304,4194304] 0 2024-03-22T07:16:00.312 INFO:tasks.workunit.client.1.smithi069.stdout:3/359: rename d5/d33/l6a to d5/d33/l74 0 2024-03-22T07:16:00.312 INFO:tasks.workunit.client.1.smithi069.stdout:3/360: write d5/d9/d26/d3f/d35/f37 [443889,46880] 0 2024-03-22T07:16:00.312 INFO:tasks.workunit.client.1.smithi069.stdout:3/361: dread - d5/f4e zero size 2024-03-22T07:16:00.312 INFO:tasks.workunit.client.1.smithi069.stdout:3/362: creat d5/d9/d26/d36/d41/f75 x:0 0 0 2024-03-22T07:16:00.312 INFO:tasks.workunit.client.1.smithi069.stdout:3/363: chown d5/d9/db/d3b/l66 632 1 2024-03-22T07:16:00.313 INFO:tasks.workunit.client.1.smithi069.stdout:8/420: rmdir d1/d1c 39 2024-03-22T07:16:00.313 INFO:tasks.workunit.client.0.smithi063.stdout:9/516: mknod d8/d19/d20/d22/d29/c9f 0 2024-03-22T07:16:00.315 INFO:tasks.workunit.client.1.smithi069.stdout:7/363: dread d9/d76/d57/d44/f47 [0,4194304] 0 2024-03-22T07:16:00.319 INFO:tasks.workunit.client.1.smithi069.stdout:6/299: dwrite d4/f47 [0,4194304] 0 2024-03-22T07:16:00.319 INFO:tasks.workunit.client.1.smithi069.stdout:3/364: dread d5/d9/d26/d3f/d35/f50 [4194304,4194304] 0 2024-03-22T07:16:00.319 INFO:tasks.workunit.client.1.smithi069.stdout:3/365: read - d5/d9/d26/f5a zero size 2024-03-22T07:16:00.319 INFO:tasks.workunit.client.1.smithi069.stdout:6/300: creat d4/dd/d12/d13/f5e x:0 0 0 2024-03-22T07:16:00.320 INFO:tasks.workunit.client.0.smithi063.stdout:0/763: mknod da/d49/cf4 0 2024-03-22T07:16:00.323 INFO:tasks.workunit.client.0.smithi063.stdout:7/482: write d5/d38/f81 [1882262,89072] 0 2024-03-22T07:16:00.326 INFO:tasks.workunit.client.1.smithi069.stdout:0/372: mknod d3/d7/d15/d29/d5a/d5d/c76 0 2024-03-22T07:16:00.327 INFO:tasks.workunit.client.0.smithi063.stdout:0/764: mkdir da/de/d17/d25/d3c/d64/df5 0 2024-03-22T07:16:00.329 INFO:tasks.workunit.client.0.smithi063.stdout:7/483: mknod d5/da/d28/d4b/ca0 0 2024-03-22T07:16:00.330 INFO:tasks.workunit.client.1.smithi069.stdout:6/301: dread d4/f6 [0,4194304] 0 2024-03-22T07:16:00.331 INFO:tasks.workunit.client.0.smithi063.stdout:0/765: mknod da/de/d17/d4e/cf6 0 2024-03-22T07:16:00.331 INFO:tasks.workunit.client.0.smithi063.stdout:9/517: write d8/d17/f26 [3214707,107784] 0 2024-03-22T07:16:00.332 INFO:tasks.workunit.client.1.smithi069.stdout:3/366: truncate d5/d9/d23/f2b 636438 0 2024-03-22T07:16:00.332 INFO:tasks.workunit.client.1.smithi069.stdout:3/367: creat d5/d9/f76 x:0 0 0 2024-03-22T07:16:00.332 INFO:tasks.workunit.client.0.smithi063.stdout:7/484: symlink d5/da/d28/d4b/la1 0 2024-03-22T07:16:00.333 INFO:tasks.workunit.client.1.smithi069.stdout:7/364: rmdir d9/d42 39 2024-03-22T07:16:00.336 INFO:tasks.workunit.client.0.smithi063.stdout:0/766: truncate da/ff 3981558 0 2024-03-22T07:16:00.336 INFO:tasks.workunit.client.0.smithi063.stdout:0/767: truncate da/f2b 755642 0 2024-03-22T07:16:00.336 INFO:tasks.workunit.client.0.smithi063.stdout:0/768: write da/de/d17/d25/f68 [2408497,81450] 0 2024-03-22T07:16:00.337 INFO:tasks.workunit.client.1.smithi069.stdout:7/365: symlink d9/db/d48/l7f 0 2024-03-22T07:16:00.337 INFO:tasks.workunit.client.1.smithi069.stdout:7/366: truncate d9/d42/f64 505079 0 2024-03-22T07:16:00.339 INFO:tasks.workunit.client.0.smithi063.stdout:0/769: getdents da/de/d17/d25/d3c/d43/d76 0 2024-03-22T07:16:00.339 INFO:tasks.workunit.client.0.smithi063.stdout:0/770: write da/de/d17/d25/d3c/d43/d76/d92/fbf [323196,61982] 0 2024-03-22T07:16:00.340 INFO:tasks.workunit.client.1.smithi069.stdout:3/368: getdents d5 0 2024-03-22T07:16:00.343 INFO:tasks.workunit.client.1.smithi069.stdout:0/373: read d3/d7/f49 [1779813,33552] 0 2024-03-22T07:16:00.344 INFO:tasks.workunit.client.1.smithi069.stdout:6/302: dread d4/dd/d12/f1d [0,4194304] 0 2024-03-22T07:16:00.344 INFO:tasks.workunit.client.1.smithi069.stdout:6/303: write d4/dd/f42 [628559,4646] 0 2024-03-22T07:16:00.344 INFO:tasks.workunit.client.1.smithi069.stdout:6/304: write d4/dd/f42 [878346,7928] 0 2024-03-22T07:16:00.344 INFO:tasks.workunit.client.1.smithi069.stdout:6/305: stat d4/dd/d12/d2b/f36 0 2024-03-22T07:16:00.344 INFO:tasks.workunit.client.1.smithi069.stdout:8/421: dwrite d1/de/d6c/d4e/d5b/f76 [4194304,4194304] 0 2024-03-22T07:16:00.344 INFO:tasks.workunit.client.1.smithi069.stdout:7/367: read d9/db/d1a/f39 [254964,105042] 0 2024-03-22T07:16:00.344 INFO:tasks.workunit.client.1.smithi069.stdout:7/368: chown d9/d76/d57/d44/d53/d2e/f6a 4218 1 2024-03-22T07:16:00.344 INFO:tasks.workunit.client.1.smithi069.stdout:0/374: creat d3/d7/d5e/f77 x:0 0 0 2024-03-22T07:16:00.346 INFO:tasks.workunit.client.1.smithi069.stdout:8/422: truncate d1/d1c/d3e/d66/f6e 3080931 0 2024-03-22T07:16:00.347 INFO:tasks.workunit.client.1.smithi069.stdout:7/369: creat d9/d76/d57/d75/d7a/f80 x:0 0 0 2024-03-22T07:16:00.348 INFO:tasks.workunit.client.1.smithi069.stdout:6/306: dread d4/f14 [0,4194304] 0 2024-03-22T07:16:00.348 INFO:tasks.workunit.client.1.smithi069.stdout:7/370: write d9/db/d48/f5b [277767,62224] 0 2024-03-22T07:16:00.348 INFO:tasks.workunit.client.1.smithi069.stdout:7/371: write f8 [2157667,122998] 0 2024-03-22T07:16:00.349 INFO:tasks.workunit.client.1.smithi069.stdout:0/375: symlink d3/d7/d15/d1f/l78 0 2024-03-22T07:16:00.349 INFO:tasks.workunit.client.1.smithi069.stdout:0/376: fdatasync d3/d7/f5c 0 2024-03-22T07:16:00.350 INFO:tasks.workunit.client.1.smithi069.stdout:8/423: symlink d1/d1c/d3e/d66/l84 0 2024-03-22T07:16:00.351 INFO:tasks.workunit.client.1.smithi069.stdout:6/307: mkdir d4/dd/d12/d2b/d5f 0 2024-03-22T07:16:00.353 INFO:tasks.workunit.client.0.smithi063.stdout:3/646: sync 2024-03-22T07:16:00.353 INFO:tasks.workunit.client.0.smithi063.stdout:1/707: sync 2024-03-22T07:16:00.353 INFO:tasks.workunit.client.0.smithi063.stdout:1/708: chown d4/d14/d57/f80 73 1 2024-03-22T07:16:00.353 INFO:tasks.workunit.client.0.smithi063.stdout:0/771: dread da/de/d60/faf [0,4194304] 0 2024-03-22T07:16:00.354 INFO:tasks.workunit.client.1.smithi069.stdout:8/424: mknod d1/d1c/d48/d56/c85 0 2024-03-22T07:16:00.357 INFO:tasks.workunit.client.1.smithi069.stdout:6/308: getdents d4 0 2024-03-22T07:16:00.358 INFO:tasks.workunit.client.1.smithi069.stdout:2/261: sync 2024-03-22T07:16:00.358 INFO:tasks.workunit.client.1.smithi069.stdout:4/418: sync 2024-03-22T07:16:00.358 INFO:tasks.workunit.client.1.smithi069.stdout:1/320: sync 2024-03-22T07:16:00.359 INFO:tasks.workunit.client.0.smithi063.stdout:9/518: dwrite d8/d19/d41/f4a [4194304,4194304] 0 2024-03-22T07:16:00.359 INFO:tasks.workunit.client.1.smithi069.stdout:5/249: sync 2024-03-22T07:16:00.359 INFO:tasks.workunit.client.1.smithi069.stdout:9/297: sync 2024-03-22T07:16:00.360 INFO:tasks.workunit.client.1.smithi069.stdout:6/309: rename d4/d7/f50 to d4/f60 0 2024-03-22T07:16:00.362 INFO:tasks.workunit.client.0.smithi063.stdout:3/647: mknod da/d63/d65/d92/cd7 0 2024-03-22T07:16:00.362 INFO:tasks.workunit.client.1.smithi069.stdout:1/321: mknod d1/d2a/d50/d4c/d63/c69 0 2024-03-22T07:16:00.362 INFO:tasks.workunit.client.1.smithi069.stdout:1/322: truncate d1/f3b 661202 0 2024-03-22T07:16:00.363 INFO:tasks.workunit.client.1.smithi069.stdout:5/250: mknod d1/d7/d8/d43/c54 0 2024-03-22T07:16:00.363 INFO:tasks.workunit.client.1.smithi069.stdout:5/251: fdatasync d1/f4f 0 2024-03-22T07:16:00.363 INFO:tasks.workunit.client.1.smithi069.stdout:5/252: write d1/d7/d8/f30 [1281865,15800] 0 2024-03-22T07:16:00.364 INFO:tasks.workunit.client.0.smithi063.stdout:0/772: mknod da/de/d17/d25/d99/de5/cf7 0 2024-03-22T07:16:00.366 INFO:tasks.workunit.client.1.smithi069.stdout:6/310: chown d4/dd/d12/d44 22488 1 2024-03-22T07:16:00.367 INFO:tasks.workunit.client.1.smithi069.stdout:1/323: mknod d1/d2a/d50/c6a 0 2024-03-22T07:16:00.368 INFO:tasks.workunit.client.1.smithi069.stdout:1/324: write d1/d25/d41/f19 [5890851,110076] 0 2024-03-22T07:16:00.368 INFO:tasks.workunit.client.1.smithi069.stdout:1/325: write d1/d13/f27 [5599393,81897] 0 2024-03-22T07:16:00.368 INFO:tasks.workunit.client.1.smithi069.stdout:1/326: truncate d1/f42 681878 0 2024-03-22T07:16:00.368 INFO:tasks.workunit.client.1.smithi069.stdout:1/327: dread - d1/d2a/d50/d4c/d55/f56 zero size 2024-03-22T07:16:00.368 INFO:tasks.workunit.client.1.smithi069.stdout:1/328: creat d1/d13/d36/f6b x:0 0 0 2024-03-22T07:16:00.368 INFO:tasks.workunit.client.1.smithi069.stdout:1/329: readlink d1/d13/l1f 0 2024-03-22T07:16:00.369 INFO:tasks.workunit.client.0.smithi063.stdout:3/648: mkdir da/d16/d26/d69/dae/dd8 0 2024-03-22T07:16:00.369 INFO:tasks.workunit.client.0.smithi063.stdout:3/649: write da/d63/d65/fac [127712,80734] 0 2024-03-22T07:16:00.370 INFO:tasks.workunit.client.0.smithi063.stdout:0/773: creat da/de/d17/d25/d3c/d64/df5/ff8 x:0 0 0 2024-03-22T07:16:00.371 INFO:tasks.workunit.client.0.smithi063.stdout:9/519: getdents d8/d19/d20/d22/d29/d31/d53/d6a 0 2024-03-22T07:16:00.371 INFO:tasks.workunit.client.1.smithi069.stdout:5/253: mkdir d1/d7/d8/d43/d55 0 2024-03-22T07:16:00.372 INFO:tasks.workunit.client.1.smithi069.stdout:3/369: dwrite d5/f18 [4194304,4194304] 0 2024-03-22T07:16:00.372 INFO:tasks.workunit.client.1.smithi069.stdout:3/370: write d5/d9/f76 [504405,20801] 0 2024-03-22T07:16:00.373 INFO:tasks.workunit.client.0.smithi063.stdout:3/650: creat da/d63/d65/d7b/fd9 x:0 0 0 2024-03-22T07:16:00.373 INFO:tasks.workunit.client.0.smithi063.stdout:3/651: chown da/d16/d3f/f79 579156 1 2024-03-22T07:16:00.373 INFO:tasks.workunit.client.0.smithi063.stdout:3/652: write da/d63/d65/fac [571302,21487] 0 2024-03-22T07:16:00.374 INFO:tasks.workunit.client.0.smithi063.stdout:0/774: mknod da/de/d17/d25/d3c/d53/cf9 0 2024-03-22T07:16:00.375 INFO:tasks.workunit.client.0.smithi063.stdout:9/520: link d8/d19/d20/d22/d56/f82 d8/d19/d20/fa0 0 2024-03-22T07:16:00.376 INFO:tasks.workunit.client.0.smithi063.stdout:0/775: dread da/f82 [0,4194304] 0 2024-03-22T07:16:00.376 INFO:tasks.workunit.client.0.smithi063.stdout:0/776: creat da/de/d60/ffa x:0 0 0 2024-03-22T07:16:00.378 INFO:tasks.workunit.client.1.smithi069.stdout:9/298: dread d2/f27 [0,4194304] 0 2024-03-22T07:16:00.378 INFO:tasks.workunit.client.1.smithi069.stdout:9/299: readlink d2/d21/l47 0 2024-03-22T07:16:00.378 INFO:tasks.workunit.client.1.smithi069.stdout:2/262: rename d18/d4d/d3e/d50 to d18/d4d/d27/d33/d60 0 2024-03-22T07:16:00.378 INFO:tasks.workunit.client.1.smithi069.stdout:2/263: truncate d18/d4d/d27/d33/d60/f5a 122240 0 2024-03-22T07:16:00.378 INFO:tasks.workunit.client.1.smithi069.stdout:2/264: creat d18/d4d/d3e/d40/f61 x:0 0 0 2024-03-22T07:16:00.378 INFO:tasks.workunit.client.1.smithi069.stdout:2/265: chown d18/d4d/c21 1315110 1 2024-03-22T07:16:00.380 INFO:tasks.workunit.client.1.smithi069.stdout:9/300: dread d2/dd/d3c/f3e [0,4194304] 0 2024-03-22T07:16:00.380 INFO:tasks.workunit.client.1.smithi069.stdout:0/377: dread d3/d7/d15/d1f/f41 [0,4194304] 0 2024-03-22T07:16:00.382 INFO:tasks.workunit.client.1.smithi069.stdout:6/311: rmdir d4/dd/d12/d2b 39 2024-03-22T07:16:00.383 INFO:tasks.workunit.client.0.smithi063.stdout:3/653: mknod da/d63/d65/d7b/cda 0 2024-03-22T07:16:00.387 INFO:tasks.workunit.client.0.smithi063.stdout:9/521: rename d8/d19/d20/d22/d56/f75 to d8/d15/fa1 0 2024-03-22T07:16:00.389 INFO:tasks.workunit.client.1.smithi069.stdout:2/266: link d18/d4d/f2d d18/d4d/d27/d56/f62 0 2024-03-22T07:16:00.389 INFO:tasks.workunit.client.1.smithi069.stdout:4/419: rename d6/dc/d1d/c61 to d6/d16/c84 0 2024-03-22T07:16:00.389 INFO:tasks.workunit.client.1.smithi069.stdout:4/420: creat d6/dc/d1d/d31/d1f/d58/f85 x:0 0 0 2024-03-22T07:16:00.389 INFO:tasks.workunit.client.1.smithi069.stdout:0/378: rename d3/d7/d15/d1f to d3/d7/d15/d1f/d79 22 2024-03-22T07:16:00.390 INFO:tasks.workunit.client.1.smithi069.stdout:0/379: chown d3/d7/d15 635 1 2024-03-22T07:16:00.390 INFO:tasks.workunit.client.0.smithi063.stdout:3/654: stat da/d63/d65/f5e 0 2024-03-22T07:16:00.392 INFO:tasks.workunit.client.1.smithi069.stdout:2/267: mkdir d18/d4d/d27/d33/d60/d63 0 2024-03-22T07:16:00.392 INFO:tasks.workunit.client.1.smithi069.stdout:2/268: truncate d18/d4d/d25/f36 254896 0 2024-03-22T07:16:00.392 INFO:tasks.workunit.client.1.smithi069.stdout:2/269: read d18/d4d/d27/d33/d60/f5a [76724,76421] 0 2024-03-22T07:16:00.392 INFO:tasks.workunit.client.1.smithi069.stdout:2/270: chown d18/d4d/d25/f47 1 1 2024-03-22T07:16:00.393 INFO:tasks.workunit.client.0.smithi063.stdout:0/777: rename da/de/d17/d25/d3c/d43/d76/f9d to da/de/d17/d25/d3c/d53/d89/ffb 0 2024-03-22T07:16:00.394 INFO:tasks.workunit.client.1.smithi069.stdout:4/421: creat d6/dc/d1d/d31/f86 x:0 0 0 2024-03-22T07:16:00.395 INFO:tasks.workunit.client.0.smithi063.stdout:3/655: creat da/d16/d89/da9/dd3/fdb x:0 0 0 2024-03-22T07:16:00.395 INFO:tasks.workunit.client.0.smithi063.stdout:3/656: write da/d63/d65/d92/fb0 [1005605,26042] 0 2024-03-22T07:16:00.396 INFO:tasks.workunit.client.0.smithi063.stdout:0/778: stat da/de/d39/c3b 0 2024-03-22T07:16:00.397 INFO:tasks.workunit.client.1.smithi069.stdout:2/271: mkdir d18/d4d/d27/d33/d60/d64 0 2024-03-22T07:16:00.402 INFO:tasks.workunit.client.1.smithi069.stdout:8/425: dwrite d1/de/d6c/f6d [0,4194304] 0 2024-03-22T07:16:00.402 INFO:tasks.workunit.client.1.smithi069.stdout:2/272: rename d18/d4d/f30 to d18/d4d/d3e/f65 0 2024-03-22T07:16:00.402 INFO:tasks.workunit.client.1.smithi069.stdout:4/422: link d6/l7 d6/dc/d1d/d55/l87 0 2024-03-22T07:16:00.402 INFO:tasks.workunit.client.1.smithi069.stdout:2/273: fdatasync d18/d4d/d3e/f65 0 2024-03-22T07:16:00.402 INFO:tasks.workunit.client.1.smithi069.stdout:2/274: write d18/d4d/d3e/d40/f61 [858868,5333] 0 2024-03-22T07:16:00.405 INFO:tasks.workunit.client.1.smithi069.stdout:3/371: dread d5/d9/db/f3d [0,4194304] 0 2024-03-22T07:16:00.409 INFO:tasks.workunit.client.1.smithi069.stdout:3/372: unlink d5/d9/d26/d3f/f32 0 2024-03-22T07:16:00.411 INFO:tasks.workunit.client.1.smithi069.stdout:1/330: dwrite d1/d2a/f4e [0,4194304] 0 2024-03-22T07:16:00.411 INFO:tasks.workunit.client.0.smithi063.stdout:3/657: getdents da/d16/d26/d69/dae 0 2024-03-22T07:16:00.412 INFO:tasks.workunit.client.0.smithi063.stdout:0/779: dread da/de/d17/d4e/f9e [0,4194304] 0 2024-03-22T07:16:00.414 INFO:tasks.workunit.client.0.smithi063.stdout:3/658: rmdir da/d63/d65/d92/dc5 0 2024-03-22T07:16:00.415 INFO:tasks.workunit.client.1.smithi069.stdout:1/331: creat d1/d2a/f6c x:0 0 0 2024-03-22T07:16:00.415 INFO:tasks.workunit.client.1.smithi069.stdout:1/332: readlink d1/lf 0 2024-03-22T07:16:00.415 INFO:tasks.workunit.client.1.smithi069.stdout:1/333: readlink d1/d13/l23 0 2024-03-22T07:16:00.415 INFO:tasks.workunit.client.1.smithi069.stdout:2/275: dread fa [0,4194304] 0 2024-03-22T07:16:00.415 INFO:tasks.workunit.client.1.smithi069.stdout:5/254: dwrite d1/d7/d8/f24 [0,4194304] 0 2024-03-22T07:16:00.416 INFO:tasks.workunit.client.1.smithi069.stdout:5/255: chown d1/d6/d26/d33 236859 1 2024-03-22T07:16:00.416 INFO:tasks.workunit.client.0.smithi063.stdout:0/780: symlink da/de/d60/d56/lfc 0 2024-03-22T07:16:00.418 INFO:tasks.workunit.client.0.smithi063.stdout:0/781: rename da/de/d17/d25/d3c/d53/d89/leb to da/de/d17/d25/d3c/d43/lfd 0 2024-03-22T07:16:00.419 INFO:tasks.workunit.client.0.smithi063.stdout:0/782: fdatasync da/de/d17/d25/d3c/f3f 0 2024-03-22T07:16:00.419 INFO:tasks.workunit.client.0.smithi063.stdout:0/783: truncate da/fe0 620303 0 2024-03-22T07:16:00.422 INFO:tasks.workunit.client.1.smithi069.stdout:8/426: dread d1/f9 [4194304,4194304] 0 2024-03-22T07:16:00.422 INFO:tasks.workunit.client.1.smithi069.stdout:8/427: dread - d1/d1c/d3e/d66/f78 zero size 2024-03-22T07:16:00.426 INFO:tasks.workunit.client.1.smithi069.stdout:9/301: dwrite d2/dd/d3c/d2e/f4e [0,4194304] 0 2024-03-22T07:16:00.426 INFO:tasks.workunit.client.1.smithi069.stdout:9/302: chown d2/dd/d3c/d2e/f4b 346778 1 2024-03-22T07:16:00.426 INFO:tasks.workunit.client.1.smithi069.stdout:9/303: chown d2/d51/f56 144 1 2024-03-22T07:16:00.426 INFO:tasks.workunit.client.1.smithi069.stdout:0/380: dwrite d3/d7/d15/f45 [0,4194304] 0 2024-03-22T07:16:00.427 INFO:tasks.workunit.client.1.smithi069.stdout:1/334: read d1/d2a/f32 [4155824,117942] 0 2024-03-22T07:16:00.427 INFO:tasks.workunit.client.0.smithi063.stdout:9/522: dwrite d8/d15/f2a [0,4194304] 0 2024-03-22T07:16:00.428 INFO:tasks.workunit.client.0.smithi063.stdout:1/709: dwrite d4/df/d13/d2b/d6c/da8/fcc [0,4194304] 0 2024-03-22T07:16:00.428 INFO:tasks.workunit.client.0.smithi063.stdout:1/710: creat d4/ff1 x:0 0 0 2024-03-22T07:16:00.431 INFO:tasks.workunit.client.0.smithi063.stdout:9/523: rename d8/d19/d41 to d8/d19/d20/d22/d29/d31/d53/d6a/da2 0 2024-03-22T07:16:00.439 INFO:tasks.workunit.client.1.smithi069.stdout:2/276: symlink d18/d4d/d27/d33/d4f/l66 0 2024-03-22T07:16:00.441 INFO:tasks.workunit.client.0.smithi063.stdout:1/711: mknod d4/cf2 0 2024-03-22T07:16:00.442 INFO:tasks.workunit.client.1.smithi069.stdout:5/256: symlink d1/d7/d8/d43/l56 0 2024-03-22T07:16:00.442 INFO:tasks.workunit.client.1.smithi069.stdout:5/257: dread - d1/d28/f41 zero size 2024-03-22T07:16:00.445 INFO:tasks.workunit.client.0.smithi063.stdout:1/712: rename d4/d71/d88/f9f to d4/d5a/d91/ff3 0 2024-03-22T07:16:00.447 INFO:tasks.workunit.client.0.smithi063.stdout:3/659: dwrite da/d16/d89/d8c/dc1/fcb [0,4194304] 0 2024-03-22T07:16:00.448 INFO:tasks.workunit.client.0.smithi063.stdout:1/713: getdents d4/d5a 0 2024-03-22T07:16:00.448 INFO:tasks.workunit.client.0.smithi063.stdout:1/714: fdatasync d4/df/d13/f25 0 2024-03-22T07:16:00.448 INFO:tasks.workunit.client.1.smithi069.stdout:8/428: mknod d1/d1c/d48/d56/c86 0 2024-03-22T07:16:00.450 INFO:tasks.workunit.client.1.smithi069.stdout:0/381: dwrite d3/da/f11 [0,4194304] 0 2024-03-22T07:16:00.450 INFO:tasks.workunit.client.1.smithi069.stdout:0/382: creat d3/d7/d15/d29/d6e/f7a x:0 0 0 2024-03-22T07:16:00.450 INFO:tasks.workunit.client.1.smithi069.stdout:0/383: write d3/d7/d15/d29/f63 [651753,23369] 0 2024-03-22T07:16:00.451 INFO:tasks.workunit.client.0.smithi063.stdout:1/715: symlink d4/df/d13/d2b/lf4 0 2024-03-22T07:16:00.452 INFO:tasks.workunit.client.0.smithi063.stdout:3/660: getdents da/d63 0 2024-03-22T07:16:00.454 INFO:tasks.workunit.client.1.smithi069.stdout:3/373: rmdir d5/d9/d26 39 2024-03-22T07:16:00.456 INFO:tasks.workunit.client.0.smithi063.stdout:1/716: creat d4/df/d13/d2b/d6c/da8/ff5 x:0 0 0 2024-03-22T07:16:00.457 INFO:tasks.workunit.client.1.smithi069.stdout:0/384: dread d3/d7/f49 [0,4194304] 0 2024-03-22T07:16:00.457 INFO:tasks.workunit.client.1.smithi069.stdout:0/385: creat d3/d7/d15/d29/d5a/f7b x:0 0 0 2024-03-22T07:16:00.458 INFO:tasks.workunit.client.1.smithi069.stdout:9/304: rename d2/f5c to d2/dd/d3c/d48/f5d 0 2024-03-22T07:16:00.458 INFO:tasks.workunit.client.1.smithi069.stdout:9/305: creat d2/d51/f5e x:0 0 0 2024-03-22T07:16:00.458 INFO:tasks.workunit.client.0.smithi063.stdout:3/661: creat da/d16/d26/fdc x:0 0 0 2024-03-22T07:16:00.458 INFO:tasks.workunit.client.0.smithi063.stdout:3/662: read da/d16/d26/f27 [1732213,86592] 0 2024-03-22T07:16:00.458 INFO:tasks.workunit.client.0.smithi063.stdout:0/784: rmdir da/de/d17/d25/d3c/d43 39 2024-03-22T07:16:00.459 INFO:tasks.workunit.client.0.smithi063.stdout:1/717: mkdir d4/df/d13/d2b/d3a/df6 0 2024-03-22T07:16:00.460 INFO:tasks.workunit.client.0.smithi063.stdout:0/785: mknod da/de/d17/d25/d99/dbc/cfe 0 2024-03-22T07:16:00.462 INFO:tasks.workunit.client.0.smithi063.stdout:1/718: mknod d4/d5a/d91/cf7 0 2024-03-22T07:16:00.462 INFO:tasks.workunit.client.0.smithi063.stdout:1/719: write d4/d5a/fb0 [703736,92603] 0 2024-03-22T07:16:00.462 INFO:tasks.workunit.client.0.smithi063.stdout:1/720: getdents d4/d5a 0 2024-03-22T07:16:00.462 INFO:tasks.workunit.client.0.smithi063.stdout:1/721: truncate d4/df/d13/d2b/d3a/f42 4517186 0 2024-03-22T07:16:00.462 INFO:tasks.workunit.client.0.smithi063.stdout:0/786: dread da/f82 [0,4194304] 0 2024-03-22T07:16:00.463 INFO:tasks.workunit.client.1.smithi069.stdout:9/306: dread d2/dd/d3c/f20 [0,4194304] 0 2024-03-22T07:16:00.464 INFO:tasks.workunit.client.1.smithi069.stdout:1/335: link d1/d13/l5a d1/d2a/d50/d4c/d55/d5f/l6d 0 2024-03-22T07:16:00.464 INFO:tasks.workunit.client.0.smithi063.stdout:1/722: rename d4/d14/f62 to d4/d5a/ff8 0 2024-03-22T07:16:00.466 INFO:tasks.workunit.client.0.smithi063.stdout:0/787: rename da/de/d17/d25/d3c/d43/d80/fc4 to da/de/d17/d25/d3c/d43/db5/fff 0 2024-03-22T07:16:00.470 INFO:tasks.workunit.client.1.smithi069.stdout:8/429: link d1/d1c/l50 d1/d1c/d3e/l87 0 2024-03-22T07:16:00.470 INFO:tasks.workunit.client.0.smithi063.stdout:1/723: mknod d4/df/de7/cf9 0 2024-03-22T07:16:00.470 INFO:tasks.workunit.client.0.smithi063.stdout:1/724: fsync d4/df/d13/d2b/d6c/d85/fab 0 2024-03-22T07:16:00.474 INFO:tasks.workunit.client.1.smithi069.stdout:3/374: symlink d5/d9/db/d3b/l77 0 2024-03-22T07:16:00.474 INFO:tasks.workunit.client.0.smithi063.stdout:0/788: creat da/de/d17/d25/d3c/d53/d89/f100 x:0 0 0 2024-03-22T07:16:00.474 INFO:tasks.workunit.client.0.smithi063.stdout:9/524: dwrite d8/f14 [0,4194304] 0 2024-03-22T07:16:00.475 INFO:tasks.workunit.client.0.smithi063.stdout:3/663: dread da/f6a [0,4194304] 0 2024-03-22T07:16:00.475 INFO:tasks.workunit.client.0.smithi063.stdout:0/789: write da/de/d17/f45 [86231,53741] 0 2024-03-22T07:16:00.476 INFO:tasks.workunit.client.1.smithi069.stdout:0/386: symlink d3/d7/d24/d6a/l7c 0 2024-03-22T07:16:00.478 INFO:tasks.workunit.client.0.smithi063.stdout:1/725: unlink d4/d71/fd4 0 2024-03-22T07:16:00.479 INFO:tasks.workunit.client.1.smithi069.stdout:2/277: rename d18/d4d/d27/d33/d4f/l66 to d18/d4d/d27/d33/d60/d64/l67 0 2024-03-22T07:16:00.481 INFO:tasks.workunit.client.0.smithi063.stdout:3/664: mkdir da/d63/d65/d4b/ddd 0 2024-03-22T07:16:00.481 INFO:tasks.workunit.client.0.smithi063.stdout:3/665: write da/d16/d89/da9/dd3/fdb [505453,29193] 0 2024-03-22T07:16:00.482 INFO:tasks.workunit.client.1.smithi069.stdout:9/307: dwrite d2/d51/f56 [0,4194304] 0 2024-03-22T07:16:00.484 INFO:tasks.workunit.client.0.smithi063.stdout:0/790: creat da/de/d17/d25/d3c/d43/db5/f101 x:0 0 0 2024-03-22T07:16:00.484 INFO:tasks.workunit.client.0.smithi063.stdout:0/791: dread - da/de/d60/d9b/fa6 zero size 2024-03-22T07:16:00.486 INFO:tasks.workunit.client.0.smithi063.stdout:1/726: rename d4/d14/l37 to d4/df/de7/lfa 0 2024-03-22T07:16:00.486 INFO:tasks.workunit.client.1.smithi069.stdout:1/336: unlink d1/d2a/d5e/c66 0 2024-03-22T07:16:00.488 INFO:tasks.workunit.client.0.smithi063.stdout:1/727: mknod d4/d8b/cfb 0 2024-03-22T07:16:00.488 INFO:tasks.workunit.client.0.smithi063.stdout:1/728: fdatasync d4/df/d13/d39/d63/f66 0 2024-03-22T07:16:00.492 INFO:tasks.workunit.client.1.smithi069.stdout:8/430: mkdir d1/d1c/d3e/d88 0 2024-03-22T07:16:00.492 INFO:tasks.workunit.client.1.smithi069.stdout:8/431: fsync d1/de/d6c/d4e/d5b/f76 0 2024-03-22T07:16:00.492 INFO:tasks.workunit.client.1.smithi069.stdout:8/432: write d1/f9 [2989640,93521] 0 2024-03-22T07:16:00.492 INFO:tasks.workunit.client.1.smithi069.stdout:3/375: write d5/d33/f39 [84699,2654] 0 2024-03-22T07:16:00.492 INFO:tasks.workunit.client.1.smithi069.stdout:3/376: creat d5/d9/d23/f78 x:0 0 0 2024-03-22T07:16:00.493 INFO:tasks.workunit.client.1.smithi069.stdout:2/278: dread d18/d4d/d34/f53 [0,4194304] 0 2024-03-22T07:16:00.494 INFO:tasks.workunit.client.1.smithi069.stdout:0/387: link d3/d7/d15/d1f/f4e d3/d7/d15/d29/d5a/d5d/f7d 0 2024-03-22T07:16:00.496 INFO:tasks.workunit.client.0.smithi063.stdout:6/584: sync 2024-03-22T07:16:00.496 INFO:tasks.workunit.client.0.smithi063.stdout:4/523: sync 2024-03-22T07:16:00.498 INFO:tasks.workunit.client.0.smithi063.stdout:4/524: read - d1/d5/d43/f39 zero size 2024-03-22T07:16:00.498 INFO:tasks.workunit.client.0.smithi063.stdout:4/525: creat d1/d5/d43/d27/d2c/d4a/d78/fb9 x:0 0 0 2024-03-22T07:16:00.498 INFO:tasks.workunit.client.1.smithi069.stdout:8/433: truncate d1/de/d6c/d4e/d5b/f76 2586558 0 2024-03-22T07:16:00.499 INFO:tasks.workunit.client.0.smithi063.stdout:0/792: write da/de/d17/f1a [1993422,20464] 0 2024-03-22T07:16:00.499 INFO:tasks.workunit.client.0.smithi063.stdout:0/793: write da/de/d17/f28 [432044,96800] 0 2024-03-22T07:16:00.499 INFO:tasks.workunit.client.0.smithi063.stdout:6/585: rmdir d3/d6/d2a/d38/da5/d32/db9 0 2024-03-22T07:16:00.499 INFO:tasks.workunit.client.0.smithi063.stdout:6/586: dread - d3/d15/d16/d57/fa9 zero size 2024-03-22T07:16:00.500 INFO:tasks.workunit.client.1.smithi069.stdout:3/377: truncate d5/d9/d26/d3f/f21 741931 0 2024-03-22T07:16:00.500 INFO:tasks.workunit.client.1.smithi069.stdout:3/378: dread - d5/d9/d26/d3f/d35/f58 zero size 2024-03-22T07:16:00.502 INFO:tasks.workunit.client.0.smithi063.stdout:0/794: mknod da/de/d17/d25/d3c/d53/c102 0 2024-03-22T07:16:00.502 INFO:tasks.workunit.client.1.smithi069.stdout:2/279: symlink d18/d4d/d27/d56/l68 0 2024-03-22T07:16:00.503 INFO:tasks.workunit.client.0.smithi063.stdout:6/587: truncate d3/d6/f10 2312533 0 2024-03-22T07:16:00.504 INFO:tasks.workunit.client.0.smithi063.stdout:6/588: chown d3/d15/d16/d60/d6d/d4c/l3c 514 1 2024-03-22T07:16:00.504 INFO:tasks.workunit.client.0.smithi063.stdout:6/589: stat d3/f4 0 2024-03-22T07:16:00.504 INFO:tasks.workunit.client.1.smithi069.stdout:0/388: creat d3/d7/d5e/f7e x:0 0 0 2024-03-22T07:16:00.504 INFO:tasks.workunit.client.1.smithi069.stdout:0/389: truncate d3/da/f52 6154532 0 2024-03-22T07:16:00.506 INFO:tasks.workunit.client.0.smithi063.stdout:4/526: rename d1/d5/d3c/f82 to d1/d5/d43/d27/d2c/d3d/d54/fba 0 2024-03-22T07:16:00.506 INFO:tasks.workunit.client.0.smithi063.stdout:4/527: dread - d1/d5/dc/f89 zero size 2024-03-22T07:16:00.506 INFO:tasks.workunit.client.0.smithi063.stdout:0/795: rmdir da/de/d17/d25/d3c/d43/dc9 39 2024-03-22T07:16:00.507 INFO:tasks.workunit.client.0.smithi063.stdout:0/796: creat da/de/d17/d4e/f103 x:0 0 0 2024-03-22T07:16:00.507 INFO:tasks.workunit.client.1.smithi069.stdout:0/390: unlink d3/c58 0 2024-03-22T07:16:00.508 INFO:tasks.workunit.client.0.smithi063.stdout:6/590: mkdir d3/d15/da3/dc2 0 2024-03-22T07:16:00.510 INFO:tasks.workunit.client.0.smithi063.stdout:0/797: unlink da/de/d39/f8c 0 2024-03-22T07:16:00.510 INFO:tasks.workunit.client.1.smithi069.stdout:2/280: dread d18/d4d/d34/f53 [0,4194304] 0 2024-03-22T07:16:00.514 INFO:tasks.workunit.client.0.smithi063.stdout:9/525: dwrite d8/d17/f1d [0,4194304] 0 2024-03-22T07:16:00.514 INFO:tasks.workunit.client.0.smithi063.stdout:1/729: dread d4/d5a/fa9 [0,4194304] 0 2024-03-22T07:16:00.514 INFO:tasks.workunit.client.0.smithi063.stdout:1/730: chown d4/df/d13/d39/d63/cae 13 1 2024-03-22T07:16:00.515 INFO:tasks.workunit.client.0.smithi063.stdout:1/731: write d4/f9 [900009,103830] 0 2024-03-22T07:16:00.515 INFO:tasks.workunit.client.0.smithi063.stdout:1/732: mknod d4/d5a/cfc 0 2024-03-22T07:16:00.516 INFO:tasks.workunit.client.0.smithi063.stdout:1/733: fdatasync d4/f1f 0 2024-03-22T07:16:00.517 INFO:tasks.workunit.client.1.smithi069.stdout:8/434: dread d1/d1c/f22 [0,4194304] 0 2024-03-22T07:16:00.517 INFO:tasks.workunit.client.0.smithi063.stdout:0/798: write da/ff [1452749,7086] 0 2024-03-22T07:16:00.518 INFO:tasks.workunit.client.0.smithi063.stdout:0/799: chown da/de/d17/d25/d3c 2976937 1 2024-03-22T07:16:00.518 INFO:tasks.workunit.client.0.smithi063.stdout:0/800: fdatasync da/de/d17/f21 0 2024-03-22T07:16:00.518 INFO:tasks.workunit.client.0.smithi063.stdout:0/801: readlink da/de/d17/d25/d3c/d43/l6f 0 2024-03-22T07:16:00.518 INFO:tasks.workunit.client.0.smithi063.stdout:1/734: mkdir d4/d8b/da7/dfd 0 2024-03-22T07:16:00.519 INFO:tasks.workunit.client.1.smithi069.stdout:8/435: symlink d1/de/d6c/d4e/d5b/l89 0 2024-03-22T07:16:00.519 INFO:tasks.workunit.client.1.smithi069.stdout:8/436: chown d1/de/c80 7 1 2024-03-22T07:16:00.520 INFO:tasks.workunit.client.0.smithi063.stdout:9/526: write d8/d19/d20/d22/d29/d31/f4e [933842,87084] 0 2024-03-22T07:16:00.520 INFO:tasks.workunit.client.0.smithi063.stdout:9/527: write d8/d17/f69 [1505480,115192] 0 2024-03-22T07:16:00.523 INFO:tasks.workunit.client.0.smithi063.stdout:0/802: link da/de/d17/d25/f65 da/de/d17/d25/d3c/d43/f104 0 2024-03-22T07:16:00.523 INFO:tasks.workunit.client.0.smithi063.stdout:0/803: creat da/de/d17/d25/d3c/d64/df5/f105 x:0 0 0 2024-03-22T07:16:00.523 INFO:tasks.workunit.client.0.smithi063.stdout:0/804: readlink da/de/d17/d25/d3c/l7e 0 2024-03-22T07:16:00.523 INFO:tasks.workunit.client.0.smithi063.stdout:0/805: write da/de/d17/d25/d3c/d5f/fe8 [391917,32483] 0 2024-03-22T07:16:00.524 INFO:tasks.workunit.client.1.smithi069.stdout:0/391: write d3/da/f11 [2582498,121296] 0 2024-03-22T07:16:00.524 INFO:tasks.workunit.client.1.smithi069.stdout:0/392: chown d3/l43 1214 1 2024-03-22T07:16:00.524 INFO:tasks.workunit.client.1.smithi069.stdout:0/393: read - d3/d7/d15/d29/d5a/f7b zero size 2024-03-22T07:16:00.524 INFO:tasks.workunit.client.1.smithi069.stdout:0/394: write d3/d7/d24/d6a/f6f [912080,5750] 0 2024-03-22T07:16:00.524 INFO:tasks.workunit.client.1.smithi069.stdout:8/437: truncate d1/d1c/f52 3217246 0 2024-03-22T07:16:00.525 INFO:tasks.workunit.client.1.smithi069.stdout:8/438: write d1/d1c/d3e/f32 [1749401,122304] 0 2024-03-22T07:16:00.525 INFO:tasks.workunit.client.1.smithi069.stdout:8/439: fdatasync d1/d1c/d3e/f36 0 2024-03-22T07:16:00.527 INFO:tasks.workunit.client.1.smithi069.stdout:9/308: dwrite d2/dd/d3c/d2e/f32 [4194304,4194304] 0 2024-03-22T07:16:00.527 INFO:tasks.workunit.client.1.smithi069.stdout:9/309: dread - d2/d51/f5e zero size 2024-03-22T07:16:00.527 INFO:tasks.workunit.client.1.smithi069.stdout:9/310: write d2/dd/f37 [416443,32775] 0 2024-03-22T07:16:00.528 INFO:tasks.workunit.client.0.smithi063.stdout:0/806: rename da/de/d17/d25/d3c/d43/l6f to da/de/d17/d25/l106 0 2024-03-22T07:16:00.538 INFO:tasks.workunit.client.1.smithi069.stdout:8/440: rename d1/de/d6c/l7b to d1/d1c/d48/d56/l8a 0 2024-03-22T07:16:00.538 INFO:tasks.workunit.client.1.smithi069.stdout:8/441: write d1/d1c/f47 [953061,98627] 0 2024-03-22T07:16:00.541 INFO:tasks.workunit.client.1.smithi069.stdout:1/337: dwrite d1/d13/d36/f3e [0,4194304] 0 2024-03-22T07:16:00.543 INFO:tasks.workunit.client.0.smithi063.stdout:0/807: getdents da 0 2024-03-22T07:16:00.546 INFO:tasks.workunit.client.0.smithi063.stdout:0/808: creat da/de/d17/d25/d3c/d43/d76/d92/f107 x:0 0 0 2024-03-22T07:16:00.548 INFO:tasks.workunit.client.1.smithi069.stdout:9/311: mknod d2/d31/c5f 0 2024-03-22T07:16:00.548 INFO:tasks.workunit.client.1.smithi069.stdout:9/312: stat d2/dd/d3c/d2e/f4b 0 2024-03-22T07:16:00.550 INFO:tasks.workunit.client.1.smithi069.stdout:8/442: creat d1/d1c/d48/d5e/f8b x:0 0 0 2024-03-22T07:16:00.550 INFO:tasks.workunit.client.0.smithi063.stdout:9/528: read d8/d19/d20/d22/d29/d31/d53/d6a/da2/f4a [7262394,45541] 0 2024-03-22T07:16:00.551 INFO:tasks.workunit.client.0.smithi063.stdout:9/529: dread - d8/d15/d2f/f93 zero size 2024-03-22T07:16:00.554 INFO:tasks.workunit.client.0.smithi063.stdout:9/530: creat d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/fa3 x:0 0 0 2024-03-22T07:16:00.555 INFO:tasks.workunit.client.0.smithi063.stdout:9/531: chown d8/d19/d2b/c2c 495 1 2024-03-22T07:16:00.560 INFO:tasks.workunit.client.1.smithi069.stdout:9/313: mknod d2/d51/c60 0 2024-03-22T07:16:00.564 INFO:tasks.workunit.client.1.smithi069.stdout:9/314: unlink d2/dd/d3c/c25 0 2024-03-22T07:16:00.566 INFO:tasks.workunit.client.1.smithi069.stdout:9/315: write d2/dd/d3c/f43 [12868,30794] 0 2024-03-22T07:16:00.568 INFO:tasks.workunit.client.1.smithi069.stdout:3/379: dwrite d5/d9/f13 [0,4194304] 0 2024-03-22T07:16:00.568 INFO:tasks.workunit.client.1.smithi069.stdout:3/380: write d5/d9/f68 [1545013,95763] 0 2024-03-22T07:16:00.570 INFO:tasks.workunit.client.0.smithi063.stdout:1/735: dwrite d4/f1f [0,4194304] 0 2024-03-22T07:16:00.572 INFO:tasks.workunit.client.0.smithi063.stdout:1/736: rename d4/df/d13/d2b/d6c/da8/db2 to d4/d8b/dfe 0 2024-03-22T07:16:00.574 INFO:tasks.workunit.client.1.smithi069.stdout:3/381: mknod d5/d9/d26/d3f/d56/c79 0 2024-03-22T07:16:00.576 INFO:tasks.workunit.client.0.smithi063.stdout:6/591: dwrite d3/d15/f62 [0,4194304] 0 2024-03-22T07:16:00.577 INFO:tasks.workunit.client.1.smithi069.stdout:3/382: creat d5/d9/d26/d3f/d35/f7a x:0 0 0 2024-03-22T07:16:00.578 INFO:tasks.workunit.client.1.smithi069.stdout:3/383: mkdir d5/d33/d7b 0 2024-03-22T07:16:00.578 INFO:tasks.workunit.client.0.smithi063.stdout:4/528: dwrite d1/d5/d43/d27/d2c/d4a/f4d [0,4194304] 0 2024-03-22T07:16:00.578 INFO:tasks.workunit.client.0.smithi063.stdout:4/529: write d1/d5/dc/d95/fb5 [918279,84965] 0 2024-03-22T07:16:00.580 INFO:tasks.workunit.client.0.smithi063.stdout:4/530: getdents d1/d5/d43/d27/d2c/d3d/d54/d5f 0 2024-03-22T07:16:00.581 INFO:tasks.workunit.client.0.smithi063.stdout:4/531: chown d1/d5/f86 141301 1 2024-03-22T07:16:00.584 INFO:tasks.workunit.client.0.smithi063.stdout:4/532: getdents d1/d5/d85 0 2024-03-22T07:16:00.590 INFO:tasks.workunit.client.1.smithi069.stdout:2/281: dwrite d18/d4d/d3e/f5d [4194304,4194304] 0 2024-03-22T07:16:00.591 INFO:tasks.workunit.client.1.smithi069.stdout:0/395: dwrite d3/d7/d15/d29/d5a/f6c [0,4194304] 0 2024-03-22T07:16:00.592 INFO:tasks.workunit.client.1.smithi069.stdout:0/396: chown d3/d7/f1e 16 1 2024-03-22T07:16:00.593 INFO:tasks.workunit.client.1.smithi069.stdout:2/282: mknod d18/c69 0 2024-03-22T07:16:00.599 INFO:tasks.workunit.client.0.smithi063.stdout:5/523: sync 2024-03-22T07:16:00.600 INFO:tasks.workunit.client.0.smithi063.stdout:8/727: sync 2024-03-22T07:16:00.601 INFO:tasks.workunit.client.0.smithi063.stdout:4/533: dread d1/d5/dc/f1d [0,4194304] 0 2024-03-22T07:16:00.604 INFO:tasks.workunit.client.0.smithi063.stdout:0/809: dwrite da/de/d17/d25/d3c/d43/fee [0,4194304] 0 2024-03-22T07:16:00.604 INFO:tasks.workunit.client.1.smithi069.stdout:8/443: dwrite d1/d1c/d3e/f61 [0,4194304] 0 2024-03-22T07:16:00.605 INFO:tasks.workunit.client.0.smithi063.stdout:9/532: dwrite d8/f9d [0,4194304] 0 2024-03-22T07:16:00.605 INFO:tasks.workunit.client.0.smithi063.stdout:4/534: rmdir d1/d5/d3c 39 2024-03-22T07:16:00.606 INFO:tasks.workunit.client.1.smithi069.stdout:1/338: dwrite d1/d13/f39 [0,4194304] 0 2024-03-22T07:16:00.609 INFO:tasks.workunit.client.1.smithi069.stdout:1/339: dread d1/fb [0,4194304] 0 2024-03-22T07:16:00.609 INFO:tasks.workunit.client.1.smithi069.stdout:9/316: dwrite d2/d51/f57 [0,4194304] 0 2024-03-22T07:16:00.609 INFO:tasks.workunit.client.1.smithi069.stdout:9/317: write d2/dd/d3c/f35 [388321,34071] 0 2024-03-22T07:16:00.609 INFO:tasks.workunit.client.1.smithi069.stdout:9/318: chown d2/dd/d3c/c36 8033 1 2024-03-22T07:16:00.610 INFO:tasks.workunit.client.0.smithi063.stdout:9/533: truncate d8/fa 363246 0 2024-03-22T07:16:00.610 INFO:tasks.workunit.client.0.smithi063.stdout:9/534: creat d8/d19/d20/d22/fa4 x:0 0 0 2024-03-22T07:16:00.610 INFO:tasks.workunit.client.0.smithi063.stdout:9/535: readlink d8/d15/l5e 0 2024-03-22T07:16:00.612 INFO:tasks.workunit.client.1.smithi069.stdout:8/444: creat d1/d1c/d48/f8c x:0 0 0 2024-03-22T07:16:00.612 INFO:tasks.workunit.client.1.smithi069.stdout:8/445: read - d1/d1c/d48/f8c zero size 2024-03-22T07:16:00.612 INFO:tasks.workunit.client.1.smithi069.stdout:8/446: stat d1/de/l35 0 2024-03-22T07:16:00.612 INFO:tasks.workunit.client.0.smithi063.stdout:9/536: mkdir d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/da5 0 2024-03-22T07:16:00.613 INFO:tasks.workunit.client.0.smithi063.stdout:4/535: mkdir d1/d5/d3c/d8f/dbb 0 2024-03-22T07:16:00.614 INFO:tasks.workunit.client.0.smithi063.stdout:6/592: dwrite d3/d15/d16/d60/d6d/d4e/d5a/d88/fc1 [0,4194304] 0 2024-03-22T07:16:00.615 INFO:tasks.workunit.client.0.smithi063.stdout:9/537: symlink d8/d15/d63/la6 0 2024-03-22T07:16:00.615 INFO:tasks.workunit.client.0.smithi063.stdout:9/538: chown d8/d19/d20/d22/d29/d31/d4b 256504070 1 2024-03-22T07:16:00.620 INFO:tasks.workunit.client.0.smithi063.stdout:9/539: mknod d8/d19/d20/ca7 0 2024-03-22T07:16:00.620 INFO:tasks.workunit.client.0.smithi063.stdout:9/540: truncate d8/d19/d20/d22/d29/d31/d4b/f4d 1447241 0 2024-03-22T07:16:00.620 INFO:tasks.workunit.client.0.smithi063.stdout:9/541: mknod d8/d19/d20/d22/d29/d31/d53/d6a/da2/ca8 0 2024-03-22T07:16:00.621 INFO:tasks.workunit.client.0.smithi063.stdout:1/737: dwrite d4/d8b/da7/fc6 [0,4194304] 0 2024-03-22T07:16:00.622 INFO:tasks.workunit.client.0.smithi063.stdout:8/728: dread d0/d1b/f20 [0,4194304] 0 2024-03-22T07:16:00.622 INFO:tasks.workunit.client.0.smithi063.stdout:6/593: rmdir d3/d6/d2a/d38/da5/d32/d51/d64 0 2024-03-22T07:16:00.622 INFO:tasks.workunit.client.0.smithi063.stdout:6/594: chown d3/d6/d2a/d38/da5 135035804 1 2024-03-22T07:16:00.622 INFO:tasks.workunit.client.0.smithi063.stdout:6/595: fsync d3/d6/f10 0 2024-03-22T07:16:00.623 INFO:tasks.workunit.client.1.smithi069.stdout:3/384: getdents d5/d9/d26/d3f/d35 0 2024-03-22T07:16:00.623 INFO:tasks.workunit.client.1.smithi069.stdout:3/385: read d5/d9/d26/d3f/f21 [337621,20493] 0 2024-03-22T07:16:00.623 INFO:tasks.workunit.client.1.smithi069.stdout:3/386: getdents d5/d9/d26/d3f/d35 0 2024-03-22T07:16:00.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='client.14602 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:00.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:16:00.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:16:00.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:16:00.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:16:00.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:16:00.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:16:00.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr fail", "who": "smithi063.mnxayh"}]: dispatch 2024-03-22T07:16:00.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: osdmap e48: 6 total, 6 up, 6 in 2024-03-22T07:16:00.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "mgr fail", "who": "smithi063.mnxayh"}]': finished 2024-03-22T07:16:00.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: mgrmap e21: smithi069.gfziwh(active, starting, since 0.0196224s) 2024-03-22T07:16:00.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:16:00.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:16:00.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.xeelpk"}]: dispatch 2024-03-22T07:16:00.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.oytapz"}]: dispatch 2024-03-22T07:16:00.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.wwtsjn"}]: dispatch 2024-03-22T07:16:00.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.cijxah"}]: dispatch 2024-03-22T07:16:00.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mgr metadata", "who": "smithi069.gfziwh", "id": "smithi069.gfziwh"}]: dispatch 2024-03-22T07:16:00.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:16:00.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:16:00.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:16:00.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:16:00.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:16:00.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:16:00.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:16:00.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:16:00.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:16:00.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:00 smithi063 ceph-mon[29200]: Manager daemon smithi069.gfziwh is now available 2024-03-22T07:16:00.631 INFO:tasks.workunit.client.1.smithi069.stdout:9/319: truncate d2/f27 3292638 0 2024-03-22T07:16:00.634 INFO:tasks.workunit.client.1.smithi069.stdout:8/447: mknod d1/d1c/d3e/d88/c8d 0 2024-03-22T07:16:00.634 INFO:tasks.workunit.client.1.smithi069.stdout:8/448: write d1/de/f30 [122881,130540] 0 2024-03-22T07:16:00.634 INFO:tasks.workunit.client.1.smithi069.stdout:8/449: write d1/de/d6c/f72 [72487,25482] 0 2024-03-22T07:16:00.634 INFO:tasks.workunit.client.1.smithi069.stdout:8/450: chown d1/f29 11534 1 2024-03-22T07:16:00.637 INFO:tasks.workunit.client.1.smithi069.stdout:3/387: write d5/d9/f13 [2066439,2270] 0 2024-03-22T07:16:00.637 INFO:tasks.workunit.client.1.smithi069.stdout:9/320: rmdir d2/d31 39 2024-03-22T07:16:00.638 INFO:tasks.workunit.client.1.smithi069.stdout:9/321: write d2/d51/f5e [893698,47280] 0 2024-03-22T07:16:00.642 INFO:tasks.workunit.client.1.smithi069.stdout:3/388: creat d5/d9/d26/d36/d41/d51/f7c x:0 0 0 2024-03-22T07:16:00.642 INFO:tasks.workunit.client.1.smithi069.stdout:3/389: chown d5/d9/d26/l29 2205 1 2024-03-22T07:16:00.643 INFO:tasks.workunit.client.1.smithi069.stdout:9/322: symlink d2/dd/d3c/l61 0 2024-03-22T07:16:00.644 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='client.14602 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:00.644 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:16:00.644 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:16:00.644 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:16:00.644 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:16:00.644 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' 2024-03-22T07:16:00.644 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:16:00.644 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr fail", "who": "smithi063.mnxayh"}]: dispatch 2024-03-22T07:16:00.644 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: osdmap e48: 6 total, 6 up, 6 in 2024-03-22T07:16:00.644 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.14208 172.21.15.63:0/2064825851' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "mgr fail", "who": "smithi063.mnxayh"}]': finished 2024-03-22T07:16:00.645 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: mgrmap e21: smithi069.gfziwh(active, starting, since 0.0196224s) 2024-03-22T07:16:00.645 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:16:00.645 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:16:00.645 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.xeelpk"}]: dispatch 2024-03-22T07:16:00.645 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.oytapz"}]: dispatch 2024-03-22T07:16:00.645 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.wwtsjn"}]: dispatch 2024-03-22T07:16:00.645 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.cijxah"}]: dispatch 2024-03-22T07:16:00.645 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mgr metadata", "who": "smithi069.gfziwh", "id": "smithi069.gfziwh"}]: dispatch 2024-03-22T07:16:00.645 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:16:00.645 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:16:00.645 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:16:00.645 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:16:00.645 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:16:00.645 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:16:00.645 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:16:00.645 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:16:00.646 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:16:00.646 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:00 smithi069 ceph-mon[36441]: Manager daemon smithi069.gfziwh is now available 2024-03-22T07:16:00.646 INFO:tasks.workunit.client.1.smithi069.stdout:8/451: write d1/de/d20/f4f [588485,37650] 0 2024-03-22T07:16:00.646 INFO:tasks.workunit.client.1.smithi069.stdout:3/390: rename d5/f4a to d5/d9/db/f7d 0 2024-03-22T07:16:00.646 INFO:tasks.workunit.client.1.smithi069.stdout:3/391: write d5/d9/db/f7d [273492,50533] 0 2024-03-22T07:16:00.646 INFO:tasks.workunit.client.1.smithi069.stdout:3/392: getdents d5/d9/d26/d3f/d54 0 2024-03-22T07:16:00.647 INFO:tasks.workunit.client.1.smithi069.stdout:9/323: mknod d2/c62 0 2024-03-22T07:16:00.650 INFO:tasks.workunit.client.1.smithi069.stdout:3/393: link d5/d9/db/d3b/l66 d5/d9/d26/d3f/l7e 0 2024-03-22T07:16:00.654 INFO:tasks.workunit.client.1.smithi069.stdout:9/324: rename d2/dd/d3c/l4f to d2/d51/l63 0 2024-03-22T07:16:00.657 INFO:tasks.workunit.client.0.smithi063.stdout:5/524: dwrite d8/d13/d1e/f2f [0,4194304] 0 2024-03-22T07:16:00.667 INFO:tasks.workunit.client.1.smithi069.stdout:1/340: dwrite d1/d25/f38 [0,4194304] 0 2024-03-22T07:16:00.667 INFO:tasks.workunit.client.1.smithi069.stdout:1/341: stat d1/f48 0 2024-03-22T07:16:00.667 INFO:tasks.workunit.client.1.smithi069.stdout:1/342: fdatasync d1/d13/d36/f6b 0 2024-03-22T07:16:00.667 INFO:tasks.workunit.client.1.smithi069.stdout:1/343: write d1/f42 [1388769,125432] 0 2024-03-22T07:16:00.667 INFO:tasks.workunit.client.1.smithi069.stdout:1/344: chown d1/d2a/d50/d4c/d63/c69 14747076 1 2024-03-22T07:16:00.667 INFO:tasks.workunit.client.1.smithi069.stdout:1/345: fsync d1/d13/f39 0 2024-03-22T07:16:00.668 INFO:tasks.workunit.client.0.smithi063.stdout:0/810: dwrite da/de/d60/f33 [0,4194304] 0 2024-03-22T07:16:00.668 INFO:tasks.workunit.client.0.smithi063.stdout:0/811: creat da/de/d60/d9b/f108 x:0 0 0 2024-03-22T07:16:00.668 INFO:tasks.workunit.client.1.smithi069.stdout:0/397: dwrite d3/d7/d15/f20 [0,4194304] 0 2024-03-22T07:16:00.669 INFO:tasks.workunit.client.1.smithi069.stdout:0/398: truncate d3/d7/d17/f3f 994749 0 2024-03-22T07:16:00.673 INFO:tasks.workunit.client.1.smithi069.stdout:1/346: mkdir d1/d13/d36/d6e 0 2024-03-22T07:16:00.680 INFO:tasks.workunit.client.0.smithi063.stdout:1/738: dwrite d4/d5a/fb0 [0,4194304] 0 2024-03-22T07:16:00.690 INFO:tasks.workunit.client.1.smithi069.stdout:9/325: dwrite d2/dd/f26 [0,4194304] 0 2024-03-22T07:16:00.690 INFO:tasks.workunit.client.1.smithi069.stdout:9/326: creat d2/d51/f64 x:0 0 0 2024-03-22T07:16:00.690 INFO:tasks.workunit.client.1.smithi069.stdout:9/327: write d2/d51/f64 [184064,44866] 0 2024-03-22T07:16:00.696 INFO:tasks.workunit.client.0.smithi063.stdout:8/729: mknod d0/d26/d2d/d83/cf9 0 2024-03-22T07:16:00.697 INFO:tasks.workunit.client.1.smithi069.stdout:0/399: link d3/d7/d17/d1c/c59 d3/d7/d15/d29/d6e/c7f 0 2024-03-22T07:16:00.698 INFO:tasks.workunit.client.1.smithi069.stdout:0/400: chown d3/d7/d24/l4d 12856881 1 2024-03-22T07:16:00.700 INFO:tasks.workunit.client.0.smithi063.stdout:5/525: creat d8/d13/d1e/d31/d58/d7e/d82/f96 x:0 0 0 2024-03-22T07:16:00.700 INFO:tasks.workunit.client.0.smithi063.stdout:5/526: write d8/f25 [289242,125631] 0 2024-03-22T07:16:00.703 INFO:tasks.workunit.client.1.smithi069.stdout:1/347: mknod d1/c6f 0 2024-03-22T07:16:00.706 INFO:tasks.workunit.client.1.smithi069.stdout:8/452: dwrite d1/de/f30 [0,4194304] 0 2024-03-22T07:16:00.719 INFO:tasks.workunit.client.1.smithi069.stdout:9/328: creat d2/dd/d3c/d2e/f65 x:0 0 0 2024-03-22T07:16:00.719 INFO:tasks.workunit.client.1.smithi069.stdout:9/329: fsync d2/d21/f3a 0 2024-03-22T07:16:00.719 INFO:tasks.workunit.client.1.smithi069.stdout:9/330: creat d2/dd/d3c/d2e/f66 x:0 0 0 2024-03-22T07:16:00.721 INFO:tasks.workunit.client.1.smithi069.stdout:1/348: link d1/d2a/f4e d1/d2a/d50/d4c/d63/f70 0 2024-03-22T07:16:00.721 INFO:tasks.workunit.client.1.smithi069.stdout:1/349: chown d1/f48 265830 1 2024-03-22T07:16:00.723 INFO:tasks.workunit.client.1.smithi069.stdout:8/453: mkdir d1/de/d6c/d8e 0 2024-03-22T07:16:00.727 INFO:tasks.workunit.client.1.smithi069.stdout:0/401: dwrite d3/d7/d17/f6b [4194304,4194304] 0 2024-03-22T07:16:00.728 INFO:tasks.workunit.client.1.smithi069.stdout:9/331: creat d2/d31/f67 x:0 0 0 2024-03-22T07:16:00.730 INFO:tasks.workunit.client.1.smithi069.stdout:1/350: creat d1/d13/d36/d5b/f71 x:0 0 0 2024-03-22T07:16:00.732 INFO:tasks.workunit.client.0.smithi063.stdout:5/527: dwrite d8/d13/d1e/f27 [4194304,4194304] 0 2024-03-22T07:16:00.733 INFO:tasks.workunit.client.0.smithi063.stdout:5/528: stat d8/d13/d1e/f27 0 2024-03-22T07:16:00.733 INFO:tasks.workunit.client.0.smithi063.stdout:5/529: write d8/d13/d1e/d31/d64/d65/f71 [729017,60528] 0 2024-03-22T07:16:00.734 INFO:tasks.workunit.client.1.smithi069.stdout:7/372: sync 2024-03-22T07:16:00.734 INFO:tasks.workunit.client.1.smithi069.stdout:7/373: read - d9/d76/d57/d44/d53/d2e/f7c zero size 2024-03-22T07:16:00.735 INFO:tasks.workunit.client.1.smithi069.stdout:7/374: fdatasync d9/db/d48/d4b/f4f 0 2024-03-22T07:16:00.735 INFO:tasks.workunit.client.1.smithi069.stdout:6/312: sync 2024-03-22T07:16:00.739 INFO:tasks.workunit.client.1.smithi069.stdout:1/351: rename d1/d13/f1b to d1/d25/d41/d1e/f72 0 2024-03-22T07:16:00.745 INFO:tasks.workunit.client.1.smithi069.stdout:7/375: creat d9/d76/d57/d75/f81 x:0 0 0 2024-03-22T07:16:00.745 INFO:tasks.workunit.client.1.smithi069.stdout:7/376: fsync d9/d76/d57/d44/d53/d74/d30/f54 0 2024-03-22T07:16:00.748 INFO:tasks.workunit.client.1.smithi069.stdout:7/377: creat d9/db/d1a/f82 x:0 0 0 2024-03-22T07:16:00.748 INFO:tasks.workunit.client.1.smithi069.stdout:7/378: chown d9/db/d48/d4b/f77 1660950 1 2024-03-22T07:16:00.749 INFO:tasks.workunit.client.1.smithi069.stdout:7/379: dread d9/d76/d57/d44/d53/d2e/f4d [0,4194304] 0 2024-03-22T07:16:00.749 INFO:tasks.workunit.client.1.smithi069.stdout:6/313: truncate d4/d7/f3e 2922675 0 2024-03-22T07:16:00.752 INFO:tasks.workunit.client.1.smithi069.stdout:7/380: creat d9/d76/d57/d44/d53/d74/d6e/f83 x:0 0 0 2024-03-22T07:16:00.752 INFO:tasks.workunit.client.1.smithi069.stdout:7/381: truncate d9/db/d1a/f39 1216918 0 2024-03-22T07:16:00.753 INFO:tasks.workunit.client.1.smithi069.stdout:6/314: creat d4/d7/da/d3f/f61 x:0 0 0 2024-03-22T07:16:00.757 INFO:tasks.workunit.client.1.smithi069.stdout:7/382: unlink d9/d76/d57/d44/d53/d74/d6e/c70 0 2024-03-22T07:16:00.758 INFO:tasks.workunit.client.1.smithi069.stdout:9/332: dwrite d2/d51/f57 [0,4194304] 0 2024-03-22T07:16:00.758 INFO:tasks.workunit.client.1.smithi069.stdout:0/402: dwrite d3/d7/d24/f34 [0,4194304] 0 2024-03-22T07:16:00.759 INFO:tasks.workunit.client.1.smithi069.stdout:0/403: chown d3/d7/d15/d29/d5a/d5d 13 1 2024-03-22T07:16:00.760 INFO:tasks.workunit.client.1.smithi069.stdout:6/315: creat d4/dd/d12/d44/d58/f62 x:0 0 0 2024-03-22T07:16:00.761 INFO:tasks.workunit.client.1.smithi069.stdout:0/404: dread d3/da/f3a [0,4194304] 0 2024-03-22T07:16:00.762 INFO:tasks.workunit.client.1.smithi069.stdout:1/352: dwrite d1/d2a/f6c [0,4194304] 0 2024-03-22T07:16:00.762 INFO:tasks.workunit.client.1.smithi069.stdout:9/333: symlink d2/dd/l68 0 2024-03-22T07:16:00.763 INFO:tasks.workunit.client.1.smithi069.stdout:6/316: truncate d4/dd/d12/f32 1335707 0 2024-03-22T07:16:00.763 INFO:tasks.workunit.client.1.smithi069.stdout:6/317: readlink d4/d7/l2d 0 2024-03-22T07:16:00.763 INFO:tasks.workunit.client.1.smithi069.stdout:6/318: creat d4/dd/d12/d44/d58/f63 x:0 0 0 2024-03-22T07:16:00.765 INFO:tasks.workunit.client.1.smithi069.stdout:0/405: symlink d3/d7/d15/d29/l80 0 2024-03-22T07:16:00.768 INFO:tasks.workunit.client.0.smithi063.stdout:1/739: mkdir d4/df/d13/d39/d63/dff 0 2024-03-22T07:16:00.775 INFO:tasks.workunit.client.1.smithi069.stdout:1/353: mkdir d1/d13/d36/d6e/d73 0 2024-03-22T07:16:00.777 INFO:tasks.workunit.client.1.smithi069.stdout:9/334: creat d2/dd/f69 x:0 0 0 2024-03-22T07:16:00.777 INFO:tasks.workunit.client.1.smithi069.stdout:9/335: write d2/dd/f37 [31166,88083] 0 2024-03-22T07:16:00.777 INFO:tasks.workunit.client.1.smithi069.stdout:0/406: symlink d3/d7/d5e/l81 0 2024-03-22T07:16:00.777 INFO:tasks.workunit.client.1.smithi069.stdout:0/407: chown d3/d7/d17/l2c 4 1 2024-03-22T07:16:00.779 INFO:tasks.workunit.client.1.smithi069.stdout:1/354: rmdir d1/d25/d41 39 2024-03-22T07:16:00.781 INFO:tasks.workunit.client.1.smithi069.stdout:0/408: symlink d3/d7/d24/d6a/l82 0 2024-03-22T07:16:00.781 INFO:tasks.workunit.client.0.smithi063.stdout:1/740: dread d4/d14/f21 [0,4194304] 0 2024-03-22T07:16:00.781 INFO:tasks.workunit.client.0.smithi063.stdout:2/574: sync 2024-03-22T07:16:00.781 INFO:tasks.workunit.client.0.smithi063.stdout:7/485: sync 2024-03-22T07:16:00.783 INFO:tasks.workunit.client.1.smithi069.stdout:0/409: creat d3/d7/d17/f83 x:0 0 0 2024-03-22T07:16:00.783 INFO:tasks.workunit.client.1.smithi069.stdout:0/410: write d3/d7/d5e/f77 [69219,127836] 0 2024-03-22T07:16:00.788 INFO:tasks.workunit.client.1.smithi069.stdout:6/319: dwrite d4/dd/d41/f51 [0,4194304] 0 2024-03-22T07:16:00.789 INFO:tasks.workunit.client.1.smithi069.stdout:6/320: write d4/dd/d12/f1f [4876378,90791] 0 2024-03-22T07:16:00.789 INFO:tasks.workunit.client.1.smithi069.stdout:6/321: write d4/dd/d12/d44/d58/f63 [581168,85085] 0 2024-03-22T07:16:00.793 INFO:tasks.workunit.client.0.smithi063.stdout:9/542: rename d8/d19/d2b/l5a to d8/d19/d20/d22/d29/d31/d53/d6a/la9 0 2024-03-22T07:16:00.794 INFO:tasks.workunit.client.0.smithi063.stdout:8/730: symlink d0/d1b/d28/d3e/lfa 0 2024-03-22T07:16:00.795 INFO:tasks.workunit.client.0.smithi063.stdout:8/731: write d0/f64 [291405,29234] 0 2024-03-22T07:16:00.796 INFO:tasks.workunit.client.0.smithi063.stdout:6/596: unlink d3/d15/d16/d60/d6d/l33 0 2024-03-22T07:16:00.800 INFO:tasks.workunit.client.0.smithi063.stdout:1/741: write d4/d14/f21 [4237120,17580] 0 2024-03-22T07:16:00.800 INFO:tasks.workunit.client.0.smithi063.stdout:1/742: chown d4/df/d13/d39/d63/dff 3481 1 2024-03-22T07:16:00.800 INFO:tasks.workunit.client.0.smithi063.stdout:0/812: rmdir da/de/d17/d25/d95/d98/db3 39 2024-03-22T07:16:00.803 INFO:tasks.workunit.client.1.smithi069.stdout:6/322: symlink d4/d7/d5b/l64 0 2024-03-22T07:16:00.806 INFO:tasks.workunit.client.1.smithi069.stdout:7/383: dwrite d9/db/f16 [0,4194304] 0 2024-03-22T07:16:00.807 INFO:tasks.workunit.client.1.smithi069.stdout:7/384: readlink l5 0 2024-03-22T07:16:00.807 INFO:tasks.workunit.client.1.smithi069.stdout:0/411: rename d3/da to d3/d7/d15/d29/d5a/d84 0 2024-03-22T07:16:00.809 INFO:tasks.workunit.client.1.smithi069.stdout:9/336: dwrite d2/dd/d3c/d2e/f32 [0,4194304] 0 2024-03-22T07:16:00.809 INFO:tasks.workunit.client.1.smithi069.stdout:9/337: stat d2/dd/d3c/d2e/f65 0 2024-03-22T07:16:00.809 INFO:tasks.workunit.client.1.smithi069.stdout:9/338: creat d2/dd/d3c/d2e/f6a x:0 0 0 2024-03-22T07:16:00.809 INFO:tasks.workunit.client.1.smithi069.stdout:9/339: chown d2/dd/d3c/f3b 11701550 1 2024-03-22T07:16:00.810 INFO:tasks.workunit.client.0.smithi063.stdout:5/530: symlink d8/d13/d1e/l97 0 2024-03-22T07:16:00.810 INFO:tasks.workunit.client.0.smithi063.stdout:5/531: fsync d8/d13/f4b 0 2024-03-22T07:16:00.811 INFO:tasks.workunit.client.0.smithi063.stdout:5/532: fsync d8/f2b 0 2024-03-22T07:16:00.811 INFO:tasks.workunit.client.0.smithi063.stdout:0/813: dread da/ff [0,4194304] 0 2024-03-22T07:16:00.811 INFO:tasks.workunit.client.0.smithi063.stdout:0/814: fsync da/de/d60/d56/d8b/fb9 0 2024-03-22T07:16:00.811 INFO:tasks.workunit.client.1.smithi069.stdout:6/323: mknod d4/dd/d12/c65 0 2024-03-22T07:16:00.811 INFO:tasks.workunit.client.0.smithi063.stdout:2/575: symlink d0/de/d58/d7f/d97/da2/lb6 0 2024-03-22T07:16:00.814 INFO:tasks.workunit.client.0.smithi063.stdout:9/543: unlink d8/d19/d20/d22/d5d/l92 0 2024-03-22T07:16:00.814 INFO:tasks.workunit.client.0.smithi063.stdout:9/544: fdatasync d8/d19/d2b/f89 0 2024-03-22T07:16:00.814 INFO:tasks.workunit.client.0.smithi063.stdout:9/545: stat d8/d19/d20/d22/d29/d31/d37/f59 0 2024-03-22T07:16:00.814 INFO:tasks.workunit.client.1.smithi069.stdout:7/385: symlink d9/d76/d57/d44/l84 0 2024-03-22T07:16:00.814 INFO:tasks.workunit.client.1.smithi069.stdout:1/355: dwrite d1/d2a/d50/d4c/d63/f70 [0,4194304] 0 2024-03-22T07:16:00.818 INFO:tasks.workunit.client.0.smithi063.stdout:1/743: unlink d4/df/c33 0 2024-03-22T07:16:00.819 INFO:tasks.workunit.client.0.smithi063.stdout:5/533: mknod d8/d13/d1e/d31/d58/d7e/c98 0 2024-03-22T07:16:00.820 INFO:tasks.workunit.client.1.smithi069.stdout:9/340: dread d2/d51/f57 [0,4194304] 0 2024-03-22T07:16:00.822 INFO:tasks.workunit.client.0.smithi063.stdout:0/815: rename da/de/d17/d25/d3c/d53/d89/fd1 to da/de/d17/d25/d3c/d43/d76/d92/f109 0 2024-03-22T07:16:00.825 INFO:tasks.workunit.client.0.smithi063.stdout:2/576: creat d0/de/d1b/d8b/fb7 x:0 0 0 2024-03-22T07:16:00.825 INFO:tasks.workunit.client.0.smithi063.stdout:2/577: truncate d0/de/d58/d7f/fa7 894930 0 2024-03-22T07:16:00.826 INFO:tasks.workunit.client.0.smithi063.stdout:9/546: creat d8/d19/d20/d22/d29/faa x:0 0 0 2024-03-22T07:16:00.826 INFO:tasks.workunit.client.0.smithi063.stdout:9/547: truncate d8/d19/d20/d22/d5d/f64 924168 0 2024-03-22T07:16:00.827 INFO:tasks.workunit.client.1.smithi069.stdout:7/386: unlink d9/d42/f64 0 2024-03-22T07:16:00.828 INFO:tasks.workunit.client.0.smithi063.stdout:1/744: creat d4/df/de7/f100 x:0 0 0 2024-03-22T07:16:00.830 INFO:tasks.workunit.client.0.smithi063.stdout:9/548: getdents d8/d19/d20/d22 0 2024-03-22T07:16:00.830 INFO:tasks.workunit.client.0.smithi063.stdout:9/549: readlink d8/d17/l33 0 2024-03-22T07:16:00.830 INFO:tasks.workunit.client.0.smithi063.stdout:9/550: read - d8/d19/d20/fa0 zero size 2024-03-22T07:16:00.831 INFO:tasks.workunit.client.0.smithi063.stdout:1/745: mknod d4/d8b/da7/c101 0 2024-03-22T07:16:00.833 INFO:tasks.workunit.client.1.smithi069.stdout:1/356: rmdir d1/d25/d41 39 2024-03-22T07:16:00.834 INFO:tasks.workunit.client.1.smithi069.stdout:0/412: getdents d3/d7/d15/d29 0 2024-03-22T07:16:00.834 INFO:tasks.workunit.client.1.smithi069.stdout:0/413: creat d3/d7/d17/f85 x:0 0 0 2024-03-22T07:16:00.834 INFO:tasks.workunit.client.0.smithi063.stdout:9/551: write d8/d19/d20/d22/d29/d31/d37/f59 [819975,92870] 0 2024-03-22T07:16:00.835 INFO:tasks.workunit.client.1.smithi069.stdout:7/387: unlink d9/d76/d57/d44/l84 0 2024-03-22T07:16:00.837 INFO:tasks.workunit.client.0.smithi063.stdout:2/578: dread d0/de/d1b/d32/f3b [0,4194304] 0 2024-03-22T07:16:00.837 INFO:tasks.workunit.client.0.smithi063.stdout:2/579: read - d0/de/d35/d36/f6d zero size 2024-03-22T07:16:00.839 INFO:tasks.workunit.client.1.smithi069.stdout:0/414: mknod d3/d71/c86 0 2024-03-22T07:16:00.839 INFO:tasks.workunit.client.0.smithi063.stdout:9/552: mkdir d8/d19/d20/d22/d5d/d76/dab 0 2024-03-22T07:16:00.840 INFO:tasks.workunit.client.1.smithi069.stdout:7/388: creat d9/db/d1a/f85 x:0 0 0 2024-03-22T07:16:00.842 INFO:tasks.workunit.client.0.smithi063.stdout:9/553: symlink d8/d19/d20/d22/d5d/lac 0 2024-03-22T07:16:00.845 INFO:tasks.workunit.client.1.smithi069.stdout:7/389: getdents d9 0 2024-03-22T07:16:00.845 INFO:tasks.workunit.client.0.smithi063.stdout:9/554: getdents d8/d19/d20 0 2024-03-22T07:16:00.845 INFO:tasks.workunit.client.0.smithi063.stdout:9/555: chown d8/d19/d20/d22/d29/d31 0 1 2024-03-22T07:16:00.848 INFO:tasks.workunit.client.0.smithi063.stdout:9/556: symlink d8/d15/d2f/lad 0 2024-03-22T07:16:00.848 INFO:tasks.workunit.client.1.smithi069.stdout:7/390: link d9/l6c d9/d42/l86 0 2024-03-22T07:16:00.850 INFO:tasks.workunit.client.0.smithi063.stdout:6/597: creat d3/d6/d2a/d38/da5/d32/fc3 x:0 0 0 2024-03-22T07:16:00.852 INFO:tasks.workunit.client.1.smithi069.stdout:7/391: creat d9/d76/d57/d44/d53/d1d/f87 x:0 0 0 2024-03-22T07:16:00.852 INFO:tasks.workunit.client.0.smithi063.stdout:6/598: truncate d3/d15/d16/d60/d6d/d4e/d5a/d88/f8d 523128 0 2024-03-22T07:16:00.855 INFO:tasks.workunit.client.1.smithi069.stdout:7/392: rename d9/d76/d57/d44/d53/d2e/l66 to d9/db/d48/l88 0 2024-03-22T07:16:00.855 INFO:tasks.workunit.client.1.smithi069.stdout:7/393: write d9/d76/d57/d44/d53/f60 [669704,66889] 0 2024-03-22T07:16:00.855 INFO:tasks.workunit.client.1.smithi069.stdout:7/394: chown d9/d76/d57/f59 150 1 2024-03-22T07:16:00.855 INFO:tasks.workunit.client.0.smithi063.stdout:8/732: mknod d0/d1b/cfb 0 2024-03-22T07:16:00.855 INFO:tasks.workunit.client.0.smithi063.stdout:8/733: truncate d0/d1b/d28/d3e/fc2 162337 0 2024-03-22T07:16:00.856 INFO:tasks.workunit.client.0.smithi063.stdout:8/734: creat d0/d26/d2d/d59/ffc x:0 0 0 2024-03-22T07:16:00.856 INFO:tasks.workunit.client.0.smithi063.stdout:6/599: rmdir d3/d15/d16/d60/d6d/d4e/d5f 39 2024-03-22T07:16:00.858 INFO:tasks.workunit.client.1.smithi069.stdout:9/341: dwrite d2/f5 [4194304,4194304] 0 2024-03-22T07:16:00.863 INFO:tasks.workunit.client.1.smithi069.stdout:6/324: dwrite d4/d7/f3e [0,4194304] 0 2024-03-22T07:16:00.863 INFO:tasks.workunit.client.1.smithi069.stdout:6/325: dread - d4/d7/da/d3f/f61 zero size 2024-03-22T07:16:00.864 INFO:tasks.workunit.client.0.smithi063.stdout:0/816: dwrite da/de/d60/f33 [0,4194304] 0 2024-03-22T07:16:00.864 INFO:tasks.workunit.client.0.smithi063.stdout:6/600: mknod d3/cc4 0 2024-03-22T07:16:00.868 INFO:tasks.workunit.client.0.smithi063.stdout:3/666: sync 2024-03-22T07:16:00.868 INFO:tasks.workunit.client.0.smithi063.stdout:3/667: chown da/d16/d89/fb5 87952 1 2024-03-22T07:16:00.868 INFO:tasks.workunit.client.0.smithi063.stdout:3/668: readlink l9 0 2024-03-22T07:16:00.868 INFO:tasks.workunit.client.0.smithi063.stdout:0/817: dread f5 [0,4194304] 0 2024-03-22T07:16:00.870 INFO:tasks.workunit.client.0.smithi063.stdout:3/669: symlink da/d63/d65/d81/lde 0 2024-03-22T07:16:00.872 INFO:tasks.workunit.client.0.smithi063.stdout:6/601: mknod d3/cc5 0 2024-03-22T07:16:00.872 INFO:tasks.workunit.client.0.smithi063.stdout:6/602: chown d3/d6/d2a/d38/da5/f4b 7871 1 2024-03-22T07:16:00.873 INFO:tasks.workunit.client.0.smithi063.stdout:8/735: dread d0/d24/d33/d4c/fd4 [0,4194304] 0 2024-03-22T07:16:00.873 INFO:tasks.workunit.client.0.smithi063.stdout:0/818: rename da/de/d17/d25/d95/fd6 to da/de/d17/d25/d3c/d64/d78/f10a 0 2024-03-22T07:16:00.876 INFO:tasks.workunit.client.0.smithi063.stdout:0/819: creat da/de/d17/d25/d99/dbc/f10b x:0 0 0 2024-03-22T07:16:00.881 INFO:tasks.workunit.client.0.smithi063.stdout:3/670: rename da/l14 to da/d16/ldf 0 2024-03-22T07:16:00.887 INFO:tasks.workunit.client.0.smithi063.stdout:9/557: read d8/d19/d20/d22/d29/d31/d53/d6a/da2/f74 [1829769,18610] 0 2024-03-22T07:16:00.887 INFO:tasks.workunit.client.0.smithi063.stdout:9/558: chown d8/d15 23039472 1 2024-03-22T07:16:00.889 INFO:tasks.workunit.client.0.smithi063.stdout:3/671: dread da/d16/f42 [4194304,4194304] 0 2024-03-22T07:16:00.889 INFO:tasks.workunit.client.0.smithi063.stdout:3/672: stat da/d16/d26/f29 0 2024-03-22T07:16:00.892 INFO:tasks.workunit.client.0.smithi063.stdout:3/673: rename da/d16/d3f/l87 to da/d63/d65/d81/d9b/d9f/dcf/le0 0 2024-03-22T07:16:00.894 INFO:tasks.workunit.client.1.smithi069.stdout:0/415: dwrite d3/d7/fb [0,4194304] 0 2024-03-22T07:16:00.894 INFO:tasks.workunit.client.1.smithi069.stdout:1/357: dwrite d1/d2a/d50/d4c/d55/f56 [0,4194304] 0 2024-03-22T07:16:00.894 INFO:tasks.workunit.client.1.smithi069.stdout:1/358: chown d1/d13/d36/f67 16015 1 2024-03-22T07:16:00.895 INFO:tasks.workunit.client.0.smithi063.stdout:2/580: dwrite d0/de/d12/d20/f2c [0,4194304] 0 2024-03-22T07:16:00.895 INFO:tasks.workunit.client.0.smithi063.stdout:2/581: truncate d0/de/d1b/f48 43247 0 2024-03-22T07:16:00.895 INFO:tasks.workunit.client.0.smithi063.stdout:2/582: fsync d0/de/d12/d20/f2a 0 2024-03-22T07:16:00.895 INFO:tasks.workunit.client.1.smithi069.stdout:6/326: dwrite d4/dd/d12/d2b/f36 [4194304,4194304] 0 2024-03-22T07:16:00.896 INFO:tasks.workunit.client.0.smithi063.stdout:3/674: unlink da/d63/l82 0 2024-03-22T07:16:00.896 INFO:tasks.workunit.client.0.smithi063.stdout:8/736: write d0/d24/d5b/f44 [678401,59584] 0 2024-03-22T07:16:00.899 INFO:tasks.workunit.client.0.smithi063.stdout:5/534: dwrite d8/d13/d28/f43 [0,4194304] 0 2024-03-22T07:16:00.899 INFO:tasks.workunit.client.0.smithi063.stdout:5/535: truncate d8/f12 4723106 0 2024-03-22T07:16:00.899 INFO:tasks.workunit.client.0.smithi063.stdout:5/536: chown d8/d13/d1e/c4d 2 1 2024-03-22T07:16:00.899 INFO:tasks.workunit.client.0.smithi063.stdout:3/675: symlink da/d16/d3f/dc7/le1 0 2024-03-22T07:16:00.902 INFO:tasks.workunit.client.0.smithi063.stdout:5/537: unlink d8/d13/d1e/d31/f52 0 2024-03-22T07:16:00.904 INFO:tasks.workunit.client.0.smithi063.stdout:3/676: read da/d16/d89/d8c/dc1/f93 [85858,96637] 0 2024-03-22T07:16:00.906 INFO:tasks.workunit.client.0.smithi063.stdout:5/538: rmdir d8/d13/d1e/d3c 39 2024-03-22T07:16:00.908 INFO:tasks.workunit.client.1.smithi069.stdout:0/416: rename d3/d7/d24/d6a/l73 to d3/d7/d15/d1f/l87 0 2024-03-22T07:16:00.910 INFO:tasks.workunit.client.0.smithi063.stdout:9/559: dwrite d8/d19/d20/d22/d56/f82 [0,4194304] 0 2024-03-22T07:16:00.910 INFO:tasks.workunit.client.0.smithi063.stdout:9/560: chown d8/d17/l33 3 1 2024-03-22T07:16:00.910 INFO:tasks.workunit.client.0.smithi063.stdout:9/561: fsync d8/d19/d20/d22/d29/d31/f86 0 2024-03-22T07:16:00.910 INFO:tasks.workunit.client.0.smithi063.stdout:9/562: chown d8/d17/f94 0 1 2024-03-22T07:16:00.930 INFO:tasks.workunit.client.1.smithi069.stdout:6/327: dwrite d4/dd/f25 [0,4194304] 0 2024-03-22T07:16:00.934 INFO:tasks.workunit.client.1.smithi069.stdout:1/359: rename d1/d25/f5c to d1/d2a/d50/d4c/f74 0 2024-03-22T07:16:00.934 INFO:tasks.workunit.client.1.smithi069.stdout:0/417: rename d3/d7 to d3/d7/d88 22 2024-03-22T07:16:00.936 INFO:tasks.workunit.client.0.smithi063.stdout:0/820: fsync da/de/d17/d25/d3c/fd3 0 2024-03-22T07:16:00.938 INFO:tasks.workunit.client.1.smithi069.stdout:0/418: symlink d3/d7/d24/l89 0 2024-03-22T07:16:00.939 INFO:tasks.workunit.client.0.smithi063.stdout:0/821: rename da/de/f31 to da/de/d17/d4e/f10c 0 2024-03-22T07:16:00.943 INFO:tasks.workunit.client.1.smithi069.stdout:0/419: creat d3/d7/d5e/f8a x:0 0 0 2024-03-22T07:16:00.944 INFO:tasks.workunit.client.1.smithi069.stdout:0/420: symlink d3/d7/d15/d1f/l8b 0 2024-03-22T07:16:00.945 INFO:tasks.workunit.client.1.smithi069.stdout:0/421: fsync d3/d7/d15/f46 0 2024-03-22T07:16:00.954 INFO:tasks.workunit.client.0.smithi063.stdout:0/822: write da/de/d17/d4e/f9e [836614,121694] 0 2024-03-22T07:16:00.954 INFO:tasks.workunit.client.0.smithi063.stdout:0/823: write da/de/d60/fd2 [554623,67481] 0 2024-03-22T07:16:00.960 INFO:tasks.workunit.client.0.smithi063.stdout:9/563: dwrite d8/d19/d20/d22/d29/d31/f51 [0,4194304] 0 2024-03-22T07:16:00.961 INFO:tasks.workunit.client.0.smithi063.stdout:9/564: fsync d8/d17/f69 0 2024-03-22T07:16:00.961 INFO:tasks.workunit.client.0.smithi063.stdout:3/677: dwrite da/d16/d89/d8c/f9e [0,4194304] 0 2024-03-22T07:16:00.961 INFO:tasks.workunit.client.0.smithi063.stdout:3/678: write da/d16/d89/da9/dd3/fdb [532831,37575] 0 2024-03-22T07:16:00.961 INFO:tasks.workunit.client.0.smithi063.stdout:5/539: dwrite d8/d13/d24/f50 [4194304,4194304] 0 2024-03-22T07:16:00.962 INFO:tasks.workunit.client.0.smithi063.stdout:9/565: symlink d8/d19/d20/d22/lae 0 2024-03-22T07:16:00.963 INFO:tasks.workunit.client.0.smithi063.stdout:2/583: dwrite d0/de/d1b/d32/dac/faf [0,4194304] 0 2024-03-22T07:16:00.967 INFO:tasks.workunit.client.1.smithi069.stdout:1/360: dwrite d1/d13/d36/f68 [0,4194304] 0 2024-03-22T07:16:00.974 INFO:tasks.workunit.client.0.smithi063.stdout:2/584: symlink d0/de/d58/lb8 0 2024-03-22T07:16:00.974 INFO:tasks.workunit.client.1.smithi069.stdout:0/422: dwrite d3/d7/d17/f3f [0,4194304] 0 2024-03-22T07:16:00.974 INFO:tasks.workunit.client.0.smithi063.stdout:5/540: symlink d8/d13/l99 0 2024-03-22T07:16:00.976 INFO:tasks.workunit.client.0.smithi063.stdout:5/541: unlink d8/d13/d28/d66/l7a 0 2024-03-22T07:16:00.976 INFO:tasks.workunit.client.0.smithi063.stdout:5/542: write d8/d13/d28/d57/d5f/f94 [827439,5758] 0 2024-03-22T07:16:00.978 INFO:tasks.workunit.client.0.smithi063.stdout:5/543: rmdir d8/d13/d28/d57 39 2024-03-22T07:16:00.978 INFO:tasks.workunit.client.0.smithi063.stdout:5/544: chown d8/f2b 1669188 1 2024-03-22T07:16:00.978 INFO:tasks.workunit.client.0.smithi063.stdout:5/545: fdatasync d8/f1c 0 2024-03-22T07:16:00.979 INFO:tasks.workunit.client.1.smithi069.stdout:1/361: dread d1/d25/d41/f19 [4194304,4194304] 0 2024-03-22T07:16:00.982 INFO:tasks.workunit.client.1.smithi069.stdout:1/362: rename d1/f18 to d1/d25/f75 0 2024-03-22T07:16:00.985 INFO:tasks.workunit.client.1.smithi069.stdout:1/363: creat d1/d25/d41/d1e/f76 x:0 0 0 2024-03-22T07:16:00.986 INFO:tasks.workunit.client.1.smithi069.stdout:1/364: write d1/d13/f2d [2472371,40507] 0 2024-03-22T07:16:00.986 INFO:tasks.workunit.client.1.smithi069.stdout:1/365: chown d1/d25/d41/d1e/l60 395545 1 2024-03-22T07:16:00.987 INFO:tasks.workunit.client.1.smithi069.stdout:0/423: dread d3/d7/d15/d1f/f26 [0,4194304] 0 2024-03-22T07:16:00.987 INFO:tasks.workunit.client.0.smithi063.stdout:5/546: unlink d8/l2c 0 2024-03-22T07:16:00.993 INFO:tasks.workunit.client.1.smithi069.stdout:0/424: rename d3/d7/lf to d3/d7/d15/l8c 0 2024-03-22T07:16:00.993 INFO:tasks.workunit.client.1.smithi069.stdout:0/425: write d3/d7/d15/d29/f69 [776591,61097] 0 2024-03-22T07:16:00.993 INFO:tasks.workunit.client.1.smithi069.stdout:0/426: truncate d3/f13 4686680 0 2024-03-22T07:16:00.997 INFO:tasks.workunit.client.0.smithi063.stdout:0/824: dwrite da/de/fcf [0,4194304] 0 2024-03-22T07:16:00.998 INFO:tasks.workunit.client.0.smithi063.stdout:0/825: truncate da/de/d60/f33 1640131 0 2024-03-22T07:16:01.000 INFO:tasks.workunit.client.0.smithi063.stdout:0/826: creat da/de/d17/d25/d3c/d43/f10d x:0 0 0 2024-03-22T07:16:01.000 INFO:tasks.workunit.client.0.smithi063.stdout:0/827: fdatasync da/f82 0 2024-03-22T07:16:01.000 INFO:tasks.workunit.client.0.smithi063.stdout:0/828: fdatasync da/de/d17/d25/f65 0 2024-03-22T07:16:01.015 INFO:tasks.workunit.client.0.smithi063.stdout:2/585: dwrite d0/de/d12/d20/f99 [8388608,4194304] 0 2024-03-22T07:16:01.015 INFO:tasks.workunit.client.0.smithi063.stdout:2/586: stat d0/de/d1b/d32/dac 0 2024-03-22T07:16:01.016 INFO:tasks.workunit.client.0.smithi063.stdout:3/679: dwrite da/d16/d89/d8c/dc1/f93 [0,4194304] 0 2024-03-22T07:16:01.016 INFO:tasks.workunit.client.0.smithi063.stdout:3/680: dread - da/d63/f75 zero size 2024-03-22T07:16:01.016 INFO:tasks.workunit.client.0.smithi063.stdout:2/587: truncate d0/de/d12/d20/f88 2541310 0 2024-03-22T07:16:01.016 INFO:tasks.workunit.client.0.smithi063.stdout:2/588: chown d0/de/d12/d20/f99 5 1 2024-03-22T07:16:01.017 INFO:tasks.workunit.client.0.smithi063.stdout:9/566: dwrite d8/fa [0,4194304] 0 2024-03-22T07:16:01.017 INFO:tasks.workunit.client.0.smithi063.stdout:9/567: truncate d8/d19/d20/d22/d29/faa 979339 0 2024-03-22T07:16:01.018 INFO:tasks.workunit.client.1.smithi069.stdout:0/427: dwrite d3/d7/d15/d29/d5a/d84/f3b [0,4194304] 0 2024-03-22T07:16:01.020 INFO:tasks.workunit.client.1.smithi069.stdout:0/428: mkdir d3/d7/d17/d8d 0 2024-03-22T07:16:01.024 INFO:tasks.workunit.client.0.smithi063.stdout:2/589: rename d0/de/d1b/d32/f54 to d0/de/d35/d36/fb9 0 2024-03-22T07:16:01.035 INFO:tasks.workunit.client.1.smithi069.stdout:0/429: write d3/d7/d15/f20 [2612620,85841] 0 2024-03-22T07:16:01.037 INFO:tasks.workunit.client.0.smithi063.stdout:3/681: dwrite da/d16/f97 [4194304,4194304] 0 2024-03-22T07:16:01.038 INFO:tasks.workunit.client.1.smithi069.stdout:0/430: write d3/d7/d15/d29/d5a/d84/f3a [597177,93318] 0 2024-03-22T07:16:01.038 INFO:tasks.workunit.client.1.smithi069.stdout:0/431: dread - d3/d7/f5c zero size 2024-03-22T07:16:01.042 INFO:tasks.workunit.client.0.smithi063.stdout:9/568: dwrite d8/d19/d20/d22/d9b/f9e [0,4194304] 0 2024-03-22T07:16:01.048 INFO:tasks.workunit.client.0.smithi063.stdout:2/590: dwrite d0/de/d35/d36/fb9 [0,4194304] 0 2024-03-22T07:16:01.048 INFO:tasks.workunit.client.0.smithi063.stdout:2/591: dread - d0/de/d1b/fae zero size 2024-03-22T07:16:01.050 INFO:tasks.workunit.client.0.smithi063.stdout:3/682: write da/d63/d65/f77 [8491617,80513] 0 2024-03-22T07:16:01.091 INFO:tasks.workunit.client.0.smithi063.stdout:9/569: dwrite d8/d19/d20/d22/d29/d31/d4b/f4d [0,4194304] 0 2024-03-22T07:16:01.091 INFO:tasks.workunit.client.0.smithi063.stdout:9/570: fdatasync d8/f1a 0 2024-03-22T07:16:01.091 INFO:tasks.workunit.client.0.smithi063.stdout:9/571: readlink d8/l72 0 2024-03-22T07:16:01.093 INFO:tasks.workunit.client.1.smithi069.stdout:0/432: dwrite d3/d7/f1e [0,4194304] 0 2024-03-22T07:16:01.093 INFO:tasks.workunit.client.1.smithi069.stdout:0/433: creat d3/d7/f8e x:0 0 0 2024-03-22T07:16:01.093 INFO:tasks.workunit.client.1.smithi069.stdout:0/434: dread - d3/d7/d17/f83 zero size 2024-03-22T07:16:01.093 INFO:tasks.workunit.client.1.smithi069.stdout:0/435: creat d3/d7/d17/f8f x:0 0 0 2024-03-22T07:16:01.095 INFO:tasks.workunit.client.0.smithi063.stdout:9/572: symlink d8/d19/d20/d22/d56/d8f/laf 0 2024-03-22T07:16:01.095 INFO:tasks.workunit.client.0.smithi063.stdout:9/573: write d8/d19/d20/d22/fa4 [132844,34033] 0 2024-03-22T07:16:01.095 INFO:tasks.workunit.client.0.smithi063.stdout:3/683: dwrite da/d16/d26/f36 [0,4194304] 0 2024-03-22T07:16:01.095 INFO:tasks.workunit.client.0.smithi063.stdout:3/684: chown da/d63/d65/d4b/fb2 987353 1 2024-03-22T07:16:01.095 INFO:tasks.workunit.client.0.smithi063.stdout:3/685: truncate da/d63/d65/d4b/f85 1030806 0 2024-03-22T07:16:01.099 INFO:tasks.workunit.client.1.smithi069.stdout:0/436: mkdir d3/d7/d90 0 2024-03-22T07:16:01.099 INFO:tasks.workunit.client.0.smithi063.stdout:3/686: mkdir da/d16/d3f/de2 0 2024-03-22T07:16:01.100 INFO:tasks.workunit.client.0.smithi063.stdout:3/687: rmdir da/d16/d26/d69 39 2024-03-22T07:16:01.100 INFO:tasks.workunit.client.1.smithi069.stdout:0/437: creat d3/d7/d17/d1c/f91 x:0 0 0 2024-03-22T07:16:01.101 INFO:tasks.workunit.client.1.smithi069.stdout:0/438: write d3/d7/d15/d29/d5a/f7b [528763,94444] 0 2024-03-22T07:16:01.112 INFO:tasks.workunit.client.1.smithi069.stdout:0/439: write d3/d7/d15/d29/d5a/d84/f38 [1259055,39967] 0 2024-03-22T07:16:01.113 INFO:tasks.workunit.client.1.smithi069.stdout:0/440: dread d3/f13 [4194304,4194304] 0 2024-03-22T07:16:01.123 INFO:tasks.workunit.client.1.smithi069.stdout:0/441: write d3/d7/d15/d29/d5a/d84/f11 [6326356,86871] 0 2024-03-22T07:16:01.124 INFO:tasks.workunit.client.1.smithi069.stdout:0/442: symlink d3/d7/d24/d6a/l92 0 2024-03-22T07:16:01.124 INFO:tasks.workunit.client.1.smithi069.stdout:0/443: write d3/d7/d15/d29/d5a/d84/f3a [192298,107855] 0 2024-03-22T07:16:01.124 INFO:tasks.workunit.client.1.smithi069.stdout:0/444: creat d3/d7/d15/d29/d5a/f93 x:0 0 0 2024-03-22T07:16:01.139 INFO:tasks.workunit.client.1.smithi069.stdout:0/445: dwrite d3/d7/d17/f4c [0,4194304] 0 2024-03-22T07:16:01.143 INFO:tasks.workunit.client.1.smithi069.stdout:0/446: write d3/d7/d17/f6b [3788921,64598] 0 2024-03-22T07:16:01.143 INFO:tasks.workunit.client.1.smithi069.stdout:0/447: dread - d3/d7/d17/f85 zero size 2024-03-22T07:16:01.143 INFO:tasks.workunit.client.1.smithi069.stdout:0/448: chown d3/d7/d17/c3d 3 1 2024-03-22T07:16:01.143 INFO:tasks.workunit.client.1.smithi069.stdout:0/449: rmdir d3/d71 39 2024-03-22T07:16:01.144 INFO:tasks.workunit.client.1.smithi069.stdout:0/450: creat d3/d7/d90/f94 x:0 0 0 2024-03-22T07:16:01.144 INFO:tasks.workunit.client.1.smithi069.stdout:0/451: dread - d3/d7/d15/d29/d6e/f70 zero size 2024-03-22T07:16:01.153 INFO:tasks.workunit.client.1.smithi069.stdout:0/452: dread d3/d7/fb [0,4194304] 0 2024-03-22T07:16:01.167 INFO:tasks.workunit.client.0.smithi063.stdout:4/536: sync 2024-03-22T07:16:01.168 INFO:tasks.workunit.client.1.smithi069.stdout:0/453: dread d3/d7/d15/d29/d5a/d84/f60 [0,4194304] 0 2024-03-22T07:16:01.168 INFO:tasks.workunit.client.1.smithi069.stdout:0/454: stat d3/d7/f64 0 2024-03-22T07:16:01.169 INFO:tasks.workunit.client.0.smithi063.stdout:2/592: dwrite d0/de/d1b/f50 [0,4194304] 0 2024-03-22T07:16:01.170 INFO:tasks.workunit.client.1.smithi069.stdout:0/455: link d3/d7/d17/f85 d3/d7/d15/d29/d6e/f95 0 2024-03-22T07:16:01.171 INFO:tasks.workunit.client.0.smithi063.stdout:2/593: symlink d0/de/d1b/lba 0 2024-03-22T07:16:01.174 INFO:tasks.workunit.client.0.smithi063.stdout:2/594: mkdir d0/de/d58/dbb 0 2024-03-22T07:16:01.175 INFO:tasks.workunit.client.0.smithi063.stdout:2/595: write d0/de/d1b/f8c [1618337,68466] 0 2024-03-22T07:16:01.176 INFO:tasks.workunit.client.0.smithi063.stdout:2/596: mknod d0/de/d58/d7f/d97/cbc 0 2024-03-22T07:16:01.176 INFO:tasks.workunit.client.0.smithi063.stdout:2/597: write d0/de/d12/d20/f52 [993299,110922] 0 2024-03-22T07:16:01.176 INFO:tasks.workunit.client.0.smithi063.stdout:9/574: dwrite d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/fa3 [0,4194304] 0 2024-03-22T07:16:01.181 INFO:tasks.workunit.client.0.smithi063.stdout:2/598: mknod d0/cbd 0 2024-03-22T07:16:01.181 INFO:tasks.workunit.client.0.smithi063.stdout:2/599: read d0/de/d35/f6f [8054,15541] 0 2024-03-22T07:16:01.182 INFO:tasks.workunit.client.0.smithi063.stdout:9/575: rmdir d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48 39 2024-03-22T07:16:01.183 INFO:tasks.workunit.client.0.smithi063.stdout:9/576: getdents d8/d19/d39 0 2024-03-22T07:16:01.184 INFO:tasks.workunit.client.0.smithi063.stdout:3/688: dwrite da/d63/d65/d81/d9b/fc9 [0,4194304] 0 2024-03-22T07:16:01.185 INFO:tasks.workunit.client.0.smithi063.stdout:9/577: unlink d8/d17/d1e/l9c 0 2024-03-22T07:16:01.187 INFO:tasks.workunit.client.0.smithi063.stdout:3/689: mknod da/d63/d65/ce3 0 2024-03-22T07:16:01.188 INFO:tasks.workunit.client.0.smithi063.stdout:3/690: stat da/d63/d65/d7b/l7f 0 2024-03-22T07:16:01.189 INFO:tasks.workunit.client.0.smithi063.stdout:9/578: unlink d8/d19/f3a 0 2024-03-22T07:16:01.192 INFO:tasks.workunit.client.1.smithi069.stdout:0/456: dwrite d3/d7/d90/f94 [0,4194304] 0 2024-03-22T07:16:01.192 INFO:tasks.workunit.client.1.smithi069.stdout:0/457: chown d3/d7/d15/d29/d5a/d5d/f7d 8186575 1 2024-03-22T07:16:01.192 INFO:tasks.workunit.client.1.smithi069.stdout:0/458: chown d3/d7/l19 1 1 2024-03-22T07:16:01.192 INFO:tasks.workunit.client.0.smithi063.stdout:3/691: mknod da/d63/d65/ce4 0 2024-03-22T07:16:01.192 INFO:tasks.workunit.client.0.smithi063.stdout:3/692: readlink da/d16/l4a 0 2024-03-22T07:16:01.192 INFO:tasks.workunit.client.0.smithi063.stdout:4/537: dwrite d1/d5/d43/d27/d2c/d3d/d54/f6b [0,4194304] 0 2024-03-22T07:16:01.206 INFO:tasks.workunit.client.0.smithi063.stdout:4/538: write d1/d5/dc/d95/f60 [426525,63831] 0 2024-03-22T07:16:01.211 INFO:tasks.workunit.client.0.smithi063.stdout:3/693: dread da/f71 [0,4194304] 0 2024-03-22T07:16:01.211 INFO:tasks.workunit.client.0.smithi063.stdout:3/694: read da/d16/d3f/f50 [55274,119538] 0 2024-03-22T07:16:01.211 INFO:tasks.workunit.client.0.smithi063.stdout:3/695: creat da/d16/d89/da9/dd3/fe5 x:0 0 0 2024-03-22T07:16:01.213 INFO:tasks.workunit.client.0.smithi063.stdout:3/696: rename da/d16/l7d to da/d63/d65/d81/d9b/d9f/le6 0 2024-03-22T07:16:01.214 INFO:tasks.workunit.client.0.smithi063.stdout:3/697: creat da/d63/d65/d4b/ddd/fe7 x:0 0 0 2024-03-22T07:16:01.214 INFO:tasks.workunit.client.0.smithi063.stdout:3/698: readlink da/d16/d89/d8c/dc1/la3 0 2024-03-22T07:16:01.214 INFO:tasks.workunit.client.0.smithi063.stdout:3/699: read - da/d16/d3f/f6f zero size 2024-03-22T07:16:01.215 INFO:tasks.workunit.client.0.smithi063.stdout:3/700: creat da/d63/d65/d81/fe8 x:0 0 0 2024-03-22T07:16:01.216 INFO:tasks.workunit.client.0.smithi063.stdout:9/579: dwrite d8/d19/d39/f8b [0,4194304] 0 2024-03-22T07:16:01.216 INFO:tasks.workunit.client.0.smithi063.stdout:9/580: chown d8/d19/d20/d22/d29/d31/d4b/f4d 386 1 2024-03-22T07:16:01.219 INFO:tasks.workunit.client.1.smithi069.stdout:0/459: dwrite d3/d7/fb [0,4194304] 0 2024-03-22T07:16:01.219 INFO:tasks.workunit.client.0.smithi063.stdout:3/701: mknod da/ce9 0 2024-03-22T07:16:01.219 INFO:tasks.workunit.client.1.smithi069.stdout:4/423: sync 2024-03-22T07:16:01.219 INFO:tasks.workunit.client.1.smithi069.stdout:5/258: sync 2024-03-22T07:16:01.220 INFO:tasks.workunit.client.0.smithi063.stdout:3/702: rename da/d16/d26/fa7 to da/d16/d89/fea 0 2024-03-22T07:16:01.220 INFO:tasks.workunit.client.0.smithi063.stdout:3/703: write da/d16/d3f/f44 [1886603,9900] 0 2024-03-22T07:16:01.232 INFO:tasks.workunit.client.1.smithi069.stdout:4/424: dread d6/d12/d18/f4f [0,4194304] 0 2024-03-22T07:16:01.232 INFO:tasks.workunit.client.1.smithi069.stdout:4/425: creat d6/d16/f88 x:0 0 0 2024-03-22T07:16:01.235 INFO:tasks.workunit.client.1.smithi069.stdout:4/426: mknod d6/d16/d83/c89 0 2024-03-22T07:16:01.236 INFO:tasks.workunit.client.0.smithi063.stdout:3/704: dread da/d16/d3f/f70 [0,4194304] 0 2024-03-22T07:16:01.236 INFO:tasks.workunit.client.0.smithi063.stdout:3/705: write da/d16/d89/da9/dd3/fe5 [150846,91159] 0 2024-03-22T07:16:01.236 INFO:tasks.workunit.client.0.smithi063.stdout:3/706: dread - da/d63/f75 zero size 2024-03-22T07:16:01.236 INFO:tasks.workunit.client.0.smithi063.stdout:3/707: dread - da/d16/d3f/f6f zero size 2024-03-22T07:16:01.236 INFO:tasks.workunit.client.1.smithi069.stdout:4/427: mknod d6/d16/c8a 0 2024-03-22T07:16:01.237 INFO:tasks.workunit.client.1.smithi069.stdout:4/428: mkdir d6/d12/d18/d8b 0 2024-03-22T07:16:01.239 INFO:tasks.workunit.client.1.smithi069.stdout:4/429: symlink d6/d12/l8c 0 2024-03-22T07:16:01.242 INFO:tasks.workunit.client.1.smithi069.stdout:4/430: mknod d6/d16/d3a/c8d 0 2024-03-22T07:16:01.244 INFO:tasks.workunit.client.1.smithi069.stdout:4/431: creat d6/d16/d83/f8e x:0 0 0 2024-03-22T07:16:01.245 INFO:tasks.workunit.client.0.smithi063.stdout:9/581: dwrite d8/d19/d20/d22/d56/f82 [4194304,4194304] 0 2024-03-22T07:16:01.246 INFO:tasks.workunit.client.1.smithi069.stdout:5/259: dwrite d1/d6/d26/d3e/f4a [0,4194304] 0 2024-03-22T07:16:01.247 INFO:tasks.workunit.client.1.smithi069.stdout:0/460: dwrite d3/d7/fb [4194304,4194304] 0 2024-03-22T07:16:01.249 INFO:tasks.workunit.client.1.smithi069.stdout:4/432: write d6/dc/d23/f5c [3671738,77842] 0 2024-03-22T07:16:01.249 INFO:tasks.workunit.client.1.smithi069.stdout:4/433: dread - d6/dc/d1d/d31/d1f/f7f zero size 2024-03-22T07:16:01.252 INFO:tasks.workunit.client.1.smithi069.stdout:5/260: mknod d1/d7/c57 0 2024-03-22T07:16:01.254 INFO:tasks.workunit.client.1.smithi069.stdout:0/461: mknod d3/d7/d15/d29/c96 0 2024-03-22T07:16:01.257 INFO:tasks.workunit.client.1.smithi069.stdout:5/261: link d1/d6/f1e d1/d7/d8/f58 0 2024-03-22T07:16:01.257 INFO:tasks.workunit.client.0.smithi063.stdout:3/708: dwrite da/d16/d89/d8c/f9e [0,4194304] 0 2024-03-22T07:16:01.257 INFO:tasks.workunit.client.0.smithi063.stdout:3/709: fdatasync da/d16/d3f/f6f 0 2024-03-22T07:16:01.259 INFO:tasks.workunit.client.1.smithi069.stdout:0/462: rename d3/d7/d24/d6a/c74 to d3/d7/d15/d1f/c97 0 2024-03-22T07:16:01.261 INFO:tasks.workunit.client.0.smithi063.stdout:9/582: write d8/d15/d2f/f47 [2079269,118099] 0 2024-03-22T07:16:01.262 INFO:tasks.workunit.client.0.smithi063.stdout:3/710: creat da/d63/d65/feb x:0 0 0 2024-03-22T07:16:01.262 INFO:tasks.workunit.client.1.smithi069.stdout:5/262: rename d1/d6/d26/d3e/f4a to d1/d7/f59 0 2024-03-22T07:16:01.265 INFO:tasks.workunit.client.1.smithi069.stdout:0/463: truncate d3/d7/d15/d29/d5a/d84/f38 4058999 0 2024-03-22T07:16:01.267 INFO:tasks.workunit.client.1.smithi069.stdout:5/263: unlink d1/d7/d8/c47 0 2024-03-22T07:16:01.270 INFO:tasks.workunit.client.1.smithi069.stdout:0/464: rename d3/d71 to d3/d7/d15/d29/d5a/d5d/d98 0 2024-03-22T07:16:01.271 INFO:tasks.workunit.client.0.smithi063.stdout:3/711: dread da/d16/f42 [0,4194304] 0 2024-03-22T07:16:01.272 INFO:tasks.workunit.client.1.smithi069.stdout:0/465: unlink d3/d7/d15/d1f/l8b 0 2024-03-22T07:16:01.273 INFO:tasks.workunit.client.0.smithi063.stdout:3/712: symlink da/d63/d65/d83/dc6/lec 0 2024-03-22T07:16:01.275 INFO:tasks.workunit.client.0.smithi063.stdout:3/713: rename da/d16/c4d to da/d16/d3f/dc7/ced 0 2024-03-22T07:16:01.275 INFO:tasks.workunit.client.0.smithi063.stdout:3/714: creat da/d16/d89/d8c/dc1/fee x:0 0 0 2024-03-22T07:16:01.275 INFO:tasks.workunit.client.0.smithi063.stdout:3/715: chown da/d16/d89/d8c/dc1/fcb 1289507 1 2024-03-22T07:16:01.276 INFO:tasks.workunit.client.0.smithi063.stdout:3/716: fsync da/d16/d3f/f6f 0 2024-03-22T07:16:01.276 INFO:tasks.workunit.client.0.smithi063.stdout:3/717: truncate da/d16/d26/d69/f8f 481301 0 2024-03-22T07:16:01.276 INFO:tasks.workunit.client.0.smithi063.stdout:3/718: stat da/d16/d89/d8c/cb6 0 2024-03-22T07:16:01.276 INFO:tasks.workunit.client.1.smithi069.stdout:5/264: write d1/d6/f1e [4073451,7378] 0 2024-03-22T07:16:01.276 INFO:tasks.workunit.client.1.smithi069.stdout:4/434: dwrite d6/d16/f88 [0,4194304] 0 2024-03-22T07:16:01.278 INFO:tasks.workunit.client.0.smithi063.stdout:3/719: mknod da/d63/d65/d81/d9b/d9f/dcf/cef 0 2024-03-22T07:16:01.279 INFO:tasks.workunit.client.0.smithi063.stdout:3/720: mknod da/d16/d26/d69/cf0 0 2024-03-22T07:16:01.280 INFO:tasks.workunit.client.1.smithi069.stdout:5/265: symlink d1/d7/d8/l5a 0 2024-03-22T07:16:01.280 INFO:tasks.workunit.client.1.smithi069.stdout:5/266: write d1/d28/f4d [531636,103452] 0 2024-03-22T07:16:01.280 INFO:tasks.workunit.client.1.smithi069.stdout:5/267: creat d1/d10/f5b x:0 0 0 2024-03-22T07:16:01.281 INFO:tasks.workunit.client.1.smithi069.stdout:4/435: mkdir d6/dc/d23/d59/d8f 0 2024-03-22T07:16:01.281 INFO:tasks.workunit.client.1.smithi069.stdout:4/436: dread - d6/d12/f73 zero size 2024-03-22T07:16:01.284 INFO:tasks.workunit.client.1.smithi069.stdout:5/268: rename d1/d6/f1e to d1/d7/d8/f5c 0 2024-03-22T07:16:01.284 INFO:tasks.workunit.client.1.smithi069.stdout:5/269: chown d1/d6/d26/d33/f4e 3781 1 2024-03-22T07:16:01.287 INFO:tasks.workunit.client.1.smithi069.stdout:4/437: rename d6/dc/d1d/d31/d4a/d65 to d6/dc/d1d/d31/d1f/d58/d7c/d90 0 2024-03-22T07:16:01.287 INFO:tasks.workunit.client.1.smithi069.stdout:4/438: dread - d6/dc/d1d/d55/f5f zero size 2024-03-22T07:16:01.287 INFO:tasks.workunit.client.1.smithi069.stdout:4/439: dread - d6/dc/d1d/d31/d1f/d3f/d51/f81 zero size 2024-03-22T07:16:01.287 INFO:tasks.workunit.client.1.smithi069.stdout:4/440: creat d6/dc/d1d/d31/d1f/d3f/d51/f91 x:0 0 0 2024-03-22T07:16:01.287 INFO:tasks.workunit.client.1.smithi069.stdout:4/441: truncate d6/d12/d18/d21/f79 601593 0 2024-03-22T07:16:01.293 INFO:tasks.workunit.client.1.smithi069.stdout:5/270: dread d1/d7/d8/f30 [0,4194304] 0 2024-03-22T07:16:01.294 INFO:tasks.workunit.client.0.smithi063.stdout:3/721: dread da/d16/d89/d8c/f9e [0,4194304] 0 2024-03-22T07:16:01.294 INFO:tasks.workunit.client.0.smithi063.stdout:9/583: dwrite d8/d17/f43 [0,4194304] 0 2024-03-22T07:16:01.296 INFO:tasks.workunit.client.1.smithi069.stdout:5/271: mkdir d1/d6/d26/d33/d5d 0 2024-03-22T07:16:01.297 INFO:tasks.workunit.client.0.smithi063.stdout:3/722: symlink da/d16/d3f/de2/lf1 0 2024-03-22T07:16:01.298 INFO:tasks.workunit.client.1.smithi069.stdout:4/442: dread d6/d12/d18/d21/f49 [0,4194304] 0 2024-03-22T07:16:01.299 INFO:tasks.workunit.client.0.smithi063.stdout:3/723: mknod da/d63/d65/d4b/cf2 0 2024-03-22T07:16:01.299 INFO:tasks.workunit.client.0.smithi063.stdout:3/724: stat da/d16/d89/d8c/dc1/fcb 0 2024-03-22T07:16:01.299 INFO:tasks.workunit.client.0.smithi063.stdout:3/725: write da/d63/d65/d4b/f85 [1494512,96827] 0 2024-03-22T07:16:01.304 INFO:tasks.workunit.client.1.smithi069.stdout:5/272: symlink d1/d6/l5e 0 2024-03-22T07:16:01.304 INFO:tasks.workunit.client.1.smithi069.stdout:4/443: dread d6/dc/ff [4194304,4194304] 0 2024-03-22T07:16:01.304 INFO:tasks.workunit.client.1.smithi069.stdout:4/444: creat d6/d4b/f92 x:0 0 0 2024-03-22T07:16:01.304 INFO:tasks.workunit.client.1.smithi069.stdout:4/445: stat d6/d16/d3a/c8d 0 2024-03-22T07:16:01.306 INFO:tasks.workunit.client.1.smithi069.stdout:5/273: creat d1/d7/d8/d43/d55/f5f x:0 0 0 2024-03-22T07:16:01.308 INFO:tasks.workunit.client.0.smithi063.stdout:3/726: dread da/f3d [0,4194304] 0 2024-03-22T07:16:01.308 INFO:tasks.workunit.client.0.smithi063.stdout:3/727: write da/d16/d3f/f6f [991280,53431] 0 2024-03-22T07:16:01.308 INFO:tasks.workunit.client.0.smithi063.stdout:3/728: write da/d16/d3f/f6f [1143221,40813] 0 2024-03-22T07:16:01.311 INFO:tasks.workunit.client.1.smithi069.stdout:4/446: truncate d6/dc/d1d/d31/d1f/f42 2109856 0 2024-03-22T07:16:01.315 INFO:tasks.workunit.client.0.smithi063.stdout:1/746: sync 2024-03-22T07:16:01.315 INFO:tasks.workunit.client.0.smithi063.stdout:9/584: dwrite d8/d19/d20/d22/f4f [0,4194304] 0 2024-03-22T07:16:01.315 INFO:tasks.workunit.client.0.smithi063.stdout:7/486: sync 2024-03-22T07:16:01.317 INFO:tasks.workunit.client.0.smithi063.stdout:1/747: write d4/df/d13/d2b/d3a/f70 [160741,15439] 0 2024-03-22T07:16:01.317 INFO:tasks.workunit.client.0.smithi063.stdout:7/487: read d5/da/d5f/f5c [470373,123378] 0 2024-03-22T07:16:01.318 INFO:tasks.workunit.client.0.smithi063.stdout:9/585: unlink d8/d17/d1e/l2e 0 2024-03-22T07:16:01.319 INFO:tasks.workunit.client.0.smithi063.stdout:9/586: read d8/d17/f69 [766956,87263] 0 2024-03-22T07:16:01.320 INFO:tasks.workunit.client.0.smithi063.stdout:9/587: fsync d8/d19/d20/d22/d29/d31/d37/f59 0 2024-03-22T07:16:01.320 INFO:tasks.workunit.client.0.smithi063.stdout:1/748: rmdir d4/d71 39 2024-03-22T07:16:01.323 INFO:tasks.workunit.client.1.smithi069.stdout:5/274: rename d1/d7/d8/d43/d55 to d1/d10/d60 0 2024-03-22T07:16:01.323 INFO:tasks.workunit.client.1.smithi069.stdout:5/275: truncate d1/f3 1183060 0 2024-03-22T07:16:01.323 INFO:tasks.workunit.client.0.smithi063.stdout:3/729: rmdir da/d63/d65 39 2024-03-22T07:16:01.325 INFO:tasks.workunit.client.0.smithi063.stdout:3/730: mknod da/d63/d65/d81/d9b/d9f/dcf/cf3 0 2024-03-22T07:16:01.329 INFO:tasks.workunit.client.1.smithi069.stdout:4/447: rename d6/dc/d1d/d31/d1f/d3f/f64 to d6/dc/d23/f93 0 2024-03-22T07:16:01.330 INFO:tasks.workunit.client.1.smithi069.stdout:4/448: mknod d6/dc/d1d/d31/d1f/c94 0 2024-03-22T07:16:01.330 INFO:tasks.workunit.client.1.smithi069.stdout:4/449: dread - d6/dc/d1d/d31/d1f/d3f/d51/f91 zero size 2024-03-22T07:16:01.330 INFO:tasks.workunit.client.1.smithi069.stdout:4/450: write d6/dc/d1d/d31/d1f/f7a [137784,796] 0 2024-03-22T07:16:01.330 INFO:tasks.workunit.client.1.smithi069.stdout:4/451: write d6/dc/d1d/d31/d1f/f5b [3984145,39136] 0 2024-03-22T07:16:01.331 INFO:tasks.workunit.client.1.smithi069.stdout:5/276: symlink d1/d6/d26/l61 0 2024-03-22T07:16:01.332 INFO:tasks.workunit.client.1.smithi069.stdout:5/277: write d1/d7/d8/f18 [1757959,76899] 0 2024-03-22T07:16:01.332 INFO:tasks.workunit.client.1.smithi069.stdout:5/278: stat d1/d7/d8/f15 0 2024-03-22T07:16:01.332 INFO:tasks.workunit.client.0.smithi063.stdout:1/749: write d4/d71/dbb/dc0/fea [856708,42664] 0 2024-03-22T07:16:01.332 INFO:tasks.workunit.client.0.smithi063.stdout:1/750: chown d4/df/d13/d2b/d6c/d85/fab 194 1 2024-03-22T07:16:01.333 INFO:tasks.workunit.client.1.smithi069.stdout:4/452: truncate d6/d12/f6f 10375 0 2024-03-22T07:16:01.333 INFO:tasks.workunit.client.1.smithi069.stdout:4/453: read d6/dc/d23/f5c [212925,34351] 0 2024-03-22T07:16:01.334 INFO:tasks.workunit.client.0.smithi063.stdout:1/751: creat d4/d5a/d91/f102 x:0 0 0 2024-03-22T07:16:01.334 INFO:tasks.workunit.client.0.smithi063.stdout:1/752: fdatasync d4/df/f11 0 2024-03-22T07:16:01.335 INFO:tasks.workunit.client.1.smithi069.stdout:5/279: truncate d1/d6/d26/d33/f38 762491 0 2024-03-22T07:16:01.335 INFO:tasks.workunit.client.1.smithi069.stdout:4/454: rmdir d6/d16/d3a 39 2024-03-22T07:16:01.336 INFO:tasks.workunit.client.0.smithi063.stdout:1/753: link d4/d14/f18 d4/df/d13/d2b/f103 0 2024-03-22T07:16:01.338 INFO:tasks.workunit.client.0.smithi063.stdout:1/754: creat d4/d5a/f104 x:0 0 0 2024-03-22T07:16:01.339 INFO:tasks.workunit.client.1.smithi069.stdout:5/280: mknod d1/d6/d26/d3e/c62 0 2024-03-22T07:16:01.339 INFO:tasks.workunit.client.1.smithi069.stdout:5/281: write d1/d7/d8/f15 [530674,31114] 0 2024-03-22T07:16:01.339 INFO:tasks.workunit.client.1.smithi069.stdout:5/282: fsync d1/d28/f4d 0 2024-03-22T07:16:01.339 INFO:tasks.workunit.client.1.smithi069.stdout:5/283: write d1/d28/f41 [22043,7737] 0 2024-03-22T07:16:01.340 INFO:tasks.workunit.client.0.smithi063.stdout:1/755: symlink d4/df/d13/d39/d63/l105 0 2024-03-22T07:16:01.340 INFO:tasks.workunit.client.0.smithi063.stdout:1/756: write d4/d5a/fcd [330165,21850] 0 2024-03-22T07:16:01.342 INFO:tasks.workunit.client.0.smithi063.stdout:1/757: rename d4/d5a/l7a to d4/df/d13/d2b/d86/l106 0 2024-03-22T07:16:01.345 INFO:tasks.workunit.client.1.smithi069.stdout:5/284: dread d1/d28/f4d [0,4194304] 0 2024-03-22T07:16:01.347 INFO:tasks.workunit.client.1.smithi069.stdout:5/285: getdents d1/d7 0 2024-03-22T07:16:01.350 INFO:tasks.workunit.client.1.smithi069.stdout:5/286: creat d1/d6/d21/f63 x:0 0 0 2024-03-22T07:16:01.351 INFO:tasks.workunit.client.0.smithi063.stdout:7/488: dwrite d5/d16/d1b/d21/f2a [0,4194304] 0 2024-03-22T07:16:01.352 INFO:tasks.workunit.client.1.smithi069.stdout:5/287: mkdir d1/d7/d8/d64 0 2024-03-22T07:16:01.353 INFO:tasks.workunit.client.1.smithi069.stdout:5/288: mknod d1/d6/d26/c65 0 2024-03-22T07:16:01.355 INFO:tasks.workunit.client.1.smithi069.stdout:5/289: mknod d1/d7/d8/d43/c66 0 2024-03-22T07:16:01.363 INFO:tasks.workunit.client.0.smithi063.stdout:3/731: dwrite da/d16/d26/fbd [0,4194304] 0 2024-03-22T07:16:01.364 INFO:tasks.workunit.client.0.smithi063.stdout:3/732: creat da/d63/d65/d83/ff4 x:0 0 0 2024-03-22T07:16:01.364 INFO:tasks.workunit.client.0.smithi063.stdout:7/489: dread d5/da/d5f/f8a [0,4194304] 0 2024-03-22T07:16:01.364 INFO:tasks.workunit.client.0.smithi063.stdout:7/490: stat d5/d16/d1b/d21/d62/d73/d1d/d29 0 2024-03-22T07:16:01.364 INFO:tasks.workunit.client.0.smithi063.stdout:7/491: truncate d5/d16/d1b/d21/d62/d73/d1d/f3f 4303529 0 2024-03-22T07:16:01.369 INFO:tasks.workunit.client.0.smithi063.stdout:7/492: dread d5/d16/f63 [0,4194304] 0 2024-03-22T07:16:01.369 INFO:tasks.workunit.client.0.smithi063.stdout:9/588: dwrite d8/d19/d20/d22/d29/d31/f51 [0,4194304] 0 2024-03-22T07:16:01.370 INFO:tasks.workunit.client.0.smithi063.stdout:1/758: dwrite d4/df/d13/d39/fd6 [0,4194304] 0 2024-03-22T07:16:01.371 INFO:tasks.workunit.client.1.smithi069.stdout:4/455: dwrite d6/dc/d1d/d31/d1f/f62 [0,4194304] 0 2024-03-22T07:16:01.371 INFO:tasks.workunit.client.1.smithi069.stdout:4/456: chown d6/d16/f88 0 1 2024-03-22T07:16:01.371 INFO:tasks.workunit.client.1.smithi069.stdout:2/283: sync 2024-03-22T07:16:01.371 INFO:tasks.workunit.client.1.smithi069.stdout:2/284: fsync d18/f3a 0 2024-03-22T07:16:01.372 INFO:tasks.workunit.client.0.smithi063.stdout:9/589: rename d8/d19/d20/d22/d56/d8f to d8/d19/d20/d22/d29/db0 0 2024-03-22T07:16:01.372 INFO:tasks.workunit.client.0.smithi063.stdout:9/590: write d8/d19/d20/d22/d5d/f64 [2000242,93959] 0 2024-03-22T07:16:01.375 INFO:tasks.workunit.client.1.smithi069.stdout:4/457: creat d6/dc/d1d/d31/d1f/d58/d7c/d90/f95 x:0 0 0 2024-03-22T07:16:01.377 INFO:tasks.workunit.client.1.smithi069.stdout:2/285: rename d18/d4d/d34/c52 to d18/d4d/d25/c6a 0 2024-03-22T07:16:01.377 INFO:tasks.workunit.client.1.smithi069.stdout:2/286: chown d18/d4d/d27/d33/d60 406 1 2024-03-22T07:16:01.377 INFO:tasks.workunit.client.0.smithi063.stdout:3/733: write da/d16/f97 [4468259,31109] 0 2024-03-22T07:16:01.378 INFO:tasks.workunit.client.0.smithi063.stdout:1/759: dread d4/df/d13/d2b/f84 [0,4194304] 0 2024-03-22T07:16:01.380 INFO:tasks.workunit.client.0.smithi063.stdout:3/734: creat da/d63/d65/d81/da5/ff5 x:0 0 0 2024-03-22T07:16:01.380 INFO:tasks.workunit.client.0.smithi063.stdout:3/735: read da/d16/d3f/f50 [31944,130724] 0 2024-03-22T07:16:01.380 INFO:tasks.workunit.client.0.smithi063.stdout:3/736: chown da/f3d 53082 1 2024-03-22T07:16:01.380 INFO:tasks.workunit.client.1.smithi069.stdout:4/458: creat d6/dc/d1d/f96 x:0 0 0 2024-03-22T07:16:01.381 INFO:tasks.workunit.client.0.smithi063.stdout:7/493: dread d5/d16/f1e [0,4194304] 0 2024-03-22T07:16:01.382 INFO:tasks.workunit.client.1.smithi069.stdout:4/459: link d6/d12/d18/d21/f79 d6/d4b/f97 0 2024-03-22T07:16:01.382 INFO:tasks.workunit.client.0.smithi063.stdout:1/760: rename d4/df/d13/d39/d63/fda to d4/df/d13/d2b/d6c/d85/f107 0 2024-03-22T07:16:01.383 INFO:tasks.workunit.client.0.smithi063.stdout:3/737: truncate da/d63/d65/f66 206643 0 2024-03-22T07:16:01.384 INFO:tasks.workunit.client.1.smithi069.stdout:4/460: mknod d6/d12/d18/d8b/c98 0 2024-03-22T07:16:01.384 INFO:tasks.workunit.client.0.smithi063.stdout:7/494: unlink d5/d16/l77 0 2024-03-22T07:16:01.386 INFO:tasks.workunit.client.0.smithi063.stdout:1/761: symlink d4/d8b/da7/l108 0 2024-03-22T07:16:01.386 INFO:tasks.workunit.client.1.smithi069.stdout:4/461: rename d6/l54 to d6/d12/d5d/l99 0 2024-03-22T07:16:01.388 INFO:tasks.workunit.client.1.smithi069.stdout:4/462: stat d6/dc/l63 0 2024-03-22T07:16:01.389 INFO:tasks.workunit.client.0.smithi063.stdout:7/495: symlink d5/da/d5f/la2 0 2024-03-22T07:16:01.389 INFO:tasks.workunit.client.0.smithi063.stdout:7/496: chown d5/d16/d1b/d21/f70 62 1 2024-03-22T07:16:01.390 INFO:tasks.workunit.client.1.smithi069.stdout:4/463: mkdir d6/d16/d83/d9a 0 2024-03-22T07:16:01.390 INFO:tasks.workunit.client.1.smithi069.stdout:4/464: chown d6/d16/c84 1632671857 1 2024-03-22T07:16:01.390 INFO:tasks.workunit.client.0.smithi063.stdout:1/762: truncate d4/d71/d88/da1/fd2 1392800 0 2024-03-22T07:16:01.390 INFO:tasks.workunit.client.0.smithi063.stdout:9/591: dwrite d8/d19/d39/f6f [0,4194304] 0 2024-03-22T07:16:01.391 INFO:tasks.workunit.client.0.smithi063.stdout:3/738: symlink da/d63/d65/d92/d96/lf6 0 2024-03-22T07:16:01.392 INFO:tasks.workunit.client.0.smithi063.stdout:3/739: chown da/d16/d26/d69/dae/ld6 842259 1 2024-03-22T07:16:01.392 INFO:tasks.workunit.client.0.smithi063.stdout:7/497: creat d5/da/d5f/d33/fa3 x:0 0 0 2024-03-22T07:16:01.393 INFO:tasks.workunit.client.0.smithi063.stdout:7/498: chown d5/da/d28/f9a 316 1 2024-03-22T07:16:01.396 INFO:tasks.workunit.client.0.smithi063.stdout:9/592: mkdir d8/d15/d63/db1 0 2024-03-22T07:16:01.398 INFO:tasks.workunit.client.0.smithi063.stdout:3/740: creat da/d16/d3f/ff7 x:0 0 0 2024-03-22T07:16:01.400 INFO:tasks.workunit.client.0.smithi063.stdout:7/499: creat d5/d16/d1b/d21/d62/d73/d1d/d4f/fa4 x:0 0 0 2024-03-22T07:16:01.403 INFO:tasks.workunit.client.1.smithi069.stdout:4/465: rename d6/dc/d1d/d31 to d6/d9b 0 2024-03-22T07:16:01.403 INFO:tasks.workunit.client.1.smithi069.stdout:4/466: dread - d6/d4b/f92 zero size 2024-03-22T07:16:01.403 INFO:tasks.workunit.client.1.smithi069.stdout:4/467: chown d6/d16/d83/c89 359592 1 2024-03-22T07:16:01.405 INFO:tasks.workunit.client.0.smithi063.stdout:7/500: rmdir d5/d16/d1b/d21/d62/d73/d1d/d29/d3b 39 2024-03-22T07:16:01.408 INFO:tasks.workunit.client.0.smithi063.stdout:9/593: rmdir d8/d19/d20/d22/d29 39 2024-03-22T07:16:01.410 INFO:tasks.workunit.client.0.smithi063.stdout:3/741: dread da/d16/d26/f76 [0,4194304] 0 2024-03-22T07:16:01.410 INFO:tasks.workunit.client.0.smithi063.stdout:1/763: dwrite d4/df/d13/d2b/d3a/fce [0,4194304] 0 2024-03-22T07:16:01.410 INFO:tasks.workunit.client.0.smithi063.stdout:7/501: creat d5/d16/d43/fa5 x:0 0 0 2024-03-22T07:16:01.412 INFO:tasks.workunit.client.0.smithi063.stdout:9/594: mkdir d8/d19/d20/d22/db2 0 2024-03-22T07:16:01.412 INFO:tasks.workunit.client.0.smithi063.stdout:9/595: chown d8/d19 126 1 2024-03-22T07:16:01.418 INFO:tasks.workunit.client.0.smithi063.stdout:7/502: write d5/d16/f32 [126827,37407] 0 2024-03-22T07:16:01.419 INFO:tasks.workunit.client.0.smithi063.stdout:3/742: symlink da/d16/d26/d69/dae/lf8 0 2024-03-22T07:16:01.422 INFO:tasks.workunit.client.0.smithi063.stdout:9/596: write d8/d19/d20/d22/d29/d31/d53/d6a/da2/f6b [1010537,105660] 0 2024-03-22T07:16:01.422 INFO:tasks.workunit.client.0.smithi063.stdout:7/503: creat d5/d16/d1b/d21/d62/d73/fa6 x:0 0 0 2024-03-22T07:16:01.425 INFO:tasks.workunit.client.0.smithi063.stdout:9/597: unlink d8/d17/d1e/f7e 0 2024-03-22T07:16:01.425 INFO:tasks.workunit.client.0.smithi063.stdout:9/598: chown d8/d19 10614 1 2024-03-22T07:16:01.426 INFO:tasks.workunit.client.0.smithi063.stdout:1/764: rmdir d4/d8b/dfe 0 2024-03-22T07:16:01.427 INFO:tasks.workunit.client.0.smithi063.stdout:1/765: read - d4/df/d13/d2b/d6c/da8/fc5 zero size 2024-03-22T07:16:01.429 INFO:tasks.workunit.client.0.smithi063.stdout:9/599: creat d8/d15/d63/fb3 x:0 0 0 2024-03-22T07:16:01.429 INFO:tasks.workunit.client.0.smithi063.stdout:9/600: dread - d8/d15/d2f/f93 zero size 2024-03-22T07:16:01.431 INFO:tasks.workunit.client.0.smithi063.stdout:6/603: sync 2024-03-22T07:16:01.433 INFO:tasks.workunit.client.0.smithi063.stdout:6/604: creat d3/d15/d16/fc6 x:0 0 0 2024-03-22T07:16:01.433 INFO:tasks.workunit.client.0.smithi063.stdout:6/605: fdatasync d3/d15/f85 0 2024-03-22T07:16:01.437 INFO:tasks.workunit.client.0.smithi063.stdout:6/606: rename d3/d15/d16/d60/d6d/d4c/d8b/fa4 to d3/d6/d35/d3a/fc7 0 2024-03-22T07:16:01.442 INFO:tasks.workunit.client.0.smithi063.stdout:3/743: dwrite da/d16/d26/d69/f6b [0,4194304] 0 2024-03-22T07:16:01.442 INFO:tasks.workunit.client.0.smithi063.stdout:9/601: dread d8/d19/d20/d22/d5d/f64 [0,4194304] 0 2024-03-22T07:16:01.442 INFO:tasks.workunit.client.0.smithi063.stdout:9/602: chown d8/d19/d20/d22/d29/d31/d4b/c81 1541527 1 2024-03-22T07:16:01.445 INFO:tasks.workunit.client.0.smithi063.stdout:9/603: mkdir d8/d19/d20/d22/d29/d31/d4b/db4 0 2024-03-22T07:16:01.447 INFO:tasks.workunit.client.0.smithi063.stdout:9/604: mkdir d8/d19/d39/db5 0 2024-03-22T07:16:01.452 INFO:tasks.workunit.client.0.smithi063.stdout:2/600: sync 2024-03-22T07:16:01.452 INFO:tasks.workunit.client.0.smithi063.stdout:8/737: sync 2024-03-22T07:16:01.452 INFO:tasks.workunit.client.0.smithi063.stdout:5/547: sync 2024-03-22T07:16:01.453 INFO:tasks.workunit.client.0.smithi063.stdout:2/601: readlink d0/l7 0 2024-03-22T07:16:01.453 INFO:tasks.workunit.client.0.smithi063.stdout:2/602: fdatasync d0/de/d12/d20/fb3 0 2024-03-22T07:16:01.455 INFO:tasks.workunit.client.0.smithi063.stdout:8/738: mkdir d0/d1b/d28/d3e/db2/dbd/dfd 0 2024-03-22T07:16:01.455 INFO:tasks.workunit.client.0.smithi063.stdout:8/739: write d0/d26/d2d/d59/ffc [84412,29632] 0 2024-03-22T07:16:01.455 INFO:tasks.workunit.client.0.smithi063.stdout:8/740: fsync d0/d24/d33/d4c/f94 0 2024-03-22T07:16:01.457 INFO:tasks.workunit.client.0.smithi063.stdout:5/548: symlink d8/d13/d28/d57/d5f/d87/l9a 0 2024-03-22T07:16:01.459 INFO:tasks.workunit.client.0.smithi063.stdout:9/605: dread d8/d19/d20/d22/f65 [0,4194304] 0 2024-03-22T07:16:01.459 INFO:tasks.workunit.client.0.smithi063.stdout:9/606: stat d8/d19/d20/d22/f27 0 2024-03-22T07:16:01.461 INFO:tasks.workunit.client.0.smithi063.stdout:1/766: dwrite d4/f32 [0,4194304] 0 2024-03-22T07:16:01.461 INFO:tasks.workunit.client.0.smithi063.stdout:5/549: creat d8/d13/d1e/d3c/f9b x:0 0 0 2024-03-22T07:16:01.463 INFO:tasks.workunit.client.0.smithi063.stdout:6/607: dwrite d3/d6/d2a/f5e [0,4194304] 0 2024-03-22T07:16:01.467 INFO:tasks.workunit.client.0.smithi063.stdout:5/550: symlink d8/d5e/l9c 0 2024-03-22T07:16:01.468 INFO:tasks.workunit.client.0.smithi063.stdout:5/551: write d8/d13/d1e/f7b [452340,89631] 0 2024-03-22T07:16:01.468 INFO:tasks.workunit.client.0.smithi063.stdout:5/552: stat d8/d13/d24/d81 0 2024-03-22T07:16:01.470 INFO:tasks.workunit.client.0.smithi063.stdout:6/608: creat d3/d15/d16/d57/fc8 x:0 0 0 2024-03-22T07:16:01.470 INFO:tasks.workunit.client.0.smithi063.stdout:6/609: getdents d3/d15/da3/dc2 0 2024-03-22T07:16:01.471 INFO:tasks.workunit.client.0.smithi063.stdout:5/553: symlink d8/d13/d28/l9d 0 2024-03-22T07:16:01.471 INFO:tasks.workunit.client.0.smithi063.stdout:5/554: chown d8/d13/f18 346605 1 2024-03-22T07:16:01.473 INFO:tasks.workunit.client.0.smithi063.stdout:5/555: mkdir d8/d13/d1e/d31/d58/d7e/d9e 0 2024-03-22T07:16:01.473 INFO:tasks.workunit.client.0.smithi063.stdout:5/556: write d8/d13/d1e/d31/f4e [1184166,20284] 0 2024-03-22T07:16:01.475 INFO:tasks.workunit.client.0.smithi063.stdout:6/610: dread d3/d15/d16/f25 [0,4194304] 0 2024-03-22T07:16:01.476 INFO:tasks.workunit.client.0.smithi063.stdout:6/611: mknod d3/d15/d16/d69/d8c/cc9 0 2024-03-22T07:16:01.477 INFO:tasks.workunit.client.0.smithi063.stdout:6/612: chown d3/d15/d16/f9a 4 1 2024-03-22T07:16:01.478 INFO:tasks.workunit.client.0.smithi063.stdout:6/613: mknod d3/d6/d35/d3a/cca 0 2024-03-22T07:16:01.496 INFO:tasks.workunit.client.0.smithi063.stdout:2/603: dwrite d0/de/d12/d20/fb3 [0,4194304] 0 2024-03-22T07:16:01.497 INFO:tasks.workunit.client.0.smithi063.stdout:2/604: chown d0/de/d12/d5a/l5f 247130 1 2024-03-22T07:16:01.498 INFO:tasks.workunit.client.0.smithi063.stdout:2/605: mknod d0/de/d58/d7f/d97/cbe 0 2024-03-22T07:16:01.499 INFO:tasks.workunit.client.0.smithi063.stdout:2/606: creat d0/de/d1b/d32/dac/fbf x:0 0 0 2024-03-22T07:16:01.499 INFO:tasks.workunit.client.0.smithi063.stdout:2/607: write d0/f41 [2360812,59481] 0 2024-03-22T07:16:01.500 INFO:tasks.workunit.client.0.smithi063.stdout:8/741: dwrite d0/d26/d2d/d8f/ff1 [0,4194304] 0 2024-03-22T07:16:01.501 INFO:tasks.workunit.client.0.smithi063.stdout:2/608: rename d0/de/d58/c94 to d0/de/d1b/d32/dac/cc0 0 2024-03-22T07:16:01.502 INFO:tasks.workunit.client.0.smithi063.stdout:5/557: dwrite d8/d13/f39 [0,4194304] 0 2024-03-22T07:16:01.502 INFO:tasks.workunit.client.0.smithi063.stdout:1/767: dwrite d4/d14/f21 [0,4194304] 0 2024-03-22T07:16:01.503 INFO:tasks.workunit.client.0.smithi063.stdout:9/607: dwrite d8/d19/d20/d22/d9b/f9e [0,4194304] 0 2024-03-22T07:16:01.503 INFO:tasks.workunit.client.0.smithi063.stdout:9/608: getdents d8/d19/d20/d22/d5d/d76/dab 0 2024-03-22T07:16:01.503 INFO:tasks.workunit.client.0.smithi063.stdout:9/609: stat d8/d19/d20/d22/d56/f71 0 2024-03-22T07:16:01.503 INFO:tasks.workunit.client.0.smithi063.stdout:9/610: stat d8/d19/d20/d22/f65 0 2024-03-22T07:16:01.503 INFO:tasks.workunit.client.0.smithi063.stdout:6/614: dwrite d3/d15/d16/d60/d6d/d4c/f6f [0,4194304] 0 2024-03-22T07:16:01.505 INFO:tasks.workunit.client.0.smithi063.stdout:5/558: creat d8/d13/d28/d7d/f9f x:0 0 0 2024-03-22T07:16:01.509 INFO:tasks.workunit.client.0.smithi063.stdout:9/611: symlink d8/d15/d63/lb6 0 2024-03-22T07:16:01.511 INFO:tasks.workunit.client.0.smithi063.stdout:6/615: creat d3/d15/d16/d60/fcb x:0 0 0 2024-03-22T07:16:01.513 INFO:tasks.workunit.client.0.smithi063.stdout:2/609: dread d0/de/d12/d20/f52 [0,4194304] 0 2024-03-22T07:16:01.513 INFO:tasks.workunit.client.0.smithi063.stdout:6/616: creat d3/d6/d2a/d38/da5/d32/d51/fcc x:0 0 0 2024-03-22T07:16:01.513 INFO:tasks.workunit.client.0.smithi063.stdout:6/617: chown d3/d15/d16/d60/d6d/d4e/d5a 9282137 1 2024-03-22T07:16:01.517 INFO:tasks.workunit.client.0.smithi063.stdout:6/618: mkdir d3/d6/dcd 0 2024-03-22T07:16:01.517 INFO:tasks.workunit.client.0.smithi063.stdout:6/619: readlink d3/d15/d16/d60/d6d/l81 0 2024-03-22T07:16:01.517 INFO:tasks.workunit.client.0.smithi063.stdout:6/620: truncate d3/f14 1750583 0 2024-03-22T07:16:01.517 INFO:tasks.workunit.client.0.smithi063.stdout:6/621: fdatasync d3/d6/d2a/d38/f9f 0 2024-03-22T07:16:01.521 INFO:tasks.workunit.client.0.smithi063.stdout:6/622: truncate d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/f5b 70266 0 2024-03-22T07:16:01.529 INFO:tasks.workunit.client.0.smithi063.stdout:2/610: dread d0/de/d12/d5a/f81 [0,4194304] 0 2024-03-22T07:16:01.529 INFO:tasks.workunit.client.0.smithi063.stdout:2/611: fsync d0/de/d12/d20/f9b 0 2024-03-22T07:16:01.529 INFO:tasks.workunit.client.0.smithi063.stdout:2/612: mkdir d0/de/d1b/d8b/dc1 0 2024-03-22T07:16:01.533 INFO:tasks.workunit.client.0.smithi063.stdout:6/623: dread d3/f9 [0,4194304] 0 2024-03-22T07:16:01.533 INFO:tasks.workunit.client.0.smithi063.stdout:6/624: truncate d3/f4 4396278 0 2024-03-22T07:16:01.534 INFO:tasks.workunit.client.1.smithi069.stdout:1/366: sync 2024-03-22T07:16:01.534 INFO:tasks.workunit.client.1.smithi069.stdout:8/454: sync 2024-03-22T07:16:01.534 INFO:tasks.workunit.client.1.smithi069.stdout:1/367: truncate d1/d2a/d50/d4c/d55/f62 30737 0 2024-03-22T07:16:01.535 INFO:tasks.workunit.client.1.smithi069.stdout:8/455: getdents d1/d1c/d3e 0 2024-03-22T07:16:01.537 INFO:tasks.workunit.client.1.smithi069.stdout:8/456: creat d1/d1c/d3e/d88/f8f x:0 0 0 2024-03-22T07:16:01.539 INFO:tasks.workunit.client.1.smithi069.stdout:6/328: sync 2024-03-22T07:16:01.539 INFO:tasks.workunit.client.1.smithi069.stdout:6/329: write d4/dd/d12/d13/f5e [41714,6066] 0 2024-03-22T07:16:01.539 INFO:tasks.workunit.client.1.smithi069.stdout:7/395: sync 2024-03-22T07:16:01.540 INFO:tasks.workunit.client.1.smithi069.stdout:9/342: sync 2024-03-22T07:16:01.540 INFO:tasks.workunit.client.1.smithi069.stdout:3/394: sync 2024-03-22T07:16:01.540 INFO:tasks.workunit.client.1.smithi069.stdout:3/395: truncate d5/d9/db/f63 715653 0 2024-03-22T07:16:01.540 INFO:tasks.workunit.client.1.smithi069.stdout:9/343: write d2/dd/d3c/f3e [949430,81494] 0 2024-03-22T07:16:01.544 INFO:tasks.workunit.client.1.smithi069.stdout:7/396: link d9/d76/d57/d44/d53/d74/c5a d9/d76/d57/d44/c89 0 2024-03-22T07:16:01.544 INFO:tasks.workunit.client.1.smithi069.stdout:7/397: write d9/d76/d57/d44/d53/d74/d30/f54 [4843069,34821] 0 2024-03-22T07:16:01.545 INFO:tasks.workunit.client.1.smithi069.stdout:3/396: mkdir d5/d9/d26/d3f/d56/d7f 0 2024-03-22T07:16:01.546 INFO:tasks.workunit.client.0.smithi063.stdout:8/742: dwrite d0/d1b/d4f/dde/df5/f5c [0,4194304] 0 2024-03-22T07:16:01.546 INFO:tasks.workunit.client.0.smithi063.stdout:8/743: write d0/f19 [3150088,104291] 0 2024-03-22T07:16:01.546 INFO:tasks.workunit.client.0.smithi063.stdout:8/744: write d0/d1b/d4f/dde/ff8 [311514,1433] 0 2024-03-22T07:16:01.547 INFO:tasks.workunit.client.1.smithi069.stdout:3/397: mknod d5/d9/d23/c80 0 2024-03-22T07:16:01.547 INFO:tasks.workunit.client.1.smithi069.stdout:3/398: write d5/d9/d26/d3f/d35/f6b [909320,79844] 0 2024-03-22T07:16:01.549 INFO:tasks.workunit.client.0.smithi063.stdout:8/745: creat d0/d9e/df0/ffe x:0 0 0 2024-03-22T07:16:01.549 INFO:tasks.workunit.client.1.smithi069.stdout:3/399: unlink d5/d9/l57 0 2024-03-22T07:16:01.549 INFO:tasks.workunit.client.1.smithi069.stdout:3/400: chown d5/f4e 1246618 1 2024-03-22T07:16:01.550 INFO:tasks.workunit.client.0.smithi063.stdout:6/625: write d3/d6/d35/d3a/f40 [2556199,66416] 0 2024-03-22T07:16:01.553 INFO:tasks.workunit.client.0.smithi063.stdout:6/626: rename d3/d6/d35/d3a/fc7 to d3/d6/dcd/fce 0 2024-03-22T07:16:01.554 INFO:tasks.workunit.client.1.smithi069.stdout:9/344: write d2/d31/f40 [6805556,72472] 0 2024-03-22T07:16:01.555 INFO:tasks.workunit.client.1.smithi069.stdout:9/345: symlink d2/d21/l6b 0 2024-03-22T07:16:01.556 INFO:tasks.workunit.client.0.smithi063.stdout:6/627: unlink d3/d15/d16/d60/d6d/d4e/d5a/f84 0 2024-03-22T07:16:01.557 INFO:tasks.workunit.client.1.smithi069.stdout:9/346: mknod d2/dd/d3c/c6c 0 2024-03-22T07:16:01.558 INFO:tasks.workunit.client.1.smithi069.stdout:7/398: write f4 [1059966,96040] 0 2024-03-22T07:16:01.558 INFO:tasks.workunit.client.1.smithi069.stdout:7/399: dread - d9/db/d48/d4b/f77 zero size 2024-03-22T07:16:01.558 INFO:tasks.workunit.client.1.smithi069.stdout:7/400: fsync d9/d76/d57/d44/d53/d2e/f4d 0 2024-03-22T07:16:01.559 INFO:tasks.workunit.client.1.smithi069.stdout:6/330: dread d4/f6 [0,4194304] 0 2024-03-22T07:16:01.559 INFO:tasks.workunit.client.1.smithi069.stdout:9/347: mkdir d2/d6d 0 2024-03-22T07:16:01.559 INFO:tasks.workunit.client.1.smithi069.stdout:9/348: fsync d2/f55 0 2024-03-22T07:16:01.560 INFO:tasks.workunit.client.1.smithi069.stdout:7/401: mkdir d9/d42/d8a 0 2024-03-22T07:16:01.560 INFO:tasks.workunit.client.1.smithi069.stdout:7/402: truncate d9/db/f19 8685959 0 2024-03-22T07:16:01.563 INFO:tasks.workunit.client.0.smithi063.stdout:9/612: dwrite d8/d19/d20/d22/d29/faa [0,4194304] 0 2024-03-22T07:16:01.564 INFO:tasks.workunit.client.1.smithi069.stdout:6/331: symlink d4/l66 0 2024-03-22T07:16:01.566 INFO:tasks.workunit.client.0.smithi063.stdout:9/613: mkdir d8/d19/d20/d22/d29/d31/d4b/db7 0 2024-03-22T07:16:01.566 INFO:tasks.workunit.client.0.smithi063.stdout:9/614: fsync d8/d19/d20/d22/d29/d31/f86 0 2024-03-22T07:16:01.566 INFO:tasks.workunit.client.1.smithi069.stdout:3/401: write d5/d9/db/f30 [3830639,24071] 0 2024-03-22T07:16:01.566 INFO:tasks.workunit.client.1.smithi069.stdout:7/403: rmdir d9/d76/d57/d75/d7a 39 2024-03-22T07:16:01.567 INFO:tasks.workunit.client.0.smithi063.stdout:9/615: mknod d8/d19/d20/d22/d29/d31/d4b/cb8 0 2024-03-22T07:16:01.568 INFO:tasks.workunit.client.0.smithi063.stdout:2/613: dwrite d0/fa [0,4194304] 0 2024-03-22T07:16:01.568 INFO:tasks.workunit.client.1.smithi069.stdout:8/457: dwrite d1/f9 [4194304,4194304] 0 2024-03-22T07:16:01.570 INFO:tasks.workunit.client.1.smithi069.stdout:7/404: mknod d9/d42/c8b 0 2024-03-22T07:16:01.570 INFO:tasks.workunit.client.1.smithi069.stdout:7/405: write d9/db/d48/d4b/f77 [378612,77970] 0 2024-03-22T07:16:01.570 INFO:tasks.workunit.client.1.smithi069.stdout:8/458: rmdir d1/de/d20 39 2024-03-22T07:16:01.570 INFO:tasks.workunit.client.1.smithi069.stdout:8/459: chown d1/d1c/d48/d5e/f79 481 1 2024-03-22T07:16:01.572 INFO:tasks.workunit.client.1.smithi069.stdout:8/460: rename d1/de/d20/c3a to d1/de/c90 0 2024-03-22T07:16:01.574 INFO:tasks.workunit.client.1.smithi069.stdout:3/402: dread d5/d9/d26/d3f/d35/f50 [4194304,4194304] 0 2024-03-22T07:16:01.575 INFO:tasks.workunit.client.0.smithi063.stdout:1/768: dwrite d4/df/d13/d39/f8e [0,4194304] 0 2024-03-22T07:16:01.575 INFO:tasks.workunit.client.0.smithi063.stdout:1/769: write d4/df/d13/d2b/d6c/da6/fe4 [746327,88321] 0 2024-03-22T07:16:01.578 INFO:tasks.workunit.client.0.smithi063.stdout:2/614: dread d0/f53 [0,4194304] 0 2024-03-22T07:16:01.580 INFO:tasks.workunit.client.1.smithi069.stdout:3/403: unlink d5/c14 0 2024-03-22T07:16:01.580 INFO:tasks.workunit.client.1.smithi069.stdout:3/404: write d5/d9/d26/d36/d41/f72 [566837,111120] 0 2024-03-22T07:16:01.582 INFO:tasks.workunit.client.1.smithi069.stdout:3/405: mknod d5/d9/d26/d3f/d56/d7f/c81 0 2024-03-22T07:16:01.583 INFO:tasks.workunit.client.0.smithi063.stdout:1/770: link d4/cb8 d4/df/d13/d2b/d86/c109 0 2024-03-22T07:16:01.583 INFO:tasks.workunit.client.1.smithi069.stdout:6/332: dread d4/dd/d12/d13/f5e [0,4194304] 0 2024-03-22T07:16:01.585 INFO:tasks.workunit.client.1.smithi069.stdout:3/406: creat d5/d9/d26/d3f/f82 x:0 0 0 2024-03-22T07:16:01.585 INFO:tasks.workunit.client.1.smithi069.stdout:3/407: chown d5/d9/d26/d3f/d56/c79 499 1 2024-03-22T07:16:01.587 INFO:tasks.workunit.client.0.smithi063.stdout:1/771: rename d4/df/d13/d39/d63/cae to d4/df/d13/d39/d63/dff/c10a 0 2024-03-22T07:16:01.589 INFO:tasks.workunit.client.1.smithi069.stdout:3/408: mknod d5/d9/d26/d36/d41/c83 0 2024-03-22T07:16:01.589 INFO:tasks.workunit.client.0.smithi063.stdout:1/772: mknod d4/d14/d57/c10b 0 2024-03-22T07:16:01.590 INFO:tasks.workunit.client.0.smithi063.stdout:1/773: truncate d4/df/d13/d2b/f84 1548426 0 2024-03-22T07:16:01.590 INFO:tasks.workunit.client.1.smithi069.stdout:9/349: rmdir d2/dd/d3c 39 2024-03-22T07:16:01.590 INFO:tasks.workunit.client.1.smithi069.stdout:9/350: chown d2/dd/d3c/d2e/f65 0 1 2024-03-22T07:16:01.590 INFO:tasks.workunit.client.1.smithi069.stdout:9/351: chown d2/c29 2690440 1 2024-03-22T07:16:01.593 INFO:tasks.workunit.client.1.smithi069.stdout:9/352: mkdir d2/dd/d6e 0 2024-03-22T07:16:01.597 INFO:tasks.workunit.client.0.smithi063.stdout:8/746: dwrite d0/d1b/d28/fa3 [0,4194304] 0 2024-03-22T07:16:01.601 INFO:tasks.workunit.client.0.smithi063.stdout:2/615: dread d0/de/d35/d36/f3a [0,4194304] 0 2024-03-22T07:16:01.601 INFO:tasks.workunit.client.0.smithi063.stdout:2/616: fdatasync d0/de/d74/f7e 0 2024-03-22T07:16:01.603 INFO:tasks.workunit.client.0.smithi063.stdout:8/747: link d0/d24/d5b/f60 d0/d26/d2d/d59/dba/dc9/fff 0 2024-03-22T07:16:01.603 INFO:tasks.workunit.client.0.smithi063.stdout:8/748: readlink d0/d1b/d4f/dde/df5/la6 0 2024-03-22T07:16:01.604 INFO:tasks.workunit.client.0.smithi063.stdout:8/749: mknod d0/d26/d2d/c100 0 2024-03-22T07:16:01.609 INFO:tasks.workunit.client.0.smithi063.stdout:2/617: write d0/f23 [526613,104030] 0 2024-03-22T07:16:01.609 INFO:tasks.workunit.client.1.smithi069.stdout:9/353: write d2/d51/f57 [59047,69068] 0 2024-03-22T07:16:01.610 INFO:tasks.workunit.client.0.smithi063.stdout:8/750: dread d0/d24/d33/d4c/f7d [0,4194304] 0 2024-03-22T07:16:01.610 INFO:tasks.workunit.client.0.smithi063.stdout:8/751: truncate d0/d26/d2d/d59/ffc 403890 0 2024-03-22T07:16:01.610 INFO:tasks.workunit.client.0.smithi063.stdout:8/752: stat d0/d1b/d28/lb5 0 2024-03-22T07:16:01.610 INFO:tasks.workunit.client.0.smithi063.stdout:8/753: readlink d0/d1b/d4f/dde/df5/l76 0 2024-03-22T07:16:01.611 INFO:tasks.workunit.client.0.smithi063.stdout:6/628: dwrite d3/d6/d35/f67 [0,4194304] 0 2024-03-22T07:16:01.611 INFO:tasks.workunit.client.0.smithi063.stdout:6/629: write d3/d15/d16/d57/fa9 [873874,117801] 0 2024-03-22T07:16:01.611 INFO:tasks.workunit.client.0.smithi063.stdout:6/630: dread - d3/d6/d2a/d38/da5/d32/d51/fcc zero size 2024-03-22T07:16:01.613 INFO:tasks.workunit.client.0.smithi063.stdout:8/754: link d0/d24/f29 d0/d26/d2d/d59/dba/df4/f101 0 2024-03-22T07:16:01.615 INFO:tasks.workunit.client.0.smithi063.stdout:6/631: mknod d3/d6/d2a/d38/da5/d32/d51/ccf 0 2024-03-22T07:16:01.615 INFO:tasks.workunit.client.0.smithi063.stdout:6/632: chown d3/d15/d16/d60/d6d/d4e/d5a/d88/cb0 6 1 2024-03-22T07:16:01.615 INFO:tasks.workunit.client.0.smithi063.stdout:6/633: chown d3/d15/d16/d60/d6d/d4e/fa7 58 1 2024-03-22T07:16:01.616 INFO:tasks.workunit.client.0.smithi063.stdout:6/634: mkdir d3/d15/d16/d60/d7b/dd0 0 2024-03-22T07:16:01.616 INFO:tasks.workunit.client.0.smithi063.stdout:6/635: truncate d3/d15/d16/d60/d6d/d4e/fa7 163856 0 2024-03-22T07:16:01.620 INFO:tasks.workunit.client.1.smithi069.stdout:8/461: dwrite d1/d1c/f47 [0,4194304] 0 2024-03-22T07:16:01.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:01 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:16:01.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:01 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:16:01.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:01 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:16:01.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:01 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi069.gfziwh/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:16:01.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:01 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi069.gfziwh/trash_purge_schedule"}]: dispatch 2024-03-22T07:16:01.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:01 smithi063 ceph-mon[29200]: mgrmap e22: smithi069.gfziwh(active, since 1.02506s) 2024-03-22T07:16:01.624 INFO:tasks.workunit.client.1.smithi069.stdout:8/462: creat d1/de/d6c/d8e/f91 x:0 0 0 2024-03-22T07:16:01.624 INFO:tasks.workunit.client.1.smithi069.stdout:8/463: readlink d1/de/l1e 0 2024-03-22T07:16:01.624 INFO:tasks.workunit.client.1.smithi069.stdout:8/464: write d1/d1c/d48/f4b [1364492,100622] 0 2024-03-22T07:16:01.625 INFO:tasks.workunit.client.1.smithi069.stdout:8/465: dread d1/d1c/d48/f7a [0,4194304] 0 2024-03-22T07:16:01.626 INFO:tasks.workunit.client.1.smithi069.stdout:8/466: creat d1/de/d6c/f92 x:0 0 0 2024-03-22T07:16:01.629 INFO:tasks.workunit.client.1.smithi069.stdout:8/467: mkdir d1/d93 0 2024-03-22T07:16:01.629 INFO:tasks.workunit.client.0.smithi063.stdout:9/616: dwrite d8/d19/d20/d22/d56/f82 [4194304,4194304] 0 2024-03-22T07:16:01.632 INFO:tasks.workunit.client.0.smithi063.stdout:9/617: creat d8/d19/d20/d22/db2/fb9 x:0 0 0 2024-03-22T07:16:01.636 INFO:tasks.workunit.client.1.smithi069.stdout:7/406: dwrite d9/d76/d57/d44/d53/d74/d30/f54 [0,4194304] 0 2024-03-22T07:16:01.636 INFO:tasks.workunit.client.1.smithi069.stdout:7/407: stat d9/d42 0 2024-03-22T07:16:01.636 INFO:tasks.workunit.client.0.smithi063.stdout:1/774: dwrite d4/df/d13/d2b/d6c/da8/fdf [0,4194304] 0 2024-03-22T07:16:01.638 INFO:tasks.workunit.client.1.smithi069.stdout:6/333: dwrite d4/dd/d12/d13/f23 [0,4194304] 0 2024-03-22T07:16:01.639 INFO:tasks.workunit.client.1.smithi069.stdout:3/409: dwrite d5/d9/d23/f2b [0,4194304] 0 2024-03-22T07:16:01.640 INFO:tasks.workunit.client.0.smithi063.stdout:1/775: rmdir d4/df/d13/d2b/d6c/d85 39 2024-03-22T07:16:01.640 INFO:tasks.workunit.client.1.smithi069.stdout:7/408: rename d9/d76/d57/d44/c89 to d9/db/d48/c8c 0 2024-03-22T07:16:01.640 INFO:tasks.workunit.client.1.smithi069.stdout:9/354: dread d2/dd/d3c/f3e [0,4194304] 0 2024-03-22T07:16:01.640 INFO:tasks.workunit.client.1.smithi069.stdout:9/355: chown d2/dd/l2a 236 1 2024-03-22T07:16:01.641 INFO:tasks.workunit.client.0.smithi063.stdout:1/776: creat d4/df/f10c x:0 0 0 2024-03-22T07:16:01.643 INFO:tasks.workunit.client.0.smithi063.stdout:4/539: sync 2024-03-22T07:16:01.643 INFO:tasks.workunit.client.0.smithi063.stdout:1/777: mkdir d4/df/d13/d10d 0 2024-03-22T07:16:01.644 INFO:tasks.workunit.client.0.smithi063.stdout:1/778: write d4/df/d13/d2b/d3a/f70 [1261379,45794] 0 2024-03-22T07:16:01.646 INFO:tasks.workunit.client.0.smithi063.stdout:4/540: unlink d1/d5/d43/d27/d2c/c3a 0 2024-03-22T07:16:01.646 INFO:tasks.workunit.client.0.smithi063.stdout:4/541: chown d1/d5/d43/d27/d2c/d3d/d54/d5f/d79/ca1 15138 1 2024-03-22T07:16:01.647 INFO:tasks.workunit.client.0.smithi063.stdout:4/542: fdatasync d1/d5/d43/d27/d2c/d3d/d54/fba 0 2024-03-22T07:16:01.647 INFO:tasks.workunit.client.0.smithi063.stdout:9/618: dread d8/d19/f23 [0,4194304] 0 2024-03-22T07:16:01.647 INFO:tasks.workunit.client.0.smithi063.stdout:9/619: write d8/f9a [563619,4011] 0 2024-03-22T07:16:01.649 INFO:tasks.workunit.client.0.smithi063.stdout:4/543: creat d1/d5/d3c/d8f/fbc x:0 0 0 2024-03-22T07:16:01.649 INFO:tasks.workunit.client.0.smithi063.stdout:9/620: truncate d8/d19/d20/d22/d29/d31/f4e 1161382 0 2024-03-22T07:16:01.650 INFO:tasks.workunit.client.0.smithi063.stdout:9/621: chown d8/d19/d20/d22/d29/d31/d53/d6a/da2/f6b 208 1 2024-03-22T07:16:01.651 INFO:tasks.workunit.client.0.smithi063.stdout:4/544: creat d1/d5/dc/d95/d59/fbd x:0 0 0 2024-03-22T07:16:01.652 INFO:tasks.workunit.client.1.smithi069.stdout:6/334: link d4/d7/d5b/l5c d4/d7/l67 0 2024-03-22T07:16:01.652 INFO:tasks.workunit.client.0.smithi063.stdout:0/829: sync 2024-03-22T07:16:01.654 INFO:tasks.workunit.client.1.smithi069.stdout:3/410: mknod d5/d33/d7b/c84 0 2024-03-22T07:16:01.654 INFO:tasks.workunit.client.1.smithi069.stdout:3/411: write d5/f5d [5046264,113034] 0 2024-03-22T07:16:01.654 INFO:tasks.workunit.client.1.smithi069.stdout:3/412: readlink d5/l17 0 2024-03-22T07:16:01.654 INFO:tasks.workunit.client.1.smithi069.stdout:3/413: fdatasync d5/d33/f39 0 2024-03-22T07:16:01.655 INFO:tasks.workunit.client.0.smithi063.stdout:6/636: dwrite d3/d15/f23 [0,4194304] 0 2024-03-22T07:16:01.655 INFO:tasks.workunit.client.0.smithi063.stdout:6/637: dread - d3/d6/d35/d3a/fbe zero size 2024-03-22T07:16:01.655 INFO:tasks.workunit.client.0.smithi063.stdout:6/638: chown d3/d15/d16 0 1 2024-03-22T07:16:01.655 INFO:tasks.workunit.client.0.smithi063.stdout:0/830: mkdir da/de/d17/d25/d3c/d5f/d10e 0 2024-03-22T07:16:01.655 INFO:tasks.workunit.client.1.smithi069.stdout:9/356: rename d2/dd/d3c/c36 to d2/d21/c6f 0 2024-03-22T07:16:01.656 INFO:tasks.workunit.client.1.smithi069.stdout:9/357: chown d2/dd/d3c/f3b 56 1 2024-03-22T07:16:01.656 INFO:tasks.workunit.client.0.smithi063.stdout:6/639: rename d3/f96 to d3/d6/d35/fd1 0 2024-03-22T07:16:01.658 INFO:tasks.workunit.client.0.smithi063.stdout:0/831: mknod da/de/d60/d9b/c10f 0 2024-03-22T07:16:01.659 INFO:tasks.workunit.client.1.smithi069.stdout:9/358: unlink d2/dd/f37 0 2024-03-22T07:16:01.659 INFO:tasks.workunit.client.0.smithi063.stdout:6/640: creat d3/d15/d16/d60/d6d/d4e/fd2 x:0 0 0 2024-03-22T07:16:01.661 INFO:tasks.workunit.client.1.smithi069.stdout:6/335: unlink d4/l52 0 2024-03-22T07:16:01.661 INFO:tasks.workunit.client.0.smithi063.stdout:0/832: getdents da/de/d67 0 2024-03-22T07:16:01.661 INFO:tasks.workunit.client.0.smithi063.stdout:0/833: truncate da/de/d17/d25/f68 2615366 0 2024-03-22T07:16:01.662 INFO:tasks.workunit.client.0.smithi063.stdout:6/641: symlink d3/d6/d35/d3a/ld3 0 2024-03-22T07:16:01.662 INFO:tasks.workunit.client.0.smithi063.stdout:6/642: read d3/d15/d16/d60/d6d/d4e/d5a/d88/f8d [30556,112113] 0 2024-03-22T07:16:01.663 INFO:tasks.workunit.client.1.smithi069.stdout:8/468: dwrite d1/d1c/d48/f7a [0,4194304] 0 2024-03-22T07:16:01.663 INFO:tasks.workunit.client.1.smithi069.stdout:8/469: chown d1/d1c/d48/d56/l8a 64517 1 2024-03-22T07:16:01.663 INFO:tasks.workunit.client.0.smithi063.stdout:9/622: write f2 [250438,27249] 0 2024-03-22T07:16:01.664 INFO:tasks.workunit.client.1.smithi069.stdout:7/409: dwrite d9/d76/d57/d44/d53/d2e/f6a [0,4194304] 0 2024-03-22T07:16:01.665 INFO:tasks.workunit.client.0.smithi063.stdout:8/755: dwrite d0/d1b/fc3 [0,4194304] 0 2024-03-22T07:16:01.667 INFO:tasks.workunit.client.0.smithi063.stdout:4/545: write d1/d5/d3c/f52 [2072365,73671] 0 2024-03-22T07:16:01.667 INFO:tasks.workunit.client.1.smithi069.stdout:9/359: mknod d2/dd/d3c/d53/c70 0 2024-03-22T07:16:01.669 INFO:tasks.workunit.client.0.smithi063.stdout:8/756: getdents d0/d26/d2d/d59/dba/df4 0 2024-03-22T07:16:01.669 INFO:tasks.workunit.client.1.smithi069.stdout:6/336: symlink d4/d7/l68 0 2024-03-22T07:16:01.671 INFO:tasks.workunit.client.0.smithi063.stdout:4/546: creat d1/d5/d43/d27/d2c/fbe x:0 0 0 2024-03-22T07:16:01.671 INFO:tasks.workunit.client.0.smithi063.stdout:4/547: write d1/d5/dc/f89 [136245,84473] 0 2024-03-22T07:16:01.672 INFO:tasks.workunit.client.0.smithi063.stdout:4/548: dread - d1/d5/d43/d27/d2c/d4a/d80/fa8 zero size 2024-03-22T07:16:01.673 INFO:tasks.workunit.client.0.smithi063.stdout:1/779: dwrite d4/df/d13/d39/d63/f77 [0,4194304] 0 2024-03-22T07:16:01.673 INFO:tasks.workunit.client.1.smithi069.stdout:8/470: unlink d1/de/f5a 0 2024-03-22T07:16:01.674 INFO:tasks.workunit.client.1.smithi069.stdout:8/471: chown d1/d1c/c21 446025681 1 2024-03-22T07:16:01.675 INFO:tasks.workunit.client.0.smithi063.stdout:8/757: creat d0/d26/d2d/d83/dbc/da1/f102 x:0 0 0 2024-03-22T07:16:01.675 INFO:tasks.workunit.client.0.smithi063.stdout:4/549: mkdir d1/d5/dc/dbf 0 2024-03-22T07:16:01.678 INFO:tasks.workunit.client.1.smithi069.stdout:7/410: creat d9/d76/d57/d44/d53/d74/f8d x:0 0 0 2024-03-22T07:16:01.680 INFO:tasks.workunit.client.0.smithi063.stdout:9/623: write d8/d19/d39/f6f [3772545,57612] 0 2024-03-22T07:16:01.681 INFO:tasks.workunit.client.0.smithi063.stdout:9/624: chown d8/d19/d20/d22/d29/db0/laf 3 1 2024-03-22T07:16:01.681 INFO:tasks.workunit.client.1.smithi069.stdout:9/360: mkdir d2/d71 0 2024-03-22T07:16:01.681 INFO:tasks.workunit.client.1.smithi069.stdout:6/337: symlink d4/dd/d41/l69 0 2024-03-22T07:16:01.684 INFO:tasks.workunit.client.1.smithi069.stdout:8/472: truncate d1/de/d20/f4f 3509628 0 2024-03-22T07:16:01.685 INFO:tasks.workunit.client.0.smithi063.stdout:4/550: mkdir d1/d5/d3c/d8f/dc0 0 2024-03-22T07:16:01.685 INFO:tasks.workunit.client.0.smithi063.stdout:4/551: fsync d1/d5/d43/d27/f4b 0 2024-03-22T07:16:01.686 INFO:tasks.workunit.client.1.smithi069.stdout:9/361: rmdir d2/d6d 0 2024-03-22T07:16:01.686 INFO:tasks.workunit.client.1.smithi069.stdout:9/362: chown d2/d21/f3a 3 1 2024-03-22T07:16:01.687 INFO:tasks.workunit.client.1.smithi069.stdout:9/363: write d2/dd/f10 [3075019,105414] 0 2024-03-22T07:16:01.687 INFO:tasks.workunit.client.1.smithi069.stdout:9/364: write d2/d31/f67 [107302,44975] 0 2024-03-22T07:16:01.688 INFO:tasks.workunit.client.1.smithi069.stdout:8/473: creat d1/de/d6c/d4e/d5b/f94 x:0 0 0 2024-03-22T07:16:01.689 INFO:tasks.workunit.client.1.smithi069.stdout:3/414: rmdir d5 39 2024-03-22T07:16:01.691 INFO:tasks.workunit.client.1.smithi069.stdout:0/466: sync 2024-03-22T07:16:01.691 INFO:tasks.workunit.client.1.smithi069.stdout:5/290: sync 2024-03-22T07:16:01.691 INFO:tasks.workunit.client.1.smithi069.stdout:0/467: write d3/d7/d17/f3f [4369209,32350] 0 2024-03-22T07:16:01.691 INFO:tasks.workunit.client.0.smithi063.stdout:9/625: creat d8/d19/d20/d22/d56/fba x:0 0 0 2024-03-22T07:16:01.692 INFO:tasks.workunit.client.1.smithi069.stdout:7/411: dread f8 [0,4194304] 0 2024-03-22T07:16:01.693 INFO:tasks.workunit.client.1.smithi069.stdout:7/412: write d9/d76/d57/d75/d7a/f80 [980312,61343] 0 2024-03-22T07:16:01.693 INFO:tasks.workunit.client.1.smithi069.stdout:9/365: rename d2/dd/d3c/f20 to d2/d21/f72 0 2024-03-22T07:16:01.696 INFO:tasks.workunit.client.1.smithi069.stdout:8/474: creat d1/d1c/d3e/d66/f95 x:0 0 0 2024-03-22T07:16:01.697 INFO:tasks.workunit.client.1.smithi069.stdout:5/291: mknod d1/d7/d3d/d3f/c67 0 2024-03-22T07:16:01.698 INFO:tasks.workunit.client.0.smithi063.stdout:0/834: dwrite da/de/d17/d25/d3c/d53/d89/ffb [0,4194304] 0 2024-03-22T07:16:01.698 INFO:tasks.workunit.client.0.smithi063.stdout:9/626: creat d8/d19/d2b/fbb x:0 0 0 2024-03-22T07:16:01.699 INFO:tasks.workunit.client.1.smithi069.stdout:9/366: truncate d2/f27 67612 0 2024-03-22T07:16:01.700 INFO:tasks.workunit.client.1.smithi069.stdout:9/367: creat d2/d21/f73 x:0 0 0 2024-03-22T07:16:01.701 INFO:tasks.workunit.client.1.smithi069.stdout:5/292: mknod d1/d10/d60/c68 0 2024-03-22T07:16:01.703 INFO:tasks.workunit.client.1.smithi069.stdout:3/415: dread d5/d9/db/f63 [0,4194304] 0 2024-03-22T07:16:01.703 INFO:tasks.workunit.client.1.smithi069.stdout:3/416: getdents d5/d9/d26/d3f/d54 0 2024-03-22T07:16:01.703 INFO:tasks.workunit.client.0.smithi063.stdout:0/835: rmdir da/de/d17/d25/d3c/d53/d89 39 2024-03-22T07:16:01.703 INFO:tasks.workunit.client.1.smithi069.stdout:5/293: mknod d1/d6/d26/d3e/c69 0 2024-03-22T07:16:01.705 INFO:tasks.workunit.client.0.smithi063.stdout:0/836: mkdir da/de/d17/d25/d3c/d64/d110 0 2024-03-22T07:16:01.706 INFO:tasks.workunit.client.1.smithi069.stdout:3/417: unlink d5/d9/d61/f6f 0 2024-03-22T07:16:01.707 INFO:tasks.workunit.client.1.smithi069.stdout:3/418: write d5/d9/db/f65 [186235,82245] 0 2024-03-22T07:16:01.707 INFO:tasks.workunit.client.1.smithi069.stdout:5/294: mknod d1/d10/d44/d50/c6a 0 2024-03-22T07:16:01.709 INFO:tasks.workunit.client.1.smithi069.stdout:5/295: link d1/d7/d8/d43/c66 d1/d6/d26/d3e/c6b 0 2024-03-22T07:16:01.709 INFO:tasks.workunit.client.1.smithi069.stdout:5/296: fdatasync d1/d7/f40 0 2024-03-22T07:16:01.709 INFO:tasks.workunit.client.1.smithi069.stdout:5/297: fsync d1/f4f 0 2024-03-22T07:16:01.709 INFO:tasks.workunit.client.1.smithi069.stdout:5/298: fdatasync d1/d7/d8/f18 0 2024-03-22T07:16:01.710 INFO:tasks.workunit.client.1.smithi069.stdout:5/299: fsync d1/d7/f40 0 2024-03-22T07:16:01.710 INFO:tasks.workunit.client.1.smithi069.stdout:5/300: write d1/d28/f4d [1205852,66599] 0 2024-03-22T07:16:01.710 INFO:tasks.workunit.client.1.smithi069.stdout:5/301: write d1/d7/d8/f4b [921966,8716] 0 2024-03-22T07:16:01.712 INFO:tasks.workunit.client.1.smithi069.stdout:3/419: dread d5/d9/f5e [0,4194304] 0 2024-03-22T07:16:01.712 INFO:tasks.workunit.client.1.smithi069.stdout:3/420: write d5/f4e [401611,53383] 0 2024-03-22T07:16:01.722 INFO:tasks.workunit.client.1.smithi069.stdout:0/468: dread d3/d7/fb [4194304,4194304] 0 2024-03-22T07:16:01.722 INFO:tasks.workunit.client.0.smithi063.stdout:9/627: write d8/d19/d20/fa0 [5733937,66948] 0 2024-03-22T07:16:01.725 INFO:tasks.workunit.client.0.smithi063.stdout:9/628: mknod d8/d19/d39/cbc 0 2024-03-22T07:16:01.726 INFO:tasks.workunit.client.1.smithi069.stdout:0/469: unlink d3/d7/d15/d29/d5a/d84/l25 0 2024-03-22T07:16:01.727 INFO:tasks.workunit.client.1.smithi069.stdout:0/470: mkdir d3/d7/d15/d29/d5a/d99 0 2024-03-22T07:16:01.728 INFO:tasks.workunit.client.0.smithi063.stdout:9/629: creat d8/d19/d20/d22/d5d/d76/d95/fbd x:0 0 0 2024-03-22T07:16:01.728 INFO:tasks.workunit.client.0.smithi063.stdout:9/630: creat d8/d19/d20/d22/d9b/fbe x:0 0 0 2024-03-22T07:16:01.729 INFO:tasks.workunit.client.1.smithi069.stdout:0/471: mkdir d3/d9a 0 2024-03-22T07:16:01.729 INFO:tasks.workunit.client.1.smithi069.stdout:0/472: chown d3/d7/d15/l8c 264453409 1 2024-03-22T07:16:01.730 INFO:tasks.workunit.client.1.smithi069.stdout:9/368: write d2/d21/f72 [6410235,98081] 0 2024-03-22T07:16:01.730 INFO:tasks.workunit.client.1.smithi069.stdout:9/369: readlink d2/d51/l63 0 2024-03-22T07:16:01.730 INFO:tasks.workunit.client.1.smithi069.stdout:0/473: symlink d3/d7/d15/d29/d5a/d99/l9b 0 2024-03-22T07:16:01.730 INFO:tasks.workunit.client.1.smithi069.stdout:0/474: write d3/d7/d15/d29/d5a/f93 [138380,32208] 0 2024-03-22T07:16:01.731 INFO:tasks.workunit.client.1.smithi069.stdout:0/475: readlink d3/d7/d15/l8c 0 2024-03-22T07:16:01.731 INFO:tasks.workunit.client.1.smithi069.stdout:9/370: truncate d2/d51/f64 90691 0 2024-03-22T07:16:01.731 INFO:tasks.workunit.client.0.smithi063.stdout:1/780: dwrite d4/df/d13/d2b/d6c/da8/fcc [4194304,4194304] 0 2024-03-22T07:16:01.731 INFO:tasks.workunit.client.0.smithi063.stdout:1/781: getdents d4/df/d2c 0 2024-03-22T07:16:01.732 INFO:tasks.workunit.client.0.smithi063.stdout:1/782: chown d4/df/d13/d2b/d6c/d85/d9e/fa5 158370 1 2024-03-22T07:16:01.732 INFO:tasks.workunit.client.0.smithi063.stdout:1/783: chown d4/df/d13/d2b/l2e 9 1 2024-03-22T07:16:01.733 INFO:tasks.workunit.client.1.smithi069.stdout:5/302: dread d1/f9 [4194304,4194304] 0 2024-03-22T07:16:01.735 INFO:tasks.workunit.client.1.smithi069.stdout:0/476: rename d3/d7/d17/l2c to d3/d7/d15/d29/d5a/d5d/l9c 0 2024-03-22T07:16:01.737 INFO:tasks.workunit.client.0.smithi063.stdout:8/758: dwrite d0/d26/f92 [0,4194304] 0 2024-03-22T07:16:01.737 INFO:tasks.workunit.client.0.smithi063.stdout:4/552: dwrite d1/d5/dc/d95/d59/f87 [0,4194304] 0 2024-03-22T07:16:01.737 INFO:tasks.workunit.client.0.smithi063.stdout:4/553: fsync d1/f2 0 2024-03-22T07:16:01.737 INFO:tasks.workunit.client.1.smithi069.stdout:9/371: symlink d2/d71/l74 0 2024-03-22T07:16:01.741 INFO:tasks.workunit.client.1.smithi069.stdout:9/372: mkdir d2/dd/d3c/d53/d75 0 2024-03-22T07:16:01.741 INFO:tasks.workunit.client.1.smithi069.stdout:9/373: chown d2/d51/f57 84021 1 2024-03-22T07:16:01.742 INFO:tasks.workunit.client.1.smithi069.stdout:9/374: mkdir d2/d76 0 2024-03-22T07:16:01.748 INFO:tasks.workunit.client.1.smithi069.stdout:2/287: sync 2024-03-22T07:16:01.748 INFO:tasks.workunit.client.1.smithi069.stdout:4/468: sync 2024-03-22T07:16:01.748 INFO:tasks.workunit.client.1.smithi069.stdout:2/288: chown d18/d4d/d27/c4e 174389 1 2024-03-22T07:16:01.749 INFO:tasks.workunit.client.1.smithi069.stdout:6/338: dwrite d4/dd/d12/d44/d58/f62 [0,4194304] 0 2024-03-22T07:16:01.750 INFO:tasks.workunit.client.1.smithi069.stdout:3/421: dwrite d5/f7 [0,4194304] 0 2024-03-22T07:16:01.750 INFO:tasks.workunit.client.1.smithi069.stdout:3/422: write d5/f5d [6847438,114221] 0 2024-03-22T07:16:01.750 INFO:tasks.workunit.client.0.smithi063.stdout:4/554: write d1/d5/d43/d27/d2c/d3d/d54/f6b [1723590,82358] 0 2024-03-22T07:16:01.750 INFO:tasks.workunit.client.0.smithi063.stdout:4/555: stat d1/d5/dc/d95/f6e 0 2024-03-22T07:16:01.752 INFO:tasks.workunit.client.1.smithi069.stdout:7/413: dwrite d9/db/d48/d4b/f77 [0,4194304] 0 2024-03-22T07:16:01.752 INFO:tasks.workunit.client.1.smithi069.stdout:0/477: dread d3/d7/d15/f46 [0,4194304] 0 2024-03-22T07:16:01.752 INFO:tasks.workunit.client.0.smithi063.stdout:4/556: creat d1/d5/dc/dbf/fc1 x:0 0 0 2024-03-22T07:16:01.752 INFO:tasks.workunit.client.0.smithi063.stdout:4/557: readlink d1/d5/dc/l3b 0 2024-03-22T07:16:01.753 INFO:tasks.workunit.client.1.smithi069.stdout:8/475: dwrite d1/f9 [8388608,4194304] 0 2024-03-22T07:16:01.753 INFO:tasks.workunit.client.1.smithi069.stdout:8/476: getdents d1/d1c/d70 0 2024-03-22T07:16:01.753 INFO:tasks.workunit.client.1.smithi069.stdout:7/414: dread d9/db/d1a/f49 [0,4194304] 0 2024-03-22T07:16:01.753 INFO:tasks.workunit.client.1.smithi069.stdout:7/415: fdatasync d9/db/d1a/f82 0 2024-03-22T07:16:01.754 INFO:tasks.workunit.client.0.smithi063.stdout:0/837: dwrite da/de/d17/d25/d3c/d64/fb2 [4194304,4194304] 0 2024-03-22T07:16:01.755 INFO:tasks.workunit.client.0.smithi063.stdout:0/838: getdents da/de/d17/d25/d95/d98 0 2024-03-22T07:16:01.755 INFO:tasks.workunit.client.0.smithi063.stdout:0/839: chown da/de/d17/d25/d95/d98/db3 494452 1 2024-03-22T07:16:01.755 INFO:tasks.workunit.client.0.smithi063.stdout:0/840: fdatasync da/de/d60/f33 0 2024-03-22T07:16:01.755 INFO:tasks.workunit.client.0.smithi063.stdout:0/841: chown da/de/d60/d56/d8b/fb9 19 1 2024-03-22T07:16:01.756 INFO:tasks.workunit.client.0.smithi063.stdout:0/842: readlink da/de/d17/d25/d3c/d43/d76/lbd 0 2024-03-22T07:16:01.756 INFO:tasks.workunit.client.0.smithi063.stdout:0/843: creat da/de/d17/d25/d95/f111 x:0 0 0 2024-03-22T07:16:01.757 INFO:tasks.workunit.client.0.smithi063.stdout:0/844: chown da/de/d60/d9b/c10f 75 1 2024-03-22T07:16:01.758 INFO:tasks.workunit.client.1.smithi069.stdout:1/368: sync 2024-03-22T07:16:01.758 INFO:tasks.workunit.client.0.smithi063.stdout:0/845: creat da/de/d17/d25/d95/d98/db3/f112 x:0 0 0 2024-03-22T07:16:01.761 INFO:tasks.workunit.client.0.smithi063.stdout:0/846: mkdir da/de/d17/d25/d99/de5/d113 0 2024-03-22T07:16:01.762 INFO:tasks.workunit.client.1.smithi069.stdout:6/339: rmdir d4/dd/d12/d2b/d5f 0 2024-03-22T07:16:01.762 INFO:tasks.workunit.client.1.smithi069.stdout:6/340: truncate d4/dd/d12/d13/f5e 218886 0 2024-03-22T07:16:01.765 INFO:tasks.workunit.client.1.smithi069.stdout:2/289: write d18/d4d/d27/f49 [2467598,67949] 0 2024-03-22T07:16:01.765 INFO:tasks.workunit.client.1.smithi069.stdout:2/290: chown d18/d4d/d3e/f5d 0 1 2024-03-22T07:16:01.765 INFO:tasks.workunit.client.1.smithi069.stdout:2/291: creat d18/d4d/d27/d33/f6b x:0 0 0 2024-03-22T07:16:01.766 INFO:tasks.workunit.client.0.smithi063.stdout:4/558: dread d1/f21 [4194304,4194304] 0 2024-03-22T07:16:01.766 INFO:tasks.workunit.client.0.smithi063.stdout:4/559: truncate d1/d5/d43/d27/d2c/d3d/d54/fa6 58032 0 2024-03-22T07:16:01.766 INFO:tasks.workunit.client.0.smithi063.stdout:4/560: chown d1/d5/d43/d27/d2c/d4a/ca3 140 1 2024-03-22T07:16:01.767 INFO:tasks.workunit.client.1.smithi069.stdout:0/478: mkdir d3/d7/d15/d29/d5a/d5d/d9d 0 2024-03-22T07:16:01.769 INFO:tasks.workunit.client.0.smithi063.stdout:4/561: unlink d1/d5/l48 0 2024-03-22T07:16:01.769 INFO:tasks.workunit.client.0.smithi063.stdout:4/562: readlink d1/d5/dc/d95/l61 0 2024-03-22T07:16:01.769 INFO:tasks.workunit.client.0.smithi063.stdout:4/563: fsync d1/d5/d43/f35 0 2024-03-22T07:16:01.769 INFO:tasks.workunit.client.1.smithi069.stdout:8/477: creat d1/d1c/f96 x:0 0 0 2024-03-22T07:16:01.769 INFO:tasks.workunit.client.1.smithi069.stdout:8/478: readlink d1/d1c/l33 0 2024-03-22T07:16:01.770 INFO:tasks.workunit.client.1.smithi069.stdout:8/479: readlink d1/de/l35 0 2024-03-22T07:16:01.770 INFO:tasks.workunit.client.1.smithi069.stdout:6/341: dread d4/dd/d12/f1d [0,4194304] 0 2024-03-22T07:16:01.770 INFO:tasks.workunit.client.1.smithi069.stdout:7/416: mkdir d9/d8e 0 2024-03-22T07:16:01.771 INFO:tasks.workunit.client.0.smithi063.stdout:4/564: mkdir d1/d5/dc2 0 2024-03-22T07:16:01.771 INFO:tasks.workunit.client.0.smithi063.stdout:4/565: chown d1/f2 23709137 1 2024-03-22T07:16:01.773 INFO:tasks.workunit.client.0.smithi063.stdout:4/566: creat d1/d5/dc/d95/fc3 x:0 0 0 2024-03-22T07:16:01.774 INFO:tasks.workunit.client.0.smithi063.stdout:4/567: symlink d1/d5/dc/dbf/lc4 0 2024-03-22T07:16:01.775 INFO:tasks.workunit.client.1.smithi069.stdout:1/369: mknod d1/d2a/d50/d4c/c77 0 2024-03-22T07:16:01.775 INFO:tasks.workunit.client.1.smithi069.stdout:3/423: dread d5/d9/d26/d3f/d35/f45 [0,4194304] 0 2024-03-22T07:16:01.775 INFO:tasks.workunit.client.1.smithi069.stdout:3/424: chown d5/d33/d7b 12 1 2024-03-22T07:16:01.775 INFO:tasks.workunit.client.1.smithi069.stdout:3/425: write d5/d9/d26/f5a [342543,105129] 0 2024-03-22T07:16:01.778 INFO:tasks.workunit.client.1.smithi069.stdout:1/370: dread d1/d25/d41/d1e/f72 [0,4194304] 0 2024-03-22T07:16:01.778 INFO:tasks.workunit.client.1.smithi069.stdout:1/371: fdatasync d1/d25/d41/f19 0 2024-03-22T07:16:01.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:01 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:16:01.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:01 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:16:01.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:01 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:16:01.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:01 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi069.gfziwh/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:16:01.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:01 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi069.gfziwh/trash_purge_schedule"}]: dispatch 2024-03-22T07:16:01.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:01 smithi069 ceph-mon[36441]: mgrmap e22: smithi069.gfziwh(active, since 1.02506s) 2024-03-22T07:16:01.782 INFO:tasks.workunit.client.1.smithi069.stdout:8/480: link d1/d1c/d48/c6a d1/de/d6c/c97 0 2024-03-22T07:16:01.782 INFO:tasks.workunit.client.1.smithi069.stdout:6/342: readlink d4/l2e 0 2024-03-22T07:16:01.783 INFO:tasks.workunit.client.1.smithi069.stdout:6/343: chown d4/dd/d12/d13/f40 1822125844 1 2024-03-22T07:16:01.784 INFO:tasks.workunit.client.1.smithi069.stdout:3/426: symlink d5/d9/d26/d3f/d54/l85 0 2024-03-22T07:16:01.784 INFO:tasks.workunit.client.1.smithi069.stdout:3/427: chown d5/d9/db/f65 4577 1 2024-03-22T07:16:01.784 INFO:tasks.workunit.client.1.smithi069.stdout:3/428: write d5/d9/db/f7d [587125,83960] 0 2024-03-22T07:16:01.784 INFO:tasks.workunit.client.1.smithi069.stdout:3/429: dread - d5/d9/d23/f6d zero size 2024-03-22T07:16:01.784 INFO:tasks.workunit.client.1.smithi069.stdout:3/430: dread - d5/d9/d26/d36/d41/d51/f7c zero size 2024-03-22T07:16:01.786 INFO:tasks.workunit.client.1.smithi069.stdout:1/372: symlink d1/d13/d36/d6e/l78 0 2024-03-22T07:16:01.787 INFO:tasks.workunit.client.1.smithi069.stdout:8/481: mkdir d1/de/d6c/d4e/d98 0 2024-03-22T07:16:01.787 INFO:tasks.workunit.client.1.smithi069.stdout:8/482: write d1/de/d20/f65 [727174,106234] 0 2024-03-22T07:16:01.787 INFO:tasks.workunit.client.1.smithi069.stdout:8/483: readlink d1/de/d20/l42 0 2024-03-22T07:16:01.790 INFO:tasks.workunit.client.1.smithi069.stdout:1/373: unlink d1/c46 0 2024-03-22T07:16:01.790 INFO:tasks.workunit.client.1.smithi069.stdout:1/374: write d1/d13/f58 [402264,102921] 0 2024-03-22T07:16:01.791 INFO:tasks.workunit.client.1.smithi069.stdout:8/484: symlink d1/d1c/l99 0 2024-03-22T07:16:01.791 INFO:tasks.workunit.client.1.smithi069.stdout:8/485: fdatasync d1/de/d6c/d73/f74 0 2024-03-22T07:16:01.791 INFO:tasks.workunit.client.1.smithi069.stdout:7/417: dread d9/db/f16 [0,4194304] 0 2024-03-22T07:16:01.793 INFO:tasks.workunit.client.1.smithi069.stdout:1/375: truncate d1/d13/f27 2583583 0 2024-03-22T07:16:01.795 INFO:tasks.workunit.client.1.smithi069.stdout:7/418: creat d9/d76/d57/d75/d7a/f8f x:0 0 0 2024-03-22T07:16:01.801 INFO:tasks.workunit.client.1.smithi069.stdout:8/486: dread d1/d1c/d3e/f36 [0,4194304] 0 2024-03-22T07:16:01.801 INFO:tasks.workunit.client.1.smithi069.stdout:8/487: mknod d1/de/d20/c9a 0 2024-03-22T07:16:01.801 INFO:tasks.workunit.client.1.smithi069.stdout:8/488: chown d1/d1c/d48/d56/l8a 1 1 2024-03-22T07:16:01.803 INFO:tasks.workunit.client.1.smithi069.stdout:8/489: mknod d1/de/d6c/c9b 0 2024-03-22T07:16:01.804 INFO:tasks.workunit.client.1.smithi069.stdout:8/490: chown d1/d1c/d3e/d66/l84 422 1 2024-03-22T07:16:01.804 INFO:tasks.workunit.client.1.smithi069.stdout:8/491: dread - d1/d1c/d48/f8c zero size 2024-03-22T07:16:01.804 INFO:tasks.workunit.client.1.smithi069.stdout:8/492: read - d1/d1c/d3e/d88/f8f zero size 2024-03-22T07:16:01.804 INFO:tasks.workunit.client.0.smithi063.stdout:9/631: dwrite d8/d19/d20/d22/d9b/fbe [0,4194304] 0 2024-03-22T07:16:01.807 INFO:tasks.workunit.client.0.smithi063.stdout:9/632: link d8/f9a d8/d15/fbf 0 2024-03-22T07:16:01.810 INFO:tasks.workunit.client.0.smithi063.stdout:9/633: unlink l5 0 2024-03-22T07:16:01.810 INFO:tasks.workunit.client.1.smithi069.stdout:1/376: write d1/d2a/d50/d4c/d63/f70 [3516549,97036] 0 2024-03-22T07:16:01.810 INFO:tasks.workunit.client.1.smithi069.stdout:1/377: readlink d1/l2c 0 2024-03-22T07:16:01.812 INFO:tasks.workunit.client.1.smithi069.stdout:9/375: dwrite d2/dd/d3c/f43 [0,4194304] 0 2024-03-22T07:16:01.812 INFO:tasks.workunit.client.1.smithi069.stdout:9/376: write d2/dd/d3c/d2e/f3f [2348417,53331] 0 2024-03-22T07:16:01.813 INFO:tasks.workunit.client.0.smithi063.stdout:9/634: dread d8/d15/fbf [0,4194304] 0 2024-03-22T07:16:01.813 INFO:tasks.workunit.client.0.smithi063.stdout:9/635: chown d8/d15/d2f 1447007 1 2024-03-22T07:16:01.813 INFO:tasks.workunit.client.0.smithi063.stdout:9/636: stat d8/d19/l49 0 2024-03-22T07:16:01.813 INFO:tasks.workunit.client.0.smithi063.stdout:9/637: chown d8/d19/d39/f8b 30494 1 2024-03-22T07:16:01.813 INFO:tasks.workunit.client.0.smithi063.stdout:9/638: symlink d8/d15/d2f/lc0 0 2024-03-22T07:16:01.814 INFO:tasks.workunit.client.0.smithi063.stdout:1/784: dwrite d4/d14/f1b [0,4194304] 0 2024-03-22T07:16:01.814 INFO:tasks.workunit.client.0.smithi063.stdout:8/759: dwrite d0/f89 [4194304,4194304] 0 2024-03-22T07:16:01.815 INFO:tasks.workunit.client.0.smithi063.stdout:9/639: link d8/d19/c84 d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/da5/cc1 0 2024-03-22T07:16:01.817 INFO:tasks.workunit.client.1.smithi069.stdout:9/377: unlink d2/d51/c5a 0 2024-03-22T07:16:01.819 INFO:tasks.workunit.client.0.smithi063.stdout:1/785: creat d4/d71/dbb/dc0/dd8/f10e x:0 0 0 2024-03-22T07:16:01.819 INFO:tasks.workunit.client.0.smithi063.stdout:1/786: chown d4/df/d13/d39/l45 2079362 1 2024-03-22T07:16:01.819 INFO:tasks.workunit.client.0.smithi063.stdout:1/787: write d4/df/d13/d39/d63/f77 [4700814,64806] 0 2024-03-22T07:16:01.819 INFO:tasks.workunit.client.0.smithi063.stdout:1/788: dread - d4/d71/fec zero size 2024-03-22T07:16:01.819 INFO:tasks.workunit.client.0.smithi063.stdout:1/789: truncate d4/d14/f1b 4746641 0 2024-03-22T07:16:01.820 INFO:tasks.workunit.client.1.smithi069.stdout:9/378: write d2/d51/f5e [887649,64052] 0 2024-03-22T07:16:01.820 INFO:tasks.workunit.client.1.smithi069.stdout:9/379: dread - d2/dd/f69 zero size 2024-03-22T07:16:01.820 INFO:tasks.workunit.client.1.smithi069.stdout:9/380: truncate d2/dd/d3c/d48/f5d 1032808 0 2024-03-22T07:16:01.820 INFO:tasks.workunit.client.1.smithi069.stdout:9/381: mkdir d2/d21/d77 0 2024-03-22T07:16:01.821 INFO:tasks.workunit.client.0.smithi063.stdout:8/760: rename d0/d26/d2d/d59/ccf to d0/d26/d2d/d59/dba/df4/c103 0 2024-03-22T07:16:01.821 INFO:tasks.workunit.client.0.smithi063.stdout:8/761: creat d0/d26/d2d/d59/dc0/f104 x:0 0 0 2024-03-22T07:16:01.823 INFO:tasks.workunit.client.0.smithi063.stdout:9/640: link d8/d19/d2b/c2c d8/d19/d20/d22/d29/d31/d53/d6a/cc2 0 2024-03-22T07:16:01.823 INFO:tasks.workunit.client.1.smithi069.stdout:1/378: write d1/d2a/f6c [3430544,6520] 0 2024-03-22T07:16:01.823 INFO:tasks.workunit.client.1.smithi069.stdout:1/379: write d1/d13/f39 [4261045,51346] 0 2024-03-22T07:16:01.823 INFO:tasks.workunit.client.1.smithi069.stdout:1/380: write d1/d13/d36/d5b/f71 [398774,12225] 0 2024-03-22T07:16:01.824 INFO:tasks.workunit.client.0.smithi063.stdout:1/790: unlink d4/d14/caf 0 2024-03-22T07:16:01.824 INFO:tasks.workunit.client.0.smithi063.stdout:1/791: chown d4/d8b/da7/cad 163 1 2024-03-22T07:16:01.824 INFO:tasks.workunit.client.0.smithi063.stdout:1/792: readlink d4/df/d13/d2b/l98 0 2024-03-22T07:16:01.824 INFO:tasks.workunit.client.0.smithi063.stdout:1/793: write d4/df/f64 [813093,41162] 0 2024-03-22T07:16:01.825 INFO:tasks.workunit.client.0.smithi063.stdout:8/762: mknod d0/d1b/d28/d7b/d45/c105 0 2024-03-22T07:16:01.826 INFO:tasks.workunit.client.0.smithi063.stdout:9/641: rename d8/d15/d63/db1 to d8/d19/d20/d22/d5d/d76/dc3 0 2024-03-22T07:16:01.829 INFO:tasks.workunit.client.0.smithi063.stdout:9/642: rename d8/f13 to d8/d19/d20/d22/d29/d31/d53/d6a/fc4 0 2024-03-22T07:16:01.831 INFO:tasks.workunit.client.0.smithi063.stdout:1/794: rename d4/df/d13/d39/d6f/lba to d4/df/d2c/l10f 0 2024-03-22T07:16:01.834 INFO:tasks.workunit.client.1.smithi069.stdout:4/469: dwrite f5 [8388608,4194304] 0 2024-03-22T07:16:01.835 INFO:tasks.workunit.client.1.smithi069.stdout:3/431: dwrite d5/d9/db/f65 [0,4194304] 0 2024-03-22T07:16:01.835 INFO:tasks.workunit.client.1.smithi069.stdout:6/344: dwrite d4/dd/d12/d13/f5e [0,4194304] 0 2024-03-22T07:16:01.835 INFO:tasks.workunit.client.1.smithi069.stdout:6/345: chown d4/dd/d12/d13/f5e 11651404 1 2024-03-22T07:16:01.838 INFO:tasks.workunit.client.1.smithi069.stdout:2/292: dwrite d18/d4d/d27/d33/d60/f5a [0,4194304] 0 2024-03-22T07:16:01.838 INFO:tasks.workunit.client.0.smithi063.stdout:4/568: dwrite d1/d5/d43/f35 [0,4194304] 0 2024-03-22T07:16:01.840 INFO:tasks.workunit.client.1.smithi069.stdout:7/419: dwrite d9/d76/d57/d75/d7a/f80 [0,4194304] 0 2024-03-22T07:16:01.840 INFO:tasks.workunit.client.1.smithi069.stdout:1/381: dread d1/d13/d36/f3e [0,4194304] 0 2024-03-22T07:16:01.840 INFO:tasks.workunit.client.1.smithi069.stdout:9/382: write d2/f5 [1447111,68033] 0 2024-03-22T07:16:01.841 INFO:tasks.workunit.client.1.smithi069.stdout:4/470: link d6/d12/d18/d21/f79 d6/dc/d1d/d55/f9c 0 2024-03-22T07:16:01.844 INFO:tasks.workunit.client.0.smithi063.stdout:0/847: dwrite da/de/d17/d25/d3c/d43/d80/fd8 [0,4194304] 0 2024-03-22T07:16:01.844 INFO:tasks.workunit.client.0.smithi063.stdout:0/848: write da/de/d60/d56/d77/fe9 [1011372,20220] 0 2024-03-22T07:16:01.846 INFO:tasks.workunit.client.1.smithi069.stdout:6/346: unlink d4/d7/ce 0 2024-03-22T07:16:01.846 INFO:tasks.workunit.client.1.smithi069.stdout:0/479: dwrite d3/d7/d15/d29/d5a/d5d/f7d [0,4194304] 0 2024-03-22T07:16:01.850 INFO:tasks.workunit.client.0.smithi063.stdout:0/849: mkdir da/de/d17/d25/d3c/d114 0 2024-03-22T07:16:01.861 INFO:tasks.workunit.client.0.smithi063.stdout:0/850: dread da/de/d17/d4e/f9e [0,4194304] 0 2024-03-22T07:16:01.862 INFO:tasks.workunit.client.0.smithi063.stdout:0/851: mkdir da/de/d60/d9b/d115 0 2024-03-22T07:16:01.862 INFO:tasks.workunit.client.0.smithi063.stdout:0/852: readlink da/de/d17/d25/d3c/d43/d76/lc0 0 2024-03-22T07:16:01.863 INFO:tasks.workunit.client.0.smithi063.stdout:0/853: mkdir da/de/d17/d25/d3c/d43/d116 0 2024-03-22T07:16:01.865 INFO:tasks.workunit.client.1.smithi069.stdout:6/347: dread d4/dd/f33 [0,4194304] 0 2024-03-22T07:16:01.865 INFO:tasks.workunit.client.0.smithi063.stdout:0/854: creat da/de/d17/d25/d3c/d64/d110/f117 x:0 0 0 2024-03-22T07:16:01.867 INFO:tasks.workunit.client.0.smithi063.stdout:0/855: creat da/de/d17/d25/d3c/d53/f118 x:0 0 0 2024-03-22T07:16:01.868 INFO:tasks.workunit.client.1.smithi069.stdout:1/382: dread d1/d13/d36/f43 [0,4194304] 0 2024-03-22T07:16:01.868 INFO:tasks.workunit.client.1.smithi069.stdout:1/383: creat d1/d2a/d50/d4c/d55/f79 x:0 0 0 2024-03-22T07:16:01.879 INFO:tasks.workunit.client.1.smithi069.stdout:2/293: mkdir d18/d4d/d27/d33/d60/d64/d6c 0 2024-03-22T07:16:01.879 INFO:tasks.workunit.client.0.smithi063.stdout:0/856: dread da/de/d17/d25/d3c/d5f/f4b [0,4194304] 0 2024-03-22T07:16:01.888 INFO:tasks.workunit.client.0.smithi063.stdout:4/569: dwrite d1/d5/dc/d95/f49 [0,4194304] 0 2024-03-22T07:16:01.888 INFO:tasks.workunit.client.0.smithi063.stdout:1/795: dwrite d4/d14/f94 [0,4194304] 0 2024-03-22T07:16:01.889 INFO:tasks.workunit.client.0.smithi063.stdout:1/796: chown d4/f12 1974 1 2024-03-22T07:16:01.890 INFO:tasks.workunit.client.1.smithi069.stdout:4/471: dwrite d6/d9b/d1f/f7a [0,4194304] 0 2024-03-22T07:16:01.890 INFO:tasks.workunit.client.1.smithi069.stdout:4/472: write d6/d12/f76 [766551,106694] 0 2024-03-22T07:16:01.890 INFO:tasks.workunit.client.1.smithi069.stdout:4/473: chown d6/d12/d5d/l99 26586 1 2024-03-22T07:16:01.890 INFO:tasks.workunit.client.1.smithi069.stdout:4/474: dread d6/dc/d23/f77 [0,4194304] 0 2024-03-22T07:16:01.890 INFO:tasks.workunit.client.1.smithi069.stdout:4/475: chown d6/dc/d23/d59/l67 975513 1 2024-03-22T07:16:01.892 INFO:tasks.workunit.client.0.smithi063.stdout:0/857: dread da/de/d17/d25/d3c/d53/d89/ffb [0,4194304] 0 2024-03-22T07:16:01.892 INFO:tasks.workunit.client.0.smithi063.stdout:9/643: dwrite d8/d17/f94 [0,4194304] 0 2024-03-22T07:16:01.892 INFO:tasks.workunit.client.0.smithi063.stdout:0/858: write da/f2b [909819,96406] 0 2024-03-22T07:16:01.892 INFO:tasks.workunit.client.0.smithi063.stdout:0/859: read - da/de/d60/ffa zero size 2024-03-22T07:16:01.893 INFO:tasks.workunit.client.0.smithi063.stdout:4/570: dread d1/d5/d43/d27/f4b [0,4194304] 0 2024-03-22T07:16:01.895 INFO:tasks.workunit.client.0.smithi063.stdout:7/504: sync 2024-03-22T07:16:01.895 INFO:tasks.workunit.client.0.smithi063.stdout:3/744: sync 2024-03-22T07:16:01.895 INFO:tasks.workunit.client.0.smithi063.stdout:5/559: sync 2024-03-22T07:16:01.895 INFO:tasks.workunit.client.0.smithi063.stdout:3/745: fdatasync da/d16/f97 0 2024-03-22T07:16:01.895 INFO:tasks.workunit.client.0.smithi063.stdout:3/746: fsync da/d16/d3f/ff7 0 2024-03-22T07:16:01.901 INFO:tasks.workunit.client.0.smithi063.stdout:1/797: dread d4/d71/dbb/dc0/fea [0,4194304] 0 2024-03-22T07:16:01.905 INFO:tasks.workunit.client.0.smithi063.stdout:0/860: dread da/de/d60/d56/d8b/fb9 [0,4194304] 0 2024-03-22T07:16:01.906 INFO:tasks.workunit.client.0.smithi063.stdout:9/644: link d8/d17/f43 d8/d19/d20/d22/d29/d31/d37/fc5 0 2024-03-22T07:16:01.906 INFO:tasks.workunit.client.0.smithi063.stdout:9/645: chown d8/d19/d2b/f5f 50157442 1 2024-03-22T07:16:01.911 INFO:tasks.workunit.client.0.smithi063.stdout:7/505: rename d5/d16/d1b/d21/d62/d73/d1d/d29/d3b to d5/da/d28/da7 0 2024-03-22T07:16:01.911 INFO:tasks.workunit.client.0.smithi063.stdout:7/506: fdatasync d5/da/d5f/f97 0 2024-03-22T07:16:01.911 INFO:tasks.workunit.client.0.smithi063.stdout:7/507: chown d5/d16/d1b/d21/f86 24001945 1 2024-03-22T07:16:01.911 INFO:tasks.workunit.client.0.smithi063.stdout:7/508: chown d5/d16/d43/d78/l92 1 1 2024-03-22T07:16:01.913 INFO:tasks.workunit.client.1.smithi069.stdout:0/480: symlink d3/d7/d15/d29/d5a/d5d/l9e 0 2024-03-22T07:16:01.914 INFO:tasks.workunit.client.0.smithi063.stdout:1/798: mknod d4/d71/d88/da1/c110 0 2024-03-22T07:16:01.917 INFO:tasks.workunit.client.1.smithi069.stdout:6/348: mknod d4/dd/d12/d44/c6a 0 2024-03-22T07:16:01.920 INFO:tasks.workunit.client.0.smithi063.stdout:9/646: dread d8/d19/d20/d22/d29/d31/f51 [0,4194304] 0 2024-03-22T07:16:01.921 INFO:tasks.workunit.client.0.smithi063.stdout:0/861: link da/de/c1c da/de/d17/d25/d3c/d53/d89/df1/c119 0 2024-03-22T07:16:01.923 INFO:tasks.workunit.client.0.smithi063.stdout:1/799: getdents d4/df/d13/d2b/d6c/d85/d9e 0 2024-03-22T07:16:01.925 INFO:tasks.workunit.client.1.smithi069.stdout:8/493: rmdir d1/d1c 39 2024-03-22T07:16:01.928 INFO:tasks.workunit.client.0.smithi063.stdout:0/862: rmdir da/de/d60/d9b/d115 0 2024-03-22T07:16:01.929 INFO:tasks.workunit.client.0.smithi063.stdout:2/618: sync 2024-03-22T07:16:01.929 INFO:tasks.workunit.client.0.smithi063.stdout:6/643: sync 2024-03-22T07:16:01.929 INFO:tasks.workunit.client.0.smithi063.stdout:6/644: write d3/d15/f17 [26342,56519] 0 2024-03-22T07:16:01.934 INFO:tasks.workunit.client.0.smithi063.stdout:0/863: link da/de/d17/d25/f70 da/de/d17/d25/d3c/d53/d89/df1/f11a 0 2024-03-22T07:16:01.934 INFO:tasks.workunit.client.0.smithi063.stdout:9/647: write d8/d15/f2a [1497057,119210] 0 2024-03-22T07:16:01.936 INFO:tasks.workunit.client.0.smithi063.stdout:3/747: dwrite da/f6a [0,4194304] 0 2024-03-22T07:16:01.937 INFO:tasks.workunit.client.1.smithi069.stdout:8/494: dread d1/de/f1a [0,4194304] 0 2024-03-22T07:16:01.937 INFO:tasks.workunit.client.1.smithi069.stdout:8/495: truncate d1/de/d20/f65 1565528 0 2024-03-22T07:16:01.937 INFO:tasks.workunit.client.1.smithi069.stdout:8/496: read d1/de/f1a [1878347,3231] 0 2024-03-22T07:16:01.938 INFO:tasks.workunit.client.0.smithi063.stdout:4/571: dwrite d1/d5/d43/d27/d2c/d4a/f9d [0,4194304] 0 2024-03-22T07:16:01.938 INFO:tasks.workunit.client.0.smithi063.stdout:6/645: mkdir d3/d15/d16/d60/d6d/d4e/d5a/dd4 0 2024-03-22T07:16:01.940 INFO:tasks.workunit.client.0.smithi063.stdout:1/800: rmdir d4/d8b/da7 39 2024-03-22T07:16:01.949 INFO:tasks.workunit.client.0.smithi063.stdout:2/619: mkdir d0/de/d1b/d32/dac/dc2 0 2024-03-22T07:16:01.949 INFO:tasks.workunit.client.0.smithi063.stdout:7/509: dwrite d5/da/f35 [0,4194304] 0 2024-03-22T07:16:01.949 INFO:tasks.workunit.client.0.smithi063.stdout:7/510: chown d5/da/f25 461 1 2024-03-22T07:16:01.954 INFO:tasks.workunit.client.0.smithi063.stdout:5/560: dwrite d8/d13/d1e/d31/d64/d65/f7c [0,4194304] 0 2024-03-22T07:16:01.955 INFO:tasks.workunit.client.0.smithi063.stdout:4/572: dread d1/d5/d43/d27/d2c/d4a/f4d [0,4194304] 0 2024-03-22T07:16:01.956 INFO:tasks.workunit.client.0.smithi063.stdout:9/648: rmdir d8/d19/d20/d22/d5d/d76/d95 39 2024-03-22T07:16:01.956 INFO:tasks.workunit.client.1.smithi069.stdout:6/349: dwrite d4/d7/da/d3f/f61 [0,4194304] 0 2024-03-22T07:16:01.958 INFO:tasks.workunit.client.0.smithi063.stdout:2/620: symlink d0/de/d58/lc3 0 2024-03-22T07:16:01.958 INFO:tasks.workunit.client.0.smithi063.stdout:2/621: write d0/de/d35/f44 [43832,104979] 0 2024-03-22T07:16:01.961 INFO:tasks.workunit.client.0.smithi063.stdout:7/511: mkdir d5/da/d28/d4b/da8 0 2024-03-22T07:16:01.961 INFO:tasks.workunit.client.0.smithi063.stdout:7/512: chown d5/d16/d1b/c23 1 1 2024-03-22T07:16:01.962 INFO:tasks.workunit.client.0.smithi063.stdout:9/649: dread d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/f66 [0,4194304] 0 2024-03-22T07:16:01.962 INFO:tasks.workunit.client.0.smithi063.stdout:9/650: write d8/d19/d20/d22/d56/f71 [330845,1479] 0 2024-03-22T07:16:01.966 INFO:tasks.workunit.client.0.smithi063.stdout:2/622: link d0/de/d1b/d32/f3b d0/de/d35/d36/fc4 0 2024-03-22T07:16:01.968 INFO:tasks.workunit.client.0.smithi063.stdout:5/561: rename d8/d13/d28/d57/d5f/d87 to d8/d13/d1e/d3c/da0 0 2024-03-22T07:16:01.969 INFO:tasks.workunit.client.0.smithi063.stdout:5/562: fsync d8/d13/d1e/d3c/f9b 0 2024-03-22T07:16:01.969 INFO:tasks.workunit.client.0.smithi063.stdout:5/563: fsync d8/d13/d1e/d3c/da0/f88 0 2024-03-22T07:16:01.969 INFO:tasks.workunit.client.0.smithi063.stdout:5/564: chown d8/c1d 6751884 1 2024-03-22T07:16:01.969 INFO:tasks.workunit.client.0.smithi063.stdout:5/565: creat d8/d13/d1e/d3c/fa1 x:0 0 0 2024-03-22T07:16:01.969 INFO:tasks.workunit.client.0.smithi063.stdout:5/566: write d8/d13/d1e/d31/d58/d7e/d82/f96 [737146,126967] 0 2024-03-22T07:16:01.970 INFO:tasks.workunit.client.0.smithi063.stdout:2/623: unlink d0/de/d58/c84 0 2024-03-22T07:16:01.971 INFO:tasks.workunit.client.0.smithi063.stdout:4/573: dread d1/d5/dc/f10 [0,4194304] 0 2024-03-22T07:16:01.973 INFO:tasks.workunit.client.0.smithi063.stdout:0/864: dwrite da/de/d60/fd2 [0,4194304] 0 2024-03-22T07:16:01.973 INFO:tasks.workunit.client.0.smithi063.stdout:0/865: truncate da/de/d17/d25/d3c/d43/d80/f86 2321117 0 2024-03-22T07:16:01.974 INFO:tasks.workunit.client.0.smithi063.stdout:4/574: readlink d1/d5/d43/d27/d2c/d3d/l77 0 2024-03-22T07:16:01.974 INFO:tasks.workunit.client.0.smithi063.stdout:4/575: stat d1/d5/d3c/f52 0 2024-03-22T07:16:01.974 INFO:tasks.workunit.client.0.smithi063.stdout:4/576: dread - d1/d5/d43/d27/d2c/d4a/d80/fa8 zero size 2024-03-22T07:16:01.976 INFO:tasks.workunit.client.0.smithi063.stdout:3/748: dwrite da/f71 [0,4194304] 0 2024-03-22T07:16:01.977 INFO:tasks.workunit.client.0.smithi063.stdout:5/567: dread d8/d13/d28/d57/d5f/f94 [0,4194304] 0 2024-03-22T07:16:01.977 INFO:tasks.workunit.client.0.smithi063.stdout:5/568: write d8/f25 [1066433,74509] 0 2024-03-22T07:16:01.978 INFO:tasks.workunit.client.0.smithi063.stdout:1/801: dwrite d4/d71/d88/fb3 [0,4194304] 0 2024-03-22T07:16:01.978 INFO:tasks.workunit.client.0.smithi063.stdout:1/802: stat d4/d14/d57/c10b 0 2024-03-22T07:16:01.978 INFO:tasks.workunit.client.0.smithi063.stdout:6/646: dwrite d3/d6/d2a/d38/da5/d32/d51/fcc [0,4194304] 0 2024-03-22T07:16:01.978 INFO:tasks.workunit.client.0.smithi063.stdout:6/647: chown d3/d6/d2a/d38/da5/d32/l36 1072838 1 2024-03-22T07:16:01.981 INFO:tasks.workunit.client.0.smithi063.stdout:1/803: dread d4/fd3 [0,4194304] 0 2024-03-22T07:16:01.985 INFO:tasks.workunit.client.0.smithi063.stdout:3/749: mkdir da/d63/d65/d7b/df9 0 2024-03-22T07:16:01.987 INFO:tasks.workunit.client.0.smithi063.stdout:5/569: rename d8/d13/d28/c47 to d8/d13/d1e/d31/d58/d7e/ca2 0 2024-03-22T07:16:01.994 INFO:tasks.workunit.client.0.smithi063.stdout:6/648: dread d3/d6/d35/d3a/f40 [0,4194304] 0 2024-03-22T07:16:01.995 INFO:tasks.workunit.client.1.smithi069.stdout:2/294: mknod d18/d4d/d27/d33/c6d 0 2024-03-22T07:16:02.000 INFO:tasks.workunit.client.0.smithi063.stdout:1/804: rename d4/df/d13/d2b/d6c/da8/fdf to d4/df/de7/f111 0 2024-03-22T07:16:02.001 INFO:tasks.workunit.client.1.smithi069.stdout:4/476: link d6/d9b/d1f/d3f/d51/f81 d6/d9b/d1f/d3f/f9d 0 2024-03-22T07:16:02.001 INFO:tasks.workunit.client.1.smithi069.stdout:4/477: fsync d6/d16/f57 0 2024-03-22T07:16:02.001 INFO:tasks.workunit.client.0.smithi063.stdout:6/649: creat d3/d15/d16/d60/d6d/d4c/fd5 x:0 0 0 2024-03-22T07:16:02.003 INFO:tasks.workunit.client.1.smithi069.stdout:0/481: rename d3/d7/d24/d6a/f6f to d3/d9a/f9f 0 2024-03-22T07:16:02.004 INFO:tasks.workunit.client.0.smithi063.stdout:1/805: link d4/df/d13/c28 d4/d71/dbb/dc0/dd8/dee/c112 0 2024-03-22T07:16:02.004 INFO:tasks.workunit.client.1.smithi069.stdout:8/497: creat d1/d1c/d3e/f9c x:0 0 0 2024-03-22T07:16:02.004 INFO:tasks.workunit.client.1.smithi069.stdout:8/498: write d1/d1c/d3e/d66/f95 [156608,85609] 0 2024-03-22T07:16:02.006 INFO:tasks.workunit.client.0.smithi063.stdout:4/577: dread d1/d5/d43/d27/d2c/f3e [0,4194304] 0 2024-03-22T07:16:02.006 INFO:tasks.workunit.client.0.smithi063.stdout:3/750: dread da/d16/d3f/f79 [0,4194304] 0 2024-03-22T07:16:02.007 INFO:tasks.workunit.client.0.smithi063.stdout:4/578: mknod d1/d5/d3c/d8f/dbb/cc5 0 2024-03-22T07:16:02.008 INFO:tasks.workunit.client.0.smithi063.stdout:4/579: mknod d1/d5/d43/d27/d2c/d4a/d80/cc6 0 2024-03-22T07:16:02.008 INFO:tasks.workunit.client.0.smithi063.stdout:4/580: chown d1/d5/dc/f41 59 1 2024-03-22T07:16:02.013 INFO:tasks.workunit.client.0.smithi063.stdout:2/624: dwrite d0/de/d35/d36/fc4 [0,4194304] 0 2024-03-22T07:16:02.015 INFO:tasks.workunit.client.0.smithi063.stdout:2/625: mknod d0/de/d58/d7f/d97/da2/cc5 0 2024-03-22T07:16:02.016 INFO:tasks.workunit.client.0.smithi063.stdout:2/626: chown d0/de/d1b/f8c 1308 1 2024-03-22T07:16:02.018 INFO:tasks.workunit.client.0.smithi063.stdout:0/866: dwrite da/de/d17/d25/d3c/d5f/d97/fa4 [0,4194304] 0 2024-03-22T07:16:02.018 INFO:tasks.workunit.client.0.smithi063.stdout:0/867: stat da/f40 0 2024-03-22T07:16:02.018 INFO:tasks.workunit.client.0.smithi063.stdout:0/868: chown da/de/d17/d25/d3c/d5f/c81 4595692 1 2024-03-22T07:16:02.021 INFO:tasks.workunit.client.0.smithi063.stdout:5/570: dwrite d8/d13/d1e/d3c/da0/f91 [0,4194304] 0 2024-03-22T07:16:02.021 INFO:tasks.workunit.client.0.smithi063.stdout:6/650: dread d3/d6/d2a/d38/da5/f63 [0,4194304] 0 2024-03-22T07:16:02.021 INFO:tasks.workunit.client.0.smithi063.stdout:5/571: creat d8/d13/d1e/d31/d64/d65/fa3 x:0 0 0 2024-03-22T07:16:02.021 INFO:tasks.workunit.client.0.smithi063.stdout:5/572: fsync d8/f14 0 2024-03-22T07:16:02.025 INFO:tasks.workunit.client.1.smithi069.stdout:4/478: link d6/d12/d18/c3b d6/d4b/c9e 0 2024-03-22T07:16:02.030 INFO:tasks.workunit.client.1.smithi069.stdout:0/482: creat d3/d7/d15/d29/d5a/d84/fa0 x:0 0 0 2024-03-22T07:16:02.030 INFO:tasks.workunit.client.1.smithi069.stdout:0/483: chown d3/d7/d15/d29/c4b 551 1 2024-03-22T07:16:02.033 INFO:tasks.workunit.client.1.smithi069.stdout:8/499: creat d1/de/d6c/d4e/d5b/d82/f9d x:0 0 0 2024-03-22T07:16:02.033 INFO:tasks.workunit.client.1.smithi069.stdout:8/500: fsync d1/d1c/d48/f64 0 2024-03-22T07:16:02.034 INFO:tasks.workunit.client.0.smithi063.stdout:6/651: rename d3/d6/c7 to d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/cd6 0 2024-03-22T07:16:02.034 INFO:tasks.workunit.client.1.smithi069.stdout:4/479: symlink d6/d9b/d1f/d3f/d51/l9f 0 2024-03-22T07:16:02.036 INFO:tasks.workunit.client.1.smithi069.stdout:0/484: truncate d3/d7/f4a 3022962 0 2024-03-22T07:16:02.036 INFO:tasks.workunit.client.0.smithi063.stdout:2/627: rmdir d0/de/d74/db0 0 2024-03-22T07:16:02.038 INFO:tasks.workunit.client.1.smithi069.stdout:8/501: mkdir d1/d1c/d3e/d66/d9e 0 2024-03-22T07:16:02.040 INFO:tasks.workunit.client.0.smithi063.stdout:1/806: rmdir d4/df/de7 39 2024-03-22T07:16:02.040 INFO:tasks.workunit.client.0.smithi063.stdout:1/807: readlink d4/d5a/lc9 0 2024-03-22T07:16:02.043 INFO:tasks.workunit.client.0.smithi063.stdout:3/751: dwrite da/d63/d65/fa8 [0,4194304] 0 2024-03-22T07:16:02.044 INFO:tasks.workunit.client.0.smithi063.stdout:2/628: mkdir d0/de/d1b/d63/dc6 0 2024-03-22T07:16:02.047 INFO:tasks.workunit.client.0.smithi063.stdout:4/581: dwrite d1/d5/dc/d95/f6e [0,4194304] 0 2024-03-22T07:16:02.047 INFO:tasks.workunit.client.0.smithi063.stdout:4/582: fdatasync d1/d5/f28 0 2024-03-22T07:16:02.047 INFO:tasks.workunit.client.0.smithi063.stdout:4/583: chown d1/d5/d3c/d8f/l9e 37088 1 2024-03-22T07:16:02.047 INFO:tasks.workunit.client.0.smithi063.stdout:1/808: symlink d4/df/l113 0 2024-03-22T07:16:02.049 INFO:tasks.workunit.client.0.smithi063.stdout:0/869: dwrite da/de/d17/d25/d3c/d64/d78/f10a [0,4194304] 0 2024-03-22T07:16:02.051 INFO:tasks.workunit.client.0.smithi063.stdout:5/573: rename d8/d13/d28/d66 to d8/d13/d24/da4 0 2024-03-22T07:16:02.052 INFO:tasks.workunit.client.0.smithi063.stdout:0/870: truncate da/de/d17/d25/d3c/d64/fb2 6841136 0 2024-03-22T07:16:02.052 INFO:tasks.workunit.client.0.smithi063.stdout:0/871: chown da/de/fcf 3497539 1 2024-03-22T07:16:02.054 INFO:tasks.workunit.client.0.smithi063.stdout:5/574: mknod d8/d13/d24/ca5 0 2024-03-22T07:16:02.055 INFO:tasks.workunit.client.0.smithi063.stdout:5/575: chown d8/d13/d1e 553626901 1 2024-03-22T07:16:02.055 INFO:tasks.workunit.client.0.smithi063.stdout:6/652: dwrite d3/d15/d16/d60/d6d/fae [0,4194304] 0 2024-03-22T07:16:02.062 INFO:tasks.workunit.client.0.smithi063.stdout:6/653: symlink d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/ld7 0 2024-03-22T07:16:02.065 INFO:tasks.workunit.client.0.smithi063.stdout:2/629: rename d0/de/d1b/d32/f49 to d0/de/d12/d20/fc7 0 2024-03-22T07:16:02.068 INFO:tasks.workunit.client.0.smithi063.stdout:2/630: creat d0/de/d1b/d32/dac/dc2/fc8 x:0 0 0 2024-03-22T07:16:02.072 INFO:tasks.workunit.client.0.smithi063.stdout:0/872: read da/de/d17/d25/d3c/d53/d89/ffb [470658,130605] 0 2024-03-22T07:16:02.073 INFO:tasks.workunit.client.0.smithi063.stdout:5/576: rename d8/d13/d1e/d31/d64/d65/f7c to d8/d13/d1e/d31/fa6 0 2024-03-22T07:16:02.074 INFO:tasks.workunit.client.0.smithi063.stdout:0/873: mknod da/de/d17/d25/d3c/d5f/d10e/c11b 0 2024-03-22T07:16:02.079 INFO:tasks.workunit.client.1.smithi069.stdout:0/485: rmdir d3/d7/d24/d6a 39 2024-03-22T07:16:02.081 INFO:tasks.workunit.client.0.smithi063.stdout:0/874: dread da/de/d60/d56/d8b/fb9 [0,4194304] 0 2024-03-22T07:16:02.081 INFO:tasks.workunit.client.0.smithi063.stdout:0/875: truncate da/de/d17/d25/d3c/d43/db5/f101 147696 0 2024-03-22T07:16:02.084 INFO:tasks.workunit.client.0.smithi063.stdout:5/577: write d8/f2b [2425598,44439] 0 2024-03-22T07:16:02.084 INFO:tasks.workunit.client.0.smithi063.stdout:5/578: chown d8/d13/d1e/d31/d58 1184674 1 2024-03-22T07:16:02.086 INFO:tasks.workunit.client.0.smithi063.stdout:5/579: rmdir d8/d13/d24/d62 39 2024-03-22T07:16:02.088 INFO:tasks.workunit.client.0.smithi063.stdout:3/752: dwrite da/d16/d3f/f6f [0,4194304] 0 2024-03-22T07:16:02.088 INFO:tasks.workunit.client.0.smithi063.stdout:5/580: rename d8/d13/d1e/d3c/f49 to d8/fa7 0 2024-03-22T07:16:02.091 INFO:tasks.workunit.client.0.smithi063.stdout:4/584: dwrite d1/d5/f9 [4194304,4194304] 0 2024-03-22T07:16:02.091 INFO:tasks.workunit.client.0.smithi063.stdout:4/585: chown d1/d5/d43/d27/d2c/d4a/f9d 7978 1 2024-03-22T07:16:02.091 INFO:tasks.workunit.client.0.smithi063.stdout:4/586: fsync d1/d5/d3c/f47 0 2024-03-22T07:16:02.091 INFO:tasks.workunit.client.0.smithi063.stdout:4/587: creat d1/d5/d43/d27/d2c/d3d/d54/d5f/d79/fc7 x:0 0 0 2024-03-22T07:16:02.091 INFO:tasks.workunit.client.0.smithi063.stdout:4/588: fdatasync d1/d5/d43/d27/d2c/d3d/d54/f8d 0 2024-03-22T07:16:02.091 INFO:tasks.workunit.client.0.smithi063.stdout:4/589: chown d1/d5/d3c/d8f 719 1 2024-03-22T07:16:02.091 INFO:tasks.workunit.client.0.smithi063.stdout:4/590: write d1/d5/d43/d27/d2c/d3d/d54/f8d [291488,55933] 0 2024-03-22T07:16:02.092 INFO:tasks.workunit.client.1.smithi069.stdout:0/486: rmdir d3/d9a 39 2024-03-22T07:16:02.094 INFO:tasks.workunit.client.1.smithi069.stdout:0/487: write d3/d7/d15/d29/d5a/f7b [260750,61003] 0 2024-03-22T07:16:02.095 INFO:tasks.workunit.client.1.smithi069.stdout:4/480: rmdir d6/d82 0 2024-03-22T07:16:02.096 INFO:tasks.workunit.client.1.smithi069.stdout:8/502: dwrite d1/de/d6c/d73/f74 [0,4194304] 0 2024-03-22T07:16:02.097 INFO:tasks.workunit.client.1.smithi069.stdout:0/488: symlink d3/d7/d15/d29/d6e/la1 0 2024-03-22T07:16:02.098 INFO:tasks.workunit.client.0.smithi063.stdout:1/809: dwrite d4/df/d13/d2b/f103 [0,4194304] 0 2024-03-22T07:16:02.100 INFO:tasks.workunit.client.0.smithi063.stdout:6/654: dwrite d3/d15/d16/d60/f65 [0,4194304] 0 2024-03-22T07:16:02.100 INFO:tasks.workunit.client.0.smithi063.stdout:1/810: link d4/df/de7/lfa d4/d5a/d91/db6/l114 0 2024-03-22T07:16:02.103 INFO:tasks.workunit.client.1.smithi069.stdout:2/295: unlink f5 0 2024-03-22T07:16:02.103 INFO:tasks.workunit.client.1.smithi069.stdout:2/296: write d18/d4d/d25/f4b [4692864,49800] 0 2024-03-22T07:16:02.103 INFO:tasks.workunit.client.0.smithi063.stdout:1/811: symlink d4/df/d13/d2b/l115 0 2024-03-22T07:16:02.104 INFO:tasks.workunit.client.1.smithi069.stdout:0/489: write d3/d7/d15/f46 [3450425,94199] 0 2024-03-22T07:16:02.106 INFO:tasks.workunit.client.0.smithi063.stdout:1/812: mkdir d4/df/d13/d39/d6f/d116 0 2024-03-22T07:16:02.106 INFO:tasks.workunit.client.0.smithi063.stdout:1/813: chown d4/df/d13/d2b/d6c/da8/cef 5 1 2024-03-22T07:16:02.107 INFO:tasks.workunit.client.1.smithi069.stdout:9/383: symlink d2/dd/l78 0 2024-03-22T07:16:02.107 INFO:tasks.workunit.client.1.smithi069.stdout:9/384: dread - d2/d21/f3a zero size 2024-03-22T07:16:02.109 INFO:tasks.workunit.client.1.smithi069.stdout:0/490: truncate d3/d7/d15/d1f/f41 5855737 0 2024-03-22T07:16:02.109 INFO:tasks.workunit.client.0.smithi063.stdout:1/814: link d4/f93 d4/df/d13/d2b/d6c/f117 0 2024-03-22T07:16:02.109 INFO:tasks.workunit.client.0.smithi063.stdout:1/815: write d4/d14/d57/f80 [1093542,20231] 0 2024-03-22T07:16:02.112 INFO:tasks.workunit.client.1.smithi069.stdout:0/491: creat d3/d7/d24/d6a/fa2 x:0 0 0 2024-03-22T07:16:02.114 INFO:tasks.workunit.client.0.smithi063.stdout:2/631: dwrite d0/de/d12/d20/f70 [0,4194304] 0 2024-03-22T07:16:02.115 INFO:tasks.workunit.client.1.smithi069.stdout:2/297: write d18/d4d/f41 [8173179,46702] 0 2024-03-22T07:16:02.115 INFO:tasks.workunit.client.1.smithi069.stdout:4/481: write d6/d9b/d1f/f7a [2500117,79357] 0 2024-03-22T07:16:02.115 INFO:tasks.workunit.client.1.smithi069.stdout:8/503: mknod d1/c9f 0 2024-03-22T07:16:02.115 INFO:tasks.workunit.client.1.smithi069.stdout:8/504: chown d1/d1c/d3e/d66/l84 399658 1 2024-03-22T07:16:02.115 INFO:tasks.workunit.client.0.smithi063.stdout:2/632: symlink d0/de/d1b/d8b/lc9 0 2024-03-22T07:16:02.116 INFO:tasks.workunit.client.1.smithi069.stdout:7/420: symlink d9/d76/d57/d44/d53/d74/l90 0 2024-03-22T07:16:02.118 INFO:tasks.workunit.client.0.smithi063.stdout:2/633: rename d0/de/d1b/d8b/dc1 to d0/de/d74/dca 0 2024-03-22T07:16:02.118 INFO:tasks.workunit.client.0.smithi063.stdout:2/634: write d0/de/d74/f96 [1762738,15081] 0 2024-03-22T07:16:02.121 INFO:tasks.workunit.client.1.smithi069.stdout:2/298: rmdir d18/d4d/d27 39 2024-03-22T07:16:02.121 INFO:tasks.workunit.client.1.smithi069.stdout:7/421: dread d9/d76/d57/d44/f47 [0,4194304] 0 2024-03-22T07:16:02.121 INFO:tasks.workunit.client.1.smithi069.stdout:4/482: symlink d6/d4b/la0 0 2024-03-22T07:16:02.121 INFO:tasks.workunit.client.1.smithi069.stdout:4/483: stat d6/d9b/d1f/f62 0 2024-03-22T07:16:02.123 INFO:tasks.workunit.client.1.smithi069.stdout:8/505: mknod d1/de/d6c/d4e/d5b/d82/ca0 0 2024-03-22T07:16:02.123 INFO:tasks.workunit.client.0.smithi063.stdout:6/655: dread d3/d6/f28 [0,4194304] 0 2024-03-22T07:16:02.126 INFO:tasks.workunit.client.0.smithi063.stdout:6/656: mknod d3/d15/d16/d60/cd8 0 2024-03-22T07:16:02.126 INFO:tasks.workunit.client.0.smithi063.stdout:6/657: chown d3/d15/d16/d47/fb5 11710881 1 2024-03-22T07:16:02.126 INFO:tasks.workunit.client.0.smithi063.stdout:6/658: write d3/d6/d35/d3a/f40 [272144,120344] 0 2024-03-22T07:16:02.126 INFO:tasks.workunit.client.1.smithi069.stdout:7/422: creat d9/d76/d57/d44/d53/d74/f91 x:0 0 0 2024-03-22T07:16:02.127 INFO:tasks.workunit.client.0.smithi063.stdout:6/659: mkdir d3/d15/d16/d60/dd9 0 2024-03-22T07:16:02.128 INFO:tasks.workunit.client.1.smithi069.stdout:4/484: creat d6/fa1 x:0 0 0 2024-03-22T07:16:02.129 INFO:tasks.workunit.client.0.smithi063.stdout:6/660: mknod d3/d6/d2a/d38/da5/d32/dbf/cda 0 2024-03-22T07:16:02.129 INFO:tasks.workunit.client.0.smithi063.stdout:6/661: write d3/f82 [92655,48509] 0 2024-03-22T07:16:02.129 INFO:tasks.workunit.client.0.smithi063.stdout:6/662: fdatasync d3/d15/d16/d60/d6d/d4e/d5f/fb7 0 2024-03-22T07:16:02.129 INFO:tasks.workunit.client.0.smithi063.stdout:6/663: write d3/d15/f17 [754444,75125] 0 2024-03-22T07:16:02.129 INFO:tasks.workunit.client.0.smithi063.stdout:6/664: read d3/f9 [5996771,122499] 0 2024-03-22T07:16:02.130 INFO:tasks.workunit.client.0.smithi063.stdout:0/876: dwrite da/de/d60/d56/d77/fa3 [0,4194304] 0 2024-03-22T07:16:02.132 INFO:tasks.workunit.client.1.smithi069.stdout:4/485: mknod d6/d12/d18/d8b/ca2 0 2024-03-22T07:16:02.132 INFO:tasks.workunit.client.1.smithi069.stdout:4/486: truncate d6/d9b/d1f/d58/d7c/d90/f95 135731 0 2024-03-22T07:16:02.132 INFO:tasks.workunit.client.0.smithi063.stdout:0/877: symlink da/de/d17/d25/d99/dbc/l11c 0 2024-03-22T07:16:02.132 INFO:tasks.workunit.client.0.smithi063.stdout:0/878: stat da/de/d17/d25/d3c/d43/f83 0 2024-03-22T07:16:02.133 INFO:tasks.workunit.client.1.smithi069.stdout:7/423: rmdir d9/db/d48/d4b 39 2024-03-22T07:16:02.133 INFO:tasks.workunit.client.1.smithi069.stdout:7/424: chown d9/d76/d57/d75/d7a/f80 4 1 2024-03-22T07:16:02.134 INFO:tasks.workunit.client.1.smithi069.stdout:7/425: truncate d9/db/d1a/f49 963439 0 2024-03-22T07:16:02.140 INFO:tasks.workunit.client.1.smithi069.stdout:4/487: unlink d6/d9b/d1f/f5b 0 2024-03-22T07:16:02.140 INFO:tasks.workunit.client.1.smithi069.stdout:7/426: mknod d9/d42/d8a/c92 0 2024-03-22T07:16:02.142 INFO:tasks.workunit.client.0.smithi063.stdout:3/753: dwrite da/f71 [0,4194304] 0 2024-03-22T07:16:02.142 INFO:tasks.workunit.client.1.smithi069.stdout:4/488: write d6/d9b/d1f/d3f/f46 [108304,99191] 0 2024-03-22T07:16:02.143 INFO:tasks.workunit.client.0.smithi063.stdout:1/816: write d4/df/d13/d39/f82 [2706152,5620] 0 2024-03-22T07:16:02.144 INFO:tasks.workunit.client.1.smithi069.stdout:0/492: write d3/d7/d15/d29/d5a/d84/f3b [2826615,81170] 0 2024-03-22T07:16:02.146 INFO:tasks.workunit.client.1.smithi069.stdout:9/385: dwrite d2/dd/d3c/f3e [0,4194304] 0 2024-03-22T07:16:02.146 INFO:tasks.workunit.client.0.smithi063.stdout:1/817: truncate d4/df/d13/d39/d6f/f72 2389112 0 2024-03-22T07:16:02.146 INFO:tasks.workunit.client.0.smithi063.stdout:1/818: chown d4/df/d13/d2b/d3a/f42 2 1 2024-03-22T07:16:02.147 INFO:tasks.workunit.client.0.smithi063.stdout:1/819: mkdir d4/d71/d88/da1/ddb/d118 0 2024-03-22T07:16:02.147 INFO:tasks.workunit.client.1.smithi069.stdout:4/489: creat d6/d16/d83/d9a/fa3 x:0 0 0 2024-03-22T07:16:02.147 INFO:tasks.workunit.client.1.smithi069.stdout:7/427: symlink d9/d76/d57/d44/d53/l93 0 2024-03-22T07:16:02.155 INFO:tasks.workunit.client.1.smithi069.stdout:9/386: symlink d2/l79 0 2024-03-22T07:16:02.155 INFO:tasks.workunit.client.1.smithi069.stdout:9/387: fsync d2/d51/f57 0 2024-03-22T07:16:02.155 INFO:tasks.workunit.client.1.smithi069.stdout:9/388: creat d2/d51/f7a x:0 0 0 2024-03-22T07:16:02.156 INFO:tasks.workunit.client.0.smithi063.stdout:5/581: dwrite d8/f10 [0,4194304] 0 2024-03-22T07:16:02.156 INFO:tasks.workunit.client.1.smithi069.stdout:4/490: rename d6/d16/d83/f8e to d6/d12/d18/d8b/fa4 0 2024-03-22T07:16:02.156 INFO:tasks.workunit.client.1.smithi069.stdout:4/491: chown d6/d9b/d1f/d3f/d51 2789 1 2024-03-22T07:16:02.156 INFO:tasks.workunit.client.1.smithi069.stdout:7/428: creat d9/db/f94 x:0 0 0 2024-03-22T07:16:02.156 INFO:tasks.workunit.client.1.smithi069.stdout:0/493: rename d3/d7/d15/f46 to d3/d7/d15/fa3 0 2024-03-22T07:16:02.158 INFO:tasks.workunit.client.0.smithi063.stdout:3/754: write da/f3d [4574628,88889] 0 2024-03-22T07:16:02.160 INFO:tasks.workunit.client.1.smithi069.stdout:4/492: rename d6/fa1 to d6/d9b/d1f/d58/d7c/d90/fa5 0 2024-03-22T07:16:02.165 INFO:tasks.workunit.client.0.smithi063.stdout:1/820: write d4/df/d13/f25 [3091982,83432] 0 2024-03-22T07:16:02.167 INFO:tasks.workunit.client.1.smithi069.stdout:7/429: rename d9/l6c to d9/d42/l95 0 2024-03-22T07:16:02.171 INFO:tasks.workunit.client.1.smithi069.stdout:7/430: mkdir d9/d76/d57/d75/d7a/d96 0 2024-03-22T07:16:02.176 INFO:tasks.workunit.client.1.smithi069.stdout:7/431: creat d9/db/d48/d4b/f97 x:0 0 0 2024-03-22T07:16:02.177 INFO:tasks.workunit.client.0.smithi063.stdout:4/591: dwrite d1/f45 [0,4194304] 0 2024-03-22T07:16:02.177 INFO:tasks.workunit.client.0.smithi063.stdout:4/592: read - d1/d5/dc/f5a zero size 2024-03-22T07:16:02.177 INFO:tasks.workunit.client.0.smithi063.stdout:4/593: symlink d1/d5/dc/d95/d59/d9c/lc8 0 2024-03-22T07:16:02.177 INFO:tasks.workunit.client.0.smithi063.stdout:4/594: mkdir d1/d5/dc/d95/d59/d9c/dac/dc9 0 2024-03-22T07:16:02.177 INFO:tasks.workunit.client.1.smithi069.stdout:7/432: link d9/d76/d57/d44/d53/d74/d30/f3a d9/d76/d57/d44/d53/d74/f98 0 2024-03-22T07:16:02.178 INFO:tasks.workunit.client.1.smithi069.stdout:7/433: chown d9/d76/d57/d75/d7a/d96 27053 1 2024-03-22T07:16:02.178 INFO:tasks.workunit.client.0.smithi063.stdout:4/595: mknod d1/d5/dc/cca 0 2024-03-22T07:16:02.179 INFO:tasks.workunit.client.0.smithi063.stdout:4/596: mkdir d1/d5/d3c/d8f/dc0/dcb 0 2024-03-22T07:16:02.179 INFO:tasks.workunit.client.1.smithi069.stdout:2/299: dwrite d18/d4d/d27/f49 [0,4194304] 0 2024-03-22T07:16:02.180 INFO:tasks.workunit.client.0.smithi063.stdout:2/635: dwrite d0/de/d12/d20/f70 [0,4194304] 0 2024-03-22T07:16:02.181 INFO:tasks.workunit.client.0.smithi063.stdout:4/597: rename d1/d5/d43/d27/f30 to d1/d5/d43/d27/fcc 0 2024-03-22T07:16:02.181 INFO:tasks.workunit.client.0.smithi063.stdout:6/665: dwrite d3/d15/f22 [0,4194304] 0 2024-03-22T07:16:02.182 INFO:tasks.workunit.client.0.smithi063.stdout:6/666: creat d3/d6/d2a/d38/da5/d32/fdb x:0 0 0 2024-03-22T07:16:02.184 INFO:tasks.workunit.client.0.smithi063.stdout:0/879: dwrite da/de/d17/d25/d3c/d53/d89/ffb [0,4194304] 0 2024-03-22T07:16:02.188 INFO:tasks.workunit.client.1.smithi069.stdout:2/300: rename l4 to d18/d4d/d27/l6e 0 2024-03-22T07:16:02.191 INFO:tasks.workunit.client.1.smithi069.stdout:2/301: rmdir d18/d4d/d27/d33/d60/d64/d6c 0 2024-03-22T07:16:02.192 INFO:tasks.workunit.client.1.smithi069.stdout:2/302: stat d18/d4d/d27/d33/c37 0 2024-03-22T07:16:02.194 INFO:tasks.workunit.client.1.smithi069.stdout:2/303: creat d18/d4d/d3e/d40/f6f x:0 0 0 2024-03-22T07:16:02.195 INFO:tasks.workunit.client.0.smithi063.stdout:4/598: rename d1/d5/dc/dbf to d1/d5/dc/d95/d59/d9c/dac/dcd 0 2024-03-22T07:16:02.195 INFO:tasks.workunit.client.0.smithi063.stdout:4/599: getdents d1/d5/d3c/d8f/dbb 0 2024-03-22T07:16:02.204 INFO:tasks.workunit.client.1.smithi069.stdout:2/304: fsync d18/d4d/d34/f3f 0 2024-03-22T07:16:02.213 INFO:tasks.workunit.client.1.smithi069.stdout:5/303: sync 2024-03-22T07:16:02.213 INFO:tasks.workunit.client.1.smithi069.stdout:3/432: sync 2024-03-22T07:16:02.213 INFO:tasks.workunit.client.1.smithi069.stdout:5/304: write d1/d6/d26/d33/f4e [252090,65183] 0 2024-03-22T07:16:02.214 INFO:tasks.workunit.client.0.smithi063.stdout:2/636: dread d0/de/d1b/f50 [0,4194304] 0 2024-03-22T07:16:02.215 INFO:tasks.workunit.client.1.smithi069.stdout:3/433: unlink d5/l2f 0 2024-03-22T07:16:02.216 INFO:tasks.workunit.client.0.smithi063.stdout:6/667: write d3/d6/d2a/f5e [2718077,105995] 0 2024-03-22T07:16:02.217 INFO:tasks.workunit.client.1.smithi069.stdout:5/305: symlink d1/d6/d21/l6c 0 2024-03-22T07:16:02.218 INFO:tasks.workunit.client.1.smithi069.stdout:5/306: creat d1/d6/d26/d3e/f6d x:0 0 0 2024-03-22T07:16:02.218 INFO:tasks.workunit.client.0.smithi063.stdout:6/668: creat d3/d6/d2a/d38/da5/fdc x:0 0 0 2024-03-22T07:16:02.218 INFO:tasks.workunit.client.1.smithi069.stdout:3/434: getdents d5/d9/d26/d3f 0 2024-03-22T07:16:02.219 INFO:tasks.workunit.client.1.smithi069.stdout:3/435: fdatasync d5/d9/d26/d36/d41/f72 0 2024-03-22T07:16:02.219 INFO:tasks.workunit.client.0.smithi063.stdout:2/637: write d0/de/d35/f44 [2507,2998] 0 2024-03-22T07:16:02.219 INFO:tasks.workunit.client.0.smithi063.stdout:0/880: dread da/de/fcf [0,4194304] 0 2024-03-22T07:16:02.219 INFO:tasks.workunit.client.0.smithi063.stdout:6/669: creat d3/d6/d2a/d38/fdd x:0 0 0 2024-03-22T07:16:02.220 INFO:tasks.workunit.client.0.smithi063.stdout:6/670: write d3/d15/d16/d60/d6d/d4e/d5a/d88/f8d [812923,130270] 0 2024-03-22T07:16:02.220 INFO:tasks.workunit.client.0.smithi063.stdout:0/881: mknod da/de/d17/d25/d3c/d43/d80/c11d 0 2024-03-22T07:16:02.221 INFO:tasks.workunit.client.1.smithi069.stdout:5/307: creat d1/d6/d26/d3a/f6e x:0 0 0 2024-03-22T07:16:02.221 INFO:tasks.workunit.client.0.smithi063.stdout:6/671: mkdir d3/d15/d16/d57/dde 0 2024-03-22T07:16:02.222 INFO:tasks.workunit.client.0.smithi063.stdout:2/638: mknod d0/de/d1b/d32/dac/ccb 0 2024-03-22T07:16:02.222 INFO:tasks.workunit.client.0.smithi063.stdout:0/882: symlink da/d49/l11e 0 2024-03-22T07:16:02.222 INFO:tasks.workunit.client.0.smithi063.stdout:0/883: chown da/de/ccb 1 1 2024-03-22T07:16:02.223 INFO:tasks.workunit.client.1.smithi069.stdout:5/308: rename d1/f4f to d1/d6/f6f 0 2024-03-22T07:16:02.224 INFO:tasks.workunit.client.0.smithi063.stdout:6/672: symlink d3/d15/d16/d69/ldf 0 2024-03-22T07:16:02.225 INFO:tasks.workunit.client.1.smithi069.stdout:5/309: link d1/d10/l29 d1/d7/l70 0 2024-03-22T07:16:02.227 INFO:tasks.workunit.client.1.smithi069.stdout:5/310: symlink d1/d6/d21/l71 0 2024-03-22T07:16:02.227 INFO:tasks.workunit.client.1.smithi069.stdout:5/311: write d1/d6/d26/d3a/f6e [571065,67507] 0 2024-03-22T07:16:02.238 INFO:tasks.workunit.client.0.smithi063.stdout:5/582: dwrite d8/d13/d1e/d3c/f4c [0,4194304] 0 2024-03-22T07:16:02.238 INFO:tasks.workunit.client.0.smithi063.stdout:0/884: dread da/d49/f6d [4194304,4194304] 0 2024-03-22T07:16:02.238 INFO:tasks.workunit.client.1.smithi069.stdout:1/384: sync 2024-03-22T07:16:02.238 INFO:tasks.workunit.client.1.smithi069.stdout:1/385: readlink d1/d13/d36/d6e/l78 0 2024-03-22T07:16:02.238 INFO:tasks.workunit.client.1.smithi069.stdout:6/350: sync 2024-03-22T07:16:02.239 INFO:tasks.workunit.client.0.smithi063.stdout:3/755: dwrite da/d63/d65/d81/d9b/d9f/f90 [0,4194304] 0 2024-03-22T07:16:02.240 INFO:tasks.workunit.client.1.smithi069.stdout:6/351: write d4/fb [8620002,37044] 0 2024-03-22T07:16:02.240 INFO:tasks.workunit.client.0.smithi063.stdout:0/885: symlink da/de/d17/d25/d3c/d64/d110/l11f 0 2024-03-22T07:16:02.241 INFO:tasks.workunit.client.0.smithi063.stdout:0/886: fsync da/d49/f6d 0 2024-03-22T07:16:02.241 INFO:tasks.workunit.client.0.smithi063.stdout:0/887: write da/de/d17/d25/d3c/d53/d89/fc3 [446792,40231] 0 2024-03-22T07:16:02.241 INFO:tasks.workunit.client.0.smithi063.stdout:5/583: dread d8/d13/d1e/d3c/f8c [0,4194304] 0 2024-03-22T07:16:02.241 INFO:tasks.workunit.client.0.smithi063.stdout:5/584: read - d8/d13/f48 zero size 2024-03-22T07:16:02.243 INFO:tasks.workunit.client.1.smithi069.stdout:5/312: dread d1/d7/d8/f5c [0,4194304] 0 2024-03-22T07:16:02.244 INFO:tasks.workunit.client.0.smithi063.stdout:3/756: unlink da/d63/d65/d81/d9b/d9f/fa2 0 2024-03-22T07:16:02.244 INFO:tasks.workunit.client.0.smithi063.stdout:3/757: getdents da/d63/d65/d7b/df9 0 2024-03-22T07:16:02.244 INFO:tasks.workunit.client.1.smithi069.stdout:5/313: mkdir d1/d6/d26/d33/d5d/d72 0 2024-03-22T07:16:02.246 INFO:tasks.workunit.client.0.smithi063.stdout:3/758: mkdir da/d63/d65/d4b/dfa 0 2024-03-22T07:16:02.247 INFO:tasks.workunit.client.0.smithi063.stdout:3/759: mkdir da/d16/d89/d8c/dfb 0 2024-03-22T07:16:02.250 INFO:tasks.workunit.client.0.smithi063.stdout:1/821: dwrite d4/df/d2c/f90 [0,4194304] 0 2024-03-22T07:16:02.250 INFO:tasks.workunit.client.0.smithi063.stdout:1/822: chown d4/df/d13/d2b/d86/fa3 9327 1 2024-03-22T07:16:02.251 INFO:tasks.workunit.client.0.smithi063.stdout:1/823: stat d4/d5a/lc1 0 2024-03-22T07:16:02.251 INFO:tasks.workunit.client.0.smithi063.stdout:1/824: mknod d4/d71/dbb/dc0/c119 0 2024-03-22T07:16:02.252 INFO:tasks.workunit.client.0.smithi063.stdout:1/825: creat d4/d5a/d91/db6/f11a x:0 0 0 2024-03-22T07:16:02.253 INFO:tasks.workunit.client.1.smithi069.stdout:6/352: write d4/dd/f4a [4121855,118991] 0 2024-03-22T07:16:02.253 INFO:tasks.workunit.client.0.smithi063.stdout:1/826: stat d4/df/d13/c28 0 2024-03-22T07:16:02.253 INFO:tasks.workunit.client.0.smithi063.stdout:1/827: creat d4/df/d13/f11b x:0 0 0 2024-03-22T07:16:02.254 INFO:tasks.workunit.client.0.smithi063.stdout:1/828: mkdir d4/d71/d88/da1/ddb/d11c 0 2024-03-22T07:16:02.256 INFO:tasks.workunit.client.0.smithi063.stdout:1/829: read d4/df/f44 [1112595,111923] 0 2024-03-22T07:16:02.256 INFO:tasks.workunit.client.1.smithi069.stdout:9/389: dwrite d2/d51/f56 [0,4194304] 0 2024-03-22T07:16:02.257 INFO:tasks.workunit.client.1.smithi069.stdout:0/494: dwrite d3/d7/f4a [0,4194304] 0 2024-03-22T07:16:02.257 INFO:tasks.workunit.client.0.smithi063.stdout:1/830: unlink d4/d5a/fcd 0 2024-03-22T07:16:02.258 INFO:tasks.workunit.client.0.smithi063.stdout:1/831: unlink d4/f93 0 2024-03-22T07:16:02.259 INFO:tasks.workunit.client.1.smithi069.stdout:9/390: symlink d2/l7b 0 2024-03-22T07:16:02.261 INFO:tasks.workunit.client.1.smithi069.stdout:2/305: dwrite d18/d4d/d27/d33/f35 [0,4194304] 0 2024-03-22T07:16:02.261 INFO:tasks.workunit.client.1.smithi069.stdout:2/306: stat d18/f1b 0 2024-03-22T07:16:02.263 INFO:tasks.workunit.client.1.smithi069.stdout:0/495: symlink d3/d7/d15/d29/d5a/d5d/la4 0 2024-03-22T07:16:02.265 INFO:tasks.workunit.client.0.smithi063.stdout:4/600: dwrite d1/d5/dc/d95/fc3 [0,4194304] 0 2024-03-22T07:16:02.266 INFO:tasks.workunit.client.0.smithi063.stdout:6/673: dwrite d3/f9 [8388608,4194304] 0 2024-03-22T07:16:02.266 INFO:tasks.workunit.client.0.smithi063.stdout:4/601: fdatasync d1/d5/dc/d95/f44 0 2024-03-22T07:16:02.266 INFO:tasks.workunit.client.1.smithi069.stdout:9/391: mkdir d2/dd/d3c/d48/d7c 0 2024-03-22T07:16:02.268 INFO:tasks.workunit.client.0.smithi063.stdout:1/832: write d4/df/d13/d2b/d3a/f42 [793807,20418] 0 2024-03-22T07:16:02.268 INFO:tasks.workunit.client.0.smithi063.stdout:1/833: readlink d4/df/d13/d2b/d6c/d85/d9e/lcf 0 2024-03-22T07:16:02.268 INFO:tasks.workunit.client.0.smithi063.stdout:1/834: write d4/d8b/f97 [725237,127506] 0 2024-03-22T07:16:02.270 INFO:tasks.workunit.client.1.smithi069.stdout:5/314: read d1/d7/d8/f12 [8860149,39175] 0 2024-03-22T07:16:02.270 INFO:tasks.workunit.client.1.smithi069.stdout:5/315: write d1/d7/d8/f18 [2735352,48744] 0 2024-03-22T07:16:02.270 INFO:tasks.workunit.client.1.smithi069.stdout:5/316: fsync d1/d7/f59 0 2024-03-22T07:16:02.270 INFO:tasks.workunit.client.1.smithi069.stdout:5/317: stat d1/d7/d3d/d3f/c67 0 2024-03-22T07:16:02.271 INFO:tasks.workunit.client.1.smithi069.stdout:5/318: readlink d1/d7/l70 0 2024-03-22T07:16:02.273 INFO:tasks.workunit.client.0.smithi063.stdout:2/639: dwrite d0/de/d1b/d32/dac/faf [0,4194304] 0 2024-03-22T07:16:02.273 INFO:tasks.workunit.client.0.smithi063.stdout:2/640: read - d0/de/d1b/d32/dac/fab zero size 2024-03-22T07:16:02.274 INFO:tasks.workunit.client.0.smithi063.stdout:4/602: creat d1/d5/dc/d95/da4/fce x:0 0 0 2024-03-22T07:16:02.277 INFO:tasks.workunit.client.1.smithi069.stdout:0/496: rename d3/d7/d15/d29/d5a/d84/f3a to d3/d7/d90/fa5 0 2024-03-22T07:16:02.277 INFO:tasks.workunit.client.1.smithi069.stdout:0/497: fsync d3/d7/d15/d1f/f2b 0 2024-03-22T07:16:02.277 INFO:tasks.workunit.client.0.smithi063.stdout:2/641: rename d0/de/c89 to d0/de/d35/ccc 0 2024-03-22T07:16:02.282 INFO:tasks.workunit.client.0.smithi063.stdout:2/642: unlink d0/de/d1b/d32/c71 0 2024-03-22T07:16:02.283 INFO:tasks.workunit.client.1.smithi069.stdout:7/434: dwrite d9/f6f [4194304,4194304] 0 2024-03-22T07:16:02.285 INFO:tasks.workunit.client.0.smithi063.stdout:4/603: link d1/d5/c18 d1/d5/d85/ccf 0 2024-03-22T07:16:02.285 INFO:tasks.workunit.client.0.smithi063.stdout:4/604: chown d1/d5/dc/f10 160 1 2024-03-22T07:16:02.285 INFO:tasks.workunit.client.1.smithi069.stdout:2/307: write d18/d4d/d34/f5f [3504150,26844] 0 2024-03-22T07:16:02.289 INFO:tasks.workunit.client.1.smithi069.stdout:9/392: unlink d2/d71/l74 0 2024-03-22T07:16:02.289 INFO:tasks.workunit.client.0.smithi063.stdout:2/643: creat d0/de/d58/d7f/d97/da2/fcd x:0 0 0 2024-03-22T07:16:02.290 INFO:tasks.workunit.client.1.smithi069.stdout:5/319: rename d1/d7/d8/f5c to d1/d7/d3d/d3f/f73 0 2024-03-22T07:16:02.290 INFO:tasks.workunit.client.1.smithi069.stdout:5/320: read - d1/d6/d26/d3e/f6d zero size 2024-03-22T07:16:02.296 INFO:tasks.workunit.client.0.smithi063.stdout:2/644: dread d0/de/f14 [0,4194304] 0 2024-03-22T07:16:02.296 INFO:tasks.workunit.client.0.smithi063.stdout:2/645: stat d0/de/d1b/c2f 0 2024-03-22T07:16:02.297 INFO:tasks.workunit.client.0.smithi063.stdout:2/646: mknod d0/de/d74/dca/cce 0 2024-03-22T07:16:02.298 INFO:tasks.workunit.client.1.smithi069.stdout:5/321: dread d1/d7/d8/f58 [0,4194304] 0 2024-03-22T07:16:02.299 INFO:tasks.workunit.client.1.smithi069.stdout:9/393: symlink d2/dd/d3c/d2e/l7d 0 2024-03-22T07:16:02.299 INFO:tasks.workunit.client.1.smithi069.stdout:9/394: creat d2/f7e x:0 0 0 2024-03-22T07:16:02.299 INFO:tasks.workunit.client.0.smithi063.stdout:2/647: mknod d0/de/d1b/d32/ccf 0 2024-03-22T07:16:02.300 INFO:tasks.workunit.client.0.smithi063.stdout:2/648: write d0/de/f14 [545929,84575] 0 2024-03-22T07:16:02.300 INFO:tasks.workunit.client.0.smithi063.stdout:2/649: stat d0/c3 0 2024-03-22T07:16:02.302 INFO:tasks.workunit.client.1.smithi069.stdout:2/308: dread d18/d4d/d27/d33/d60/f5a [0,4194304] 0 2024-03-22T07:16:02.305 INFO:tasks.workunit.client.0.smithi063.stdout:8/763: sync 2024-03-22T07:16:02.306 INFO:tasks.workunit.client.1.smithi069.stdout:9/395: mkdir d2/dd/d3c/d48/d7f 0 2024-03-22T07:16:02.306 INFO:tasks.workunit.client.1.smithi069.stdout:9/396: dread - d2/f7e zero size 2024-03-22T07:16:02.306 INFO:tasks.workunit.client.1.smithi069.stdout:9/397: readlink d2/l3d 0 2024-03-22T07:16:02.306 INFO:tasks.workunit.client.1.smithi069.stdout:9/398: chown d2/dd/d3c/c4c 18727896 1 2024-03-22T07:16:02.307 INFO:tasks.workunit.client.0.smithi063.stdout:8/764: mkdir d0/d26/d2d/d59/dc0/d106 0 2024-03-22T07:16:02.307 INFO:tasks.workunit.client.0.smithi063.stdout:8/765: fsync d0/d24/d5b/f44 0 2024-03-22T07:16:02.308 INFO:tasks.workunit.client.0.smithi063.stdout:0/888: dwrite da/de/d17/d25/d3c/fd3 [0,4194304] 0 2024-03-22T07:16:02.309 INFO:tasks.workunit.client.0.smithi063.stdout:5/585: dwrite d8/f14 [0,4194304] 0 2024-03-22T07:16:02.309 INFO:tasks.workunit.client.0.smithi063.stdout:0/889: truncate da/de/d60/d9b/fa5 1013805 0 2024-03-22T07:16:02.310 INFO:tasks.workunit.client.0.smithi063.stdout:1/835: dwrite d4/f38 [0,4194304] 0 2024-03-22T07:16:02.311 INFO:tasks.workunit.client.1.smithi069.stdout:6/353: dwrite d4/dd/d41/f51 [0,4194304] 0 2024-03-22T07:16:02.311 INFO:tasks.workunit.client.1.smithi069.stdout:6/354: truncate d4/d7/f35 358951 0 2024-03-22T07:16:02.313 INFO:tasks.workunit.client.1.smithi069.stdout:6/355: truncate d4/dd/d12/d13/f23 3036173 0 2024-03-22T07:16:02.314 INFO:tasks.workunit.client.1.smithi069.stdout:6/356: chown d4/d7/da/d3f/c4d 26 1 2024-03-22T07:16:02.317 INFO:tasks.workunit.client.0.smithi063.stdout:2/650: write d0/de/d35/d36/fc4 [3953908,102294] 0 2024-03-22T07:16:02.317 INFO:tasks.workunit.client.0.smithi063.stdout:3/760: dwrite da/d16/d26/d69/f8f [0,4194304] 0 2024-03-22T07:16:02.317 INFO:tasks.workunit.client.0.smithi063.stdout:1/836: mkdir d4/df/d13/d11d 0 2024-03-22T07:16:02.318 INFO:tasks.workunit.client.1.smithi069.stdout:0/498: dwrite d3/d7/d17/f3f [0,4194304] 0 2024-03-22T07:16:02.318 INFO:tasks.workunit.client.1.smithi069.stdout:6/357: link d4/f46 d4/f6b 0 2024-03-22T07:16:02.321 INFO:tasks.workunit.client.1.smithi069.stdout:6/358: creat d4/f6c x:0 0 0 2024-03-22T07:16:02.321 INFO:tasks.workunit.client.1.smithi069.stdout:6/359: chown d4/dd/c18 520172 1 2024-03-22T07:16:02.322 INFO:tasks.workunit.client.1.smithi069.stdout:6/360: creat d4/d7/da/d3f/f6d x:0 0 0 2024-03-22T07:16:02.322 INFO:tasks.workunit.client.1.smithi069.stdout:6/361: readlink d4/d7/da/l2a 0 2024-03-22T07:16:02.323 INFO:tasks.workunit.client.0.smithi063.stdout:4/605: getdents d1/d5/dc 0 2024-03-22T07:16:02.325 INFO:tasks.workunit.client.0.smithi063.stdout:8/766: write d0/d26/f92 [3915048,111369] 0 2024-03-22T07:16:02.326 INFO:tasks.workunit.client.1.smithi069.stdout:6/362: getdents d4/d7/d5b 0 2024-03-22T07:16:02.326 INFO:tasks.workunit.client.0.smithi063.stdout:5/586: dread d8/d13/f5d [0,4194304] 0 2024-03-22T07:16:02.327 INFO:tasks.workunit.client.1.smithi069.stdout:9/399: dread d2/f5 [0,4194304] 0 2024-03-22T07:16:02.329 INFO:tasks.workunit.client.1.smithi069.stdout:6/363: creat d4/d7/f6e x:0 0 0 2024-03-22T07:16:02.334 INFO:tasks.workunit.client.1.smithi069.stdout:9/400: creat d2/dd/d3c/d2e/f80 x:0 0 0 2024-03-22T07:16:02.336 INFO:tasks.workunit.client.0.smithi063.stdout:3/761: mkdir da/d63/d65/d4b/dfa/dfc 0 2024-03-22T07:16:02.336 INFO:tasks.workunit.client.0.smithi063.stdout:3/762: fdatasync da/d16/d89/da9/dd3/fdb 0 2024-03-22T07:16:02.336 INFO:tasks.workunit.client.0.smithi063.stdout:3/763: write da/d16/d3f/ff7 [72678,33144] 0 2024-03-22T07:16:02.337 INFO:tasks.workunit.client.1.smithi069.stdout:6/364: creat d4/dd/d12/d44/f6f x:0 0 0 2024-03-22T07:16:02.338 INFO:tasks.workunit.client.0.smithi063.stdout:1/837: mkdir d4/df/d11e 0 2024-03-22T07:16:02.338 INFO:tasks.workunit.client.0.smithi063.stdout:1/838: write d4/df/d13/d2b/d6c/d85/d9e/fd0 [329900,123342] 0 2024-03-22T07:16:02.338 INFO:tasks.workunit.client.0.smithi063.stdout:1/839: creat d4/df/d13/d2b/d86/f11f x:0 0 0 2024-03-22T07:16:02.342 INFO:tasks.workunit.client.1.smithi069.stdout:2/309: dwrite d18/d4d/d27/d33/d60/f5a [0,4194304] 0 2024-03-22T07:16:02.344 INFO:tasks.workunit.client.0.smithi063.stdout:8/767: truncate d0/d24/d33/fee 328557 0 2024-03-22T07:16:02.345 INFO:tasks.workunit.client.0.smithi063.stdout:5/587: rename d8/d13/d24/d62/c85 to d8/d13/d1e/d31/ca8 0 2024-03-22T07:16:02.348 INFO:tasks.workunit.client.1.smithi069.stdout:2/310: creat d18/f70 x:0 0 0 2024-03-22T07:16:02.348 INFO:tasks.workunit.client.1.smithi069.stdout:2/311: fdatasync d18/d4d/d3e/f65 0 2024-03-22T07:16:02.349 INFO:tasks.workunit.client.0.smithi063.stdout:5/588: rmdir d8/d13/d1e/d31/d64 39 2024-03-22T07:16:02.350 INFO:tasks.workunit.client.0.smithi063.stdout:1/840: write d4/df/f64 [219816,79845] 0 2024-03-22T07:16:02.351 INFO:tasks.workunit.client.0.smithi063.stdout:0/890: dwrite da/de/d17/d25/d3c/f3f [0,4194304] 0 2024-03-22T07:16:02.351 INFO:tasks.workunit.client.0.smithi063.stdout:0/891: read da/de/d60/d9b/f9f [3085643,70902] 0 2024-03-22T07:16:02.351 INFO:tasks.workunit.client.0.smithi063.stdout:0/892: write da/de/d17/d25/d95/d98/db3/f112 [734381,107187] 0 2024-03-22T07:16:02.352 INFO:tasks.workunit.client.1.smithi069.stdout:2/312: rename d18/d4d/d34/l42 to d18/d4d/d27/d33/d60/d64/l71 0 2024-03-22T07:16:02.353 INFO:tasks.workunit.client.0.smithi063.stdout:5/589: mknod d8/d5e/ca9 0 2024-03-22T07:16:02.355 INFO:tasks.workunit.client.1.smithi069.stdout:5/322: truncate d1/d7/d8/f18 2339922 0 2024-03-22T07:16:02.363 INFO:tasks.workunit.client.0.smithi063.stdout:0/893: mknod da/de/d17/d25/d95/c120 0 2024-03-22T07:16:02.368 INFO:tasks.workunit.client.0.smithi063.stdout:0/894: mkdir da/d49/d121 0 2024-03-22T07:16:02.371 INFO:tasks.workunit.client.0.smithi063.stdout:0/895: link da/de/d17/d25/d3c/d5f/d97/fa4 da/de/d17/d25/d3c/d43/d80/f122 0 2024-03-22T07:16:02.373 INFO:tasks.workunit.client.0.smithi063.stdout:0/896: mkdir da/de/d60/d56/d8b/d123 0 2024-03-22T07:16:02.374 INFO:tasks.workunit.client.0.smithi063.stdout:0/897: mknod da/de/d17/d25/d3c/d43/c124 0 2024-03-22T07:16:02.374 INFO:tasks.workunit.client.0.smithi063.stdout:0/898: truncate da/d49/f74 528241 0 2024-03-22T07:16:02.382 INFO:tasks.workunit.client.0.smithi063.stdout:3/764: dwrite da/d63/d65/d83/fc8 [0,4194304] 0 2024-03-22T07:16:02.382 INFO:tasks.workunit.client.0.smithi063.stdout:2/651: dwrite d0/de/d12/f27 [0,4194304] 0 2024-03-22T07:16:02.385 INFO:tasks.workunit.client.0.smithi063.stdout:4/606: dwrite d1/d5/f67 [0,4194304] 0 2024-03-22T07:16:02.385 INFO:tasks.workunit.client.1.smithi069.stdout:0/499: dwrite d3/d9a/f9f [0,4194304] 0 2024-03-22T07:16:02.386 INFO:tasks.workunit.client.1.smithi069.stdout:0/500: read d3/f39 [3842957,109952] 0 2024-03-22T07:16:02.386 INFO:tasks.workunit.client.0.smithi063.stdout:2/652: mknod d0/de/d58/d7f/d97/da2/cd0 0 2024-03-22T07:16:02.389 INFO:tasks.workunit.client.0.smithi063.stdout:4/607: symlink d1/d5/d3c/d65/ld0 0 2024-03-22T07:16:02.389 INFO:tasks.workunit.client.1.smithi069.stdout:6/365: dwrite d4/dd/d12/f32 [0,4194304] 0 2024-03-22T07:16:02.390 INFO:tasks.workunit.client.0.smithi063.stdout:5/590: dwrite d8/d13/d1e/f27 [0,4194304] 0 2024-03-22T07:16:02.390 INFO:tasks.workunit.client.0.smithi063.stdout:2/653: mkdir d0/de/d12/dd1 0 2024-03-22T07:16:02.390 INFO:tasks.workunit.client.0.smithi063.stdout:2/654: read - d0/de/d12/d5a/fa6 zero size 2024-03-22T07:16:02.391 INFO:tasks.workunit.client.0.smithi063.stdout:8/768: dwrite d0/d1b/d28/d3e/fec [0,4194304] 0 2024-03-22T07:16:02.391 INFO:tasks.workunit.client.1.smithi069.stdout:6/366: creat d4/d7/d5b/f70 x:0 0 0 2024-03-22T07:16:02.392 INFO:tasks.workunit.client.0.smithi063.stdout:4/608: creat d1/d5/d3c/d8f/dbb/fd1 x:0 0 0 2024-03-22T07:16:02.392 INFO:tasks.workunit.client.0.smithi063.stdout:4/609: write d1/d5/d43/d27/d2c/d3d/d54/d5f/f66 [5069753,25221] 0 2024-03-22T07:16:02.393 INFO:tasks.workunit.client.1.smithi069.stdout:9/401: dwrite d2/dd/d3c/d48/f4a [0,4194304] 0 2024-03-22T07:16:02.397 INFO:tasks.workunit.client.0.smithi063.stdout:1/841: dwrite d4/df/d13/d2b/d86/fa3 [0,4194304] 0 2024-03-22T07:16:02.399 INFO:tasks.workunit.client.0.smithi063.stdout:0/899: dwrite da/de/d17/d25/d3c/d5f/fb8 [0,4194304] 0 2024-03-22T07:16:02.399 INFO:tasks.workunit.client.0.smithi063.stdout:0/900: stat da/de/d17/d25/d3c/fd3 0 2024-03-22T07:16:02.399 INFO:tasks.workunit.client.0.smithi063.stdout:2/655: rename d0/de/d1b/d32/dac/f98 to d0/de/d74/dca/fd2 0 2024-03-22T07:16:02.399 INFO:tasks.workunit.client.0.smithi063.stdout:2/656: creat d0/de/d12/d20/fd3 x:0 0 0 2024-03-22T07:16:02.404 INFO:tasks.workunit.client.0.smithi063.stdout:8/769: link d0/d24/d33/d4c/l8c d0/d1b/d4f/l107 0 2024-03-22T07:16:02.408 INFO:tasks.workunit.client.0.smithi063.stdout:4/610: dread d1/d5/d3c/f83 [0,4194304] 0 2024-03-22T07:16:02.408 INFO:tasks.workunit.client.0.smithi063.stdout:4/611: dread - d1/d5/d3c/d8f/dbb/fd1 zero size 2024-03-22T07:16:02.408 INFO:tasks.workunit.client.0.smithi063.stdout:4/612: stat d1 0 2024-03-22T07:16:02.408 INFO:tasks.workunit.client.1.smithi069.stdout:6/367: dread d4/d7/da/f1b [0,4194304] 0 2024-03-22T07:16:02.409 INFO:tasks.workunit.client.0.smithi063.stdout:4/613: dread d1/d5/d43/d27/d2c/d3d/d54/fa6 [0,4194304] 0 2024-03-22T07:16:02.413 INFO:tasks.workunit.client.0.smithi063.stdout:0/901: mkdir da/de/d17/d25/d3c/d43/d76/d92/d125 0 2024-03-22T07:16:02.414 INFO:tasks.workunit.client.1.smithi069.stdout:8/506: sync 2024-03-22T07:16:02.416 INFO:tasks.workunit.client.1.smithi069.stdout:8/507: mknod d1/de/d6c/d4e/d5b/ca1 0 2024-03-22T07:16:02.416 INFO:tasks.workunit.client.0.smithi063.stdout:8/770: link d0/d1b/d28/l31 d0/d26/d2d/d8f/l108 0 2024-03-22T07:16:02.417 INFO:tasks.workunit.client.1.smithi069.stdout:8/508: mknod d1/de/d20/ca2 0 2024-03-22T07:16:02.417 INFO:tasks.workunit.client.1.smithi069.stdout:8/509: stat d1/d1c/d3e/d66/f78 0 2024-03-22T07:16:02.417 INFO:tasks.workunit.client.1.smithi069.stdout:8/510: dread - d1/d1c/d3e/d66/f78 zero size 2024-03-22T07:16:02.418 INFO:tasks.workunit.client.0.smithi063.stdout:4/614: link d1/d5/d43/d27/d2c/d3d/d54/d5f/c8e d1/d5/dc/d95/d59/d9c/dac/dc9/cd2 0 2024-03-22T07:16:02.418 INFO:tasks.workunit.client.0.smithi063.stdout:0/902: symlink da/de/d17/d25/d3c/d64/df5/l126 0 2024-03-22T07:16:02.418 INFO:tasks.workunit.client.0.smithi063.stdout:0/903: chown da/de/d17/d25/d3c/d64/d78/c8e 45990515 1 2024-03-22T07:16:02.420 INFO:tasks.workunit.client.0.smithi063.stdout:7/513: sync 2024-03-22T07:16:02.420 INFO:tasks.workunit.client.0.smithi063.stdout:7/514: truncate d5/d16/d1b/d21/f86 177027 0 2024-03-22T07:16:02.420 INFO:tasks.workunit.client.0.smithi063.stdout:9/651: sync 2024-03-22T07:16:02.420 INFO:tasks.workunit.client.1.smithi069.stdout:8/511: rename d1/de/d6c/d4e/c58 to d1/de/ca3 0 2024-03-22T07:16:02.421 INFO:tasks.workunit.client.1.smithi069.stdout:8/512: write d1/d1c/f68 [5180722,26913] 0 2024-03-22T07:16:02.423 INFO:tasks.workunit.client.0.smithi063.stdout:3/765: dwrite da/d16/d26/f8b [0,4194304] 0 2024-03-22T07:16:02.423 INFO:tasks.workunit.client.0.smithi063.stdout:3/766: write da/d16/d89/d8c/f9e [5156219,59481] 0 2024-03-22T07:16:02.423 INFO:tasks.workunit.client.0.smithi063.stdout:3/767: write da/d16/d89/da9/fcc [194051,115649] 0 2024-03-22T07:16:02.425 INFO:tasks.workunit.client.1.smithi069.stdout:0/501: dwrite d3/d7/d15/d29/d5a/d84/fa0 [0,4194304] 0 2024-03-22T07:16:02.425 INFO:tasks.workunit.client.0.smithi063.stdout:7/515: mknod d5/d16/d43/d9d/ca9 0 2024-03-22T07:16:02.425 INFO:tasks.workunit.client.1.smithi069.stdout:5/323: dread d1/d7/d8/f18 [0,4194304] 0 2024-03-22T07:16:02.427 INFO:tasks.workunit.client.0.smithi063.stdout:9/652: creat d8/d19/d20/d22/d5d/d76/fc6 x:0 0 0 2024-03-22T07:16:02.428 INFO:tasks.workunit.client.0.smithi063.stdout:4/615: getdents d1/d5/dc/d95/d59/d9c 0 2024-03-22T07:16:02.428 INFO:tasks.workunit.client.1.smithi069.stdout:0/502: rename d3/d7/d15/f45 to d3/d7/d90/fa6 0 2024-03-22T07:16:02.428 INFO:tasks.workunit.client.1.smithi069.stdout:0/503: creat d3/d7/d15/d29/d5a/fa7 x:0 0 0 2024-03-22T07:16:02.429 INFO:tasks.workunit.client.1.smithi069.stdout:5/324: mknod d1/d6/d26/d33/d5d/d72/c74 0 2024-03-22T07:16:02.432 INFO:tasks.workunit.client.1.smithi069.stdout:0/504: link d3/d7/d17/l2a d3/d7/la8 0 2024-03-22T07:16:02.434 INFO:tasks.workunit.client.1.smithi069.stdout:5/325: rename d1/d7/d8/f30 to d1/d28/f75 0 2024-03-22T07:16:02.436 INFO:tasks.workunit.client.0.smithi063.stdout:3/768: rename da/fca to da/d16/d3f/de2/ffd 0 2024-03-22T07:16:02.439 INFO:tasks.workunit.client.1.smithi069.stdout:0/505: dread d3/d7/f49 [0,4194304] 0 2024-03-22T07:16:02.439 INFO:tasks.workunit.client.0.smithi063.stdout:7/516: creat d5/da/d28/d4b/da8/faa x:0 0 0 2024-03-22T07:16:02.440 INFO:tasks.workunit.client.1.smithi069.stdout:0/506: stat d3/d7/la8 0 2024-03-22T07:16:02.441 INFO:tasks.workunit.client.0.smithi063.stdout:9/653: link d8/d19/d20/d22/d29/d31/d53/d6a/da2/f80 d8/d19/d20/d22/d29/d31/d53/d6a/da2/fc7 0 2024-03-22T07:16:02.443 INFO:tasks.workunit.client.1.smithi069.stdout:0/507: unlink d3/d7/d5e/f77 0 2024-03-22T07:16:02.445 INFO:tasks.workunit.client.1.smithi069.stdout:0/508: creat d3/d7/d15/d29/d5a/d5d/d9d/fa9 x:0 0 0 2024-03-22T07:16:02.447 INFO:tasks.workunit.client.0.smithi063.stdout:3/769: mkdir da/d16/d89/d8c/dc1/dfe 0 2024-03-22T07:16:02.459 INFO:tasks.workunit.client.1.smithi069.stdout:0/509: dread d3/d7/d15/f20 [0,4194304] 0 2024-03-22T07:16:02.459 INFO:tasks.workunit.client.1.smithi069.stdout:0/510: chown d3/d7/d17/f56 429001478 1 2024-03-22T07:16:02.459 INFO:tasks.workunit.client.0.smithi063.stdout:5/591: dwrite d8/d13/d1e/f38 [4194304,4194304] 0 2024-03-22T07:16:02.460 INFO:tasks.workunit.client.0.smithi063.stdout:9/654: dread d8/d19/d2b/f89 [0,4194304] 0 2024-03-22T07:16:02.461 INFO:tasks.workunit.client.0.smithi063.stdout:3/770: dread da/d16/d26/f27 [0,4194304] 0 2024-03-22T07:16:02.463 INFO:tasks.workunit.client.0.smithi063.stdout:1/842: dwrite d4/df/d13/f3c [0,4194304] 0 2024-03-22T07:16:02.463 INFO:tasks.workunit.client.0.smithi063.stdout:9/655: creat d8/d19/d20/d22/d29/d31/d4b/db7/fc8 x:0 0 0 2024-03-22T07:16:02.465 INFO:tasks.workunit.client.0.smithi063.stdout:1/843: creat d4/df/d13/d2b/d3a/df6/f120 x:0 0 0 2024-03-22T07:16:02.466 INFO:tasks.workunit.client.0.smithi063.stdout:1/844: truncate d4/d71/fe1 741269 0 2024-03-22T07:16:02.467 INFO:tasks.workunit.client.0.smithi063.stdout:9/656: creat d8/d15/fc9 x:0 0 0 2024-03-22T07:16:02.469 INFO:tasks.workunit.client.0.smithi063.stdout:1/845: mknod d4/df/d13/d2b/c121 0 2024-03-22T07:16:02.470 INFO:tasks.workunit.client.1.smithi069.stdout:0/511: write d3/d7/d15/d1f/f26 [3099597,85274] 0 2024-03-22T07:16:02.470 INFO:tasks.workunit.client.0.smithi063.stdout:9/657: truncate d8/d17/f1d 3589875 0 2024-03-22T07:16:02.470 INFO:tasks.workunit.client.0.smithi063.stdout:1/846: chown d4/df/d13/d2b/d6c/d85/d9e/led 30392 1 2024-03-22T07:16:02.474 INFO:tasks.workunit.client.1.smithi069.stdout:9/402: dwrite d2/dd/d3c/d2e/f4b [4194304,4194304] 0 2024-03-22T07:16:02.474 INFO:tasks.workunit.client.0.smithi063.stdout:9/658: mkdir d8/d19/d20/d22/d29/dca 0 2024-03-22T07:16:02.475 INFO:tasks.workunit.client.1.smithi069.stdout:9/403: mkdir d2/d21/d77/d81 0 2024-03-22T07:16:02.475 INFO:tasks.workunit.client.1.smithi069.stdout:9/404: mknod d2/d21/c82 0 2024-03-22T07:16:02.478 INFO:tasks.workunit.client.0.smithi063.stdout:3/771: write da/d16/d26/f8b [2207987,36949] 0 2024-03-22T07:16:02.478 INFO:tasks.workunit.client.0.smithi063.stdout:3/772: fsync da/d16/d26/fbd 0 2024-03-22T07:16:02.483 INFO:tasks.workunit.client.0.smithi063.stdout:4/616: dwrite d1/d5/f2f [0,4194304] 0 2024-03-22T07:16:02.483 INFO:tasks.workunit.client.0.smithi063.stdout:8/771: dwrite d0/f7c [0,4194304] 0 2024-03-22T07:16:02.483 INFO:tasks.workunit.client.0.smithi063.stdout:4/617: chown d1/d5/d43/d27/d2c/d3d/d54/d5f/d79/fc7 67 1 2024-03-22T07:16:02.484 INFO:tasks.workunit.client.1.smithi069.stdout:0/512: write d3/d9a/f9f [2037863,113775] 0 2024-03-22T07:16:02.485 INFO:tasks.workunit.client.0.smithi063.stdout:0/904: dwrite da/de/d17/d25/d3c/d43/d80/f86 [0,4194304] 0 2024-03-22T07:16:02.487 INFO:tasks.workunit.client.1.smithi069.stdout:8/513: dwrite d1/d1c/d48/d5e/f79 [0,4194304] 0 2024-03-22T07:16:02.487 INFO:tasks.workunit.client.1.smithi069.stdout:8/514: chown d1/de/d6c/d4e/l53 29712 1 2024-03-22T07:16:02.488 INFO:tasks.workunit.client.0.smithi063.stdout:4/618: creat d1/d5/dc/d95/da4/fd3 x:0 0 0 2024-03-22T07:16:02.488 INFO:tasks.workunit.client.0.smithi063.stdout:2/657: dwrite d0/f53 [4194304,4194304] 0 2024-03-22T07:16:02.488 INFO:tasks.workunit.client.0.smithi063.stdout:8/772: dread d0/d24/f74 [0,4194304] 0 2024-03-22T07:16:02.489 INFO:tasks.workunit.client.0.smithi063.stdout:8/773: dread - d0/d1b/d28/d7b/d45/fda zero size 2024-03-22T07:16:02.491 INFO:tasks.workunit.client.0.smithi063.stdout:0/905: link da/de/d39/l42 da/de/d17/d25/d3c/d5f/l127 0 2024-03-22T07:16:02.491 INFO:tasks.workunit.client.0.smithi063.stdout:0/906: fsync da/de/d17/d25/d3c/d64/fb2 0 2024-03-22T07:16:02.491 INFO:tasks.workunit.client.1.smithi069.stdout:5/326: dwrite d1/f3 [0,4194304] 0 2024-03-22T07:16:02.493 INFO:tasks.workunit.client.0.smithi063.stdout:7/517: dwrite d5/d16/d43/f56 [0,4194304] 0 2024-03-22T07:16:02.493 INFO:tasks.workunit.client.0.smithi063.stdout:7/518: write d5/d16/d43/fa5 [911228,10456] 0 2024-03-22T07:16:02.493 INFO:tasks.workunit.client.0.smithi063.stdout:7/519: readlink d5/d16/d43/l8e 0 2024-03-22T07:16:02.496 INFO:tasks.workunit.client.0.smithi063.stdout:4/619: rename d1/d5/dc/d95/l88 to d1/d5/dc/d95/ld4 0 2024-03-22T07:16:02.496 INFO:tasks.workunit.client.1.smithi069.stdout:8/515: creat d1/de/fa4 x:0 0 0 2024-03-22T07:16:02.499 INFO:tasks.workunit.client.0.smithi063.stdout:8/774: creat d0/d26/f109 x:0 0 0 2024-03-22T07:16:02.500 INFO:tasks.workunit.client.1.smithi069.stdout:5/327: unlink d1/d6/d26/d3e/c69 0 2024-03-22T07:16:02.500 INFO:tasks.workunit.client.1.smithi069.stdout:5/328: dread - d1/d6/f6f zero size 2024-03-22T07:16:02.502 INFO:tasks.workunit.client.1.smithi069.stdout:5/329: symlink d1/d7/d8/d64/l76 0 2024-03-22T07:16:02.502 INFO:tasks.workunit.client.1.smithi069.stdout:5/330: stat d1/d7/d8/d64 0 2024-03-22T07:16:02.505 INFO:tasks.workunit.client.0.smithi063.stdout:3/773: dread da/d16/d26/fbf [0,4194304] 0 2024-03-22T07:16:02.506 INFO:tasks.workunit.client.0.smithi063.stdout:2/658: dread d0/f41 [0,4194304] 0 2024-03-22T07:16:02.509 INFO:tasks.workunit.client.0.smithi063.stdout:1/847: dwrite d4/df/d13/f40 [4194304,4194304] 0 2024-03-22T07:16:02.518 INFO:tasks.workunit.client.1.smithi069.stdout:5/331: read d1/d7/f22 [1265349,10854] 0 2024-03-22T07:16:02.518 INFO:tasks.workunit.client.1.smithi069.stdout:5/332: fsync d1/d6/d26/d33/f4e 0 2024-03-22T07:16:02.520 INFO:tasks.workunit.client.1.smithi069.stdout:5/333: creat d1/d7/d3d/d3f/f77 x:0 0 0 2024-03-22T07:16:02.521 INFO:tasks.workunit.client.0.smithi063.stdout:7/520: symlink d5/d16/d1b/d21/d62/d73/d1d/d29/d4a/lab 0 2024-03-22T07:16:02.521 INFO:tasks.workunit.client.0.smithi063.stdout:7/521: chown d5/d16/l54 2 1 2024-03-22T07:16:02.521 INFO:tasks.workunit.client.0.smithi063.stdout:9/659: dwrite d8/d15/d2f/f47 [0,4194304] 0 2024-03-22T07:16:02.522 INFO:tasks.workunit.client.0.smithi063.stdout:4/620: symlink d1/d5/d3c/d65/ld5 0 2024-03-22T07:16:02.522 INFO:tasks.workunit.client.0.smithi063.stdout:4/621: write d1/d5/d3c/d65/f69 [610804,48005] 0 2024-03-22T07:16:02.528 INFO:tasks.workunit.client.0.smithi063.stdout:4/622: creat d1/d5/dc2/fd6 x:0 0 0 2024-03-22T07:16:02.528 INFO:tasks.workunit.client.0.smithi063.stdout:4/623: truncate d1/d5/d43/d27/d2c/d3d/d54/d5f/d79/fb0 622518 0 2024-03-22T07:16:02.528 INFO:tasks.workunit.client.1.smithi069.stdout:9/405: dwrite d2/dd/f69 [0,4194304] 0 2024-03-22T07:16:02.535 INFO:tasks.workunit.client.0.smithi063.stdout:2/659: rename d0/de/d35/ccc to d0/de/d35/d36/cd4 0 2024-03-22T07:16:02.537 INFO:tasks.workunit.client.1.smithi069.stdout:0/513: dwrite d3/d7/d17/f4c [0,4194304] 0 2024-03-22T07:16:02.538 INFO:tasks.workunit.client.1.smithi069.stdout:0/514: creat d3/d7/d17/faa x:0 0 0 2024-03-22T07:16:02.540 INFO:tasks.workunit.client.0.smithi063.stdout:7/522: dread d5/da/f2c [0,4194304] 0 2024-03-22T07:16:02.542 INFO:tasks.workunit.client.0.smithi063.stdout:2/660: link d0/de/d1b/d63/d6e/f72 d0/de/d74/dca/fd5 0 2024-03-22T07:16:02.544 INFO:tasks.workunit.client.0.smithi063.stdout:0/907: dwrite f5 [0,4194304] 0 2024-03-22T07:16:02.544 INFO:tasks.workunit.client.0.smithi063.stdout:0/908: chown da/de/d17/d25/d3c/d43/dc9/ld7 4 1 2024-03-22T07:16:02.544 INFO:tasks.workunit.client.0.smithi063.stdout:1/848: rename d4/d71/d88/da1/c110 to d4/df/d13/d39/c122 0 2024-03-22T07:16:02.548 INFO:tasks.workunit.client.0.smithi063.stdout:7/523: unlink d5/da/d4e/l66 0 2024-03-22T07:16:02.548 INFO:tasks.workunit.client.0.smithi063.stdout:7/524: truncate d5/da/d5f/d33/d57/f67 833450 0 2024-03-22T07:16:02.548 INFO:tasks.workunit.client.0.smithi063.stdout:7/525: read d5/da/d5f/f5c [625659,62708] 0 2024-03-22T07:16:02.548 INFO:tasks.workunit.client.0.smithi063.stdout:7/526: creat d5/d16/d1b/d21/fac x:0 0 0 2024-03-22T07:16:02.549 INFO:tasks.workunit.client.0.smithi063.stdout:7/527: creat d5/da/d5f/d33/d57/fad x:0 0 0 2024-03-22T07:16:02.549 INFO:tasks.workunit.client.0.smithi063.stdout:7/528: read d5/da/d5f/f8a [984761,58212] 0 2024-03-22T07:16:02.551 INFO:tasks.workunit.client.0.smithi063.stdout:8/775: dwrite d0/d24/f35 [0,4194304] 0 2024-03-22T07:16:02.553 INFO:tasks.workunit.client.1.smithi069.stdout:8/516: dwrite d1/d1c/f22 [0,4194304] 0 2024-03-22T07:16:02.554 INFO:tasks.workunit.client.1.smithi069.stdout:8/517: rmdir d1/d1c 39 2024-03-22T07:16:02.554 INFO:tasks.workunit.client.1.smithi069.stdout:8/518: stat d1/d1c/d48/d5e/f79 0 2024-03-22T07:16:02.556 INFO:tasks.workunit.client.1.smithi069.stdout:8/519: creat d1/d1c/d48/d5e/fa5 x:0 0 0 2024-03-22T07:16:02.557 INFO:tasks.workunit.client.0.smithi063.stdout:7/529: symlink d5/da/d28/d4b/da8/lae 0 2024-03-22T07:16:02.557 INFO:tasks.workunit.client.0.smithi063.stdout:7/530: chown d5/da/d28/f9a 31592 1 2024-03-22T07:16:02.557 INFO:tasks.workunit.client.1.smithi069.stdout:8/520: rename d1/de/d6c/d4e/l55 to d1/de/d6c/d4e/d98/la6 0 2024-03-22T07:16:02.562 INFO:tasks.workunit.client.0.smithi063.stdout:9/660: dwrite d8/d19/d39/f8b [0,4194304] 0 2024-03-22T07:16:02.562 INFO:tasks.workunit.client.0.smithi063.stdout:2/661: rmdir d0/de/d58/d7f 39 2024-03-22T07:16:02.562 INFO:tasks.workunit.client.0.smithi063.stdout:2/662: write d0/de/d12/f1f [2625333,15728] 0 2024-03-22T07:16:02.563 INFO:tasks.workunit.client.0.smithi063.stdout:9/661: symlink d8/d19/d20/d22/d29/lcb 0 2024-03-22T07:16:02.563 INFO:tasks.workunit.client.0.smithi063.stdout:9/662: write d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/f7c [74677,110344] 0 2024-03-22T07:16:02.563 INFO:tasks.workunit.client.0.smithi063.stdout:9/663: creat d8/d19/d20/d22/d5d/d76/fcc x:0 0 0 2024-03-22T07:16:02.563 INFO:tasks.workunit.client.0.smithi063.stdout:9/664: readlink d8/d19/d20/d22/d29/lcb 0 2024-03-22T07:16:02.564 INFO:tasks.workunit.client.1.smithi069.stdout:4/493: sync 2024-03-22T07:16:02.564 INFO:tasks.workunit.client.1.smithi069.stdout:4/494: write d6/d4b/f92 [522521,106093] 0 2024-03-22T07:16:02.564 INFO:tasks.workunit.client.1.smithi069.stdout:4/495: chown d6/ca 188501688 1 2024-03-22T07:16:02.564 INFO:tasks.workunit.client.1.smithi069.stdout:4/496: chown d6/d12/f70 27 1 2024-03-22T07:16:02.564 INFO:tasks.workunit.client.1.smithi069.stdout:4/497: stat d6/lb 0 2024-03-22T07:16:02.565 INFO:tasks.workunit.client.0.smithi063.stdout:4/624: dwrite d1/d5/dc/fb8 [0,4194304] 0 2024-03-22T07:16:02.565 INFO:tasks.workunit.client.1.smithi069.stdout:9/406: dwrite d2/dd/d3c/d2e/f3f [0,4194304] 0 2024-03-22T07:16:02.565 INFO:tasks.workunit.client.1.smithi069.stdout:9/407: chown d2/dd/l68 13254503 1 2024-03-22T07:16:02.565 INFO:tasks.workunit.client.1.smithi069.stdout:9/408: chown d2/dd/d3c/d2e/l49 109328636 1 2024-03-22T07:16:02.566 INFO:tasks.workunit.client.0.smithi063.stdout:2/663: mkdir d0/de/d1b/d63/dd6 0 2024-03-22T07:16:02.567 INFO:tasks.workunit.client.0.smithi063.stdout:3/774: dwrite da/d16/d26/f48 [0,4194304] 0 2024-03-22T07:16:02.569 INFO:tasks.workunit.client.1.smithi069.stdout:5/334: dwrite d1/f1f [4194304,4194304] 0 2024-03-22T07:16:02.570 INFO:tasks.workunit.client.1.smithi069.stdout:4/498: symlink d6/d9b/d56/la6 0 2024-03-22T07:16:02.571 INFO:tasks.workunit.client.0.smithi063.stdout:9/665: creat d8/d19/d39/db5/fcd x:0 0 0 2024-03-22T07:16:02.571 INFO:tasks.workunit.client.0.smithi063.stdout:9/666: write d8/f10 [4628704,53348] 0 2024-03-22T07:16:02.574 INFO:tasks.workunit.client.0.smithi063.stdout:3/775: read da/d16/d26/f76 [136082,100395] 0 2024-03-22T07:16:02.576 INFO:tasks.workunit.client.0.smithi063.stdout:4/625: unlink d1/d5/f86 0 2024-03-22T07:16:02.577 INFO:tasks.workunit.client.1.smithi069.stdout:8/521: write d1/de/d20/f3c [3326231,71776] 0 2024-03-22T07:16:02.577 INFO:tasks.workunit.client.1.smithi069.stdout:8/522: write d1/f1d [5330103,87469] 0 2024-03-22T07:16:02.578 INFO:tasks.workunit.client.0.smithi063.stdout:3/776: creat da/d63/d65/d92/fff x:0 0 0 2024-03-22T07:16:02.578 INFO:tasks.workunit.client.0.smithi063.stdout:6/674: sync 2024-03-22T07:16:02.579 INFO:tasks.workunit.client.0.smithi063.stdout:3/777: symlink da/d16/d89/l100 0 2024-03-22T07:16:02.580 INFO:tasks.workunit.client.1.smithi069.stdout:9/409: dread d2/d51/f57 [0,4194304] 0 2024-03-22T07:16:02.581 INFO:tasks.workunit.client.1.smithi069.stdout:5/335: mkdir d1/d6/d26/d33/d5d/d72/d78 0 2024-03-22T07:16:02.582 INFO:tasks.workunit.client.0.smithi063.stdout:7/531: dread d5/d16/f2e [0,4194304] 0 2024-03-22T07:16:02.584 INFO:tasks.workunit.client.0.smithi063.stdout:7/532: link d5/da/d4e/l75 d5/d16/d43/laf 0 2024-03-22T07:16:02.584 INFO:tasks.workunit.client.0.smithi063.stdout:7/533: read d5/da/d5f/f5c [1018922,81469] 0 2024-03-22T07:16:02.585 INFO:tasks.workunit.client.1.smithi069.stdout:4/499: rename d6/d9b/c1b to d6/d9b/d1f/d3f/ca7 0 2024-03-22T07:16:02.587 INFO:tasks.workunit.client.0.smithi063.stdout:8/776: read d0/d1b/d28/d7b/d4a/f5a [972533,74082] 0 2024-03-22T07:16:02.588 INFO:tasks.workunit.client.1.smithi069.stdout:9/410: truncate d2/d51/f56 3369907 0 2024-03-22T07:16:02.588 INFO:tasks.workunit.client.1.smithi069.stdout:9/411: chown d2/dd/d3c/d2e/f6a 42871 1 2024-03-22T07:16:02.589 INFO:tasks.workunit.client.0.smithi063.stdout:7/534: rename d5/da/d5f/f84 to d5/da/d28/da7/fb0 0 2024-03-22T07:16:02.589 INFO:tasks.workunit.client.0.smithi063.stdout:7/535: chown d5/da/d28/da7/f41 105146 1 2024-03-22T07:16:02.589 INFO:tasks.workunit.client.0.smithi063.stdout:7/536: write d5/d38/f7b [890472,34830] 0 2024-03-22T07:16:02.590 INFO:tasks.workunit.client.1.smithi069.stdout:9/412: mkdir d2/d83 0 2024-03-22T07:16:02.590 INFO:tasks.workunit.client.1.smithi069.stdout:9/413: write d2/d51/f7a [298411,130164] 0 2024-03-22T07:16:02.591 INFO:tasks.workunit.client.1.smithi069.stdout:5/336: dread d1/d6/ff [0,4194304] 0 2024-03-22T07:16:02.591 INFO:tasks.workunit.client.1.smithi069.stdout:5/337: chown d1/d28/f4d 1805945 1 2024-03-22T07:16:02.591 INFO:tasks.workunit.client.1.smithi069.stdout:5/338: truncate d1/d6/f6f 198261 0 2024-03-22T07:16:02.591 INFO:tasks.workunit.client.1.smithi069.stdout:5/339: creat d1/d6/d21/f79 x:0 0 0 2024-03-22T07:16:02.591 INFO:tasks.workunit.client.1.smithi069.stdout:5/340: chown d1/d6/d21/f79 292202717 1 2024-03-22T07:16:02.592 INFO:tasks.workunit.client.1.smithi069.stdout:9/414: getdents d2/dd/d3c/d2e 0 2024-03-22T07:16:02.592 INFO:tasks.workunit.client.1.smithi069.stdout:9/415: fdatasync d2/d51/f5e 0 2024-03-22T07:16:02.592 INFO:tasks.workunit.client.1.smithi069.stdout:9/416: dread - d2/d51/f59 zero size 2024-03-22T07:16:02.592 INFO:tasks.workunit.client.0.smithi063.stdout:8/777: rename d0/d24/d33/d4c/l8c to d0/d26/d2d/d59/dba/df4/l10a 0 2024-03-22T07:16:02.592 INFO:tasks.workunit.client.0.smithi063.stdout:8/778: write d0/d24/d33/d4c/f7d [787784,121015] 0 2024-03-22T07:16:02.593 INFO:tasks.workunit.client.0.smithi063.stdout:9/667: write d8/d19/d20/d22/d29/d31/f86 [637293,4165] 0 2024-03-22T07:16:02.594 INFO:tasks.workunit.client.1.smithi069.stdout:5/341: symlink d1/d6/d26/d33/d5d/d72/d78/l7a 0 2024-03-22T07:16:02.594 INFO:tasks.workunit.client.1.smithi069.stdout:5/342: write d1/d6/f6f [959926,36105] 0 2024-03-22T07:16:02.596 INFO:tasks.workunit.client.1.smithi069.stdout:5/343: creat d1/d10/d60/f7b x:0 0 0 2024-03-22T07:16:02.596 INFO:tasks.workunit.client.0.smithi063.stdout:4/626: dread d1/d5/d3c/f47 [4194304,4194304] 0 2024-03-22T07:16:02.597 INFO:tasks.workunit.client.0.smithi063.stdout:4/627: symlink d1/d5/d43/d27/d2c/d4a/d80/ld7 0 2024-03-22T07:16:02.598 INFO:tasks.workunit.client.1.smithi069.stdout:5/344: link d1/d7/d8/d43/l56 d1/d28/l7c 0 2024-03-22T07:16:02.598 INFO:tasks.workunit.client.1.smithi069.stdout:5/345: write d1/d7/d8/f18 [3135603,127617] 0 2024-03-22T07:16:02.598 INFO:tasks.workunit.client.0.smithi063.stdout:4/628: mknod d1/d5/dc/d95/d59/d9c/dac/dc9/cd8 0 2024-03-22T07:16:02.599 INFO:tasks.workunit.client.1.smithi069.stdout:5/346: mkdir d1/d6/d26/d7d 0 2024-03-22T07:16:02.599 INFO:tasks.workunit.client.1.smithi069.stdout:5/347: dread - d1/d6/d21/f63 zero size 2024-03-22T07:16:02.600 INFO:tasks.workunit.client.0.smithi063.stdout:4/629: creat d1/d5/d43/d27/d2c/d4a/d80/fd9 x:0 0 0 2024-03-22T07:16:02.601 INFO:tasks.workunit.client.0.smithi063.stdout:3/778: dread da/d16/f38 [0,4194304] 0 2024-03-22T07:16:02.601 INFO:tasks.workunit.client.0.smithi063.stdout:3/779: unlink da/d16/d3f/dc7/fd5 0 2024-03-22T07:16:02.602 INFO:tasks.workunit.client.0.smithi063.stdout:3/780: symlink da/d16/d89/l101 0 2024-03-22T07:16:02.605 INFO:tasks.workunit.client.0.smithi063.stdout:7/537: dread d5/d16/d1b/d21/d62/d73/d1d/d4f/f95 [0,4194304] 0 2024-03-22T07:16:02.605 INFO:tasks.workunit.client.0.smithi063.stdout:7/538: write d5/d16/d1b/d21/d62/d73/d1d/d29/f6a [927875,66168] 0 2024-03-22T07:16:02.606 INFO:tasks.workunit.client.1.smithi069.stdout:5/348: dread d1/d6/d26/d33/f38 [0,4194304] 0 2024-03-22T07:16:02.606 INFO:tasks.workunit.client.1.smithi069.stdout:5/349: creat d1/d28/f7e x:0 0 0 2024-03-22T07:16:02.606 INFO:tasks.workunit.client.0.smithi063.stdout:6/675: dread d3/d15/f62 [0,4194304] 0 2024-03-22T07:16:02.606 INFO:tasks.workunit.client.0.smithi063.stdout:6/676: chown d3/d6/l9e 810642304 1 2024-03-22T07:16:02.606 INFO:tasks.workunit.client.0.smithi063.stdout:6/677: creat d3/d15/d16/d60/d6d/d4c/fe0 x:0 0 0 2024-03-22T07:16:02.606 INFO:tasks.workunit.client.0.smithi063.stdout:3/781: rmdir da/d16/d89/d8c/dfb 0 2024-03-22T07:16:02.606 INFO:tasks.workunit.client.1.smithi069.stdout:5/350: mknod d1/d7/d8/c7f 0 2024-03-22T07:16:02.607 INFO:tasks.workunit.client.0.smithi063.stdout:6/678: link d3/f82 d3/d6/d2a/d38/da5/fe1 0 2024-03-22T07:16:02.608 INFO:tasks.workunit.client.0.smithi063.stdout:6/679: creat d3/d15/d16/d69/d8c/fe2 x:0 0 0 2024-03-22T07:16:02.608 INFO:tasks.workunit.client.0.smithi063.stdout:6/680: chown d3/d15/d16/d60/d6d/d4e/l92 394 1 2024-03-22T07:16:02.609 INFO:tasks.workunit.client.0.smithi063.stdout:4/630: read d1/d5/dc/d95/d59/f71 [2313904,114295] 0 2024-03-22T07:16:02.609 INFO:tasks.workunit.client.0.smithi063.stdout:6/681: creat d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/fe3 x:0 0 0 2024-03-22T07:16:02.610 INFO:tasks.workunit.client.0.smithi063.stdout:6/682: read - d3/d6/d2a/d38/da5/fdc zero size 2024-03-22T07:16:02.613 INFO:tasks.workunit.client.1.smithi069.stdout:9/417: write d2/dd/f26 [2606120,1687] 0 2024-03-22T07:16:02.614 INFO:tasks.workunit.client.1.smithi069.stdout:9/418: write d2/dd/f69 [4746249,97623] 0 2024-03-22T07:16:02.614 INFO:tasks.workunit.client.1.smithi069.stdout:9/419: fsync d2/dd/d3c/d2e/f32 0 2024-03-22T07:16:02.615 INFO:tasks.workunit.client.0.smithi063.stdout:4/631: rename d1/d5/d43/d27/d2c/d3d/d54/d5f to d1/d5/dc/dda 0 2024-03-22T07:16:02.615 INFO:tasks.workunit.client.0.smithi063.stdout:9/668: write d8/d19/d20/d22/d5d/f64 [1390775,119387] 0 2024-03-22T07:16:02.616 INFO:tasks.workunit.client.1.smithi069.stdout:9/420: mkdir d2/dd/d3c/d2e/d84 0 2024-03-22T07:16:02.617 INFO:tasks.workunit.client.0.smithi063.stdout:7/539: dread d5/d16/d1b/f80 [0,4194304] 0 2024-03-22T07:16:02.617 INFO:tasks.workunit.client.0.smithi063.stdout:7/540: write d5/da/d28/da7/fb0 [3333463,11104] 0 2024-03-22T07:16:02.617 INFO:tasks.workunit.client.0.smithi063.stdout:7/541: fdatasync d5/d16/d1b/f52 0 2024-03-22T07:16:02.617 INFO:tasks.workunit.client.0.smithi063.stdout:7/542: fdatasync d5/d16/d1b/d21/d62/d73/d1d/d29/f47 0 2024-03-22T07:16:02.618 INFO:tasks.workunit.client.0.smithi063.stdout:4/632: mkdir d1/d5/d43/d27/d2c/d3d/d54/ddb 0 2024-03-22T07:16:02.618 INFO:tasks.workunit.client.1.smithi069.stdout:9/421: truncate d2/dd/d3c/d2e/f30 2172024 0 2024-03-22T07:16:02.620 INFO:tasks.workunit.client.1.smithi069.stdout:4/500: dread d6/d12/f41 [0,4194304] 0 2024-03-22T07:16:02.620 INFO:tasks.workunit.client.0.smithi063.stdout:7/543: mknod d5/d16/d1b/d21/d85/cb1 0 2024-03-22T07:16:02.621 INFO:tasks.workunit.client.0.smithi063.stdout:7/544: write d5/d16/f88 [840164,79889] 0 2024-03-22T07:16:02.621 INFO:tasks.workunit.client.0.smithi063.stdout:6/683: write d3/d6/d2a/d38/da5/f63 [3041708,35923] 0 2024-03-22T07:16:02.621 INFO:tasks.workunit.client.0.smithi063.stdout:6/684: fdatasync d3/d15/d16/d60/d6d/d4e/fa7 0 2024-03-22T07:16:02.622 INFO:tasks.workunit.client.0.smithi063.stdout:4/633: link d1/d5/d43/f14 d1/d5/d3c/d8f/dc0/dcb/fdc 0 2024-03-22T07:16:02.623 INFO:tasks.workunit.client.1.smithi069.stdout:5/351: read d1/f1f [8342713,11432] 0 2024-03-22T07:16:02.623 INFO:tasks.workunit.client.1.smithi069.stdout:4/501: creat d6/d9b/d56/fa8 x:0 0 0 2024-03-22T07:16:02.623 INFO:tasks.workunit.client.1.smithi069.stdout:8/523: dwrite d1/d1c/d48/d56/f57 [0,4194304] 0 2024-03-22T07:16:02.623 INFO:tasks.workunit.client.1.smithi069.stdout:8/524: creat d1/de/d6c/d8e/fa7 x:0 0 0 2024-03-22T07:16:02.624 INFO:tasks.workunit.client.1.smithi069.stdout:4/502: unlink d6/dc/c24 0 2024-03-22T07:16:02.625 INFO:tasks.workunit.client.1.smithi069.stdout:4/503: dread d6/dc/d1d/d55/f9c [0,4194304] 0 2024-03-22T07:16:02.626 INFO:tasks.workunit.client.0.smithi063.stdout:7/545: creat d5/da/d28/da7/fb2 x:0 0 0 2024-03-22T07:16:02.629 INFO:tasks.workunit.client.0.smithi063.stdout:9/669: write d8/d19/d20/d22/d29/d31/d4b/f4d [2997292,111265] 0 2024-03-22T07:16:02.629 INFO:tasks.workunit.client.0.smithi063.stdout:6/685: symlink d3/d15/d16/d60/d6d/d4e/d5a/le4 0 2024-03-22T07:16:02.629 INFO:tasks.workunit.client.1.smithi069.stdout:8/525: mkdir d1/d1c/d3e/d88/da8 0 2024-03-22T07:16:02.629 INFO:tasks.workunit.client.1.smithi069.stdout:8/526: chown d1/d1c/f22 1015 1 2024-03-22T07:16:02.630 INFO:tasks.workunit.client.0.smithi063.stdout:0/909: dwrite da/de/d17/d25/d3c/d64/d78/f10a [0,4194304] 0 2024-03-22T07:16:02.630 INFO:tasks.workunit.client.1.smithi069.stdout:0/515: dwrite d3/d7/d15/d1f/f2b [0,4194304] 0 2024-03-22T07:16:02.630 INFO:tasks.workunit.client.1.smithi069.stdout:0/516: dread - d3/d7/f8e zero size 2024-03-22T07:16:02.630 INFO:tasks.workunit.client.1.smithi069.stdout:0/517: chown d3/d7/d5e/l81 17106 1 2024-03-22T07:16:02.630 INFO:tasks.workunit.client.1.smithi069.stdout:0/518: write d3/d7/d15/d1f/f26 [4717211,54726] 0 2024-03-22T07:16:02.631 INFO:tasks.workunit.client.0.smithi063.stdout:4/634: unlink d1/d5/d43/d27/d2c/fbe 0 2024-03-22T07:16:02.631 INFO:tasks.workunit.client.0.smithi063.stdout:1/849: dwrite d4/df/d13/d39/d6f/dd9/fe9 [0,4194304] 0 2024-03-22T07:16:02.631 INFO:tasks.workunit.client.0.smithi063.stdout:3/782: dwrite da/f57 [0,4194304] 0 2024-03-22T07:16:02.632 INFO:tasks.workunit.client.1.smithi069.stdout:3/436: sync 2024-03-22T07:16:02.632 INFO:tasks.workunit.client.1.smithi069.stdout:1/386: sync 2024-03-22T07:16:02.632 INFO:tasks.workunit.client.1.smithi069.stdout:3/437: readlink d5/d33/l74 0 2024-03-22T07:16:02.632 INFO:tasks.workunit.client.1.smithi069.stdout:1/387: write d1/d13/f2d [1806165,29124] 0 2024-03-22T07:16:02.633 INFO:tasks.workunit.client.1.smithi069.stdout:1/388: stat d1/f48 0 2024-03-22T07:16:02.633 INFO:tasks.workunit.client.0.smithi063.stdout:7/546: mkdir d5/da/d5f/d33/d57/db3 0 2024-03-22T07:16:02.635 INFO:tasks.workunit.client.0.smithi063.stdout:9/670: mknod d8/d19/d20/d22/d5d/d76/dc3/cce 0 2024-03-22T07:16:02.635 INFO:tasks.workunit.client.0.smithi063.stdout:9/671: chown d8/d17/f1d 20657304 1 2024-03-22T07:16:02.639 INFO:tasks.workunit.client.0.smithi063.stdout:6/686: link d3/d15/d16/d60/d6d/d4e/d5a/fba d3/d6/d2a/d38/da5/d32/dbf/fe5 0 2024-03-22T07:16:02.640 INFO:tasks.workunit.client.1.smithi069.stdout:1/389: dread d1/d13/f27 [0,4194304] 0 2024-03-22T07:16:02.640 INFO:tasks.workunit.client.1.smithi069.stdout:0/519: creat d3/d7/d15/d29/d5a/d99/fab x:0 0 0 2024-03-22T07:16:02.641 INFO:tasks.workunit.client.0.smithi063.stdout:0/910: symlink da/de/d60/d56/d8b/d123/l128 0 2024-03-22T07:16:02.641 INFO:tasks.workunit.client.0.smithi063.stdout:4/635: write d1/d5/d43/d27/d2c/f9f [2966549,11581] 0 2024-03-22T07:16:02.641 INFO:tasks.workunit.client.0.smithi063.stdout:4/636: readlink d1/l36 0 2024-03-22T07:16:02.642 INFO:tasks.workunit.client.1.smithi069.stdout:5/352: rmdir d1/d6/d21 39 2024-03-22T07:16:02.642 INFO:tasks.workunit.client.1.smithi069.stdout:4/504: dread d6/dc/d23/f93 [0,4194304] 0 2024-03-22T07:16:02.642 INFO:tasks.workunit.client.1.smithi069.stdout:4/505: creat d6/d16/d68/fa9 x:0 0 0 2024-03-22T07:16:02.642 INFO:tasks.workunit.client.1.smithi069.stdout:4/506: write d6/d16/f71 [260111,113616] 0 2024-03-22T07:16:02.642 INFO:tasks.workunit.client.0.smithi063.stdout:3/783: dread da/d16/d26/d69/f6b [0,4194304] 0 2024-03-22T07:16:02.643 INFO:tasks.workunit.client.0.smithi063.stdout:3/784: fdatasync da/d16/d3f/ff7 0 2024-03-22T07:16:02.643 INFO:tasks.workunit.client.0.smithi063.stdout:1/850: truncate d4/df/d13/d2b/f84 1064167 0 2024-03-22T07:16:02.644 INFO:tasks.workunit.client.1.smithi069.stdout:3/438: rmdir d5/d9/db 39 2024-03-22T07:16:02.644 INFO:tasks.workunit.client.1.smithi069.stdout:3/439: truncate d5/d9/f76 603391 0 2024-03-22T07:16:02.644 INFO:tasks.workunit.client.1.smithi069.stdout:1/390: dread d1/d13/f58 [0,4194304] 0 2024-03-22T07:16:02.644 INFO:tasks.workunit.client.1.smithi069.stdout:1/391: write d1/d25/d41/d1e/f51 [2005015,67018] 0 2024-03-22T07:16:02.644 INFO:tasks.workunit.client.1.smithi069.stdout:1/392: dread - d1/d25/d41/f4b zero size 2024-03-22T07:16:02.645 INFO:tasks.workunit.client.0.smithi063.stdout:7/547: creat d5/d16/d1b/d21/d62/d73/d1d/d29/fb4 x:0 0 0 2024-03-22T07:16:02.645 INFO:tasks.workunit.client.1.smithi069.stdout:5/353: dread d1/d6/d26/d33/f4e [0,4194304] 0 2024-03-22T07:16:02.647 INFO:tasks.workunit.client.0.smithi063.stdout:6/687: link d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/f72 d3/d15/da3/fe6 0 2024-03-22T07:16:02.649 INFO:tasks.workunit.client.1.smithi069.stdout:5/354: dread d1/d6/d26/d3a/f6e [0,4194304] 0 2024-03-22T07:16:02.650 INFO:tasks.workunit.client.0.smithi063.stdout:0/911: mkdir da/de/d39/d129 0 2024-03-22T07:16:02.650 INFO:tasks.workunit.client.0.smithi063.stdout:0/912: write da/de/d39/fe6 [1047692,14461] 0 2024-03-22T07:16:02.653 INFO:tasks.workunit.client.1.smithi069.stdout:1/393: rename d1/d2a/d50/d4c/d63 to d1/d25/d41/d7a 0 2024-03-22T07:16:02.654 INFO:tasks.workunit.client.0.smithi063.stdout:9/672: dread f2 [0,4194304] 0 2024-03-22T07:16:02.654 INFO:tasks.workunit.client.0.smithi063.stdout:9/673: write d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/f85 [1094221,75815] 0 2024-03-22T07:16:02.656 INFO:tasks.workunit.client.0.smithi063.stdout:3/785: read da/d63/d65/d83/fc8 [3172093,91939] 0 2024-03-22T07:16:02.656 INFO:tasks.workunit.client.0.smithi063.stdout:4/637: symlink d1/d5/d43/d27/d2c/ldd 0 2024-03-22T07:16:02.657 INFO:tasks.workunit.client.1.smithi069.stdout:0/520: rmdir d3/d7/d15/d1f 39 2024-03-22T07:16:02.658 INFO:tasks.workunit.client.1.smithi069.stdout:4/507: mkdir d6/daa 0 2024-03-22T07:16:02.658 INFO:tasks.workunit.client.1.smithi069.stdout:4/508: write d6/d12/d18/f4f [3362076,98902] 0 2024-03-22T07:16:02.659 INFO:tasks.workunit.client.1.smithi069.stdout:4/509: chown d6/d12/l66 548213140 1 2024-03-22T07:16:02.659 INFO:tasks.workunit.client.1.smithi069.stdout:4/510: dread - d6/d16/f7e zero size 2024-03-22T07:16:02.659 INFO:tasks.workunit.client.0.smithi063.stdout:1/851: mkdir d4/d14/d123 0 2024-03-22T07:16:02.660 INFO:tasks.workunit.client.0.smithi063.stdout:1/852: dread - d4/df/d13/d2b/d3a/df6/f120 zero size 2024-03-22T07:16:02.661 INFO:tasks.workunit.client.0.smithi063.stdout:0/913: mkdir da/de/d60/d56/d12a 0 2024-03-22T07:16:02.662 INFO:tasks.workunit.client.1.smithi069.stdout:8/527: dwrite d1/d1c/d3e/d88/f8f [0,4194304] 0 2024-03-22T07:16:02.662 INFO:tasks.workunit.client.1.smithi069.stdout:8/528: chown d1/d1c/d48/d5e/l6b 6 1 2024-03-22T07:16:02.662 INFO:tasks.workunit.client.1.smithi069.stdout:8/529: creat d1/de/d6c/d8e/fa9 x:0 0 0 2024-03-22T07:16:02.663 INFO:tasks.workunit.client.1.smithi069.stdout:5/355: rename d1/f1f to d1/d6/d26/d33/f80 0 2024-03-22T07:16:02.666 INFO:tasks.workunit.client.1.smithi069.stdout:4/511: mknod d6/d9b/d1f/d58/cab 0 2024-03-22T07:16:02.667 INFO:tasks.workunit.client.0.smithi063.stdout:3/786: truncate da/d16/d3f/f44 4133 0 2024-03-22T07:16:02.668 INFO:tasks.workunit.client.0.smithi063.stdout:3/787: write da/fb [2005660,41265] 0 2024-03-22T07:16:02.671 INFO:tasks.workunit.client.0.smithi063.stdout:4/638: rename d1/d5/dc/fb8 to d1/d5/dc/d95/fde 0 2024-03-22T07:16:02.671 INFO:tasks.workunit.client.0.smithi063.stdout:4/639: fdatasync d1/d5/dc/d95/da4/fce 0 2024-03-22T07:16:02.672 INFO:tasks.workunit.client.0.smithi063.stdout:4/640: write d1/d5/dc/f41 [2472972,105541] 0 2024-03-22T07:16:02.672 INFO:tasks.workunit.client.1.smithi069.stdout:0/521: dread d3/f13 [0,4194304] 0 2024-03-22T07:16:02.672 INFO:tasks.workunit.client.1.smithi069.stdout:0/522: write d3/d7/d15/fa3 [1023381,44416] 0 2024-03-22T07:16:02.673 INFO:tasks.workunit.client.0.smithi063.stdout:4/641: read d1/d5/d43/f31 [602421,100928] 0 2024-03-22T07:16:02.673 INFO:tasks.workunit.client.0.smithi063.stdout:4/642: truncate d1/d5/d3c/f83 4400747 0 2024-03-22T07:16:02.675 INFO:tasks.workunit.client.1.smithi069.stdout:8/530: rename d1/d1c/d3e/l3f to d1/d93/laa 0 2024-03-22T07:16:02.675 INFO:tasks.workunit.client.0.smithi063.stdout:0/914: truncate da/de/d17/f1a 133435 0 2024-03-22T07:16:02.675 INFO:tasks.workunit.client.0.smithi063.stdout:0/915: truncate da/de/d17/f3a 3447419 0 2024-03-22T07:16:02.675 INFO:tasks.workunit.client.0.smithi063.stdout:0/916: dread - da/de/d60/ffa zero size 2024-03-22T07:16:02.675 INFO:tasks.workunit.client.0.smithi063.stdout:3/788: dread da/d63/d65/d4b/f85 [0,4194304] 0 2024-03-22T07:16:02.675 INFO:tasks.workunit.client.1.smithi069.stdout:5/356: unlink d1/d6/d26/l61 0 2024-03-22T07:16:02.676 INFO:tasks.workunit.client.0.smithi063.stdout:9/674: link d8/d19/d2b/c40 d8/d17/ccf 0 2024-03-22T07:16:02.676 INFO:tasks.workunit.client.1.smithi069.stdout:0/523: getdents d3/d7/d15/d29/d5a/d5d/d98 0 2024-03-22T07:16:02.682 INFO:tasks.workunit.client.0.smithi063.stdout:1/853: mknod d4/df/d13/d2b/d6c/da8/c124 0 2024-03-22T07:16:02.682 INFO:tasks.workunit.client.0.smithi063.stdout:1/854: write d4/d14/f46 [734080,5525] 0 2024-03-22T07:16:02.683 INFO:tasks.workunit.client.1.smithi069.stdout:4/512: read d6/f30 [3714922,52833] 0 2024-03-22T07:16:02.683 INFO:tasks.workunit.client.1.smithi069.stdout:4/513: readlink d6/dc/l63 0 2024-03-22T07:16:02.683 INFO:tasks.workunit.client.0.smithi063.stdout:0/917: mknod da/de/d17/d25/d3c/d64/d110/c12b 0 2024-03-22T07:16:02.685 INFO:tasks.workunit.client.1.smithi069.stdout:0/524: creat d3/d7/d5e/fac x:0 0 0 2024-03-22T07:16:02.685 INFO:tasks.workunit.client.1.smithi069.stdout:4/514: dread d6/dc/d1d/d55/f9c [0,4194304] 0 2024-03-22T07:16:02.686 INFO:tasks.workunit.client.1.smithi069.stdout:7/435: sync 2024-03-22T07:16:02.686 INFO:tasks.workunit.client.1.smithi069.stdout:6/368: sync 2024-03-22T07:16:02.686 INFO:tasks.workunit.client.1.smithi069.stdout:2/313: sync 2024-03-22T07:16:02.686 INFO:tasks.workunit.client.1.smithi069.stdout:7/436: fdatasync d9/d76/d57/d44/d53/d74/d30/d3d/f7b 0 2024-03-22T07:16:02.686 INFO:tasks.workunit.client.1.smithi069.stdout:7/437: read - d9/db/d48/d4b/f97 zero size 2024-03-22T07:16:02.686 INFO:tasks.workunit.client.1.smithi069.stdout:2/314: truncate d18/d4d/d3e/d40/f5c 934700 0 2024-03-22T07:16:02.686 INFO:tasks.workunit.client.1.smithi069.stdout:7/438: chown f8 306 1 2024-03-22T07:16:02.686 INFO:tasks.workunit.client.1.smithi069.stdout:6/369: write d4/dd/d41/f51 [5087802,86819] 0 2024-03-22T07:16:02.687 INFO:tasks.workunit.client.1.smithi069.stdout:4/515: creat d6/d9b/d1f/d58/d7c/fac x:0 0 0 2024-03-22T07:16:02.687 INFO:tasks.workunit.client.0.smithi063.stdout:9/675: mknod d8/d19/d20/d22/d29/d31/d4b/cd0 0 2024-03-22T07:16:02.687 INFO:tasks.workunit.client.0.smithi063.stdout:1/855: creat d4/df/d13/d39/d63/f125 x:0 0 0 2024-03-22T07:16:02.688 INFO:tasks.workunit.client.0.smithi063.stdout:0/918: rmdir da/de/d39/d129 0 2024-03-22T07:16:02.688 INFO:tasks.workunit.client.1.smithi069.stdout:7/439: mkdir d9/d76/d57/d44/d53/d74/d30/d99 0 2024-03-22T07:16:02.689 INFO:tasks.workunit.client.1.smithi069.stdout:8/531: write d1/d1c/d3e/d88/f8f [36832,59327] 0 2024-03-22T07:16:02.689 INFO:tasks.workunit.client.1.smithi069.stdout:2/315: mkdir d18/d4d/d27/d72 0 2024-03-22T07:16:02.690 INFO:tasks.workunit.client.0.smithi063.stdout:3/789: dread da/d16/d89/d8c/dc1/f93 [0,4194304] 0 2024-03-22T07:16:02.690 INFO:tasks.workunit.client.0.smithi063.stdout:1/856: link d4/cb8 d4/d5a/c126 0 2024-03-22T07:16:02.691 INFO:tasks.workunit.client.0.smithi063.stdout:1/857: creat d4/df/d13/d2b/d3a/df6/f127 x:0 0 0 2024-03-22T07:16:02.691 INFO:tasks.workunit.client.1.smithi069.stdout:6/370: symlink d4/dd/d12/l71 0 2024-03-22T07:16:02.691 INFO:tasks.workunit.client.0.smithi063.stdout:3/790: link da/d63/d65/d81/d9b/l9d da/d16/d89/d8c/dc1/l102 0 2024-03-22T07:16:02.693 INFO:tasks.workunit.client.1.smithi069.stdout:4/516: mknod d6/d9b/d1f/d58/cad 0 2024-03-22T07:16:02.693 INFO:tasks.workunit.client.1.smithi069.stdout:4/517: read d6/dc/ff [3278771,126530] 0 2024-03-22T07:16:02.693 INFO:tasks.workunit.client.1.smithi069.stdout:4/518: chown d6/d12/d18/d8b/fa4 1510455893 1 2024-03-22T07:16:02.693 INFO:tasks.workunit.client.1.smithi069.stdout:4/519: chown d6/dc/l44 209273 1 2024-03-22T07:16:02.693 INFO:tasks.workunit.client.1.smithi069.stdout:1/394: dwrite d1/d13/d36/f6b [0,4194304] 0 2024-03-22T07:16:02.694 INFO:tasks.workunit.client.0.smithi063.stdout:7/548: dwrite d5/d16/f2e [4194304,4194304] 0 2024-03-22T07:16:02.694 INFO:tasks.workunit.client.0.smithi063.stdout:7/549: chown d5/da/d28/da7/fb2 302011763 1 2024-03-22T07:16:02.694 INFO:tasks.workunit.client.0.smithi063.stdout:1/858: symlink d4/d14/d123/l128 0 2024-03-22T07:16:02.695 INFO:tasks.workunit.client.1.smithi069.stdout:7/440: creat d9/d42/d8a/f9a x:0 0 0 2024-03-22T07:16:02.696 INFO:tasks.workunit.client.0.smithi063.stdout:1/859: mknod d4/df/d13/d2b/d6c/d85/d9e/c129 0 2024-03-22T07:16:02.697 INFO:tasks.workunit.client.0.smithi063.stdout:6/688: dwrite d3/d15/d16/f59 [0,4194304] 0 2024-03-22T07:16:02.697 INFO:tasks.workunit.client.0.smithi063.stdout:6/689: truncate d3/d6/d2a/d38/da5/fdc 256457 0 2024-03-22T07:16:02.697 INFO:tasks.workunit.client.0.smithi063.stdout:6/690: write d3/d15/d16/f9a [1866550,82226] 0 2024-03-22T07:16:02.697 INFO:tasks.workunit.client.0.smithi063.stdout:7/550: write d5/d38/f7b [423825,121249] 0 2024-03-22T07:16:02.697 INFO:tasks.workunit.client.0.smithi063.stdout:1/860: mkdir d4/df/d13/d2b/d6c/d85/d12a 0 2024-03-22T07:16:02.699 INFO:tasks.workunit.client.1.smithi069.stdout:8/532: symlink d1/d1c/lab 0 2024-03-22T07:16:02.699 INFO:tasks.workunit.client.1.smithi069.stdout:8/533: stat d1/de 0 2024-03-22T07:16:02.699 INFO:tasks.workunit.client.1.smithi069.stdout:8/534: creat d1/de/d6c/d4e/d5b/d82/fac x:0 0 0 2024-03-22T07:16:02.699 INFO:tasks.workunit.client.1.smithi069.stdout:8/535: readlink d1/de/d20/l42 0 2024-03-22T07:16:02.700 INFO:tasks.workunit.client.0.smithi063.stdout:6/691: mkdir d3/d15/d16/d60/d6d/d4c/de7 0 2024-03-22T07:16:02.700 INFO:tasks.workunit.client.0.smithi063.stdout:7/551: mkdir d5/d16/d43/d78/db5 0 2024-03-22T07:16:02.701 INFO:tasks.workunit.client.1.smithi069.stdout:0/525: write d3/d7/f1e [1539418,17414] 0 2024-03-22T07:16:02.701 INFO:tasks.workunit.client.1.smithi069.stdout:3/440: dwrite d5/d9/d26/d3f/d35/f58 [0,4194304] 0 2024-03-22T07:16:02.701 INFO:tasks.workunit.client.1.smithi069.stdout:3/441: fsync d5/d9/d26/d36/f52 0 2024-03-22T07:16:02.701 INFO:tasks.workunit.client.1.smithi069.stdout:0/526: fdatasync d3/d7/d15/d29/d5a/d84/f60 0 2024-03-22T07:16:02.701 INFO:tasks.workunit.client.1.smithi069.stdout:9/422: dread d2/dd/d3c/d2e/f30 [0,4194304] 0 2024-03-22T07:16:02.702 INFO:tasks.workunit.client.0.smithi063.stdout:1/861: rename d4/f38 to d4/d71/d88/da1/ddb/f12b 0 2024-03-22T07:16:02.704 INFO:tasks.workunit.client.0.smithi063.stdout:6/692: mkdir d3/d15/d16/d60/d6d/d4c/de8 0 2024-03-22T07:16:02.704 INFO:tasks.workunit.client.0.smithi063.stdout:6/693: dread - d3/d15/d16/d69/d8c/fe2 zero size 2024-03-22T07:16:02.704 INFO:tasks.workunit.client.1.smithi069.stdout:2/316: getdents d18/d4d 0 2024-03-22T07:16:02.706 INFO:tasks.workunit.client.1.smithi069.stdout:6/371: mkdir d4/dd/d12/d2b/d72 0 2024-03-22T07:16:02.706 INFO:tasks.workunit.client.1.smithi069.stdout:4/520: getdents d6/d12 0 2024-03-22T07:16:02.706 INFO:tasks.workunit.client.1.smithi069.stdout:4/521: fsync d6/d4b/f97 0 2024-03-22T07:16:02.707 INFO:tasks.workunit.client.0.smithi063.stdout:7/552: symlink d5/d16/d1b/d21/d85/lb6 0 2024-03-22T07:16:02.707 INFO:tasks.workunit.client.0.smithi063.stdout:7/553: write d5/da/d5f/d33/d57/fad [720564,88160] 0 2024-03-22T07:16:02.708 INFO:tasks.workunit.client.0.smithi063.stdout:7/554: dread - d5/da/d28/da7/fb2 zero size 2024-03-22T07:16:02.708 INFO:tasks.workunit.client.0.smithi063.stdout:1/862: dread d4/df/d13/f25 [0,4194304] 0 2024-03-22T07:16:02.709 INFO:tasks.workunit.client.1.smithi069.stdout:1/395: mknod d1/d25/d41/c7b 0 2024-03-22T07:16:02.710 INFO:tasks.workunit.client.1.smithi069.stdout:7/441: read d9/d76/d57/d44/d53/f6d [2060700,29785] 0 2024-03-22T07:16:02.711 INFO:tasks.workunit.client.0.smithi063.stdout:6/694: unlink d3/d6/d2a/fb8 0 2024-03-22T07:16:02.714 INFO:tasks.workunit.client.1.smithi069.stdout:3/442: unlink d5/d9/db/f7d 0 2024-03-22T07:16:02.715 INFO:tasks.workunit.client.0.smithi063.stdout:7/555: creat d5/d16/d1b/d21/d62/d8d/d98/fb7 x:0 0 0 2024-03-22T07:16:02.715 INFO:tasks.workunit.client.0.smithi063.stdout:7/556: chown d5/da/d5f/f97 1301436 1 2024-03-22T07:16:02.715 INFO:tasks.workunit.client.0.smithi063.stdout:4/643: dwrite d1/d5/dc/dda/d79/fc7 [0,4194304] 0 2024-03-22T07:16:02.715 INFO:tasks.workunit.client.0.smithi063.stdout:4/644: dread - d1/d5/dc/d95/da4/fce zero size 2024-03-22T07:16:02.715 INFO:tasks.workunit.client.0.smithi063.stdout:4/645: truncate d1/d5/dc/ff 5133483 0 2024-03-22T07:16:02.715 INFO:tasks.workunit.client.0.smithi063.stdout:4/646: truncate d1/d5/d43/f8c 531201 0 2024-03-22T07:16:02.718 INFO:tasks.workunit.client.0.smithi063.stdout:1/863: rmdir d4/d71/d88/da1 39 2024-03-22T07:16:02.718 INFO:tasks.workunit.client.0.smithi063.stdout:1/864: stat d4/df/d13/d2b/l115 0 2024-03-22T07:16:02.719 INFO:tasks.workunit.client.0.smithi063.stdout:3/791: write da/f52 [2514424,57634] 0 2024-03-22T07:16:02.719 INFO:tasks.workunit.client.0.smithi063.stdout:3/792: chown da/d16/d26/d69/dae 870430 1 2024-03-22T07:16:02.719 INFO:tasks.workunit.client.0.smithi063.stdout:3/793: stat da/d63/d65/d4b/dfa 0 2024-03-22T07:16:02.720 INFO:tasks.workunit.client.1.smithi069.stdout:9/423: creat d2/dd/d3c/d2e/f85 x:0 0 0 2024-03-22T07:16:02.720 INFO:tasks.workunit.client.1.smithi069.stdout:3/443: symlink d5/d9/db/d3b/l86 0 2024-03-22T07:16:02.720 INFO:tasks.workunit.client.1.smithi069.stdout:3/444: readlink d5/d9/db/d3b/l86 0 2024-03-22T07:16:02.722 INFO:tasks.workunit.client.1.smithi069.stdout:0/527: mkdir d3/d7/d15/dad 0 2024-03-22T07:16:02.722 INFO:tasks.workunit.client.1.smithi069.stdout:0/528: stat d3/d7/d24/d6a/l82 0 2024-03-22T07:16:02.722 INFO:tasks.workunit.client.1.smithi069.stdout:4/522: getdents d6/d16/d83/d9a 0 2024-03-22T07:16:02.723 INFO:tasks.workunit.client.1.smithi069.stdout:2/317: dread d18/d4d/d25/f47 [0,4194304] 0 2024-03-22T07:16:02.723 INFO:tasks.workunit.client.1.smithi069.stdout:3/445: mknod d5/d9/d26/d3f/d54/c87 0 2024-03-22T07:16:02.724 INFO:tasks.workunit.client.1.smithi069.stdout:0/529: creat d3/d7/d15/dad/fae x:0 0 0 2024-03-22T07:16:02.725 INFO:tasks.workunit.client.1.smithi069.stdout:4/523: symlink d6/d4b/lae 0 2024-03-22T07:16:02.725 INFO:tasks.workunit.client.1.smithi069.stdout:4/524: chown d6/d9b/d56/fa8 41 1 2024-03-22T07:16:02.726 INFO:tasks.workunit.client.1.smithi069.stdout:4/525: creat d6/dc/d1d/d55/faf x:0 0 0 2024-03-22T07:16:02.727 INFO:tasks.workunit.client.0.smithi063.stdout:7/557: rename d5/f7 to d5/da/d5f/fb8 0 2024-03-22T07:16:02.727 INFO:tasks.workunit.client.1.smithi069.stdout:3/446: creat d5/d9/d26/f88 x:0 0 0 2024-03-22T07:16:02.728 INFO:tasks.workunit.client.1.smithi069.stdout:3/447: dread - d5/d9/d26/d36/d41/d51/f7c zero size 2024-03-22T07:16:02.728 INFO:tasks.workunit.client.1.smithi069.stdout:0/530: stat d3/d7/d17/d1c/c59 0 2024-03-22T07:16:02.728 INFO:tasks.workunit.client.1.smithi069.stdout:0/531: write d3/d7/d15/d29/d5a/fa7 [366359,28089] 0 2024-03-22T07:16:02.728 INFO:tasks.workunit.client.1.smithi069.stdout:0/532: dread - d3/d7/d5e/f7e zero size 2024-03-22T07:16:02.728 INFO:tasks.workunit.client.1.smithi069.stdout:1/396: dread d1/d2a/f32 [0,4194304] 0 2024-03-22T07:16:02.729 INFO:tasks.workunit.client.1.smithi069.stdout:7/442: dread d9/d76/d57/d44/d53/d74/d30/d3d/f40 [0,4194304] 0 2024-03-22T07:16:02.729 INFO:tasks.workunit.client.1.smithi069.stdout:7/443: write d9/d76/d57/d44/d53/d74/f8d [263988,38368] 0 2024-03-22T07:16:02.729 INFO:tasks.workunit.client.1.smithi069.stdout:7/444: dread - d9/db/f94 zero size 2024-03-22T07:16:02.731 INFO:tasks.workunit.client.0.smithi063.stdout:0/919: dwrite da/f82 [0,4194304] 0 2024-03-22T07:16:02.732 INFO:tasks.workunit.client.1.smithi069.stdout:4/526: symlink d6/d9b/d4a/lb0 0 2024-03-22T07:16:02.734 INFO:tasks.workunit.client.0.smithi063.stdout:9/676: dread d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/f85 [0,4194304] 0 2024-03-22T07:16:02.734 INFO:tasks.workunit.client.0.smithi063.stdout:1/865: dread d4/d5a/fa9 [0,4194304] 0 2024-03-22T07:16:02.735 INFO:tasks.workunit.client.1.smithi069.stdout:3/448: mknod d5/d33/c89 0 2024-03-22T07:16:02.735 INFO:tasks.workunit.client.1.smithi069.stdout:0/533: rename d3/d7/cc to d3/d7/d15/d29/d6e/caf 0 2024-03-22T07:16:02.737 INFO:tasks.workunit.client.0.smithi063.stdout:4/647: getdents d1/d5/dc/dda/d79/daa 0 2024-03-22T07:16:02.737 INFO:tasks.workunit.client.0.smithi063.stdout:7/558: dread d5/da/f25 [0,4194304] 0 2024-03-22T07:16:02.738 INFO:tasks.workunit.client.0.smithi063.stdout:7/559: creat d5/d16/d1b/d21/d85/fb9 x:0 0 0 2024-03-22T07:16:02.738 INFO:tasks.workunit.client.0.smithi063.stdout:7/560: fdatasync d5/d16/f1e 0 2024-03-22T07:16:02.739 INFO:tasks.workunit.client.1.smithi069.stdout:1/397: link d1/d25/c30 d1/d13/c7c 0 2024-03-22T07:16:02.739 INFO:tasks.workunit.client.1.smithi069.stdout:1/398: readlink d1/d2a/d50/l44 0 2024-03-22T07:16:02.741 INFO:tasks.workunit.client.1.smithi069.stdout:7/445: creat d9/d76/d57/d44/f9b x:0 0 0 2024-03-22T07:16:02.741 INFO:tasks.workunit.client.0.smithi063.stdout:3/794: mknod da/d16/c103 0 2024-03-22T07:16:02.741 INFO:tasks.workunit.client.0.smithi063.stdout:3/795: chown da/c43 36 1 2024-03-22T07:16:02.741 INFO:tasks.workunit.client.0.smithi063.stdout:3/796: chown da/d63/d65/d81/d9b/l9d 1417255 1 2024-03-22T07:16:02.742 INFO:tasks.workunit.client.1.smithi069.stdout:3/449: mkdir d5/d9/d26/d3f/d56/d8a 0 2024-03-22T07:16:02.742 INFO:tasks.workunit.client.1.smithi069.stdout:3/450: chown d5/d9/l1f 458404288 1 2024-03-22T07:16:02.742 INFO:tasks.workunit.client.0.smithi063.stdout:1/866: write d4/df/d13/d2b/f84 [1446520,106854] 0 2024-03-22T07:16:02.742 INFO:tasks.workunit.client.0.smithi063.stdout:1/867: chown d4/d71/dbb/dc0 7242 1 2024-03-22T07:16:02.742 INFO:tasks.workunit.client.0.smithi063.stdout:1/868: chown d4/df/d13/d2b/d6c/d85/d9e/lcf 0 1 2024-03-22T07:16:02.746 INFO:tasks.workunit.client.1.smithi069.stdout:0/534: unlink d3/d7/d17/c37 0 2024-03-22T07:16:02.748 INFO:tasks.workunit.client.1.smithi069.stdout:0/535: mknod d3/d7/d15/d29/d5a/d5d/d9d/cb0 0 2024-03-22T07:16:02.748 INFO:tasks.workunit.client.1.smithi069.stdout:0/536: read d3/d7/d15/d29/d5a/d84/f3b [496513,81083] 0 2024-03-22T07:16:02.748 INFO:tasks.workunit.client.0.smithi063.stdout:9/677: symlink d8/d19/d20/d22/d5d/d76/dab/ld1 0 2024-03-22T07:16:02.749 INFO:tasks.workunit.client.1.smithi069.stdout:3/451: dread d5/d9/d26/f5a [0,4194304] 0 2024-03-22T07:16:02.750 INFO:tasks.workunit.client.1.smithi069.stdout:0/537: symlink d3/d7/d15/d1f/lb1 0 2024-03-22T07:16:02.752 INFO:tasks.workunit.client.1.smithi069.stdout:2/318: dread f6 [0,4194304] 0 2024-03-22T07:16:02.753 INFO:tasks.workunit.client.1.smithi069.stdout:2/319: chown d18/l23 0 1 2024-03-22T07:16:02.754 INFO:tasks.workunit.client.1.smithi069.stdout:3/452: rename d5/d9/d26/f88 to d5/d9/d26/f8b 0 2024-03-22T07:16:02.755 INFO:tasks.workunit.client.1.smithi069.stdout:3/453: truncate d5/d9/db/f30 3798945 0 2024-03-22T07:16:02.756 INFO:tasks.workunit.client.0.smithi063.stdout:4/648: mknod d1/cdf 0 2024-03-22T07:16:02.756 INFO:tasks.workunit.client.0.smithi063.stdout:4/649: chown d1/d5/dc/dda/d79/ca1 0 1 2024-03-22T07:16:02.756 INFO:tasks.workunit.client.0.smithi063.stdout:4/650: chown d1/d5/dc/d95/d59/fa9 0 1 2024-03-22T07:16:02.756 INFO:tasks.workunit.client.1.smithi069.stdout:3/454: symlink d5/d9/d23/l8c 0 2024-03-22T07:16:02.756 INFO:tasks.workunit.client.1.smithi069.stdout:3/455: fdatasync d5/d9/d26/d3f/f3e 0 2024-03-22T07:16:02.758 INFO:tasks.workunit.client.1.smithi069.stdout:1/399: dread d1/d13/f45 [0,4194304] 0 2024-03-22T07:16:02.758 INFO:tasks.workunit.client.1.smithi069.stdout:3/456: creat d5/d9/d26/d3f/d35/f8d x:0 0 0 2024-03-22T07:16:02.761 INFO:tasks.workunit.client.1.smithi069.stdout:7/446: rmdir d9/d42/d8a 39 2024-03-22T07:16:02.764 INFO:tasks.workunit.client.1.smithi069.stdout:7/447: fsync d9/d76/d57/d44/d53/f43 0 2024-03-22T07:16:02.764 INFO:tasks.workunit.client.1.smithi069.stdout:7/448: dread - d9/db/f69 zero size 2024-03-22T07:16:02.765 INFO:tasks.workunit.client.0.smithi063.stdout:3/797: unlink da/d16/d3f/dc7/ced 0 2024-03-22T07:16:02.765 INFO:tasks.workunit.client.0.smithi063.stdout:3/798: dread - da/d63/d65/d81/da5/fad zero size 2024-03-22T07:16:02.765 INFO:tasks.workunit.client.0.smithi063.stdout:3/799: truncate da/d16/d89/d8c/dc1/fee 519328 0 2024-03-22T07:16:02.765 INFO:tasks.workunit.client.0.smithi063.stdout:3/800: creat da/d16/d3f/f104 x:0 0 0 2024-03-22T07:16:02.767 INFO:tasks.workunit.client.1.smithi069.stdout:7/449: link d9/db/d1a/f2c d9/d76/d57/d75/f9c 0 2024-03-22T07:16:02.770 INFO:tasks.workunit.client.0.smithi063.stdout:1/869: symlink d4/d71/d88/da1/l12c 0 2024-03-22T07:16:02.773 INFO:tasks.workunit.client.0.smithi063.stdout:9/678: rename d8/d19/f23 to d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/da5/fd2 0 2024-03-22T07:16:02.774 INFO:tasks.workunit.client.1.smithi069.stdout:2/320: dread d18/f3a [0,4194304] 0 2024-03-22T07:16:02.774 INFO:tasks.workunit.client.1.smithi069.stdout:2/321: readlink d18/d4d/d27/d56/l68 0 2024-03-22T07:16:02.775 INFO:tasks.workunit.client.0.smithi063.stdout:4/651: mknod d1/d5/dc2/ce0 0 2024-03-22T07:16:02.776 INFO:tasks.workunit.client.0.smithi063.stdout:3/801: write da/f57 [2357389,29587] 0 2024-03-22T07:16:02.777 INFO:tasks.workunit.client.1.smithi069.stdout:2/322: rmdir d18 39 2024-03-22T07:16:02.778 INFO:tasks.workunit.client.1.smithi069.stdout:2/323: mkdir d18/d4d/d27/d33/d60/d63/d73 0 2024-03-22T07:16:02.778 INFO:tasks.workunit.client.1.smithi069.stdout:2/324: fdatasync d18/d4d/d34/f5f 0 2024-03-22T07:16:02.779 INFO:tasks.workunit.client.0.smithi063.stdout:1/870: truncate d4/d14/f94 1971594 0 2024-03-22T07:16:02.779 INFO:tasks.workunit.client.1.smithi069.stdout:2/325: symlink d18/d4d/d3e/d40/l74 0 2024-03-22T07:16:02.780 INFO:tasks.workunit.client.1.smithi069.stdout:8/536: dwrite d1/f14 [0,4194304] 0 2024-03-22T07:16:02.781 INFO:tasks.workunit.client.0.smithi063.stdout:4/652: truncate d1/f45 3540967 0 2024-03-22T07:16:02.783 INFO:tasks.workunit.client.0.smithi063.stdout:3/802: getdents da/d63/d65/d81 0 2024-03-22T07:16:02.785 INFO:tasks.workunit.client.0.smithi063.stdout:9/679: rmdir d8/d19/d20/d22/d29/d31/d4b/db7 39 2024-03-22T07:16:02.791 INFO:tasks.workunit.client.1.smithi069.stdout:2/326: write d18/f2b [2211778,56091] 0 2024-03-22T07:16:02.791 INFO:tasks.workunit.client.1.smithi069.stdout:2/327: write d18/d4d/d3e/f65 [1791436,41640] 0 2024-03-22T07:16:02.792 INFO:tasks.workunit.client.1.smithi069.stdout:2/328: symlink d18/d4d/d3e/d40/l75 0 2024-03-22T07:16:02.792 INFO:tasks.workunit.client.0.smithi063.stdout:4/653: unlink d1/d5/dc/d95/da4/fce 0 2024-03-22T07:16:02.792 INFO:tasks.workunit.client.0.smithi063.stdout:9/680: dread d8/d15/d2f/f47 [0,4194304] 0 2024-03-22T07:16:02.793 INFO:tasks.workunit.client.1.smithi069.stdout:2/329: creat d18/d4d/d3e/d40/f76 x:0 0 0 2024-03-22T07:16:02.794 INFO:tasks.workunit.client.0.smithi063.stdout:3/803: mkdir da/d63/d105 0 2024-03-22T07:16:02.794 INFO:tasks.workunit.client.0.smithi063.stdout:1/871: getdents d4/d71/dbb/dc0/dd8 0 2024-03-22T07:16:02.794 INFO:tasks.workunit.client.0.smithi063.stdout:9/681: read d8/d15/f58 [676420,124619] 0 2024-03-22T07:16:02.795 INFO:tasks.workunit.client.0.smithi063.stdout:1/872: rmdir d4/d71/d88/da1 39 2024-03-22T07:16:02.800 INFO:tasks.workunit.client.0.smithi063.stdout:9/682: unlink d8/d19/d20/d22/d5d/lac 0 2024-03-22T07:16:02.803 INFO:tasks.workunit.client.0.smithi063.stdout:9/683: dread d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/f85 [0,4194304] 0 2024-03-22T07:16:02.803 INFO:tasks.workunit.client.0.smithi063.stdout:9/684: readlink d8/l28 0 2024-03-22T07:16:02.806 INFO:tasks.workunit.client.1.smithi069.stdout:2/330: dread d18/d4d/d34/f5f [0,4194304] 0 2024-03-22T07:16:02.806 INFO:tasks.workunit.client.1.smithi069.stdout:2/331: truncate d18/f70 640455 0 2024-03-22T07:16:02.806 INFO:tasks.workunit.client.0.smithi063.stdout:9/685: truncate d8/d17/d1e/f5c 884169 0 2024-03-22T07:16:02.806 INFO:tasks.workunit.client.0.smithi063.stdout:9/686: write d8/d19/d20/d22/d29/d31/f4e [1382177,35043] 0 2024-03-22T07:16:02.817 INFO:tasks.workunit.client.1.smithi069.stdout:6/372: dwrite d4/d7/d5b/f70 [0,4194304] 0 2024-03-22T07:16:02.817 INFO:tasks.workunit.client.1.smithi069.stdout:9/424: dwrite d2/dd/f10 [0,4194304] 0 2024-03-22T07:16:02.819 INFO:tasks.workunit.client.0.smithi063.stdout:9/687: write d8/f1a [3231703,106222] 0 2024-03-22T07:16:02.821 INFO:tasks.workunit.client.1.smithi069.stdout:7/450: dread d9/d76/d57/d44/d53/d74/f8d [0,4194304] 0 2024-03-22T07:16:02.823 INFO:tasks.workunit.client.1.smithi069.stdout:9/425: getdents d2/dd 0 2024-03-22T07:16:02.825 INFO:tasks.workunit.client.1.smithi069.stdout:7/451: link d9/f46 d9/d42/d8a/f9d 0 2024-03-22T07:16:02.825 INFO:tasks.workunit.client.1.smithi069.stdout:7/452: fsync d9/f6f 0 2024-03-22T07:16:02.825 INFO:tasks.workunit.client.1.smithi069.stdout:9/426: mkdir d2/dd/d3c/d48/d7f/d86 0 2024-03-22T07:16:02.827 INFO:tasks.workunit.client.0.smithi063.stdout:7/561: dwrite d5/d16/d1b/d21/d62/d73/d1d/f51 [0,4194304] 0 2024-03-22T07:16:02.827 INFO:tasks.workunit.client.1.smithi069.stdout:4/527: dwrite d6/d12/d18/d21/f2f [0,4194304] 0 2024-03-22T07:16:02.828 INFO:tasks.workunit.client.1.smithi069.stdout:0/538: dwrite d3/d7/f12 [0,4194304] 0 2024-03-22T07:16:02.828 INFO:tasks.workunit.client.0.smithi063.stdout:0/920: dwrite da/de/d60/fd2 [4194304,4194304] 0 2024-03-22T07:16:02.829 INFO:tasks.workunit.client.1.smithi069.stdout:7/453: dread d9/d76/d57/d44/f47 [0,4194304] 0 2024-03-22T07:16:02.829 INFO:tasks.workunit.client.1.smithi069.stdout:4/528: symlink d6/d9b/d56/lb1 0 2024-03-22T07:16:02.830 INFO:tasks.workunit.client.1.smithi069.stdout:6/373: write d4/f6 [1667705,67880] 0 2024-03-22T07:16:02.832 INFO:tasks.workunit.client.0.smithi063.stdout:9/688: dread d8/d19/d20/fa0 [0,4194304] 0 2024-03-22T07:16:02.833 INFO:tasks.workunit.client.0.smithi063.stdout:9/689: symlink d8/d19/d39/db5/ld3 0 2024-03-22T07:16:02.834 INFO:tasks.workunit.client.1.smithi069.stdout:0/539: creat d3/d7/d15/d29/d5a/d5d/fb2 x:0 0 0 2024-03-22T07:16:02.834 INFO:tasks.workunit.client.1.smithi069.stdout:0/540: dread - d3/d7/f8e zero size 2024-03-22T07:16:02.834 INFO:tasks.workunit.client.1.smithi069.stdout:0/541: dread - d3/d7/d15/d29/d6e/f7a zero size 2024-03-22T07:16:02.834 INFO:tasks.workunit.client.1.smithi069.stdout:7/454: truncate d9/d76/d57/d44/d53/f60 138201 0 2024-03-22T07:16:02.835 INFO:tasks.workunit.client.1.smithi069.stdout:4/529: mkdir d6/d12/db2 0 2024-03-22T07:16:02.835 INFO:tasks.workunit.client.1.smithi069.stdout:6/374: mkdir d4/dd/d12/d2b/d72/d73 0 2024-03-22T07:16:02.836 INFO:tasks.workunit.client.1.smithi069.stdout:3/457: dwrite d5/d9/d26/d3f/d35/f50 [0,4194304] 0 2024-03-22T07:16:02.836 INFO:tasks.workunit.client.1.smithi069.stdout:3/458: readlink d5/d9/d23/l8c 0 2024-03-22T07:16:02.836 INFO:tasks.workunit.client.1.smithi069.stdout:1/400: dwrite d1/d13/f27 [0,4194304] 0 2024-03-22T07:16:02.836 INFO:tasks.workunit.client.1.smithi069.stdout:1/401: dread - d1/d2a/d50/d4c/d55/f79 zero size 2024-03-22T07:16:02.837 INFO:tasks.workunit.client.1.smithi069.stdout:1/402: stat d1/d25/d41/d1e/c52 0 2024-03-22T07:16:02.838 INFO:tasks.workunit.client.1.smithi069.stdout:8/537: dwrite d1/d1c/d48/d5e/f79 [0,4194304] 0 2024-03-22T07:16:02.838 INFO:tasks.workunit.client.1.smithi069.stdout:0/542: rename d3/d7/d17/f1a to d3/d7/d17/d1c/fb3 0 2024-03-22T07:16:02.839 INFO:tasks.workunit.client.0.smithi063.stdout:4/654: dwrite d1/d5/d3c/d8f/dc0/dcb/fdc [0,4194304] 0 2024-03-22T07:16:02.840 INFO:tasks.workunit.client.1.smithi069.stdout:7/455: creat d9/d76/d57/d75/d7a/d96/f9e x:0 0 0 2024-03-22T07:16:02.841 INFO:tasks.workunit.client.1.smithi069.stdout:7/456: write d9/db/d1a/f21 [1569002,2911] 0 2024-03-22T07:16:02.843 INFO:tasks.workunit.client.0.smithi063.stdout:4/655: dread d1/d5/d3c/d65/f69 [0,4194304] 0 2024-03-22T07:16:02.843 INFO:tasks.workunit.client.0.smithi063.stdout:1/873: dwrite d4/df/d13/d39/d6f/dd9/feb [0,4194304] 0 2024-03-22T07:16:02.843 INFO:tasks.workunit.client.0.smithi063.stdout:1/874: write d4/d5a/d91/db6/f11a [122341,51821] 0 2024-03-22T07:16:02.843 INFO:tasks.workunit.client.0.smithi063.stdout:1/875: write f2 [4645177,121750] 0 2024-03-22T07:16:02.843 INFO:tasks.workunit.client.1.smithi069.stdout:2/332: dwrite d18/d4d/d3e/d40/f5c [0,4194304] 0 2024-03-22T07:16:02.844 INFO:tasks.workunit.client.1.smithi069.stdout:4/530: unlink d6/d12/l66 0 2024-03-22T07:16:02.844 INFO:tasks.workunit.client.1.smithi069.stdout:4/531: dread - d6/d9b/d1f/d58/f85 zero size 2024-03-22T07:16:02.846 INFO:tasks.workunit.client.1.smithi069.stdout:6/375: truncate d4/dd/d12/f1f 2130809 0 2024-03-22T07:16:02.850 INFO:tasks.workunit.client.1.smithi069.stdout:1/403: getdents d1/d2a/d50 0 2024-03-22T07:16:02.850 INFO:tasks.workunit.client.1.smithi069.stdout:1/404: chown d1/d2a/d50 223532670 1 2024-03-22T07:16:02.850 INFO:tasks.workunit.client.0.smithi063.stdout:7/562: write d5/d16/f2e [2542994,49574] 0 2024-03-22T07:16:02.851 INFO:tasks.workunit.client.0.smithi063.stdout:7/563: truncate d5/d16/d1b/d21/d62/d73/d1d/d29/fb4 256948 0 2024-03-22T07:16:02.853 INFO:tasks.workunit.client.1.smithi069.stdout:2/333: write d18/d4d/d3e/d40/f5c [3164281,703] 0 2024-03-22T07:16:02.854 INFO:tasks.workunit.client.1.smithi069.stdout:8/538: rename d1/d1c/d48/f8c to d1/d1c/d48/d5e/fad 0 2024-03-22T07:16:02.854 INFO:tasks.workunit.client.1.smithi069.stdout:8/539: dread - d1/de/d6c/d4e/d5b/f94 zero size 2024-03-22T07:16:02.855 INFO:tasks.workunit.client.0.smithi063.stdout:7/564: dread d5/d16/d1b/d21/f86 [0,4194304] 0 2024-03-22T07:16:02.855 INFO:tasks.workunit.client.0.smithi063.stdout:1/876: rename d4/d14/d57/f5d to d4/d14/d123/f12d 0 2024-03-22T07:16:02.855 INFO:tasks.workunit.client.0.smithi063.stdout:1/877: truncate d4/d71/d88/da1/ddb/f12b 4962790 0 2024-03-22T07:16:02.855 INFO:tasks.workunit.client.0.smithi063.stdout:1/878: chown d4/d71/dbb/dc0/fea 29763 1 2024-03-22T07:16:02.855 INFO:tasks.workunit.client.0.smithi063.stdout:0/921: dread da/de/d17/f2f [0,4194304] 0 2024-03-22T07:16:02.855 INFO:tasks.workunit.client.0.smithi063.stdout:0/922: readlink da/de/d60/la8 0 2024-03-22T07:16:02.855 INFO:tasks.workunit.client.0.smithi063.stdout:0/923: fsync da/de/d17/d25/d3c/d53/d89/f100 0 2024-03-22T07:16:02.855 INFO:tasks.workunit.client.0.smithi063.stdout:3/804: dwrite da/d16/d89/d8c/dc1/fee [0,4194304] 0 2024-03-22T07:16:02.856 INFO:tasks.workunit.client.1.smithi069.stdout:0/543: truncate d3/d7/d15/d29/f30 662018 0 2024-03-22T07:16:02.862 INFO:tasks.workunit.client.0.smithi063.stdout:0/924: unlink da/de/l1d 0 2024-03-22T07:16:02.863 INFO:tasks.workunit.client.0.smithi063.stdout:0/925: stat da/de/d17/d25/d3c/f3f 0 2024-03-22T07:16:02.863 INFO:tasks.workunit.client.0.smithi063.stdout:0/926: chown da/de/d60/d56 30835296 1 2024-03-22T07:16:02.865 INFO:tasks.workunit.client.1.smithi069.stdout:9/427: dwrite d2/d21/f73 [0,4194304] 0 2024-03-22T07:16:02.868 INFO:tasks.workunit.client.0.smithi063.stdout:3/805: creat da/d63/d65/d4b/dfa/dfc/f106 x:0 0 0 2024-03-22T07:16:02.868 INFO:tasks.workunit.client.0.smithi063.stdout:0/927: mkdir da/d49/d12c 0 2024-03-22T07:16:02.869 INFO:tasks.workunit.client.1.smithi069.stdout:8/540: write d1/de/f30 [1269162,95896] 0 2024-03-22T07:16:02.869 INFO:tasks.workunit.client.1.smithi069.stdout:8/541: write d1/d1c/d3e/d66/f75 [793545,70422] 0 2024-03-22T07:16:02.869 INFO:tasks.workunit.client.1.smithi069.stdout:8/542: creat d1/de/d6c/d4e/d5b/fae x:0 0 0 2024-03-22T07:16:02.869 INFO:tasks.workunit.client.1.smithi069.stdout:4/532: symlink d6/d12/d5d/lb3 0 2024-03-22T07:16:02.870 INFO:tasks.workunit.client.1.smithi069.stdout:4/533: write d6/d16/d83/d9a/fa3 [626868,56437] 0 2024-03-22T07:16:02.871 INFO:tasks.workunit.client.1.smithi069.stdout:6/376: mkdir d4/dd/d12/d2b/d72/d73/d74 0 2024-03-22T07:16:02.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:02 smithi063 ceph-mon[29200]: Deploying cephadm binary to smithi063 2024-03-22T07:16:02.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:02 smithi063 ceph-mon[29200]: Deploying cephadm binary to smithi069 2024-03-22T07:16:02.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:02 smithi063 ceph-mon[29200]: pgmap v3: 65 pgs: 65 active+clean; 2.4 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail 2024-03-22T07:16:02.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:02 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:02.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:02 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:02.874 INFO:tasks.workunit.client.1.smithi069.stdout:1/405: creat d1/d2a/d50/d4c/f7d x:0 0 0 2024-03-22T07:16:02.875 INFO:tasks.workunit.client.1.smithi069.stdout:2/334: mkdir d18/d4d/d27/d77 0 2024-03-22T07:16:02.875 INFO:tasks.workunit.client.1.smithi069.stdout:2/335: fsync d18/d4d/d3e/d40/f44 0 2024-03-22T07:16:02.875 INFO:tasks.workunit.client.0.smithi063.stdout:3/806: truncate da/d16/f42 3850768 0 2024-03-22T07:16:02.876 INFO:tasks.workunit.client.0.smithi063.stdout:3/807: truncate da/d63/d65/d81/da5/ff5 723098 0 2024-03-22T07:16:02.876 INFO:tasks.workunit.client.1.smithi069.stdout:2/336: write d18/d4d/d3e/d40/f61 [18108,68917] 0 2024-03-22T07:16:02.876 INFO:tasks.workunit.client.0.smithi063.stdout:5/592: sync 2024-03-22T07:16:02.877 INFO:tasks.workunit.client.0.smithi063.stdout:5/593: creat d8/d13/d1e/d3c/faa x:0 0 0 2024-03-22T07:16:02.878 INFO:tasks.workunit.client.1.smithi069.stdout:8/543: symlink d1/d1c/d48/laf 0 2024-03-22T07:16:02.881 INFO:tasks.workunit.client.1.smithi069.stdout:6/377: link f3 d4/dd/d12/d44/d58/f75 0 2024-03-22T07:16:02.882 INFO:tasks.workunit.client.1.smithi069.stdout:6/378: getdents d4/dd/d12/d2b/d72/d73/d74 0 2024-03-22T07:16:02.882 INFO:tasks.workunit.client.1.smithi069.stdout:6/379: dread - d4/f6c zero size 2024-03-22T07:16:02.882 INFO:tasks.workunit.client.1.smithi069.stdout:6/380: dread - d4/f6c zero size 2024-03-22T07:16:02.882 INFO:tasks.workunit.client.0.smithi063.stdout:5/594: symlink d8/d13/d28/d7d/lab 0 2024-03-22T07:16:02.883 INFO:tasks.workunit.client.0.smithi063.stdout:5/595: symlink d8/d13/d1e/d31/lac 0 2024-03-22T07:16:02.883 INFO:tasks.workunit.client.0.smithi063.stdout:5/596: read d8/d13/d1e/d31/d58/f68 [547708,63808] 0 2024-03-22T07:16:02.883 INFO:tasks.workunit.client.0.smithi063.stdout:5/597: creat d8/d13/d1e/d31/d58/fad x:0 0 0 2024-03-22T07:16:02.884 INFO:tasks.workunit.client.1.smithi069.stdout:1/406: rename d1/d13/d36/f43 to d1/d25/d41/d64/f7e 0 2024-03-22T07:16:02.884 INFO:tasks.workunit.client.1.smithi069.stdout:1/407: chown d1/fb 176377 1 2024-03-22T07:16:02.885 INFO:tasks.workunit.client.1.smithi069.stdout:2/337: mknod d18/d4d/d27/d33/d4f/c78 0 2024-03-22T07:16:02.889 INFO:tasks.workunit.client.1.smithi069.stdout:9/428: write d2/dd/d3c/d2e/f4e [3255226,87683] 0 2024-03-22T07:16:02.890 INFO:tasks.workunit.client.1.smithi069.stdout:1/408: link d1/d13/d36/f67 d1/d25/d41/d7a/f7f 0 2024-03-22T07:16:02.890 INFO:tasks.workunit.client.1.smithi069.stdout:1/409: fsync d1/d13/f26 0 2024-03-22T07:16:02.890 INFO:tasks.workunit.client.1.smithi069.stdout:1/410: creat d1/d2a/d50/d4c/d55/f80 x:0 0 0 2024-03-22T07:16:02.890 INFO:tasks.workunit.client.1.smithi069.stdout:1/411: creat d1/d13/d36/f81 x:0 0 0 2024-03-22T07:16:02.890 INFO:tasks.workunit.client.1.smithi069.stdout:1/412: chown d1/d25/d41/d1e/l33 61172674 1 2024-03-22T07:16:02.890 INFO:tasks.workunit.client.1.smithi069.stdout:1/413: chown d1/f3b 27291617 1 2024-03-22T07:16:02.891 INFO:tasks.workunit.client.1.smithi069.stdout:1/414: read d1/d13/f2d [2154972,2048] 0 2024-03-22T07:16:02.891 INFO:tasks.workunit.client.1.smithi069.stdout:2/338: mknod d18/d4d/d27/d33/d4f/c79 0 2024-03-22T07:16:02.891 INFO:tasks.workunit.client.1.smithi069.stdout:2/339: readlink d18/d4d/d27/d33/d60/d64/l67 0 2024-03-22T07:16:02.894 INFO:tasks.workunit.client.1.smithi069.stdout:6/381: write d4/dd/f25 [237234,64088] 0 2024-03-22T07:16:02.894 INFO:tasks.workunit.client.1.smithi069.stdout:6/382: write d4/dd/d12/d44/f4e [477319,79813] 0 2024-03-22T07:16:02.896 INFO:tasks.workunit.client.1.smithi069.stdout:7/457: dwrite d9/db/d1a/f39 [0,4194304] 0 2024-03-22T07:16:02.896 INFO:tasks.workunit.client.1.smithi069.stdout:7/458: fsync d9/d76/d57/d44/d53/d1d/f87 0 2024-03-22T07:16:02.898 INFO:tasks.workunit.client.1.smithi069.stdout:9/429: creat d2/d21/d77/f87 x:0 0 0 2024-03-22T07:16:02.899 INFO:tasks.workunit.client.1.smithi069.stdout:9/430: write d2/dd/d3c/d2e/f80 [489763,70967] 0 2024-03-22T07:16:02.899 INFO:tasks.workunit.client.1.smithi069.stdout:1/415: dread d1/d25/d41/d1e/f51 [0,4194304] 0 2024-03-22T07:16:02.899 INFO:tasks.workunit.client.1.smithi069.stdout:1/416: stat d1/l21 0 2024-03-22T07:16:02.901 INFO:tasks.workunit.client.1.smithi069.stdout:2/340: unlink d18/l38 0 2024-03-22T07:16:02.901 INFO:tasks.workunit.client.1.smithi069.stdout:2/341: write d18/d4d/d27/d33/f55 [699025,46693] 0 2024-03-22T07:16:02.904 INFO:tasks.workunit.client.1.smithi069.stdout:6/383: getdents d4/dd/d12/d2b 0 2024-03-22T07:16:02.909 INFO:tasks.workunit.client.1.smithi069.stdout:1/417: unlink d1/d13/f17 0 2024-03-22T07:16:02.913 INFO:tasks.workunit.client.1.smithi069.stdout:1/418: mkdir d1/d13/d82 0 2024-03-22T07:16:02.913 INFO:tasks.workunit.client.1.smithi069.stdout:1/419: chown d1/d25/f38 3715 1 2024-03-22T07:16:02.913 INFO:tasks.workunit.client.1.smithi069.stdout:9/431: write d2/dd/f69 [52884,64253] 0 2024-03-22T07:16:02.914 INFO:tasks.workunit.client.1.smithi069.stdout:3/459: dwrite d5/d9/d26/d3f/d35/f6b [0,4194304] 0 2024-03-22T07:16:02.914 INFO:tasks.workunit.client.0.smithi063.stdout:9/690: dwrite d8/f1a [4194304,4194304] 0 2024-03-22T07:16:02.914 INFO:tasks.workunit.client.0.smithi063.stdout:9/691: write d8/f1a [2776407,121660] 0 2024-03-22T07:16:02.915 INFO:tasks.workunit.client.0.smithi063.stdout:4/656: dwrite d1/d5/f2f [0,4194304] 0 2024-03-22T07:16:02.917 INFO:tasks.workunit.client.0.smithi063.stdout:7/565: dwrite d5/d16/d1b/f80 [0,4194304] 0 2024-03-22T07:16:02.920 INFO:tasks.workunit.client.1.smithi069.stdout:9/432: creat d2/dd/d3c/d2e/d84/f88 x:0 0 0 2024-03-22T07:16:02.920 INFO:tasks.workunit.client.1.smithi069.stdout:6/384: dread d4/f14 [0,4194304] 0 2024-03-22T07:16:02.921 INFO:tasks.workunit.client.0.smithi063.stdout:4/657: dread d1/d5/d43/d27/d2c/d3d/d54/fba [0,4194304] 0 2024-03-22T07:16:02.921 INFO:tasks.workunit.client.0.smithi063.stdout:7/566: dread d5/d38/f7b [0,4194304] 0 2024-03-22T07:16:02.921 INFO:tasks.workunit.client.0.smithi063.stdout:7/567: write d5/da/f25 [3104483,14896] 0 2024-03-22T07:16:02.921 INFO:tasks.workunit.client.0.smithi063.stdout:7/568: readlink d5/d16/l4c 0 2024-03-22T07:16:02.925 INFO:tasks.workunit.client.0.smithi063.stdout:7/569: rename d5/d16/d43/d78/db5 to d5/da/d28/da7/dba 0 2024-03-22T07:16:02.925 INFO:tasks.workunit.client.0.smithi063.stdout:7/570: chown d5/d16/d1b/d21/d62/d73/d1d/d40/l49 3807078 1 2024-03-22T07:16:02.927 INFO:tasks.workunit.client.0.smithi063.stdout:7/571: getdents d5/da/d5f 0 2024-03-22T07:16:02.929 INFO:tasks.workunit.client.0.smithi063.stdout:7/572: mkdir d5/d16/d1b/d21/d62/d8d/dbb 0 2024-03-22T07:16:02.930 INFO:tasks.workunit.client.0.smithi063.stdout:0/928: dwrite da/de/d17/d25/d3c/d5f/fb8 [4194304,4194304] 0 2024-03-22T07:16:02.931 INFO:tasks.workunit.client.0.smithi063.stdout:0/929: write da/de/d60/d56/d8b/f91 [88757,106751] 0 2024-03-22T07:16:02.932 INFO:tasks.workunit.client.1.smithi069.stdout:1/420: write d1/d13/f39 [3580737,98258] 0 2024-03-22T07:16:02.937 INFO:tasks.workunit.client.1.smithi069.stdout:3/460: write d5/d9/d26/d3f/d35/f50 [4100667,100998] 0 2024-03-22T07:16:02.937 INFO:tasks.workunit.client.1.smithi069.stdout:3/461: stat d5/l71 0 2024-03-22T07:16:02.941 INFO:tasks.workunit.client.1.smithi069.stdout:6/385: mkdir d4/d7/da/d3f/d76 0 2024-03-22T07:16:02.941 INFO:tasks.workunit.client.1.smithi069.stdout:8/544: dwrite d1/d1c/d3e/f36 [0,4194304] 0 2024-03-22T07:16:02.942 INFO:tasks.workunit.client.0.smithi063.stdout:5/598: dwrite d8/d13/d1e/f2f [0,4194304] 0 2024-03-22T07:16:02.942 INFO:tasks.workunit.client.0.smithi063.stdout:0/930: dread da/de/d60/f33 [0,4194304] 0 2024-03-22T07:16:02.942 INFO:tasks.workunit.client.0.smithi063.stdout:0/931: dread - da/de/d17/d25/d3c/d64/df5/f105 zero size 2024-03-22T07:16:02.942 INFO:tasks.workunit.client.1.smithi069.stdout:7/459: dwrite d9/d76/d57/d44/d53/d74/f91 [0,4194304] 0 2024-03-22T07:16:02.942 INFO:tasks.workunit.client.1.smithi069.stdout:7/460: dread - d9/d76/d57/d44/f9b zero size 2024-03-22T07:16:02.944 INFO:tasks.workunit.client.0.smithi063.stdout:5/599: truncate d8/d13/f4b 762680 0 2024-03-22T07:16:02.944 INFO:tasks.workunit.client.1.smithi069.stdout:3/462: mkdir d5/d9/d26/d36/d41/d8e 0 2024-03-22T07:16:02.945 INFO:tasks.workunit.client.1.smithi069.stdout:3/463: write d5/d9/d26/f5a [983635,4480] 0 2024-03-22T07:16:02.945 INFO:tasks.workunit.client.0.smithi063.stdout:0/932: mknod da/de/d39/c12d 0 2024-03-22T07:16:02.945 INFO:tasks.workunit.client.0.smithi063.stdout:5/600: getdents d8/d13/d24/d62 0 2024-03-22T07:16:02.945 INFO:tasks.workunit.client.0.smithi063.stdout:5/601: fdatasync d8/d13/d1e/d31/d58/d7e/f84 0 2024-03-22T07:16:02.946 INFO:tasks.workunit.client.0.smithi063.stdout:5/602: chown d8/d13/d1e/d31/d58/d7e/d82 131 1 2024-03-22T07:16:02.946 INFO:tasks.workunit.client.0.smithi063.stdout:5/603: creat d8/d13/d1e/d3c/fae x:0 0 0 2024-03-22T07:16:02.946 INFO:tasks.workunit.client.1.smithi069.stdout:6/386: creat d4/f77 x:0 0 0 2024-03-22T07:16:02.947 INFO:tasks.workunit.client.1.smithi069.stdout:2/342: dwrite d18/d4d/d3e/f65 [0,4194304] 0 2024-03-22T07:16:02.947 INFO:tasks.workunit.client.1.smithi069.stdout:2/343: fsync f1 0 2024-03-22T07:16:02.947 INFO:tasks.workunit.client.0.smithi063.stdout:0/933: creat da/de/d17/d25/d95/d98/f12e x:0 0 0 2024-03-22T07:16:02.948 INFO:tasks.workunit.client.0.smithi063.stdout:5/604: readlink d8/d13/l20 0 2024-03-22T07:16:02.949 INFO:tasks.workunit.client.0.smithi063.stdout:5/605: mkdir d8/d13/d1e/d31/daf 0 2024-03-22T07:16:02.951 INFO:tasks.workunit.client.1.smithi069.stdout:8/545: unlink d1/de/d6c/d4e/d98/la6 0 2024-03-22T07:16:02.951 INFO:tasks.workunit.client.0.smithi063.stdout:5/606: creat d8/d13/d1e/d31/daf/fb0 x:0 0 0 2024-03-22T07:16:02.951 INFO:tasks.workunit.client.0.smithi063.stdout:5/607: chown d8/d13/d28/d57/c72 180 1 2024-03-22T07:16:02.953 INFO:tasks.workunit.client.1.smithi069.stdout:3/464: creat d5/f8f x:0 0 0 2024-03-22T07:16:02.953 INFO:tasks.workunit.client.1.smithi069.stdout:3/465: dread - d5/d9/d23/f73 zero size 2024-03-22T07:16:02.956 INFO:tasks.workunit.client.1.smithi069.stdout:6/387: mkdir d4/dd/d41/d78 0 2024-03-22T07:16:02.959 INFO:tasks.workunit.client.1.smithi069.stdout:8/546: rename d1/d1c/d48/d5e/f63 to d1/d1c/d3e/fb0 0 2024-03-22T07:16:02.960 INFO:tasks.workunit.client.1.smithi069.stdout:9/433: dwrite d2/f55 [0,4194304] 0 2024-03-22T07:16:02.960 INFO:tasks.workunit.client.1.smithi069.stdout:9/434: fsync d2/d31/f40 0 2024-03-22T07:16:02.960 INFO:tasks.workunit.client.1.smithi069.stdout:3/466: getdents d5/d9/d23 0 2024-03-22T07:16:02.960 INFO:tasks.workunit.client.1.smithi069.stdout:3/467: truncate d5/f4e 895271 0 2024-03-22T07:16:02.960 INFO:tasks.workunit.client.1.smithi069.stdout:3/468: readlink d5/d9/d23/l59 0 2024-03-22T07:16:02.961 INFO:tasks.workunit.client.0.smithi063.stdout:9/692: dwrite d8/d19/d39/f8b [0,4194304] 0 2024-03-22T07:16:02.961 INFO:tasks.workunit.client.0.smithi063.stdout:9/693: chown d8/d19/d20/d22/d5d/d76/fc6 119 1 2024-03-22T07:16:02.962 INFO:tasks.workunit.client.0.smithi063.stdout:4/658: dwrite d1/d5/f67 [0,4194304] 0 2024-03-22T07:16:02.962 INFO:tasks.workunit.client.1.smithi069.stdout:6/388: symlink d4/dd/l79 0 2024-03-22T07:16:02.962 INFO:tasks.workunit.client.1.smithi069.stdout:6/389: creat d4/dd/d12/d44/f7a x:0 0 0 2024-03-22T07:16:02.962 INFO:tasks.workunit.client.1.smithi069.stdout:6/390: write d4/dd/d12/d2b/f36 [8701385,4778] 0 2024-03-22T07:16:02.962 INFO:tasks.workunit.client.1.smithi069.stdout:6/391: chown d4/dd/c18 0 1 2024-03-22T07:16:02.963 INFO:tasks.workunit.client.0.smithi063.stdout:9/694: creat d8/d19/d20/d22/fd4 x:0 0 0 2024-03-22T07:16:02.963 INFO:tasks.workunit.client.0.smithi063.stdout:9/695: readlink d8/l28 0 2024-03-22T07:16:02.964 INFO:tasks.workunit.client.1.smithi069.stdout:2/344: dread d18/d4d/f2d [0,4194304] 0 2024-03-22T07:16:02.964 INFO:tasks.workunit.client.1.smithi069.stdout:2/345: stat d18/d4d/d3e/d40/f6f 0 2024-03-22T07:16:02.964 INFO:tasks.workunit.client.1.smithi069.stdout:2/346: dread - d18/d4d/d25/f5b zero size 2024-03-22T07:16:02.964 INFO:tasks.workunit.client.1.smithi069.stdout:9/435: mkdir d2/d21/d89 0 2024-03-22T07:16:02.964 INFO:tasks.workunit.client.0.smithi063.stdout:9/696: symlink d8/d19/d20/d22/d29/db0/ld5 0 2024-03-22T07:16:02.968 INFO:tasks.workunit.client.1.smithi069.stdout:6/392: symlink d4/dd/d12/d2b/d72/d73/l7b 0 2024-03-22T07:16:02.970 INFO:tasks.workunit.client.1.smithi069.stdout:2/347: unlink d18/c69 0 2024-03-22T07:16:02.971 INFO:tasks.workunit.client.1.smithi069.stdout:9/436: mknod d2/d71/c8a 0 2024-03-22T07:16:02.971 INFO:tasks.workunit.client.1.smithi069.stdout:9/437: fsync d2/dd/d3c/d2e/f4b 0 2024-03-22T07:16:02.972 INFO:tasks.workunit.client.0.smithi063.stdout:5/608: dwrite d8/f25 [0,4194304] 0 2024-03-22T07:16:02.972 INFO:tasks.workunit.client.0.smithi063.stdout:5/609: truncate d8/d13/d1e/d31/d64/d65/f95 470548 0 2024-03-22T07:16:02.972 INFO:tasks.workunit.client.1.smithi069.stdout:7/461: dwrite d9/d42/f71 [0,4194304] 0 2024-03-22T07:16:02.973 INFO:tasks.workunit.client.0.smithi063.stdout:9/697: dread d8/d19/d20/d22/fa4 [0,4194304] 0 2024-03-22T07:16:02.973 INFO:tasks.workunit.client.0.smithi063.stdout:9/698: creat d8/d19/d20/d22/fd6 x:0 0 0 2024-03-22T07:16:02.973 INFO:tasks.workunit.client.0.smithi063.stdout:9/699: chown d8/d15/d63/lb6 42141 1 2024-03-22T07:16:02.973 INFO:tasks.workunit.client.0.smithi063.stdout:9/700: truncate d8/d19/d20/d22/fa4 1076612 0 2024-03-22T07:16:02.974 INFO:tasks.workunit.client.1.smithi069.stdout:6/393: truncate d4/dd/d12/f1d 738268 0 2024-03-22T07:16:02.974 INFO:tasks.workunit.client.1.smithi069.stdout:8/547: dread d1/de/d20/f3c [0,4194304] 0 2024-03-22T07:16:02.975 INFO:tasks.workunit.client.1.smithi069.stdout:2/348: mkdir d18/d4d/d7a 0 2024-03-22T07:16:02.976 INFO:tasks.workunit.client.1.smithi069.stdout:9/438: mkdir d2/dd/d3c/d53/d8b 0 2024-03-22T07:16:02.976 INFO:tasks.workunit.client.1.smithi069.stdout:9/439: stat d2/dd/d3c/d2e/l49 0 2024-03-22T07:16:02.978 INFO:tasks.workunit.client.1.smithi069.stdout:3/469: rename d5/d9/d61 to d5/d9/d26/d3f/d56/d7f/d90 0 2024-03-22T07:16:02.981 INFO:tasks.workunit.client.1.smithi069.stdout:7/462: symlink d9/d76/d57/d44/d53/d2e/l9f 0 2024-03-22T07:16:02.981 INFO:tasks.workunit.client.1.smithi069.stdout:2/349: dread d18/d4d/d3e/d40/f61 [0,4194304] 0 2024-03-22T07:16:02.982 INFO:tasks.workunit.client.0.smithi063.stdout:9/701: mkdir d8/d19/d20/d22/d29/d31/d4b/dd7 0 2024-03-22T07:16:02.983 INFO:tasks.workunit.client.0.smithi063.stdout:4/659: dwrite d1/d5/dc/d95/d59/f87 [0,4194304] 0 2024-03-22T07:16:02.983 INFO:tasks.workunit.client.0.smithi063.stdout:4/660: readlink d1/d5/d43/d27/l40 0 2024-03-22T07:16:02.985 INFO:tasks.workunit.client.1.smithi069.stdout:9/440: link d2/dd/d3c/d48/l5b d2/dd/d6e/l8c 0 2024-03-22T07:16:02.985 INFO:tasks.workunit.client.0.smithi063.stdout:7/573: rename d5/d16/d1b/d21 to d5/da/d28/d4b/da8/dbc 0 2024-03-22T07:16:02.986 INFO:tasks.workunit.client.0.smithi063.stdout:7/574: chown d5/da/d28/d4b/ca0 1005973771 1 2024-03-22T07:16:02.986 INFO:tasks.workunit.client.1.smithi069.stdout:7/463: mkdir d9/d76/d57/d75/d7a/d96/da0 0 2024-03-22T07:16:02.987 INFO:tasks.workunit.client.1.smithi069.stdout:7/464: stat d9/d76/d57/d75/f9c 0 2024-03-22T07:16:02.987 INFO:tasks.workunit.client.1.smithi069.stdout:7/465: stat d9/ld 0 2024-03-22T07:16:02.987 INFO:tasks.workunit.client.0.smithi063.stdout:4/661: symlink d1/d5/d43/d27/d2c/d4a/d78/le1 0 2024-03-22T07:16:02.987 INFO:tasks.workunit.client.1.smithi069.stdout:6/394: read d4/d7/da/f1b [3577997,114245] 0 2024-03-22T07:16:02.988 INFO:tasks.workunit.client.1.smithi069.stdout:2/350: rename d18/d4d/c58 to d18/d4d/d27/d72/c7b 0 2024-03-22T07:16:02.988 INFO:tasks.workunit.client.0.smithi063.stdout:0/934: rename da/de/d17/d25/c4c to da/de/ddd/c12f 0 2024-03-22T07:16:02.988 INFO:tasks.workunit.client.0.smithi063.stdout:0/935: chown da/de/d60/d56/d8b/d123 589309 1 2024-03-22T07:16:02.989 INFO:tasks.workunit.client.0.smithi063.stdout:0/936: creat da/de/d17/d25/d3c/d43/db5/f130 x:0 0 0 2024-03-22T07:16:02.989 INFO:tasks.workunit.client.0.smithi063.stdout:0/937: fdatasync da/de/d17/d25/d3c/d53/f118 0 2024-03-22T07:16:02.989 INFO:tasks.workunit.client.1.smithi069.stdout:8/548: getdents d1/de/d6c/d4e/d5b 0 2024-03-22T07:16:02.990 INFO:tasks.workunit.client.1.smithi069.stdout:8/549: readlink d1/d93/laa 0 2024-03-22T07:16:02.990 INFO:tasks.workunit.client.1.smithi069.stdout:8/550: chown d1/c9f 1 1 2024-03-22T07:16:02.990 INFO:tasks.workunit.client.1.smithi069.stdout:8/551: stat d1/d1c/c34 0 2024-03-22T07:16:02.990 INFO:tasks.workunit.client.1.smithi069.stdout:8/552: readlink d1/d1c/l50 0 2024-03-22T07:16:02.990 INFO:tasks.workunit.client.0.smithi063.stdout:7/575: dread d5/da/d28/d4b/da8/dbc/f59 [0,4194304] 0 2024-03-22T07:16:02.991 INFO:tasks.workunit.client.0.smithi063.stdout:7/576: chown d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/d4a 130189478 1 2024-03-22T07:16:02.991 INFO:tasks.workunit.client.0.smithi063.stdout:7/577: creat d5/da/d5f/d33/fbd x:0 0 0 2024-03-22T07:16:02.992 INFO:tasks.workunit.client.0.smithi063.stdout:5/610: rename d8/d13/d1e/f27 to d8/d13/d1e/d31/daf/fb1 0 2024-03-22T07:16:02.993 INFO:tasks.workunit.client.1.smithi069.stdout:7/466: mkdir d9/da1 0 2024-03-22T07:16:02.994 INFO:tasks.workunit.client.1.smithi069.stdout:2/351: mkdir d18/d4d/d27/d77/d7c 0 2024-03-22T07:16:02.997 INFO:tasks.workunit.client.0.smithi063.stdout:4/662: getdents d1/d5/d43/d27/d2c/d4a/d80 0 2024-03-22T07:16:02.998 INFO:tasks.workunit.client.0.smithi063.stdout:4/663: read d1/d5/d43/d27/f4b [279207,109194] 0 2024-03-22T07:16:02.998 INFO:tasks.workunit.client.0.smithi063.stdout:4/664: write d1/d5/d43/d27/f4b [868438,53126] 0 2024-03-22T07:16:02.999 INFO:tasks.workunit.client.1.smithi069.stdout:2/352: rename d18/d4d/d27/d33/f35 to d18/d4d/d27/d72/f7d 0 2024-03-22T07:16:02.999 INFO:tasks.workunit.client.1.smithi069.stdout:7/467: dread d9/d76/d57/d44/d53/d74/d30/d3d/f40 [0,4194304] 0 2024-03-22T07:16:02.999 INFO:tasks.workunit.client.1.smithi069.stdout:7/468: truncate d9/db/f94 168925 0 2024-03-22T07:16:03.001 INFO:tasks.workunit.client.1.smithi069.stdout:2/353: rmdir d18/d4d/d7a 0 2024-03-22T07:16:03.001 INFO:tasks.workunit.client.1.smithi069.stdout:8/553: write d1/d1c/f38 [2578203,101050] 0 2024-03-22T07:16:03.001 INFO:tasks.workunit.client.1.smithi069.stdout:8/554: readlink d1/de/l1b 0 2024-03-22T07:16:03.002 INFO:tasks.workunit.client.0.smithi063.stdout:7/578: link c3 d5/da/d5f/d33/cbe 0 2024-03-22T07:16:03.002 INFO:tasks.workunit.client.0.smithi063.stdout:7/579: read - d5/da/d28/d4b/da8/dbc/d85/fb9 zero size 2024-03-22T07:16:03.003 INFO:tasks.workunit.client.1.smithi069.stdout:5/357: sync 2024-03-22T07:16:03.003 INFO:tasks.workunit.client.1.smithi069.stdout:5/358: chown d1/d7/d8/f58 1 1 2024-03-22T07:16:03.004 INFO:tasks.workunit.client.1.smithi069.stdout:8/555: symlink d1/d1c/d3e/d88/da8/lb1 0 2024-03-22T07:16:03.004 INFO:tasks.workunit.client.0.smithi063.stdout:9/702: rename d8/d19/d20/d22/d29/dca to d8/d19/d20/d22/d29/d31/d4b/dd7/dd8 0 2024-03-22T07:16:03.005 INFO:tasks.workunit.client.0.smithi063.stdout:9/703: fsync d8/d19/d20/d22/d29/d31/d53/d6a/da2/f80 0 2024-03-22T07:16:03.005 INFO:tasks.workunit.client.0.smithi063.stdout:9/704: fdatasync d8/d19/d20/d22/fd6 0 2024-03-22T07:16:03.005 INFO:tasks.workunit.client.1.smithi069.stdout:5/359: stat d1/d6/d26/l2d 0 2024-03-22T07:16:03.006 INFO:tasks.workunit.client.0.smithi063.stdout:4/665: getdents d1/d5/d43/d27/d2c/d4a 0 2024-03-22T07:16:03.006 INFO:tasks.workunit.client.1.smithi069.stdout:6/395: write d4/dd/d12/d44/d58/f62 [1102354,91354] 0 2024-03-22T07:16:03.007 INFO:tasks.workunit.client.1.smithi069.stdout:8/556: dread d1/d1c/d3e/d66/f95 [0,4194304] 0 2024-03-22T07:16:03.007 INFO:tasks.workunit.client.1.smithi069.stdout:8/557: getdents d1/d1c/d3e/d66/d9e 0 2024-03-22T07:16:03.007 INFO:tasks.workunit.client.1.smithi069.stdout:8/558: write d1/de/d6c/d8e/f91 [392490,43647] 0 2024-03-22T07:16:03.011 INFO:tasks.workunit.client.0.smithi063.stdout:9/705: rename d8/f10 to d8/d17/d1e/fd9 0 2024-03-22T07:16:03.012 INFO:tasks.workunit.client.1.smithi069.stdout:5/360: dread d1/d7/d8/f52 [0,4194304] 0 2024-03-22T07:16:03.012 INFO:tasks.workunit.client.1.smithi069.stdout:5/361: chown d1/d10/d60/f7b 282 1 2024-03-22T07:16:03.013 INFO:tasks.workunit.client.0.smithi063.stdout:5/611: write d8/d13/d1e/f2f [1735984,75548] 0 2024-03-22T07:16:03.013 INFO:tasks.workunit.client.0.smithi063.stdout:9/706: symlink d8/lda 0 2024-03-22T07:16:03.014 INFO:tasks.workunit.client.1.smithi069.stdout:9/441: dwrite d2/dd/d3c/d2e/f3f [0,4194304] 0 2024-03-22T07:16:03.014 INFO:tasks.workunit.client.1.smithi069.stdout:9/442: getdents d2/dd/d3c/d53/d8b 0 2024-03-22T07:16:03.015 INFO:tasks.workunit.client.0.smithi063.stdout:7/580: dread d5/da/d28/d4b/da8/dbc/d62/d73/f7a [0,4194304] 0 2024-03-22T07:16:03.016 INFO:tasks.workunit.client.1.smithi069.stdout:3/470: dwrite d5/d9/d26/f4c [0,4194304] 0 2024-03-22T07:16:03.018 INFO:tasks.workunit.client.0.smithi063.stdout:5/612: rename d8/d13/l19 to d8/d13/d1e/d31/d64/d65/d86/lb2 0 2024-03-22T07:16:03.018 INFO:tasks.workunit.client.1.smithi069.stdout:6/396: rename d4/dd/d12/f1d to d4/d7/da/f7c 0 2024-03-22T07:16:03.021 INFO:tasks.workunit.client.0.smithi063.stdout:9/707: symlink d8/d19/d20/d22/d9b/ldb 0 2024-03-22T07:16:03.022 INFO:tasks.workunit.client.0.smithi063.stdout:8/779: sync 2024-03-22T07:16:03.022 INFO:tasks.workunit.client.0.smithi063.stdout:2/664: sync 2024-03-22T07:16:03.022 INFO:tasks.workunit.client.0.smithi063.stdout:8/780: chown d0/d1b/f20 585378978 1 2024-03-22T07:16:03.023 INFO:tasks.workunit.client.1.smithi069.stdout:3/471: symlink d5/d9/d26/d3f/l91 0 2024-03-22T07:16:03.023 INFO:tasks.workunit.client.1.smithi069.stdout:3/472: fsync d5/d33/f39 0 2024-03-22T07:16:03.024 INFO:tasks.workunit.client.0.smithi063.stdout:7/581: creat d5/da/d28/d4b/da8/dbc/d85/fbf x:0 0 0 2024-03-22T07:16:03.025 INFO:tasks.workunit.client.1.smithi069.stdout:8/559: rename d1/d1c/d48/d56/l8a to d1/d1c/d70/lb2 0 2024-03-22T07:16:03.025 INFO:tasks.workunit.client.1.smithi069.stdout:8/560: write d1/d1c/d48/d5e/fa5 [1047375,31227] 0 2024-03-22T07:16:03.026 INFO:tasks.workunit.client.1.smithi069.stdout:6/397: mknod d4/dd/d12/d2b/d72/c7d 0 2024-03-22T07:16:03.027 INFO:tasks.workunit.client.1.smithi069.stdout:3/473: symlink d5/d33/d7b/l92 0 2024-03-22T07:16:03.028 INFO:tasks.workunit.client.1.smithi069.stdout:3/474: write d5/d9/d26/d3f/d35/f8d [575218,26396] 0 2024-03-22T07:16:03.029 INFO:tasks.workunit.client.0.smithi063.stdout:9/708: mknod d8/d19/d20/d22/d29/d31/d4b/cdc 0 2024-03-22T07:16:03.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:02 smithi069 ceph-mon[36441]: Deploying cephadm binary to smithi063 2024-03-22T07:16:03.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:02 smithi069 ceph-mon[36441]: Deploying cephadm binary to smithi069 2024-03-22T07:16:03.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:02 smithi069 ceph-mon[36441]: pgmap v3: 65 pgs: 65 active+clean; 2.4 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail 2024-03-22T07:16:03.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:02 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:03.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:02 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:03.030 INFO:tasks.workunit.client.1.smithi069.stdout:8/561: creat d1/fb3 x:0 0 0 2024-03-22T07:16:03.032 INFO:tasks.workunit.client.1.smithi069.stdout:3/475: creat d5/d33/d7b/f93 x:0 0 0 2024-03-22T07:16:03.034 INFO:tasks.workunit.client.0.smithi063.stdout:8/781: symlink d0/d1b/d28/de8/dd1/l10b 0 2024-03-22T07:16:03.034 INFO:tasks.workunit.client.0.smithi063.stdout:7/582: truncate d5/d16/f63 241504 0 2024-03-22T07:16:03.034 INFO:tasks.workunit.client.0.smithi063.stdout:5/613: getdents d8/d13/d28/d7d 0 2024-03-22T07:16:03.034 INFO:tasks.workunit.client.0.smithi063.stdout:8/782: chown d0/d1b/d4f/dde/df5 84 1 2024-03-22T07:16:03.034 INFO:tasks.workunit.client.0.smithi063.stdout:5/614: read - d8/d13/d28/d57/f77 zero size 2024-03-22T07:16:03.034 INFO:tasks.workunit.client.1.smithi069.stdout:5/362: rename d1/d7/cd to d1/d6/c81 0 2024-03-22T07:16:03.034 INFO:tasks.workunit.client.1.smithi069.stdout:3/476: mknod d5/d9/d26/d3f/d56/d7f/d90/c94 0 2024-03-22T07:16:03.035 INFO:tasks.workunit.client.0.smithi063.stdout:9/709: dread d8/d19/d20/d22/d29/d31/d37/f59 [0,4194304] 0 2024-03-22T07:16:03.036 INFO:tasks.workunit.client.0.smithi063.stdout:7/583: mknod d5/da/d28/d4b/da8/dbc/d62/d73/cc0 0 2024-03-22T07:16:03.036 INFO:tasks.workunit.client.0.smithi063.stdout:7/584: chown d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/l79 0 1 2024-03-22T07:16:03.036 INFO:tasks.workunit.client.0.smithi063.stdout:7/585: fdatasync d5/da/d5f/f5c 0 2024-03-22T07:16:03.036 INFO:tasks.workunit.client.0.smithi063.stdout:7/586: write d5/d16/f32 [1728733,92865] 0 2024-03-22T07:16:03.036 INFO:tasks.workunit.client.0.smithi063.stdout:7/587: dread - d5/da/d28/da7/fb2 zero size 2024-03-22T07:16:03.037 INFO:tasks.workunit.client.1.smithi069.stdout:5/363: rename d1/d10/l29 to d1/d28/l82 0 2024-03-22T07:16:03.037 INFO:tasks.workunit.client.1.smithi069.stdout:5/364: creat d1/d10/d60/f83 x:0 0 0 2024-03-22T07:16:03.037 INFO:tasks.workunit.client.1.smithi069.stdout:9/443: dread d2/dd/d3c/d2e/f3f [0,4194304] 0 2024-03-22T07:16:03.037 INFO:tasks.workunit.client.1.smithi069.stdout:3/477: unlink d5/f7 0 2024-03-22T07:16:03.038 INFO:tasks.workunit.client.0.smithi063.stdout:8/783: creat d0/d1b/d28/d3e/db2/dbd/dd0/f10c x:0 0 0 2024-03-22T07:16:03.038 INFO:tasks.workunit.client.0.smithi063.stdout:8/784: read d0/d1b/fc8 [361963,80003] 0 2024-03-22T07:16:03.038 INFO:tasks.workunit.client.0.smithi063.stdout:9/710: mknod d8/d19/d20/d22/db2/cdd 0 2024-03-22T07:16:03.039 INFO:tasks.workunit.client.1.smithi069.stdout:9/444: creat d2/d71/f8d x:0 0 0 2024-03-22T07:16:03.040 INFO:tasks.workunit.client.0.smithi063.stdout:7/588: link d5/d16/d43/l8e d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d4f/lc1 0 2024-03-22T07:16:03.042 INFO:tasks.workunit.client.1.smithi069.stdout:3/478: creat d5/d9/db/d3b/f95 x:0 0 0 2024-03-22T07:16:03.042 INFO:tasks.workunit.client.1.smithi069.stdout:3/479: write d5/d9/d23/f2b [4471795,91763] 0 2024-03-22T07:16:03.043 INFO:tasks.workunit.client.0.smithi063.stdout:8/785: creat d0/d1b/f10d x:0 0 0 2024-03-22T07:16:03.043 INFO:tasks.workunit.client.0.smithi063.stdout:8/786: stat d0/d24/d5b/f44 0 2024-03-22T07:16:03.043 INFO:tasks.workunit.client.0.smithi063.stdout:8/787: fdatasync d0/d1b/d28/d3e/db2/dbd/dd0/f10c 0 2024-03-22T07:16:03.048 INFO:tasks.workunit.client.1.smithi069.stdout:2/354: dwrite d18/d4d/d25/f5b [0,4194304] 0 2024-03-22T07:16:03.048 INFO:tasks.workunit.client.1.smithi069.stdout:8/562: dread d1/de/f40 [0,4194304] 0 2024-03-22T07:16:03.048 INFO:tasks.workunit.client.1.smithi069.stdout:3/480: rename d5/f18 to d5/d9/d26/d3f/d56/d7f/f96 0 2024-03-22T07:16:03.050 INFO:tasks.workunit.client.0.smithi063.stdout:7/589: symlink d5/da/d5f/lc2 0 2024-03-22T07:16:03.050 INFO:tasks.workunit.client.1.smithi069.stdout:5/365: dread d1/d7/d8/f12 [4194304,4194304] 0 2024-03-22T07:16:03.051 INFO:tasks.workunit.client.0.smithi063.stdout:5/615: dread d8/d13/d1e/d31/daf/fb1 [4194304,4194304] 0 2024-03-22T07:16:03.052 INFO:tasks.workunit.client.0.smithi063.stdout:4/666: dwrite d1/d5/f20 [4194304,4194304] 0 2024-03-22T07:16:03.054 INFO:tasks.workunit.client.0.smithi063.stdout:7/590: mknod d5/da/d5f/cc3 0 2024-03-22T07:16:03.057 INFO:tasks.workunit.client.0.smithi063.stdout:9/711: read d8/d19/d20/d22/f27 [2539269,6206] 0 2024-03-22T07:16:03.058 INFO:tasks.workunit.client.0.smithi063.stdout:2/665: dwrite d0/de/f4c [0,4194304] 0 2024-03-22T07:16:03.058 INFO:tasks.workunit.client.1.smithi069.stdout:8/563: symlink d1/lb4 0 2024-03-22T07:16:03.059 INFO:tasks.workunit.client.0.smithi063.stdout:4/667: mknod d1/d5/d43/d27/d2c/d4a/d78/ce2 0 2024-03-22T07:16:03.060 INFO:tasks.workunit.client.0.smithi063.stdout:4/668: truncate d1/d5/d3c/f83 4532624 0 2024-03-22T07:16:03.060 INFO:tasks.workunit.client.1.smithi069.stdout:3/481: symlink d5/d9/d26/d3f/d56/l97 0 2024-03-22T07:16:03.060 INFO:tasks.workunit.client.1.smithi069.stdout:6/398: dwrite d4/dd/d12/d44/f4e [0,4194304] 0 2024-03-22T07:16:03.060 INFO:tasks.workunit.client.1.smithi069.stdout:5/366: symlink d1/d10/d44/l84 0 2024-03-22T07:16:03.061 INFO:tasks.workunit.client.0.smithi063.stdout:9/712: symlink d8/d19/d20/d22/d29/d31/d4b/dd7/dd8/lde 0 2024-03-22T07:16:03.062 INFO:tasks.workunit.client.0.smithi063.stdout:9/713: dread d8/d15/fa1 [0,4194304] 0 2024-03-22T07:16:03.063 INFO:tasks.workunit.client.0.smithi063.stdout:5/616: rename d8/d13/c2d to d8/d13/d28/d57/d5f/cb3 0 2024-03-22T07:16:03.067 INFO:tasks.workunit.client.0.smithi063.stdout:4/669: unlink d1/d5/c18 0 2024-03-22T07:16:03.067 INFO:tasks.workunit.client.1.smithi069.stdout:5/367: symlink d1/d10/d60/l85 0 2024-03-22T07:16:03.068 INFO:tasks.workunit.client.0.smithi063.stdout:9/714: rmdir d8/d19/d20/d22/d5d/d76/dab 39 2024-03-22T07:16:03.074 INFO:tasks.workunit.client.0.smithi063.stdout:4/670: mknod d1/d5/d43/d27/d2c/d4a/d78/ce3 0 2024-03-22T07:16:03.078 INFO:tasks.workunit.client.0.smithi063.stdout:5/617: rename d8/d13/d1e/d31/f4e to d8/d13/d28/d57/d5f/fb4 0 2024-03-22T07:16:03.080 INFO:tasks.workunit.client.0.smithi063.stdout:4/671: mkdir d1/d5/d3c/d65/de4 0 2024-03-22T07:16:03.081 INFO:tasks.workunit.client.0.smithi063.stdout:5/618: truncate d8/d13/d1e/f38 4579655 0 2024-03-22T07:16:03.082 INFO:tasks.workunit.client.0.smithi063.stdout:9/715: rename d8/d19/d20/d22/d5d/d76/d95/fbd to d8/d19/d39/db5/fdf 0 2024-03-22T07:16:03.084 INFO:tasks.workunit.client.0.smithi063.stdout:5/619: creat d8/d13/d24/da4/d8b/fb5 x:0 0 0 2024-03-22T07:16:03.091 INFO:tasks.workunit.client.0.smithi063.stdout:8/788: dwrite d0/d24/f5d [0,4194304] 0 2024-03-22T07:16:03.091 INFO:tasks.workunit.client.0.smithi063.stdout:2/666: dread d0/f9 [0,4194304] 0 2024-03-22T07:16:03.091 INFO:tasks.workunit.client.0.smithi063.stdout:2/667: fsync d0/de/d1b/d63/f86 0 2024-03-22T07:16:03.091 INFO:tasks.workunit.client.0.smithi063.stdout:2/668: chown d0/de/d74/f7e 126925 1 2024-03-22T07:16:03.091 INFO:tasks.workunit.client.0.smithi063.stdout:2/669: write d0/f8 [1257447,27470] 0 2024-03-22T07:16:03.094 INFO:tasks.workunit.client.0.smithi063.stdout:8/789: rename d0/d1b/d28/d3e/db2/dbd/dd0/cdc to d0/d26/d2d/d59/dc0/c10e 0 2024-03-22T07:16:03.096 INFO:tasks.workunit.client.0.smithi063.stdout:2/670: mknod d0/de/d58/d7f/d97/da2/cd7 0 2024-03-22T07:16:03.097 INFO:tasks.workunit.client.0.smithi063.stdout:8/790: rename d0/d1b/d28/de8/dd1/l10b to d0/d24/d33/dd9/l10f 0 2024-03-22T07:16:03.098 INFO:tasks.workunit.client.0.smithi063.stdout:2/671: symlink d0/de/d1b/ld8 0 2024-03-22T07:16:03.098 INFO:tasks.workunit.client.0.smithi063.stdout:2/672: truncate d0/de/d58/d7f/d97/fa1 775123 0 2024-03-22T07:16:03.099 INFO:tasks.workunit.client.0.smithi063.stdout:8/791: getdents d0/d26/d2d/d8f 0 2024-03-22T07:16:03.101 INFO:tasks.workunit.client.0.smithi063.stdout:8/792: creat d0/d26/d2d/f110 x:0 0 0 2024-03-22T07:16:03.102 INFO:tasks.workunit.client.1.smithi069.stdout:5/368: rename d1/d7/d3d/d3f to d1/d6/d86 0 2024-03-22T07:16:03.106 INFO:tasks.workunit.client.1.smithi069.stdout:5/369: unlink d1/d6/d21/f63 0 2024-03-22T07:16:03.107 INFO:tasks.workunit.client.1.smithi069.stdout:9/445: getdents d2/d71 0 2024-03-22T07:16:03.107 INFO:tasks.workunit.client.1.smithi069.stdout:9/446: fsync d2/f5 0 2024-03-22T07:16:03.107 INFO:tasks.workunit.client.1.smithi069.stdout:9/447: truncate d2/dd/f10 4323963 0 2024-03-22T07:16:03.110 INFO:tasks.workunit.client.1.smithi069.stdout:9/448: creat d2/dd/d3c/d48/d7f/d86/f8e x:0 0 0 2024-03-22T07:16:03.110 INFO:tasks.workunit.client.1.smithi069.stdout:9/449: creat d2/f8f x:0 0 0 2024-03-22T07:16:03.112 INFO:tasks.workunit.client.1.smithi069.stdout:8/564: dwrite d1/d1c/d48/f7a [0,4194304] 0 2024-03-22T07:16:03.112 INFO:tasks.workunit.client.1.smithi069.stdout:8/565: stat d1/d1c/d48/d5e/fad 0 2024-03-22T07:16:03.112 INFO:tasks.workunit.client.0.smithi063.stdout:7/591: dwrite d5/da/d5f/f8a [0,4194304] 0 2024-03-22T07:16:03.113 INFO:tasks.workunit.client.0.smithi063.stdout:7/592: fsync d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/d4a/f7e 0 2024-03-22T07:16:03.113 INFO:tasks.workunit.client.0.smithi063.stdout:7/593: creat d5/da/d28/d4b/da8/dbc/d85/fc4 x:0 0 0 2024-03-22T07:16:03.114 INFO:tasks.workunit.client.1.smithi069.stdout:3/482: dwrite d5/d9/d26/d36/f52 [0,4194304] 0 2024-03-22T07:16:03.116 INFO:tasks.workunit.client.1.smithi069.stdout:6/399: dwrite d4/f47 [0,4194304] 0 2024-03-22T07:16:03.117 INFO:tasks.workunit.client.1.smithi069.stdout:9/450: mkdir d2/d71/d90 0 2024-03-22T07:16:03.117 INFO:tasks.workunit.client.1.smithi069.stdout:9/451: write d2/dd/d3c/f35 [733299,33228] 0 2024-03-22T07:16:03.118 INFO:tasks.workunit.client.0.smithi063.stdout:5/620: dwrite d8/d13/d24/da4/d8b/fb5 [0,4194304] 0 2024-03-22T07:16:03.118 INFO:tasks.workunit.client.0.smithi063.stdout:9/716: dwrite d8/f14 [4194304,4194304] 0 2024-03-22T07:16:03.118 INFO:tasks.workunit.client.0.smithi063.stdout:8/793: dread d0/d24/f5d [0,4194304] 0 2024-03-22T07:16:03.118 INFO:tasks.workunit.client.0.smithi063.stdout:8/794: fdatasync d0/d1b/f6e 0 2024-03-22T07:16:03.118 INFO:tasks.workunit.client.0.smithi063.stdout:8/795: write d0/d1b/d28/d3e/fc2 [1050089,76114] 0 2024-03-22T07:16:03.119 INFO:tasks.workunit.client.1.smithi069.stdout:8/566: rename d1/d1c/d3e/d88/c8d to d1/de/d6c/d8e/cb5 0 2024-03-22T07:16:03.119 INFO:tasks.workunit.client.0.smithi063.stdout:7/594: mkdir d5/da/d28/d4b/da8/dbc/d62/d73/dc5 0 2024-03-22T07:16:03.119 INFO:tasks.workunit.client.0.smithi063.stdout:7/595: truncate d5/d16/d1b/f80 4604060 0 2024-03-22T07:16:03.124 INFO:tasks.workunit.client.0.smithi063.stdout:9/717: link d8/d19/d20/d22/d5d/d76/dc3/cce d8/d19/d20/d22/d5d/d76/d95/ce0 0 2024-03-22T07:16:03.124 INFO:tasks.workunit.client.0.smithi063.stdout:9/718: write d8/d19/d20/d22/d29/d31/d53/d6a/da2/f74 [98730,71672] 0 2024-03-22T07:16:03.125 INFO:tasks.workunit.client.1.smithi069.stdout:3/483: rename d5/d9/d26/d3f/l7e to d5/d33/d7b/l98 0 2024-03-22T07:16:03.125 INFO:tasks.workunit.client.0.smithi063.stdout:8/796: rmdir d0/d26/d2d/d59/dba/dc9 39 2024-03-22T07:16:03.127 INFO:tasks.workunit.client.1.smithi069.stdout:8/567: symlink d1/de/d6c/d4e/d5b/lb6 0 2024-03-22T07:16:03.128 INFO:tasks.workunit.client.1.smithi069.stdout:9/452: read d2/dd/d3c/d2e/f30 [642291,43749] 0 2024-03-22T07:16:03.128 INFO:tasks.workunit.client.0.smithi063.stdout:7/596: creat d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/d4a/fc6 x:0 0 0 2024-03-22T07:16:03.130 INFO:tasks.workunit.client.0.smithi063.stdout:9/719: rmdir d8/d19/d20/d22/d29/d31/d4b/db7 39 2024-03-22T07:16:03.131 INFO:tasks.workunit.client.1.smithi069.stdout:9/453: rename d2/d31/c5f to d2/dd/d3c/d48/d7c/c91 0 2024-03-22T07:16:03.131 INFO:tasks.workunit.client.1.smithi069.stdout:9/454: stat d2/c62 0 2024-03-22T07:16:03.132 INFO:tasks.workunit.client.1.smithi069.stdout:6/400: dread f3 [4194304,4194304] 0 2024-03-22T07:16:03.132 INFO:tasks.workunit.client.1.smithi069.stdout:6/401: write d4/dd/d12/f1f [2903813,16641] 0 2024-03-22T07:16:03.133 INFO:tasks.workunit.client.0.smithi063.stdout:8/797: symlink d0/d26/d2d/d8f/l111 0 2024-03-22T07:16:03.134 INFO:tasks.workunit.client.0.smithi063.stdout:2/673: dwrite d0/de/d74/f7e [4194304,4194304] 0 2024-03-22T07:16:03.134 INFO:tasks.workunit.client.0.smithi063.stdout:2/674: stat d0/de/d58 0 2024-03-22T07:16:03.135 INFO:tasks.workunit.client.0.smithi063.stdout:9/720: getdents d8/d19/d20/d22/d5d/d76/dc3 0 2024-03-22T07:16:03.135 INFO:tasks.workunit.client.0.smithi063.stdout:9/721: dread - d8/d19/d20/d22/db2/fb9 zero size 2024-03-22T07:16:03.139 INFO:tasks.workunit.client.1.smithi069.stdout:3/484: write d5/d9/d26/d3f/d35/f6b [83179,30227] 0 2024-03-22T07:16:03.139 INFO:tasks.workunit.client.1.smithi069.stdout:8/568: dread d1/d1c/d3e/f61 [0,4194304] 0 2024-03-22T07:16:03.139 INFO:tasks.workunit.client.1.smithi069.stdout:8/569: dread - d1/de/d6c/d4e/d5b/fae zero size 2024-03-22T07:16:03.139 INFO:tasks.workunit.client.1.smithi069.stdout:5/370: dwrite d1/d6/d86/f73 [0,4194304] 0 2024-03-22T07:16:03.139 INFO:tasks.workunit.client.1.smithi069.stdout:5/371: fdatasync d1/d6/ff 0 2024-03-22T07:16:03.140 INFO:tasks.workunit.client.0.smithi063.stdout:5/621: dread d8/d13/d24/f50 [4194304,4194304] 0 2024-03-22T07:16:03.141 INFO:tasks.workunit.client.1.smithi069.stdout:3/485: read d5/d9/f5e [231235,30367] 0 2024-03-22T07:16:03.142 INFO:tasks.workunit.client.0.smithi063.stdout:8/798: rename d0/d26/d2d/d59/dba/df4 to d0/d26/d2d/d83/d112 0 2024-03-22T07:16:03.142 INFO:tasks.workunit.client.0.smithi063.stdout:8/799: chown d0/d26/d2d/d8f/l111 1350 1 2024-03-22T07:16:03.144 INFO:tasks.workunit.client.1.smithi069.stdout:6/402: dread f3 [4194304,4194304] 0 2024-03-22T07:16:03.145 INFO:tasks.workunit.client.0.smithi063.stdout:9/722: dread d8/d19/d39/f6f [0,4194304] 0 2024-03-22T07:16:03.146 INFO:tasks.workunit.client.1.smithi069.stdout:8/570: link d1/d1c/f96 d1/d1c/d3e/fb7 0 2024-03-22T07:16:03.146 INFO:tasks.workunit.client.0.smithi063.stdout:2/675: symlink d0/de/d58/dbb/ld9 0 2024-03-22T07:16:03.147 INFO:tasks.workunit.client.1.smithi069.stdout:9/455: dread d2/dd/d3c/f35 [0,4194304] 0 2024-03-22T07:16:03.147 INFO:tasks.workunit.client.1.smithi069.stdout:5/372: mknod d1/d7/d8/c87 0 2024-03-22T07:16:03.148 INFO:tasks.workunit.client.1.smithi069.stdout:3/486: creat d5/d33/f99 x:0 0 0 2024-03-22T07:16:03.149 INFO:tasks.workunit.client.1.smithi069.stdout:6/403: mknod d4/d7/da/d3f/c7e 0 2024-03-22T07:16:03.149 INFO:tasks.workunit.client.1.smithi069.stdout:8/571: stat d1/de/c90 0 2024-03-22T07:16:03.149 INFO:tasks.workunit.client.1.smithi069.stdout:8/572: fsync d1/f1d 0 2024-03-22T07:16:03.150 INFO:tasks.workunit.client.1.smithi069.stdout:4/534: sync 2024-03-22T07:16:03.150 INFO:tasks.workunit.client.1.smithi069.stdout:0/544: sync 2024-03-22T07:16:03.151 INFO:tasks.workunit.client.0.smithi063.stdout:7/597: dwrite d5/da/d5f/d33/fbd [0,4194304] 0 2024-03-22T07:16:03.151 INFO:tasks.workunit.client.1.smithi069.stdout:3/487: mknod d5/d9/d26/d3f/d56/d7f/c9a 0 2024-03-22T07:16:03.151 INFO:tasks.workunit.client.1.smithi069.stdout:3/488: fsync d5/d9/d23/f6d 0 2024-03-22T07:16:03.151 INFO:tasks.workunit.client.1.smithi069.stdout:3/489: dread - d5/d9/d26/d36/d41/d51/f7c zero size 2024-03-22T07:16:03.152 INFO:tasks.workunit.client.1.smithi069.stdout:8/573: mknod d1/d1c/cb8 0 2024-03-22T07:16:03.152 INFO:tasks.workunit.client.1.smithi069.stdout:8/574: chown d1/de/ca3 1 1 2024-03-22T07:16:03.152 INFO:tasks.workunit.client.1.smithi069.stdout:8/575: chown d1/d1c/d48/d5e/f8b 5059129 1 2024-03-22T07:16:03.153 INFO:tasks.workunit.client.1.smithi069.stdout:5/373: mkdir d1/d6/d88 0 2024-03-22T07:16:03.153 INFO:tasks.workunit.client.1.smithi069.stdout:3/490: dread d5/d9/db/f63 [0,4194304] 0 2024-03-22T07:16:03.154 INFO:tasks.workunit.client.1.smithi069.stdout:4/535: mkdir d6/d16/d3a/db4 0 2024-03-22T07:16:03.154 INFO:tasks.workunit.client.1.smithi069.stdout:4/536: chown d6/d9b/d1f/d58/cad 35939 1 2024-03-22T07:16:03.155 INFO:tasks.workunit.client.1.smithi069.stdout:0/545: mkdir d3/d7/d15/d29/d5a/d5d/db4 0 2024-03-22T07:16:03.155 INFO:tasks.workunit.client.1.smithi069.stdout:0/546: write d3/d7/d15/dad/fae [28744,61969] 0 2024-03-22T07:16:03.155 INFO:tasks.workunit.client.1.smithi069.stdout:0/547: creat d3/d7/d17/d1c/fb5 x:0 0 0 2024-03-22T07:16:03.156 INFO:tasks.workunit.client.1.smithi069.stdout:8/576: mkdir d1/d1c/d3e/db9 0 2024-03-22T07:16:03.156 INFO:tasks.workunit.client.1.smithi069.stdout:8/577: truncate d1/de/d6c/d4e/d5b/f94 925387 0 2024-03-22T07:16:03.156 INFO:tasks.workunit.client.0.smithi063.stdout:5/622: mkdir d8/d13/d24/da4/d8b/db6 0 2024-03-22T07:16:03.156 INFO:tasks.workunit.client.0.smithi063.stdout:5/623: creat d8/d13/d1e/d3c/fb7 x:0 0 0 2024-03-22T07:16:03.157 INFO:tasks.workunit.client.1.smithi069.stdout:3/491: unlink d5/d9/f13 0 2024-03-22T07:16:03.157 INFO:tasks.workunit.client.1.smithi069.stdout:3/492: truncate d5/d9/d26/d3f/d35/f7a 391194 0 2024-03-22T07:16:03.158 INFO:tasks.workunit.client.1.smithi069.stdout:0/548: link d3/d7/d5e/l81 d3/d7/d15/dad/lb6 0 2024-03-22T07:16:03.159 INFO:tasks.workunit.client.1.smithi069.stdout:8/578: symlink d1/d1c/d3e/d66/lba 0 2024-03-22T07:16:03.159 INFO:tasks.workunit.client.1.smithi069.stdout:3/493: mknod d5/d33/d7b/c9b 0 2024-03-22T07:16:03.159 INFO:tasks.workunit.client.1.smithi069.stdout:3/494: creat d5/d33/f9c x:0 0 0 2024-03-22T07:16:03.160 INFO:tasks.workunit.client.0.smithi063.stdout:8/800: getdents d0/d1b/d28 0 2024-03-22T07:16:03.161 INFO:tasks.workunit.client.1.smithi069.stdout:0/549: rmdir d3/d7/d17/d8d 0 2024-03-22T07:16:03.161 INFO:tasks.workunit.client.1.smithi069.stdout:0/550: write d3/d7/d15/d29/d6e/f95 [980299,23595] 0 2024-03-22T07:16:03.161 INFO:tasks.workunit.client.1.smithi069.stdout:0/551: chown d3/d7/d24/d6a/l92 70779 1 2024-03-22T07:16:03.161 INFO:tasks.workunit.client.1.smithi069.stdout:0/552: write d3/d7/d15/d29/d5a/d5d/fb2 [159512,117288] 0 2024-03-22T07:16:03.161 INFO:tasks.workunit.client.1.smithi069.stdout:8/579: mkdir d1/d1c/d3e/d88/da8/dbb 0 2024-03-22T07:16:03.163 INFO:tasks.workunit.client.1.smithi069.stdout:3/495: mkdir d5/d9/d26/d3f/d54/d9d 0 2024-03-22T07:16:03.163 INFO:tasks.workunit.client.1.smithi069.stdout:3/496: creat d5/d9/d26/d3f/d35/f9e x:0 0 0 2024-03-22T07:16:03.164 INFO:tasks.workunit.client.0.smithi063.stdout:9/723: rmdir d8/d19/d20/d22/d29/d31/d4b/db4 0 2024-03-22T07:16:03.164 INFO:tasks.workunit.client.0.smithi063.stdout:9/724: fsync d8/d15/f1c 0 2024-03-22T07:16:03.165 INFO:tasks.workunit.client.1.smithi069.stdout:8/580: creat d1/d1c/d3e/d66/d9e/fbc x:0 0 0 2024-03-22T07:16:03.166 INFO:tasks.workunit.client.0.smithi063.stdout:2/676: unlink d0/de/c1a 0 2024-03-22T07:16:03.166 INFO:tasks.workunit.client.0.smithi063.stdout:2/677: fdatasync d0/de/d74/dca/fd2 0 2024-03-22T07:16:03.171 INFO:tasks.workunit.client.1.smithi069.stdout:4/537: dread d6/d12/d18/d21/f79 [0,4194304] 0 2024-03-22T07:16:03.172 INFO:tasks.workunit.client.1.smithi069.stdout:6/404: dread d4/f47 [0,4194304] 0 2024-03-22T07:16:03.174 INFO:tasks.workunit.client.0.smithi063.stdout:2/678: mknod d0/de/d1b/d63/dd6/cda 0 2024-03-22T07:16:03.174 INFO:tasks.workunit.client.0.smithi063.stdout:2/679: read d0/de/d12/d20/fc7 [463444,3932] 0 2024-03-22T07:16:03.176 INFO:tasks.workunit.client.0.smithi063.stdout:2/680: mkdir d0/de/d1b/d63/d6e/ddb 0 2024-03-22T07:16:03.179 INFO:tasks.workunit.client.0.smithi063.stdout:2/681: unlink d0/de/d1b/f50 0 2024-03-22T07:16:03.180 INFO:tasks.workunit.client.1.smithi069.stdout:4/538: mkdir d6/d9b/d4a/db5 0 2024-03-22T07:16:03.180 INFO:tasks.workunit.client.1.smithi069.stdout:6/405: mkdir d4/dd/d12/d44/d7f 0 2024-03-22T07:16:03.180 INFO:tasks.workunit.client.1.smithi069.stdout:6/406: chown d4/dd/d12/d2b/f36 404159905 1 2024-03-22T07:16:03.181 INFO:tasks.workunit.client.0.smithi063.stdout:8/801: dread d0/d1b/d28/d3e/fec [0,4194304] 0 2024-03-22T07:16:03.182 INFO:tasks.workunit.client.1.smithi069.stdout:6/407: symlink d4/d7/da/d3f/l80 0 2024-03-22T07:16:03.184 INFO:tasks.workunit.client.1.smithi069.stdout:0/553: dread d3/d7/d15/fa3 [0,4194304] 0 2024-03-22T07:16:03.184 INFO:tasks.workunit.client.1.smithi069.stdout:0/554: truncate d3/d7/d17/d1c/f91 585140 0 2024-03-22T07:16:03.184 INFO:tasks.workunit.client.1.smithi069.stdout:0/555: chown d3/d7/f8e 3879 1 2024-03-22T07:16:03.184 INFO:tasks.workunit.client.1.smithi069.stdout:0/556: truncate d3/d7/d15/d29/d5a/fa7 685194 0 2024-03-22T07:16:03.184 INFO:tasks.workunit.client.1.smithi069.stdout:6/408: symlink d4/dd/d12/d44/d7f/l81 0 2024-03-22T07:16:03.185 INFO:tasks.workunit.client.1.smithi069.stdout:0/557: creat d3/d9a/fb7 x:0 0 0 2024-03-22T07:16:03.185 INFO:tasks.workunit.client.1.smithi069.stdout:0/558: chown d3/d7/d17/l2a 48729 1 2024-03-22T07:16:03.185 INFO:tasks.workunit.client.1.smithi069.stdout:0/559: chown d3/d7/d15/d29/d5a/d84/fa0 49772355 1 2024-03-22T07:16:03.185 INFO:tasks.workunit.client.1.smithi069.stdout:0/560: truncate d3/d7/d17/f85 1284929 0 2024-03-22T07:16:03.187 INFO:tasks.workunit.client.1.smithi069.stdout:6/409: getdents d4/dd/d12/d44/d58 0 2024-03-22T07:16:03.189 INFO:tasks.workunit.client.1.smithi069.stdout:0/561: symlink d3/d7/d15/lb8 0 2024-03-22T07:16:03.190 INFO:tasks.workunit.client.1.smithi069.stdout:0/562: mknod d3/d7/d15/d29/d5a/d99/cb9 0 2024-03-22T07:16:03.191 INFO:tasks.workunit.client.1.smithi069.stdout:0/563: chown d3/d7/c10 821235 1 2024-03-22T07:16:03.191 INFO:tasks.workunit.client.1.smithi069.stdout:0/564: readlink d3/d7/d24/l89 0 2024-03-22T07:16:03.191 INFO:tasks.workunit.client.0.smithi063.stdout:9/725: rename d8/d19/d20/d22/d29/d31/d4b to d8/d19/d20/d22/de1 0 2024-03-22T07:16:03.192 INFO:tasks.workunit.client.0.smithi063.stdout:9/726: write d8/d19/d20/fa0 [6472216,10850] 0 2024-03-22T07:16:03.192 INFO:tasks.workunit.client.1.smithi069.stdout:6/410: mkdir d4/d82 0 2024-03-22T07:16:03.192 INFO:tasks.workunit.client.1.smithi069.stdout:6/411: getdents d4/d7/da/d3f/d76 0 2024-03-22T07:16:03.194 INFO:tasks.workunit.client.0.smithi063.stdout:9/727: dread d8/d19/f35 [0,4194304] 0 2024-03-22T07:16:03.195 INFO:tasks.workunit.client.0.smithi063.stdout:6/695: sync 2024-03-22T07:16:03.196 INFO:tasks.workunit.client.0.smithi063.stdout:6/696: getdents d3/d6/dcd 0 2024-03-22T07:16:03.198 INFO:tasks.workunit.client.0.smithi063.stdout:6/697: creat d3/d15/d16/d60/d6d/fe9 x:0 0 0 2024-03-22T07:16:03.199 INFO:tasks.workunit.client.0.smithi063.stdout:2/682: dread d0/de/d12/d20/f2c [0,4194304] 0 2024-03-22T07:16:03.201 INFO:tasks.workunit.client.0.smithi063.stdout:6/698: link d3/d6/d35/d3a/ld3 d3/d15/d16/d60/d6d/d4e/d5a/d88/lea 0 2024-03-22T07:16:03.201 INFO:tasks.workunit.client.0.smithi063.stdout:6/699: truncate d3/d15/f85 980129 0 2024-03-22T07:16:03.201 INFO:tasks.workunit.client.0.smithi063.stdout:6/700: readlink d3/d6/d2a/d38/da5/d32/d51/l9c 0 2024-03-22T07:16:03.201 INFO:tasks.workunit.client.0.smithi063.stdout:6/701: fdatasync d3/d15/d16/d60/d6d/d4e/d5a/d88/fc1 0 2024-03-22T07:16:03.202 INFO:tasks.workunit.client.0.smithi063.stdout:2/683: unlink d0/de/d35/f6f 0 2024-03-22T07:16:03.202 INFO:tasks.workunit.client.0.smithi063.stdout:6/702: mkdir d3/d15/d16/d57/dde/deb 0 2024-03-22T07:16:03.203 INFO:tasks.workunit.client.0.smithi063.stdout:2/684: rmdir d0/de/d74 39 2024-03-22T07:16:03.203 INFO:tasks.workunit.client.0.smithi063.stdout:2/685: dread - d0/de/d35/d36/f6d zero size 2024-03-22T07:16:03.205 INFO:tasks.workunit.client.0.smithi063.stdout:6/703: mkdir d3/d15/d16/d60/d6d/d4e/dec 0 2024-03-22T07:16:03.206 INFO:tasks.workunit.client.0.smithi063.stdout:2/686: rename d0/de/d58/d7f/d97/cbc to d0/de/d74/dca/cdc 0 2024-03-22T07:16:03.207 INFO:tasks.workunit.client.1.smithi069.stdout:0/565: write d3/d7/f12 [2217134,80736] 0 2024-03-22T07:16:03.208 INFO:tasks.workunit.client.1.smithi069.stdout:0/566: mknod d3/cba 0 2024-03-22T07:16:03.208 INFO:tasks.workunit.client.1.smithi069.stdout:0/567: dread - d3/d7/d17/f8f zero size 2024-03-22T07:16:03.208 INFO:tasks.workunit.client.1.smithi069.stdout:8/581: dwrite d1/d1c/d3e/f32 [0,4194304] 0 2024-03-22T07:16:03.208 INFO:tasks.workunit.client.1.smithi069.stdout:8/582: getdents d1/d1c/d3e/d88/da8/dbb 0 2024-03-22T07:16:03.209 INFO:tasks.workunit.client.1.smithi069.stdout:0/568: symlink d3/d7/d15/d1f/lbb 0 2024-03-22T07:16:03.209 INFO:tasks.workunit.client.0.smithi063.stdout:7/598: dwrite d5/da/d5f/f8a [0,4194304] 0 2024-03-22T07:16:03.211 INFO:tasks.workunit.client.0.smithi063.stdout:7/599: mknod d5/d38/cc7 0 2024-03-22T07:16:03.212 INFO:tasks.workunit.client.1.smithi069.stdout:8/583: truncate d1/d1c/d3e/f61 2857092 0 2024-03-22T07:16:03.212 INFO:tasks.workunit.client.1.smithi069.stdout:8/584: fsync d1/d1c/d48/f7a 0 2024-03-22T07:16:03.212 INFO:tasks.workunit.client.1.smithi069.stdout:8/585: readlink d1/de/l1e 0 2024-03-22T07:16:03.213 INFO:tasks.workunit.client.1.smithi069.stdout:5/374: dwrite d1/d6/d21/f79 [0,4194304] 0 2024-03-22T07:16:03.213 INFO:tasks.workunit.client.1.smithi069.stdout:6/412: dread d4/d7/da/d3f/f61 [0,4194304] 0 2024-03-22T07:16:03.213 INFO:tasks.workunit.client.1.smithi069.stdout:6/413: fdatasync d4/f6b 0 2024-03-22T07:16:03.213 INFO:tasks.workunit.client.1.smithi069.stdout:0/569: symlink d3/d7/d90/lbc 0 2024-03-22T07:16:03.214 INFO:tasks.workunit.client.0.smithi063.stdout:7/600: unlink d5/da/d28/d4b/da8/dbc/d62/d73/fa6 0 2024-03-22T07:16:03.214 INFO:tasks.workunit.client.0.smithi063.stdout:7/601: creat d5/da/d28/d4b/da8/fc8 x:0 0 0 2024-03-22T07:16:03.214 INFO:tasks.workunit.client.0.smithi063.stdout:7/602: dread - d5/da/d28/d4b/da8/dbc/d62/d8d/d98/fb7 zero size 2024-03-22T07:16:03.214 INFO:tasks.workunit.client.1.smithi069.stdout:8/586: mkdir d1/de/dbd 0 2024-03-22T07:16:03.214 INFO:tasks.workunit.client.1.smithi069.stdout:8/587: write d1/de/d20/f3c [1079325,77560] 0 2024-03-22T07:16:03.215 INFO:tasks.workunit.client.0.smithi063.stdout:7/603: creat d5/da/d28/d4b/da8/dbc/d62/d73/dc5/fc9 x:0 0 0 2024-03-22T07:16:03.215 INFO:tasks.workunit.client.1.smithi069.stdout:8/588: dread d1/de/d6c/d4e/d5b/f94 [0,4194304] 0 2024-03-22T07:16:03.216 INFO:tasks.workunit.client.0.smithi063.stdout:5/624: dwrite d8/d13/f63 [0,4194304] 0 2024-03-22T07:16:03.216 INFO:tasks.workunit.client.1.smithi069.stdout:4/539: dwrite d6/d16/f57 [0,4194304] 0 2024-03-22T07:16:03.216 INFO:tasks.workunit.client.1.smithi069.stdout:4/540: fdatasync d6/d9b/d1f/d58/d7c/d90/f95 0 2024-03-22T07:16:03.217 INFO:tasks.workunit.client.1.smithi069.stdout:5/375: rename d1/d28/c31 to d1/d7/d8/d43/c89 0 2024-03-22T07:16:03.217 INFO:tasks.workunit.client.1.smithi069.stdout:3/497: dwrite d5/d9/d23/f6d [0,4194304] 0 2024-03-22T07:16:03.217 INFO:tasks.workunit.client.1.smithi069.stdout:6/414: link d4/d7/da/f5a d4/d7/da/f83 0 2024-03-22T07:16:03.217 INFO:tasks.workunit.client.1.smithi069.stdout:6/415: write d4/dd/d12/d44/d58/f63 [675098,1632] 0 2024-03-22T07:16:03.218 INFO:tasks.workunit.client.0.smithi063.stdout:5/625: symlink d8/d13/d24/d81/lb8 0 2024-03-22T07:16:03.219 INFO:tasks.workunit.client.0.smithi063.stdout:2/687: dread d0/de/d74/f7e [0,4194304] 0 2024-03-22T07:16:03.219 INFO:tasks.workunit.client.0.smithi063.stdout:2/688: chown d0/de/d1b/d63/d6e/f95 1835 1 2024-03-22T07:16:03.219 INFO:tasks.workunit.client.0.smithi063.stdout:2/689: write d0/de/d58/f91 [338495,49852] 0 2024-03-22T07:16:03.219 INFO:tasks.workunit.client.0.smithi063.stdout:2/690: truncate d0/de/d58/f73 782931 0 2024-03-22T07:16:03.220 INFO:tasks.workunit.client.1.smithi069.stdout:4/541: creat d6/d9b/fb6 x:0 0 0 2024-03-22T07:16:03.220 INFO:tasks.workunit.client.1.smithi069.stdout:4/542: dread - d6/dc/d1d/d55/f5f zero size 2024-03-22T07:16:03.220 INFO:tasks.workunit.client.0.smithi063.stdout:5/626: creat d8/d13/d1e/d3c/da0/fb9 x:0 0 0 2024-03-22T07:16:03.221 INFO:tasks.workunit.client.1.smithi069.stdout:5/376: mkdir d1/d7/d8/d43/d8a 0 2024-03-22T07:16:03.222 INFO:tasks.workunit.client.0.smithi063.stdout:2/691: creat d0/de/d12/dd1/fdd x:0 0 0 2024-03-22T07:16:03.222 INFO:tasks.workunit.client.0.smithi063.stdout:2/692: truncate d0/de/d1b/d32/dac/fab 506620 0 2024-03-22T07:16:03.222 INFO:tasks.workunit.client.0.smithi063.stdout:8/802: dwrite d0/d9e/df0/ffe [0,4194304] 0 2024-03-22T07:16:03.223 INFO:tasks.workunit.client.1.smithi069.stdout:5/377: dread d1/d7/f40 [0,4194304] 0 2024-03-22T07:16:03.223 INFO:tasks.workunit.client.1.smithi069.stdout:5/378: chown d1/d10 480261461 1 2024-03-22T07:16:03.226 INFO:tasks.workunit.client.1.smithi069.stdout:3/498: creat d5/d9/d26/d36/d41/d8e/f9f x:0 0 0 2024-03-22T07:16:03.226 INFO:tasks.workunit.client.1.smithi069.stdout:3/499: readlink d5/d9/db/d3b/l86 0 2024-03-22T07:16:03.228 INFO:tasks.workunit.client.1.smithi069.stdout:6/416: write d4/dd/d12/d13/f59 [934445,45963] 0 2024-03-22T07:16:03.228 INFO:tasks.workunit.client.1.smithi069.stdout:6/417: read - d4/d7/f6e zero size 2024-03-22T07:16:03.228 INFO:tasks.workunit.client.0.smithi063.stdout:8/803: rename d0/c2 to d0/d1b/d28/de8/dc6/c113 0 2024-03-22T07:16:03.228 INFO:tasks.workunit.client.0.smithi063.stdout:8/804: write d0/d26/d2d/d59/dba/dc9/fff [520823,17583] 0 2024-03-22T07:16:03.229 INFO:tasks.workunit.client.1.smithi069.stdout:8/589: dread d1/de/d6c/d73/f74 [0,4194304] 0 2024-03-22T07:16:03.231 INFO:tasks.workunit.client.0.smithi063.stdout:2/693: symlink d0/de/d1b/d32/dac/lde 0 2024-03-22T07:16:03.234 INFO:tasks.workunit.client.0.smithi063.stdout:6/704: rmdir d3/d15/d16/d60/d6d/d4e/d5a 39 2024-03-22T07:16:03.234 INFO:tasks.workunit.client.1.smithi069.stdout:4/543: truncate d6/d12/f41 2478176 0 2024-03-22T07:16:03.234 INFO:tasks.workunit.client.1.smithi069.stdout:4/544: readlink d6/dc/l63 0 2024-03-22T07:16:03.235 INFO:tasks.workunit.client.1.smithi069.stdout:6/418: link d4/d7/da/l4c d4/d7/da/l84 0 2024-03-22T07:16:03.235 INFO:tasks.workunit.client.1.smithi069.stdout:6/419: write d4/dd/d12/d44/f7a [1043896,107310] 0 2024-03-22T07:16:03.235 INFO:tasks.workunit.client.1.smithi069.stdout:6/420: readlink d4/d7/l67 0 2024-03-22T07:16:03.237 INFO:tasks.workunit.client.1.smithi069.stdout:8/590: mknod d1/d1c/d3e/d66/cbe 0 2024-03-22T07:16:03.237 INFO:tasks.workunit.client.0.smithi063.stdout:2/694: symlink d0/de/d1b/ldf 0 2024-03-22T07:16:03.239 INFO:tasks.workunit.client.1.smithi069.stdout:8/591: getdents d1/d1c/d48/d5e 0 2024-03-22T07:16:03.240 INFO:tasks.workunit.client.0.smithi063.stdout:6/705: rename d3/d15/d16/d69/d8c/la6 to d3/d15/d16/d69/led 0 2024-03-22T07:16:03.241 INFO:tasks.workunit.client.0.smithi063.stdout:6/706: creat d3/d6/d2a/fee x:0 0 0 2024-03-22T07:16:03.243 INFO:tasks.workunit.client.1.smithi069.stdout:6/421: write d4/f47 [2153373,102721] 0 2024-03-22T07:16:03.244 INFO:tasks.workunit.client.0.smithi063.stdout:6/707: symlink d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/lef 0 2024-03-22T07:16:03.245 INFO:tasks.workunit.client.0.smithi063.stdout:7/604: getdents d5/d38 0 2024-03-22T07:16:03.246 INFO:tasks.workunit.client.1.smithi069.stdout:6/422: link d4/dd/f4a d4/dd/d41/d78/f85 0 2024-03-22T07:16:03.247 INFO:tasks.workunit.client.1.smithi069.stdout:6/423: dread - d4/f6c zero size 2024-03-22T07:16:03.247 INFO:tasks.workunit.client.1.smithi069.stdout:6/424: write d4/dd/f54 [392971,12234] 0 2024-03-22T07:16:03.248 INFO:tasks.workunit.client.1.smithi069.stdout:8/592: write d1/de/d6c/d73/f74 [1019784,121209] 0 2024-03-22T07:16:03.248 INFO:tasks.workunit.client.1.smithi069.stdout:8/593: fsync d1/de/d6c/d4e/d5b/d82/fac 0 2024-03-22T07:16:03.248 INFO:tasks.workunit.client.1.smithi069.stdout:8/594: stat d1/d1c/d3e/d88/da8/dbb 0 2024-03-22T07:16:03.248 INFO:tasks.workunit.client.1.smithi069.stdout:8/595: write d1/de/fa4 [492454,32151] 0 2024-03-22T07:16:03.250 INFO:tasks.workunit.client.1.smithi069.stdout:4/545: dread d6/d9b/f14 [0,4194304] 0 2024-03-22T07:16:03.250 INFO:tasks.workunit.client.1.smithi069.stdout:4/546: stat d6/d16/d83/c89 0 2024-03-22T07:16:03.252 INFO:tasks.workunit.client.1.smithi069.stdout:6/425: getdents d4/dd/d12/d13 0 2024-03-22T07:16:03.254 INFO:tasks.workunit.client.1.smithi069.stdout:5/379: dwrite d1/d28/f4d [0,4194304] 0 2024-03-22T07:16:03.255 INFO:tasks.workunit.client.0.smithi063.stdout:5/627: dwrite d8/d13/d28/f43 [4194304,4194304] 0 2024-03-22T07:16:03.256 INFO:tasks.workunit.client.1.smithi069.stdout:8/596: symlink d1/d1c/lbf 0 2024-03-22T07:16:03.258 INFO:tasks.workunit.client.0.smithi063.stdout:6/708: read d3/d15/d16/d60/d6d/d4e/d5a/d88/fc1 [2675412,126240] 0 2024-03-22T07:16:03.258 INFO:tasks.workunit.client.0.smithi063.stdout:6/709: chown d3/d6/d35/d3a/fbe 81 1 2024-03-22T07:16:03.261 INFO:tasks.workunit.client.1.smithi069.stdout:5/380: truncate d1/f3 95675 0 2024-03-22T07:16:03.266 INFO:tasks.workunit.client.0.smithi063.stdout:6/710: unlink d3/d15/d16/d60/d6d/c2b 0 2024-03-22T07:16:03.268 INFO:tasks.workunit.client.0.smithi063.stdout:6/711: symlink d3/d6/d2a/lf0 0 2024-03-22T07:16:03.276 INFO:tasks.workunit.client.0.smithi063.stdout:7/605: dwrite d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/fb4 [0,4194304] 0 2024-03-22T07:16:03.276 INFO:tasks.workunit.client.0.smithi063.stdout:7/606: write d5/da/d28/da7/f72 [376939,98498] 0 2024-03-22T07:16:03.276 INFO:tasks.workunit.client.0.smithi063.stdout:7/607: chown d5/da/d28/d4b/da8/dbc/d85/fbf 9823 1 2024-03-22T07:16:03.276 INFO:tasks.workunit.client.0.smithi063.stdout:7/608: read d5/da/d5f/f5c [565603,104102] 0 2024-03-22T07:16:03.276 INFO:tasks.workunit.client.0.smithi063.stdout:7/609: fdatasync d5/da/f35 0 2024-03-22T07:16:03.276 INFO:tasks.workunit.client.1.smithi069.stdout:8/597: symlink d1/d1c/lc0 0 2024-03-22T07:16:03.277 INFO:tasks.workunit.client.0.smithi063.stdout:7/610: read d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d4f/f95 [289134,11727] 0 2024-03-22T07:16:03.278 INFO:tasks.workunit.client.1.smithi069.stdout:8/598: unlink d1/d1c/d3e/d66/cbe 0 2024-03-22T07:16:03.279 INFO:tasks.workunit.client.1.smithi069.stdout:8/599: mknod d1/de/d6c/d4e/d5b/d82/cc1 0 2024-03-22T07:16:03.279 INFO:tasks.workunit.client.0.smithi063.stdout:6/712: dread d3/d6/d2a/d38/f4d [0,4194304] 0 2024-03-22T07:16:03.281 INFO:tasks.workunit.client.1.smithi069.stdout:8/600: link d1/de/c80 d1/cc2 0 2024-03-22T07:16:03.281 INFO:tasks.workunit.client.1.smithi069.stdout:8/601: stat d1/d1c/d48/f7a 0 2024-03-22T07:16:03.281 INFO:tasks.workunit.client.1.smithi069.stdout:8/602: write d1/d1c/d48/d5e/f8b [398407,95386] 0 2024-03-22T07:16:03.281 INFO:tasks.workunit.client.1.smithi069.stdout:8/603: truncate d1/d1c/d3e/d66/f78 182430 0 2024-03-22T07:16:03.282 INFO:tasks.workunit.client.1.smithi069.stdout:0/570: getdents d3/d7/d90 0 2024-03-22T07:16:03.282 INFO:tasks.workunit.client.1.smithi069.stdout:0/571: creat d3/d7/d15/d29/d5a/d5d/d9d/fbd x:0 0 0 2024-03-22T07:16:03.283 INFO:tasks.workunit.client.1.smithi069.stdout:8/604: dread d1/d1c/d3e/d66/f78 [0,4194304] 0 2024-03-22T07:16:03.283 INFO:tasks.workunit.client.1.smithi069.stdout:8/605: chown d1/fd 15445 1 2024-03-22T07:16:03.283 INFO:tasks.workunit.client.1.smithi069.stdout:8/606: readlink d1/d1c/l44 0 2024-03-22T07:16:03.283 INFO:tasks.workunit.client.1.smithi069.stdout:8/607: write d1/de/d6c/f72 [972640,46250] 0 2024-03-22T07:16:03.286 INFO:tasks.workunit.client.1.smithi069.stdout:8/608: creat d1/d1c/d48/fc3 x:0 0 0 2024-03-22T07:16:03.287 INFO:tasks.workunit.client.0.smithi063.stdout:2/695: dwrite d0/de/d12/d20/fd3 [0,4194304] 0 2024-03-22T07:16:03.287 INFO:tasks.workunit.client.1.smithi069.stdout:0/572: rmdir d3/d9a 39 2024-03-22T07:16:03.288 INFO:tasks.workunit.client.0.smithi063.stdout:2/696: creat d0/de/d1b/d63/fe0 x:0 0 0 2024-03-22T07:16:03.289 INFO:tasks.workunit.client.1.smithi069.stdout:8/609: creat d1/d1c/d48/d5e/fc4 x:0 0 0 2024-03-22T07:16:03.291 INFO:tasks.workunit.client.1.smithi069.stdout:0/573: creat d3/d7/d5e/fbe x:0 0 0 2024-03-22T07:16:03.291 INFO:tasks.workunit.client.1.smithi069.stdout:0/574: creat d3/d7/d15/d29/d5a/d99/fbf x:0 0 0 2024-03-22T07:16:03.292 INFO:tasks.workunit.client.1.smithi069.stdout:8/610: creat d1/de/d6c/d4e/d98/fc5 x:0 0 0 2024-03-22T07:16:03.295 INFO:tasks.workunit.client.1.smithi069.stdout:0/575: rename d3/d7/d15/d29/d5a/d5d/f7d to d3/d7/d24/d6a/fc0 0 2024-03-22T07:16:03.298 INFO:tasks.workunit.client.1.smithi069.stdout:8/611: mknod d1/d1c/d3e/d66/d9e/cc6 0 2024-03-22T07:16:03.298 INFO:tasks.workunit.client.1.smithi069.stdout:0/576: dread d3/d7/d15/d29/f63 [0,4194304] 0 2024-03-22T07:16:03.298 INFO:tasks.workunit.client.1.smithi069.stdout:8/612: creat d1/de/d6c/d4e/fc7 x:0 0 0 2024-03-22T07:16:03.299 INFO:tasks.workunit.client.0.smithi063.stdout:6/713: dread d3/f9 [4194304,4194304] 0 2024-03-22T07:16:03.299 INFO:tasks.workunit.client.0.smithi063.stdout:6/714: readlink d3/d15/d16/d60/d6d/d4e/d5a/d88/lea 0 2024-03-22T07:16:03.299 INFO:tasks.workunit.client.0.smithi063.stdout:6/715: creat d3/d6/d35/d3a/ff1 x:0 0 0 2024-03-22T07:16:03.301 INFO:tasks.workunit.client.1.smithi069.stdout:8/613: rename d1/de/d6c/d8e/fa9 to d1/de/d6c/d4e/fc8 0 2024-03-22T07:16:03.301 INFO:tasks.workunit.client.0.smithi063.stdout:6/716: rename d3/d6/d2a/d38/da5/d32/d51/c99 to d3/d15/d16/d60/d6d/d4e/cf2 0 2024-03-22T07:16:03.301 INFO:tasks.workunit.client.0.smithi063.stdout:6/717: dread - d3/d15/d16/d60/fcb zero size 2024-03-22T07:16:03.302 INFO:tasks.workunit.client.1.smithi069.stdout:8/614: rename d1/de/d6c/f6d to d1/d1c/d48/d56/fc9 0 2024-03-22T07:16:03.302 INFO:tasks.workunit.client.1.smithi069.stdout:8/615: creat d1/de/fca x:0 0 0 2024-03-22T07:16:03.303 INFO:tasks.workunit.client.1.smithi069.stdout:8/616: creat d1/fcb x:0 0 0 2024-03-22T07:16:03.303 INFO:tasks.workunit.client.0.smithi063.stdout:6/718: rename d3/f4 to d3/d6/d2a/d38/da5/ff3 0 2024-03-22T07:16:03.305 INFO:tasks.workunit.client.0.smithi063.stdout:6/719: rename d3/d6/d2a/d38/da5/d32/d51/ccf to d3/cf4 0 2024-03-22T07:16:03.305 INFO:tasks.workunit.client.1.smithi069.stdout:8/617: symlink d1/d1c/d3e/d66/d9e/lcc 0 2024-03-22T07:16:03.307 INFO:tasks.workunit.client.1.smithi069.stdout:8/618: write d1/de/d20/f65 [521524,116986] 0 2024-03-22T07:16:03.307 INFO:tasks.workunit.client.1.smithi069.stdout:8/619: write d1/de/fca [882483,93997] 0 2024-03-22T07:16:03.307 INFO:tasks.workunit.client.1.smithi069.stdout:8/620: write d1/d1c/d3e/d66/f95 [1014353,56742] 0 2024-03-22T07:16:03.307 INFO:tasks.workunit.client.1.smithi069.stdout:4/547: dwrite d6/dc/d1d/f96 [0,4194304] 0 2024-03-22T07:16:03.308 INFO:tasks.workunit.client.0.smithi063.stdout:8/805: dwrite d0/d26/d2d/d59/dba/dc9/fff [0,4194304] 0 2024-03-22T07:16:03.309 INFO:tasks.workunit.client.0.smithi063.stdout:8/806: creat d0/d1b/d28/d3e/db2/dbd/dfd/f114 x:0 0 0 2024-03-22T07:16:03.309 INFO:tasks.workunit.client.1.smithi069.stdout:8/621: unlink d1/de/d20/l28 0 2024-03-22T07:16:03.311 INFO:tasks.workunit.client.1.smithi069.stdout:6/426: dwrite d4/d7/da/f5a [0,4194304] 0 2024-03-22T07:16:03.311 INFO:tasks.workunit.client.1.smithi069.stdout:6/427: getdents d4/d7/da/d3f/d76 0 2024-03-22T07:16:03.311 INFO:tasks.workunit.client.1.smithi069.stdout:6/428: dread - d4/f6c zero size 2024-03-22T07:16:03.311 INFO:tasks.workunit.client.1.smithi069.stdout:5/381: dwrite d1/d28/f4d [0,4194304] 0 2024-03-22T07:16:03.312 INFO:tasks.workunit.client.0.smithi063.stdout:7/611: dwrite d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/f5d [0,4194304] 0 2024-03-22T07:16:03.312 INFO:tasks.workunit.client.0.smithi063.stdout:5/628: dwrite d8/d13/d1e/d31/daf/fb1 [0,4194304] 0 2024-03-22T07:16:03.313 INFO:tasks.workunit.client.1.smithi069.stdout:8/622: truncate d1/de/d6c/d4e/d5b/f76 1311614 0 2024-03-22T07:16:03.313 INFO:tasks.workunit.client.1.smithi069.stdout:8/623: write d1/d1c/d3e/d88/f8f [2155267,33548] 0 2024-03-22T07:16:03.316 INFO:tasks.workunit.client.1.smithi069.stdout:6/429: creat d4/dd/d12/d13/f86 x:0 0 0 2024-03-22T07:16:03.319 INFO:tasks.workunit.client.1.smithi069.stdout:6/430: symlink d4/d7/da/d3f/l87 0 2024-03-22T07:16:03.319 INFO:tasks.workunit.client.1.smithi069.stdout:6/431: stat d4/dd/d12/d44/f6f 0 2024-03-22T07:16:03.320 INFO:tasks.workunit.client.1.smithi069.stdout:6/432: chown d4/dd/d12/d2b/d72/d73/d74 38 1 2024-03-22T07:16:03.320 INFO:tasks.workunit.client.0.smithi063.stdout:8/807: write d0/d1b/d4f/dde/df5/f98 [4158637,117442] 0 2024-03-22T07:16:03.320 INFO:tasks.workunit.client.0.smithi063.stdout:6/720: dread d3/d6/fa [0,4194304] 0 2024-03-22T07:16:03.321 INFO:tasks.workunit.client.1.smithi069.stdout:5/382: getdents d1/d10/d60 0 2024-03-22T07:16:03.321 INFO:tasks.workunit.client.1.smithi069.stdout:5/383: fdatasync d1/d6/d26/d3a/f6e 0 2024-03-22T07:16:03.321 INFO:tasks.workunit.client.1.smithi069.stdout:5/384: dread - d1/d10/d60/f83 zero size 2024-03-22T07:16:03.325 INFO:tasks.workunit.client.0.smithi063.stdout:7/612: link d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/f6a d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/fca 0 2024-03-22T07:16:03.330 INFO:tasks.workunit.client.1.smithi069.stdout:4/548: dwrite d6/dc/d1d/d55/f9c [0,4194304] 0 2024-03-22T07:16:03.330 INFO:tasks.workunit.client.1.smithi069.stdout:4/549: write d6/d16/f71 [839723,113405] 0 2024-03-22T07:16:03.330 INFO:tasks.workunit.client.0.smithi063.stdout:6/721: mkdir d3/d15/d16/d60/d6d/d4e/d5a/d88/df5 0 2024-03-22T07:16:03.330 INFO:tasks.workunit.client.0.smithi063.stdout:6/722: truncate d3/d15/d16/d57/fc8 174068 0 2024-03-22T07:16:03.331 INFO:tasks.workunit.client.1.smithi069.stdout:6/433: symlink d4/dd/d12/d13/l88 0 2024-03-22T07:16:03.332 INFO:tasks.workunit.client.0.smithi063.stdout:8/808: dread d0/d1b/fc3 [0,4194304] 0 2024-03-22T07:16:03.332 INFO:tasks.workunit.client.0.smithi063.stdout:5/629: mkdir d8/dba 0 2024-03-22T07:16:03.334 INFO:tasks.workunit.client.1.smithi069.stdout:0/577: truncate d3/d7/d17/d1c/f91 189291 0 2024-03-22T07:16:03.334 INFO:tasks.workunit.client.1.smithi069.stdout:0/578: write d3/d7/d15/d29/d5a/d84/f60 [57359,29299] 0 2024-03-22T07:16:03.334 INFO:tasks.workunit.client.1.smithi069.stdout:0/579: chown d3/d7/d15/d1f/c66 172334 1 2024-03-22T07:16:03.334 INFO:tasks.workunit.client.1.smithi069.stdout:0/580: read d3/d7/f12 [1515589,61659] 0 2024-03-22T07:16:03.334 INFO:tasks.workunit.client.0.smithi063.stdout:6/723: link d3/d15/f22 d3/d15/d16/ff6 0 2024-03-22T07:16:03.335 INFO:tasks.workunit.client.0.smithi063.stdout:6/724: write d3/d6/d2a/f5e [4916052,97897] 0 2024-03-22T07:16:03.335 INFO:tasks.workunit.client.0.smithi063.stdout:6/725: dread d3/d15/d16/d60/d6d/d4e/fa7 [0,4194304] 0 2024-03-22T07:16:03.335 INFO:tasks.workunit.client.0.smithi063.stdout:8/809: creat d0/d1b/d28/d3e/db2/f115 x:0 0 0 2024-03-22T07:16:03.337 INFO:tasks.workunit.client.1.smithi069.stdout:4/550: mknod d6/d16/cb7 0 2024-03-22T07:16:03.337 INFO:tasks.workunit.client.1.smithi069.stdout:4/551: fsync d6/d9b/d1f/d3f/f9d 0 2024-03-22T07:16:03.338 INFO:tasks.workunit.client.0.smithi063.stdout:5/630: truncate d8/d13/d1e/f2f 3701465 0 2024-03-22T07:16:03.339 INFO:tasks.workunit.client.1.smithi069.stdout:6/434: truncate d4/dd/d12/d2b/f36 5765701 0 2024-03-22T07:16:03.339 INFO:tasks.workunit.client.1.smithi069.stdout:6/435: chown d4/d7/da/d3f/f6d 1727602 1 2024-03-22T07:16:03.340 INFO:tasks.workunit.client.0.smithi063.stdout:8/810: dread d0/d1b/d28/d5f/fd3 [0,4194304] 0 2024-03-22T07:16:03.341 INFO:tasks.workunit.client.1.smithi069.stdout:4/552: mkdir d6/d9b/d1f/d3f/d51/db8 0 2024-03-22T07:16:03.341 INFO:tasks.workunit.client.0.smithi063.stdout:5/631: creat d8/d13/d1e/d31/d64/fbb x:0 0 0 2024-03-22T07:16:03.343 INFO:tasks.workunit.client.1.smithi069.stdout:5/385: dwrite d1/d7/d8/f24 [0,4194304] 0 2024-03-22T07:16:03.343 INFO:tasks.workunit.client.1.smithi069.stdout:5/386: stat d1/d6/f6f 0 2024-03-22T07:16:03.344 INFO:tasks.workunit.client.1.smithi069.stdout:5/387: readlink d1/d28/l82 0 2024-03-22T07:16:03.344 INFO:tasks.workunit.client.1.smithi069.stdout:5/388: mknod d1/d6/d21/c8b 0 2024-03-22T07:16:03.345 INFO:tasks.workunit.client.0.smithi063.stdout:5/632: rename d8/d13/d1e/d3c/faa to d8/d13/d28/fbc 0 2024-03-22T07:16:03.345 INFO:tasks.workunit.client.1.smithi069.stdout:5/389: readlink d1/d6/l20 0 2024-03-22T07:16:03.348 INFO:tasks.workunit.client.0.smithi063.stdout:6/726: dread d3/d6/d2a/d38/da5/f4b [0,4194304] 0 2024-03-22T07:16:03.348 INFO:tasks.workunit.client.0.smithi063.stdout:6/727: readlink d3/d6/d2a/d38/da5/d32/l36 0 2024-03-22T07:16:03.349 INFO:tasks.workunit.client.0.smithi063.stdout:5/633: rename d8/d13/f5d to d8/d13/d28/fbd 0 2024-03-22T07:16:03.350 INFO:tasks.workunit.client.0.smithi063.stdout:5/634: rmdir d8/d13 39 2024-03-22T07:16:03.352 INFO:tasks.workunit.client.0.smithi063.stdout:7/613: dwrite d5/f89 [0,4194304] 0 2024-03-22T07:16:03.352 INFO:tasks.workunit.client.0.smithi063.stdout:7/614: write d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d4f/f95 [632873,63488] 0 2024-03-22T07:16:03.354 INFO:tasks.workunit.client.0.smithi063.stdout:6/728: dread d3/d15/f17 [0,4194304] 0 2024-03-22T07:16:03.354 INFO:tasks.workunit.client.0.smithi063.stdout:6/729: fdatasync d3/d15/d16/f25 0 2024-03-22T07:16:03.357 INFO:tasks.workunit.client.1.smithi069.stdout:0/581: dwrite d3/d9a/f9f [4194304,4194304] 0 2024-03-22T07:16:03.357 INFO:tasks.workunit.client.1.smithi069.stdout:0/582: creat d3/d7/d17/fc1 x:0 0 0 2024-03-22T07:16:03.359 INFO:tasks.workunit.client.0.smithi063.stdout:7/615: symlink d5/d16/d43/d78/lcb 0 2024-03-22T07:16:03.360 INFO:tasks.workunit.client.1.smithi069.stdout:5/390: write d1/d7/d8/f58 [1852350,47212] 0 2024-03-22T07:16:03.362 INFO:tasks.workunit.client.0.smithi063.stdout:6/730: rename d3/d15/d16/d60/d6d/d4e/dec to d3/d15/d16/d60/d6d/d4e/d5a/d88/df7 0 2024-03-22T07:16:03.367 INFO:tasks.workunit.client.0.smithi063.stdout:6/731: creat d3/d15/ff8 x:0 0 0 2024-03-22T07:16:03.367 INFO:tasks.workunit.client.0.smithi063.stdout:6/732: read d3/d6/dcd/fce [529148,107208] 0 2024-03-22T07:16:03.367 INFO:tasks.workunit.client.1.smithi069.stdout:0/583: unlink d3/d7/d5e/l81 0 2024-03-22T07:16:03.367 INFO:tasks.workunit.client.1.smithi069.stdout:0/584: readlink d3/d7/d15/l72 0 2024-03-22T07:16:03.367 INFO:tasks.workunit.client.1.smithi069.stdout:0/585: chown d3/d7/d15/d1f/f41 231 1 2024-03-22T07:16:03.368 INFO:tasks.workunit.client.0.smithi063.stdout:5/635: dwrite d8/d13/d24/f33 [0,4194304] 0 2024-03-22T07:16:03.369 INFO:tasks.workunit.client.1.smithi069.stdout:5/391: fsync d1/d6/f2e 0 2024-03-22T07:16:03.371 INFO:tasks.workunit.client.0.smithi063.stdout:5/636: rename d8/d13/d1e/d31/d58/d7e/d82/f96 to d8/d13/d1e/d31/d58/fbe 0 2024-03-22T07:16:03.371 INFO:tasks.workunit.client.0.smithi063.stdout:5/637: chown d8/d13/d1e/f7b 750 1 2024-03-22T07:16:03.374 INFO:tasks.workunit.client.1.smithi069.stdout:1/421: sync 2024-03-22T07:16:03.377 INFO:tasks.workunit.client.1.smithi069.stdout:8/624: rename d1/de to d1/d1c/d3e/d88/dcd 0 2024-03-22T07:16:03.377 INFO:tasks.workunit.client.1.smithi069.stdout:8/625: chown d1 29619058 1 2024-03-22T07:16:03.381 INFO:tasks.workunit.client.1.smithi069.stdout:0/586: dread d3/d7/d15/d29/d5a/d84/f11 [0,4194304] 0 2024-03-22T07:16:03.382 INFO:tasks.workunit.client.0.smithi063.stdout:6/733: dread d3/d6/d2a/d38/da5/f4b [4194304,4194304] 0 2024-03-22T07:16:03.382 INFO:tasks.workunit.client.1.smithi069.stdout:1/422: unlink d1/d2a/d50/d4c/d55/d5f/l6d 0 2024-03-22T07:16:03.382 INFO:tasks.workunit.client.1.smithi069.stdout:1/423: write d1/d2a/f4e [347585,27702] 0 2024-03-22T07:16:03.382 INFO:tasks.workunit.client.1.smithi069.stdout:1/424: chown d1/d13/l1f 313927 1 2024-03-22T07:16:03.382 INFO:tasks.workunit.client.0.smithi063.stdout:6/734: creat d3/d15/da3/dc2/ff9 x:0 0 0 2024-03-22T07:16:03.383 INFO:tasks.workunit.client.0.smithi063.stdout:6/735: creat d3/d15/d16/d60/d6d/d4e/d5a/dd4/ffa x:0 0 0 2024-03-22T07:16:03.384 INFO:tasks.workunit.client.0.smithi063.stdout:6/736: mknod d3/d6/d2a/d38/da5/d32/d51/cfb 0 2024-03-22T07:16:03.385 INFO:tasks.workunit.client.1.smithi069.stdout:4/553: rename d6/d9b/c6b to d6/d9b/d56/cb9 0 2024-03-22T07:16:03.385 INFO:tasks.workunit.client.0.smithi063.stdout:6/737: truncate d3/d15/d16/d60/d6d/d4c/f6f 634709 0 2024-03-22T07:16:03.387 INFO:tasks.workunit.client.1.smithi069.stdout:0/587: mknod d3/d7/d15/d29/d5a/d5d/d98/cc2 0 2024-03-22T07:16:03.391 INFO:tasks.workunit.client.1.smithi069.stdout:1/425: creat d1/d13/d82/f83 x:0 0 0 2024-03-22T07:16:03.393 INFO:tasks.workunit.client.1.smithi069.stdout:4/554: mknod d6/dc/d1d/cba 0 2024-03-22T07:16:03.396 INFO:tasks.workunit.client.0.smithi063.stdout:7/616: dwrite d5/da/d28/da7/f64 [0,4194304] 0 2024-03-22T07:16:03.397 INFO:tasks.workunit.client.1.smithi069.stdout:1/426: unlink d1/d13/d36/d6e/l78 0 2024-03-22T07:16:03.397 INFO:tasks.workunit.client.1.smithi069.stdout:5/392: dwrite d1/d10/d60/f5f [0,4194304] 0 2024-03-22T07:16:03.398 INFO:tasks.workunit.client.1.smithi069.stdout:4/555: read d6/dc/d23/f3c [2072205,71805] 0 2024-03-22T07:16:03.399 INFO:tasks.workunit.client.1.smithi069.stdout:8/626: dread d1/d1c/d48/d5e/f79 [0,4194304] 0 2024-03-22T07:16:03.400 INFO:tasks.workunit.client.0.smithi063.stdout:5/638: dwrite d8/d13/d1e/d3c/f4c [0,4194304] 0 2024-03-22T07:16:03.400 INFO:tasks.workunit.client.0.smithi063.stdout:5/639: truncate d8/d13/d1e/d31/d58/fad 631997 0 2024-03-22T07:16:03.403 INFO:tasks.workunit.client.0.smithi063.stdout:5/640: creat d8/d13/d24/d62/fbf x:0 0 0 2024-03-22T07:16:03.404 INFO:tasks.workunit.client.0.smithi063.stdout:5/641: truncate d8/f44 4102289 0 2024-03-22T07:16:03.406 INFO:tasks.workunit.client.0.smithi063.stdout:5/642: symlink d8/d13/d28/d57/lc0 0 2024-03-22T07:16:03.407 INFO:tasks.workunit.client.0.smithi063.stdout:5/643: symlink d8/d13/d1e/d31/d64/d65/d86/lc1 0 2024-03-22T07:16:03.408 INFO:tasks.workunit.client.0.smithi063.stdout:5/644: fsync d8/d13/f48 0 2024-03-22T07:16:03.408 INFO:tasks.workunit.client.1.smithi069.stdout:4/556: creat d6/fbb x:0 0 0 2024-03-22T07:16:03.409 INFO:tasks.workunit.client.1.smithi069.stdout:1/427: dread d1/d2a/f32 [0,4194304] 0 2024-03-22T07:16:03.409 INFO:tasks.workunit.client.0.smithi063.stdout:5/645: getdents d8/d13/d28/d57/d5f 0 2024-03-22T07:16:03.410 INFO:tasks.workunit.client.0.smithi063.stdout:5/646: creat d8/d13/d1e/d31/d64/fc2 x:0 0 0 2024-03-22T07:16:03.410 INFO:tasks.workunit.client.1.smithi069.stdout:5/393: getdents d1/d7/d8 0 2024-03-22T07:16:03.414 INFO:tasks.workunit.client.1.smithi069.stdout:8/627: link d1/l10 d1/d1c/d48/d56/lce 0 2024-03-22T07:16:03.416 INFO:tasks.workunit.client.1.smithi069.stdout:4/557: truncate d6/dc/d23/f3c 999035 0 2024-03-22T07:16:03.416 INFO:tasks.workunit.client.1.smithi069.stdout:4/558: chown d6/d12/f80 780614 1 2024-03-22T07:16:03.416 INFO:tasks.workunit.client.1.smithi069.stdout:4/559: truncate d6/d12/d18/d21/f49 4624677 0 2024-03-22T07:16:03.416 INFO:tasks.workunit.client.1.smithi069.stdout:4/560: chown d6/d4b/f92 64354 1 2024-03-22T07:16:03.417 INFO:tasks.workunit.client.0.smithi063.stdout:6/738: dwrite d3/d6/d2a/d38/da5/d32/fc3 [0,4194304] 0 2024-03-22T07:16:03.417 INFO:tasks.workunit.client.0.smithi063.stdout:3/808: sync 2024-03-22T07:16:03.417 INFO:tasks.workunit.client.0.smithi063.stdout:1/879: sync 2024-03-22T07:16:03.417 INFO:tasks.workunit.client.0.smithi063.stdout:0/938: sync 2024-03-22T07:16:03.417 INFO:tasks.workunit.client.0.smithi063.stdout:3/809: write da/d16/d3f/ff7 [191879,3354] 0 2024-03-22T07:16:03.417 INFO:tasks.workunit.client.0.smithi063.stdout:3/810: fdatasync da/d16/d26/f27 0 2024-03-22T07:16:03.417 INFO:tasks.workunit.client.0.smithi063.stdout:4/672: sync 2024-03-22T07:16:03.417 INFO:tasks.workunit.client.0.smithi063.stdout:9/728: sync 2024-03-22T07:16:03.418 INFO:tasks.workunit.client.1.smithi069.stdout:4/561: dread d6/d9b/d1f/d3f/f46 [0,4194304] 0 2024-03-22T07:16:03.424 INFO:tasks.workunit.client.0.smithi063.stdout:0/939: mknod da/de/d17/d25/d3c/d114/c131 0 2024-03-22T07:16:03.425 INFO:tasks.workunit.client.1.smithi069.stdout:0/588: dwrite d3/d7/d17/faa [0,4194304] 0 2024-03-22T07:16:03.425 INFO:tasks.workunit.client.1.smithi069.stdout:0/589: chown d3/d7/d15/d1f/lbb 6852463 1 2024-03-22T07:16:03.425 INFO:tasks.workunit.client.1.smithi069.stdout:0/590: fsync d3/d7/f1e 0 2024-03-22T07:16:03.426 INFO:tasks.workunit.client.0.smithi063.stdout:3/811: link da/d16/d26/d69/ca6 da/d16/d26/d69/c107 0 2024-03-22T07:16:03.426 INFO:tasks.workunit.client.0.smithi063.stdout:3/812: readlink da/d16/d89/d8c/dc1/l102 0 2024-03-22T07:16:03.428 INFO:tasks.workunit.client.0.smithi063.stdout:9/729: unlink d8/d19/d39/cbc 0 2024-03-22T07:16:03.430 INFO:tasks.workunit.client.0.smithi063.stdout:0/940: creat da/de/d17/d25/d3c/d64/f132 x:0 0 0 2024-03-22T07:16:03.431 INFO:tasks.workunit.client.0.smithi063.stdout:0/941: creat da/de/d60/d56/d8b/f133 x:0 0 0 2024-03-22T07:16:03.431 INFO:tasks.workunit.client.0.smithi063.stdout:9/730: mkdir d8/d19/de2 0 2024-03-22T07:16:03.431 INFO:tasks.workunit.client.1.smithi069.stdout:8/628: dread d1/d1c/d3e/d88/f8f [0,4194304] 0 2024-03-22T07:16:03.433 INFO:tasks.workunit.client.0.smithi063.stdout:7/617: dwrite d5/da/d5f/d33/fbd [0,4194304] 0 2024-03-22T07:16:03.433 INFO:tasks.workunit.client.0.smithi063.stdout:7/618: truncate d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/f36 4775026 0 2024-03-22T07:16:03.433 INFO:tasks.workunit.client.0.smithi063.stdout:7/619: write d5/d16/f90 [32164,78739] 0 2024-03-22T07:16:03.433 INFO:tasks.workunit.client.0.smithi063.stdout:7/620: read - d5/da/d28/d4b/da8/dbc/d85/f9f zero size 2024-03-22T07:16:03.433 INFO:tasks.workunit.client.0.smithi063.stdout:7/621: getdents d5/d38 0 2024-03-22T07:16:03.434 INFO:tasks.workunit.client.1.smithi069.stdout:4/562: write d6/d12/f41 [2449189,10929] 0 2024-03-22T07:16:03.434 INFO:tasks.workunit.client.1.smithi069.stdout:1/428: dwrite d1/d25/f4d [0,4194304] 0 2024-03-22T07:16:03.437 INFO:tasks.workunit.client.0.smithi063.stdout:7/622: truncate d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/f47 1497487 0 2024-03-22T07:16:03.437 INFO:tasks.workunit.client.0.smithi063.stdout:7/623: write d5/da/d28/d4b/da8/fc8 [275372,5312] 0 2024-03-22T07:16:03.438 INFO:tasks.workunit.client.0.smithi063.stdout:7/624: fdatasync d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/fb4 0 2024-03-22T07:16:03.438 INFO:tasks.workunit.client.0.smithi063.stdout:7/625: chown d5/da/f35 6351 1 2024-03-22T07:16:03.441 INFO:tasks.workunit.client.0.smithi063.stdout:7/626: mknod d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d4f/ccc 0 2024-03-22T07:16:03.443 INFO:tasks.workunit.client.0.smithi063.stdout:4/673: dwrite d1/d5/dc/d95/d59/fab [0,4194304] 0 2024-03-22T07:16:03.444 INFO:tasks.workunit.client.0.smithi063.stdout:9/731: dread d8/d17/d1e/f5c [0,4194304] 0 2024-03-22T07:16:03.444 INFO:tasks.workunit.client.0.smithi063.stdout:7/627: dread d5/da/d28/d4b/da8/dbc/f59 [0,4194304] 0 2024-03-22T07:16:03.444 INFO:tasks.workunit.client.1.smithi069.stdout:1/429: mkdir d1/d2a/d84 0 2024-03-22T07:16:03.445 INFO:tasks.workunit.client.0.smithi063.stdout:0/942: write da/de/d17/d25/d3c/d43/d80/fd8 [1386156,86224] 0 2024-03-22T07:16:03.445 INFO:tasks.workunit.client.1.smithi069.stdout:4/563: truncate d6/dc/d23/f5c 1100022 0 2024-03-22T07:16:03.447 INFO:tasks.workunit.client.1.smithi069.stdout:4/564: rename d6/d12/d18/d8b to d6/d16/d3a/db4/dbc 0 2024-03-22T07:16:03.447 INFO:tasks.workunit.client.0.smithi063.stdout:4/674: mknod d1/d5/d3c/d8f/dbb/ce5 0 2024-03-22T07:16:03.448 INFO:tasks.workunit.client.0.smithi063.stdout:7/628: dread d5/d16/d1b/f7c [0,4194304] 0 2024-03-22T07:16:03.450 INFO:tasks.workunit.client.0.smithi063.stdout:4/675: readlink d1/d5/d43/d27/d2c/d3d/l92 0 2024-03-22T07:16:03.450 INFO:tasks.workunit.client.0.smithi063.stdout:4/676: chown d1/d5/d3c/d65 3 1 2024-03-22T07:16:03.451 INFO:tasks.workunit.client.0.smithi063.stdout:9/732: rmdir d8/d19/d39/db5 39 2024-03-22T07:16:03.454 INFO:tasks.workunit.client.0.smithi063.stdout:4/677: creat d1/d5/dc/d95/d59/d9c/dac/fe6 x:0 0 0 2024-03-22T07:16:03.456 INFO:tasks.workunit.client.0.smithi063.stdout:9/733: creat d8/d19/d20/d22/de1/db7/fe3 x:0 0 0 2024-03-22T07:16:03.456 INFO:tasks.workunit.client.1.smithi069.stdout:0/591: getdents d3/d7/d15/dad 0 2024-03-22T07:16:03.457 INFO:tasks.workunit.client.0.smithi063.stdout:4/678: symlink d1/d5/dc/d95/d59/d9c/dac/le7 0 2024-03-22T07:16:03.458 INFO:tasks.workunit.client.0.smithi063.stdout:4/679: symlink d1/d5/d3c/d65/de4/le8 0 2024-03-22T07:16:03.458 INFO:tasks.workunit.client.0.smithi063.stdout:6/739: write d3/d6/d2a/d38/f49 [2200247,517] 0 2024-03-22T07:16:03.458 INFO:tasks.workunit.client.0.smithi063.stdout:6/740: stat d3/d15/d16/d60/f65 0 2024-03-22T07:16:03.459 INFO:tasks.workunit.client.0.smithi063.stdout:9/734: read d8/d17/d1e/f5c [4086,130681] 0 2024-03-22T07:16:03.459 INFO:tasks.workunit.client.1.smithi069.stdout:7/469: sync 2024-03-22T07:16:03.460 INFO:tasks.workunit.client.0.smithi063.stdout:4/680: link d1/d5/dc/f89 d1/d5/d3c/fe9 0 2024-03-22T07:16:03.462 INFO:tasks.workunit.client.0.smithi063.stdout:6/741: truncate d3/d15/f85 591838 0 2024-03-22T07:16:03.462 INFO:tasks.workunit.client.0.smithi063.stdout:6/742: chown d3/d15/d16/d60/d6d/d4e/d5a/le4 39808752 1 2024-03-22T07:16:03.464 INFO:tasks.workunit.client.0.smithi063.stdout:6/743: dread d3/d15/d16/d57/fc8 [0,4194304] 0 2024-03-22T07:16:03.464 INFO:tasks.workunit.client.0.smithi063.stdout:6/744: dread - d3/d15/ff8 zero size 2024-03-22T07:16:03.464 INFO:tasks.workunit.client.0.smithi063.stdout:6/745: truncate d3/d15/d16/d60/d6d/d4e/fa7 332250 0 2024-03-22T07:16:03.464 INFO:tasks.workunit.client.0.smithi063.stdout:6/746: fsync d3/d15/d16/d60/d6d/d4e/d5a/d88/f8d 0 2024-03-22T07:16:03.464 INFO:tasks.workunit.client.0.smithi063.stdout:6/747: stat d3/f19 0 2024-03-22T07:16:03.464 INFO:tasks.workunit.client.0.smithi063.stdout:4/681: mknod d1/d5/d43/d27/d2c/d3d/cea 0 2024-03-22T07:16:03.465 INFO:tasks.workunit.client.1.smithi069.stdout:8/629: dwrite d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/fae [0,4194304] 0 2024-03-22T07:16:03.466 INFO:tasks.workunit.client.0.smithi063.stdout:6/748: mknod d3/d6/d2a/d38/da5/d32/d51/cfc 0 2024-03-22T07:16:03.467 INFO:tasks.workunit.client.0.smithi063.stdout:4/682: getdents d1/d5/d3c/d8f/dc0/dcb 0 2024-03-22T07:16:03.468 INFO:tasks.workunit.client.0.smithi063.stdout:6/749: mkdir d3/d15/d16/d57/dde/deb/dfd 0 2024-03-22T07:16:03.469 INFO:tasks.workunit.client.1.smithi069.stdout:8/630: creat d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/fcf x:0 0 0 2024-03-22T07:16:03.469 INFO:tasks.workunit.client.0.smithi063.stdout:7/629: write d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/d4a/f7e [2199771,3122] 0 2024-03-22T07:16:03.470 INFO:tasks.workunit.client.1.smithi069.stdout:8/631: truncate d1/d1c/d48/d5e/f79 451582 0 2024-03-22T07:16:03.470 INFO:tasks.workunit.client.1.smithi069.stdout:8/632: chown d1/d1c/d3e/d88/dcd/d6c/d8e 713013 1 2024-03-22T07:16:03.470 INFO:tasks.workunit.client.1.smithi069.stdout:8/633: rmdir d1/d1c/d3e/d88 39 2024-03-22T07:16:03.471 INFO:tasks.workunit.client.0.smithi063.stdout:4/683: mkdir d1/d5/d3c/d8f/deb 0 2024-03-22T07:16:03.472 INFO:tasks.workunit.client.0.smithi063.stdout:6/750: rename d3/d6/d2a/d38/da5/d32/d51/cfb to d3/d15/d16/d60/d6d/d4e/d5a/d88/cfe 0 2024-03-22T07:16:03.473 INFO:tasks.workunit.client.1.smithi069.stdout:8/634: symlink d1/d1c/d48/ld0 0 2024-03-22T07:16:03.473 INFO:tasks.workunit.client.1.smithi069.stdout:8/635: fdatasync d1/d1c/d3e/f46 0 2024-03-22T07:16:03.473 INFO:tasks.workunit.client.1.smithi069.stdout:4/565: dwrite d6/d9b/d1f/d3f/d51/f91 [0,4194304] 0 2024-03-22T07:16:03.473 INFO:tasks.workunit.client.0.smithi063.stdout:3/813: dwrite da/d63/d65/d83/ff4 [0,4194304] 0 2024-03-22T07:16:03.473 INFO:tasks.workunit.client.0.smithi063.stdout:3/814: write da/f52 [4966792,52374] 0 2024-03-22T07:16:03.474 INFO:tasks.workunit.client.1.smithi069.stdout:8/636: rmdir d1/d1c/d3e/d88/dcd 39 2024-03-22T07:16:03.476 INFO:tasks.workunit.client.0.smithi063.stdout:7/630: link d5/d16/d43/d9d/ca9 d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/ccd 0 2024-03-22T07:16:03.476 INFO:tasks.workunit.client.0.smithi063.stdout:7/631: creat d5/da/d28/d4b/da8/dbc/d85/fce x:0 0 0 2024-03-22T07:16:03.476 INFO:tasks.workunit.client.0.smithi063.stdout:1/880: dwrite d4/df/d13/d2b/d6c/d85/fc7 [0,4194304] 0 2024-03-22T07:16:03.477 INFO:tasks.workunit.client.0.smithi063.stdout:6/751: mkdir d3/d15/d16/d60/d7b/dff 0 2024-03-22T07:16:03.478 INFO:tasks.workunit.client.1.smithi069.stdout:4/566: creat d6/d12/db2/fbd x:0 0 0 2024-03-22T07:16:03.478 INFO:tasks.workunit.client.1.smithi069.stdout:4/567: truncate d6/fbb 411038 0 2024-03-22T07:16:03.480 INFO:tasks.workunit.client.1.smithi069.stdout:8/637: mkdir d1/d1c/d3e/d66/d9e/dd1 0 2024-03-22T07:16:03.481 INFO:tasks.workunit.client.0.smithi063.stdout:3/815: truncate da/d16/d3f/f67 79937 0 2024-03-22T07:16:03.481 INFO:tasks.workunit.client.0.smithi063.stdout:3/816: fdatasync da/d16/d26/f29 0 2024-03-22T07:16:03.481 INFO:tasks.workunit.client.0.smithi063.stdout:3/817: fsync da/d63/d65/d81/fe8 0 2024-03-22T07:16:03.482 INFO:tasks.workunit.client.1.smithi069.stdout:8/638: symlink d1/d1c/d3e/d88/dcd/dbd/ld2 0 2024-03-22T07:16:03.486 INFO:tasks.workunit.client.1.smithi069.stdout:4/568: getdents d6/d12 0 2024-03-22T07:16:03.489 INFO:tasks.workunit.client.0.smithi063.stdout:7/632: mkdir d5/dcf 0 2024-03-22T07:16:03.489 INFO:tasks.workunit.client.1.smithi069.stdout:4/569: symlink d6/d9b/d1f/d58/lbe 0 2024-03-22T07:16:03.490 INFO:tasks.workunit.client.1.smithi069.stdout:4/570: truncate d6/d9b/d1f/d58/f69 2087887 0 2024-03-22T07:16:03.491 INFO:tasks.workunit.client.0.smithi063.stdout:1/881: rename d4/d8b/da7/faa to d4/d8b/da7/f12e 0 2024-03-22T07:16:03.491 INFO:tasks.workunit.client.1.smithi069.stdout:4/571: symlink d6/d12/d18/lbf 0 2024-03-22T07:16:03.492 INFO:tasks.workunit.client.1.smithi069.stdout:4/572: truncate d6/d16/d68/fa9 272397 0 2024-03-22T07:16:03.492 INFO:tasks.workunit.client.1.smithi069.stdout:4/573: creat d6/daa/fc0 x:0 0 0 2024-03-22T07:16:03.493 INFO:tasks.workunit.client.0.smithi063.stdout:6/752: mkdir d3/d6/d2a/d38/d100 0 2024-03-22T07:16:03.493 INFO:tasks.workunit.client.0.smithi063.stdout:6/753: fdatasync d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/f5b 0 2024-03-22T07:16:03.494 INFO:tasks.workunit.client.1.smithi069.stdout:4/574: creat d6/d16/d3a/db4/fc1 x:0 0 0 2024-03-22T07:16:03.497 INFO:tasks.workunit.client.1.smithi069.stdout:7/470: dwrite d9/d76/d57/d44/f9b [0,4194304] 0 2024-03-22T07:16:03.498 INFO:tasks.workunit.client.1.smithi069.stdout:7/471: write d9/d76/d57/d44/d53/f6d [327155,60775] 0 2024-03-22T07:16:03.498 INFO:tasks.workunit.client.1.smithi069.stdout:7/472: chown d9/d42/d8a/c92 28329 1 2024-03-22T07:16:03.498 INFO:tasks.workunit.client.1.smithi069.stdout:7/473: chown d9/d8e 340 1 2024-03-22T07:16:03.498 INFO:tasks.workunit.client.1.smithi069.stdout:7/474: write d9/d76/d57/d44/d53/f43 [2035012,37253] 0 2024-03-22T07:16:03.498 INFO:tasks.workunit.client.1.smithi069.stdout:7/475: stat d9/d76/d57/d75/d7a/f80 0 2024-03-22T07:16:03.498 INFO:tasks.workunit.client.1.smithi069.stdout:7/476: stat d9/d42/c8b 0 2024-03-22T07:16:03.498 INFO:tasks.workunit.client.1.smithi069.stdout:7/477: chown d9/d76/d57/d44/d53/d74/d30 21 1 2024-03-22T07:16:03.498 INFO:tasks.workunit.client.0.smithi063.stdout:3/818: mknod da/d63/c108 0 2024-03-22T07:16:03.498 INFO:tasks.workunit.client.0.smithi063.stdout:3/819: chown da/d16/d3f/de2 54547655 1 2024-03-22T07:16:03.499 INFO:tasks.workunit.client.0.smithi063.stdout:3/820: creat da/d63/d65/d81/da5/f109 x:0 0 0 2024-03-22T07:16:03.499 INFO:tasks.workunit.client.0.smithi063.stdout:7/633: symlink d5/da/d5f/d33/d57/ld0 0 2024-03-22T07:16:03.503 INFO:tasks.workunit.client.0.smithi063.stdout:9/735: dwrite d8/d19/d20/d22/d9b/f9e [0,4194304] 0 2024-03-22T07:16:03.503 INFO:tasks.workunit.client.1.smithi069.stdout:0/592: dwrite d3/d7/d15/d29/d5a/f75 [0,4194304] 0 2024-03-22T07:16:03.503 INFO:tasks.workunit.client.1.smithi069.stdout:7/478: rename d9/db/d1a/f56 to d9/d76/d57/d75/d7a/fa2 0 2024-03-22T07:16:03.506 INFO:tasks.workunit.client.0.smithi063.stdout:7/634: mkdir d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d4f/dd1 0 2024-03-22T07:16:03.507 INFO:tasks.workunit.client.1.smithi069.stdout:7/479: unlink d9/d76/d57/f59 0 2024-03-22T07:16:03.507 INFO:tasks.workunit.client.1.smithi069.stdout:7/480: truncate d9/db/d1a/f85 327692 0 2024-03-22T07:16:03.507 INFO:tasks.workunit.client.0.smithi063.stdout:4/684: dwrite d1/d5/dc/d95/d59/fa9 [0,4194304] 0 2024-03-22T07:16:03.507 INFO:tasks.workunit.client.0.smithi063.stdout:4/685: fsync d1/d5/dc/d95/f44 0 2024-03-22T07:16:03.511 INFO:tasks.workunit.client.1.smithi069.stdout:7/481: rmdir d9/d76/d57/d44/d53/d74/d30/d99 0 2024-03-22T07:16:03.511 INFO:tasks.workunit.client.1.smithi069.stdout:7/482: truncate d9/d76/d57/d44/d53/d2e/f7c 6252 0 2024-03-22T07:16:03.512 INFO:tasks.workunit.client.0.smithi063.stdout:9/736: symlink d8/d19/d20/d22/d5d/d76/d95/le4 0 2024-03-22T07:16:03.513 INFO:tasks.workunit.client.0.smithi063.stdout:7/635: symlink d5/da/d5f/d33/d57/ld2 0 2024-03-22T07:16:03.514 INFO:tasks.workunit.client.1.smithi069.stdout:7/483: symlink d9/da1/la3 0 2024-03-22T07:16:03.514 INFO:tasks.workunit.client.1.smithi069.stdout:0/593: dread d3/d7/d15/fa3 [0,4194304] 0 2024-03-22T07:16:03.514 INFO:tasks.workunit.client.1.smithi069.stdout:8/639: dwrite d1/d1c/d3e/d88/dcd/d6c/f72 [0,4194304] 0 2024-03-22T07:16:03.520 INFO:tasks.workunit.client.0.smithi063.stdout:4/686: creat d1/d5/d43/d27/d2c/d3d/fec x:0 0 0 2024-03-22T07:16:03.520 INFO:tasks.workunit.client.1.smithi069.stdout:8/640: truncate d1/d1c/d3e/f61 717849 0 2024-03-22T07:16:03.521 INFO:tasks.workunit.client.1.smithi069.stdout:0/594: rmdir d3/d7/d15/d29/d5a/d5d 39 2024-03-22T07:16:03.521 INFO:tasks.workunit.client.1.smithi069.stdout:0/595: fsync d3/d7/d15/d29/d6e/f7a 0 2024-03-22T07:16:03.522 INFO:tasks.workunit.client.1.smithi069.stdout:8/641: mkdir d1/d1c/d3e/d88/dd3 0 2024-03-22T07:16:03.523 INFO:tasks.workunit.client.0.smithi063.stdout:9/737: link d8/d15/f58 d8/d19/d20/d22/d29/d31/d53/d6a/fe5 0 2024-03-22T07:16:03.525 INFO:tasks.workunit.client.1.smithi069.stdout:0/596: creat d3/d9a/fc3 x:0 0 0 2024-03-22T07:16:03.526 INFO:tasks.workunit.client.1.smithi069.stdout:0/597: write d3/d7/f64 [519548,73363] 0 2024-03-22T07:16:03.526 INFO:tasks.workunit.client.0.smithi063.stdout:7/636: link d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d4f/c74 d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/cd3 0 2024-03-22T07:16:03.527 INFO:tasks.workunit.client.0.smithi063.stdout:7/637: stat d5/da/d5f 0 2024-03-22T07:16:03.527 INFO:tasks.workunit.client.1.smithi069.stdout:8/642: mknod d1/d1c/d3e/d88/dcd/d6c/cd4 0 2024-03-22T07:16:03.527 INFO:tasks.workunit.client.1.smithi069.stdout:0/598: symlink d3/lc4 0 2024-03-22T07:16:03.528 INFO:tasks.workunit.client.1.smithi069.stdout:8/643: mknod d1/d1c/d48/cd5 0 2024-03-22T07:16:03.528 INFO:tasks.workunit.client.1.smithi069.stdout:0/599: write d3/d7/d15/d29/f69 [756185,123191] 0 2024-03-22T07:16:03.528 INFO:tasks.workunit.client.1.smithi069.stdout:0/600: getdents d3/d7/d15/dad 0 2024-03-22T07:16:03.530 INFO:tasks.workunit.client.1.smithi069.stdout:0/601: mknod d3/d7/d17/d1c/cc5 0 2024-03-22T07:16:03.530 INFO:tasks.workunit.client.1.smithi069.stdout:0/602: truncate d3/d7/d15/d1f/f2b 1522584 0 2024-03-22T07:16:03.531 INFO:tasks.workunit.client.1.smithi069.stdout:0/603: symlink d3/d7/d15/d29/d5a/d99/lc6 0 2024-03-22T07:16:03.531 INFO:tasks.workunit.client.1.smithi069.stdout:0/604: chown d3/d7/d17/d1c/c68 496 1 2024-03-22T07:16:03.532 INFO:tasks.workunit.client.0.smithi063.stdout:4/687: dread d1/d5/d43/d27/d2c/d4a/f4d [0,4194304] 0 2024-03-22T07:16:03.532 INFO:tasks.workunit.client.0.smithi063.stdout:4/688: readlink d1/d5/d43/d27/d2c/d4a/d78/le1 0 2024-03-22T07:16:03.535 INFO:tasks.workunit.client.0.smithi063.stdout:4/689: symlink d1/d5/dc2/led 0 2024-03-22T07:16:03.538 INFO:tasks.workunit.client.0.smithi063.stdout:4/690: mkdir d1/d5/dc2/dee 0 2024-03-22T07:16:03.540 INFO:tasks.workunit.client.0.smithi063.stdout:9/738: write d8/d17/d1e/fd9 [1441215,63284] 0 2024-03-22T07:16:03.540 INFO:tasks.workunit.client.0.smithi063.stdout:1/882: dwrite d4/df/f10c [0,4194304] 0 2024-03-22T07:16:03.540 INFO:tasks.workunit.client.0.smithi063.stdout:7/638: write d5/da/d5f/f8a [884582,103680] 0 2024-03-22T07:16:03.540 INFO:tasks.workunit.client.0.smithi063.stdout:6/754: dwrite d3/d15/d16/d47/fb5 [0,4194304] 0 2024-03-22T07:16:03.540 INFO:tasks.workunit.client.0.smithi063.stdout:6/755: fdatasync d3/d15/d16/d57/fa9 0 2024-03-22T07:16:03.541 INFO:tasks.workunit.client.0.smithi063.stdout:4/691: unlink d1/d5/d43/d27/d2c/d4a/d78/ce3 0 2024-03-22T07:16:03.541 INFO:tasks.workunit.client.1.smithi069.stdout:4/575: dwrite d6/d12/d18/d21/f2f [4194304,4194304] 0 2024-03-22T07:16:03.541 INFO:tasks.workunit.client.1.smithi069.stdout:4/576: readlink d6/d9b/d1f/d3f/d51/l9f 0 2024-03-22T07:16:03.541 INFO:tasks.workunit.client.0.smithi063.stdout:3/821: dwrite da/d16/d26/f76 [0,4194304] 0 2024-03-22T07:16:03.542 INFO:tasks.workunit.client.0.smithi063.stdout:3/822: chown da/d63/d65/d4b/dfa/dfc 1 1 2024-03-22T07:16:03.542 INFO:tasks.workunit.client.1.smithi069.stdout:7/484: dwrite d9/d76/d57/d44/d53/f38 [0,4194304] 0 2024-03-22T07:16:03.542 INFO:tasks.workunit.client.1.smithi069.stdout:7/485: dread - d9/d76/d57/d75/d7a/d96/f9e zero size 2024-03-22T07:16:03.546 INFO:tasks.workunit.client.0.smithi063.stdout:9/739: rmdir d8/d19/d20/d22/d29/d31/d53 39 2024-03-22T07:16:03.550 INFO:tasks.workunit.client.0.smithi063.stdout:1/883: creat d4/d71/d88/da1/ddb/f12f x:0 0 0 2024-03-22T07:16:03.550 INFO:tasks.workunit.client.0.smithi063.stdout:7/639: write d5/da/d28/d4b/f9c [7525017,35311] 0 2024-03-22T07:16:03.554 INFO:tasks.workunit.client.0.smithi063.stdout:1/884: write d4/df/d13/d39/d63/f66 [80275,107169] 0 2024-03-22T07:16:03.555 INFO:tasks.workunit.client.0.smithi063.stdout:7/640: write d5/da/f25 [2068542,89755] 0 2024-03-22T07:16:03.556 INFO:tasks.workunit.client.0.smithi063.stdout:6/756: creat d3/d6/d35/f101 x:0 0 0 2024-03-22T07:16:03.557 INFO:tasks.workunit.client.1.smithi069.stdout:8/644: write d1/d1c/d3e/f61 [1282740,31498] 0 2024-03-22T07:16:03.558 INFO:tasks.workunit.client.1.smithi069.stdout:8/645: dread - d1/d1c/d3e/d88/dcd/d6c/d4e/fc7 zero size 2024-03-22T07:16:03.558 INFO:tasks.workunit.client.0.smithi063.stdout:4/692: stat d1/d5/d43/d27/d2c/d3d/c42 0 2024-03-22T07:16:03.558 INFO:tasks.workunit.client.1.smithi069.stdout:0/605: dwrite d3/d7/d17/f56 [0,4194304] 0 2024-03-22T07:16:03.559 INFO:tasks.workunit.client.0.smithi063.stdout:3/823: mkdir da/d63/d65/d4b/d10a 0 2024-03-22T07:16:03.563 INFO:tasks.workunit.client.1.smithi069.stdout:7/486: link d9/db/d48/d4b/f65 d9/d76/d57/d44/d53/d74/fa4 0 2024-03-22T07:16:03.569 INFO:tasks.workunit.client.0.smithi063.stdout:4/693: dread d1/f45 [0,4194304] 0 2024-03-22T07:16:03.570 INFO:tasks.workunit.client.0.smithi063.stdout:4/694: chown d1/d5/dc/dda/d79 20 1 2024-03-22T07:16:03.571 INFO:tasks.workunit.client.1.smithi069.stdout:4/577: dwrite d6/d9b/d1f/d3f/d51/f81 [0,4194304] 0 2024-03-22T07:16:03.573 INFO:tasks.workunit.client.1.smithi069.stdout:7/487: symlink d9/d76/d57/d75/d7a/d96/la5 0 2024-03-22T07:16:03.574 INFO:tasks.workunit.client.1.smithi069.stdout:7/488: fsync d9/d76/d57/d44/d53/d74/d30/d3d/f7b 0 2024-03-22T07:16:03.574 INFO:tasks.workunit.client.1.smithi069.stdout:7/489: fsync d9/d76/d57/d44/f47 0 2024-03-22T07:16:03.574 INFO:tasks.workunit.client.1.smithi069.stdout:0/606: write d3/d9a/f9f [4478838,104348] 0 2024-03-22T07:16:03.575 INFO:tasks.workunit.client.1.smithi069.stdout:8/646: dread d1/d1c/d3e/f36 [0,4194304] 0 2024-03-22T07:16:03.575 INFO:tasks.workunit.client.1.smithi069.stdout:8/647: dread - d1/d1c/d3e/d88/dcd/d6c/d4e/d98/fc5 zero size 2024-03-22T07:16:03.577 INFO:tasks.workunit.client.0.smithi063.stdout:3/824: symlink da/d16/d3f/de2/l10b 0 2024-03-22T07:16:03.577 INFO:tasks.workunit.client.1.smithi069.stdout:4/578: mknod d6/d9b/d1f/d3f/d51/db8/cc2 0 2024-03-22T07:16:03.579 INFO:tasks.workunit.client.0.smithi063.stdout:1/885: dwrite d4/df/d2c/f90 [0,4194304] 0 2024-03-22T07:16:03.579 INFO:tasks.workunit.client.0.smithi063.stdout:1/886: creat d4/d8b/f130 x:0 0 0 2024-03-22T07:16:03.580 INFO:tasks.workunit.client.0.smithi063.stdout:4/695: creat d1/d5/d3c/d8f/dc0/fef x:0 0 0 2024-03-22T07:16:03.580 INFO:tasks.workunit.client.0.smithi063.stdout:4/696: readlink d1/d5/d43/d27/d2c/d4a/l70 0 2024-03-22T07:16:03.581 INFO:tasks.workunit.client.1.smithi069.stdout:8/648: truncate d1/d1c/d3e/d66/f95 276017 0 2024-03-22T07:16:03.582 INFO:tasks.workunit.client.1.smithi069.stdout:7/490: rmdir d9/d42 39 2024-03-22T07:16:03.583 INFO:tasks.workunit.client.0.smithi063.stdout:6/757: dwrite d3/d6/d2a/d38/da5/d32/fb1 [0,4194304] 0 2024-03-22T07:16:03.583 INFO:tasks.workunit.client.1.smithi069.stdout:4/579: dread d6/f30 [0,4194304] 0 2024-03-22T07:16:03.586 INFO:tasks.workunit.client.0.smithi063.stdout:4/697: creat d1/d5/dc/d95/da4/ff0 x:0 0 0 2024-03-22T07:16:03.587 INFO:tasks.workunit.client.1.smithi069.stdout:0/607: dread d3/d7/fb [0,4194304] 0 2024-03-22T07:16:03.587 INFO:tasks.workunit.client.1.smithi069.stdout:0/608: creat d3/d7/d17/d1c/fc7 x:0 0 0 2024-03-22T07:16:03.589 INFO:tasks.workunit.client.0.smithi063.stdout:3/825: getdents da/d63/d65/d81/da5 0 2024-03-22T07:16:03.593 INFO:tasks.workunit.client.1.smithi069.stdout:0/609: rename d3/d7/d15/d29/d5a/d99/l9b to d3/d7/d15/d29/d5a/d5d/lc8 0 2024-03-22T07:16:03.594 INFO:tasks.workunit.client.0.smithi063.stdout:6/758: unlink d3/d6/d2a/f48 0 2024-03-22T07:16:03.597 INFO:tasks.workunit.client.0.smithi063.stdout:3/826: creat da/d63/f10c x:0 0 0 2024-03-22T07:16:03.597 INFO:tasks.workunit.client.1.smithi069.stdout:8/649: getdents d1/d1c/d3e 0 2024-03-22T07:16:03.598 INFO:tasks.workunit.client.1.smithi069.stdout:4/580: write d6/d4b/f97 [1483077,39653] 0 2024-03-22T07:16:03.598 INFO:tasks.workunit.client.0.smithi063.stdout:6/759: symlink d3/d6/d35/l102 0 2024-03-22T07:16:03.600 INFO:tasks.workunit.client.0.smithi063.stdout:6/760: creat d3/d15/d16/d60/d6d/d4e/d5a/f103 x:0 0 0 2024-03-22T07:16:03.602 INFO:tasks.workunit.client.1.smithi069.stdout:9/456: sync 2024-03-22T07:16:03.602 INFO:tasks.workunit.client.1.smithi069.stdout:2/355: sync 2024-03-22T07:16:03.602 INFO:tasks.workunit.client.1.smithi069.stdout:2/356: dread - d18/d4d/d27/d33/d3d/f54 zero size 2024-03-22T07:16:03.603 INFO:tasks.workunit.client.0.smithi063.stdout:6/761: rmdir d3/d15/d16/d60/d6d/d4c/d8b 0 2024-03-22T07:16:03.603 INFO:tasks.workunit.client.0.smithi063.stdout:6/762: dread - d3/d15/d16/d60/d6d/d4c/fd5 zero size 2024-03-22T07:16:03.603 INFO:tasks.workunit.client.1.smithi069.stdout:4/581: truncate d6/d12/d18/f45 2177348 0 2024-03-22T07:16:03.604 INFO:tasks.workunit.client.1.smithi069.stdout:4/582: dread d6/d12/f6f [0,4194304] 0 2024-03-22T07:16:03.606 INFO:tasks.workunit.client.1.smithi069.stdout:8/650: rename d1/d1c/d48/f7a to d1/d1c/d3e/d66/d9e/fd6 0 2024-03-22T07:16:03.606 INFO:tasks.workunit.client.0.smithi063.stdout:6/763: dread d3/d6/d2a/d38/da5/fe1 [0,4194304] 0 2024-03-22T07:16:03.607 INFO:tasks.workunit.client.1.smithi069.stdout:3/500: sync 2024-03-22T07:16:03.608 INFO:tasks.workunit.client.1.smithi069.stdout:9/457: unlink d2/dd/d6e/l8c 0 2024-03-22T07:16:03.609 INFO:tasks.workunit.client.1.smithi069.stdout:0/610: dread d3/d7/d15/d29/d5a/d84/f3b [0,4194304] 0 2024-03-22T07:16:03.612 INFO:tasks.workunit.client.1.smithi069.stdout:8/651: truncate d1/d1c/d3e/d88/dcd/f2d 703933 0 2024-03-22T07:16:03.612 INFO:tasks.workunit.client.1.smithi069.stdout:8/652: fdatasync d1/d1c/f52 0 2024-03-22T07:16:03.613 INFO:tasks.workunit.client.1.smithi069.stdout:2/357: dread d18/d4d/d34/f3f [0,4194304] 0 2024-03-22T07:16:03.613 INFO:tasks.workunit.client.1.smithi069.stdout:9/458: stat d2/dd/d3c/d48/d7c/c91 0 2024-03-22T07:16:03.615 INFO:tasks.workunit.client.1.smithi069.stdout:4/583: link d6/d12/l32 d6/dc/d23/d59/lc3 0 2024-03-22T07:16:03.615 INFO:tasks.workunit.client.1.smithi069.stdout:4/584: truncate d6/d4b/f92 728014 0 2024-03-22T07:16:03.615 INFO:tasks.workunit.client.1.smithi069.stdout:2/358: getdents d18/d4d/d3e 0 2024-03-22T07:16:03.618 INFO:tasks.workunit.client.1.smithi069.stdout:9/459: creat d2/d83/f92 x:0 0 0 2024-03-22T07:16:03.618 INFO:tasks.workunit.client.1.smithi069.stdout:8/653: dread d1/d1c/d3e/d66/f75 [0,4194304] 0 2024-03-22T07:16:03.620 INFO:tasks.workunit.client.1.smithi069.stdout:4/585: unlink d6/d9b/d1f/f42 0 2024-03-22T07:16:03.623 INFO:tasks.workunit.client.1.smithi069.stdout:9/460: mknod d2/dd/d6e/c93 0 2024-03-22T07:16:03.624 INFO:tasks.workunit.client.1.smithi069.stdout:0/611: write d3/d7/d15/d29/d5a/d84/fa0 [3054093,30234] 0 2024-03-22T07:16:03.624 INFO:tasks.workunit.client.1.smithi069.stdout:0/612: fsync d3/d7/d15/f20 0 2024-03-22T07:16:03.626 INFO:tasks.workunit.client.0.smithi063.stdout:7/641: dwrite d5/da/d28/d4b/f9c [0,4194304] 0 2024-03-22T07:16:03.626 INFO:tasks.workunit.client.0.smithi063.stdout:7/642: dread - d5/da/d5f/d33/fa3 zero size 2024-03-22T07:16:03.627 INFO:tasks.workunit.client.1.smithi069.stdout:9/461: mknod d2/d31/c94 0 2024-03-22T07:16:03.627 INFO:tasks.workunit.client.1.smithi069.stdout:7/491: dwrite d9/d76/d57/d75/d7a/d96/f9e [0,4194304] 0 2024-03-22T07:16:03.627 INFO:tasks.workunit.client.0.smithi063.stdout:3/827: dread da/d16/d26/f8b [0,4194304] 0 2024-03-22T07:16:03.628 INFO:tasks.workunit.client.0.smithi063.stdout:1/887: dwrite d4/df/d13/d2b/d6c/da8/fc5 [0,4194304] 0 2024-03-22T07:16:03.628 INFO:tasks.workunit.client.0.smithi063.stdout:1/888: chown d4/df/d13/d39/f5e 226679456 1 2024-03-22T07:16:03.628 INFO:tasks.workunit.client.0.smithi063.stdout:1/889: write d4/df/d13/d39/f5e [806477,102742] 0 2024-03-22T07:16:03.629 INFO:tasks.workunit.client.0.smithi063.stdout:7/643: truncate d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/f5d 7432452 0 2024-03-22T07:16:03.629 INFO:tasks.workunit.client.0.smithi063.stdout:7/644: dread - d5/da/d28/f9a zero size 2024-03-22T07:16:03.629 INFO:tasks.workunit.client.0.smithi063.stdout:7/645: read d5/da/d28/da7/f72 [343923,34467] 0 2024-03-22T07:16:03.629 INFO:tasks.workunit.client.1.smithi069.stdout:4/586: rename d6/d9b/d4a to d6/dc4 0 2024-03-22T07:16:03.630 INFO:tasks.workunit.client.1.smithi069.stdout:4/587: read d6/d4b/f97 [1786836,67818] 0 2024-03-22T07:16:03.630 INFO:tasks.workunit.client.1.smithi069.stdout:8/654: write d1/d1c/d3e/d66/d9e/fd6 [2433231,101130] 0 2024-03-22T07:16:03.632 INFO:tasks.workunit.client.0.smithi063.stdout:4/698: dwrite d1/d5/f97 [0,4194304] 0 2024-03-22T07:16:03.632 INFO:tasks.workunit.client.1.smithi069.stdout:9/462: creat d2/d71/d90/f95 x:0 0 0 2024-03-22T07:16:03.632 INFO:tasks.workunit.client.1.smithi069.stdout:6/436: sync 2024-03-22T07:16:03.632 INFO:tasks.workunit.client.1.smithi069.stdout:3/501: dwrite d5/d9/d26/d3f/d35/f58 [0,4194304] 0 2024-03-22T07:16:03.632 INFO:tasks.workunit.client.1.smithi069.stdout:6/437: dread - d4/dd/d12/d2b/f3c zero size 2024-03-22T07:16:03.633 INFO:tasks.workunit.client.1.smithi069.stdout:3/502: chown d5/d33 1 1 2024-03-22T07:16:03.635 INFO:tasks.workunit.client.0.smithi063.stdout:1/890: creat d4/df/d13/d39/d6f/f131 x:0 0 0 2024-03-22T07:16:03.637 INFO:tasks.workunit.client.0.smithi063.stdout:3/828: getdents da/d16/d3f 0 2024-03-22T07:16:03.637 INFO:tasks.workunit.client.0.smithi063.stdout:3/829: write da/d63/d65/d4b/dfa/dfc/f106 [285634,110459] 0 2024-03-22T07:16:03.641 INFO:tasks.workunit.client.1.smithi069.stdout:0/613: rename d3/d7/d15/d29/d5a/d84/f11 to d3/d7/d24/d6a/fc9 0 2024-03-22T07:16:03.642 INFO:tasks.workunit.client.0.smithi063.stdout:4/699: dread - d1/d5/d43/d27/d2c/d3d/f99 zero size 2024-03-22T07:16:03.642 INFO:tasks.workunit.client.0.smithi063.stdout:4/700: getdents d1/d5/d43/d27/d2c/d4a 0 2024-03-22T07:16:03.642 INFO:tasks.workunit.client.1.smithi069.stdout:8/655: link d1/d1c/d3e/d88/dcd/d6c/c60 d1/cd7 0 2024-03-22T07:16:03.645 INFO:tasks.workunit.client.1.smithi069.stdout:9/463: dread d2/dd/d3c/d48/f4a [0,4194304] 0 2024-03-22T07:16:03.645 INFO:tasks.workunit.client.1.smithi069.stdout:9/464: creat d2/dd/d3c/d2e/d84/f96 x:0 0 0 2024-03-22T07:16:03.646 INFO:tasks.workunit.client.0.smithi063.stdout:3/830: creat da/d63/d65/d81/d9b/d9f/dcf/f10d x:0 0 0 2024-03-22T07:16:03.647 INFO:tasks.workunit.client.0.smithi063.stdout:3/831: dread da/d16/d3f/f44 [0,4194304] 0 2024-03-22T07:16:03.647 INFO:tasks.workunit.client.0.smithi063.stdout:2/697: sync 2024-03-22T07:16:03.648 INFO:tasks.workunit.client.1.smithi069.stdout:3/503: unlink d5/d9/d23/c34 0 2024-03-22T07:16:03.648 INFO:tasks.workunit.client.1.smithi069.stdout:3/504: creat d5/fa0 x:0 0 0 2024-03-22T07:16:03.648 INFO:tasks.workunit.client.1.smithi069.stdout:7/492: dwrite f4 [0,4194304] 0 2024-03-22T07:16:03.649 INFO:tasks.workunit.client.0.smithi063.stdout:3/832: mknod da/d63/d65/d4b/d10a/c10e 0 2024-03-22T07:16:03.649 INFO:tasks.workunit.client.1.smithi069.stdout:0/614: mkdir d3/d7/d15/d29/d5a/d99/dca 0 2024-03-22T07:16:03.650 INFO:tasks.workunit.client.1.smithi069.stdout:6/438: rmdir d4/dd/d12/d44 39 2024-03-22T07:16:03.651 INFO:tasks.workunit.client.0.smithi063.stdout:2/698: truncate d0/de/f4c 996888 0 2024-03-22T07:16:03.653 INFO:tasks.workunit.client.0.smithi063.stdout:1/891: dread d4/d71/dbb/dc0/fea [0,4194304] 0 2024-03-22T07:16:03.653 INFO:tasks.workunit.client.0.smithi063.stdout:3/833: creat da/d63/d65/d92/d96/f10f x:0 0 0 2024-03-22T07:16:03.656 INFO:tasks.workunit.client.1.smithi069.stdout:3/505: creat d5/d9/d26/d3f/d56/d7f/fa1 x:0 0 0 2024-03-22T07:16:03.656 INFO:tasks.workunit.client.0.smithi063.stdout:2/699: creat d0/de/d1b/d32/fe1 x:0 0 0 2024-03-22T07:16:03.657 INFO:tasks.workunit.client.1.smithi069.stdout:3/506: write d5/d9/f76 [499315,4676] 0 2024-03-22T07:16:03.657 INFO:tasks.workunit.client.1.smithi069.stdout:3/507: dread - d5/d9/db/d3b/f95 zero size 2024-03-22T07:16:03.657 INFO:tasks.workunit.client.1.smithi069.stdout:3/508: fsync d5/f4e 0 2024-03-22T07:16:03.657 INFO:tasks.workunit.client.1.smithi069.stdout:3/509: stat d5/d9/db/f30 0 2024-03-22T07:16:03.657 INFO:tasks.workunit.client.1.smithi069.stdout:7/493: getdents d9/d42 0 2024-03-22T07:16:03.658 INFO:tasks.workunit.client.1.smithi069.stdout:0/615: mkdir d3/d9a/dcb 0 2024-03-22T07:16:03.659 INFO:tasks.workunit.client.1.smithi069.stdout:0/616: stat d3/d7/d90/fa5 0 2024-03-22T07:16:03.660 INFO:tasks.workunit.client.1.smithi069.stdout:3/510: mkdir d5/da2 0 2024-03-22T07:16:03.660 INFO:tasks.workunit.client.1.smithi069.stdout:3/511: truncate d5/f4e 1561520 0 2024-03-22T07:16:03.661 INFO:tasks.workunit.client.0.smithi063.stdout:4/701: dread d1/d5/f67 [0,4194304] 0 2024-03-22T07:16:03.661 INFO:tasks.workunit.client.1.smithi069.stdout:3/512: chown d5/d9/d26/d3f/d54 247394944 1 2024-03-22T07:16:03.661 INFO:tasks.workunit.client.0.smithi063.stdout:3/834: rmdir da/d16/d26/d69/dae/dd8 0 2024-03-22T07:16:03.661 INFO:tasks.workunit.client.0.smithi063.stdout:3/835: chown da/d16/d26/f48 18 1 2024-03-22T07:16:03.663 INFO:tasks.workunit.client.1.smithi069.stdout:0/617: unlink d3/d7/d15/d29/d6e/la1 0 2024-03-22T07:16:03.664 INFO:tasks.workunit.client.0.smithi063.stdout:2/700: link d0/c56 d0/de/d1b/d63/d6e/ce2 0 2024-03-22T07:16:03.665 INFO:tasks.workunit.client.1.smithi069.stdout:3/513: rename d5/d9/d26/d3f/l24 to d5/d9/db/la3 0 2024-03-22T07:16:03.665 INFO:tasks.workunit.client.1.smithi069.stdout:5/394: sync 2024-03-22T07:16:03.665 INFO:tasks.workunit.client.1.smithi069.stdout:1/430: sync 2024-03-22T07:16:03.665 INFO:tasks.workunit.client.1.smithi069.stdout:1/431: stat d1/d13/d36 0 2024-03-22T07:16:03.665 INFO:tasks.workunit.client.1.smithi069.stdout:6/439: write d4/d7/f3e [3976195,8510] 0 2024-03-22T07:16:03.666 INFO:tasks.workunit.client.1.smithi069.stdout:0/618: truncate d3/d7/d15/d1f/f41 5768729 0 2024-03-22T07:16:03.666 INFO:tasks.workunit.client.0.smithi063.stdout:1/892: symlink d4/df/d13/d2b/d6c/da6/l132 0 2024-03-22T07:16:03.666 INFO:tasks.workunit.client.0.smithi063.stdout:1/893: chown d4/df/d13/d39/f6b 333 1 2024-03-22T07:16:03.668 INFO:tasks.workunit.client.1.smithi069.stdout:3/514: mknod d5/ca4 0 2024-03-22T07:16:03.668 INFO:tasks.workunit.client.1.smithi069.stdout:3/515: truncate d5/d33/d7b/f93 242190 0 2024-03-22T07:16:03.669 INFO:tasks.workunit.client.0.smithi063.stdout:2/701: mknod d0/de/d1b/d63/d6e/ce3 0 2024-03-22T07:16:03.670 INFO:tasks.workunit.client.1.smithi069.stdout:4/588: dwrite d6/dc/d23/f77 [0,4194304] 0 2024-03-22T07:16:03.671 INFO:tasks.workunit.client.0.smithi063.stdout:4/702: getdents d1/d5/dc/d95 0 2024-03-22T07:16:03.671 INFO:tasks.workunit.client.1.smithi069.stdout:5/395: rmdir d1/d7/d3d 39 2024-03-22T07:16:03.673 INFO:tasks.workunit.client.1.smithi069.stdout:1/432: rename d1/d25/d41/d1e/f76 to d1/d25/d41/d64/f85 0 2024-03-22T07:16:03.673 INFO:tasks.workunit.client.1.smithi069.stdout:6/440: dread d4/d7/da/f1b [0,4194304] 0 2024-03-22T07:16:03.673 INFO:tasks.workunit.client.0.smithi063.stdout:3/836: dread da/d16/d26/f29 [0,4194304] 0 2024-03-22T07:16:03.673 INFO:tasks.workunit.client.0.smithi063.stdout:2/702: creat d0/de/d1b/d63/dd6/fe4 x:0 0 0 2024-03-22T07:16:03.678 INFO:tasks.workunit.client.1.smithi069.stdout:7/494: write d9/db/f16 [2433828,36405] 0 2024-03-22T07:16:03.680 INFO:tasks.workunit.client.1.smithi069.stdout:4/589: dread d6/fbb [0,4194304] 0 2024-03-22T07:16:03.680 INFO:tasks.workunit.client.1.smithi069.stdout:4/590: write d6/d12/f70 [16309,33972] 0 2024-03-22T07:16:03.680 INFO:tasks.workunit.client.1.smithi069.stdout:0/619: dread d3/d7/d15/d29/d5a/d84/fa0 [0,4194304] 0 2024-03-22T07:16:03.680 INFO:tasks.workunit.client.1.smithi069.stdout:0/620: chown d3/d7/d15/d29/d5a/d5d/l9c 1445512 1 2024-03-22T07:16:03.681 INFO:tasks.workunit.client.1.smithi069.stdout:6/441: creat d4/dd/d12/d2b/d72/d73/d74/f89 x:0 0 0 2024-03-22T07:16:03.681 INFO:tasks.workunit.client.1.smithi069.stdout:8/656: dwrite d1/d1c/d48/d5e/f79 [0,4194304] 0 2024-03-22T07:16:03.681 INFO:tasks.workunit.client.1.smithi069.stdout:9/465: dwrite d2/dd/d3c/d48/f4a [0,4194304] 0 2024-03-22T07:16:03.682 INFO:tasks.workunit.client.1.smithi069.stdout:5/396: read d1/d6/f6f [845596,2499] 0 2024-03-22T07:16:03.692 INFO:tasks.workunit.client.1.smithi069.stdout:7/495: write d9/d76/d57/d44/d53/d74/d30/f3a [2566176,123470] 0 2024-03-22T07:16:03.695 INFO:tasks.workunit.client.1.smithi069.stdout:4/591: link d6/d9b/d1f/d3f/ca7 d6/d16/d68/cc5 0 2024-03-22T07:16:03.695 INFO:tasks.workunit.client.1.smithi069.stdout:6/442: dread d4/dd/f25 [4194304,4194304] 0 2024-03-22T07:16:03.698 INFO:tasks.workunit.client.1.smithi069.stdout:8/657: creat d1/d1c/d3e/d88/da8/dbb/fd8 x:0 0 0 2024-03-22T07:16:03.701 INFO:tasks.workunit.client.1.smithi069.stdout:9/466: creat d2/d21/d77/d81/f97 x:0 0 0 2024-03-22T07:16:03.701 INFO:tasks.workunit.client.1.smithi069.stdout:9/467: readlink d2/d21/l6b 0 2024-03-22T07:16:03.701 INFO:tasks.workunit.client.1.smithi069.stdout:9/468: read - d2/d71/f8d zero size 2024-03-22T07:16:03.702 INFO:tasks.workunit.client.1.smithi069.stdout:3/516: rename d5/d33 to d5/d9/d26/d3f/d56/d8a/da5 0 2024-03-22T07:16:03.706 INFO:tasks.workunit.client.1.smithi069.stdout:8/658: write d1/d1c/d48/fc3 [360787,6330] 0 2024-03-22T07:16:03.707 INFO:tasks.workunit.client.1.smithi069.stdout:9/469: creat d2/dd/d6e/f98 x:0 0 0 2024-03-22T07:16:03.709 INFO:tasks.workunit.client.1.smithi069.stdout:0/621: rename d3/d7/le to d3/d7/d15/d29/d5a/d5d/d98/lcc 0 2024-03-22T07:16:03.709 INFO:tasks.workunit.client.1.smithi069.stdout:0/622: read d3/d7/d15/d29/f69 [80035,7712] 0 2024-03-22T07:16:03.709 INFO:tasks.workunit.client.1.smithi069.stdout:0/623: truncate d3/d7/d17/d1c/f91 1074563 0 2024-03-22T07:16:03.709 INFO:tasks.workunit.client.1.smithi069.stdout:0/624: creat d3/d7/d24/d6a/fcd x:0 0 0 2024-03-22T07:16:03.710 INFO:tasks.workunit.client.1.smithi069.stdout:4/592: getdents d6/d16 0 2024-03-22T07:16:03.711 INFO:tasks.workunit.client.1.smithi069.stdout:4/593: chown d6/dc/d23/d59/lc3 500011 1 2024-03-22T07:16:03.711 INFO:tasks.workunit.client.0.smithi063.stdout:2/703: dwrite d0/de/d12/d20/f47 [0,4194304] 0 2024-03-22T07:16:03.712 INFO:tasks.workunit.client.1.smithi069.stdout:8/659: getdents d1/d1c/d3e/d88/da8 0 2024-03-22T07:16:03.714 INFO:tasks.workunit.client.1.smithi069.stdout:6/443: rename d4/dd/d12/d2b/d72/d73/d74/f89 to d4/dd/d12/d13/f8a 0 2024-03-22T07:16:03.714 INFO:tasks.workunit.client.1.smithi069.stdout:6/444: stat d4/dd/d12/f32 0 2024-03-22T07:16:03.714 INFO:tasks.workunit.client.1.smithi069.stdout:6/445: chown d4/dd/d12/d44/f4e 1273921 1 2024-03-22T07:16:03.717 INFO:tasks.workunit.client.1.smithi069.stdout:3/517: read d5/d9/d23/f2b [4163707,69300] 0 2024-03-22T07:16:03.717 INFO:tasks.workunit.client.1.smithi069.stdout:3/518: dread - d5/d9/db/d3b/f95 zero size 2024-03-22T07:16:03.718 INFO:tasks.workunit.client.1.smithi069.stdout:0/625: dread d3/d7/d15/fa3 [0,4194304] 0 2024-03-22T07:16:03.718 INFO:tasks.workunit.client.1.smithi069.stdout:4/594: mkdir d6/d16/d3a/db4/dbc/dc6 0 2024-03-22T07:16:03.718 INFO:tasks.workunit.client.1.smithi069.stdout:4/595: chown d6/d12/d18/d21/l40 589 1 2024-03-22T07:16:03.718 INFO:tasks.workunit.client.1.smithi069.stdout:4/596: chown d6/dc/d1d/c4e 1960354 1 2024-03-22T07:16:03.721 INFO:tasks.workunit.client.1.smithi069.stdout:7/496: dwrite f4 [0,4194304] 0 2024-03-22T07:16:03.721 INFO:tasks.workunit.client.1.smithi069.stdout:7/497: creat d9/d76/d57/d44/d53/d2e/fa6 x:0 0 0 2024-03-22T07:16:03.722 INFO:tasks.workunit.client.1.smithi069.stdout:8/660: creat d1/d1c/fd9 x:0 0 0 2024-03-22T07:16:03.722 INFO:tasks.workunit.client.0.smithi063.stdout:4/703: dwrite d1/d5/f67 [0,4194304] 0 2024-03-22T07:16:03.722 INFO:tasks.workunit.client.0.smithi063.stdout:3/837: dwrite da/d63/d65/d92/fff [0,4194304] 0 2024-03-22T07:16:03.722 INFO:tasks.workunit.client.0.smithi063.stdout:3/838: write da/d16/f97 [6420961,24023] 0 2024-03-22T07:16:03.723 INFO:tasks.workunit.client.1.smithi069.stdout:1/433: getdents d1/d25/d41/d64 0 2024-03-22T07:16:03.723 INFO:tasks.workunit.client.1.smithi069.stdout:8/661: write d1/d1c/d48/d5e/fa5 [658114,66143] 0 2024-03-22T07:16:03.723 INFO:tasks.workunit.client.1.smithi069.stdout:8/662: dread - d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/fcf zero size 2024-03-22T07:16:03.724 INFO:tasks.workunit.client.0.smithi063.stdout:4/704: mknod d1/d5/dc/dda/d79/daa/cf1 0 2024-03-22T07:16:03.725 INFO:tasks.workunit.client.1.smithi069.stdout:0/626: unlink d3/d7/d15/d29/d5a/d5d/la4 0 2024-03-22T07:16:03.726 INFO:tasks.workunit.client.1.smithi069.stdout:0/627: truncate d3/d7/d17/d1c/f91 1859680 0 2024-03-22T07:16:03.727 INFO:tasks.workunit.client.1.smithi069.stdout:4/597: unlink d6/dc4/lb0 0 2024-03-22T07:16:03.728 INFO:tasks.workunit.client.1.smithi069.stdout:4/598: write d6/d16/d83/d9a/fa3 [684978,93994] 0 2024-03-22T07:16:03.734 INFO:tasks.workunit.client.1.smithi069.stdout:7/498: unlink d9/f46 0 2024-03-22T07:16:03.737 INFO:tasks.workunit.client.1.smithi069.stdout:1/434: truncate d1/d2a/f6c 187272 0 2024-03-22T07:16:03.737 INFO:tasks.workunit.client.1.smithi069.stdout:1/435: write d1/f22 [1566525,69835] 0 2024-03-22T07:16:03.739 INFO:tasks.workunit.client.1.smithi069.stdout:8/663: creat d1/d1c/d3e/d66/d9e/dd1/fda x:0 0 0 2024-03-22T07:16:03.742 INFO:tasks.workunit.client.1.smithi069.stdout:4/599: mknod d6/d9b/d1f/d3f/d51/cc7 0 2024-03-22T07:16:03.743 INFO:tasks.workunit.client.1.smithi069.stdout:8/664: getdents d1/d1c/d3e/d88/dcd/d6c/d4e 0 2024-03-22T07:16:03.744 INFO:tasks.workunit.client.1.smithi069.stdout:0/628: mkdir d3/d7/d15/dce 0 2024-03-22T07:16:03.745 INFO:tasks.workunit.client.1.smithi069.stdout:0/629: write d3/d7/d5e/f61 [1070590,53132] 0 2024-03-22T07:16:03.745 INFO:tasks.workunit.client.1.smithi069.stdout:6/446: dwrite d4/dd/d41/d78/f85 [0,4194304] 0 2024-03-22T07:16:03.745 INFO:tasks.workunit.client.1.smithi069.stdout:6/447: readlink d4/d7/da/l84 0 2024-03-22T07:16:03.746 INFO:tasks.workunit.client.1.smithi069.stdout:7/499: link d9/db/d48/d4b/f4f d9/d76/d57/d44/d53/fa7 0 2024-03-22T07:16:03.746 INFO:tasks.workunit.client.1.smithi069.stdout:8/665: mkdir d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/ddb 0 2024-03-22T07:16:03.747 INFO:tasks.workunit.client.1.smithi069.stdout:6/448: mknod d4/d7/da/d3f/d76/c8b 0 2024-03-22T07:16:03.751 INFO:tasks.workunit.client.1.smithi069.stdout:6/449: rename d4/l2e to d4/dd/d12/l8c 0 2024-03-22T07:16:03.751 INFO:tasks.workunit.client.1.smithi069.stdout:6/450: symlink d4/d7/da/d3f/l8d 0 2024-03-22T07:16:03.751 INFO:tasks.workunit.client.1.smithi069.stdout:6/451: truncate d4/dd/d12/d13/f23 1373435 0 2024-03-22T07:16:03.751 INFO:tasks.workunit.client.1.smithi069.stdout:6/452: link d4/dd/l15 d4/dd/d41/l8e 0 2024-03-22T07:16:03.752 INFO:tasks.workunit.client.1.smithi069.stdout:6/453: truncate d4/d7/f35 856170 0 2024-03-22T07:16:03.754 INFO:tasks.workunit.client.1.smithi069.stdout:6/454: link d4/d7/f35 d4/dd/f8f 0 2024-03-22T07:16:03.754 INFO:tasks.workunit.client.1.smithi069.stdout:6/455: dread - d4/dd/d12/d2b/f3c zero size 2024-03-22T07:16:03.755 INFO:tasks.workunit.client.1.smithi069.stdout:6/456: dread d4/dd/d12/d2b/f30 [0,4194304] 0 2024-03-22T07:16:03.755 INFO:tasks.workunit.client.1.smithi069.stdout:6/457: fsync d4/dd/d12/d44/f7a 0 2024-03-22T07:16:03.756 INFO:tasks.workunit.client.1.smithi069.stdout:6/458: mknod d4/dd/d41/d78/c90 0 2024-03-22T07:16:03.757 INFO:tasks.workunit.client.1.smithi069.stdout:6/459: chown d4/dd/d12/d2b 47 1 2024-03-22T07:16:03.762 INFO:tasks.workunit.client.1.smithi069.stdout:0/630: dread d3/d7/d17/faa [0,4194304] 0 2024-03-22T07:16:03.769 INFO:tasks.workunit.client.1.smithi069.stdout:8/666: write d1/d1c/d3e/f16 [3514995,80184] 0 2024-03-22T07:16:03.769 INFO:tasks.workunit.client.1.smithi069.stdout:3/519: dwrite d5/d9/d26/d3f/d56/d8a/da5/d7b/f93 [0,4194304] 0 2024-03-22T07:16:03.769 INFO:tasks.workunit.client.1.smithi069.stdout:4/600: dwrite d6/d12/d18/f4f [0,4194304] 0 2024-03-22T07:16:03.769 INFO:tasks.workunit.client.0.smithi063.stdout:3/839: dwrite da/d16/d89/d8c/dc1/fcb [0,4194304] 0 2024-03-22T07:16:03.769 INFO:tasks.workunit.client.0.smithi063.stdout:3/840: readlink da/d63/d65/d81/d9b/d9f/ld4 0 2024-03-22T07:16:03.769 INFO:tasks.workunit.client.0.smithi063.stdout:3/841: creat da/d16/d89/da9/dd3/f110 x:0 0 0 2024-03-22T07:16:03.770 INFO:tasks.workunit.client.0.smithi063.stdout:4/705: dwrite d1/d5/dc/d95/fb5 [0,4194304] 0 2024-03-22T07:16:03.770 INFO:tasks.workunit.client.0.smithi063.stdout:3/842: mkdir da/d16/d26/d69/d111 0 2024-03-22T07:16:03.770 INFO:tasks.workunit.client.0.smithi063.stdout:4/706: dread d1/d5/d43/d27/d2c/d3d/d54/f8d [0,4194304] 0 2024-03-22T07:16:03.770 INFO:tasks.workunit.client.0.smithi063.stdout:3/843: write da/d16/d26/f76 [4811309,91834] 0 2024-03-22T07:16:03.770 INFO:tasks.workunit.client.0.smithi063.stdout:4/707: creat d1/d5/d85/ff2 x:0 0 0 2024-03-22T07:16:03.771 INFO:tasks.workunit.client.1.smithi069.stdout:8/667: truncate d1/d1c/d3e/d88/dcd/d20/f2f 3100989 0 2024-03-22T07:16:03.772 INFO:tasks.workunit.client.0.smithi063.stdout:3/844: truncate da/d16/d89/d8c/dc1/f93 3598275 0 2024-03-22T07:16:03.772 INFO:tasks.workunit.client.1.smithi069.stdout:3/520: rename d5/d9/db/f3d to d5/da2/fa6 0 2024-03-22T07:16:03.772 INFO:tasks.workunit.client.1.smithi069.stdout:3/521: stat d5/d9/d26/d3f/d54/c87 0 2024-03-22T07:16:03.773 INFO:tasks.workunit.client.1.smithi069.stdout:1/436: dread d1/f22 [0,4194304] 0 2024-03-22T07:16:03.773 INFO:tasks.workunit.client.1.smithi069.stdout:1/437: creat d1/d2a/d50/d4c/f86 x:0 0 0 2024-03-22T07:16:03.773 INFO:tasks.workunit.client.0.smithi063.stdout:4/708: mkdir d1/d5/d43/d27/d2c/df3 0 2024-03-22T07:16:03.773 INFO:tasks.workunit.client.0.smithi063.stdout:4/709: chown d1/d5/d43/d27/d2c/d3d/c42 1 1 2024-03-22T07:16:03.774 INFO:tasks.workunit.client.0.smithi063.stdout:4/710: readlink d1/d5/dc/l37 0 2024-03-22T07:16:03.774 INFO:tasks.workunit.client.0.smithi063.stdout:4/711: write d1/d5/f97 [4648190,46318] 0 2024-03-22T07:16:03.774 INFO:tasks.workunit.client.0.smithi063.stdout:4/712: chown d1/d5/f20 6821 1 2024-03-22T07:16:03.775 INFO:tasks.workunit.client.0.smithi063.stdout:4/713: mkdir d1/d5/d43/d27/d2c/df4 0 2024-03-22T07:16:03.776 INFO:tasks.workunit.client.0.smithi063.stdout:4/714: creat d1/d5/d43/d27/d2c/d4a/ff5 x:0 0 0 2024-03-22T07:16:03.776 INFO:tasks.workunit.client.1.smithi069.stdout:0/631: dread d3/d7/d17/f3f [0,4194304] 0 2024-03-22T07:16:03.785 INFO:tasks.workunit.client.1.smithi069.stdout:6/460: dwrite d4/f60 [0,4194304] 0 2024-03-22T07:16:03.785 INFO:tasks.workunit.client.1.smithi069.stdout:8/668: unlink d1/d1c/d3e/l87 0 2024-03-22T07:16:03.785 INFO:tasks.workunit.client.1.smithi069.stdout:8/669: fdatasync d1/d1c/d3e/fb0 0 2024-03-22T07:16:03.786 INFO:tasks.workunit.client.1.smithi069.stdout:8/670: chown d1/d1c/d3e/d88/dcd/d20/f2f 0 1 2024-03-22T07:16:03.786 INFO:tasks.workunit.client.1.smithi069.stdout:8/671: readlink d1/d1c/lbf 0 2024-03-22T07:16:03.786 INFO:tasks.workunit.client.1.smithi069.stdout:8/672: fdatasync d1/d1c/d3e/d88/dcd/d6c/f72 0 2024-03-22T07:16:03.786 INFO:tasks.workunit.client.1.smithi069.stdout:3/522: link d5/d9/db/d3b/l66 d5/d9/d26/d3f/d35/la7 0 2024-03-22T07:16:03.787 INFO:tasks.workunit.client.1.smithi069.stdout:3/523: dread d5/f4e [0,4194304] 0 2024-03-22T07:16:03.789 INFO:tasks.workunit.client.1.smithi069.stdout:1/438: write d1/d25/f38 [229397,117436] 0 2024-03-22T07:16:03.789 INFO:tasks.workunit.client.1.smithi069.stdout:7/500: getdents d9 0 2024-03-22T07:16:03.789 INFO:tasks.workunit.client.1.smithi069.stdout:7/501: chown d9/d76/d57/d75/d7a/l7d 9 1 2024-03-22T07:16:03.789 INFO:tasks.workunit.client.1.smithi069.stdout:7/502: fdatasync d9/d76/d57/d75/d7a/fa2 0 2024-03-22T07:16:03.789 INFO:tasks.workunit.client.1.smithi069.stdout:7/503: fsync d9/d76/d57/d44/d53/f60 0 2024-03-22T07:16:03.790 INFO:tasks.workunit.client.0.smithi063.stdout:4/715: dread d1/d5/d43/d27/d2c/d3d/f62 [0,4194304] 0 2024-03-22T07:16:03.791 INFO:tasks.workunit.client.1.smithi069.stdout:1/439: read d1/d25/d41/d1e/f51 [1046424,108265] 0 2024-03-22T07:16:03.791 INFO:tasks.workunit.client.1.smithi069.stdout:1/440: creat d1/d13/d36/d5b/f87 x:0 0 0 2024-03-22T07:16:03.791 INFO:tasks.workunit.client.1.smithi069.stdout:1/441: truncate d1/d13/d82/f83 222391 0 2024-03-22T07:16:03.792 INFO:tasks.workunit.client.1.smithi069.stdout:1/442: write d1/d2a/d50/d4c/d55/f80 [829758,95437] 0 2024-03-22T07:16:03.792 INFO:tasks.workunit.client.1.smithi069.stdout:3/524: mkdir d5/d9/d26/d3f/d56/d8a/da5/d7b/da8 0 2024-03-22T07:16:03.792 INFO:tasks.workunit.client.0.smithi063.stdout:4/716: rmdir d1/d5/d3c 39 2024-03-22T07:16:03.792 INFO:tasks.workunit.client.0.smithi063.stdout:4/717: write d1/d5/d3c/d8f/fbc [831002,58982] 0 2024-03-22T07:16:03.792 INFO:tasks.workunit.client.0.smithi063.stdout:4/718: chown d1/d5/dc/dda/d79/daa 56422 1 2024-03-22T07:16:03.794 INFO:tasks.workunit.client.1.smithi069.stdout:6/461: dread d4/d7/da/f5a [0,4194304] 0 2024-03-22T07:16:03.801 INFO:tasks.workunit.client.0.smithi063.stdout:4/719: dread d1/d5/d3c/f47 [4194304,4194304] 0 2024-03-22T07:16:03.802 INFO:tasks.workunit.client.1.smithi069.stdout:6/462: link d4/dd/d12/d13/l88 d4/d7/l91 0 2024-03-22T07:16:03.803 INFO:tasks.workunit.client.0.smithi063.stdout:4/720: creat d1/d5/d43/d27/d2c/d4a/d80/ff6 x:0 0 0 2024-03-22T07:16:03.804 INFO:tasks.workunit.client.0.smithi063.stdout:4/721: write d1/d5/d43/d27/d2c/d4a/d80/fd9 [585963,2013] 0 2024-03-22T07:16:03.804 INFO:tasks.workunit.client.0.smithi063.stdout:4/722: stat d1/f3 0 2024-03-22T07:16:03.808 INFO:tasks.workunit.client.1.smithi069.stdout:3/525: dread d5/d9/db/f30 [0,4194304] 0 2024-03-22T07:16:03.810 INFO:tasks.workunit.client.1.smithi069.stdout:3/526: link d5/d9/d26/d3f/f22 d5/d9/d23/fa9 0 2024-03-22T07:16:03.810 INFO:tasks.workunit.client.1.smithi069.stdout:3/527: chown d5/c8 122137530 1 2024-03-22T07:16:03.810 INFO:tasks.workunit.client.1.smithi069.stdout:3/528: truncate d5/d9/db/f63 1628265 0 2024-03-22T07:16:03.815 INFO:tasks.workunit.client.0.smithi063.stdout:4/723: read d1/f45 [2748063,39780] 0 2024-03-22T07:16:03.816 INFO:tasks.workunit.client.1.smithi069.stdout:6/463: write d4/dd/d12/d2b/f37 [3003227,731] 0 2024-03-22T07:16:03.816 INFO:tasks.workunit.client.1.smithi069.stdout:6/464: write d4/f46 [3597156,120625] 0 2024-03-22T07:16:03.822 INFO:tasks.workunit.client.1.smithi069.stdout:8/673: dwrite d1/d1c/d3e/d88/dcd/d20/f3b [0,4194304] 0 2024-03-22T07:16:03.822 INFO:tasks.workunit.client.1.smithi069.stdout:8/674: read d1/d1c/d48/d5e/fa5 [380027,118974] 0 2024-03-22T07:16:03.822 INFO:tasks.workunit.client.1.smithi069.stdout:8/675: readlink d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/lb6 0 2024-03-22T07:16:03.823 INFO:tasks.workunit.client.1.smithi069.stdout:8/676: fsync d1/d1c/d48/d56/fc9 0 2024-03-22T07:16:03.823 INFO:tasks.workunit.client.1.smithi069.stdout:7/504: dwrite d9/d76/d57/d44/f9b [0,4194304] 0 2024-03-22T07:16:03.823 INFO:tasks.workunit.client.1.smithi069.stdout:1/443: dwrite d1/d2a/f4e [0,4194304] 0 2024-03-22T07:16:03.823 INFO:tasks.workunit.client.1.smithi069.stdout:1/444: fsync d1/f35 0 2024-03-22T07:16:03.824 INFO:tasks.workunit.client.1.smithi069.stdout:1/445: read d1/d13/f2d [2379137,12222] 0 2024-03-22T07:16:03.827 INFO:tasks.workunit.client.1.smithi069.stdout:8/677: unlink d1/d1c/d3e/d88/dcd/d6c/d8e/cb5 0 2024-03-22T07:16:03.831 INFO:tasks.workunit.client.1.smithi069.stdout:3/529: dread d5/d9/d26/f4c [0,4194304] 0 2024-03-22T07:16:03.832 INFO:tasks.workunit.client.1.smithi069.stdout:8/678: mknod d1/d1c/d3e/d88/dd3/cdc 0 2024-03-22T07:16:03.832 INFO:tasks.workunit.client.1.smithi069.stdout:1/446: dread d1/d2a/f32 [0,4194304] 0 2024-03-22T07:16:03.833 INFO:tasks.workunit.client.0.smithi063.stdout:8/811: sync 2024-03-22T07:16:03.833 INFO:tasks.workunit.client.0.smithi063.stdout:8/812: rename d0/d26/d2d/d59/dc0/f104 to d0/d1b/d28/d7b/d4a/f116 0 2024-03-22T07:16:03.833 INFO:tasks.workunit.client.0.smithi063.stdout:8/813: stat d0/d24/d5b/f95 0 2024-03-22T07:16:03.833 INFO:tasks.workunit.client.0.smithi063.stdout:8/814: fsync d0/d1b/d28/d7b/d45/f58 0 2024-03-22T07:16:03.834 INFO:tasks.workunit.client.1.smithi069.stdout:6/465: dwrite d4/d7/f35 [0,4194304] 0 2024-03-22T07:16:03.834 INFO:tasks.workunit.client.1.smithi069.stdout:6/466: dread - d4/d7/f6e zero size 2024-03-22T07:16:03.834 INFO:tasks.workunit.client.1.smithi069.stdout:3/530: rmdir d5/d9/d26/d3f/d56/d8a/da5/d7b/da8 0 2024-03-22T07:16:03.835 INFO:tasks.workunit.client.1.smithi069.stdout:3/531: write d5/f8f [1017417,4243] 0 2024-03-22T07:16:03.835 INFO:tasks.workunit.client.1.smithi069.stdout:3/532: stat d5/d9/d26/d3f/f22 0 2024-03-22T07:16:03.835 INFO:tasks.workunit.client.1.smithi069.stdout:3/533: write d5/d9/d23/f78 [776548,83269] 0 2024-03-22T07:16:03.835 INFO:tasks.workunit.client.1.smithi069.stdout:6/467: write d4/dd/d12/d2b/f3a [181107,49721] 0 2024-03-22T07:16:03.845 INFO:tasks.workunit.client.0.smithi063.stdout:8/815: dread d0/d26/d2d/d83/fa8 [0,4194304] 0 2024-03-22T07:16:03.848 INFO:tasks.workunit.client.0.smithi063.stdout:8/816: creat d0/d9e/df0/f117 x:0 0 0 2024-03-22T07:16:03.848 INFO:tasks.workunit.client.0.smithi063.stdout:8/817: fdatasync d0/d1b/fc3 0 2024-03-22T07:16:03.848 INFO:tasks.workunit.client.0.smithi063.stdout:8/818: readlink d0/d1b/d4f/le5 0 2024-03-22T07:16:03.849 INFO:tasks.workunit.client.1.smithi069.stdout:6/468: symlink d4/dd/d12/l92 0 2024-03-22T07:16:03.849 INFO:tasks.workunit.client.1.smithi069.stdout:6/469: chown d4/d7/da/f1b 252499 1 2024-03-22T07:16:03.851 INFO:tasks.workunit.client.1.smithi069.stdout:8/679: getdents d1/d1c 0 2024-03-22T07:16:03.851 INFO:tasks.workunit.client.1.smithi069.stdout:8/680: fsync d1/d1c/d3e/f9c 0 2024-03-22T07:16:03.852 INFO:tasks.workunit.client.1.smithi069.stdout:8/681: symlink d1/d1c/d3e/d88/dcd/d6c/d8e/ldd 0 2024-03-22T07:16:03.853 INFO:tasks.workunit.client.1.smithi069.stdout:8/682: write d1/d1c/d48/d5e/f8b [815578,3648] 0 2024-03-22T07:16:03.859 INFO:tasks.workunit.client.1.smithi069.stdout:1/447: dwrite d1/d13/d36/d5b/f71 [0,4194304] 0 2024-03-22T07:16:03.859 INFO:tasks.workunit.client.1.smithi069.stdout:1/448: chown d1/d2a/f6c 3533 1 2024-03-22T07:16:03.859 INFO:tasks.workunit.client.1.smithi069.stdout:3/534: dwrite d5/da2/fa6 [0,4194304] 0 2024-03-22T07:16:03.861 INFO:tasks.workunit.client.0.smithi063.stdout:8/819: write d0/d1b/d28/d7b/f7f [1068266,11959] 0 2024-03-22T07:16:03.863 INFO:tasks.workunit.client.1.smithi069.stdout:3/535: symlink d5/d9/d23/laa 0 2024-03-22T07:16:03.864 INFO:tasks.workunit.client.0.smithi063.stdout:8/820: dread d0/d26/faf [0,4194304] 0 2024-03-22T07:16:03.865 INFO:tasks.workunit.client.0.smithi063.stdout:8/821: symlink d0/d26/d2d/d59/dc0/l118 0 2024-03-22T07:16:03.867 INFO:tasks.workunit.client.0.smithi063.stdout:8/822: link d0/d26/d2d/d59/dc0/l118 d0/d24/l119 0 2024-03-22T07:16:03.867 INFO:tasks.workunit.client.0.smithi063.stdout:8/823: write d0/d24/fb9 [4328242,38004] 0 2024-03-22T07:16:03.867 INFO:tasks.workunit.client.0.smithi063.stdout:8/824: read - d0/d1b/d28/d7b/d4a/f116 zero size 2024-03-22T07:16:03.869 INFO:tasks.workunit.client.0.smithi063.stdout:8/825: unlink d0/d1b/d28/d7b/l55 0 2024-03-22T07:16:03.871 INFO:tasks.workunit.client.0.smithi063.stdout:8/826: unlink d0/c9 0 2024-03-22T07:16:03.871 INFO:tasks.workunit.client.0.smithi063.stdout:8/827: chown d0/d26/d2d/d8f/ca9 1 1 2024-03-22T07:16:03.871 INFO:tasks.workunit.client.0.smithi063.stdout:8/828: write d0/d24/d33/d4c/f9c [636476,98919] 0 2024-03-22T07:16:03.872 INFO:tasks.workunit.client.1.smithi069.stdout:2/359: sync 2024-03-22T07:16:03.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:03 smithi063 ceph-mon[29200]: pgmap v4: 65 pgs: 65 active+clean; 2.4 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail 2024-03-22T07:16:03.873 INFO:tasks.workunit.client.1.smithi069.stdout:2/360: getdents d18/d4d/d27/d56 0 2024-03-22T07:16:03.874 INFO:tasks.workunit.client.1.smithi069.stdout:2/361: link d18/f2b d18/d4d/d34/f7e 0 2024-03-22T07:16:03.883 INFO:tasks.workunit.client.0.smithi063.stdout:8/829: write d0/d1b/d4f/dde/df5/f53 [1418150,37046] 0 2024-03-22T07:16:03.883 INFO:tasks.workunit.client.1.smithi069.stdout:6/470: dwrite d4/d7/da/f5a [0,4194304] 0 2024-03-22T07:16:03.885 INFO:tasks.workunit.client.1.smithi069.stdout:6/471: creat d4/dd/d41/f93 x:0 0 0 2024-03-22T07:16:03.885 INFO:tasks.workunit.client.1.smithi069.stdout:6/472: readlink d4/d7/da/l2a 0 2024-03-22T07:16:03.886 INFO:tasks.workunit.client.0.smithi063.stdout:8/830: unlink d0/d26/d2d/d83/cae 0 2024-03-22T07:16:03.886 INFO:tasks.workunit.client.0.smithi063.stdout:8/831: write d0/d26/d2d/d83/d112/f101 [4103937,74349] 0 2024-03-22T07:16:03.888 INFO:tasks.workunit.client.0.smithi063.stdout:8/832: symlink d0/d1b/d28/d7b/d45/l11a 0 2024-03-22T07:16:03.890 INFO:tasks.workunit.client.1.smithi069.stdout:2/362: dread d18/d4d/d3e/f65 [0,4194304] 0 2024-03-22T07:16:03.891 INFO:tasks.workunit.client.1.smithi069.stdout:2/363: mknod d18/d4d/d27/d33/d60/d63/d73/c7f 0 2024-03-22T07:16:03.892 INFO:tasks.workunit.client.1.smithi069.stdout:3/536: dwrite d5/d9/f4b [0,4194304] 0 2024-03-22T07:16:03.892 INFO:tasks.workunit.client.1.smithi069.stdout:3/537: write d5/d9/d26/f8b [652017,77394] 0 2024-03-22T07:16:03.892 INFO:tasks.workunit.client.1.smithi069.stdout:3/538: fdatasync d5/d9/d26/d3f/d56/d7f/f96 0 2024-03-22T07:16:03.893 INFO:tasks.workunit.client.1.smithi069.stdout:3/539: write d5/d9/f5e [596352,100828] 0 2024-03-22T07:16:03.893 INFO:tasks.workunit.client.1.smithi069.stdout:1/449: dwrite d1/f42 [0,4194304] 0 2024-03-22T07:16:03.896 INFO:tasks.workunit.client.1.smithi069.stdout:3/540: unlink d5/d9/d26/d36/d41/d51/l60 0 2024-03-22T07:16:03.896 INFO:tasks.workunit.client.1.smithi069.stdout:3/541: write d5/d9/db/f30 [418047,122448] 0 2024-03-22T07:16:03.899 INFO:tasks.workunit.client.1.smithi069.stdout:3/542: link d5/d9/f68 d5/d9/db/fab 0 2024-03-22T07:16:03.899 INFO:tasks.workunit.client.1.smithi069.stdout:3/543: creat d5/d9/d26/fac x:0 0 0 2024-03-22T07:16:03.900 INFO:tasks.workunit.client.0.smithi063.stdout:8/833: read d0/d1b/f1e [4157482,55436] 0 2024-03-22T07:16:03.900 INFO:tasks.workunit.client.0.smithi063.stdout:8/834: dread - d0/d1b/d28/d3e/db2/dbd/dd0/f10c zero size 2024-03-22T07:16:03.903 INFO:tasks.workunit.client.1.smithi069.stdout:3/544: mknod d5/cad 0 2024-03-22T07:16:03.904 INFO:tasks.workunit.client.1.smithi069.stdout:2/364: dread d18/d4d/d27/d72/f7d [0,4194304] 0 2024-03-22T07:16:03.904 INFO:tasks.workunit.client.1.smithi069.stdout:3/545: mkdir d5/d9/db/dae 0 2024-03-22T07:16:03.905 INFO:tasks.workunit.client.1.smithi069.stdout:2/365: mkdir d18/d4d/d27/d33/d60/d63/d80 0 2024-03-22T07:16:03.905 INFO:tasks.workunit.client.1.smithi069.stdout:2/366: truncate d18/d4d/f57 951553 0 2024-03-22T07:16:03.906 INFO:tasks.workunit.client.1.smithi069.stdout:2/367: read d18/f70 [173926,30571] 0 2024-03-22T07:16:03.906 INFO:tasks.workunit.client.1.smithi069.stdout:3/546: creat d5/d9/db/dae/faf x:0 0 0 2024-03-22T07:16:03.907 INFO:tasks.workunit.client.1.smithi069.stdout:2/368: symlink d18/d4d/d27/d33/d60/d64/l81 0 2024-03-22T07:16:03.907 INFO:tasks.workunit.client.1.smithi069.stdout:2/369: write d18/f2b [2661566,36483] 0 2024-03-22T07:16:03.908 INFO:tasks.workunit.client.1.smithi069.stdout:3/547: rename d5/d9/d26/d3f/d35/f7a to d5/d9/d26/d3f/d54/d9d/fb0 0 2024-03-22T07:16:03.911 INFO:tasks.workunit.client.1.smithi069.stdout:2/370: creat d18/d4d/d27/d33/f82 x:0 0 0 2024-03-22T07:16:03.912 INFO:tasks.workunit.client.1.smithi069.stdout:3/548: dread d5/d9/d26/d3f/d35/f37 [0,4194304] 0 2024-03-22T07:16:03.914 INFO:tasks.workunit.client.1.smithi069.stdout:3/549: getdents d5/d9/d26/d3f/d56/d8a/da5 0 2024-03-22T07:16:03.914 INFO:tasks.workunit.client.1.smithi069.stdout:6/473: dwrite d4/dd/d12/f32 [0,4194304] 0 2024-03-22T07:16:03.914 INFO:tasks.workunit.client.1.smithi069.stdout:1/450: dwrite d1/d25/f75 [0,4194304] 0 2024-03-22T07:16:03.915 INFO:tasks.workunit.client.1.smithi069.stdout:1/451: dread d1/f3b [0,4194304] 0 2024-03-22T07:16:03.916 INFO:tasks.workunit.client.1.smithi069.stdout:6/474: getdents d4/dd/d12/d44/d7f 0 2024-03-22T07:16:03.916 INFO:tasks.workunit.client.1.smithi069.stdout:6/475: chown d4/d7/da/d3f/l87 191459499 1 2024-03-22T07:16:03.916 INFO:tasks.workunit.client.1.smithi069.stdout:6/476: write d4/dd/d12/d44/d58/f62 [3204105,109513] 0 2024-03-22T07:16:03.916 INFO:tasks.workunit.client.1.smithi069.stdout:6/477: creat d4/d7/da/f94 x:0 0 0 2024-03-22T07:16:03.917 INFO:tasks.workunit.client.1.smithi069.stdout:3/550: symlink d5/da2/lb1 0 2024-03-22T07:16:03.919 INFO:tasks.workunit.client.1.smithi069.stdout:6/478: link d4/dd/d41/d78/f85 d4/dd/d12/d2b/d72/f95 0 2024-03-22T07:16:03.920 INFO:tasks.workunit.client.1.smithi069.stdout:3/551: readlink d5/l4d 0 2024-03-22T07:16:03.921 INFO:tasks.workunit.client.1.smithi069.stdout:1/452: rename d1/d25 to d1/d2a/d50/d88 0 2024-03-22T07:16:03.922 INFO:tasks.workunit.client.0.smithi063.stdout:8/835: dread d0/d24/f29 [4194304,4194304] 0 2024-03-22T07:16:03.924 INFO:tasks.workunit.client.0.smithi063.stdout:8/836: mknod d0/d1b/d28/d3e/c11b 0 2024-03-22T07:16:03.924 INFO:tasks.workunit.client.0.smithi063.stdout:8/837: chown d0/d26/d2d/d59/c61 0 1 2024-03-22T07:16:03.924 INFO:tasks.workunit.client.0.smithi063.stdout:8/838: fsync d0/d1b/fc3 0 2024-03-22T07:16:03.925 INFO:tasks.workunit.client.0.smithi063.stdout:8/839: mkdir d0/d26/d2d/d83/d11c 0 2024-03-22T07:16:03.925 INFO:tasks.workunit.client.0.smithi063.stdout:8/840: write d0/d24/fb9 [4586100,36230] 0 2024-03-22T07:16:03.925 INFO:tasks.workunit.client.1.smithi069.stdout:2/371: dread d18/d4d/f28 [0,4194304] 0 2024-03-22T07:16:03.925 INFO:tasks.workunit.client.1.smithi069.stdout:2/372: fsync d18/f1b 0 2024-03-22T07:16:03.925 INFO:tasks.workunit.client.1.smithi069.stdout:2/373: chown d18/d4d/f41 127739156 1 2024-03-22T07:16:03.928 INFO:tasks.workunit.client.0.smithi063.stdout:8/841: mkdir d0/d24/d33/d4c/d78/d11d 0 2024-03-22T07:16:03.928 INFO:tasks.workunit.client.0.smithi063.stdout:8/842: read - d0/d26/d2d/fb1 zero size 2024-03-22T07:16:03.928 INFO:tasks.workunit.client.0.smithi063.stdout:8/843: dread d0/d24/d33/fee [0,4194304] 0 2024-03-22T07:16:03.930 INFO:tasks.workunit.client.1.smithi069.stdout:3/552: rename d5/c2c to d5/d9/cb2 0 2024-03-22T07:16:03.932 INFO:tasks.workunit.client.1.smithi069.stdout:1/453: getdents d1/d13/d36 0 2024-03-22T07:16:03.933 INFO:tasks.workunit.client.1.smithi069.stdout:2/374: rmdir d18/d4d/d27/d33/d4f 39 2024-03-22T07:16:03.940 INFO:tasks.workunit.client.1.smithi069.stdout:3/553: write d5/d9/d26/d36/f52 [138618,94020] 0 2024-03-22T07:16:03.943 INFO:tasks.workunit.client.1.smithi069.stdout:1/454: rename d1/le to d1/d13/l89 0 2024-03-22T07:16:03.943 INFO:tasks.workunit.client.1.smithi069.stdout:1/455: stat d1/l21 0 2024-03-22T07:16:03.944 INFO:tasks.workunit.client.1.smithi069.stdout:2/375: write d18/d4d/d27/f49 [1940986,125331] 0 2024-03-22T07:16:03.944 INFO:tasks.workunit.client.1.smithi069.stdout:2/376: write d18/d4d/d25/f47 [4390859,60807] 0 2024-03-22T07:16:03.945 INFO:tasks.workunit.client.0.smithi063.stdout:8/844: dwrite d0/d24/d33/fee [0,4194304] 0 2024-03-22T07:16:03.948 INFO:tasks.workunit.client.0.smithi063.stdout:5/647: sync 2024-03-22T07:16:03.948 INFO:tasks.workunit.client.0.smithi063.stdout:5/648: fdatasync d8/d13/f48 0 2024-03-22T07:16:03.948 INFO:tasks.workunit.client.1.smithi069.stdout:6/479: rmdir d4/dd/d41 39 2024-03-22T07:16:03.948 INFO:tasks.workunit.client.1.smithi069.stdout:6/480: chown d4/d7/da/f1b 163 1 2024-03-22T07:16:03.950 INFO:tasks.workunit.client.0.smithi063.stdout:5/649: rmdir d8/d13/d24/da4 39 2024-03-22T07:16:03.950 INFO:tasks.workunit.client.1.smithi069.stdout:3/554: mknod d5/d9/d26/d3f/d56/d7f/d90/cb3 0 2024-03-22T07:16:03.950 INFO:tasks.workunit.client.1.smithi069.stdout:3/555: write d5/d9/d26/f5a [1202488,65544] 0 2024-03-22T07:16:03.951 INFO:tasks.workunit.client.0.smithi063.stdout:5/650: mknod d8/d13/d1e/d3c/cc3 0 2024-03-22T07:16:03.952 INFO:tasks.workunit.client.0.smithi063.stdout:5/651: symlink d8/d13/d1e/d31/lc4 0 2024-03-22T07:16:03.953 INFO:tasks.workunit.client.1.smithi069.stdout:2/377: dread d18/f19 [0,4194304] 0 2024-03-22T07:16:03.954 INFO:tasks.workunit.client.0.smithi063.stdout:5/652: rmdir d8/d13/d1e/d31/d58 39 2024-03-22T07:16:03.956 INFO:tasks.workunit.client.1.smithi069.stdout:1/456: rename d1/l21 to d1/d2a/d50/d88/d41/d7a/l8a 0 2024-03-22T07:16:03.956 INFO:tasks.workunit.client.1.smithi069.stdout:1/457: chown d1/d2a/d50/d88/c30 439401 1 2024-03-22T07:16:03.956 INFO:tasks.workunit.client.1.smithi069.stdout:1/458: fsync d1/d2a/d50/d88/d41/d64/f85 0 2024-03-22T07:16:03.958 INFO:tasks.workunit.client.1.smithi069.stdout:2/378: symlink d18/d4d/d27/l83 0 2024-03-22T07:16:03.958 INFO:tasks.workunit.client.1.smithi069.stdout:2/379: write d18/d4d/d27/d33/f6b [470513,64382] 0 2024-03-22T07:16:03.959 INFO:tasks.workunit.client.1.smithi069.stdout:1/459: dread d1/d13/f58 [0,4194304] 0 2024-03-22T07:16:03.959 INFO:tasks.workunit.client.1.smithi069.stdout:1/460: fsync d1/d2a/d50/d88/d41/d1e/f72 0 2024-03-22T07:16:03.960 INFO:tasks.workunit.client.1.smithi069.stdout:6/481: rename d4/dd/d12/d13/c20 to d4/dd/d12/d2b/d72/d73/d74/c96 0 2024-03-22T07:16:03.961 INFO:tasks.workunit.client.1.smithi069.stdout:2/380: symlink d18/d4d/l84 0 2024-03-22T07:16:03.964 INFO:tasks.workunit.client.1.smithi069.stdout:1/461: unlink d1/c3c 0 2024-03-22T07:16:03.964 INFO:tasks.workunit.client.1.smithi069.stdout:1/462: readlink d1/d2a/d50/l44 0 2024-03-22T07:16:03.965 INFO:tasks.workunit.client.1.smithi069.stdout:1/463: dread d1/d13/d82/f83 [0,4194304] 0 2024-03-22T07:16:03.966 INFO:tasks.workunit.client.1.smithi069.stdout:2/381: link d18/d4d/d27/c29 d18/d4d/d3e/d40/c85 0 2024-03-22T07:16:03.969 INFO:tasks.workunit.client.1.smithi069.stdout:1/464: truncate d1/d2a/d50/f40 3591147 0 2024-03-22T07:16:03.969 INFO:tasks.workunit.client.1.smithi069.stdout:1/465: dread - d1/d2a/d50/d88/d41/d7a/f7f zero size 2024-03-22T07:16:03.971 INFO:tasks.workunit.client.1.smithi069.stdout:2/382: dread d18/f1b [0,4194304] 0 2024-03-22T07:16:03.971 INFO:tasks.workunit.client.1.smithi069.stdout:2/383: stat d18 0 2024-03-22T07:16:03.972 INFO:tasks.workunit.client.1.smithi069.stdout:1/466: creat d1/d2a/d84/f8b x:0 0 0 2024-03-22T07:16:03.974 INFO:tasks.workunit.client.1.smithi069.stdout:2/384: creat d18/d4d/d3e/f86 x:0 0 0 2024-03-22T07:16:03.979 INFO:tasks.workunit.client.0.smithi063.stdout:8/845: dwrite d0/d1b/fc8 [0,4194304] 0 2024-03-22T07:16:03.986 INFO:tasks.workunit.client.1.smithi069.stdout:3/556: dwrite d5/d9/f48 [4194304,4194304] 0 2024-03-22T07:16:03.986 INFO:tasks.workunit.client.1.smithi069.stdout:3/557: getdents d5/d9/d26/d3f/d56/d8a/da5 0 2024-03-22T07:16:03.986 INFO:tasks.workunit.client.1.smithi069.stdout:6/482: dwrite d4/dd/d12/d13/f23 [0,4194304] 0 2024-03-22T07:16:03.987 INFO:tasks.workunit.client.1.smithi069.stdout:1/467: dread d1/d13/f39 [0,4194304] 0 2024-03-22T07:16:03.987 INFO:tasks.workunit.client.1.smithi069.stdout:1/468: creat d1/d2a/f8c x:0 0 0 2024-03-22T07:16:03.987 INFO:tasks.workunit.client.1.smithi069.stdout:1/469: chown d1/d13/c15 2052414 1 2024-03-22T07:16:03.991 INFO:tasks.workunit.client.0.smithi063.stdout:8/846: write d0/d24/d33/d4c/fd4 [711584,122978] 0 2024-03-22T07:16:03.991 INFO:tasks.workunit.client.1.smithi069.stdout:2/385: dread fa [0,4194304] 0 2024-03-22T07:16:03.993 INFO:tasks.workunit.client.0.smithi063.stdout:8/847: mkdir d0/d26/d2d/d59/dba/dc9/ded/d11e 0 2024-03-22T07:16:03.993 INFO:tasks.workunit.client.0.smithi063.stdout:8/848: stat d0/d24/d5b/ce1 0 2024-03-22T07:16:03.993 INFO:tasks.workunit.client.0.smithi063.stdout:8/849: read - d0/d1b/d28/d3e/db2/dbd/dd0/f10c zero size 2024-03-22T07:16:03.993 INFO:tasks.workunit.client.0.smithi063.stdout:8/850: chown d0/d26/d2d/d83/d112/l10a 10820 1 2024-03-22T07:16:03.994 INFO:tasks.workunit.client.0.smithi063.stdout:8/851: readlink d0/d26/l6c 0 2024-03-22T07:16:03.994 INFO:tasks.workunit.client.1.smithi069.stdout:6/483: dread d4/dd/f8f [0,4194304] 0 2024-03-22T07:16:03.994 INFO:tasks.workunit.client.1.smithi069.stdout:6/484: chown d4/dd/d12/c1c 3225 1 2024-03-22T07:16:03.994 INFO:tasks.workunit.client.1.smithi069.stdout:6/485: readlink d4/dd/d12/l2f 0 2024-03-22T07:16:03.994 INFO:tasks.workunit.client.1.smithi069.stdout:6/486: creat d4/d7/d5b/f97 x:0 0 0 2024-03-22T07:16:03.995 INFO:tasks.workunit.client.0.smithi063.stdout:8/852: mknod d0/d1b/c11f 0 2024-03-22T07:16:03.995 INFO:tasks.workunit.client.1.smithi069.stdout:3/558: dread d5/d9/d26/d3f/d35/f58 [0,4194304] 0 2024-03-22T07:16:03.997 INFO:tasks.workunit.client.1.smithi069.stdout:1/470: dread d1/d2a/d50/d88/d41/d7a/f70 [0,4194304] 0 2024-03-22T07:16:03.998 INFO:tasks.workunit.client.0.smithi063.stdout:8/853: rmdir d0/d26/d2d/d83 39 2024-03-22T07:16:03.998 INFO:tasks.workunit.client.0.smithi063.stdout:8/854: creat d0/d1b/d28/d3e/db2/dbd/dd0/f120 x:0 0 0 2024-03-22T07:16:03.998 INFO:tasks.workunit.client.0.smithi063.stdout:8/855: chown d0/d26/d2d/d83/d112 579415925 1 2024-03-22T07:16:03.998 INFO:tasks.workunit.client.0.smithi063.stdout:8/856: fsync d0/d26/d2d/d59/dba/dc9/ded/ff7 0 2024-03-22T07:16:04.000 INFO:tasks.workunit.client.1.smithi069.stdout:2/386: unlink d18/d4d/d34/f3f 0 2024-03-22T07:16:04.004 INFO:tasks.workunit.client.1.smithi069.stdout:1/471: rename d1/d2a/f6c to d1/d13/d36/f8d 0 2024-03-22T07:16:04.004 INFO:tasks.workunit.client.1.smithi069.stdout:1/472: readlink d1/d2a/d50/d88/d41/d1e/l60 0 2024-03-22T07:16:04.006 INFO:tasks.workunit.client.1.smithi069.stdout:2/387: rename d18/l1f to d18/d4d/d27/d33/d4f/l87 0 2024-03-22T07:16:04.008 INFO:tasks.workunit.client.1.smithi069.stdout:1/473: rename d1/d2a/d50/d4c/f7d to d1/d13/d36/d5b/f8e 0 2024-03-22T07:16:04.008 INFO:tasks.workunit.client.1.smithi069.stdout:1/474: fdatasync d1/d2a/d84/f8b 0 2024-03-22T07:16:04.022 INFO:tasks.workunit.client.1.smithi069.stdout:1/475: dread d1/d13/d36/d5b/f71 [0,4194304] 0 2024-03-22T07:16:04.023 INFO:tasks.workunit.client.0.smithi063.stdout:8/857: dwrite d0/d26/d2d/d59/dc0/fdf [0,4194304] 0 2024-03-22T07:16:04.026 INFO:tasks.workunit.client.0.smithi063.stdout:8/858: mkdir d0/d26/d2d/d83/d11c/d121 0 2024-03-22T07:16:04.026 INFO:tasks.workunit.client.0.smithi063.stdout:8/859: truncate d0/d24/f74 1078810 0 2024-03-22T07:16:04.026 INFO:tasks.workunit.client.0.smithi063.stdout:8/860: readlink d0/d1b/d28/d7b/d4a/l9d 0 2024-03-22T07:16:04.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:03 smithi069 ceph-mon[36441]: pgmap v4: 65 pgs: 65 active+clean; 2.4 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail 2024-03-22T07:16:04.063 INFO:tasks.workunit.client.1.smithi069.stdout:5/397: sync 2024-03-22T07:16:04.063 INFO:tasks.workunit.client.1.smithi069.stdout:5/398: chown d1/d7/d3d/c53 329149777 1 2024-03-22T07:16:04.065 INFO:tasks.workunit.client.1.smithi069.stdout:5/399: link d1/d7/d8/c1a d1/d6/d26/d7d/c8c 0 2024-03-22T07:16:04.071 INFO:tasks.workunit.client.1.smithi069.stdout:5/400: dread d1/d7/f22 [0,4194304] 0 2024-03-22T07:16:04.073 INFO:tasks.workunit.client.1.smithi069.stdout:5/401: getdents d1/d6 0 2024-03-22T07:16:04.073 INFO:tasks.workunit.client.1.smithi069.stdout:5/402: creat d1/d6/d26/d3e/f8d x:0 0 0 2024-03-22T07:16:04.074 INFO:tasks.workunit.client.1.smithi069.stdout:5/403: rmdir d1/d7 39 2024-03-22T07:16:04.074 INFO:tasks.workunit.client.1.smithi069.stdout:5/404: read d1/d7/d8/f52 [643456,33464] 0 2024-03-22T07:16:04.074 INFO:tasks.workunit.client.1.smithi069.stdout:5/405: truncate d1/d7/d8/f58 3698333 0 2024-03-22T07:16:04.153 INFO:tasks.workunit.client.0.smithi063.stdout:0/943: sync 2024-03-22T07:16:04.153 INFO:tasks.workunit.client.0.smithi063.stdout:0/944: fsync da/de/d17/d25/d3c/d5f/fe8 0 2024-03-22T07:16:04.154 INFO:tasks.workunit.client.0.smithi063.stdout:0/945: mknod da/de/d60/d56/d77/c134 0 2024-03-22T07:16:04.155 INFO:tasks.workunit.client.0.smithi063.stdout:0/946: mknod da/de/d60/d56/d8b/c135 0 2024-03-22T07:16:04.156 INFO:tasks.workunit.client.0.smithi063.stdout:0/947: rmdir da/de/d17/d25/d3c/d64/d78 39 2024-03-22T07:16:04.156 INFO:tasks.workunit.client.0.smithi063.stdout:0/948: write da/de/d60/d9b/fa5 [1919168,42877] 0 2024-03-22T07:16:04.157 INFO:tasks.workunit.client.0.smithi063.stdout:0/949: write da/de/d60/d56/d77/fa3 [4659937,102444] 0 2024-03-22T07:16:04.157 INFO:tasks.workunit.client.0.smithi063.stdout:0/950: readlink da/de/d17/d25/l48 0 2024-03-22T07:16:04.159 INFO:tasks.workunit.client.0.smithi063.stdout:0/951: symlink da/de/d17/d25/d99/l136 0 2024-03-22T07:16:04.161 INFO:tasks.workunit.client.0.smithi063.stdout:0/952: link da/de/d17/d25/d3c/d64/df5/f105 da/de/d60/d56/d77/f137 0 2024-03-22T07:16:04.161 INFO:tasks.workunit.client.0.smithi063.stdout:0/953: chown da/de/d17/d25/d3c/d53/d89/df1/f11a 89199 1 2024-03-22T07:16:04.162 INFO:tasks.workunit.client.0.smithi063.stdout:0/954: getdents da/de/d17/d25/d3c/d43/dc9 0 2024-03-22T07:16:04.162 INFO:tasks.workunit.client.0.smithi063.stdout:0/955: readlink da/de/d17/d25/d3c/d43/d76/lc8 0 2024-03-22T07:16:04.164 INFO:tasks.workunit.client.0.smithi063.stdout:0/956: creat da/de/d60/d56/f138 x:0 0 0 2024-03-22T07:16:04.164 INFO:tasks.workunit.client.0.smithi063.stdout:0/957: write da/de/d17/d25/d3c/d64/d110/f117 [454746,58966] 0 2024-03-22T07:16:04.164 INFO:tasks.workunit.client.0.smithi063.stdout:0/958: readlink da/de/d17/l59 0 2024-03-22T07:16:04.164 INFO:tasks.workunit.client.0.smithi063.stdout:0/959: chown da/de/ddd 45985 1 2024-03-22T07:16:04.164 INFO:tasks.workunit.client.0.smithi063.stdout:0/960: chown da/de/d60/cb7 2 1 2024-03-22T07:16:04.165 INFO:tasks.workunit.client.1.smithi069.stdout:6/487: dwrite d4/f77 [0,4194304] 0 2024-03-22T07:16:04.181 INFO:tasks.workunit.client.1.smithi069.stdout:3/559: dwrite d5/d9/d26/d3f/d56/d8a/da5/f99 [0,4194304] 0 2024-03-22T07:16:04.181 INFO:tasks.workunit.client.1.smithi069.stdout:1/476: dwrite d1/f48 [0,4194304] 0 2024-03-22T07:16:04.182 INFO:tasks.workunit.client.0.smithi063.stdout:8/861: dwrite d0/d24/f29 [4194304,4194304] 0 2024-03-22T07:16:04.182 INFO:tasks.workunit.client.0.smithi063.stdout:8/862: fdatasync d0/d26/faf 0 2024-03-22T07:16:04.186 INFO:tasks.workunit.client.0.smithi063.stdout:8/863: mkdir d0/d24/d33/d122 0 2024-03-22T07:16:04.186 INFO:tasks.workunit.client.0.smithi063.stdout:8/864: write d0/d1b/d4f/dde/df5/f40 [2065928,10614] 0 2024-03-22T07:16:04.186 INFO:tasks.workunit.client.0.smithi063.stdout:8/865: write d0/d1b/f1e [3884153,13510] 0 2024-03-22T07:16:04.189 INFO:tasks.workunit.client.0.smithi063.stdout:8/866: link d0/c88 d0/d1b/d28/d3e/db2/dbd/dfd/c123 0 2024-03-22T07:16:04.192 INFO:tasks.workunit.client.0.smithi063.stdout:0/961: dwrite da/de/d17/d25/d3c/d43/d80/f86 [0,4194304] 0 2024-03-22T07:16:04.193 INFO:tasks.workunit.client.1.smithi069.stdout:6/488: dwrite d4/dd/d12/d13/f86 [0,4194304] 0 2024-03-22T07:16:04.195 INFO:tasks.workunit.client.0.smithi063.stdout:0/962: mknod da/d49/d121/c139 0 2024-03-22T07:16:04.203 INFO:tasks.workunit.client.1.smithi069.stdout:3/560: dread d5/d9/db/f65 [0,4194304] 0 2024-03-22T07:16:04.203 INFO:tasks.workunit.client.1.smithi069.stdout:3/561: fsync d5/d9/d23/f6d 0 2024-03-22T07:16:04.203 INFO:tasks.workunit.client.1.smithi069.stdout:1/477: write d1/d13/d36/f3e [2894480,110614] 0 2024-03-22T07:16:04.203 INFO:tasks.workunit.client.1.smithi069.stdout:1/478: fsync d1/d13/d82/f83 0 2024-03-22T07:16:04.204 INFO:tasks.workunit.client.1.smithi069.stdout:3/562: mkdir d5/d9/d26/db4 0 2024-03-22T07:16:04.204 INFO:tasks.workunit.client.1.smithi069.stdout:3/563: truncate d5/d9/db/dae/faf 207266 0 2024-03-22T07:16:04.204 INFO:tasks.workunit.client.1.smithi069.stdout:5/406: dwrite d1/d7/d8/f58 [0,4194304] 0 2024-03-22T07:16:04.204 INFO:tasks.workunit.client.1.smithi069.stdout:3/564: mkdir d5/da2/db5 0 2024-03-22T07:16:04.204 INFO:tasks.workunit.client.1.smithi069.stdout:3/565: creat d5/d9/d26/d3f/d35/fb6 x:0 0 0 2024-03-22T07:16:04.204 INFO:tasks.workunit.client.1.smithi069.stdout:5/407: mknod d1/d6/d26/d3a/c8e 0 2024-03-22T07:16:04.204 INFO:tasks.workunit.client.1.smithi069.stdout:5/408: readlink d1/d6/d26/l51 0 2024-03-22T07:16:04.205 INFO:tasks.workunit.client.0.smithi063.stdout:0/963: dread da/de/d17/d4e/f9e [0,4194304] 0 2024-03-22T07:16:04.205 INFO:tasks.workunit.client.0.smithi063.stdout:0/964: fsync da/de/d17/d25/d3c/d64/fb2 0 2024-03-22T07:16:04.205 INFO:tasks.workunit.client.0.smithi063.stdout:0/965: write da/de/d17/d25/d3c/d53/d89/fc3 [3194307,27245] 0 2024-03-22T07:16:04.205 INFO:tasks.workunit.client.0.smithi063.stdout:0/966: readlink da/ld0 0 2024-03-22T07:16:04.206 INFO:tasks.workunit.client.0.smithi063.stdout:0/967: rmdir da/de/d17/d25/d3c/d43/d76 39 2024-03-22T07:16:04.207 INFO:tasks.workunit.client.1.smithi069.stdout:6/489: dread d4/dd/d12/d13/f23 [0,4194304] 0 2024-03-22T07:16:04.207 INFO:tasks.workunit.client.1.smithi069.stdout:6/490: chown d4/dd/d41/d78/f85 1925202467 1 2024-03-22T07:16:04.207 INFO:tasks.workunit.client.1.smithi069.stdout:6/491: chown d4/dd/c3d 0 1 2024-03-22T07:16:04.207 INFO:tasks.workunit.client.1.smithi069.stdout:3/566: rmdir d5/da2/db5 0 2024-03-22T07:16:04.208 INFO:tasks.workunit.client.1.smithi069.stdout:5/409: getdents d1 0 2024-03-22T07:16:04.208 INFO:tasks.workunit.client.1.smithi069.stdout:3/567: write d5/d9/db/f63 [1431957,28892] 0 2024-03-22T07:16:04.208 INFO:tasks.workunit.client.1.smithi069.stdout:5/410: write d1/d10/d60/f83 [47750,74187] 0 2024-03-22T07:16:04.209 INFO:tasks.workunit.client.0.smithi063.stdout:0/968: rename da/de/d17/d25/f27 to da/de/d17/d25/d3c/d53/d89/df1/f13a 0 2024-03-22T07:16:04.210 INFO:tasks.workunit.client.1.smithi069.stdout:6/492: symlink d4/d7/d5b/l98 0 2024-03-22T07:16:04.210 INFO:tasks.workunit.client.0.smithi063.stdout:4/724: sync 2024-03-22T07:16:04.210 INFO:tasks.workunit.client.0.smithi063.stdout:6/764: sync 2024-03-22T07:16:04.210 INFO:tasks.workunit.client.0.smithi063.stdout:7/646: sync 2024-03-22T07:16:04.210 INFO:tasks.workunit.client.0.smithi063.stdout:7/647: write d5/da/d28/d4b/da8/dbc/f70 [871765,49516] 0 2024-03-22T07:16:04.210 INFO:tasks.workunit.client.0.smithi063.stdout:2/704: sync 2024-03-22T07:16:04.210 INFO:tasks.workunit.client.0.smithi063.stdout:5/653: sync 2024-03-22T07:16:04.211 INFO:tasks.workunit.client.0.smithi063.stdout:9/740: sync 2024-03-22T07:16:04.211 INFO:tasks.workunit.client.0.smithi063.stdout:1/894: sync 2024-03-22T07:16:04.211 INFO:tasks.workunit.client.0.smithi063.stdout:3/845: sync 2024-03-22T07:16:04.212 INFO:tasks.workunit.client.1.smithi069.stdout:3/568: mknod d5/d9/cb7 0 2024-03-22T07:16:04.214 INFO:tasks.workunit.client.0.smithi063.stdout:6/765: symlink d3/d15/d16/d57/dde/deb/l104 0 2024-03-22T07:16:04.214 INFO:tasks.workunit.client.0.smithi063.stdout:6/766: write d3/d6/d2a/d38/f49 [4694984,27158] 0 2024-03-22T07:16:04.216 INFO:tasks.workunit.client.1.smithi069.stdout:9/470: sync 2024-03-22T07:16:04.216 INFO:tasks.workunit.client.1.smithi069.stdout:6/493: link d4/dd/d41/f93 d4/d7/da/d3f/f99 0 2024-03-22T07:16:04.216 INFO:tasks.workunit.client.0.smithi063.stdout:7/648: mkdir d5/da/d28/d4b/da8/dd4 0 2024-03-22T07:16:04.216 INFO:tasks.workunit.client.0.smithi063.stdout:7/649: dread - d5/da/d28/d4b/da8/dbc/d85/fbf zero size 2024-03-22T07:16:04.216 INFO:tasks.workunit.client.0.smithi063.stdout:7/650: chown d5/da/f35 46483 1 2024-03-22T07:16:04.222 INFO:tasks.workunit.client.1.smithi069.stdout:0/632: sync 2024-03-22T07:16:04.222 INFO:tasks.workunit.client.1.smithi069.stdout:4/601: sync 2024-03-22T07:16:04.222 INFO:tasks.workunit.client.1.smithi069.stdout:4/602: write d6/dc/f1a [155342,37221] 0 2024-03-22T07:16:04.222 INFO:tasks.workunit.client.0.smithi063.stdout:5/654: creat d8/d13/d1e/d31/d64/d65/d6a/fc5 x:0 0 0 2024-03-22T07:16:04.222 INFO:tasks.workunit.client.0.smithi063.stdout:5/655: fdatasync d8/f44 0 2024-03-22T07:16:04.226 INFO:tasks.workunit.client.1.smithi069.stdout:9/471: dread d2/d31/f40 [4194304,4194304] 0 2024-03-22T07:16:04.227 INFO:tasks.workunit.client.1.smithi069.stdout:8/683: sync 2024-03-22T07:16:04.227 INFO:tasks.workunit.client.1.smithi069.stdout:7/505: sync 2024-03-22T07:16:04.227 INFO:tasks.workunit.client.1.smithi069.stdout:8/684: fdatasync d1/d1c/d3e/d66/f78 0 2024-03-22T07:16:04.228 INFO:tasks.workunit.client.0.smithi063.stdout:4/725: write d1/d5/d3c/f47 [5382777,51807] 0 2024-03-22T07:16:04.228 INFO:tasks.workunit.client.0.smithi063.stdout:4/726: write d1/d5/f28 [849857,374] 0 2024-03-22T07:16:04.228 INFO:tasks.workunit.client.0.smithi063.stdout:4/727: readlink d1/d5/dc/d95/l61 0 2024-03-22T07:16:04.228 INFO:tasks.workunit.client.0.smithi063.stdout:4/728: fdatasync d1/d5/d43/d27/fcc 0 2024-03-22T07:16:04.228 INFO:tasks.workunit.client.0.smithi063.stdout:4/729: chown d1/d5/dc/d95/d59/d9c/lc8 89258 1 2024-03-22T07:16:04.228 INFO:tasks.workunit.client.0.smithi063.stdout:1/895: creat d4/d71/f133 x:0 0 0 2024-03-22T07:16:04.229 INFO:tasks.workunit.client.1.smithi069.stdout:2/388: sync 2024-03-22T07:16:04.229 INFO:tasks.workunit.client.1.smithi069.stdout:0/633: mkdir d3/d7/d15/d29/dcf 0 2024-03-22T07:16:04.229 INFO:tasks.workunit.client.1.smithi069.stdout:4/603: symlink d6/d16/d83/lc8 0 2024-03-22T07:16:04.230 INFO:tasks.workunit.client.1.smithi069.stdout:9/472: symlink d2/dd/d3c/d48/l99 0 2024-03-22T07:16:04.230 INFO:tasks.workunit.client.1.smithi069.stdout:9/473: dread - d2/f8f zero size 2024-03-22T07:16:04.230 INFO:tasks.workunit.client.0.smithi063.stdout:2/705: dread d0/de/d12/d20/f70 [0,4194304] 0 2024-03-22T07:16:04.230 INFO:tasks.workunit.client.0.smithi063.stdout:3/846: rename da/d16/d3f/f44 to da/d16/d89/d8c/dc1/dfe/f112 0 2024-03-22T07:16:04.230 INFO:tasks.workunit.client.0.smithi063.stdout:2/706: chown d0/de/d12/d20 52224 1 2024-03-22T07:16:04.230 INFO:tasks.workunit.client.0.smithi063.stdout:2/707: chown d0/de/d1b/d32/dac/fab 0 1 2024-03-22T07:16:04.230 INFO:tasks.workunit.client.0.smithi063.stdout:0/969: getdents da/de/d17/d4e 0 2024-03-22T07:16:04.230 INFO:tasks.workunit.client.0.smithi063.stdout:0/970: truncate da/de/d17/d25/d95/d98/f12e 653169 0 2024-03-22T07:16:04.230 INFO:tasks.workunit.client.0.smithi063.stdout:0/971: fdatasync da/de/d17/d25/d3c/d5f/d97/fa4 0 2024-03-22T07:16:04.231 INFO:tasks.workunit.client.0.smithi063.stdout:6/767: mknod d3/d15/d16/d60/d6d/d4c/c105 0 2024-03-22T07:16:04.231 INFO:tasks.workunit.client.1.smithi069.stdout:8/685: mkdir d1/d1c/d3e/dde 0 2024-03-22T07:16:04.231 INFO:tasks.workunit.client.1.smithi069.stdout:8/686: fsync d1/d1c/d3e/d88/da8/dbb/fd8 0 2024-03-22T07:16:04.232 INFO:tasks.workunit.client.0.smithi063.stdout:7/651: creat d5/da/d4e/fd5 x:0 0 0 2024-03-22T07:16:04.232 INFO:tasks.workunit.client.1.smithi069.stdout:0/634: truncate d3/d7/d15/d1f/f4e 151177 0 2024-03-22T07:16:04.235 INFO:tasks.workunit.client.0.smithi063.stdout:9/741: getdents d8/d19/d20/d22 0 2024-03-22T07:16:04.235 INFO:tasks.workunit.client.0.smithi063.stdout:9/742: fdatasync d8/fa 0 2024-03-22T07:16:04.235 INFO:tasks.workunit.client.1.smithi069.stdout:4/604: mknod d6/daa/cc9 0 2024-03-22T07:16:04.235 INFO:tasks.workunit.client.1.smithi069.stdout:4/605: dread - d6/d9b/d1f/d58/d7c/d90/fa5 zero size 2024-03-22T07:16:04.236 INFO:tasks.workunit.client.1.smithi069.stdout:4/606: creat d6/dc/d23/d59/fca x:0 0 0 2024-03-22T07:16:04.237 INFO:tasks.workunit.client.0.smithi063.stdout:4/730: mkdir d1/d5/d43/d27/d2c/df3/df7 0 2024-03-22T07:16:04.237 INFO:tasks.workunit.client.0.smithi063.stdout:4/731: stat d1/d5/dc/cca 0 2024-03-22T07:16:04.237 INFO:tasks.workunit.client.0.smithi063.stdout:4/732: write d1/d5/dc/d95/d59/d9c/dac/fe6 [626173,11150] 0 2024-03-22T07:16:04.237 INFO:tasks.workunit.client.0.smithi063.stdout:4/733: write d1/d5/dc/d95/d59/fa9 [4844656,10894] 0 2024-03-22T07:16:04.237 INFO:tasks.workunit.client.0.smithi063.stdout:4/734: dread - d1/d5/d85/ff2 zero size 2024-03-22T07:16:04.238 INFO:tasks.workunit.client.1.smithi069.stdout:0/635: creat d3/d7/d15/dad/fd0 x:0 0 0 2024-03-22T07:16:04.238 INFO:tasks.workunit.client.1.smithi069.stdout:0/636: write d3/d7/f49 [4826313,120411] 0 2024-03-22T07:16:04.238 INFO:tasks.workunit.client.1.smithi069.stdout:0/637: chown d3/d7/f49 11 1 2024-03-22T07:16:04.239 INFO:tasks.workunit.client.1.smithi069.stdout:0/638: stat d3/d7 0 2024-03-22T07:16:04.239 INFO:tasks.workunit.client.1.smithi069.stdout:2/389: dread d18/d4d/d27/d33/d60/f5a [0,4194304] 0 2024-03-22T07:16:04.239 INFO:tasks.workunit.client.1.smithi069.stdout:2/390: write d18/d4d/f22 [9206582,76529] 0 2024-03-22T07:16:04.240 INFO:tasks.workunit.client.0.smithi063.stdout:3/847: rename da/d16/d89/d8c/cb6 to da/d16/d89/da9/dd3/c113 0 2024-03-22T07:16:04.240 INFO:tasks.workunit.client.1.smithi069.stdout:4/607: creat d6/d16/d3a/fcb x:0 0 0 2024-03-22T07:16:04.240 INFO:tasks.workunit.client.1.smithi069.stdout:4/608: fsync d6/d9b/d1f/d58/f69 0 2024-03-22T07:16:04.240 INFO:tasks.workunit.client.0.smithi063.stdout:2/708: symlink d0/de/d58/le5 0 2024-03-22T07:16:04.242 INFO:tasks.workunit.client.1.smithi069.stdout:8/687: getdents d1 0 2024-03-22T07:16:04.242 INFO:tasks.workunit.client.0.smithi063.stdout:1/896: write d4/df/d13/d2b/d6c/d85/fc7 [2649526,16819] 0 2024-03-22T07:16:04.243 INFO:tasks.workunit.client.1.smithi069.stdout:7/506: write d9/db/d1a/f2c [345716,37758] 0 2024-03-22T07:16:04.244 INFO:tasks.workunit.client.0.smithi063.stdout:0/972: link da/de/d17/d25/d3c/d64/df5/ff8 da/d49/d121/f13b 0 2024-03-22T07:16:04.247 INFO:tasks.workunit.client.1.smithi069.stdout:0/639: unlink d3/d9a/f9f 0 2024-03-22T07:16:04.247 INFO:tasks.workunit.client.1.smithi069.stdout:0/640: chown d3/d7/d15/d1f/lbb 0 1 2024-03-22T07:16:04.247 INFO:tasks.workunit.client.0.smithi063.stdout:6/768: creat d3/d6/d2a/d38/d100/f106 x:0 0 0 2024-03-22T07:16:04.247 INFO:tasks.workunit.client.0.smithi063.stdout:7/652: symlink d5/da/d28/ld6 0 2024-03-22T07:16:04.247 INFO:tasks.workunit.client.0.smithi063.stdout:7/653: read d5/f39 [4151890,85072] 0 2024-03-22T07:16:04.249 INFO:tasks.workunit.client.0.smithi063.stdout:2/709: dread d0/de/f4c [0,4194304] 0 2024-03-22T07:16:04.250 INFO:tasks.workunit.client.0.smithi063.stdout:4/735: mknod d1/d5/d3c/d65/cf8 0 2024-03-22T07:16:04.250 INFO:tasks.workunit.client.0.smithi063.stdout:4/736: stat d1/d5/dc/f5a 0 2024-03-22T07:16:04.251 INFO:tasks.workunit.client.1.smithi069.stdout:4/609: mknod d6/d9b/ccc 0 2024-03-22T07:16:04.251 INFO:tasks.workunit.client.1.smithi069.stdout:8/688: mkdir d1/d93/ddf 0 2024-03-22T07:16:04.251 INFO:tasks.workunit.client.1.smithi069.stdout:2/391: rmdir d18/d4d/d27/d33/d60 39 2024-03-22T07:16:04.251 INFO:tasks.workunit.client.1.smithi069.stdout:8/689: chown d1/d1c/f38 1514 1 2024-03-22T07:16:04.251 INFO:tasks.workunit.client.1.smithi069.stdout:2/392: fsync d18/d4d/d3e/d40/f44 0 2024-03-22T07:16:04.251 INFO:tasks.workunit.client.1.smithi069.stdout:8/690: read - d1/d1c/d3e/d88/dcd/d6c/d4e/fc8 zero size 2024-03-22T07:16:04.251 INFO:tasks.workunit.client.1.smithi069.stdout:8/691: fsync d1/d1c/d3e/f36 0 2024-03-22T07:16:04.251 INFO:tasks.workunit.client.1.smithi069.stdout:7/507: rmdir d9/d76/d57/d44/d53 39 2024-03-22T07:16:04.252 INFO:tasks.workunit.client.0.smithi063.stdout:3/848: mkdir da/d63/d65/d83/dc6/d114 0 2024-03-22T07:16:04.252 INFO:tasks.workunit.client.1.smithi069.stdout:0/641: mknod d3/d7/d15/d29/d5a/d84/cd1 0 2024-03-22T07:16:04.252 INFO:tasks.workunit.client.1.smithi069.stdout:0/642: write d3/f39 [299900,98258] 0 2024-03-22T07:16:04.253 INFO:tasks.workunit.client.1.smithi069.stdout:5/411: dwrite d1/d6/d26/d3e/f8d [0,4194304] 0 2024-03-22T07:16:04.253 INFO:tasks.workunit.client.1.smithi069.stdout:5/412: fdatasync d1/d7/f40 0 2024-03-22T07:16:04.253 INFO:tasks.workunit.client.1.smithi069.stdout:5/413: chown d1/d10/d44/d50/c6a 23220 1 2024-03-22T07:16:04.255 INFO:tasks.workunit.client.1.smithi069.stdout:5/414: dread d1/f3 [0,4194304] 0 2024-03-22T07:16:04.256 INFO:tasks.workunit.client.1.smithi069.stdout:3/569: dwrite d5/d9/d26/d3f/d35/f6b [4194304,4194304] 0 2024-03-22T07:16:04.256 INFO:tasks.workunit.client.1.smithi069.stdout:3/570: chown d5/d9/d26/d3f/d56/d7f 2878 1 2024-03-22T07:16:04.257 INFO:tasks.workunit.client.0.smithi063.stdout:1/897: write d4/df/f15 [880345,118824] 0 2024-03-22T07:16:04.258 INFO:tasks.workunit.client.1.smithi069.stdout:4/610: creat d6/dc/fcd x:0 0 0 2024-03-22T07:16:04.258 INFO:tasks.workunit.client.0.smithi063.stdout:0/973: symlink da/de/ddd/l13c 0 2024-03-22T07:16:04.259 INFO:tasks.workunit.client.0.smithi063.stdout:6/769: mkdir d3/d15/d16/d60/d6d/d107 0 2024-03-22T07:16:04.259 INFO:tasks.workunit.client.0.smithi063.stdout:6/770: readlink d3/d15/d16/d60/d6d/d4e/lac 0 2024-03-22T07:16:04.260 INFO:tasks.workunit.client.1.smithi069.stdout:6/494: dwrite d4/f46 [4194304,4194304] 0 2024-03-22T07:16:04.260 INFO:tasks.workunit.client.0.smithi063.stdout:3/849: mknod da/d16/d3f/c115 0 2024-03-22T07:16:04.261 INFO:tasks.workunit.client.0.smithi063.stdout:3/850: fdatasync da/d63/d65/d81/d9b/fc9 0 2024-03-22T07:16:04.262 INFO:tasks.workunit.client.1.smithi069.stdout:2/393: creat d18/d4d/d27/d56/f88 x:0 0 0 2024-03-22T07:16:04.262 INFO:tasks.workunit.client.0.smithi063.stdout:1/898: mknod d4/c134 0 2024-03-22T07:16:04.263 INFO:tasks.workunit.client.0.smithi063.stdout:1/899: readlink d4/df/de7/lfa 0 2024-03-22T07:16:04.263 INFO:tasks.workunit.client.0.smithi063.stdout:0/974: mknod da/de/d67/c13d 0 2024-03-22T07:16:04.265 INFO:tasks.workunit.client.1.smithi069.stdout:8/692: unlink d1/d1c/l2b 0 2024-03-22T07:16:04.265 INFO:tasks.workunit.client.1.smithi069.stdout:8/693: readlink d1/d1c/lc0 0 2024-03-22T07:16:04.265 INFO:tasks.workunit.client.0.smithi063.stdout:6/771: mknod d3/d15/d16/d60/d6d/d4c/c108 0 2024-03-22T07:16:04.266 INFO:tasks.workunit.client.1.smithi069.stdout:0/643: rename d3/d7/d24/d6a/l92 to d3/d9a/dcb/ld2 0 2024-03-22T07:16:04.268 INFO:tasks.workunit.client.0.smithi063.stdout:3/851: rename da/d63/d65/ce3 to da/d63/d65/d81/d9b/d9f/c116 0 2024-03-22T07:16:04.268 INFO:tasks.workunit.client.0.smithi063.stdout:2/710: dread d0/de/d12/d20/f5d [0,4194304] 0 2024-03-22T07:16:04.270 INFO:tasks.workunit.client.0.smithi063.stdout:1/900: mknod d4/d71/d88/da1/ddb/c135 0 2024-03-22T07:16:04.270 INFO:tasks.workunit.client.1.smithi069.stdout:5/415: mknod d1/d7/d3d/c8f 0 2024-03-22T07:16:04.271 INFO:tasks.workunit.client.0.smithi063.stdout:0/975: symlink da/de/d17/d25/d95/d98/l13e 0 2024-03-22T07:16:04.271 INFO:tasks.workunit.client.0.smithi063.stdout:9/743: dwrite d8/f9d [0,4194304] 0 2024-03-22T07:16:04.274 INFO:tasks.workunit.client.0.smithi063.stdout:3/852: creat da/d63/d65/d7b/f117 x:0 0 0 2024-03-22T07:16:04.274 INFO:tasks.workunit.client.1.smithi069.stdout:3/571: mknod d5/d9/d26/d36/d41/cb8 0 2024-03-22T07:16:04.275 INFO:tasks.workunit.client.0.smithi063.stdout:5/656: dwrite d8/fe [4194304,4194304] 0 2024-03-22T07:16:04.276 INFO:tasks.workunit.client.0.smithi063.stdout:0/976: rename da/de/d17/d25/d3c/d53/d89/l8d to da/de/d17/d25/d3c/d43/d116/l13f 0 2024-03-22T07:16:04.276 INFO:tasks.workunit.client.0.smithi063.stdout:0/977: truncate da/de/d17/d25/f65 2277978 0 2024-03-22T07:16:04.276 INFO:tasks.workunit.client.0.smithi063.stdout:0/978: write da/de/d17/d25/d99/dbc/f10b [57730,1515] 0 2024-03-22T07:16:04.277 INFO:tasks.workunit.client.0.smithi063.stdout:0/979: dread da/de/d17/d25/d3c/d43/db5/f101 [0,4194304] 0 2024-03-22T07:16:04.279 INFO:tasks.workunit.client.1.smithi069.stdout:6/495: rmdir d4/d82 0 2024-03-22T07:16:04.279 INFO:tasks.workunit.client.1.smithi069.stdout:6/496: write d4/f14 [2822245,125782] 0 2024-03-22T07:16:04.283 INFO:tasks.workunit.client.1.smithi069.stdout:2/394: rmdir d18/d4d/d27/d33/d60/d63 39 2024-03-22T07:16:04.283 INFO:tasks.workunit.client.0.smithi063.stdout:5/657: creat d8/fc6 x:0 0 0 2024-03-22T07:16:04.283 INFO:tasks.workunit.client.0.smithi063.stdout:5/658: fsync d8/d13/d28/f43 0 2024-03-22T07:16:04.284 INFO:tasks.workunit.client.0.smithi063.stdout:2/711: dread d0/de/f30 [0,4194304] 0 2024-03-22T07:16:04.284 INFO:tasks.workunit.client.0.smithi063.stdout:2/712: creat d0/de/d12/d20/fe6 x:0 0 0 2024-03-22T07:16:04.286 INFO:tasks.workunit.client.1.smithi069.stdout:8/694: chown d1/d1c/d48/c6a 5 1 2024-03-22T07:16:04.286 INFO:tasks.workunit.client.1.smithi069.stdout:8/695: dread - d1/d1c/d3e/d88/dcd/d6c/f92 zero size 2024-03-22T07:16:04.286 INFO:tasks.workunit.client.1.smithi069.stdout:8/696: chown d1/d1c/d3e/d88/dcd/d20/c7f 7931299 1 2024-03-22T07:16:04.287 INFO:tasks.workunit.client.0.smithi063.stdout:9/744: rename d8/d17/d1e/c61 to d8/d19/d20/d22/d29/db0/ce6 0 2024-03-22T07:16:04.287 INFO:tasks.workunit.client.0.smithi063.stdout:9/745: read d8/d15/f58 [452627,78832] 0 2024-03-22T07:16:04.287 INFO:tasks.workunit.client.0.smithi063.stdout:9/746: fdatasync d8/d15/d2f/f93 0 2024-03-22T07:16:04.287 INFO:tasks.workunit.client.0.smithi063.stdout:4/737: dwrite d1/d5/dc/d95/fb5 [0,4194304] 0 2024-03-22T07:16:04.288 INFO:tasks.workunit.client.0.smithi063.stdout:0/980: mkdir da/de/d17/d25/d3c/d43/db5/d140 0 2024-03-22T07:16:04.288 INFO:tasks.workunit.client.1.smithi069.stdout:3/572: write d5/d9/d26/f6c [1157729,2329] 0 2024-03-22T07:16:04.288 INFO:tasks.workunit.client.1.smithi069.stdout:3/573: stat d5/d9 0 2024-03-22T07:16:04.288 INFO:tasks.workunit.client.1.smithi069.stdout:3/574: write d5/d9/d26/d3f/d56/d7f/fa1 [768444,119959] 0 2024-03-22T07:16:04.288 INFO:tasks.workunit.client.1.smithi069.stdout:3/575: fdatasync d5/d9/d26/d36/d41/d8e/f9f 0 2024-03-22T07:16:04.290 INFO:tasks.workunit.client.0.smithi063.stdout:3/853: getdents da/d63/d65/d92 0 2024-03-22T07:16:04.290 INFO:tasks.workunit.client.1.smithi069.stdout:5/416: creat d1/d7/d8/f90 x:0 0 0 2024-03-22T07:16:04.291 INFO:tasks.workunit.client.1.smithi069.stdout:5/417: creat d1/d28/f91 x:0 0 0 2024-03-22T07:16:04.297 INFO:tasks.workunit.client.1.smithi069.stdout:6/497: mkdir d4/dd/d41/d78/d9a 0 2024-03-22T07:16:04.297 INFO:tasks.workunit.client.1.smithi069.stdout:8/697: mknod d1/d1c/d48/d5e/ce0 0 2024-03-22T07:16:04.297 INFO:tasks.workunit.client.1.smithi069.stdout:2/395: dread d18/f19 [0,4194304] 0 2024-03-22T07:16:04.297 INFO:tasks.workunit.client.1.smithi069.stdout:2/396: fdatasync d18/d4d/d25/f36 0 2024-03-22T07:16:04.297 INFO:tasks.workunit.client.0.smithi063.stdout:5/659: symlink d8/d13/d1e/d3c/da0/lc7 0 2024-03-22T07:16:04.297 INFO:tasks.workunit.client.0.smithi063.stdout:2/713: truncate d0/de/d1b/d63/f86 134423 0 2024-03-22T07:16:04.297 INFO:tasks.workunit.client.0.smithi063.stdout:9/747: symlink d8/d19/d20/d22/de1/dd7/dd8/le7 0 2024-03-22T07:16:04.298 INFO:tasks.workunit.client.1.smithi069.stdout:3/576: mkdir d5/d9/d26/d3f/d56/d8a/da5/d7b/db9 0 2024-03-22T07:16:04.298 INFO:tasks.workunit.client.1.smithi069.stdout:0/644: getdents d3/d7/d15/d29/d5a/d5d 0 2024-03-22T07:16:04.298 INFO:tasks.workunit.client.1.smithi069.stdout:0/645: dread - d3/d7/d17/f83 zero size 2024-03-22T07:16:04.299 INFO:tasks.workunit.client.0.smithi063.stdout:4/738: creat d1/d5/dc/d95/d59/d9c/dac/dc9/ff9 x:0 0 0 2024-03-22T07:16:04.299 INFO:tasks.workunit.client.0.smithi063.stdout:4/739: truncate d1/d5/d43/d27/d2c/d4a/d78/fb2 80170 0 2024-03-22T07:16:04.300 INFO:tasks.workunit.client.0.smithi063.stdout:3/854: stat da/d16/d26/l2c 0 2024-03-22T07:16:04.300 INFO:tasks.workunit.client.1.smithi069.stdout:5/418: symlink d1/d7/d8/l92 0 2024-03-22T07:16:04.302 INFO:tasks.workunit.client.1.smithi069.stdout:8/698: rename d1/d1c/d3e/d88/dcd/c5f to d1/d1c/d48/d5e/ce1 0 2024-03-22T07:16:04.303 INFO:tasks.workunit.client.1.smithi069.stdout:3/577: symlink d5/d9/d26/d3f/d56/d8a/da5/lba 0 2024-03-22T07:16:04.304 INFO:tasks.workunit.client.0.smithi063.stdout:2/714: mknod d0/de/d1b/d63/d6e/ce7 0 2024-03-22T07:16:04.305 INFO:tasks.workunit.client.1.smithi069.stdout:0/646: mknod d3/d9a/cd3 0 2024-03-22T07:16:04.306 INFO:tasks.workunit.client.0.smithi063.stdout:4/740: truncate d1/d5/d43/f17 2386522 0 2024-03-22T07:16:04.307 INFO:tasks.workunit.client.1.smithi069.stdout:5/419: creat d1/d6/d88/f93 x:0 0 0 2024-03-22T07:16:04.308 INFO:tasks.workunit.client.0.smithi063.stdout:3/855: mknod da/d63/d65/d4b/c118 0 2024-03-22T07:16:04.308 INFO:tasks.workunit.client.0.smithi063.stdout:3/856: truncate da/d63/d65/f77 13496711 0 2024-03-22T07:16:04.309 INFO:tasks.workunit.client.1.smithi069.stdout:6/498: write d4/dd/d41/d78/f85 [2188187,125734] 0 2024-03-22T07:16:04.309 INFO:tasks.workunit.client.0.smithi063.stdout:5/660: mknod d8/cc8 0 2024-03-22T07:16:04.310 INFO:tasks.workunit.client.0.smithi063.stdout:2/715: symlink d0/de/d58/le8 0 2024-03-22T07:16:04.311 INFO:tasks.workunit.client.1.smithi069.stdout:8/699: creat d1/d1c/d3e/d66/fe2 x:0 0 0 2024-03-22T07:16:04.312 INFO:tasks.workunit.client.1.smithi069.stdout:3/578: creat d5/d9/db/fbb x:0 0 0 2024-03-22T07:16:04.313 INFO:tasks.workunit.client.1.smithi069.stdout:3/579: dread - d5/d9/d26/d36/d41/f75 zero size 2024-03-22T07:16:04.313 INFO:tasks.workunit.client.1.smithi069.stdout:3/580: write d5/d9/d26/d3f/d56/d8a/da5/f9c [167481,96357] 0 2024-03-22T07:16:04.313 INFO:tasks.workunit.client.0.smithi063.stdout:9/748: link d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/c91 d8/ce8 0 2024-03-22T07:16:04.313 INFO:tasks.workunit.client.1.smithi069.stdout:5/420: mknod d1/d6/d26/d3a/c94 0 2024-03-22T07:16:04.316 INFO:tasks.workunit.client.0.smithi063.stdout:5/661: dread d8/d13/d1e/f7b [0,4194304] 0 2024-03-22T07:16:04.316 INFO:tasks.workunit.client.0.smithi063.stdout:5/662: readlink d8/lb 0 2024-03-22T07:16:04.317 INFO:tasks.workunit.client.0.smithi063.stdout:3/857: mknod da/c119 0 2024-03-22T07:16:04.318 INFO:tasks.workunit.client.1.smithi069.stdout:8/700: mkdir d1/d93/ddf/de3 0 2024-03-22T07:16:04.319 INFO:tasks.workunit.client.0.smithi063.stdout:2/716: mkdir d0/de/d12/d20/de9 0 2024-03-22T07:16:04.323 INFO:tasks.workunit.client.0.smithi063.stdout:4/741: getdents d1/d5/d3c/d8f/dc0 0 2024-03-22T07:16:04.323 INFO:tasks.workunit.client.0.smithi063.stdout:5/663: getdents d8/d13/d24/d81 0 2024-03-22T07:16:04.323 INFO:tasks.workunit.client.1.smithi069.stdout:3/581: mkdir d5/d9/d26/d3f/d56/d8a/da5/d7b/dbc 0 2024-03-22T07:16:04.323 INFO:tasks.workunit.client.1.smithi069.stdout:6/499: write d4/dd/f33 [311382,84185] 0 2024-03-22T07:16:04.323 INFO:tasks.workunit.client.1.smithi069.stdout:5/421: getdents d1/d10/d44/d50 0 2024-03-22T07:16:04.323 INFO:tasks.workunit.client.1.smithi069.stdout:5/422: dread d1/d7/f40 [0,4194304] 0 2024-03-22T07:16:04.324 INFO:tasks.workunit.client.0.smithi063.stdout:5/664: write d8/d13/d28/d57/d5f/fb4 [830454,50776] 0 2024-03-22T07:16:04.325 INFO:tasks.workunit.client.0.smithi063.stdout:2/717: mkdir d0/de/d1b/d63/dea 0 2024-03-22T07:16:04.328 INFO:tasks.workunit.client.1.smithi069.stdout:8/701: rename d1/d1c/d48/d5e/f79 to d1/d1c/fe4 0 2024-03-22T07:16:04.328 INFO:tasks.workunit.client.0.smithi063.stdout:4/742: link d1/d5/dc/dda/d79/daa/fb7 d1/d5/dc/d95/d59/d9c/ffa 0 2024-03-22T07:16:04.329 INFO:tasks.workunit.client.1.smithi069.stdout:3/582: mknod d5/d9/d26/db4/cbd 0 2024-03-22T07:16:04.331 INFO:tasks.workunit.client.1.smithi069.stdout:5/423: mkdir d1/d6/d26/d33/d5d/d72/d95 0 2024-03-22T07:16:04.332 INFO:tasks.workunit.client.1.smithi069.stdout:4/611: dwrite d6/d12/f70 [0,4194304] 0 2024-03-22T07:16:04.337 INFO:tasks.workunit.client.1.smithi069.stdout:4/612: mkdir d6/daa/dce 0 2024-03-22T07:16:04.338 INFO:tasks.workunit.client.1.smithi069.stdout:4/613: write d6/fbb [381427,49654] 0 2024-03-22T07:16:04.339 INFO:tasks.workunit.client.1.smithi069.stdout:4/614: rmdir d6/d16/d83/d9a 39 2024-03-22T07:16:04.349 INFO:tasks.workunit.client.0.smithi063.stdout:0/981: dwrite da/de/d17/d4e/f10c [4194304,4194304] 0 2024-03-22T07:16:04.350 INFO:tasks.workunit.client.0.smithi063.stdout:0/982: truncate da/f82 1602482 0 2024-03-22T07:16:04.352 INFO:tasks.workunit.client.1.smithi069.stdout:4/615: write d6/d9b/d1f/f62 [2874007,62517] 0 2024-03-22T07:16:04.352 INFO:tasks.workunit.client.1.smithi069.stdout:4/616: write d6/dc/d23/d59/fca [447080,115193] 0 2024-03-22T07:16:04.353 INFO:tasks.workunit.client.0.smithi063.stdout:0/983: symlink da/de/d17/d25/d3c/d43/d76/d92/d125/l141 0 2024-03-22T07:16:04.354 INFO:tasks.workunit.client.0.smithi063.stdout:0/984: getdents da/de/d60/d56/d8b/d123 0 2024-03-22T07:16:04.355 INFO:tasks.workunit.client.1.smithi069.stdout:4/617: unlink d6/daa/cc9 0 2024-03-22T07:16:04.355 INFO:tasks.workunit.client.1.smithi069.stdout:4/618: creat d6/d16/d3a/db4/fcf x:0 0 0 2024-03-22T07:16:04.355 INFO:tasks.workunit.client.1.smithi069.stdout:4/619: stat d6/dc4/db5 0 2024-03-22T07:16:04.355 INFO:tasks.workunit.client.1.smithi069.stdout:4/620: truncate d6/d9b/d1f/d58/d7c/fac 141103 0 2024-03-22T07:16:04.357 INFO:tasks.workunit.client.1.smithi069.stdout:4/621: mkdir d6/d16/d83/dd0 0 2024-03-22T07:16:04.357 INFO:tasks.workunit.client.1.smithi069.stdout:4/622: truncate d6/d16/d68/f78 628497 0 2024-03-22T07:16:04.358 INFO:tasks.workunit.client.1.smithi069.stdout:4/623: mknod d6/d9b/d1f/d58/cd1 0 2024-03-22T07:16:04.358 INFO:tasks.workunit.client.1.smithi069.stdout:4/624: dread - d6/d12/f80 zero size 2024-03-22T07:16:04.359 INFO:tasks.workunit.client.1.smithi069.stdout:4/625: rename d6/l7 to d6/d12/d5d/ld2 0 2024-03-22T07:16:04.370 INFO:tasks.workunit.client.0.smithi063.stdout:6/772: dwrite d3/d6/d2a/d38/f9f [0,4194304] 0 2024-03-22T07:16:04.371 INFO:tasks.workunit.client.0.smithi063.stdout:6/773: dread - d3/d6/d35/d3a/fbe zero size 2024-03-22T07:16:04.371 INFO:tasks.workunit.client.0.smithi063.stdout:6/774: truncate d3/d6/d2a/d38/da5/fdc 1251291 0 2024-03-22T07:16:04.371 INFO:tasks.workunit.client.0.smithi063.stdout:1/901: dwrite d4/df/d13/d2b/d3a/fce [0,4194304] 0 2024-03-22T07:16:04.386 INFO:tasks.workunit.client.0.smithi063.stdout:1/902: dread d4/df/d13/f25 [0,4194304] 0 2024-03-22T07:16:04.386 INFO:tasks.workunit.client.0.smithi063.stdout:6/775: dread d3/d6/d2a/f5e [0,4194304] 0 2024-03-22T07:16:04.388 INFO:tasks.workunit.client.1.smithi069.stdout:2/397: dwrite d18/d4d/d27/d56/f62 [0,4194304] 0 2024-03-22T07:16:04.388 INFO:tasks.workunit.client.1.smithi069.stdout:0/647: dwrite d3/d7/d15/d29/f69 [0,4194304] 0 2024-03-22T07:16:04.388 INFO:tasks.workunit.client.1.smithi069.stdout:0/648: dread - d3/d7/d15/d29/d6e/f7a zero size 2024-03-22T07:16:04.389 INFO:tasks.workunit.client.0.smithi063.stdout:9/749: dwrite d8/d19/d20/d22/d29/d31/d53/d6a/f97 [0,4194304] 0 2024-03-22T07:16:04.389 INFO:tasks.workunit.client.0.smithi063.stdout:9/750: dread - d8/d19/d39/db5/fcd zero size 2024-03-22T07:16:04.395 INFO:tasks.workunit.client.1.smithi069.stdout:4/626: dwrite d6/d9b/d1f/d3f/d51/f81 [0,4194304] 0 2024-03-22T07:16:04.395 INFO:tasks.workunit.client.1.smithi069.stdout:8/702: dwrite d1/d1c/d48/f64 [0,4194304] 0 2024-03-22T07:16:04.395 INFO:tasks.workunit.client.1.smithi069.stdout:3/583: dwrite d5/d9/d26/d3f/f3e [0,4194304] 0 2024-03-22T07:16:04.395 INFO:tasks.workunit.client.0.smithi063.stdout:5/665: rmdir d8 39 2024-03-22T07:16:04.396 INFO:tasks.workunit.client.1.smithi069.stdout:5/424: dwrite d1/d6/d26/d3a/f6e [0,4194304] 0 2024-03-22T07:16:04.396 INFO:tasks.workunit.client.1.smithi069.stdout:5/425: readlink d1/d28/l82 0 2024-03-22T07:16:04.397 INFO:tasks.workunit.client.0.smithi063.stdout:5/666: mknod d8/d13/d1e/d31/d58/d7e/cc9 0 2024-03-22T07:16:04.397 INFO:tasks.workunit.client.0.smithi063.stdout:5/667: dread - d8/d13/d1e/d31/d64/d65/d6a/fc5 zero size 2024-03-22T07:16:04.398 INFO:tasks.workunit.client.0.smithi063.stdout:4/743: dwrite d1/d5/dc2/fd6 [0,4194304] 0 2024-03-22T07:16:04.398 INFO:tasks.workunit.client.0.smithi063.stdout:4/744: readlink d1/d5/dc/l37 0 2024-03-22T07:16:04.399 INFO:tasks.workunit.client.0.smithi063.stdout:0/985: dwrite da/de/d17/d25/d3c/d53/d89/fc3 [4194304,4194304] 0 2024-03-22T07:16:04.399 INFO:tasks.workunit.client.0.smithi063.stdout:0/986: chown da/de/d17/cb0 387 1 2024-03-22T07:16:04.399 INFO:tasks.workunit.client.0.smithi063.stdout:5/668: unlink d8/d13/d1e/d31/d64/d65/d6a/fc5 0 2024-03-22T07:16:04.400 INFO:tasks.workunit.client.0.smithi063.stdout:5/669: chown d8/d13/d1e/d31/d58/d7e/c98 5393 1 2024-03-22T07:16:04.400 INFO:tasks.workunit.client.0.smithi063.stdout:5/670: readlink d8/d13/d28/d57/l69 0 2024-03-22T07:16:04.400 INFO:tasks.workunit.client.0.smithi063.stdout:0/987: read da/de/d17/d25/d3c/d53/d89/df1/f11a [125484,9406] 0 2024-03-22T07:16:04.400 INFO:tasks.workunit.client.0.smithi063.stdout:2/718: dwrite d0/de/d12/d5a/f81 [0,4194304] 0 2024-03-22T07:16:04.401 INFO:tasks.workunit.client.0.smithi063.stdout:6/776: dread d3/d6/d2a/d38/da5/f63 [0,4194304] 0 2024-03-22T07:16:04.402 INFO:tasks.workunit.client.0.smithi063.stdout:9/751: write d8/d19/d20/fa0 [739710,85888] 0 2024-03-22T07:16:04.402 INFO:tasks.workunit.client.0.smithi063.stdout:9/752: chown d8/d19/d20/d22/d29/d31/d53/d6a/c7d 0 1 2024-03-22T07:16:04.402 INFO:tasks.workunit.client.1.smithi069.stdout:0/649: mkdir d3/d7/d17/d1c/dd4 0 2024-03-22T07:16:04.407 INFO:tasks.workunit.client.0.smithi063.stdout:5/671: mkdir d8/d13/d28/d57/dca 0 2024-03-22T07:16:04.407 INFO:tasks.workunit.client.0.smithi063.stdout:5/672: fsync d8/d13/d1e/f80 0 2024-03-22T07:16:04.408 INFO:tasks.workunit.client.1.smithi069.stdout:8/703: rename d1/d1c/d3e/c37 to d1/d1c/d3e/d88/dcd/d6c/d4e/d98/ce5 0 2024-03-22T07:16:04.408 INFO:tasks.workunit.client.1.smithi069.stdout:8/704: chown d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/l89 7016099 1 2024-03-22T07:16:04.409 INFO:tasks.workunit.client.1.smithi069.stdout:0/650: rename d3/d9a/dcb to d3/d9a/dcb/dd5 22 2024-03-22T07:16:04.409 INFO:tasks.workunit.client.1.smithi069.stdout:2/398: dwrite d18/f2b [0,4194304] 0 2024-03-22T07:16:04.412 INFO:tasks.workunit.client.1.smithi069.stdout:3/584: unlink d5/d9/d26/d3f/d56/d7f/c81 0 2024-03-22T07:16:04.414 INFO:tasks.workunit.client.1.smithi069.stdout:4/627: write d6/d12/d18/d21/f2f [7108497,89415] 0 2024-03-22T07:16:04.414 INFO:tasks.workunit.client.1.smithi069.stdout:4/628: creat d6/d16/d3a/db4/fd3 x:0 0 0 2024-03-22T07:16:04.414 INFO:tasks.workunit.client.1.smithi069.stdout:4/629: read d6/d9b/d1f/d58/d7c/fac [117663,105676] 0 2024-03-22T07:16:04.415 INFO:tasks.workunit.client.0.smithi063.stdout:2/719: symlink d0/de/d12/d20/de9/leb 0 2024-03-22T07:16:04.416 INFO:tasks.workunit.client.1.smithi069.stdout:5/426: creat d1/d10/d44/d50/f96 x:0 0 0 2024-03-22T07:16:04.418 INFO:tasks.workunit.client.0.smithi063.stdout:9/753: mknod d8/d19/d20/d22/d5d/d76/dab/ce9 0 2024-03-22T07:16:04.419 INFO:tasks.workunit.client.0.smithi063.stdout:9/754: stat d8/d19/d2b/fbb 0 2024-03-22T07:16:04.419 INFO:tasks.workunit.client.0.smithi063.stdout:9/755: stat d8/d17/d1e/l1f 0 2024-03-22T07:16:04.419 INFO:tasks.workunit.client.0.smithi063.stdout:5/673: getdents d8/d13/d24/da4/d8b 0 2024-03-22T07:16:04.421 INFO:tasks.workunit.client.1.smithi069.stdout:0/651: read d3/d7/d15/d29/f69 [2064711,38125] 0 2024-03-22T07:16:04.421 INFO:tasks.workunit.client.1.smithi069.stdout:2/399: mkdir d18/d4d/d27/d33/d60/d64/d89 0 2024-03-22T07:16:04.422 INFO:tasks.workunit.client.0.smithi063.stdout:2/720: symlink d0/de/d58/d7f/d97/da2/lec 0 2024-03-22T07:16:04.423 INFO:tasks.workunit.client.0.smithi063.stdout:6/777: getdents d3/d15/d16/d60/d6d/d4e/d5a 0 2024-03-22T07:16:04.423 INFO:tasks.workunit.client.0.smithi063.stdout:6/778: dread - d3/d6/d2a/d38/da5/d32/fdb zero size 2024-03-22T07:16:04.424 INFO:tasks.workunit.client.1.smithi069.stdout:8/705: dread d1/fd [0,4194304] 0 2024-03-22T07:16:04.424 INFO:tasks.workunit.client.0.smithi063.stdout:5/674: dread d8/d13/f4b [0,4194304] 0 2024-03-22T07:16:04.425 INFO:tasks.workunit.client.0.smithi063.stdout:5/675: write d8/d13/d28/fbc [514266,34307] 0 2024-03-22T07:16:04.425 INFO:tasks.workunit.client.1.smithi069.stdout:2/400: dread d18/d4d/d27/d33/f55 [0,4194304] 0 2024-03-22T07:16:04.425 INFO:tasks.workunit.client.1.smithi069.stdout:3/585: mkdir d5/d9/d26/d3f/dbe 0 2024-03-22T07:16:04.427 INFO:tasks.workunit.client.1.smithi069.stdout:4/630: creat d6/d16/d3a/db4/dbc/fd4 x:0 0 0 2024-03-22T07:16:04.429 INFO:tasks.workunit.client.1.smithi069.stdout:0/652: mkdir d3/d7/d17/d1c/dd6 0 2024-03-22T07:16:04.429 INFO:tasks.workunit.client.1.smithi069.stdout:0/653: fdatasync d3/d7/d15/d1f/f4e 0 2024-03-22T07:16:04.430 INFO:tasks.workunit.client.1.smithi069.stdout:0/654: read d3/d7/d15/d29/d5a/f93 [48932,6958] 0 2024-03-22T07:16:04.430 INFO:tasks.workunit.client.1.smithi069.stdout:0/655: dread - d3/d7/d15/d29/d5a/d99/fbf zero size 2024-03-22T07:16:04.430 INFO:tasks.workunit.client.1.smithi069.stdout:0/656: read d3/d7/d15/d29/f69 [837442,29908] 0 2024-03-22T07:16:04.430 INFO:tasks.workunit.client.1.smithi069.stdout:0/657: chown d3/d7/d24/l44 0 1 2024-03-22T07:16:04.430 INFO:tasks.workunit.client.1.smithi069.stdout:8/706: stat d1/d1c/d48/l5c 0 2024-03-22T07:16:04.431 INFO:tasks.workunit.client.1.smithi069.stdout:2/401: symlink d18/d4d/d27/d72/l8a 0 2024-03-22T07:16:04.431 INFO:tasks.workunit.client.1.smithi069.stdout:2/402: readlink d18/d4d/d27/d33/l3b 0 2024-03-22T07:16:04.433 INFO:tasks.workunit.client.1.smithi069.stdout:4/631: truncate d6/dc/d23/f5c 634956 0 2024-03-22T07:16:04.434 INFO:tasks.workunit.client.0.smithi063.stdout:9/756: rename d8/ce8 to d8/d17/cea 0 2024-03-22T07:16:04.434 INFO:tasks.workunit.client.0.smithi063.stdout:9/757: creat d8/d15/feb x:0 0 0 2024-03-22T07:16:04.434 INFO:tasks.workunit.client.0.smithi063.stdout:0/988: dwrite da/de/f24 [0,4194304] 0 2024-03-22T07:16:04.435 INFO:tasks.workunit.client.1.smithi069.stdout:0/658: symlink d3/d7/d17/ld7 0 2024-03-22T07:16:04.435 INFO:tasks.workunit.client.0.smithi063.stdout:9/758: dread d8/d15/fa1 [0,4194304] 0 2024-03-22T07:16:04.435 INFO:tasks.workunit.client.0.smithi063.stdout:9/759: fdatasync d8/f9a 0 2024-03-22T07:16:04.435 INFO:tasks.workunit.client.0.smithi063.stdout:9/760: dread - d8/f54 zero size 2024-03-22T07:16:04.437 INFO:tasks.workunit.client.0.smithi063.stdout:6/779: creat d3/d6/d2a/d38/da5/d32/dbf/f109 x:0 0 0 2024-03-22T07:16:04.437 INFO:tasks.workunit.client.0.smithi063.stdout:6/780: readlink d3/d6/d2a/d38/da5/d32/l36 0 2024-03-22T07:16:04.437 INFO:tasks.workunit.client.0.smithi063.stdout:6/781: dread - d3/d15/d16/d60/d6d/d4c/f7d zero size 2024-03-22T07:16:04.437 INFO:tasks.workunit.client.1.smithi069.stdout:8/707: mkdir d1/d1c/d3e/de6 0 2024-03-22T07:16:04.439 INFO:tasks.workunit.client.1.smithi069.stdout:4/632: link d6/d4b/f97 d6/dc/d1d/d55/fd5 0 2024-03-22T07:16:04.439 INFO:tasks.workunit.client.0.smithi063.stdout:0/989: symlink da/de/d60/l142 0 2024-03-22T07:16:04.441 INFO:tasks.workunit.client.1.smithi069.stdout:0/659: getdents d3/d7/d24 0 2024-03-22T07:16:04.443 INFO:tasks.workunit.client.1.smithi069.stdout:8/708: mkdir d1/d1c/d3e/d88/da8/de7 0 2024-03-22T07:16:04.443 INFO:tasks.workunit.client.0.smithi063.stdout:2/721: rmdir d0/de/d1b/d63/dd6 39 2024-03-22T07:16:04.444 INFO:tasks.workunit.client.0.smithi063.stdout:9/761: mknod d8/d19/d20/d22/d29/db0/cec 0 2024-03-22T07:16:04.444 INFO:tasks.workunit.client.1.smithi069.stdout:5/427: dwrite d1/d6/d26/d33/f4c [0,4194304] 0 2024-03-22T07:16:04.447 INFO:tasks.workunit.client.0.smithi063.stdout:9/762: truncate d8/d19/d20/d22/d29/d31/d53/d6a/da2/fc7 376683 0 2024-03-22T07:16:04.448 INFO:tasks.workunit.client.0.smithi063.stdout:6/782: getdents d3/d15/d16/d60/d6d/d4e 0 2024-03-22T07:16:04.448 INFO:tasks.workunit.client.1.smithi069.stdout:4/633: rename d6/d9b/d56/cb9 to d6/d9b/d1f/cd6 0 2024-03-22T07:16:04.449 INFO:tasks.workunit.client.1.smithi069.stdout:0/660: creat d3/d7/d15/d29/fd8 x:0 0 0 2024-03-22T07:16:04.450 INFO:tasks.workunit.client.0.smithi063.stdout:2/722: mkdir d0/ded 0 2024-03-22T07:16:04.450 INFO:tasks.workunit.client.0.smithi063.stdout:2/723: fsync d0/f9 0 2024-03-22T07:16:04.451 INFO:tasks.workunit.client.0.smithi063.stdout:8/867: sync 2024-03-22T07:16:04.451 INFO:tasks.workunit.client.0.smithi063.stdout:9/763: symlink d8/d19/d20/d22/db2/led 0 2024-03-22T07:16:04.452 INFO:tasks.workunit.client.0.smithi063.stdout:6/783: truncate d3/d15/d16/d60/d6d/d4e/d5a/d88/fc1 3565158 0 2024-03-22T07:16:04.454 INFO:tasks.workunit.client.0.smithi063.stdout:2/724: creat d0/de/d74/fee x:0 0 0 2024-03-22T07:16:04.454 INFO:tasks.workunit.client.0.smithi063.stdout:2/725: readlink d0/de/d12/d5a/lb4 0 2024-03-22T07:16:04.454 INFO:tasks.workunit.client.1.smithi069.stdout:0/661: read d3/d7/d24/d6a/fc0 [65520,36153] 0 2024-03-22T07:16:04.455 INFO:tasks.workunit.client.0.smithi063.stdout:8/868: mknod d0/d1b/d28/de8/dc6/c124 0 2024-03-22T07:16:04.455 INFO:tasks.workunit.client.1.smithi069.stdout:4/634: mknod d6/d4b/cd7 0 2024-03-22T07:16:04.456 INFO:tasks.workunit.client.0.smithi063.stdout:6/784: truncate d3/d15/f17 143138 0 2024-03-22T07:16:04.458 INFO:tasks.workunit.client.1.smithi069.stdout:3/586: dwrite d5/d9/d26/d3f/d35/f37 [0,4194304] 0 2024-03-22T07:16:04.458 INFO:tasks.workunit.client.1.smithi069.stdout:3/587: chown d5/d9/d23/f78 190159562 1 2024-03-22T07:16:04.462 INFO:tasks.workunit.client.1.smithi069.stdout:0/662: creat d3/fd9 x:0 0 0 2024-03-22T07:16:04.465 INFO:tasks.workunit.client.0.smithi063.stdout:6/785: rename d3/d15/d16/d47 to d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/d10a 0 2024-03-22T07:16:04.465 INFO:tasks.workunit.client.0.smithi063.stdout:6/786: chown d3/d15/d16/d60/d6d/d4c/de7 5857 1 2024-03-22T07:16:04.465 INFO:tasks.workunit.client.1.smithi069.stdout:1/479: sync 2024-03-22T07:16:04.465 INFO:tasks.workunit.client.1.smithi069.stdout:1/480: chown d1/d13/d36/f54 2 1 2024-03-22T07:16:04.466 INFO:tasks.workunit.client.1.smithi069.stdout:4/635: read d6/d12/d18/f4f [2619857,83484] 0 2024-03-22T07:16:04.466 INFO:tasks.workunit.client.0.smithi063.stdout:5/676: dwrite d8/d13/f4b [0,4194304] 0 2024-03-22T07:16:04.468 INFO:tasks.workunit.client.0.smithi063.stdout:6/787: unlink d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/d10a/f87 0 2024-03-22T07:16:04.469 INFO:tasks.workunit.client.0.smithi063.stdout:6/788: stat d3/d6/d2a/d38/da5/d32/dbf/lc0 0 2024-03-22T07:16:04.469 INFO:tasks.workunit.client.1.smithi069.stdout:3/588: creat d5/d9/d26/d3f/d56/d7f/d90/fbf x:0 0 0 2024-03-22T07:16:04.471 INFO:tasks.workunit.client.1.smithi069.stdout:1/481: getdents d1/d2a/d50/d88 0 2024-03-22T07:16:04.472 INFO:tasks.workunit.client.1.smithi069.stdout:1/482: write d1/f22 [2383230,15048] 0 2024-03-22T07:16:04.472 INFO:tasks.workunit.client.1.smithi069.stdout:1/483: fsync d1/d2a/d50/f40 0 2024-03-22T07:16:04.472 INFO:tasks.workunit.client.0.smithi063.stdout:5/677: link d8/d13/d24/da4/d8b/fb5 d8/d13/d28/d57/fcb 0 2024-03-22T07:16:04.473 INFO:tasks.workunit.client.1.smithi069.stdout:4/636: mknod d6/d16/d3a/cd8 0 2024-03-22T07:16:04.475 INFO:tasks.workunit.client.0.smithi063.stdout:8/869: write d0/d24/f42 [1522124,84390] 0 2024-03-22T07:16:04.476 INFO:tasks.workunit.client.0.smithi063.stdout:6/789: rename d3/d15/d16/d57 to d3/d15/d16/d60/d6d/d107/d10b 0 2024-03-22T07:16:04.476 INFO:tasks.workunit.client.1.smithi069.stdout:4/637: dread d6/d12/f6f [0,4194304] 0 2024-03-22T07:16:04.476 INFO:tasks.workunit.client.1.smithi069.stdout:3/589: creat d5/d9/d26/d3f/d56/d8a/da5/fc0 x:0 0 0 2024-03-22T07:16:04.477 INFO:tasks.workunit.client.1.smithi069.stdout:4/638: symlink d6/d16/d83/dd0/ld9 0 2024-03-22T07:16:04.477 INFO:tasks.workunit.client.1.smithi069.stdout:4/639: creat d6/dc/d1d/d55/fda x:0 0 0 2024-03-22T07:16:04.478 INFO:tasks.workunit.client.0.smithi063.stdout:6/790: rename d3/d6/d2a/d38/da5/d32/d51/l79 to d3/d15/da3/dc2/l10c 0 2024-03-22T07:16:04.478 INFO:tasks.workunit.client.0.smithi063.stdout:6/791: dread - d3/d15/d16/d60/d6d/d4e/fd2 zero size 2024-03-22T07:16:04.478 INFO:tasks.workunit.client.1.smithi069.stdout:3/590: mkdir d5/d9/d26/d3f/dc1 0 2024-03-22T07:16:04.479 INFO:tasks.workunit.client.1.smithi069.stdout:3/591: write d5/d9/d26/d3f/d56/d7f/fa1 [518057,108328] 0 2024-03-22T07:16:04.481 INFO:tasks.workunit.client.0.smithi063.stdout:5/678: dread d8/d13/d1e/f2f [0,4194304] 0 2024-03-22T07:16:04.481 INFO:tasks.workunit.client.0.smithi063.stdout:5/679: read - d8/d13/d1e/d3c/fae zero size 2024-03-22T07:16:04.484 INFO:tasks.workunit.client.0.smithi063.stdout:6/792: dread d3/d15/d16/d69/f83 [4194304,4194304] 0 2024-03-22T07:16:04.485 INFO:tasks.workunit.client.0.smithi063.stdout:5/680: creat d8/dba/fcc x:0 0 0 2024-03-22T07:16:04.486 INFO:tasks.workunit.client.1.smithi069.stdout:3/592: creat d5/d9/d26/d3f/d56/d8a/da5/d7b/dbc/fc2 x:0 0 0 2024-03-22T07:16:04.486 INFO:tasks.workunit.client.1.smithi069.stdout:3/593: stat d5/d9/d26/d3f/d54/c5c 0 2024-03-22T07:16:04.486 INFO:tasks.workunit.client.1.smithi069.stdout:3/594: symlink d5/d9/d26/d3f/d35/lc3 0 2024-03-22T07:16:04.486 INFO:tasks.workunit.client.1.smithi069.stdout:3/595: readlink d5/d9/d26/l44 0 2024-03-22T07:16:04.486 INFO:tasks.workunit.client.1.smithi069.stdout:3/596: write d5/d9/d23/f78 [347739,66437] 0 2024-03-22T07:16:04.487 INFO:tasks.workunit.client.1.smithi069.stdout:3/597: rename d5/d9/d26/d3f/d56/d7f/d90/fbf to d5/d9/d26/d3f/d35/fc4 0 2024-03-22T07:16:04.487 INFO:tasks.workunit.client.1.smithi069.stdout:3/598: readlink d5/d9/d26/d3f/l91 0 2024-03-22T07:16:04.487 INFO:tasks.workunit.client.0.smithi063.stdout:6/793: symlink d3/d15/d16/d60/d6d/d107/d10b/l10d 0 2024-03-22T07:16:04.488 INFO:tasks.workunit.client.0.smithi063.stdout:6/794: readlink d3/d15/d16/d60/d6d/d107/d10b/dde/deb/l104 0 2024-03-22T07:16:04.488 INFO:tasks.workunit.client.0.smithi063.stdout:8/870: dread d0/d1b/d28/d7b/d45/f58 [0,4194304] 0 2024-03-22T07:16:04.488 INFO:tasks.workunit.client.0.smithi063.stdout:8/871: truncate d0/d26/d2d/d59/dc0/fdf 4355027 0 2024-03-22T07:16:04.488 INFO:tasks.workunit.client.1.smithi069.stdout:3/599: symlink d5/lc5 0 2024-03-22T07:16:04.489 INFO:tasks.workunit.client.0.smithi063.stdout:8/872: symlink d0/d1b/d28/d3e/db2/dbd/dd0/l125 0 2024-03-22T07:16:04.490 INFO:tasks.workunit.client.0.smithi063.stdout:8/873: fdatasync d0/d24/d33/d4c/f7d 0 2024-03-22T07:16:04.491 INFO:tasks.workunit.client.0.smithi063.stdout:8/874: symlink d0/d24/d33/d122/l126 0 2024-03-22T07:16:04.491 INFO:tasks.workunit.client.0.smithi063.stdout:8/875: write d0/d24/d33/d4c/fd4 [3538607,128490] 0 2024-03-22T07:16:04.491 INFO:tasks.workunit.client.0.smithi063.stdout:8/876: creat d0/d1b/d28/d7b/d45/f127 x:0 0 0 2024-03-22T07:16:04.491 INFO:tasks.workunit.client.1.smithi069.stdout:3/600: truncate d5/f5d 6235575 0 2024-03-22T07:16:04.491 INFO:tasks.workunit.client.1.smithi069.stdout:3/601: write d5/d9/d26/d3f/d35/fc4 [351989,125583] 0 2024-03-22T07:16:04.491 INFO:tasks.workunit.client.1.smithi069.stdout:3/602: write d5/d9/d26/d3f/d56/d8a/da5/d7b/dbc/fc2 [679267,76300] 0 2024-03-22T07:16:04.491 INFO:tasks.workunit.client.1.smithi069.stdout:3/603: truncate d5/d9/d26/d36/d41/f75 187016 0 2024-03-22T07:16:04.493 INFO:tasks.workunit.client.1.smithi069.stdout:1/484: dread d1/d2a/d50/d4c/d55/f56 [0,4194304] 0 2024-03-22T07:16:04.495 INFO:tasks.workunit.client.1.smithi069.stdout:3/604: dread d5/d9/d26/f6c [0,4194304] 0 2024-03-22T07:16:04.495 INFO:tasks.workunit.client.1.smithi069.stdout:1/485: creat d1/d13/d36/f8f x:0 0 0 2024-03-22T07:16:04.496 INFO:tasks.workunit.client.1.smithi069.stdout:3/605: mkdir d5/d9/db/d3b/dc6 0 2024-03-22T07:16:04.497 INFO:tasks.workunit.client.0.smithi063.stdout:6/795: dread d3/d15/d16/d69/f83 [0,4194304] 0 2024-03-22T07:16:04.498 INFO:tasks.workunit.client.1.smithi069.stdout:8/709: dwrite d1/d1c/f68 [0,4194304] 0 2024-03-22T07:16:04.498 INFO:tasks.workunit.client.1.smithi069.stdout:1/486: creat d1/d2a/d50/d88/d41/d7a/f90 x:0 0 0 2024-03-22T07:16:04.501 INFO:tasks.workunit.client.1.smithi069.stdout:1/487: unlink d1/d2a/d50/d88/l2f 0 2024-03-22T07:16:04.501 INFO:tasks.workunit.client.1.smithi069.stdout:0/663: dwrite d3/d7/d17/d1c/fb3 [8388608,4194304] 0 2024-03-22T07:16:04.501 INFO:tasks.workunit.client.1.smithi069.stdout:0/664: write d3/d7/d17/d1c/fc7 [1005379,112900] 0 2024-03-22T07:16:04.502 INFO:tasks.workunit.client.0.smithi063.stdout:5/681: dwrite d8/d13/d1e/d31/daf/fb0 [0,4194304] 0 2024-03-22T07:16:04.502 INFO:tasks.workunit.client.0.smithi063.stdout:9/764: dwrite d8/d15/d2f/f93 [0,4194304] 0 2024-03-22T07:16:04.502 INFO:tasks.workunit.client.0.smithi063.stdout:9/765: stat d8/d15/d63/la6 0 2024-03-22T07:16:04.502 INFO:tasks.workunit.client.0.smithi063.stdout:9/766: chown d8/d19/d20/c18 33454841 1 2024-03-22T07:16:04.502 INFO:tasks.workunit.client.0.smithi063.stdout:0/990: dwrite da/de/d17/d25/d3c/d64/fb2 [4194304,4194304] 0 2024-03-22T07:16:04.504 INFO:tasks.workunit.client.1.smithi069.stdout:2/403: dwrite d18/d4d/d25/f47 [0,4194304] 0 2024-03-22T07:16:04.504 INFO:tasks.workunit.client.1.smithi069.stdout:2/404: truncate d18/d4d/f28 4477670 0 2024-03-22T07:16:04.513 INFO:tasks.workunit.client.0.smithi063.stdout:0/991: unlink da/de/d17/d25/d3c/d43/dc9/ld7 0 2024-03-22T07:16:04.517 INFO:tasks.workunit.client.0.smithi063.stdout:0/992: symlink da/de/d17/d25/d3c/d53/l143 0 2024-03-22T07:16:04.517 INFO:tasks.workunit.client.0.smithi063.stdout:5/682: dread d8/d13/d24/f50 [4194304,4194304] 0 2024-03-22T07:16:04.519 INFO:tasks.workunit.client.1.smithi069.stdout:8/710: dread d1/d1c/d3e/d66/d9e/fd6 [0,4194304] 0 2024-03-22T07:16:04.519 INFO:tasks.workunit.client.1.smithi069.stdout:8/711: write d1/d1c/d3e/d66/f6e [4484724,130937] 0 2024-03-22T07:16:04.519 INFO:tasks.workunit.client.1.smithi069.stdout:8/712: stat d1/d1c/cb8 0 2024-03-22T07:16:04.519 INFO:tasks.workunit.client.1.smithi069.stdout:8/713: chown d1/d1c/d3e/d88/dcd/ca3 5445 1 2024-03-22T07:16:04.519 INFO:tasks.workunit.client.1.smithi069.stdout:0/665: stat d3/d7/d15/d1f/l87 0 2024-03-22T07:16:04.520 INFO:tasks.workunit.client.0.smithi063.stdout:0/993: symlink da/de/d60/d56/d8b/d123/l144 0 2024-03-22T07:16:04.520 INFO:tasks.workunit.client.1.smithi069.stdout:8/714: dread d1/d1c/d3e/d88/dcd/fa4 [0,4194304] 0 2024-03-22T07:16:04.520 INFO:tasks.workunit.client.1.smithi069.stdout:8/715: read - d1/d1c/d3e/f9c zero size 2024-03-22T07:16:04.520 INFO:tasks.workunit.client.1.smithi069.stdout:8/716: chown d1/d1c/d48/c6a 0 1 2024-03-22T07:16:04.521 INFO:tasks.workunit.client.0.smithi063.stdout:0/994: unlink da/de/d17/d4e/f10c 0 2024-03-22T07:16:04.523 INFO:tasks.workunit.client.1.smithi069.stdout:4/640: getdents d6/d16/d83/dd0 0 2024-03-22T07:16:04.526 INFO:tasks.workunit.client.1.smithi069.stdout:0/666: link d3/d7/d15/d29/d5a/d84/f60 d3/d9a/fda 0 2024-03-22T07:16:04.526 INFO:tasks.workunit.client.1.smithi069.stdout:0/667: write d3/d7/d15/d29/d6e/f7a [152722,103584] 0 2024-03-22T07:16:04.528 INFO:tasks.workunit.client.1.smithi069.stdout:4/641: truncate d6/dc/d23/d59/fca 68070 0 2024-03-22T07:16:04.529 INFO:tasks.workunit.client.1.smithi069.stdout:4/642: chown d6/d12/f73 1030 1 2024-03-22T07:16:04.529 INFO:tasks.workunit.client.0.smithi063.stdout:8/877: dwrite d0/d1b/d28/d7b/d45/f58 [0,4194304] 0 2024-03-22T07:16:04.529 INFO:tasks.workunit.client.0.smithi063.stdout:8/878: fdatasync d0/d1b/d28/d5f/fd8 0 2024-03-22T07:16:04.529 INFO:tasks.workunit.client.0.smithi063.stdout:8/879: chown d0/d24/d33/fee 252 1 2024-03-22T07:16:04.531 INFO:tasks.workunit.client.1.smithi069.stdout:8/717: getdents d1/d1c/d3e/d88/da8/dbb 0 2024-03-22T07:16:04.531 INFO:tasks.workunit.client.1.smithi069.stdout:8/718: readlink d1/l4a 0 2024-03-22T07:16:04.535 INFO:tasks.workunit.client.1.smithi069.stdout:4/643: creat d6/d16/d3a/db4/dbc/dc6/fdb x:0 0 0 2024-03-22T07:16:04.535 INFO:tasks.workunit.client.1.smithi069.stdout:4/644: chown d6/d16/d83/lc8 3 1 2024-03-22T07:16:04.552 INFO:tasks.workunit.client.0.smithi063.stdout:6/796: dwrite d3/d6/d2a/d38/da5/fdc [0,4194304] 0 2024-03-22T07:16:04.552 INFO:tasks.workunit.client.1.smithi069.stdout:3/606: dwrite d5/d9/d26/d3f/d56/d8a/da5/fc0 [0,4194304] 0 2024-03-22T07:16:04.552 INFO:tasks.workunit.client.1.smithi069.stdout:3/607: fsync d5/d9/d26/d3f/d54/d9d/fb0 0 2024-03-22T07:16:04.553 INFO:tasks.workunit.client.0.smithi063.stdout:6/797: creat d3/d15/d16/d60/d6d/d4e/d5a/d88/f10e x:0 0 0 2024-03-22T07:16:04.567 INFO:tasks.workunit.client.1.smithi069.stdout:3/608: dread d5/d9/d26/d3f/d35/f37 [0,4194304] 0 2024-03-22T07:16:04.568 INFO:tasks.workunit.client.1.smithi069.stdout:3/609: creat d5/fc7 x:0 0 0 2024-03-22T07:16:04.569 INFO:tasks.workunit.client.1.smithi069.stdout:3/610: truncate d5/d9/d26/f5a 837210 0 2024-03-22T07:16:04.569 INFO:tasks.workunit.client.0.smithi063.stdout:6/798: write d3/d6/d35/f67 [298509,124856] 0 2024-03-22T07:16:04.569 INFO:tasks.workunit.client.1.smithi069.stdout:3/611: readlink d5/l17 0 2024-03-22T07:16:04.570 INFO:tasks.workunit.client.1.smithi069.stdout:3/612: write d5/d9/d26/d3f/d35/fb6 [442111,116676] 0 2024-03-22T07:16:04.570 INFO:tasks.workunit.client.1.smithi069.stdout:3/613: creat d5/d9/d26/d36/fc8 x:0 0 0 2024-03-22T07:16:04.571 INFO:tasks.workunit.client.0.smithi063.stdout:6/799: rename d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/d10a/c74 to d3/d15/d16/d60/d6d/d4c/de8/c10f 0 2024-03-22T07:16:04.572 INFO:tasks.workunit.client.1.smithi069.stdout:3/614: link d5/d9/d26/d3f/d35/f45 d5/d9/d26/d3f/d35/fc9 0 2024-03-22T07:16:04.572 INFO:tasks.workunit.client.1.smithi069.stdout:3/615: chown d5/d9/d26/d36/f52 266721242 1 2024-03-22T07:16:04.572 INFO:tasks.workunit.client.1.smithi069.stdout:3/616: write d5/d9/db/fbb [192090,90657] 0 2024-03-22T07:16:04.580 INFO:tasks.workunit.client.0.smithi063.stdout:9/767: dwrite d8/d19/d20/d22/d5d/f64 [4194304,4194304] 0 2024-03-22T07:16:04.580 INFO:tasks.workunit.client.0.smithi063.stdout:9/768: dread - d8/d19/d20/d22/db2/fb9 zero size 2024-03-22T07:16:04.582 INFO:tasks.workunit.client.1.smithi069.stdout:2/405: dwrite d18/d4d/d34/f5f [0,4194304] 0 2024-03-22T07:16:04.582 INFO:tasks.workunit.client.1.smithi069.stdout:8/719: dwrite d1/d1c/d3e/d88/dcd/d6c/f72 [0,4194304] 0 2024-03-22T07:16:04.582 INFO:tasks.workunit.client.1.smithi069.stdout:3/617: write d5/d9/d23/f6d [2346167,79260] 0 2024-03-22T07:16:04.583 INFO:tasks.workunit.client.0.smithi063.stdout:0/995: dwrite da/de/d17/d25/d3c/d53/d89/ffb [0,4194304] 0 2024-03-22T07:16:04.583 INFO:tasks.workunit.client.0.smithi063.stdout:5/683: dwrite d8/f1c [0,4194304] 0 2024-03-22T07:16:04.583 INFO:tasks.workunit.client.1.smithi069.stdout:1/488: dwrite d1/d2a/f32 [0,4194304] 0 2024-03-22T07:16:04.583 INFO:tasks.workunit.client.1.smithi069.stdout:1/489: creat d1/d13/d36/f91 x:0 0 0 2024-03-22T07:16:04.584 INFO:tasks.workunit.client.1.smithi069.stdout:4/645: dwrite d6/dc/ff [0,4194304] 0 2024-03-22T07:16:04.584 INFO:tasks.workunit.client.1.smithi069.stdout:4/646: write d6/d12/f6f [805688,120668] 0 2024-03-22T07:16:04.584 INFO:tasks.workunit.client.1.smithi069.stdout:4/647: write d6/d12/db2/fbd [49948,45312] 0 2024-03-22T07:16:04.584 INFO:tasks.workunit.client.1.smithi069.stdout:4/648: write d6/dc/f1a [1247189,128179] 0 2024-03-22T07:16:04.584 INFO:tasks.workunit.client.0.smithi063.stdout:0/996: dread da/de/d17/d25/d3c/d64/d110/f117 [0,4194304] 0 2024-03-22T07:16:04.585 INFO:tasks.workunit.client.1.smithi069.stdout:2/406: mkdir d18/d4d/d27/d33/d60/d64/d89/d8b 0 2024-03-22T07:16:04.589 INFO:tasks.workunit.client.1.smithi069.stdout:9/474: sync 2024-03-22T07:16:04.589 INFO:tasks.workunit.client.1.smithi069.stdout:7/508: sync 2024-03-22T07:16:04.589 INFO:tasks.workunit.client.1.smithi069.stdout:6/500: sync 2024-03-22T07:16:04.589 INFO:tasks.workunit.client.1.smithi069.stdout:6/501: readlink d4/dd/d12/l8c 0 2024-03-22T07:16:04.589 INFO:tasks.workunit.client.1.smithi069.stdout:6/502: fdatasync d4/dd/d12/d13/f5e 0 2024-03-22T07:16:04.589 INFO:tasks.workunit.client.1.smithi069.stdout:7/509: write d9/d76/d57/d75/d7a/f8f [539725,21137] 0 2024-03-22T07:16:04.590 INFO:tasks.workunit.client.0.smithi063.stdout:5/684: rename d8/d13/d28/d57/f77 to d8/d13/d1e/d31/daf/fcd 0 2024-03-22T07:16:04.590 INFO:tasks.workunit.client.1.smithi069.stdout:8/720: truncate d1/d1c/fe4 1928989 0 2024-03-22T07:16:04.591 INFO:tasks.workunit.client.0.smithi063.stdout:0/997: mknod da/d49/d12c/c145 0 2024-03-22T07:16:04.594 INFO:tasks.workunit.client.1.smithi069.stdout:3/618: mkdir d5/d9/db/dca 0 2024-03-22T07:16:04.594 INFO:tasks.workunit.client.0.smithi063.stdout:6/800: dread d3/d6/d2a/d38/f49 [0,4194304] 0 2024-03-22T07:16:04.595 INFO:tasks.workunit.client.0.smithi063.stdout:6/801: stat d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb 0 2024-03-22T07:16:04.595 INFO:tasks.workunit.client.0.smithi063.stdout:6/802: readlink d3/d6/l9e 0 2024-03-22T07:16:04.595 INFO:tasks.workunit.client.0.smithi063.stdout:6/803: fsync d3/d15/d16/d60/d6d/d4c/f20 0 2024-03-22T07:16:04.597 INFO:tasks.workunit.client.0.smithi063.stdout:8/880: dread d0/d24/d33/d4c/fd4 [0,4194304] 0 2024-03-22T07:16:04.597 INFO:tasks.workunit.client.0.smithi063.stdout:8/881: write d0/d1b/f1e [1013107,91292] 0 2024-03-22T07:16:04.601 INFO:tasks.workunit.client.1.smithi069.stdout:9/475: mknod d2/dd/d3c/d48/d7f/d86/c9a 0 2024-03-22T07:16:04.601 INFO:tasks.workunit.client.1.smithi069.stdout:9/476: creat d2/d21/d77/d81/f9b x:0 0 0 2024-03-22T07:16:04.603 INFO:tasks.workunit.client.0.smithi063.stdout:9/769: dwrite d8/d19/d20/d22/d56/f71 [0,4194304] 0 2024-03-22T07:16:04.603 INFO:tasks.workunit.client.0.smithi063.stdout:9/770: readlink d8/d15/d2f/lad 0 2024-03-22T07:16:04.603 INFO:tasks.workunit.client.0.smithi063.stdout:9/771: fsync d8/d17/f1d 0 2024-03-22T07:16:04.604 INFO:tasks.workunit.client.1.smithi069.stdout:6/503: rename d4/dd/d12/d44/f7a to d4/dd/d12/d44/f9b 0 2024-03-22T07:16:04.604 INFO:tasks.workunit.client.1.smithi069.stdout:6/504: chown d4/d7/d5b/f70 45 1 2024-03-22T07:16:04.604 INFO:tasks.workunit.client.0.smithi063.stdout:5/685: unlink d8/d13/d1e/d31/lc4 0 2024-03-22T07:16:04.608 INFO:tasks.workunit.client.1.smithi069.stdout:7/510: mkdir d9/d42/da8 0 2024-03-22T07:16:04.611 INFO:tasks.workunit.client.0.smithi063.stdout:8/882: stat d0/d1b/d28/d3e/c8b 0 2024-03-22T07:16:04.614 INFO:tasks.workunit.client.1.smithi069.stdout:3/619: dread d5/f8f [0,4194304] 0 2024-03-22T07:16:04.614 INFO:tasks.workunit.client.0.smithi063.stdout:5/686: creat d8/d13/d24/da4/d8b/fce x:0 0 0 2024-03-22T07:16:04.615 INFO:tasks.workunit.client.0.smithi063.stdout:8/883: mkdir d0/d1b/d28/de8/d128 0 2024-03-22T07:16:04.617 INFO:tasks.workunit.client.0.smithi063.stdout:5/687: write d8/d13/d1e/d31/d64/d65/f71 [178634,25744] 0 2024-03-22T07:16:04.617 INFO:tasks.workunit.client.0.smithi063.stdout:5/688: chown d8/f1c 467281 1 2024-03-22T07:16:04.617 INFO:tasks.workunit.client.0.smithi063.stdout:8/884: mknod d0/d26/d2d/d59/dc0/d106/c129 0 2024-03-22T07:16:04.622 INFO:tasks.workunit.client.1.smithi069.stdout:6/505: creat d4/dd/d41/d78/f9c x:0 0 0 2024-03-22T07:16:04.623 INFO:tasks.workunit.client.1.smithi069.stdout:3/620: symlink d5/d9/d26/d3f/d56/d8a/da5/d7b/db9/lcb 0 2024-03-22T07:16:04.623 INFO:tasks.workunit.client.1.smithi069.stdout:3/621: creat d5/d9/d26/d36/fcc x:0 0 0 2024-03-22T07:16:04.625 INFO:tasks.workunit.client.1.smithi069.stdout:9/477: getdents d2/d71/d90 0 2024-03-22T07:16:04.625 INFO:tasks.workunit.client.1.smithi069.stdout:9/478: dread - d2/d51/f59 zero size 2024-03-22T07:16:04.626 INFO:tasks.workunit.client.1.smithi069.stdout:6/506: mkdir d4/dd/d12/d2b/d72/d73/d9d 0 2024-03-22T07:16:04.628 INFO:tasks.workunit.client.1.smithi069.stdout:9/479: creat d2/dd/d3c/d53/d8b/f9c x:0 0 0 2024-03-22T07:16:04.631 INFO:tasks.workunit.client.1.smithi069.stdout:6/507: unlink d4/dd/d41/l8e 0 2024-03-22T07:16:04.648 INFO:tasks.workunit.client.1.smithi069.stdout:6/508: read d4/f46 [7000117,122049] 0 2024-03-22T07:16:04.648 INFO:tasks.workunit.client.1.smithi069.stdout:6/509: write d4/d7/da/f7c [109039,103162] 0 2024-03-22T07:16:04.654 INFO:tasks.workunit.client.1.smithi069.stdout:1/490: dwrite d1/d2a/d50/d4c/d55/f56 [0,4194304] 0 2024-03-22T07:16:04.654 INFO:tasks.workunit.client.1.smithi069.stdout:1/491: truncate d1/d13/d36/f91 706436 0 2024-03-22T07:16:04.655 INFO:tasks.workunit.client.0.smithi063.stdout:5/689: dwrite d8/d13/d1e/d3c/fa1 [0,4194304] 0 2024-03-22T07:16:04.656 INFO:tasks.workunit.client.1.smithi069.stdout:7/511: dwrite d9/db/d1a/f49 [0,4194304] 0 2024-03-22T07:16:04.656 INFO:tasks.workunit.client.1.smithi069.stdout:4/649: dwrite d6/d9b/d56/fa8 [0,4194304] 0 2024-03-22T07:16:04.656 INFO:tasks.workunit.client.1.smithi069.stdout:2/407: dwrite d18/d4d/d34/f7e [0,4194304] 0 2024-03-22T07:16:04.656 INFO:tasks.workunit.client.1.smithi069.stdout:3/622: dwrite d5/d9/d26/d36/d41/f75 [0,4194304] 0 2024-03-22T07:16:04.657 INFO:tasks.workunit.client.1.smithi069.stdout:3/623: stat d5/d9/d26/d3f/d56/d8a/da5/d7b/dbc 0 2024-03-22T07:16:04.657 INFO:tasks.workunit.client.1.smithi069.stdout:3/624: truncate d5/f5d 6790470 0 2024-03-22T07:16:04.659 INFO:tasks.workunit.client.1.smithi069.stdout:1/492: symlink d1/d2a/d50/d4c/d55/l92 0 2024-03-22T07:16:04.659 INFO:tasks.workunit.client.0.smithi063.stdout:0/998: dwrite da/de/d17/f28 [4194304,4194304] 0 2024-03-22T07:16:04.659 INFO:tasks.workunit.client.0.smithi063.stdout:9/772: dwrite d8/d19/d2b/f5f [4194304,4194304] 0 2024-03-22T07:16:04.660 INFO:tasks.workunit.client.0.smithi063.stdout:9/773: readlink d8/d19/d20/d22/lae 0 2024-03-22T07:16:04.660 INFO:tasks.workunit.client.0.smithi063.stdout:6/804: dwrite d3/d15/d16/d60/d6d/d107/d10b/fa9 [0,4194304] 0 2024-03-22T07:16:04.660 INFO:tasks.workunit.client.0.smithi063.stdout:6/805: dread - d3/d15/d16/d60/fb3 zero size 2024-03-22T07:16:04.660 INFO:tasks.workunit.client.0.smithi063.stdout:5/690: creat d8/d13/d1e/d3c/da0/fcf x:0 0 0 2024-03-22T07:16:04.661 INFO:tasks.workunit.client.0.smithi063.stdout:0/999: mknod da/d49/c146 0 2024-03-22T07:16:04.663 INFO:tasks.workunit.client.1.smithi069.stdout:7/512: mkdir d9/d76/d57/d44/d53/d74/d30/da9 0 2024-03-22T07:16:04.663 INFO:tasks.workunit.client.1.smithi069.stdout:7/513: write d9/d76/d57/d44/d53/d2e/f4d [101286,11563] 0 2024-03-22T07:16:04.663 INFO:tasks.workunit.client.1.smithi069.stdout:7/514: chown d9/d76/d57/d44/f47 22128092 1 2024-03-22T07:16:04.666 INFO:tasks.workunit.client.0.smithi063.stdout:9/774: mkdir d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/da5/dee 0 2024-03-22T07:16:04.666 INFO:tasks.workunit.client.1.smithi069.stdout:2/408: creat d18/d4d/d27/f8c x:0 0 0 2024-03-22T07:16:04.669 INFO:tasks.workunit.client.1.smithi069.stdout:4/650: dread d6/d12/d18/d21/f6a [0,4194304] 0 2024-03-22T07:16:04.669 INFO:tasks.workunit.client.1.smithi069.stdout:4/651: fsync d6/d16/d3a/db4/fc1 0 2024-03-22T07:16:04.671 INFO:tasks.workunit.client.0.smithi063.stdout:6/806: truncate d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/d10a/fb5 3319373 0 2024-03-22T07:16:04.671 INFO:tasks.workunit.client.1.smithi069.stdout:1/493: mkdir d1/d2a/d5e/d93 0 2024-03-22T07:16:04.673 INFO:tasks.workunit.client.0.smithi063.stdout:5/691: symlink d8/dba/ld0 0 2024-03-22T07:16:04.673 INFO:tasks.workunit.client.1.smithi069.stdout:2/409: creat d18/d4d/d27/d33/d60/d64/d89/f8d x:0 0 0 2024-03-22T07:16:04.673 INFO:tasks.workunit.client.1.smithi069.stdout:2/410: chown d18/d4d/d34/f4a 37741533 1 2024-03-22T07:16:04.674 INFO:tasks.workunit.client.1.smithi069.stdout:4/652: symlink d6/d12/d18/ldc 0 2024-03-22T07:16:04.676 INFO:tasks.workunit.client.0.smithi063.stdout:9/775: mknod d8/d19/d20/d22/de1/dd7/cef 0 2024-03-22T07:16:04.677 INFO:tasks.workunit.client.1.smithi069.stdout:4/653: link d6/d16/f71 d6/d16/d83/fdd 0 2024-03-22T07:16:04.677 INFO:tasks.workunit.client.0.smithi063.stdout:1/903: sync 2024-03-22T07:16:04.677 INFO:tasks.workunit.client.0.smithi063.stdout:3/858: sync 2024-03-22T07:16:04.677 INFO:tasks.workunit.client.0.smithi063.stdout:4/745: sync 2024-03-22T07:16:04.677 INFO:tasks.workunit.client.0.smithi063.stdout:4/746: stat d1/d5/dc/d95/da4/fd3 0 2024-03-22T07:16:04.677 INFO:tasks.workunit.client.0.smithi063.stdout:7/654: sync 2024-03-22T07:16:04.678 INFO:tasks.workunit.client.0.smithi063.stdout:5/692: creat d8/d13/d28/fd1 x:0 0 0 2024-03-22T07:16:04.678 INFO:tasks.workunit.client.0.smithi063.stdout:5/693: fsync d8/dba/fcc 0 2024-03-22T07:16:04.679 INFO:tasks.workunit.client.1.smithi069.stdout:9/480: rename d2/d21/d77/d81 to d2/dd/d3c/d53/d9d 0 2024-03-22T07:16:04.679 INFO:tasks.workunit.client.1.smithi069.stdout:9/481: write d2/dd/d3c/d2e/f4b [81847,99074] 0 2024-03-22T07:16:04.681 INFO:tasks.workunit.client.0.smithi063.stdout:4/747: mknod d1/d5/d3c/d65/de4/cfb 0 2024-03-22T07:16:04.682 INFO:tasks.workunit.client.1.smithi069.stdout:2/411: dread f6 [0,4194304] 0 2024-03-22T07:16:04.685 INFO:tasks.workunit.client.0.smithi063.stdout:7/655: link d5/da/d5f/f8a d5/da/d28/d4b/da8/dbc/d62/d73/fd7 0 2024-03-22T07:16:04.685 INFO:tasks.workunit.client.1.smithi069.stdout:9/482: creat d2/dd/d3c/d48/d7c/f9e x:0 0 0 2024-03-22T07:16:04.685 INFO:tasks.workunit.client.1.smithi069.stdout:9/483: creat d2/f9f x:0 0 0 2024-03-22T07:16:04.685 INFO:tasks.workunit.client.1.smithi069.stdout:9/484: dread - d2/d71/d90/f95 zero size 2024-03-22T07:16:04.687 INFO:tasks.workunit.client.1.smithi069.stdout:1/494: rename d1/d2a/l3a to d1/d2a/d50/l94 0 2024-03-22T07:16:04.688 INFO:tasks.workunit.client.0.smithi063.stdout:7/656: truncate d5/da/d28/d4b/da8/dbc/f2a 3466896 0 2024-03-22T07:16:04.690 INFO:tasks.workunit.client.0.smithi063.stdout:9/776: dread d8/fd [0,4194304] 0 2024-03-22T07:16:04.690 INFO:tasks.workunit.client.0.smithi063.stdout:9/777: fdatasync d8/d19/d20/d22/f65 0 2024-03-22T07:16:04.690 INFO:tasks.workunit.client.0.smithi063.stdout:7/657: mkdir d5/d16/d43/d9d/dd8 0 2024-03-22T07:16:04.691 INFO:tasks.workunit.client.1.smithi069.stdout:1/495: mkdir d1/d13/d36/d5b/d95 0 2024-03-22T07:16:04.693 INFO:tasks.workunit.client.1.smithi069.stdout:1/496: creat d1/d2a/d50/d88/d41/d1e/f96 x:0 0 0 2024-03-22T07:16:04.694 INFO:tasks.workunit.client.0.smithi063.stdout:1/904: read d4/df/d13/d2b/d6c/da8/fc5 [3602437,21188] 0 2024-03-22T07:16:04.694 INFO:tasks.workunit.client.0.smithi063.stdout:1/905: dread - d4/df/de7/f100 zero size 2024-03-22T07:16:04.695 INFO:tasks.workunit.client.0.smithi063.stdout:1/906: fsync d4/df/d13/d39/d6f/f72 0 2024-03-22T07:16:04.695 INFO:tasks.workunit.client.0.smithi063.stdout:2/726: sync 2024-03-22T07:16:04.697 INFO:tasks.workunit.client.0.smithi063.stdout:2/727: chown d0/c19 1582679054 1 2024-03-22T07:16:04.701 INFO:tasks.workunit.client.0.smithi063.stdout:9/778: write d8/d19/d20/d22/d29/faa [2303249,16532] 0 2024-03-22T07:16:04.702 INFO:tasks.workunit.client.0.smithi063.stdout:1/907: dread d4/df/fbe [0,4194304] 0 2024-03-22T07:16:04.708 INFO:tasks.workunit.client.0.smithi063.stdout:6/807: dwrite d3/d6/d2a/d38/da5/f90 [4194304,4194304] 0 2024-03-22T07:16:04.708 INFO:tasks.workunit.client.0.smithi063.stdout:6/808: stat d3/d6 0 2024-03-22T07:16:04.708 INFO:tasks.workunit.client.0.smithi063.stdout:6/809: stat d3/d6/d2a/d38/da5/fdc 0 2024-03-22T07:16:04.708 INFO:tasks.workunit.client.0.smithi063.stdout:6/810: chown d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/ld7 429665 1 2024-03-22T07:16:04.708 INFO:tasks.workunit.client.0.smithi063.stdout:6/811: truncate d3/d15/d16/d69/d8c/fe2 672672 0 2024-03-22T07:16:04.711 INFO:tasks.workunit.client.1.smithi069.stdout:7/515: dwrite d9/d76/d57/d44/d53/d74/d30/d3d/f7b [0,4194304] 0 2024-03-22T07:16:04.712 INFO:tasks.workunit.client.1.smithi069.stdout:7/516: chown d9/d76/d57/d44/d53/d1d/f87 2478734 1 2024-03-22T07:16:04.712 INFO:tasks.workunit.client.1.smithi069.stdout:1/497: dread d1/d2a/d50/f40 [0,4194304] 0 2024-03-22T07:16:04.712 INFO:tasks.workunit.client.1.smithi069.stdout:1/498: readlink d1/d13/l49 0 2024-03-22T07:16:04.712 INFO:tasks.workunit.client.1.smithi069.stdout:1/499: creat d1/d2a/d50/d4c/d55/f97 x:0 0 0 2024-03-22T07:16:04.712 INFO:tasks.workunit.client.1.smithi069.stdout:1/500: truncate d1/d13/d36/d5b/f87 414872 0 2024-03-22T07:16:04.713 INFO:tasks.workunit.client.0.smithi063.stdout:6/812: mkdir d3/d15/d110 0 2024-03-22T07:16:04.714 INFO:tasks.workunit.client.0.smithi063.stdout:6/813: chown d3/d15/d16/d60/d6d 0 1 2024-03-22T07:16:04.714 INFO:tasks.workunit.client.0.smithi063.stdout:6/814: creat d3/d15/d16/f111 x:0 0 0 2024-03-22T07:16:04.714 INFO:tasks.workunit.client.0.smithi063.stdout:6/815: write d3/d15/d16/d60/d6d/d4e/d5a/d88/fc1 [1893653,2220] 0 2024-03-22T07:16:04.714 INFO:tasks.workunit.client.0.smithi063.stdout:6/816: readlink d3/l9b 0 2024-03-22T07:16:04.714 INFO:tasks.workunit.client.0.smithi063.stdout:6/817: read d3/d6/d2a/f5e [84478,109935] 0 2024-03-22T07:16:04.714 INFO:tasks.workunit.client.0.smithi063.stdout:6/818: link d3/d15/f17 d3/d15/da3/dc2/f112 0 2024-03-22T07:16:04.715 INFO:tasks.workunit.client.0.smithi063.stdout:6/819: dread - d3/d6/d2a/d38/da5/d32/fdb zero size 2024-03-22T07:16:04.715 INFO:tasks.workunit.client.0.smithi063.stdout:1/908: dread d4/df/d13/d39/fa0 [0,4194304] 0 2024-03-22T07:16:04.715 INFO:tasks.workunit.client.0.smithi063.stdout:1/909: stat d4/d71/d88/da1 0 2024-03-22T07:16:04.715 INFO:tasks.workunit.client.0.smithi063.stdout:1/910: symlink d4/d14/d123/l136 0 2024-03-22T07:16:04.717 INFO:tasks.workunit.client.0.smithi063.stdout:6/820: write d3/d6/d2a/d38/f6c [3952501,83041] 0 2024-03-22T07:16:04.717 INFO:tasks.workunit.client.0.smithi063.stdout:1/911: rename d4/c134 to d4/df/d13/d2b/d6c/d85/c137 0 2024-03-22T07:16:04.718 INFO:tasks.workunit.client.1.smithi069.stdout:1/501: dread d1/d2a/d50/d88/f4d [0,4194304] 0 2024-03-22T07:16:04.719 INFO:tasks.workunit.client.1.smithi069.stdout:1/502: mknod d1/d13/d36/c98 0 2024-03-22T07:16:04.720 INFO:tasks.workunit.client.0.smithi063.stdout:1/912: mkdir d4/d71/dbb/dc0/dd8/d138 0 2024-03-22T07:16:04.721 INFO:tasks.workunit.client.0.smithi063.stdout:6/821: dread d3/d15/d16/d60/d6d/d4c/f75 [0,4194304] 0 2024-03-22T07:16:04.721 INFO:tasks.workunit.client.1.smithi069.stdout:4/654: dwrite d6/d9b/d1f/d3f/f9d [0,4194304] 0 2024-03-22T07:16:04.722 INFO:tasks.workunit.client.1.smithi069.stdout:4/655: fdatasync d6/dc/ff 0 2024-03-22T07:16:04.722 INFO:tasks.workunit.client.0.smithi063.stdout:3/859: write da/d16/d3f/f50 [732227,31691] 0 2024-03-22T07:16:04.722 INFO:tasks.workunit.client.0.smithi063.stdout:3/860: write da/d63/d65/d81/fe8 [907743,104150] 0 2024-03-22T07:16:04.722 INFO:tasks.workunit.client.0.smithi063.stdout:4/748: dwrite d1/d5/dc/fa7 [0,4194304] 0 2024-03-22T07:16:04.722 INFO:tasks.workunit.client.0.smithi063.stdout:6/822: mknod d3/d6/d2a/d38/da5/c113 0 2024-03-22T07:16:04.722 INFO:tasks.workunit.client.0.smithi063.stdout:6/823: readlink d3/d15/da3/lb6 0 2024-03-22T07:16:04.723 INFO:tasks.workunit.client.1.smithi069.stdout:7/517: dread d9/d76/d57/d44/f9b [0,4194304] 0 2024-03-22T07:16:04.723 INFO:tasks.workunit.client.1.smithi069.stdout:1/503: rename d1/f35 to d1/d2a/d50/d88/d41/d64/f99 0 2024-03-22T07:16:04.724 INFO:tasks.workunit.client.0.smithi063.stdout:9/779: dwrite d8/d15/d63/fb3 [0,4194304] 0 2024-03-22T07:16:04.724 INFO:tasks.workunit.client.0.smithi063.stdout:9/780: write d8/d19/d20/fa0 [6719135,105022] 0 2024-03-22T07:16:04.724 INFO:tasks.workunit.client.0.smithi063.stdout:5/694: dwrite d8/f44 [4194304,4194304] 0 2024-03-22T07:16:04.725 INFO:tasks.workunit.client.0.smithi063.stdout:3/861: dread da/d16/d3f/f4c [0,4194304] 0 2024-03-22T07:16:04.725 INFO:tasks.workunit.client.1.smithi069.stdout:4/656: unlink d6/d16/c84 0 2024-03-22T07:16:04.726 INFO:tasks.workunit.client.1.smithi069.stdout:1/504: mkdir d1/d2a/d84/d9a 0 2024-03-22T07:16:04.727 INFO:tasks.workunit.client.1.smithi069.stdout:1/505: readlink d1/d2a/d50/d88/d41/l59 0 2024-03-22T07:16:04.731 INFO:tasks.workunit.client.0.smithi063.stdout:4/749: mknod d1/d5/dc2/cfc 0 2024-03-22T07:16:04.731 INFO:tasks.workunit.client.0.smithi063.stdout:4/750: dread - d1/d5/dc/d95/da4/ff0 zero size 2024-03-22T07:16:04.731 INFO:tasks.workunit.client.0.smithi063.stdout:6/824: mkdir d3/d15/d16/d60/d6d/d4e/d5a/d88/df5/d114 0 2024-03-22T07:16:04.731 INFO:tasks.workunit.client.1.smithi069.stdout:5/428: sync 2024-03-22T07:16:04.731 INFO:tasks.workunit.client.0.smithi063.stdout:1/913: dread d4/d5a/ff8 [0,4194304] 0 2024-03-22T07:16:04.733 INFO:tasks.workunit.client.1.smithi069.stdout:5/429: rmdir d1/d6/d26/d33/d5d 39 2024-03-22T07:16:04.735 INFO:tasks.workunit.client.1.smithi069.stdout:5/430: symlink d1/d6/d26/d33/d5d/d72/l97 0 2024-03-22T07:16:04.735 INFO:tasks.workunit.client.1.smithi069.stdout:4/657: dread d6/f30 [0,4194304] 0 2024-03-22T07:16:04.736 INFO:tasks.workunit.client.0.smithi063.stdout:1/914: write d4/df/f44 [241940,28889] 0 2024-03-22T07:16:04.736 INFO:tasks.workunit.client.0.smithi063.stdout:1/915: creat d4/d5a/d91/f139 x:0 0 0 2024-03-22T07:16:04.736 INFO:tasks.workunit.client.0.smithi063.stdout:9/781: mkdir d8/d19/d20/d22/d29/df0 0 2024-03-22T07:16:04.736 INFO:tasks.workunit.client.1.smithi069.stdout:5/431: symlink d1/d7/d8/l98 0 2024-03-22T07:16:04.736 INFO:tasks.workunit.client.1.smithi069.stdout:5/432: chown d1/d6/d26/d33/f80 192526 1 2024-03-22T07:16:04.739 INFO:tasks.workunit.client.0.smithi063.stdout:3/862: mknod da/d16/d26/d69/c11a 0 2024-03-22T07:16:04.740 INFO:tasks.workunit.client.0.smithi063.stdout:3/863: read da/d16/d26/f29 [728993,1420] 0 2024-03-22T07:16:04.740 INFO:tasks.workunit.client.1.smithi069.stdout:4/658: dread d6/d12/f76 [0,4194304] 0 2024-03-22T07:16:04.740 INFO:tasks.workunit.client.1.smithi069.stdout:4/659: creat d6/d16/d3a/db4/fde x:0 0 0 2024-03-22T07:16:04.740 INFO:tasks.workunit.client.1.smithi069.stdout:4/660: readlink d6/d16/d83/dd0/ld9 0 2024-03-22T07:16:04.744 INFO:tasks.workunit.client.0.smithi063.stdout:9/782: mkdir d8/d19/d20/d22/d29/d31/d53/d6a/da2/df1 0 2024-03-22T07:16:04.744 INFO:tasks.workunit.client.0.smithi063.stdout:1/916: dread d4/f3b [0,4194304] 0 2024-03-22T07:16:04.745 INFO:tasks.workunit.client.1.smithi069.stdout:7/518: write f2 [6518912,18239] 0 2024-03-22T07:16:04.745 INFO:tasks.workunit.client.1.smithi069.stdout:7/519: dread - d9/d76/d57/d44/d53/d2e/fa6 zero size 2024-03-22T07:16:04.745 INFO:tasks.workunit.client.1.smithi069.stdout:7/520: write d9/d76/d57/d44/d53/f60 [313825,89390] 0 2024-03-22T07:16:04.746 INFO:tasks.workunit.client.0.smithi063.stdout:3/864: mknod da/d63/d65/d81/d9b/c11b 0 2024-03-22T07:16:04.746 INFO:tasks.workunit.client.0.smithi063.stdout:1/917: creat d4/d71/dbb/dc0/dd8/d138/f13a x:0 0 0 2024-03-22T07:16:04.747 INFO:tasks.workunit.client.0.smithi063.stdout:1/918: mknod d4/d71/dbb/dc0/dd8/dee/c13b 0 2024-03-22T07:16:04.748 INFO:tasks.workunit.client.1.smithi069.stdout:7/521: truncate d9/db/f17 819935 0 2024-03-22T07:16:04.748 INFO:tasks.workunit.client.1.smithi069.stdout:7/522: read d9/d76/d57/d44/d53/f6d [1937157,124877] 0 2024-03-22T07:16:04.749 INFO:tasks.workunit.client.0.smithi063.stdout:1/919: rmdir d4/df/d13/d39/d63 39 2024-03-22T07:16:04.749 INFO:tasks.workunit.client.0.smithi063.stdout:1/920: write d4/df/f48 [4554799,55214] 0 2024-03-22T07:16:04.750 INFO:tasks.workunit.client.0.smithi063.stdout:1/921: creat d4/d8b/da7/f13c x:0 0 0 2024-03-22T07:16:04.751 INFO:tasks.workunit.client.0.smithi063.stdout:1/922: symlink d4/d14/l13d 0 2024-03-22T07:16:04.751 INFO:tasks.workunit.client.0.smithi063.stdout:6/825: write d3/d15/f23 [613120,35159] 0 2024-03-22T07:16:04.751 INFO:tasks.workunit.client.0.smithi063.stdout:6/826: read d3/d6/f2c [4132931,48300] 0 2024-03-22T07:16:04.754 INFO:tasks.workunit.client.1.smithi069.stdout:7/523: creat d9/d76/d57/d44/d53/d1d/faa x:0 0 0 2024-03-22T07:16:04.754 INFO:tasks.workunit.client.0.smithi063.stdout:1/923: getdents d4/df/d13/d2b/d3a 0 2024-03-22T07:16:04.754 INFO:tasks.workunit.client.0.smithi063.stdout:1/924: read d4/df/f15 [102242,109701] 0 2024-03-22T07:16:04.754 INFO:tasks.workunit.client.0.smithi063.stdout:6/827: link d3/d6/d2a/d38/da5/c113 d3/d15/d16/d60/d6d/d4e/d5a/d88/df5/c115 0 2024-03-22T07:16:04.754 INFO:tasks.workunit.client.0.smithi063.stdout:6/828: stat d3/d15/d16/d60/d6d/d4c/f20 0 2024-03-22T07:16:04.756 INFO:tasks.workunit.client.1.smithi069.stdout:4/661: dread d6/d16/f57 [0,4194304] 0 2024-03-22T07:16:04.756 INFO:tasks.workunit.client.1.smithi069.stdout:4/662: write d6/d12/f33 [1652502,110390] 0 2024-03-22T07:16:04.757 INFO:tasks.workunit.client.1.smithi069.stdout:7/524: truncate d9/d76/d57/d44/d53/f6d 904758 0 2024-03-22T07:16:04.757 INFO:tasks.workunit.client.0.smithi063.stdout:1/925: dread d4/d71/d88/da1/fd2 [0,4194304] 0 2024-03-22T07:16:04.757 INFO:tasks.workunit.client.0.smithi063.stdout:1/926: chown d4/df/d13/d2b/d6c/d85/d9e/led 15 1 2024-03-22T07:16:04.758 INFO:tasks.workunit.client.0.smithi063.stdout:6/829: rename d3/d15/d16/d60/d6d/d4c/l53 to d3/d6/d2a/d38/da5/d32/l116 0 2024-03-22T07:16:04.758 INFO:tasks.workunit.client.0.smithi063.stdout:6/830: chown d3/d15/d16/d60/d6d/d4e/d5a/d88/fb2 232 1 2024-03-22T07:16:04.758 INFO:tasks.workunit.client.0.smithi063.stdout:6/831: dread - d3/d15/d16/d60/fcb zero size 2024-03-22T07:16:04.758 INFO:tasks.workunit.client.0.smithi063.stdout:6/832: write d3/d15/d16/d60/d6d/d4e/d5a/dd4/ffa [220288,67130] 0 2024-03-22T07:16:04.759 INFO:tasks.workunit.client.1.smithi069.stdout:7/525: mkdir d9/db/d48/dab 0 2024-03-22T07:16:04.761 INFO:tasks.workunit.client.0.smithi063.stdout:1/927: rename d4/df/d13/d2b/d86/f11f to d4/f13e 0 2024-03-22T07:16:04.761 INFO:tasks.workunit.client.1.smithi069.stdout:4/663: dread d6/d9b/d1f/f62 [0,4194304] 0 2024-03-22T07:16:04.762 INFO:tasks.workunit.client.0.smithi063.stdout:6/833: symlink d3/d15/d95/l117 0 2024-03-22T07:16:04.762 INFO:tasks.workunit.client.0.smithi063.stdout:5/695: dwrite d8/d13/d1e/d3c/da0/fb9 [0,4194304] 0 2024-03-22T07:16:04.764 INFO:tasks.workunit.client.1.smithi069.stdout:4/664: link d6/dc/d23/d59/l67 d6/d9b/ldf 0 2024-03-22T07:16:04.764 INFO:tasks.workunit.client.1.smithi069.stdout:4/665: chown d6/dc/d23/f3c 238856 1 2024-03-22T07:16:04.764 INFO:tasks.workunit.client.1.smithi069.stdout:4/666: write d6/d16/d3a/fcb [376531,114298] 0 2024-03-22T07:16:04.765 INFO:tasks.workunit.client.1.smithi069.stdout:4/667: write d6/dc/d23/d59/fca [185114,91466] 0 2024-03-22T07:16:04.765 INFO:tasks.workunit.client.1.smithi069.stdout:4/668: fsync d6/d9b/d1f/d3f/d51/f91 0 2024-03-22T07:16:04.765 INFO:tasks.workunit.client.1.smithi069.stdout:4/669: fdatasync d6/d12/d18/f45 0 2024-03-22T07:16:04.765 INFO:tasks.workunit.client.1.smithi069.stdout:4/670: dread d6/d16/d68/fa9 [0,4194304] 0 2024-03-22T07:16:04.766 INFO:tasks.workunit.client.0.smithi063.stdout:6/834: creat d3/d15/d16/d60/d6d/d107/f118 x:0 0 0 2024-03-22T07:16:04.766 INFO:tasks.workunit.client.0.smithi063.stdout:3/865: fdatasync da/d16/d3f/f50 0 2024-03-22T07:16:04.768 INFO:tasks.workunit.client.1.smithi069.stdout:4/671: unlink d6/d9b/d1f/d3f/d51/f91 0 2024-03-22T07:16:04.768 INFO:tasks.workunit.client.1.smithi069.stdout:4/672: fsync d6/d12/d18/d21/f6a 0 2024-03-22T07:16:04.769 INFO:tasks.workunit.client.1.smithi069.stdout:4/673: dread d6/d16/d68/f78 [0,4194304] 0 2024-03-22T07:16:04.769 INFO:tasks.workunit.client.1.smithi069.stdout:4/674: fdatasync d6/d9b/d1f/d58/f85 0 2024-03-22T07:16:04.770 INFO:tasks.workunit.client.0.smithi063.stdout:6/835: rmdir d3/d15/d95 39 2024-03-22T07:16:04.775 INFO:tasks.workunit.client.1.smithi069.stdout:1/506: dwrite d1/d2a/d50/d88/f38 [0,4194304] 0 2024-03-22T07:16:04.776 INFO:tasks.workunit.client.0.smithi063.stdout:6/836: truncate d3/d6/f3e 924748 0 2024-03-22T07:16:04.776 INFO:tasks.workunit.client.1.smithi069.stdout:5/433: rename d1/d7/d8 to d1/d6/d86/d99 0 2024-03-22T07:16:04.777 INFO:tasks.workunit.client.1.smithi069.stdout:5/434: chown d1/d10/d44/d50 271070 1 2024-03-22T07:16:04.779 INFO:tasks.workunit.client.1.smithi069.stdout:7/526: dread d9/d76/d57/d44/d53/d74/f91 [0,4194304] 0 2024-03-22T07:16:04.780 INFO:tasks.workunit.client.1.smithi069.stdout:7/527: creat d9/d76/d57/d44/fac x:0 0 0 2024-03-22T07:16:04.782 INFO:tasks.workunit.client.1.smithi069.stdout:1/507: mkdir d1/d2a/d5e/d9b 0 2024-03-22T07:16:04.782 INFO:tasks.workunit.client.1.smithi069.stdout:1/508: dread - d1/d2a/d50/d88/d41/d64/f85 zero size 2024-03-22T07:16:04.783 INFO:tasks.workunit.client.1.smithi069.stdout:5/435: truncate d1/d6/d21/f79 448170 0 2024-03-22T07:16:04.785 INFO:tasks.workunit.client.1.smithi069.stdout:5/436: truncate d1/d6/d86/d99/f12 3147866 0 2024-03-22T07:16:04.785 INFO:tasks.workunit.client.0.smithi063.stdout:3/866: dwrite da/d16/d26/fdc [0,4194304] 0 2024-03-22T07:16:04.786 INFO:tasks.workunit.client.0.smithi063.stdout:3/867: stat da/d16/c103 0 2024-03-22T07:16:04.786 INFO:tasks.workunit.client.1.smithi069.stdout:5/437: rename d1/f3 to d1/d7/d3d/f9a 0 2024-03-22T07:16:04.787 INFO:tasks.workunit.client.0.smithi063.stdout:6/837: rename d3/d6/c5c to d3/d6/d2a/d38/c119 0 2024-03-22T07:16:04.791 INFO:tasks.workunit.client.1.smithi069.stdout:1/509: write d1/d2a/d50/d88/d41/d64/f7e [56706,130722] 0 2024-03-22T07:16:04.792 INFO:tasks.workunit.client.1.smithi069.stdout:1/510: getdents d1/d13/d36/d5b/d95 0 2024-03-22T07:16:04.792 INFO:tasks.workunit.client.1.smithi069.stdout:1/511: chown d1/d2a/d50/d4c/c77 46481 1 2024-03-22T07:16:04.792 INFO:tasks.workunit.client.1.smithi069.stdout:1/512: readlink d1/l2c 0 2024-03-22T07:16:04.792 INFO:tasks.workunit.client.1.smithi069.stdout:1/513: getdents d1/d2a/d50/d4c/d55/d5f 0 2024-03-22T07:16:04.835 INFO:tasks.workunit.client.0.smithi063.stdout:4/751: dwrite d1/d5/d43/f39 [0,4194304] 0 2024-03-22T07:16:04.835 INFO:tasks.workunit.client.0.smithi063.stdout:4/752: write d1/d5/d43/d27/f4b [415685,30873] 0 2024-03-22T07:16:04.835 INFO:tasks.workunit.client.0.smithi063.stdout:4/753: stat d1/d5/d3c/d8f/dbb/cc5 0 2024-03-22T07:16:04.835 INFO:tasks.workunit.client.0.smithi063.stdout:4/754: chown d1/d5/d43/f31 31318 1 2024-03-22T07:16:04.837 INFO:tasks.workunit.client.1.smithi069.stdout:5/438: dwrite d1/d6/d26/d33/f4c [0,4194304] 0 2024-03-22T07:16:04.838 INFO:tasks.workunit.client.0.smithi063.stdout:4/755: mkdir d1/d5/d43/d27/d2c/d4a/d78/dfd 0 2024-03-22T07:16:04.839 INFO:tasks.workunit.client.1.smithi069.stdout:5/439: link d1/d7/f59 d1/d6/d88/f9b 0 2024-03-22T07:16:04.839 INFO:tasks.workunit.client.0.smithi063.stdout:4/756: symlink d1/d5/d43/d27/d2c/d4a/d80/lfe 0 2024-03-22T07:16:04.840 INFO:tasks.workunit.client.1.smithi069.stdout:5/440: mkdir d1/d6/d26/d3e/d9c 0 2024-03-22T07:16:04.840 INFO:tasks.workunit.client.0.smithi063.stdout:4/757: rmdir d1/d5/d3c/d65/de4 39 2024-03-22T07:16:04.841 INFO:tasks.workunit.client.1.smithi069.stdout:5/441: unlink d1/d6/ff 0 2024-03-22T07:16:04.842 INFO:tasks.workunit.client.0.smithi063.stdout:4/758: rename d1/d5/dc/d95/d59/fbd to d1/d5/d43/d27/d2c/df4/fff 0 2024-03-22T07:16:04.842 INFO:tasks.workunit.client.0.smithi063.stdout:4/759: stat d1/d5/f9 0 2024-03-22T07:16:04.842 INFO:tasks.workunit.client.0.smithi063.stdout:4/760: write d1/d5/d43/d27/d2c/d4a/f9d [5063598,22824] 0 2024-03-22T07:16:04.842 INFO:tasks.workunit.client.0.smithi063.stdout:4/761: fsync d1/d5/d43/d27/d2c/d3d/d54/fa6 0 2024-03-22T07:16:04.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:04 smithi063 ceph-mon[29200]: [22/Mar/2024:07:16:02] ENGINE Bus STARTING 2024-03-22T07:16:04.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:04 smithi063 ceph-mon[29200]: [22/Mar/2024:07:16:03] ENGINE Serving on https://172.21.15.69:7150 2024-03-22T07:16:04.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:04 smithi063 ceph-mon[29200]: [22/Mar/2024:07:16:03] ENGINE Serving on http://172.21.15.69:8765 2024-03-22T07:16:04.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:04 smithi063 ceph-mon[29200]: [22/Mar/2024:07:16:03] ENGINE Bus STARTED 2024-03-22T07:16:04.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:04 smithi063 ceph-mon[29200]: mgrmap e23: smithi069.gfziwh(active, since 3s) 2024-03-22T07:16:04.886 INFO:tasks.workunit.client.0.smithi063.stdout:4/762: dwrite d1/d5/d43/d27/d2c/d4a/d80/fa8 [0,4194304] 0 2024-03-22T07:16:04.889 INFO:tasks.workunit.client.0.smithi063.stdout:4/763: symlink d1/d5/dc/dda/l100 0 2024-03-22T07:16:04.889 INFO:tasks.workunit.client.0.smithi063.stdout:4/764: mknod d1/d5/d43/d27/d2c/d4a/d78/dfd/c101 0 2024-03-22T07:16:04.890 INFO:tasks.workunit.client.0.smithi063.stdout:4/765: mkdir d1/d5/dc2/d102 0 2024-03-22T07:16:04.891 INFO:tasks.workunit.client.0.smithi063.stdout:4/766: link d1/d5/d43/d27/d2c/ldd d1/d5/d43/d27/d2c/df4/l103 0 2024-03-22T07:16:04.906 INFO:tasks.workunit.client.0.smithi063.stdout:9/783: dwrite d8/d17/f43 [0,4194304] 0 2024-03-22T07:16:04.912 INFO:tasks.workunit.client.0.smithi063.stdout:4/767: dread d1/d5/dc/f41 [0,4194304] 0 2024-03-22T07:16:04.916 INFO:tasks.workunit.client.0.smithi063.stdout:4/768: mkdir d1/d5/dc/d95/da4/d104 0 2024-03-22T07:16:04.916 INFO:tasks.workunit.client.0.smithi063.stdout:4/769: write d1/d5/d3c/fe9 [761289,73542] 0 2024-03-22T07:16:04.918 INFO:tasks.workunit.client.0.smithi063.stdout:9/784: dread d8/d19/d20/d22/d56/f71 [0,4194304] 0 2024-03-22T07:16:04.919 INFO:tasks.workunit.client.0.smithi063.stdout:1/928: dwrite d4/df/d13/d39/d63/f77 [0,4194304] 0 2024-03-22T07:16:04.919 INFO:tasks.workunit.client.0.smithi063.stdout:1/929: creat d4/df/f13f x:0 0 0 2024-03-22T07:16:04.921 INFO:tasks.workunit.client.0.smithi063.stdout:4/770: mkdir d1/d5/d43/d27/d2c/d105 0 2024-03-22T07:16:04.923 INFO:tasks.workunit.client.0.smithi063.stdout:9/785: mknod d8/d15/d2f/cf2 0 2024-03-22T07:16:04.923 INFO:tasks.workunit.client.0.smithi063.stdout:9/786: fsync d8/fd 0 2024-03-22T07:16:04.926 INFO:tasks.workunit.client.0.smithi063.stdout:1/930: symlink d4/d71/dbb/dc0/dd8/dee/l140 0 2024-03-22T07:16:04.929 INFO:tasks.workunit.client.0.smithi063.stdout:4/771: dread d1/d5/d85/f8a [0,4194304] 0 2024-03-22T07:16:04.929 INFO:tasks.workunit.client.0.smithi063.stdout:4/772: chown d1/d5/dc/dda/d79/daa/fb7 136381 1 2024-03-22T07:16:04.929 INFO:tasks.workunit.client.0.smithi063.stdout:4/773: truncate d1/d5/dc/dda/d79/daa/fb6 269910 0 2024-03-22T07:16:04.929 INFO:tasks.workunit.client.0.smithi063.stdout:9/787: rename d8/d19/d20/d22/d29/d31/d53/d6a/cc2 to d8/d19/d20/d22/d5d/d76/d95/cf3 0 2024-03-22T07:16:04.929 INFO:tasks.workunit.client.0.smithi063.stdout:9/788: write d8/d19/d20/d22/f27 [1473455,78301] 0 2024-03-22T07:16:04.930 INFO:tasks.workunit.client.0.smithi063.stdout:1/931: creat d4/df/d13/d11d/f141 x:0 0 0 2024-03-22T07:16:04.930 INFO:tasks.workunit.client.0.smithi063.stdout:1/932: fdatasync d4/df/d13/d39/d6f/dd9/feb 0 2024-03-22T07:16:04.930 INFO:tasks.workunit.client.0.smithi063.stdout:1/933: stat d4/d71/d88/da1/l12c 0 2024-03-22T07:16:04.933 INFO:tasks.workunit.client.1.smithi069.stdout:4/675: dwrite d6/dc/d23/f3c [0,4194304] 0 2024-03-22T07:16:04.934 INFO:tasks.workunit.client.1.smithi069.stdout:4/676: creat d6/d16/d83/fe0 x:0 0 0 2024-03-22T07:16:04.935 INFO:tasks.workunit.client.1.smithi069.stdout:7/528: dwrite d9/d76/d57/d44/d53/d2e/f7c [0,4194304] 0 2024-03-22T07:16:04.936 INFO:tasks.workunit.client.0.smithi063.stdout:9/789: mkdir d8/d19/d39/df4 0 2024-03-22T07:16:04.937 INFO:tasks.workunit.client.0.smithi063.stdout:9/790: chown d8/d19/d20/fa0 8151 1 2024-03-22T07:16:04.937 INFO:tasks.workunit.client.0.smithi063.stdout:9/791: symlink d8/d19/de2/lf5 0 2024-03-22T07:16:04.937 INFO:tasks.workunit.client.0.smithi063.stdout:9/792: creat d8/d19/d2b/ff6 x:0 0 0 2024-03-22T07:16:04.939 INFO:tasks.workunit.client.1.smithi069.stdout:4/677: mkdir d6/dc4/de1 0 2024-03-22T07:16:04.940 INFO:tasks.workunit.client.1.smithi069.stdout:7/529: getdents d9/da1 0 2024-03-22T07:16:04.940 INFO:tasks.workunit.client.0.smithi063.stdout:3/868: dwrite da/d16/d89/da9/dd3/f110 [0,4194304] 0 2024-03-22T07:16:04.942 INFO:tasks.workunit.client.1.smithi069.stdout:1/514: dwrite d1/d2a/f32 [0,4194304] 0 2024-03-22T07:16:04.942 INFO:tasks.workunit.client.1.smithi069.stdout:1/515: truncate d1/f3b 1148534 0 2024-03-22T07:16:04.947 INFO:tasks.workunit.client.1.smithi069.stdout:1/516: rename d1/d2a/f8c to d1/d2a/d50/d4c/d55/d5f/f9c 0 2024-03-22T07:16:04.947 INFO:tasks.workunit.client.0.smithi063.stdout:6/838: dwrite d3/d6/d2a/d38/da5/f63 [4194304,4194304] 0 2024-03-22T07:16:04.948 INFO:tasks.workunit.client.1.smithi069.stdout:4/678: write d6/d9b/f14 [463430,23275] 0 2024-03-22T07:16:04.951 INFO:tasks.workunit.client.1.smithi069.stdout:1/517: unlink d1/d13/d36/f91 0 2024-03-22T07:16:04.951 INFO:tasks.workunit.client.1.smithi069.stdout:1/518: stat d1/d13 0 2024-03-22T07:16:04.954 INFO:tasks.workunit.client.0.smithi063.stdout:3/869: dread da/d16/d3f/dc7/fcd [0,4194304] 0 2024-03-22T07:16:04.954 INFO:tasks.workunit.client.1.smithi069.stdout:4/679: getdents d6/d9b/d1f 0 2024-03-22T07:16:04.954 INFO:tasks.workunit.client.1.smithi069.stdout:4/680: dread - d6/daa/fc0 zero size 2024-03-22T07:16:04.954 INFO:tasks.workunit.client.1.smithi069.stdout:4/681: creat d6/d12/db2/fe2 x:0 0 0 2024-03-22T07:16:04.955 INFO:tasks.workunit.client.1.smithi069.stdout:4/682: chown d6/d9b/d1f/d3f/d51 152089 1 2024-03-22T07:16:04.958 INFO:tasks.workunit.client.0.smithi063.stdout:3/870: getdents da/d63/d65/db9 0 2024-03-22T07:16:04.958 INFO:tasks.workunit.client.1.smithi069.stdout:1/519: rename d1/d13/d36/f6b to d1/d2a/d50/d88/d41/d64/f9d 0 2024-03-22T07:16:04.958 INFO:tasks.workunit.client.1.smithi069.stdout:1/520: fdatasync d1/d2a/d50/f40 0 2024-03-22T07:16:04.963 INFO:tasks.workunit.client.1.smithi069.stdout:4/683: mknod d6/d12/d18/d21/ce3 0 2024-03-22T07:16:04.963 INFO:tasks.workunit.client.1.smithi069.stdout:4/684: read d6/d4b/f97 [2116607,47580] 0 2024-03-22T07:16:04.963 INFO:tasks.workunit.client.1.smithi069.stdout:4/685: fdatasync d6/d4b/f4d 0 2024-03-22T07:16:04.964 INFO:tasks.workunit.client.0.smithi063.stdout:3/871: symlink da/d63/d65/d83/dc6/l11c 0 2024-03-22T07:16:04.964 INFO:tasks.workunit.client.0.smithi063.stdout:3/872: creat da/d16/d89/da9/f11d x:0 0 0 2024-03-22T07:16:04.964 INFO:tasks.workunit.client.0.smithi063.stdout:3/873: truncate da/d16/d26/f8b 1550824 0 2024-03-22T07:16:04.968 INFO:tasks.workunit.client.0.smithi063.stdout:3/874: truncate da/f71 2957573 0 2024-03-22T07:16:04.970 INFO:tasks.workunit.client.0.smithi063.stdout:3/875: dread da/d16/d89/fb5 [0,4194304] 0 2024-03-22T07:16:04.970 INFO:tasks.workunit.client.0.smithi063.stdout:3/876: fdatasync da/d63/d65/d92/fb0 0 2024-03-22T07:16:04.970 INFO:tasks.workunit.client.0.smithi063.stdout:3/877: write da/d63/d65/d81/da5/ff5 [1674111,83078] 0 2024-03-22T07:16:04.971 INFO:tasks.workunit.client.1.smithi069.stdout:0/668: sync 2024-03-22T07:16:04.971 INFO:tasks.workunit.client.1.smithi069.stdout:0/669: write d3/d7/d15/d29/d5a/fa7 [1641326,113392] 0 2024-03-22T07:16:04.971 INFO:tasks.workunit.client.1.smithi069.stdout:0/670: write d3/d7/d5e/f7e [71460,79174] 0 2024-03-22T07:16:04.972 INFO:tasks.workunit.client.0.smithi063.stdout:3/878: creat da/d63/d65/d83/dc6/f11e x:0 0 0 2024-03-22T07:16:04.973 INFO:tasks.workunit.client.1.smithi069.stdout:0/671: creat d3/d7/d15/fdb x:0 0 0 2024-03-22T07:16:04.973 INFO:tasks.workunit.client.1.smithi069.stdout:0/672: chown d3/d7/d90/fa5 1317838 1 2024-03-22T07:16:04.973 INFO:tasks.workunit.client.0.smithi063.stdout:9/793: dwrite d8/d19/d20/d22/d56/f71 [0,4194304] 0 2024-03-22T07:16:04.973 INFO:tasks.workunit.client.0.smithi063.stdout:4/774: dwrite d1/d5/f97 [0,4194304] 0 2024-03-22T07:16:04.974 INFO:tasks.workunit.client.1.smithi069.stdout:0/673: symlink d3/d7/d5e/ldc 0 2024-03-22T07:16:04.974 INFO:tasks.workunit.client.1.smithi069.stdout:8/721: sync 2024-03-22T07:16:04.975 INFO:tasks.workunit.client.0.smithi063.stdout:9/794: rmdir d8/d19/d20/d22/de1/dd7/dd8 39 2024-03-22T07:16:04.975 INFO:tasks.workunit.client.1.smithi069.stdout:0/674: dread d3/d7/d17/d1c/f91 [0,4194304] 0 2024-03-22T07:16:04.976 INFO:tasks.workunit.client.1.smithi069.stdout:8/722: mkdir d1/d1c/d3e/d88/dcd/de8 0 2024-03-22T07:16:04.976 INFO:tasks.workunit.client.1.smithi069.stdout:8/723: dread - d1/d1c/d48/d5e/fc4 zero size 2024-03-22T07:16:04.976 INFO:tasks.workunit.client.0.smithi063.stdout:9/795: symlink d8/lf7 0 2024-03-22T07:16:04.977 INFO:tasks.workunit.client.0.smithi063.stdout:9/796: dread - d8/f54 zero size 2024-03-22T07:16:04.977 INFO:tasks.workunit.client.1.smithi069.stdout:0/675: rmdir d3/d7/d15/dad 39 2024-03-22T07:16:04.978 INFO:tasks.workunit.client.1.smithi069.stdout:4/686: write d6/d12/f70 [3601807,98075] 0 2024-03-22T07:16:04.979 INFO:tasks.workunit.client.1.smithi069.stdout:4/687: fsync d6/d12/f76 0 2024-03-22T07:16:04.979 INFO:tasks.workunit.client.0.smithi063.stdout:9/797: creat d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/ff8 x:0 0 0 2024-03-22T07:16:04.979 INFO:tasks.workunit.client.1.smithi069.stdout:0/676: symlink d3/d7/d15/dad/ldd 0 2024-03-22T07:16:04.981 INFO:tasks.workunit.client.0.smithi063.stdout:9/798: unlink d8/fa 0 2024-03-22T07:16:04.981 INFO:tasks.workunit.client.0.smithi063.stdout:9/799: write d8/d19/d20/fa0 [1675320,25420] 0 2024-03-22T07:16:04.989 INFO:tasks.workunit.client.1.smithi069.stdout:1/521: dwrite d1/d13/d36/f68 [4194304,4194304] 0 2024-03-22T07:16:04.991 INFO:tasks.workunit.client.1.smithi069.stdout:1/522: symlink d1/d2a/d50/l9e 0 2024-03-22T07:16:04.992 INFO:tasks.workunit.client.1.smithi069.stdout:1/523: write d1/d13/d36/f8d [311942,92145] 0 2024-03-22T07:16:04.993 INFO:tasks.workunit.client.0.smithi063.stdout:3/879: read da/d63/d65/fa8 [2578885,91702] 0 2024-03-22T07:16:04.993 INFO:tasks.workunit.client.0.smithi063.stdout:3/880: fsync da/d16/f97 0 2024-03-22T07:16:04.994 INFO:tasks.workunit.client.0.smithi063.stdout:6/839: dwrite d3/d15/d16/d60/d6d/d4e/d5a/f103 [0,4194304] 0 2024-03-22T07:16:04.994 INFO:tasks.workunit.client.0.smithi063.stdout:6/840: stat d3/d6/d2a/d38/c9d 0 2024-03-22T07:16:04.994 INFO:tasks.workunit.client.1.smithi069.stdout:1/524: mkdir d1/d13/d36/d9f 0 2024-03-22T07:16:04.995 INFO:tasks.workunit.client.1.smithi069.stdout:1/525: creat d1/d2a/d5e/d93/fa0 x:0 0 0 2024-03-22T07:16:04.996 INFO:tasks.workunit.client.1.smithi069.stdout:4/688: dread d6/d9b/d1f/f7a [0,4194304] 0 2024-03-22T07:16:04.997 INFO:tasks.workunit.client.1.smithi069.stdout:7/530: dwrite d9/d76/d57/d44/d53/d74/d6e/f83 [0,4194304] 0 2024-03-22T07:16:04.997 INFO:tasks.workunit.client.1.smithi069.stdout:4/689: write d6/d4b/f4d [5081741,13322] 0 2024-03-22T07:16:04.997 INFO:tasks.workunit.client.1.smithi069.stdout:7/531: write d9/d76/d57/d44/d53/d2e/fa6 [1024241,2225] 0 2024-03-22T07:16:04.997 INFO:tasks.workunit.client.1.smithi069.stdout:7/532: creat d9/d76/d57/d75/d7a/fad x:0 0 0 2024-03-22T07:16:04.997 INFO:tasks.workunit.client.1.smithi069.stdout:4/690: write d6/dc/d1d/d55/faf [465576,128330] 0 2024-03-22T07:16:04.997 INFO:tasks.workunit.client.1.smithi069.stdout:7/533: write d9/d76/d57/d44/fac [364491,52536] 0 2024-03-22T07:16:04.998 INFO:tasks.workunit.client.1.smithi069.stdout:7/534: creat d9/d76/d57/d44/d53/d74/d30/d3d/fae x:0 0 0 2024-03-22T07:16:04.999 INFO:tasks.workunit.client.0.smithi063.stdout:9/800: write d8/d19/d39/f8b [81313,106145] 0 2024-03-22T07:16:05.008 INFO:tasks.workunit.client.1.smithi069.stdout:7/535: creat d9/db/d48/d4b/faf x:0 0 0 2024-03-22T07:16:05.010 INFO:tasks.workunit.client.1.smithi069.stdout:4/691: rmdir d6/d12/db2 39 2024-03-22T07:16:05.013 INFO:tasks.workunit.client.0.smithi063.stdout:4/775: dwrite d1/f5b [0,4194304] 0 2024-03-22T07:16:05.015 INFO:tasks.workunit.client.1.smithi069.stdout:4/692: mkdir d6/daa/dce/de4 0 2024-03-22T07:16:05.016 INFO:tasks.workunit.client.1.smithi069.stdout:4/693: mknod d6/d12/d18/ce5 0 2024-03-22T07:16:05.017 INFO:tasks.workunit.client.0.smithi063.stdout:4/776: mknod d1/d5/dc/d95/da4/d104/c106 0 2024-03-22T07:16:05.021 INFO:tasks.workunit.client.1.smithi069.stdout:0/677: dwrite d3/d7/d15/f32 [0,4194304] 0 2024-03-22T07:16:05.021 INFO:tasks.workunit.client.1.smithi069.stdout:0/678: readlink d3/d7/d15/d29/l54 0 2024-03-22T07:16:05.022 INFO:tasks.workunit.client.1.smithi069.stdout:0/679: creat d3/d7/d17/fde x:0 0 0 2024-03-22T07:16:05.022 INFO:tasks.workunit.client.1.smithi069.stdout:0/680: fsync d3/d7/d15/d29/d5a/d84/f3b 0 2024-03-22T07:16:05.025 INFO:tasks.workunit.client.0.smithi063.stdout:6/841: dread d3/d6/d2a/d38/da5/d32/fc3 [0,4194304] 0 2024-03-22T07:16:05.027 INFO:tasks.workunit.client.1.smithi069.stdout:0/681: dread d3/d7/d15/d29/f63 [0,4194304] 0 2024-03-22T07:16:05.028 INFO:tasks.workunit.client.1.smithi069.stdout:0/682: read d3/d7/d15/d1f/f4e [74731,69238] 0 2024-03-22T07:16:05.028 INFO:tasks.workunit.client.1.smithi069.stdout:0/683: creat d3/d7/d24/d6a/fdf x:0 0 0 2024-03-22T07:16:05.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:04 smithi069 ceph-mon[36441]: [22/Mar/2024:07:16:02] ENGINE Bus STARTING 2024-03-22T07:16:05.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:04 smithi069 ceph-mon[36441]: [22/Mar/2024:07:16:03] ENGINE Serving on https://172.21.15.69:7150 2024-03-22T07:16:05.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:04 smithi069 ceph-mon[36441]: [22/Mar/2024:07:16:03] ENGINE Serving on http://172.21.15.69:8765 2024-03-22T07:16:05.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:04 smithi069 ceph-mon[36441]: [22/Mar/2024:07:16:03] ENGINE Bus STARTED 2024-03-22T07:16:05.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:04 smithi069 ceph-mon[36441]: mgrmap e23: smithi069.gfziwh(active, since 3s) 2024-03-22T07:16:05.030 INFO:tasks.workunit.client.0.smithi063.stdout:4/777: write d1/d5/dc/ff [3273278,10747] 0 2024-03-22T07:16:05.030 INFO:tasks.workunit.client.0.smithi063.stdout:4/778: chown d1/d5/d43/d27/d2c/d4a/d80/ld7 7329 1 2024-03-22T07:16:05.031 INFO:tasks.workunit.client.1.smithi069.stdout:0/684: link d3/d7/d15/d29/d5a/d5d/d9d/fbd d3/d7/d90/fe0 0 2024-03-22T07:16:05.034 INFO:tasks.workunit.client.1.smithi069.stdout:1/526: dwrite d1/d2a/d50/d88/f75 [0,4194304] 0 2024-03-22T07:16:05.034 INFO:tasks.workunit.client.1.smithi069.stdout:1/527: fsync d1/d2a/d50/d88/d41/d7a/f7f 0 2024-03-22T07:16:05.035 INFO:tasks.workunit.client.1.smithi069.stdout:0/685: creat d3/d7/d15/fe1 x:0 0 0 2024-03-22T07:16:05.036 INFO:tasks.workunit.client.1.smithi069.stdout:7/536: dwrite d9/d76/d57/d44/d53/fa7 [0,4194304] 0 2024-03-22T07:16:05.038 INFO:tasks.workunit.client.0.smithi063.stdout:3/881: dwrite da/d16/f38 [0,4194304] 0 2024-03-22T07:16:05.038 INFO:tasks.workunit.client.0.smithi063.stdout:3/882: truncate da/d16/d3f/ff7 293756 0 2024-03-22T07:16:05.038 INFO:tasks.workunit.client.0.smithi063.stdout:9/801: getdents d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48 0 2024-03-22T07:16:05.042 INFO:tasks.workunit.client.0.smithi063.stdout:9/802: dread d8/d15/f58 [0,4194304] 0 2024-03-22T07:16:05.043 INFO:tasks.workunit.client.1.smithi069.stdout:1/528: dread d1/d13/f58 [0,4194304] 0 2024-03-22T07:16:05.044 INFO:tasks.workunit.client.1.smithi069.stdout:4/694: dread d6/d12/f70 [0,4194304] 0 2024-03-22T07:16:05.045 INFO:tasks.workunit.client.1.smithi069.stdout:4/695: dread d6/d9b/d1f/d58/d7c/fac [0,4194304] 0 2024-03-22T07:16:05.045 INFO:tasks.workunit.client.1.smithi069.stdout:4/696: creat d6/dc/d23/d59/fe6 x:0 0 0 2024-03-22T07:16:05.047 INFO:tasks.workunit.client.1.smithi069.stdout:0/686: symlink d3/d7/d15/d29/d5a/d99/dca/le2 0 2024-03-22T07:16:05.047 INFO:tasks.workunit.client.1.smithi069.stdout:0/687: chown d3/d7/d15/d29/l80 1 1 2024-03-22T07:16:05.047 INFO:tasks.workunit.client.1.smithi069.stdout:0/688: truncate d3/d7/d15/f32 4362920 0 2024-03-22T07:16:05.047 INFO:tasks.workunit.client.1.smithi069.stdout:0/689: write d3/d7/f5c [314099,74708] 0 2024-03-22T07:16:05.049 INFO:tasks.workunit.client.1.smithi069.stdout:7/537: read d9/d42/d8a/f9d [399,70634] 0 2024-03-22T07:16:05.049 INFO:tasks.workunit.client.1.smithi069.stdout:7/538: fdatasync d9/d76/d57/d44/f9b 0 2024-03-22T07:16:05.050 INFO:tasks.workunit.client.1.smithi069.stdout:1/529: mkdir d1/d13/d36/da1 0 2024-03-22T07:16:05.052 INFO:tasks.workunit.client.1.smithi069.stdout:4/697: link d6/d4b/la0 d6/d4b/le7 0 2024-03-22T07:16:05.052 INFO:tasks.workunit.client.1.smithi069.stdout:4/698: creat d6/daa/fe8 x:0 0 0 2024-03-22T07:16:05.052 INFO:tasks.workunit.client.1.smithi069.stdout:4/699: chown d6/d16/d3a/db4/dbc/dc6/fdb 24952 1 2024-03-22T07:16:05.052 INFO:tasks.workunit.client.1.smithi069.stdout:1/530: write d1/d2a/d50/f40 [2772420,85819] 0 2024-03-22T07:16:05.054 INFO:tasks.workunit.client.1.smithi069.stdout:0/690: link d3/d7/d15/d29/l2d d3/d7/d15/d29/d5a/d5d/db4/le3 0 2024-03-22T07:16:05.057 INFO:tasks.workunit.client.1.smithi069.stdout:7/539: truncate d9/db/f19 6584965 0 2024-03-22T07:16:05.057 INFO:tasks.workunit.client.1.smithi069.stdout:7/540: read d9/f6f [7992573,17166] 0 2024-03-22T07:16:05.057 INFO:tasks.workunit.client.1.smithi069.stdout:7/541: write d9/d76/d57/d44/d53/d74/d6e/f83 [4813656,101907] 0 2024-03-22T07:16:05.058 INFO:tasks.workunit.client.1.smithi069.stdout:4/700: mkdir d6/d4b/de9 0 2024-03-22T07:16:05.059 INFO:tasks.workunit.client.1.smithi069.stdout:4/701: creat d6/d9b/d56/fea x:0 0 0 2024-03-22T07:16:05.059 INFO:tasks.workunit.client.1.smithi069.stdout:4/702: creat d6/dc/feb x:0 0 0 2024-03-22T07:16:05.059 INFO:tasks.workunit.client.1.smithi069.stdout:4/703: write d6/dc/d1d/d55/f9c [1985344,22702] 0 2024-03-22T07:16:05.060 INFO:tasks.workunit.client.0.smithi063.stdout:4/779: dwrite d1/d5/dc/d95/d59/d9c/dac/dc9/ff9 [0,4194304] 0 2024-03-22T07:16:05.063 INFO:tasks.workunit.client.1.smithi069.stdout:4/704: mknod d6/d16/cec 0 2024-03-22T07:16:05.064 INFO:tasks.workunit.client.0.smithi063.stdout:6/842: dwrite d3/d15/d16/d60/d6d/d4e/d5f/f93 [0,4194304] 0 2024-03-22T07:16:05.066 INFO:tasks.workunit.client.1.smithi069.stdout:4/705: mknod d6/dc4/db5/ced 0 2024-03-22T07:16:05.066 INFO:tasks.workunit.client.0.smithi063.stdout:4/780: mkdir d1/d5/d43/d107 0 2024-03-22T07:16:05.068 INFO:tasks.workunit.client.1.smithi069.stdout:4/706: link d6/d9b/d1f/d3f/d51/l9f d6/d4b/de9/lee 0 2024-03-22T07:16:05.070 INFO:tasks.workunit.client.1.smithi069.stdout:4/707: mknod d6/dc4/db5/cef 0 2024-03-22T07:16:05.071 INFO:tasks.workunit.client.1.smithi069.stdout:4/708: dread - d6/d16/d3a/db4/dbc/fa4 zero size 2024-03-22T07:16:05.071 INFO:tasks.workunit.client.1.smithi069.stdout:4/709: getdents d6/d9b/d1f 0 2024-03-22T07:16:05.073 INFO:tasks.workunit.client.1.smithi069.stdout:4/710: creat d6/d12/d5d/ff0 x:0 0 0 2024-03-22T07:16:05.076 INFO:tasks.workunit.client.0.smithi063.stdout:6/843: mknod d3/d15/d16/d60/d6d/d4e/c11a 0 2024-03-22T07:16:05.077 INFO:tasks.workunit.client.1.smithi069.stdout:4/711: unlink d6/dc/d1d/cba 0 2024-03-22T07:16:05.077 INFO:tasks.workunit.client.1.smithi069.stdout:4/712: write d6/d9b/fb6 [2077,12237] 0 2024-03-22T07:16:05.077 INFO:tasks.workunit.client.1.smithi069.stdout:4/713: chown d6/dc/d23/d59/l67 1773155 1 2024-03-22T07:16:05.078 INFO:tasks.workunit.client.1.smithi069.stdout:4/714: link d6/dc/d1d/l29 d6/dc4/de1/lf1 0 2024-03-22T07:16:05.079 INFO:tasks.workunit.client.0.smithi063.stdout:9/803: dwrite d8/d15/d63/fb3 [0,4194304] 0 2024-03-22T07:16:05.079 INFO:tasks.workunit.client.0.smithi063.stdout:9/804: write d8/d19/d39/f6f [2082641,100900] 0 2024-03-22T07:16:05.079 INFO:tasks.workunit.client.0.smithi063.stdout:9/805: truncate d8/d19/d39/db5/fdf 472311 0 2024-03-22T07:16:05.080 INFO:tasks.workunit.client.1.smithi069.stdout:4/715: rename d6/dc/d1d/l29 to d6/d9b/d1f/d58/lf2 0 2024-03-22T07:16:05.081 INFO:tasks.workunit.client.1.smithi069.stdout:9/485: sync 2024-03-22T07:16:05.082 INFO:tasks.workunit.client.1.smithi069.stdout:6/510: sync 2024-03-22T07:16:05.082 INFO:tasks.workunit.client.1.smithi069.stdout:3/625: sync 2024-03-22T07:16:05.082 INFO:tasks.workunit.client.0.smithi063.stdout:4/781: creat d1/d5/dc/d95/d59/d9c/dac/dcd/f108 x:0 0 0 2024-03-22T07:16:05.083 INFO:tasks.workunit.client.1.smithi069.stdout:1/531: dwrite d1/d13/d36/d5b/f8e [0,4194304] 0 2024-03-22T07:16:05.083 INFO:tasks.workunit.client.1.smithi069.stdout:9/486: unlink d2/d51/l63 0 2024-03-22T07:16:05.086 INFO:tasks.workunit.client.1.smithi069.stdout:6/511: creat d4/dd/d12/d44/f9e x:0 0 0 2024-03-22T07:16:05.086 INFO:tasks.workunit.client.1.smithi069.stdout:6/512: truncate d4/dd/d12/f1f 5089615 0 2024-03-22T07:16:05.087 INFO:tasks.workunit.client.1.smithi069.stdout:3/626: truncate d5/d9/d26/d3f/d35/f37 2944130 0 2024-03-22T07:16:05.087 INFO:tasks.workunit.client.1.smithi069.stdout:3/627: write d5/d9/d26/d36/d41/d51/f7c [160060,127089] 0 2024-03-22T07:16:05.088 INFO:tasks.workunit.client.1.smithi069.stdout:4/716: read d6/d12/d18/f45 [197544,99335] 0 2024-03-22T07:16:05.088 INFO:tasks.workunit.client.1.smithi069.stdout:4/717: creat d6/d12/d18/d21/ff3 x:0 0 0 2024-03-22T07:16:05.090 INFO:tasks.workunit.client.1.smithi069.stdout:7/542: dwrite d9/d76/d57/d44/d53/d2e/fa6 [0,4194304] 0 2024-03-22T07:16:05.091 INFO:tasks.workunit.client.1.smithi069.stdout:7/543: write d9/d76/d57/d44/d53/d1d/faa [459755,33246] 0 2024-03-22T07:16:05.091 INFO:tasks.workunit.client.1.smithi069.stdout:7/544: chown d9/d76/d57/d44/d53/d1d 1067880 1 2024-03-22T07:16:05.093 INFO:tasks.workunit.client.1.smithi069.stdout:1/532: symlink d1/d13/d36/la2 0 2024-03-22T07:16:05.093 INFO:tasks.workunit.client.1.smithi069.stdout:1/533: chown d1/d13/f45 4386 1 2024-03-22T07:16:05.095 INFO:tasks.workunit.client.0.smithi063.stdout:4/782: write d1/d5/d3c/d8f/dc0/dcb/fdc [2591819,109394] 0 2024-03-22T07:16:05.095 INFO:tasks.workunit.client.1.smithi069.stdout:6/513: creat d4/dd/d12/d2b/f9f x:0 0 0 2024-03-22T07:16:05.095 INFO:tasks.workunit.client.0.smithi063.stdout:6/844: creat d3/d6/f11b x:0 0 0 2024-03-22T07:16:05.097 INFO:tasks.workunit.client.1.smithi069.stdout:3/628: symlink d5/d9/d26/d3f/d56/d8a/da5/lcd 0 2024-03-22T07:16:05.101 INFO:tasks.workunit.client.1.smithi069.stdout:4/718: rename d6/daa/fc0 to d6/d16/d83/ff4 0 2024-03-22T07:16:05.102 INFO:tasks.workunit.client.1.smithi069.stdout:3/629: read d5/f8f [354683,57512] 0 2024-03-22T07:16:05.103 INFO:tasks.workunit.client.1.smithi069.stdout:3/630: dread - d5/d9/d26/d36/fcc zero size 2024-03-22T07:16:05.104 INFO:tasks.workunit.client.1.smithi069.stdout:7/545: mknod d9/d76/d57/d44/cb0 0 2024-03-22T07:16:05.106 INFO:tasks.workunit.client.1.smithi069.stdout:1/534: creat d1/d13/fa3 x:0 0 0 2024-03-22T07:16:05.106 INFO:tasks.workunit.client.1.smithi069.stdout:1/535: write d1/d13/d36/f8d [342131,66910] 0 2024-03-22T07:16:05.107 INFO:tasks.workunit.client.1.smithi069.stdout:1/536: dread d1/d2a/d50/d4c/d55/f62 [0,4194304] 0 2024-03-22T07:16:05.108 INFO:tasks.workunit.client.1.smithi069.stdout:6/514: mknod d4/dd/d12/d44/d58/ca0 0 2024-03-22T07:16:05.110 INFO:tasks.workunit.client.1.smithi069.stdout:4/719: creat d6/d12/db2/ff5 x:0 0 0 2024-03-22T07:16:05.111 INFO:tasks.workunit.client.0.smithi063.stdout:4/783: mkdir d1/d5/dc/dda/d109 0 2024-03-22T07:16:05.113 INFO:tasks.workunit.client.0.smithi063.stdout:6/845: mkdir d3/d15/d16/d60/d6d/d4e/d5a/d88/df5/d114/d11c 0 2024-03-22T07:16:05.114 INFO:tasks.workunit.client.0.smithi063.stdout:6/846: fsync d3/d6/d2a/d38/da5/d32/fdb 0 2024-03-22T07:16:05.114 INFO:tasks.workunit.client.1.smithi069.stdout:3/631: symlink d5/d9/d26/d3f/dc1/lce 0 2024-03-22T07:16:05.114 INFO:tasks.workunit.client.1.smithi069.stdout:3/632: fdatasync d5/d9/d26/d3f/d56/d7f/f96 0 2024-03-22T07:16:05.114 INFO:tasks.workunit.client.1.smithi069.stdout:3/633: write d5/d9/f76 [407420,58654] 0 2024-03-22T07:16:05.114 INFO:tasks.workunit.client.1.smithi069.stdout:3/634: creat d5/d9/db/d3b/fcf x:0 0 0 2024-03-22T07:16:05.116 INFO:tasks.workunit.client.1.smithi069.stdout:7/546: truncate d9/d76/d57/d44/d53/d74/f98 3446871 0 2024-03-22T07:16:05.117 INFO:tasks.workunit.client.0.smithi063.stdout:4/784: creat d1/d5/d85/f10a x:0 0 0 2024-03-22T07:16:05.117 INFO:tasks.workunit.client.0.smithi063.stdout:4/785: stat d1/d5/dc2/ce0 0 2024-03-22T07:16:05.117 INFO:tasks.workunit.client.0.smithi063.stdout:8/885: sync 2024-03-22T07:16:05.118 INFO:tasks.workunit.client.0.smithi063.stdout:6/847: mknod d3/d15/d16/d60/d6d/d4e/d5f/c11d 0 2024-03-22T07:16:05.118 INFO:tasks.workunit.client.0.smithi063.stdout:6/848: readlink d3/d15/d16/d60/d6d/d4c/l2f 0 2024-03-22T07:16:05.118 INFO:tasks.workunit.client.0.smithi063.stdout:6/849: dread - d3/d6/d35/f101 zero size 2024-03-22T07:16:05.120 INFO:tasks.workunit.client.1.smithi069.stdout:1/537: rename d1/d2a/d50/d88/d41/d7a/f7f to d1/d2a/d5e/fa4 0 2024-03-22T07:16:05.120 INFO:tasks.workunit.client.1.smithi069.stdout:1/538: dread - d1/d2a/d50/d4c/f86 zero size 2024-03-22T07:16:05.120 INFO:tasks.workunit.client.1.smithi069.stdout:6/515: write d4/dd/d12/d44/f4e [3982001,125734] 0 2024-03-22T07:16:05.120 INFO:tasks.workunit.client.0.smithi063.stdout:4/786: mkdir d1/d5/d43/d27/d2c/df4/d10b 0 2024-03-22T07:16:05.122 INFO:tasks.workunit.client.0.smithi063.stdout:8/886: mknod d0/d1b/d28/de8/dc6/c12a 0 2024-03-22T07:16:05.124 INFO:tasks.workunit.client.0.smithi063.stdout:8/887: rename d0/d1b/d28/d3e/c81 to d0/d9e/c12b 0 2024-03-22T07:16:05.125 INFO:tasks.workunit.client.0.smithi063.stdout:8/888: chown d0/d1b/d28/d3e/c8b 92 1 2024-03-22T07:16:05.125 INFO:tasks.workunit.client.0.smithi063.stdout:8/889: write d0/d26/d2d/d59/ffc [1373119,69373] 0 2024-03-22T07:16:05.125 INFO:tasks.workunit.client.0.smithi063.stdout:8/890: read - d0/d9e/df0/f117 zero size 2024-03-22T07:16:05.125 INFO:tasks.workunit.client.0.smithi063.stdout:8/891: chown d0/d1b/f20 1436 1 2024-03-22T07:16:05.125 INFO:tasks.workunit.client.1.smithi069.stdout:3/635: write d5/d9/d26/d3f/d35/f45 [1764747,79472] 0 2024-03-22T07:16:05.127 INFO:tasks.workunit.client.0.smithi063.stdout:8/892: creat d0/d1b/d28/de8/dc6/f12c x:0 0 0 2024-03-22T07:16:05.128 INFO:tasks.workunit.client.1.smithi069.stdout:1/539: rmdir d1/d13/d36/d9f 0 2024-03-22T07:16:05.128 INFO:tasks.workunit.client.1.smithi069.stdout:1/540: creat d1/d2a/d50/d4c/d55/fa5 x:0 0 0 2024-03-22T07:16:05.128 INFO:tasks.workunit.client.1.smithi069.stdout:1/541: write d1/d2a/f4e [4881631,16293] 0 2024-03-22T07:16:05.129 INFO:tasks.workunit.client.0.smithi063.stdout:8/893: getdents d0/d24 0 2024-03-22T07:16:05.129 INFO:tasks.workunit.client.0.smithi063.stdout:9/806: dwrite d8/f54 [0,4194304] 0 2024-03-22T07:16:05.130 INFO:tasks.workunit.client.0.smithi063.stdout:6/850: dread d3/f19 [4194304,4194304] 0 2024-03-22T07:16:05.130 INFO:tasks.workunit.client.0.smithi063.stdout:8/894: mkdir d0/d26/d2d/d83/d11c/d121/d12d 0 2024-03-22T07:16:05.131 INFO:tasks.workunit.client.0.smithi063.stdout:8/895: chown d0/d26/d2d/d59/dc0/d106/c129 196507 1 2024-03-22T07:16:05.131 INFO:tasks.workunit.client.1.smithi069.stdout:6/516: unlink d4/dd/c18 0 2024-03-22T07:16:05.131 INFO:tasks.workunit.client.1.smithi069.stdout:6/517: chown d4/dd/d41/f51 318465279 1 2024-03-22T07:16:05.131 INFO:tasks.workunit.client.1.smithi069.stdout:6/518: truncate d4/d7/da/d3f/f99 338419 0 2024-03-22T07:16:05.132 INFO:tasks.workunit.client.1.smithi069.stdout:6/519: write d4/dd/f4a [1537076,87577] 0 2024-03-22T07:16:05.132 INFO:tasks.workunit.client.1.smithi069.stdout:6/520: chown d4/dd 1 1 2024-03-22T07:16:05.132 INFO:tasks.workunit.client.1.smithi069.stdout:6/521: read d4/f47 [2885045,118900] 0 2024-03-22T07:16:05.133 INFO:tasks.workunit.client.0.smithi063.stdout:7/658: sync 2024-03-22T07:16:05.133 INFO:tasks.workunit.client.0.smithi063.stdout:2/728: sync 2024-03-22T07:16:05.133 INFO:tasks.workunit.client.0.smithi063.stdout:2/729: chown d0/de/d12/d20/fc7 459637099 1 2024-03-22T07:16:05.133 INFO:tasks.workunit.client.0.smithi063.stdout:3/883: dwrite da/d16/d3f/f67 [0,4194304] 0 2024-03-22T07:16:05.134 INFO:tasks.workunit.client.0.smithi063.stdout:3/884: stat da/d63/d65/d81/fa4 0 2024-03-22T07:16:05.138 INFO:tasks.workunit.client.1.smithi069.stdout:1/542: mkdir d1/d2a/d5e/da6 0 2024-03-22T07:16:05.139 INFO:tasks.workunit.client.1.smithi069.stdout:1/543: creat d1/d2a/d84/fa7 x:0 0 0 2024-03-22T07:16:05.139 INFO:tasks.workunit.client.0.smithi063.stdout:9/807: mkdir d8/d19/d20/d22/de1/db7/df9 0 2024-03-22T07:16:05.139 INFO:tasks.workunit.client.0.smithi063.stdout:9/808: truncate d8/d19/d20/d22/fd6 925716 0 2024-03-22T07:16:05.139 INFO:tasks.workunit.client.0.smithi063.stdout:5/696: sync 2024-03-22T07:16:05.139 INFO:tasks.workunit.client.0.smithi063.stdout:1/934: sync 2024-03-22T07:16:05.139 INFO:tasks.workunit.client.0.smithi063.stdout:1/935: truncate d4/df/d13/d39/d63/f66 2203726 0 2024-03-22T07:16:05.139 INFO:tasks.workunit.client.0.smithi063.stdout:6/851: stat d3/d6/d2a/d38/c119 0 2024-03-22T07:16:05.139 INFO:tasks.workunit.client.0.smithi063.stdout:6/852: fdatasync d3/d6/f11b 0 2024-03-22T07:16:05.143 INFO:tasks.workunit.client.1.smithi069.stdout:6/522: truncate d4/d7/da/f83 3251759 0 2024-03-22T07:16:05.143 INFO:tasks.workunit.client.1.smithi069.stdout:6/523: write d4/f6 [1103976,18081] 0 2024-03-22T07:16:05.147 INFO:tasks.workunit.client.0.smithi063.stdout:8/896: creat d0/d26/d2d/d59/dc0/d106/f12e x:0 0 0 2024-03-22T07:16:05.147 INFO:tasks.workunit.client.0.smithi063.stdout:6/853: dread d3/d15/d16/d60/d6d/d4e/d5a/dd4/ffa [0,4194304] 0 2024-03-22T07:16:05.147 INFO:tasks.workunit.client.0.smithi063.stdout:6/854: write d3/d15/d16/d60/d6d/d4e/d5a/d88/f8d [1876752,107593] 0 2024-03-22T07:16:05.147 INFO:tasks.workunit.client.0.smithi063.stdout:6/855: write d3/d15/d16/d60/d6d/d4e/d5a/dd4/ffa [639377,76016] 0 2024-03-22T07:16:05.148 INFO:tasks.workunit.client.0.smithi063.stdout:3/885: dread da/f6a [0,4194304] 0 2024-03-22T07:16:05.148 INFO:tasks.workunit.client.0.smithi063.stdout:9/809: creat d8/d19/d20/d22/d29/df0/ffa x:0 0 0 2024-03-22T07:16:05.148 INFO:tasks.workunit.client.0.smithi063.stdout:5/697: mknod d8/d13/d1e/d31/d64/d65/d86/cd2 0 2024-03-22T07:16:05.149 INFO:tasks.workunit.client.0.smithi063.stdout:1/936: unlink d4/df/d13/d39/f82 0 2024-03-22T07:16:05.149 INFO:tasks.workunit.client.0.smithi063.stdout:6/856: getdents d3/d15/d95 0 2024-03-22T07:16:05.151 INFO:tasks.workunit.client.0.smithi063.stdout:3/886: unlink da/d16/d26/d69/c11a 0 2024-03-22T07:16:05.151 INFO:tasks.workunit.client.0.smithi063.stdout:3/887: write da/d63/d65/d92/d96/f10f [421300,108282] 0 2024-03-22T07:16:05.153 INFO:tasks.workunit.client.0.smithi063.stdout:9/810: mkdir d8/d19/d20/d22/d29/d31/d53/d6a/da2/df1/dfb 0 2024-03-22T07:16:05.154 INFO:tasks.workunit.client.1.smithi069.stdout:7/547: rename d9/d76 to d9/db/d1a/db1 0 2024-03-22T07:16:05.156 INFO:tasks.workunit.client.0.smithi063.stdout:9/811: dread d8/d19/d20/d22/d29/d31/d53/d6a/da2/f80 [0,4194304] 0 2024-03-22T07:16:05.157 INFO:tasks.workunit.client.1.smithi069.stdout:1/544: dread d1/d13/d36/d5b/f71 [0,4194304] 0 2024-03-22T07:16:05.159 INFO:tasks.workunit.client.0.smithi063.stdout:3/888: dread da/d63/d65/d4b/f85 [0,4194304] 0 2024-03-22T07:16:05.159 INFO:tasks.workunit.client.0.smithi063.stdout:5/698: read d8/d13/f18 [5928320,107678] 0 2024-03-22T07:16:05.161 INFO:tasks.workunit.client.0.smithi063.stdout:3/889: mkdir da/d63/d65/d4b/ddd/d11f 0 2024-03-22T07:16:05.163 INFO:tasks.workunit.client.0.smithi063.stdout:5/699: rename d8/d13/d1e/d3c/da0/f91 to d8/d13/d24/d62/fd3 0 2024-03-22T07:16:05.164 INFO:tasks.workunit.client.1.smithi069.stdout:6/524: dread d4/dd/d12/d2b/d72/f95 [0,4194304] 0 2024-03-22T07:16:05.165 INFO:tasks.workunit.client.0.smithi063.stdout:3/890: rename da/d16/d3f/f104 to da/d63/d65/d4b/d10a/f120 0 2024-03-22T07:16:05.165 INFO:tasks.workunit.client.0.smithi063.stdout:7/659: dwrite d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/d4a/fc6 [0,4194304] 0 2024-03-22T07:16:05.166 INFO:tasks.workunit.client.0.smithi063.stdout:7/660: truncate d5/da/d28/d4b/da8/dbc/d62/d73/fd7 4631901 0 2024-03-22T07:16:05.167 INFO:tasks.workunit.client.0.smithi063.stdout:5/700: link d8/d13/d1e/d31/daf/fb0 d8/d13/d24/d81/fd4 0 2024-03-22T07:16:05.169 INFO:tasks.workunit.client.1.smithi069.stdout:1/545: creat d1/d13/d36/d6e/d73/fa8 x:0 0 0 2024-03-22T07:16:05.170 INFO:tasks.workunit.client.1.smithi069.stdout:7/548: getdents d9/db 0 2024-03-22T07:16:05.171 INFO:tasks.workunit.client.0.smithi063.stdout:7/661: link d5/da/d5f/d33/fbd d5/da/d28/d4b/da8/dbc/fd9 0 2024-03-22T07:16:05.175 INFO:tasks.workunit.client.0.smithi063.stdout:5/701: creat d8/d13/d1e/d31/d64/d65/fd5 x:0 0 0 2024-03-22T07:16:05.176 INFO:tasks.workunit.client.1.smithi069.stdout:1/546: symlink d1/d2a/la9 0 2024-03-22T07:16:05.176 INFO:tasks.workunit.client.1.smithi069.stdout:1/547: dread d1/f3b [0,4194304] 0 2024-03-22T07:16:05.176 INFO:tasks.workunit.client.1.smithi069.stdout:1/548: chown d1/c6f 11527 1 2024-03-22T07:16:05.177 INFO:tasks.workunit.client.1.smithi069.stdout:1/549: readlink d1/d2a/d50/l44 0 2024-03-22T07:16:05.177 INFO:tasks.workunit.client.1.smithi069.stdout:1/550: truncate d1/d2a/d50/d4c/d55/f97 750490 0 2024-03-22T07:16:05.177 INFO:tasks.workunit.client.1.smithi069.stdout:7/549: creat d9/db/d1a/db1/d57/d44/d53/d74/fb2 x:0 0 0 2024-03-22T07:16:05.177 INFO:tasks.workunit.client.1.smithi069.stdout:1/551: rmdir d1/d2a/d50/d4c/d55/d5f 39 2024-03-22T07:16:05.178 INFO:tasks.workunit.client.0.smithi063.stdout:8/897: dwrite d0/d1b/f6e [0,4194304] 0 2024-03-22T07:16:05.178 INFO:tasks.workunit.client.0.smithi063.stdout:2/730: dwrite d0/de/d12/d20/f52 [0,4194304] 0 2024-03-22T07:16:05.178 INFO:tasks.workunit.client.0.smithi063.stdout:8/898: write d0/d1b/d28/d3e/db2/f115 [606334,37826] 0 2024-03-22T07:16:05.178 INFO:tasks.workunit.client.0.smithi063.stdout:8/899: write d0/d26/d2d/d83/fcc [989203,7790] 0 2024-03-22T07:16:05.178 INFO:tasks.workunit.client.0.smithi063.stdout:8/900: chown d0/d26/l6c 9 1 2024-03-22T07:16:05.178 INFO:tasks.workunit.client.0.smithi063.stdout:7/662: symlink d5/da/d28/d4b/da8/dbc/d62/lda 0 2024-03-22T07:16:05.179 INFO:tasks.workunit.client.1.smithi069.stdout:6/525: write d4/f77 [3199331,101195] 0 2024-03-22T07:16:05.180 INFO:tasks.workunit.client.1.smithi069.stdout:1/552: mknod d1/d13/d36/d6e/caa 0 2024-03-22T07:16:05.180 INFO:tasks.workunit.client.1.smithi069.stdout:7/550: dread d9/db/d1a/db1/d57/d44/d53/f6d [0,4194304] 0 2024-03-22T07:16:05.180 INFO:tasks.workunit.client.1.smithi069.stdout:7/551: write d9/db/d1a/db1/d57/d75/d7a/f8f [1581987,70852] 0 2024-03-22T07:16:05.180 INFO:tasks.workunit.client.1.smithi069.stdout:7/552: truncate d9/db/f69 336700 0 2024-03-22T07:16:05.180 INFO:tasks.workunit.client.1.smithi069.stdout:7/553: fdatasync d9/db/d1a/db1/d57/d44/d53/d2e/f7c 0 2024-03-22T07:16:05.182 INFO:tasks.workunit.client.0.smithi063.stdout:8/901: truncate d0/d24/f29 1290574 0 2024-03-22T07:16:05.182 INFO:tasks.workunit.client.0.smithi063.stdout:8/902: creat d0/d1b/d28/de8/dc6/f12f x:0 0 0 2024-03-22T07:16:05.190 INFO:tasks.workunit.client.0.smithi063.stdout:1/937: dwrite d4/d5a/d91/f139 [0,4194304] 0 2024-03-22T07:16:05.192 INFO:tasks.workunit.client.0.smithi063.stdout:7/663: dread d5/da/d5f/f8a [0,4194304] 0 2024-03-22T07:16:05.192 INFO:tasks.workunit.client.0.smithi063.stdout:7/664: truncate d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/d4a/f7e 4768772 0 2024-03-22T07:16:05.193 INFO:tasks.workunit.client.0.smithi063.stdout:7/665: creat d5/da/d28/d4b/da8/fdb x:0 0 0 2024-03-22T07:16:05.194 INFO:tasks.workunit.client.1.smithi069.stdout:1/553: write d1/d13/d36/f68 [237963,97001] 0 2024-03-22T07:16:05.195 INFO:tasks.workunit.client.1.smithi069.stdout:1/554: symlink d1/d13/d36/lab 0 2024-03-22T07:16:05.195 INFO:tasks.workunit.client.1.smithi069.stdout:1/555: chown d1/d13/c15 466802 1 2024-03-22T07:16:05.195 INFO:tasks.workunit.client.1.smithi069.stdout:1/556: fdatasync d1/d13/fa3 0 2024-03-22T07:16:05.197 INFO:tasks.workunit.client.0.smithi063.stdout:8/903: dread d0/d26/d2d/d59/dba/dc9/fff [0,4194304] 0 2024-03-22T07:16:05.197 INFO:tasks.workunit.client.1.smithi069.stdout:1/557: creat d1/d2a/d50/fac x:0 0 0 2024-03-22T07:16:05.197 INFO:tasks.workunit.client.1.smithi069.stdout:1/558: write d1/d2a/d84/fa7 [4908,45852] 0 2024-03-22T07:16:05.197 INFO:tasks.workunit.client.0.smithi063.stdout:1/938: unlink d4/df/f4e 0 2024-03-22T07:16:05.198 INFO:tasks.workunit.client.1.smithi069.stdout:1/559: creat d1/d13/d36/d5b/d95/fad x:0 0 0 2024-03-22T07:16:05.208 INFO:tasks.workunit.client.1.smithi069.stdout:2/412: sync 2024-03-22T07:16:05.208 INFO:tasks.workunit.client.1.smithi069.stdout:2/413: truncate d18/d4d/d3e/d40/f5c 4737475 0 2024-03-22T07:16:05.209 INFO:tasks.workunit.client.1.smithi069.stdout:5/442: sync 2024-03-22T07:16:05.209 INFO:tasks.workunit.client.1.smithi069.stdout:2/414: symlink d18/d4d/d25/l8e 0 2024-03-22T07:16:05.211 INFO:tasks.workunit.client.1.smithi069.stdout:5/443: dread d1/d6/d26/d33/f4e [0,4194304] 0 2024-03-22T07:16:05.212 INFO:tasks.workunit.client.1.smithi069.stdout:5/444: write d1/f9 [1363904,90552] 0 2024-03-22T07:16:05.212 INFO:tasks.workunit.client.0.smithi063.stdout:3/891: dwrite da/d63/d65/d4b/ddd/fe7 [0,4194304] 0 2024-03-22T07:16:05.214 INFO:tasks.workunit.client.1.smithi069.stdout:7/554: dwrite d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/f7b [0,4194304] 0 2024-03-22T07:16:05.214 INFO:tasks.workunit.client.1.smithi069.stdout:5/445: mkdir d1/d6/d26/d3e/d9c/d9d 0 2024-03-22T07:16:05.215 INFO:tasks.workunit.client.1.smithi069.stdout:5/446: write d1/f9 [1284370,55393] 0 2024-03-22T07:16:05.216 INFO:tasks.workunit.client.1.smithi069.stdout:1/560: dread d1/d13/f27 [0,4194304] 0 2024-03-22T07:16:05.216 INFO:tasks.workunit.client.0.smithi063.stdout:7/666: truncate d5/da/f25 112779 0 2024-03-22T07:16:05.216 INFO:tasks.workunit.client.0.smithi063.stdout:8/904: truncate d0/d1b/d28/d7b/d45/f46 1517411 0 2024-03-22T07:16:05.216 INFO:tasks.workunit.client.1.smithi069.stdout:5/447: creat d1/d6/d26/d33/d5d/d72/f9e x:0 0 0 2024-03-22T07:16:05.217 INFO:tasks.workunit.client.1.smithi069.stdout:1/561: mkdir d1/d2a/d50/d88/dae 0 2024-03-22T07:16:05.218 INFO:tasks.workunit.client.0.smithi063.stdout:3/892: mkdir da/d63/d65/d4b/dfa/dfc/d121 0 2024-03-22T07:16:05.221 INFO:tasks.workunit.client.0.smithi063.stdout:7/667: symlink d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d4f/dd1/ldc 0 2024-03-22T07:16:05.224 INFO:tasks.workunit.client.1.smithi069.stdout:2/415: dread d18/d4d/d3e/d40/f51 [0,4194304] 0 2024-03-22T07:16:05.224 INFO:tasks.workunit.client.1.smithi069.stdout:2/416: fdatasync d18/d4d/d27/d33/f82 0 2024-03-22T07:16:05.226 INFO:tasks.workunit.client.0.smithi063.stdout:8/905: creat d0/d26/d2d/d59/dba/f130 x:0 0 0 2024-03-22T07:16:05.228 INFO:tasks.workunit.client.0.smithi063.stdout:2/731: dwrite d0/de/d12/d20/f3f [0,4194304] 0 2024-03-22T07:16:05.228 INFO:tasks.workunit.client.0.smithi063.stdout:5/702: dwrite d8/f1c [0,4194304] 0 2024-03-22T07:16:05.228 INFO:tasks.workunit.client.0.smithi063.stdout:2/732: chown d0/de/d35/l9a 900253045 1 2024-03-22T07:16:05.230 INFO:tasks.workunit.client.1.smithi069.stdout:7/555: dread d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/f7b [0,4194304] 0 2024-03-22T07:16:05.230 INFO:tasks.workunit.client.1.smithi069.stdout:7/556: fdatasync d9/db/d1a/db1/d57/d44/d53/d2e/f4d 0 2024-03-22T07:16:05.233 INFO:tasks.workunit.client.0.smithi063.stdout:5/703: rename d8/d13/d24/ca5 to d8/d13/d28/d57/dca/cd6 0 2024-03-22T07:16:05.234 INFO:tasks.workunit.client.0.smithi063.stdout:5/704: chown d8/d13/d1e/d31/d64/d65/f71 877191 1 2024-03-22T07:16:05.235 INFO:tasks.workunit.client.0.smithi063.stdout:2/733: creat d0/de/d58/d7f/d97/fef x:0 0 0 2024-03-22T07:16:05.235 INFO:tasks.workunit.client.0.smithi063.stdout:2/734: fdatasync d0/de/d1b/d63/f6c 0 2024-03-22T07:16:05.236 INFO:tasks.workunit.client.0.smithi063.stdout:5/705: mkdir d8/dba/dd7 0 2024-03-22T07:16:05.240 INFO:tasks.workunit.client.0.smithi063.stdout:5/706: rename d8/f1c to d8/d13/d24/fd8 0 2024-03-22T07:16:05.240 INFO:tasks.workunit.client.0.smithi063.stdout:5/707: dread - d8/dba/fcc zero size 2024-03-22T07:16:05.243 INFO:tasks.workunit.client.0.smithi063.stdout:3/893: write da/d63/d65/d83/ff4 [2442303,54585] 0 2024-03-22T07:16:05.243 INFO:tasks.workunit.client.0.smithi063.stdout:8/906: dread d0/d24/d33/d4c/f7d [0,4194304] 0 2024-03-22T07:16:05.247 INFO:tasks.workunit.client.0.smithi063.stdout:8/907: rename d0/d26/d2d/d83/dbc/da1/f102 to d0/d26/d2d/d59/dc0/f131 0 2024-03-22T07:16:05.259 INFO:tasks.workunit.client.1.smithi069.stdout:7/557: dread d9/db/d1a/f49 [0,4194304] 0 2024-03-22T07:16:05.260 INFO:tasks.workunit.client.0.smithi063.stdout:8/908: write d0/d1b/d4f/dde/df5/f53 [4707621,86460] 0 2024-03-22T07:16:05.262 INFO:tasks.workunit.client.1.smithi069.stdout:7/558: rmdir d9/db/d1a/db1/d57/d44/d53/d74/d30/da9 0 2024-03-22T07:16:05.269 INFO:tasks.workunit.client.0.smithi063.stdout:6/857: dwrite d3/d6/f3e [0,4194304] 0 2024-03-22T07:16:05.276 INFO:tasks.workunit.client.1.smithi069.stdout:5/448: dwrite d1/d6/d88/f93 [0,4194304] 0 2024-03-22T07:16:05.278 INFO:tasks.workunit.client.1.smithi069.stdout:5/449: mkdir d1/d6/d26/d7d/d9f 0 2024-03-22T07:16:05.279 INFO:tasks.workunit.client.1.smithi069.stdout:5/450: unlink d1/d6/d26/d3a/c8e 0 2024-03-22T07:16:05.279 INFO:tasks.workunit.client.1.smithi069.stdout:5/451: write d1/d10/d60/f83 [200683,118264] 0 2024-03-22T07:16:05.280 INFO:tasks.workunit.client.0.smithi063.stdout:1/939: dwrite d4/f3b [0,4194304] 0 2024-03-22T07:16:05.283 INFO:tasks.workunit.client.0.smithi063.stdout:1/940: mknod d4/df/d2c/c142 0 2024-03-22T07:16:05.283 INFO:tasks.workunit.client.0.smithi063.stdout:1/941: fdatasync d4/d5a/d91/db6/fb7 0 2024-03-22T07:16:05.291 INFO:tasks.workunit.client.1.smithi069.stdout:1/562: dwrite d1/d13/d82/f83 [0,4194304] 0 2024-03-22T07:16:05.293 INFO:tasks.workunit.client.1.smithi069.stdout:1/563: creat d1/d2a/d5e/da6/faf x:0 0 0 2024-03-22T07:16:05.294 INFO:tasks.workunit.client.1.smithi069.stdout:1/564: symlink d1/d13/d36/d6e/lb0 0 2024-03-22T07:16:05.299 INFO:tasks.workunit.client.1.smithi069.stdout:2/417: dwrite d18/d4d/d34/f7e [0,4194304] 0 2024-03-22T07:16:05.299 INFO:tasks.workunit.client.1.smithi069.stdout:2/418: dread - d18/d4d/d3e/d40/f76 zero size 2024-03-22T07:16:05.300 INFO:tasks.workunit.client.0.smithi063.stdout:7/668: dwrite d5/da/f35 [4194304,4194304] 0 2024-03-22T07:16:05.300 INFO:tasks.workunit.client.0.smithi063.stdout:7/669: creat d5/da/d28/da7/fdd x:0 0 0 2024-03-22T07:16:05.300 INFO:tasks.workunit.client.0.smithi063.stdout:7/670: chown d5/f13 187905912 1 2024-03-22T07:16:05.301 INFO:tasks.workunit.client.0.smithi063.stdout:7/671: fsync d5/da/d5f/d33/d57/f9e 0 2024-03-22T07:16:05.301 INFO:tasks.workunit.client.0.smithi063.stdout:2/735: dwrite d0/de/d1b/d8b/fb7 [0,4194304] 0 2024-03-22T07:16:05.301 INFO:tasks.workunit.client.1.smithi069.stdout:1/565: dread d1/d13/f45 [0,4194304] 0 2024-03-22T07:16:05.301 INFO:tasks.workunit.client.0.smithi063.stdout:3/894: dwrite da/d16/d89/d8c/dc1/f93 [0,4194304] 0 2024-03-22T07:16:05.302 INFO:tasks.workunit.client.0.smithi063.stdout:5/708: dwrite d8/d13/d1e/d31/d58/d7e/f84 [0,4194304] 0 2024-03-22T07:16:05.303 INFO:tasks.workunit.client.1.smithi069.stdout:2/419: rename d18/d4d/d34/f4a to d18/d4d/d27/d33/d60/d64/f8f 0 2024-03-22T07:16:05.304 INFO:tasks.workunit.client.0.smithi063.stdout:7/672: symlink d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/lde 0 2024-03-22T07:16:05.305 INFO:tasks.workunit.client.0.smithi063.stdout:1/942: write d4/df/d13/f40 [4197688,99356] 0 2024-03-22T07:16:05.306 INFO:tasks.workunit.client.0.smithi063.stdout:2/736: mknod d0/de/d58/cf0 0 2024-03-22T07:16:05.307 INFO:tasks.workunit.client.1.smithi069.stdout:1/566: rename d1/d2a/d5e/fa4 to d1/d2a/d5e/d9b/fb1 0 2024-03-22T07:16:05.307 INFO:tasks.workunit.client.0.smithi063.stdout:3/895: creat da/d16/f122 x:0 0 0 2024-03-22T07:16:05.308 INFO:tasks.workunit.client.0.smithi063.stdout:3/896: chown da/d16/d3f/f67 52877 1 2024-03-22T07:16:05.309 INFO:tasks.workunit.client.1.smithi069.stdout:2/420: unlink d18/d4d/f41 0 2024-03-22T07:16:05.309 INFO:tasks.workunit.client.0.smithi063.stdout:7/673: mknod d5/da/d28/d4b/da8/dbc/d62/d73/cdf 0 2024-03-22T07:16:05.310 INFO:tasks.workunit.client.1.smithi069.stdout:1/567: mknod d1/d2a/d84/cb2 0 2024-03-22T07:16:05.312 INFO:tasks.workunit.client.0.smithi063.stdout:3/897: creat da/d16/d89/f123 x:0 0 0 2024-03-22T07:16:05.313 INFO:tasks.workunit.client.1.smithi069.stdout:1/568: symlink d1/d13/d36/d6e/d73/lb3 0 2024-03-22T07:16:05.315 INFO:tasks.workunit.client.0.smithi063.stdout:3/898: creat da/d63/d65/d92/f124 x:0 0 0 2024-03-22T07:16:05.315 INFO:tasks.workunit.client.1.smithi069.stdout:7/559: dwrite d9/db/d1a/db1/d57/d75/f81 [0,4194304] 0 2024-03-22T07:16:05.316 INFO:tasks.workunit.client.1.smithi069.stdout:7/560: dread d9/db/d48/d4b/f65 [0,4194304] 0 2024-03-22T07:16:05.316 INFO:tasks.workunit.client.0.smithi063.stdout:1/943: dread d4/df/d13/f25 [0,4194304] 0 2024-03-22T07:16:05.317 INFO:tasks.workunit.client.0.smithi063.stdout:5/709: dread d8/d13/d28/fbd [0,4194304] 0 2024-03-22T07:16:05.318 INFO:tasks.workunit.client.1.smithi069.stdout:7/561: mkdir d9/db/d1a/db1/db3 0 2024-03-22T07:16:05.319 INFO:tasks.workunit.client.1.smithi069.stdout:7/562: stat d9/db/d1a/db1/d57/d44/d53/d1d/faa 0 2024-03-22T07:16:05.319 INFO:tasks.workunit.client.1.smithi069.stdout:7/563: write d9/db/f16 [1989523,128097] 0 2024-03-22T07:16:05.319 INFO:tasks.workunit.client.0.smithi063.stdout:2/737: dread d0/de/d1b/d63/f6c [0,4194304] 0 2024-03-22T07:16:05.320 INFO:tasks.workunit.client.1.smithi069.stdout:5/452: dwrite d1/d6/d21/f79 [0,4194304] 0 2024-03-22T07:16:05.320 INFO:tasks.workunit.client.0.smithi063.stdout:1/944: mknod d4/d71/dbb/dc0/c143 0 2024-03-22T07:16:05.321 INFO:tasks.workunit.client.0.smithi063.stdout:8/909: dwrite d0/d24/d33/fe6 [0,4194304] 0 2024-03-22T07:16:05.322 INFO:tasks.workunit.client.1.smithi069.stdout:7/564: creat d9/db/d1a/db1/d57/d75/d7a/d96/fb4 x:0 0 0 2024-03-22T07:16:05.328 INFO:tasks.workunit.client.1.smithi069.stdout:7/565: mkdir d9/d42/d8a/db5 0 2024-03-22T07:16:05.329 INFO:tasks.workunit.client.0.smithi063.stdout:1/945: symlink d4/df/d13/d2b/d6c/da6/l144 0 2024-03-22T07:16:05.329 INFO:tasks.workunit.client.1.smithi069.stdout:7/566: write d9/db/f94 [11998,69036] 0 2024-03-22T07:16:05.332 INFO:tasks.workunit.client.0.smithi063.stdout:3/899: write da/d16/d89/da9/dd3/f110 [747952,119835] 0 2024-03-22T07:16:05.332 INFO:tasks.workunit.client.0.smithi063.stdout:3/900: chown da/d16/d3f/c5d 770 1 2024-03-22T07:16:05.332 INFO:tasks.workunit.client.0.smithi063.stdout:6/858: dwrite d3/d15/d16/d60/d6d/d4e/d5a/d88/fb2 [0,4194304] 0 2024-03-22T07:16:05.332 INFO:tasks.workunit.client.0.smithi063.stdout:6/859: stat d3/d6/d2a/d38/da5/d32/l36 0 2024-03-22T07:16:05.332 INFO:tasks.workunit.client.0.smithi063.stdout:6/860: truncate d3/d6/f3e 5154574 0 2024-03-22T07:16:05.334 INFO:tasks.workunit.client.0.smithi063.stdout:5/710: rmdir d8/d13/d28 39 2024-03-22T07:16:05.336 INFO:tasks.workunit.client.0.smithi063.stdout:2/738: write d0/de/d12/d20/fb3 [412216,124359] 0 2024-03-22T07:16:05.337 INFO:tasks.workunit.client.0.smithi063.stdout:1/946: creat d4/d14/d57/f145 x:0 0 0 2024-03-22T07:16:05.347 INFO:tasks.workunit.client.0.smithi063.stdout:3/901: symlink da/d63/d65/d92/l125 0 2024-03-22T07:16:05.347 INFO:tasks.workunit.client.0.smithi063.stdout:3/902: fdatasync da/d16/d26/f27 0 2024-03-22T07:16:05.347 INFO:tasks.workunit.client.0.smithi063.stdout:3/903: truncate da/d63/d65/d7b/fd9 914744 0 2024-03-22T07:16:05.348 INFO:tasks.workunit.client.0.smithi063.stdout:6/861: mkdir d3/d15/d16/d11e 0 2024-03-22T07:16:05.348 INFO:tasks.workunit.client.0.smithi063.stdout:6/862: chown d3/d6/d2a/d38/d100/f106 239666883 1 2024-03-22T07:16:05.349 INFO:tasks.workunit.client.0.smithi063.stdout:6/863: creat d3/d6/d35/f11f x:0 0 0 2024-03-22T07:16:05.349 INFO:tasks.workunit.client.0.smithi063.stdout:6/864: read d3/d6/d2a/d38/da5/f90 [3549012,51161] 0 2024-03-22T07:16:05.350 INFO:tasks.workunit.client.0.smithi063.stdout:2/739: creat d0/de/d35/d36/ff1 x:0 0 0 2024-03-22T07:16:05.350 INFO:tasks.workunit.client.0.smithi063.stdout:2/740: chown d0/de/d12/c18 49409202 1 2024-03-22T07:16:05.351 INFO:tasks.workunit.client.0.smithi063.stdout:2/741: write d0/de/d58/f73 [737154,118098] 0 2024-03-22T07:16:05.353 INFO:tasks.workunit.client.0.smithi063.stdout:1/947: truncate d4/df/d13/d2b/d6c/da8/fc5 2796270 0 2024-03-22T07:16:05.353 INFO:tasks.workunit.client.0.smithi063.stdout:1/948: stat d4/d14/d123/l128 0 2024-03-22T07:16:05.353 INFO:tasks.workunit.client.0.smithi063.stdout:1/949: stat d4/df/d13/d39/f8e 0 2024-03-22T07:16:05.353 INFO:tasks.workunit.client.0.smithi063.stdout:1/950: fdatasync d4/d14/d57/f145 0 2024-03-22T07:16:05.357 INFO:tasks.workunit.client.0.smithi063.stdout:1/951: symlink d4/df/d13/d10d/l146 0 2024-03-22T07:16:05.359 INFO:tasks.workunit.client.0.smithi063.stdout:1/952: symlink d4/df/d13/d11d/l147 0 2024-03-22T07:16:05.360 INFO:tasks.workunit.client.0.smithi063.stdout:1/953: mknod d4/df/d13/d11d/c148 0 2024-03-22T07:16:05.361 INFO:tasks.workunit.client.0.smithi063.stdout:1/954: symlink d4/df/d13/d2b/d3a/l149 0 2024-03-22T07:16:05.362 INFO:tasks.workunit.client.0.smithi063.stdout:7/674: dwrite d5/f13 [0,4194304] 0 2024-03-22T07:16:05.362 INFO:tasks.workunit.client.0.smithi063.stdout:7/675: stat d5/da/d5f/f97 0 2024-03-22T07:16:05.367 INFO:tasks.workunit.client.1.smithi069.stdout:5/453: dwrite d1/d6/d86/d99/f12 [0,4194304] 0 2024-03-22T07:16:05.369 INFO:tasks.workunit.client.1.smithi069.stdout:5/454: mkdir d1/d6/d26/d3e/da0 0 2024-03-22T07:16:05.369 INFO:tasks.workunit.client.1.smithi069.stdout:5/455: creat d1/d6/d88/fa1 x:0 0 0 2024-03-22T07:16:05.371 INFO:tasks.workunit.client.1.smithi069.stdout:5/456: mknod d1/d6/d26/d3a/ca2 0 2024-03-22T07:16:05.372 INFO:tasks.workunit.client.1.smithi069.stdout:2/421: dwrite d18/d4d/d34/f7e [0,4194304] 0 2024-03-22T07:16:05.372 INFO:tasks.workunit.client.1.smithi069.stdout:5/457: mknod d1/d6/d26/d33/d5d/d72/d78/ca3 0 2024-03-22T07:16:05.373 INFO:tasks.workunit.client.0.smithi063.stdout:7/676: read d5/da/f35 [5807161,91720] 0 2024-03-22T07:16:05.374 INFO:tasks.workunit.client.1.smithi069.stdout:5/458: creat d1/d6/d26/d33/d5d/fa4 x:0 0 0 2024-03-22T07:16:05.375 INFO:tasks.workunit.client.1.smithi069.stdout:5/459: creat d1/d7/fa5 x:0 0 0 2024-03-22T07:16:05.375 INFO:tasks.workunit.client.1.smithi069.stdout:5/460: stat d1/d7/d3d 0 2024-03-22T07:16:05.383 INFO:tasks.workunit.client.1.smithi069.stdout:2/422: write d18/d4d/d25/f47 [3886397,122997] 0 2024-03-22T07:16:05.386 INFO:tasks.workunit.client.1.smithi069.stdout:1/569: dwrite d1/d2a/d5e/d9b/fb1 [0,4194304] 0 2024-03-22T07:16:05.387 INFO:tasks.workunit.client.1.smithi069.stdout:1/570: creat d1/d13/d36/fb4 x:0 0 0 2024-03-22T07:16:05.387 INFO:tasks.workunit.client.1.smithi069.stdout:1/571: stat d1/d2a/d50/l9e 0 2024-03-22T07:16:05.390 INFO:tasks.workunit.client.1.smithi069.stdout:5/461: write d1/d6/d26/d3a/f6e [4176491,110250] 0 2024-03-22T07:16:05.391 INFO:tasks.workunit.client.1.smithi069.stdout:2/423: dread d18/f70 [0,4194304] 0 2024-03-22T07:16:05.391 INFO:tasks.workunit.client.1.smithi069.stdout:2/424: readlink d18/d4d/d27/l83 0 2024-03-22T07:16:05.392 INFO:tasks.workunit.client.0.smithi063.stdout:8/910: dwrite d0/d26/d2d/d59/dba/dc9/fff [0,4194304] 0 2024-03-22T07:16:05.392 INFO:tasks.workunit.client.0.smithi063.stdout:6/865: dwrite d3/d6/d35/f67 [4194304,4194304] 0 2024-03-22T07:16:05.392 INFO:tasks.workunit.client.0.smithi063.stdout:5/711: dwrite d8/d13/d24/d62/fd3 [4194304,4194304] 0 2024-03-22T07:16:05.393 INFO:tasks.workunit.client.1.smithi069.stdout:5/462: creat d1/d6/d26/d3e/d9c/d9d/fa6 x:0 0 0 2024-03-22T07:16:05.394 INFO:tasks.workunit.client.0.smithi063.stdout:6/866: creat d3/d15/d16/d60/d6d/d107/d10b/dde/f120 x:0 0 0 2024-03-22T07:16:05.394 INFO:tasks.workunit.client.0.smithi063.stdout:6/867: fsync d3/d6/d2a/fee 0 2024-03-22T07:16:05.396 INFO:tasks.workunit.client.0.smithi063.stdout:5/712: creat d8/fd9 x:0 0 0 2024-03-22T07:16:05.397 INFO:tasks.workunit.client.0.smithi063.stdout:6/868: symlink d3/d15/d16/d69/d8c/l121 0 2024-03-22T07:16:05.398 INFO:tasks.workunit.client.0.smithi063.stdout:5/713: mkdir d8/d13/d1e/d3c/da0/dda 0 2024-03-22T07:16:05.400 INFO:tasks.workunit.client.0.smithi063.stdout:6/869: mkdir d3/d15/d16/d60/d6d/d4e/d5a/d88/df5/d114/d122 0 2024-03-22T07:16:05.400 INFO:tasks.workunit.client.0.smithi063.stdout:6/870: fsync d3/f82 0 2024-03-22T07:16:05.402 INFO:tasks.workunit.client.0.smithi063.stdout:3/904: dwrite da/f52 [0,4194304] 0 2024-03-22T07:16:05.402 INFO:tasks.workunit.client.0.smithi063.stdout:3/905: write da/f3d [8743679,52976] 0 2024-03-22T07:16:05.403 INFO:tasks.workunit.client.0.smithi063.stdout:5/714: creat d8/d13/fdb x:0 0 0 2024-03-22T07:16:05.403 INFO:tasks.workunit.client.0.smithi063.stdout:5/715: chown d8/d13/d1e/d31/d64 239255 1 2024-03-22T07:16:05.403 INFO:tasks.workunit.client.0.smithi063.stdout:5/716: write d8/d13/f18 [4756990,87532] 0 2024-03-22T07:16:05.403 INFO:tasks.workunit.client.0.smithi063.stdout:5/717: dread - d8/d13/d1e/d31/d64/d65/d86/f92 zero size 2024-03-22T07:16:05.404 INFO:tasks.workunit.client.0.smithi063.stdout:6/871: mkdir d3/d15/d123 0 2024-03-22T07:16:05.405 INFO:tasks.workunit.client.1.smithi069.stdout:7/567: dwrite d9/db/d1a/db1/d57/d75/f9c [0,4194304] 0 2024-03-22T07:16:05.405 INFO:tasks.workunit.client.1.smithi069.stdout:7/568: write d9/db/d1a/db1/d57/d75/d7a/f8f [1943016,31548] 0 2024-03-22T07:16:05.405 INFO:tasks.workunit.client.1.smithi069.stdout:5/463: dread d1/d6/f2e [0,4194304] 0 2024-03-22T07:16:05.405 INFO:tasks.workunit.client.1.smithi069.stdout:7/569: readlink d9/da1/la3 0 2024-03-22T07:16:05.408 INFO:tasks.workunit.client.0.smithi063.stdout:3/906: rename da/d63/d65/d81/fa4 to da/d63/d105/f126 0 2024-03-22T07:16:05.410 INFO:tasks.workunit.client.0.smithi063.stdout:5/718: dread d8/d13/d1e/d31/d58/f68 [0,4194304] 0 2024-03-22T07:16:05.410 INFO:tasks.workunit.client.0.smithi063.stdout:5/719: write d8/d13/d1e/d3c/f9b [117732,31486] 0 2024-03-22T07:16:05.411 INFO:tasks.workunit.client.0.smithi063.stdout:6/872: truncate d3/f89 1513315 0 2024-03-22T07:16:05.411 INFO:tasks.workunit.client.0.smithi063.stdout:6/873: fdatasync d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/f5b 0 2024-03-22T07:16:05.414 INFO:tasks.workunit.client.0.smithi063.stdout:6/874: mknod d3/d6/d35/d3a/c124 0 2024-03-22T07:16:05.414 INFO:tasks.workunit.client.0.smithi063.stdout:6/875: truncate d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/fe3 476382 0 2024-03-22T07:16:05.414 INFO:tasks.workunit.client.0.smithi063.stdout:6/876: readlink d3/d15/d16/d60/d6d/d4e/d5a/le4 0 2024-03-22T07:16:05.415 INFO:tasks.workunit.client.0.smithi063.stdout:2/742: dwrite d0/de/d12/f1f [0,4194304] 0 2024-03-22T07:16:05.415 INFO:tasks.workunit.client.0.smithi063.stdout:1/955: dwrite d4/df/d13/d2b/f84 [0,4194304] 0 2024-03-22T07:16:05.415 INFO:tasks.workunit.client.0.smithi063.stdout:1/956: fdatasync d4/df/d13/d39/f8e 0 2024-03-22T07:16:05.416 INFO:tasks.workunit.client.0.smithi063.stdout:8/911: dwrite d0/d1b/d28/d3e/db2/dbd/dd0/f120 [0,4194304] 0 2024-03-22T07:16:05.417 INFO:tasks.workunit.client.0.smithi063.stdout:8/912: creat d0/d26/d2d/d59/dc0/f132 x:0 0 0 2024-03-22T07:16:05.417 INFO:tasks.workunit.client.0.smithi063.stdout:5/720: unlink d8/d13/c26 0 2024-03-22T07:16:05.424 INFO:tasks.workunit.client.1.smithi069.stdout:1/572: dwrite d1/d13/f26 [0,4194304] 0 2024-03-22T07:16:05.424 INFO:tasks.workunit.client.1.smithi069.stdout:1/573: write d1/d2a/d50/d88/d41/d7a/f90 [123550,60676] 0 2024-03-22T07:16:05.424 INFO:tasks.workunit.client.1.smithi069.stdout:1/574: dread - d1/d13/d36/f81 zero size 2024-03-22T07:16:05.424 INFO:tasks.workunit.client.1.smithi069.stdout:1/575: stat d1/d2a/d50/d88/d41/d64 0 2024-03-22T07:16:05.430 INFO:tasks.workunit.client.0.smithi063.stdout:1/957: write d4/df/d2c/f90 [6635998,38401] 0 2024-03-22T07:16:05.430 INFO:tasks.workunit.client.1.smithi069.stdout:1/576: rmdir d1/d2a/d50/d4c 39 2024-03-22T07:16:05.430 INFO:tasks.workunit.client.1.smithi069.stdout:2/425: dwrite d18/d4d/f28 [0,4194304] 0 2024-03-22T07:16:05.430 INFO:tasks.workunit.client.1.smithi069.stdout:2/426: fdatasync d18/d4d/d3e/f65 0 2024-03-22T07:16:05.431 INFO:tasks.workunit.client.1.smithi069.stdout:1/577: mkdir d1/d2a/d50/d88/dae/db5 0 2024-03-22T07:16:05.432 INFO:tasks.workunit.client.1.smithi069.stdout:1/578: mkdir d1/d2a/db6 0 2024-03-22T07:16:05.438 INFO:tasks.workunit.client.1.smithi069.stdout:1/579: rename d1/d2a/d50/d4c/f86 to d1/d2a/d5e/d9b/fb7 0 2024-03-22T07:16:05.438 INFO:tasks.workunit.client.1.smithi069.stdout:7/570: dwrite d9/db/d1a/db1/d57/d44/d53/d2e/fa6 [0,4194304] 0 2024-03-22T07:16:05.438 INFO:tasks.workunit.client.0.smithi063.stdout:2/743: symlink d0/de/d58/d7f/d97/da2/lf2 0 2024-03-22T07:16:05.439 INFO:tasks.workunit.client.0.smithi063.stdout:2/744: chown d0/de/d35/d36 6 1 2024-03-22T07:16:05.443 INFO:tasks.workunit.client.0.smithi063.stdout:5/721: link d8/cc8 d8/d13/d24/da4/d8b/db6/cdc 0 2024-03-22T07:16:05.445 INFO:tasks.workunit.client.0.smithi063.stdout:1/958: symlink d4/df/d13/d39/d6f/d116/l14a 0 2024-03-22T07:16:05.446 INFO:tasks.workunit.client.0.smithi063.stdout:1/959: read d4/d8b/da7/f12e [633648,120415] 0 2024-03-22T07:16:05.447 INFO:tasks.workunit.client.0.smithi063.stdout:1/960: rename d4/cf2 to d4/df/d13/d39/d6f/d116/c14b 0 2024-03-22T07:16:05.450 INFO:tasks.workunit.client.0.smithi063.stdout:1/961: creat d4/df/d13/d39/f14c x:0 0 0 2024-03-22T07:16:05.451 INFO:tasks.workunit.client.0.smithi063.stdout:2/745: dread d0/de/d12/f27 [0,4194304] 0 2024-03-22T07:16:05.451 INFO:tasks.workunit.client.0.smithi063.stdout:2/746: creat d0/de/d1b/d32/ff3 x:0 0 0 2024-03-22T07:16:05.452 INFO:tasks.workunit.client.1.smithi069.stdout:7/571: rmdir d9/db/d1a/db1/d57/d75/d7a/d96 39 2024-03-22T07:16:05.452 INFO:tasks.workunit.client.0.smithi063.stdout:5/722: dread d8/d13/d1e/f2f [0,4194304] 0 2024-03-22T07:16:05.455 INFO:tasks.workunit.client.0.smithi063.stdout:2/747: symlink d0/de/d1b/d63/dc6/lf4 0 2024-03-22T07:16:05.457 INFO:tasks.workunit.client.1.smithi069.stdout:8/724: sync 2024-03-22T07:16:05.458 INFO:tasks.workunit.client.1.smithi069.stdout:8/725: dread - d1/d1c/d48/d5e/fad zero size 2024-03-22T07:16:05.458 INFO:tasks.workunit.client.1.smithi069.stdout:8/726: fsync d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/f67 0 2024-03-22T07:16:05.458 INFO:tasks.workunit.client.0.smithi063.stdout:5/723: truncate d8/d13/d28/d57/fcb 969274 0 2024-03-22T07:16:05.460 INFO:tasks.workunit.client.0.smithi063.stdout:2/748: symlink d0/de/d35/lf5 0 2024-03-22T07:16:05.460 INFO:tasks.workunit.client.0.smithi063.stdout:2/749: fsync d0/de/f30 0 2024-03-22T07:16:05.460 INFO:tasks.workunit.client.0.smithi063.stdout:7/677: write d5/da/d28/d4b/da8/dbc/d62/d73/d1d/f3f [1429569,128907] 0 2024-03-22T07:16:05.461 INFO:tasks.workunit.client.1.smithi069.stdout:8/727: creat d1/d93/ddf/fe9 x:0 0 0 2024-03-22T07:16:05.463 INFO:tasks.workunit.client.0.smithi063.stdout:5/724: symlink d8/d13/d1e/d3c/da0/ldd 0 2024-03-22T07:16:05.463 INFO:tasks.workunit.client.1.smithi069.stdout:0/691: sync 2024-03-22T07:16:05.464 INFO:tasks.workunit.client.0.smithi063.stdout:2/750: rmdir d0/de/d58/d7f/d97/da2 39 2024-03-22T07:16:05.466 INFO:tasks.workunit.client.0.smithi063.stdout:7/678: creat d5/da/d5f/d33/d57/db3/fe0 x:0 0 0 2024-03-22T07:16:05.467 INFO:tasks.workunit.client.1.smithi069.stdout:2/427: dwrite d18/d4d/d34/f7e [4194304,4194304] 0 2024-03-22T07:16:05.467 INFO:tasks.workunit.client.1.smithi069.stdout:2/428: write d18/d4d/d27/d56/f88 [544192,58977] 0 2024-03-22T07:16:05.468 INFO:tasks.workunit.client.0.smithi063.stdout:5/725: mkdir d8/d13/d24/da4/d8b/db6/dde 0 2024-03-22T07:16:05.469 INFO:tasks.workunit.client.0.smithi063.stdout:3/907: dwrite da/d63/d65/d92/d96/f10f [0,4194304] 0 2024-03-22T07:16:05.470 INFO:tasks.workunit.client.1.smithi069.stdout:9/487: sync 2024-03-22T07:16:05.470 INFO:tasks.workunit.client.1.smithi069.stdout:9/488: chown d2/d31/f40 8375184 1 2024-03-22T07:16:05.470 INFO:tasks.workunit.client.1.smithi069.stdout:9/489: fdatasync d2/dd/d3c/d53/d9d/f97 0 2024-03-22T07:16:05.470 INFO:tasks.workunit.client.1.smithi069.stdout:3/636: sync 2024-03-22T07:16:05.470 INFO:tasks.workunit.client.1.smithi069.stdout:5/464: dwrite d1/d6/f6f [0,4194304] 0 2024-03-22T07:16:05.470 INFO:tasks.workunit.client.1.smithi069.stdout:6/526: sync 2024-03-22T07:16:05.470 INFO:tasks.workunit.client.1.smithi069.stdout:6/527: write d4/f6c [1019339,63773] 0 2024-03-22T07:16:05.470 INFO:tasks.workunit.client.1.smithi069.stdout:6/528: write d4/f6b [8362481,63840] 0 2024-03-22T07:16:05.470 INFO:tasks.workunit.client.1.smithi069.stdout:6/529: creat d4/dd/d12/d13/fa1 x:0 0 0 2024-03-22T07:16:05.470 INFO:tasks.workunit.client.1.smithi069.stdout:4/720: sync 2024-03-22T07:16:05.470 INFO:tasks.workunit.client.1.smithi069.stdout:4/721: creat d6/d12/d5d/ff6 x:0 0 0 2024-03-22T07:16:05.471 INFO:tasks.workunit.client.1.smithi069.stdout:4/722: chown d6/d12/db2/ff5 1468 1 2024-03-22T07:16:05.473 INFO:tasks.workunit.client.1.smithi069.stdout:5/465: unlink d1/d7/f25 0 2024-03-22T07:16:05.474 INFO:tasks.workunit.client.0.smithi063.stdout:1/962: dwrite d4/df/d13/d2b/d6c/d85/fc7 [4194304,4194304] 0 2024-03-22T07:16:05.474 INFO:tasks.workunit.client.0.smithi063.stdout:7/679: symlink d5/da/d28/d4b/da8/dbc/le1 0 2024-03-22T07:16:05.474 INFO:tasks.workunit.client.1.smithi069.stdout:1/580: dwrite d1/f42 [4194304,4194304] 0 2024-03-22T07:16:05.476 INFO:tasks.workunit.client.0.smithi063.stdout:5/726: mkdir d8/d13/d1e/d31/d64/d65/ddf 0 2024-03-22T07:16:05.476 INFO:tasks.workunit.client.0.smithi063.stdout:5/727: dread - d8/d13/d1e/d3c/fae zero size 2024-03-22T07:16:05.476 INFO:tasks.workunit.client.1.smithi069.stdout:6/530: symlink d4/d7/da/d3f/la2 0 2024-03-22T07:16:05.476 INFO:tasks.workunit.client.1.smithi069.stdout:6/531: stat d4/d7/da/f94 0 2024-03-22T07:16:05.478 INFO:tasks.workunit.client.1.smithi069.stdout:3/637: dread d5/d9/d26/d3f/d56/d7f/fa1 [0,4194304] 0 2024-03-22T07:16:05.480 INFO:tasks.workunit.client.0.smithi063.stdout:6/877: truncate d3/d6/d35/f67 864138 0 2024-03-22T07:16:05.481 INFO:tasks.workunit.client.1.smithi069.stdout:4/723: link d6/d9b/d1f/d58/f85 d6/dc4/ff7 0 2024-03-22T07:16:05.485 INFO:tasks.workunit.client.1.smithi069.stdout:2/429: link d18/d4d/d3e/d40/f44 d18/d4d/d27/d33/d60/d64/d89/f90 0 2024-03-22T07:16:05.485 INFO:tasks.workunit.client.0.smithi063.stdout:2/751: rename d0/de/d12/d20/f88 to d0/de/d35/d36/ff6 0 2024-03-22T07:16:05.485 INFO:tasks.workunit.client.0.smithi063.stdout:2/752: write d0/de/d74/dca/fd2 [726914,32241] 0 2024-03-22T07:16:05.485 INFO:tasks.workunit.client.0.smithi063.stdout:2/753: stat d0/de/d35/d36/l4f 0 2024-03-22T07:16:05.485 INFO:tasks.workunit.client.1.smithi069.stdout:7/572: dwrite d9/db/d1a/db1/d57/d75/d7a/f80 [4194304,4194304] 0 2024-03-22T07:16:05.486 INFO:tasks.workunit.client.1.smithi069.stdout:5/466: mkdir d1/d6/d26/d33/d5d/d72/da7 0 2024-03-22T07:16:05.486 INFO:tasks.workunit.client.0.smithi063.stdout:6/878: dread d3/d15/d16/d60/d6d/d4c/f6f [0,4194304] 0 2024-03-22T07:16:05.486 INFO:tasks.workunit.client.0.smithi063.stdout:6/879: creat d3/d6/d2a/d38/da5/f125 x:0 0 0 2024-03-22T07:16:05.488 INFO:tasks.workunit.client.1.smithi069.stdout:8/728: dwrite d1/d1c/f38 [0,4194304] 0 2024-03-22T07:16:05.489 INFO:tasks.workunit.client.0.smithi063.stdout:1/963: mkdir d4/d71/dbb/d14d 0 2024-03-22T07:16:05.492 INFO:tasks.workunit.client.0.smithi063.stdout:3/908: mkdir da/d16/d26/d127 0 2024-03-22T07:16:05.492 INFO:tasks.workunit.client.1.smithi069.stdout:1/581: getdents d1/d2a/d5e/d9b 0 2024-03-22T07:16:05.492 INFO:tasks.workunit.client.1.smithi069.stdout:1/582: readlink d1/d2a/d50/d88/d41/d1e/l33 0 2024-03-22T07:16:05.493 INFO:tasks.workunit.client.1.smithi069.stdout:3/638: creat d5/d9/d26/d36/d41/fd0 x:0 0 0 2024-03-22T07:16:05.494 INFO:tasks.workunit.client.1.smithi069.stdout:4/724: mkdir d6/d16/d68/df8 0 2024-03-22T07:16:05.495 INFO:tasks.workunit.client.0.smithi063.stdout:7/680: rename d5/da/d5f/f8c to d5/da/fe2 0 2024-03-22T07:16:05.495 INFO:tasks.workunit.client.0.smithi063.stdout:7/681: creat d5/d16/fe3 x:0 0 0 2024-03-22T07:16:05.495 INFO:tasks.workunit.client.1.smithi069.stdout:2/430: mkdir d18/d91 0 2024-03-22T07:16:05.497 INFO:tasks.workunit.client.1.smithi069.stdout:7/573: rmdir d9/db/d1a/db1/d57/d75/d7a/d96 39 2024-03-22T07:16:05.498 INFO:tasks.workunit.client.1.smithi069.stdout:5/467: creat d1/d6/fa8 x:0 0 0 2024-03-22T07:16:05.498 INFO:tasks.workunit.client.1.smithi069.stdout:5/468: creat d1/d6/fa9 x:0 0 0 2024-03-22T07:16:05.498 INFO:tasks.workunit.client.0.smithi063.stdout:1/964: creat d4/d71/dbb/f14e x:0 0 0 2024-03-22T07:16:05.500 INFO:tasks.workunit.client.0.smithi063.stdout:3/909: truncate da/d16/d3f/f79 1078163 0 2024-03-22T07:16:05.501 INFO:tasks.workunit.client.1.smithi069.stdout:1/583: dread d1/d2a/f4e [0,4194304] 0 2024-03-22T07:16:05.502 INFO:tasks.workunit.client.0.smithi063.stdout:5/728: dwrite d8/d13/f39 [0,4194304] 0 2024-03-22T07:16:05.502 INFO:tasks.workunit.client.0.smithi063.stdout:5/729: chown d8/d13/d1e/d3c/fb7 0 1 2024-03-22T07:16:05.502 INFO:tasks.workunit.client.0.smithi063.stdout:6/880: rename d3/d15/d16/d69/d8c/l121 to d3/d6/dcd/l126 0 2024-03-22T07:16:05.503 INFO:tasks.workunit.client.1.smithi069.stdout:8/729: link d1/d1c/d3e/d66/d9e/dd1/fda d1/d1c/d3e/d88/dcd/d6c/d73/fea 0 2024-03-22T07:16:05.503 INFO:tasks.workunit.client.1.smithi069.stdout:8/730: truncate d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/f76 732620 0 2024-03-22T07:16:05.505 INFO:tasks.workunit.client.0.smithi063.stdout:7/682: creat d5/da/d28/d4b/da8/dbc/d62/d73/fe4 x:0 0 0 2024-03-22T07:16:05.505 INFO:tasks.workunit.client.0.smithi063.stdout:7/683: dread - d5/da/d28/d4b/da8/dbc/fac zero size 2024-03-22T07:16:05.505 INFO:tasks.workunit.client.0.smithi063.stdout:1/965: stat d4/df/d13/d2b/d86/l106 0 2024-03-22T07:16:05.505 INFO:tasks.workunit.client.0.smithi063.stdout:1/966: dread - d4/df/d13/f11b zero size 2024-03-22T07:16:05.506 INFO:tasks.workunit.client.0.smithi063.stdout:1/967: creat d4/df/d13/d39/d6f/f14f x:0 0 0 2024-03-22T07:16:05.506 INFO:tasks.workunit.client.1.smithi069.stdout:2/431: write d18/d4d/d34/f5f [387806,60794] 0 2024-03-22T07:16:05.508 INFO:tasks.workunit.client.1.smithi069.stdout:4/725: truncate d6/d12/d18/d21/f6a 3841839 0 2024-03-22T07:16:05.510 INFO:tasks.workunit.client.0.smithi063.stdout:5/730: mknod d8/d13/d24/da4/d8b/ce0 0 2024-03-22T07:16:05.510 INFO:tasks.workunit.client.0.smithi063.stdout:5/731: read d8/d13/d1e/d31/f46 [1560229,23416] 0 2024-03-22T07:16:05.510 INFO:tasks.workunit.client.1.smithi069.stdout:7/574: creat d9/db/d1a/db1/fb6 x:0 0 0 2024-03-22T07:16:05.514 INFO:tasks.workunit.client.0.smithi063.stdout:1/968: symlink d4/df/d13/d11d/l150 0 2024-03-22T07:16:05.518 INFO:tasks.workunit.client.0.smithi063.stdout:2/754: getdents d0/de/d1b/d63/dc6 0 2024-03-22T07:16:05.519 INFO:tasks.workunit.client.0.smithi063.stdout:1/969: mkdir d4/df/d13/d39/d63/d151 0 2024-03-22T07:16:05.523 INFO:tasks.workunit.client.0.smithi063.stdout:2/755: rename d0/de/d58/f7b to d0/de/d74/dca/ff7 0 2024-03-22T07:16:05.523 INFO:tasks.workunit.client.0.smithi063.stdout:2/756: write d0/de/d1b/d63/fe0 [858291,81179] 0 2024-03-22T07:16:05.528 INFO:tasks.workunit.client.0.smithi063.stdout:1/970: truncate d4/d71/d88/da1/ddb/f12b 1418463 0 2024-03-22T07:16:05.528 INFO:tasks.workunit.client.1.smithi069.stdout:4/726: dread d6/d9b/d56/fa8 [0,4194304] 0 2024-03-22T07:16:05.528 INFO:tasks.workunit.client.1.smithi069.stdout:4/727: dread d6/d16/d68/fa9 [0,4194304] 0 2024-03-22T07:16:05.529 INFO:tasks.workunit.client.1.smithi069.stdout:4/728: write d6/d12/f6f [1804323,117249] 0 2024-03-22T07:16:05.529 INFO:tasks.workunit.client.0.smithi063.stdout:5/732: dread d8/f14 [0,4194304] 0 2024-03-22T07:16:05.530 INFO:tasks.workunit.client.0.smithi063.stdout:3/910: dwrite da/d16/d89/da9/fcc [0,4194304] 0 2024-03-22T07:16:05.530 INFO:tasks.workunit.client.0.smithi063.stdout:2/757: symlink d0/de/d58/d7f/lf8 0 2024-03-22T07:16:05.531 INFO:tasks.workunit.client.0.smithi063.stdout:1/971: mkdir d4/df/d13/d39/d6f/d152 0 2024-03-22T07:16:05.532 INFO:tasks.workunit.client.1.smithi069.stdout:5/469: mknod d1/d10/d60/caa 0 2024-03-22T07:16:05.536 INFO:tasks.workunit.client.1.smithi069.stdout:8/731: link d1/fd d1/d1c/d3e/dde/feb 0 2024-03-22T07:16:05.536 INFO:tasks.workunit.client.1.smithi069.stdout:8/732: chown d1/d1c/d3e/d88/dd3 29 1 2024-03-22T07:16:05.536 INFO:tasks.workunit.client.1.smithi069.stdout:8/733: truncate d1/d1c/d3e/d66/f78 452719 0 2024-03-22T07:16:05.539 INFO:tasks.workunit.client.0.smithi063.stdout:6/881: dwrite d3/d6/d2a/d38/da5/d32/d51/fcc [4194304,4194304] 0 2024-03-22T07:16:05.540 INFO:tasks.workunit.client.0.smithi063.stdout:6/882: chown d3/d15/d16/d60/d6d/d4e/d5a/d88/c8e 340799 1 2024-03-22T07:16:05.540 INFO:tasks.workunit.client.1.smithi069.stdout:7/575: dwrite d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/f7b [0,4194304] 0 2024-03-22T07:16:05.541 INFO:tasks.workunit.client.0.smithi063.stdout:3/911: unlink da/d63/d65/d81/d9b/d9f/c116 0 2024-03-22T07:16:05.541 INFO:tasks.workunit.client.0.smithi063.stdout:3/912: getdents da/d63/d65/d81/da5 0 2024-03-22T07:16:05.541 INFO:tasks.workunit.client.0.smithi063.stdout:4/787: sync 2024-03-22T07:16:05.541 INFO:tasks.workunit.client.0.smithi063.stdout:7/684: dwrite d5/da/d28/d4b/da8/dbc/d62/d73/d1d/f3f [0,4194304] 0 2024-03-22T07:16:05.541 INFO:tasks.workunit.client.0.smithi063.stdout:9/812: sync 2024-03-22T07:16:05.541 INFO:tasks.workunit.client.1.smithi069.stdout:4/729: creat d6/daa/dce/de4/ff9 x:0 0 0 2024-03-22T07:16:05.542 INFO:tasks.workunit.client.1.smithi069.stdout:5/470: mkdir d1/d6/d86/dab 0 2024-03-22T07:16:05.543 INFO:tasks.workunit.client.0.smithi063.stdout:2/758: mkdir d0/ded/df9 0 2024-03-22T07:16:05.545 INFO:tasks.workunit.client.0.smithi063.stdout:1/972: creat d4/d14/f153 x:0 0 0 2024-03-22T07:16:05.545 INFO:tasks.workunit.client.0.smithi063.stdout:1/973: chown d4/d71/d88/fb3 0 1 2024-03-22T07:16:05.551 INFO:tasks.workunit.client.1.smithi069.stdout:1/584: getdents d1/d2a/d5e/d93 0 2024-03-22T07:16:05.553 INFO:tasks.workunit.client.0.smithi063.stdout:6/883: creat d3/d15/d16/d60/d6d/d4e/d5a/d88/df5/d114/f127 x:0 0 0 2024-03-22T07:16:05.553 INFO:tasks.workunit.client.0.smithi063.stdout:6/884: truncate d3/d15/d16/d60/d6d/d4c/f75 932810 0 2024-03-22T07:16:05.553 INFO:tasks.workunit.client.0.smithi063.stdout:6/885: readlink d3/d15/d16/lbd 0 2024-03-22T07:16:05.554 INFO:tasks.workunit.client.0.smithi063.stdout:4/788: mknod d1/d5/d43/d27/d2c/d3d/d54/ddb/c10c 0 2024-03-22T07:16:05.555 INFO:tasks.workunit.client.0.smithi063.stdout:5/733: dwrite d8/d13/d24/f33 [4194304,4194304] 0 2024-03-22T07:16:05.556 INFO:tasks.workunit.client.1.smithi069.stdout:8/734: creat d1/d1c/fec x:0 0 0 2024-03-22T07:16:05.557 INFO:tasks.workunit.client.0.smithi063.stdout:2/759: mkdir d0/de/d58/d7f/d97/da2/dfa 0 2024-03-22T07:16:05.557 INFO:tasks.workunit.client.1.smithi069.stdout:2/432: link d18/d4d/d3e/f65 d18/d4d/d27/d77/f92 0 2024-03-22T07:16:05.559 INFO:tasks.workunit.client.0.smithi063.stdout:1/974: rmdir d4/d71/d88/da1/ddb/d11c 0 2024-03-22T07:16:05.559 INFO:tasks.workunit.client.0.smithi063.stdout:1/975: truncate d4/df/fbe 2397736 0 2024-03-22T07:16:05.559 INFO:tasks.workunit.client.1.smithi069.stdout:7/576: symlink d9/db/d1a/db1/d57/d44/d53/d2e/lb7 0 2024-03-22T07:16:05.560 INFO:tasks.workunit.client.0.smithi063.stdout:6/886: rename d3/d15/d16/f9a to d3/d15/d123/f128 0 2024-03-22T07:16:05.561 INFO:tasks.workunit.client.1.smithi069.stdout:4/730: mknod d6/dc4/cfa 0 2024-03-22T07:16:05.562 INFO:tasks.workunit.client.1.smithi069.stdout:5/471: creat d1/d10/d60/fac x:0 0 0 2024-03-22T07:16:05.563 INFO:tasks.workunit.client.0.smithi063.stdout:2/760: mkdir d0/de/d1b/d32/dfb 0 2024-03-22T07:16:05.564 INFO:tasks.workunit.client.0.smithi063.stdout:2/761: truncate d0/de/d1b/fae 607666 0 2024-03-22T07:16:05.564 INFO:tasks.workunit.client.0.smithi063.stdout:2/762: dread - d0/de/d1b/d63/d6e/f72 zero size 2024-03-22T07:16:05.564 INFO:tasks.workunit.client.1.smithi069.stdout:1/585: creat d1/d2a/d50/d88/fb8 x:0 0 0 2024-03-22T07:16:05.564 INFO:tasks.workunit.client.1.smithi069.stdout:8/735: mknod d1/d1c/d3e/d88/da8/ced 0 2024-03-22T07:16:05.565 INFO:tasks.workunit.client.1.smithi069.stdout:2/433: fsync d18/d4d/d3e/d40/f6f 0 2024-03-22T07:16:05.565 INFO:tasks.workunit.client.0.smithi063.stdout:1/976: mkdir d4/df/d13/d2b/d6c/d85/d154 0 2024-03-22T07:16:05.565 INFO:tasks.workunit.client.0.smithi063.stdout:1/977: fdatasync d4/df/d13/d2b/d86/fa3 0 2024-03-22T07:16:05.566 INFO:tasks.workunit.client.0.smithi063.stdout:4/789: dread d1/d5/dc/d95/f44 [0,4194304] 0 2024-03-22T07:16:05.567 INFO:tasks.workunit.client.0.smithi063.stdout:5/734: creat d8/d13/d28/d57/dca/fe1 x:0 0 0 2024-03-22T07:16:05.568 INFO:tasks.workunit.client.1.smithi069.stdout:7/577: rename d9/db/d1a/db1/d57/d75/d7a/d96/f9e to d9/db/d1a/db1/db3/fb8 0 2024-03-22T07:16:05.569 INFO:tasks.workunit.client.1.smithi069.stdout:1/586: link d1/d13/l49 d1/d2a/d84/d9a/lb9 0 2024-03-22T07:16:05.570 INFO:tasks.workunit.client.0.smithi063.stdout:9/813: dread d8/d17/d1e/fd9 [0,4194304] 0 2024-03-22T07:16:05.570 INFO:tasks.workunit.client.0.smithi063.stdout:9/814: chown d8/d19/d20/d22/d56/fba 3 1 2024-03-22T07:16:05.570 INFO:tasks.workunit.client.0.smithi063.stdout:9/815: fsync d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/ff8 0 2024-03-22T07:16:05.570 INFO:tasks.workunit.client.0.smithi063.stdout:7/685: dwrite d5/d16/f63 [0,4194304] 0 2024-03-22T07:16:05.571 INFO:tasks.workunit.client.0.smithi063.stdout:7/686: fsync d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/f47 0 2024-03-22T07:16:05.571 INFO:tasks.workunit.client.0.smithi063.stdout:1/978: rename d4/df/d11e to d4/d8b/d155 0 2024-03-22T07:16:05.571 INFO:tasks.workunit.client.0.smithi063.stdout:1/979: truncate d4/d14/f46 1923421 0 2024-03-22T07:16:05.571 INFO:tasks.workunit.client.1.smithi069.stdout:4/731: dread d6/d16/f88 [0,4194304] 0 2024-03-22T07:16:05.571 INFO:tasks.workunit.client.1.smithi069.stdout:8/736: unlink d1/d1c/d3e/d88/dcd/c90 0 2024-03-22T07:16:05.572 INFO:tasks.workunit.client.0.smithi063.stdout:9/816: rmdir d8/d17 39 2024-03-22T07:16:05.572 INFO:tasks.workunit.client.0.smithi063.stdout:9/817: write d8/d19/d20/d22/d29/df0/ffa [375086,89840] 0 2024-03-22T07:16:05.572 INFO:tasks.workunit.client.0.smithi063.stdout:9/818: read d8/d19/d20/fa0 [1187390,117270] 0 2024-03-22T07:16:05.573 INFO:tasks.workunit.client.0.smithi063.stdout:3/913: dwrite da/d16/d3f/f50 [0,4194304] 0 2024-03-22T07:16:05.573 INFO:tasks.workunit.client.0.smithi063.stdout:3/914: dread - da/d63/d65/d81/da5/fad zero size 2024-03-22T07:16:05.573 INFO:tasks.workunit.client.0.smithi063.stdout:3/915: chown da/d63/d65/d81/d9b/fba 0 1 2024-03-22T07:16:05.573 INFO:tasks.workunit.client.0.smithi063.stdout:3/916: creat da/d16/f128 x:0 0 0 2024-03-22T07:16:05.573 INFO:tasks.workunit.client.1.smithi069.stdout:2/434: mkdir d18/d4d/d27/d33/d60/d63/d93 0 2024-03-22T07:16:05.575 INFO:tasks.workunit.client.0.smithi063.stdout:7/687: truncate d5/da/d28/d4b/da8/dbc/d62/d73/fd7 4127313 0 2024-03-22T07:16:05.577 INFO:tasks.workunit.client.1.smithi069.stdout:5/472: rename d1/d10/d60/c68 to d1/d6/d26/d3e/cad 0 2024-03-22T07:16:05.578 INFO:tasks.workunit.client.1.smithi069.stdout:5/473: chown d1/d28/l82 342346666 1 2024-03-22T07:16:05.578 INFO:tasks.workunit.client.1.smithi069.stdout:5/474: fsync d1/d6/d26/d33/d5d/fa4 0 2024-03-22T07:16:05.578 INFO:tasks.workunit.client.0.smithi063.stdout:4/790: rename d1/d5/d43/d27/d2c/d4a/d78/le1 to d1/d5/d43/d27/l10d 0 2024-03-22T07:16:05.579 INFO:tasks.workunit.client.1.smithi069.stdout:7/578: mknod d9/db/d1a/db1/d57/d44/d53/d74/d6e/cb9 0 2024-03-22T07:16:05.579 INFO:tasks.workunit.client.0.smithi063.stdout:1/980: truncate d4/f1f 3023837 0 2024-03-22T07:16:05.581 INFO:tasks.workunit.client.0.smithi063.stdout:6/887: dwrite d3/d6/d35/f11f [0,4194304] 0 2024-03-22T07:16:05.584 INFO:tasks.workunit.client.0.smithi063.stdout:9/819: mknod d8/d19/d20/d22/d5d/d76/d95/cfc 0 2024-03-22T07:16:05.584 INFO:tasks.workunit.client.1.smithi069.stdout:1/587: creat d1/d2a/d5e/fba x:0 0 0 2024-03-22T07:16:05.585 INFO:tasks.workunit.client.1.smithi069.stdout:8/737: rmdir d1/d1c/d70 39 2024-03-22T07:16:05.585 INFO:tasks.workunit.client.1.smithi069.stdout:5/475: mkdir d1/d6/d21/dae 0 2024-03-22T07:16:05.586 INFO:tasks.workunit.client.0.smithi063.stdout:5/735: write d8/f10 [294407,60929] 0 2024-03-22T07:16:05.586 INFO:tasks.workunit.client.1.smithi069.stdout:7/579: creat d9/da1/fba x:0 0 0 2024-03-22T07:16:05.587 INFO:tasks.workunit.client.0.smithi063.stdout:3/917: symlink da/d63/d65/d83/l129 0 2024-03-22T07:16:05.589 INFO:tasks.workunit.client.1.smithi069.stdout:5/476: creat d1/d6/d26/d33/d5d/d72/d95/faf x:0 0 0 2024-03-22T07:16:05.589 INFO:tasks.workunit.client.1.smithi069.stdout:4/732: dwrite d6/daa/dce/de4/ff9 [0,4194304] 0 2024-03-22T07:16:05.589 INFO:tasks.workunit.client.0.smithi063.stdout:7/688: getdents d5/d16/d1b 0 2024-03-22T07:16:05.591 INFO:tasks.workunit.client.1.smithi069.stdout:5/477: mknod d1/d10/d60/cb0 0 2024-03-22T07:16:05.592 INFO:tasks.workunit.client.0.smithi063.stdout:4/791: symlink d1/d5/dc2/l10e 0 2024-03-22T07:16:05.594 INFO:tasks.workunit.client.1.smithi069.stdout:8/738: dread d1/d1c/d3e/f61 [0,4194304] 0 2024-03-22T07:16:05.594 INFO:tasks.workunit.client.1.smithi069.stdout:8/739: write d1/d1c/d3e/f46 [1597384,95143] 0 2024-03-22T07:16:05.594 INFO:tasks.workunit.client.0.smithi063.stdout:1/981: rmdir d4/df/d13/d2b/d6c/da6 39 2024-03-22T07:16:05.595 INFO:tasks.workunit.client.0.smithi063.stdout:1/982: creat d4/d14/d57/f156 x:0 0 0 2024-03-22T07:16:05.596 INFO:tasks.workunit.client.1.smithi069.stdout:5/478: creat d1/d6/d21/fb1 x:0 0 0 2024-03-22T07:16:05.596 INFO:tasks.workunit.client.1.smithi069.stdout:5/479: chown d1/d6/d21/c8b 1768 1 2024-03-22T07:16:05.596 INFO:tasks.workunit.client.1.smithi069.stdout:5/480: dread - d1/d6/d26/d33/d5d/fa4 zero size 2024-03-22T07:16:05.597 INFO:tasks.workunit.client.1.smithi069.stdout:5/481: mkdir d1/d6/d26/d3a/db2 0 2024-03-22T07:16:05.598 INFO:tasks.workunit.client.1.smithi069.stdout:5/482: dread - d1/d10/d60/fac zero size 2024-03-22T07:16:05.598 INFO:tasks.workunit.client.1.smithi069.stdout:5/483: dread - d1/d6/d88/fa1 zero size 2024-03-22T07:16:05.600 INFO:tasks.workunit.client.1.smithi069.stdout:5/484: rename d1/d28/c39 to d1/d10/cb3 0 2024-03-22T07:16:05.600 INFO:tasks.workunit.client.1.smithi069.stdout:5/485: chown d1/d6/d26/d3e/f8d 63 1 2024-03-22T07:16:05.600 INFO:tasks.workunit.client.1.smithi069.stdout:5/486: fdatasync d1/d6/d21/fb1 0 2024-03-22T07:16:05.600 INFO:tasks.workunit.client.1.smithi069.stdout:5/487: readlink d1/d6/l20 0 2024-03-22T07:16:05.601 INFO:tasks.workunit.client.0.smithi063.stdout:9/820: stat d8/d19/d2b/l42 0 2024-03-22T07:16:05.601 INFO:tasks.workunit.client.0.smithi063.stdout:9/821: readlink d8/d19/d20/d22/d29/lcb 0 2024-03-22T07:16:05.601 INFO:tasks.workunit.client.0.smithi063.stdout:9/822: fsync d8/d19/d20/d22/fd6 0 2024-03-22T07:16:05.602 INFO:tasks.workunit.client.0.smithi063.stdout:9/823: write d8/d19/d20/d22/d29/d31/f86 [888817,70857] 0 2024-03-22T07:16:05.602 INFO:tasks.workunit.client.1.smithi069.stdout:5/488: creat d1/d10/d44/fb4 x:0 0 0 2024-03-22T07:16:05.605 INFO:tasks.workunit.client.1.smithi069.stdout:5/489: link d1/d10/d60/f7b d1/d6/d26/d33/d5d/d72/d78/fb5 0 2024-03-22T07:16:05.605 INFO:tasks.workunit.client.1.smithi069.stdout:5/490: readlink d1/d6/d86/d99/l92 0 2024-03-22T07:16:05.605 INFO:tasks.workunit.client.1.smithi069.stdout:5/491: fsync d1/d6/f6f 0 2024-03-22T07:16:05.605 INFO:tasks.workunit.client.0.smithi063.stdout:5/736: creat d8/d13/d24/da4/d8b/db6/fe2 x:0 0 0 2024-03-22T07:16:05.605 INFO:tasks.workunit.client.0.smithi063.stdout:5/737: write d8/d13/d1e/d31/d64/d65/d86/f92 [810115,73532] 0 2024-03-22T07:16:05.606 INFO:tasks.workunit.client.1.smithi069.stdout:7/580: dread d9/db/d1a/db1/d57/d44/d53/d74/d30/f54 [0,4194304] 0 2024-03-22T07:16:05.606 INFO:tasks.workunit.client.1.smithi069.stdout:1/588: rmdir d1/d2a/d84 39 2024-03-22T07:16:05.608 INFO:tasks.workunit.client.1.smithi069.stdout:5/492: mknod d1/d6/d21/dae/cb6 0 2024-03-22T07:16:05.609 INFO:tasks.workunit.client.1.smithi069.stdout:5/493: chown d1/d6/d26/d33/d5d/fa4 533661 1 2024-03-22T07:16:05.610 INFO:tasks.workunit.client.1.smithi069.stdout:7/581: creat d9/d42/d8a/db5/fbb x:0 0 0 2024-03-22T07:16:05.610 INFO:tasks.workunit.client.1.smithi069.stdout:7/582: truncate d9/db/d1a/db1/d57/d44/d53/d1d/f87 856175 0 2024-03-22T07:16:05.610 INFO:tasks.workunit.client.1.smithi069.stdout:7/583: chown d9/c27 0 1 2024-03-22T07:16:05.612 INFO:tasks.workunit.client.1.smithi069.stdout:7/584: mknod d9/db/d1a/db1/d57/d75/d7a/d96/cbc 0 2024-03-22T07:16:05.612 INFO:tasks.workunit.client.0.smithi063.stdout:1/983: creat d4/df/d13/d2b/d6c/f157 x:0 0 0 2024-03-22T07:16:05.612 INFO:tasks.workunit.client.0.smithi063.stdout:1/984: truncate d4/d71/d88/da1/ddb/f12f 980404 0 2024-03-22T07:16:05.614 INFO:tasks.workunit.client.1.smithi069.stdout:7/585: mkdir d9/db/d48/d4b/dbd 0 2024-03-22T07:16:05.615 INFO:tasks.workunit.client.0.smithi063.stdout:9/824: mknod d8/d19/d20/cfd 0 2024-03-22T07:16:05.615 INFO:tasks.workunit.client.0.smithi063.stdout:1/985: mkdir d4/d71/d88/d158 0 2024-03-22T07:16:05.617 INFO:tasks.workunit.client.0.smithi063.stdout:9/825: rmdir d8/d19/d20/d22/d29/d31 39 2024-03-22T07:16:05.617 INFO:tasks.workunit.client.0.smithi063.stdout:1/986: getdents d4/df/d13/d2b/d6c/d85 0 2024-03-22T07:16:05.620 INFO:tasks.workunit.client.0.smithi063.stdout:9/826: rmdir d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48 39 2024-03-22T07:16:05.620 INFO:tasks.workunit.client.0.smithi063.stdout:5/738: write d8/d13/d1e/d31/d58/d7e/f84 [240843,11699] 0 2024-03-22T07:16:05.621 INFO:tasks.workunit.client.0.smithi063.stdout:9/827: getdents d8/d19/d20/d22/de1/dd7 0 2024-03-22T07:16:05.623 INFO:tasks.workunit.client.0.smithi063.stdout:5/739: mknod d8/d13/d24/ce3 0 2024-03-22T07:16:05.625 INFO:tasks.workunit.client.1.smithi069.stdout:5/494: dread d1/d7/f59 [0,4194304] 0 2024-03-22T07:16:05.625 INFO:tasks.workunit.client.0.smithi063.stdout:1/987: mknod d4/df/d13/d2b/d6c/c159 0 2024-03-22T07:16:05.627 INFO:tasks.workunit.client.0.smithi063.stdout:9/828: creat d8/d19/d20/d22/d29/d31/d53/ffe x:0 0 0 2024-03-22T07:16:05.628 INFO:tasks.workunit.client.1.smithi069.stdout:7/586: dread f4 [0,4194304] 0 2024-03-22T07:16:05.628 INFO:tasks.workunit.client.1.smithi069.stdout:7/587: write d9/db/d1a/db1/d57/d44/d53/d74/fa4 [645294,40819] 0 2024-03-22T07:16:05.629 INFO:tasks.workunit.client.0.smithi063.stdout:5/740: unlink d8/d13/d1e/d31/d64/c67 0 2024-03-22T07:16:05.632 INFO:tasks.workunit.client.0.smithi063.stdout:1/988: getdents d4/d71/dbb/dc0/dd8 0 2024-03-22T07:16:05.632 INFO:tasks.workunit.client.0.smithi063.stdout:1/989: dread - d4/d5a/d91/f102 zero size 2024-03-22T07:16:05.632 INFO:tasks.workunit.client.0.smithi063.stdout:1/990: creat d4/df/d13/d2b/d86/f15a x:0 0 0 2024-03-22T07:16:05.632 INFO:tasks.workunit.client.0.smithi063.stdout:1/991: chown d4/d5a/ff8 4 1 2024-03-22T07:16:05.633 INFO:tasks.workunit.client.0.smithi063.stdout:3/918: dwrite da/d16/d89/d8c/dc1/fcb [0,4194304] 0 2024-03-22T07:16:05.634 INFO:tasks.workunit.client.0.smithi063.stdout:9/829: rename d8/d19/d20/d22/db2/cdd to d8/d19/d20/d22/de1/cff 0 2024-03-22T07:16:05.634 INFO:tasks.workunit.client.0.smithi063.stdout:9/830: write d8/d19/d20/d22/d56/f82 [5869379,99722] 0 2024-03-22T07:16:05.634 INFO:tasks.workunit.client.0.smithi063.stdout:5/741: truncate d8/f44 2100599 0 2024-03-22T07:16:05.636 INFO:tasks.workunit.client.0.smithi063.stdout:1/992: rmdir d4/df/d13/d39/d6f/dd9 39 2024-03-22T07:16:05.638 INFO:tasks.workunit.client.0.smithi063.stdout:3/919: rename da/d16/f42 to da/d63/d65/d4b/d10a/f12a 0 2024-03-22T07:16:05.641 INFO:tasks.workunit.client.0.smithi063.stdout:5/742: link d8/d13/c1a d8/d13/d1e/ce4 0 2024-03-22T07:16:05.644 INFO:tasks.workunit.client.1.smithi069.stdout:5/495: dread d1/d6/d88/f93 [0,4194304] 0 2024-03-22T07:16:05.644 INFO:tasks.workunit.client.1.smithi069.stdout:5/496: write d1/d6/d21/fb1 [670411,119724] 0 2024-03-22T07:16:05.644 INFO:tasks.workunit.client.1.smithi069.stdout:5/497: creat d1/d10/d44/fb7 x:0 0 0 2024-03-22T07:16:05.644 INFO:tasks.workunit.client.1.smithi069.stdout:5/498: chown d1/d6/d26/d3e/d9c/d9d 6 1 2024-03-22T07:16:05.645 INFO:tasks.workunit.client.0.smithi063.stdout:3/920: dread da/f1c [0,4194304] 0 2024-03-22T07:16:05.645 INFO:tasks.workunit.client.0.smithi063.stdout:3/921: creat da/d63/d65/d83/dc6/f12b x:0 0 0 2024-03-22T07:16:05.646 INFO:tasks.workunit.client.1.smithi069.stdout:4/733: dwrite d6/dc/d23/d59/fca [0,4194304] 0 2024-03-22T07:16:05.649 INFO:tasks.workunit.client.1.smithi069.stdout:7/588: write d9/db/d1a/db1/d57/d75/f81 [103148,80550] 0 2024-03-22T07:16:05.649 INFO:tasks.workunit.client.0.smithi063.stdout:1/993: dread d4/f12 [0,4194304] 0 2024-03-22T07:16:05.650 INFO:tasks.workunit.client.0.smithi063.stdout:1/994: dread - d4/d5a/f9a zero size 2024-03-22T07:16:05.650 INFO:tasks.workunit.client.0.smithi063.stdout:1/995: creat d4/d8b/f15b x:0 0 0 2024-03-22T07:16:05.651 INFO:tasks.workunit.client.1.smithi069.stdout:7/589: mknod d9/db/d1a/db1/d57/cbe 0 2024-03-22T07:16:05.651 INFO:tasks.workunit.client.1.smithi069.stdout:7/590: dread - d9/d42/d8a/db5/fbb zero size 2024-03-22T07:16:05.651 INFO:tasks.workunit.client.1.smithi069.stdout:7/591: chown d9/db/d1a/db1/d57/d44/c68 142614 1 2024-03-22T07:16:05.651 INFO:tasks.workunit.client.1.smithi069.stdout:7/592: stat d9/db/d1a/db1/d57/d44/d53/d1d/faa 0 2024-03-22T07:16:05.651 INFO:tasks.workunit.client.1.smithi069.stdout:7/593: chown d9/db/d1a/db1/d57/d44/d53/d2e/f7c 470514 1 2024-03-22T07:16:05.652 INFO:tasks.workunit.client.1.smithi069.stdout:4/734: rename d6/dc to d6/d16/d3a/db4/dfb 0 2024-03-22T07:16:05.653 INFO:tasks.workunit.client.1.smithi069.stdout:7/594: unlink d9/db/d1a/db1/d57/d44/cb0 0 2024-03-22T07:16:05.653 INFO:tasks.workunit.client.1.smithi069.stdout:7/595: truncate d9/db/d1a/f21 2370547 0 2024-03-22T07:16:05.653 INFO:tasks.workunit.client.1.smithi069.stdout:7/596: dread - d9/da1/fba zero size 2024-03-22T07:16:05.654 INFO:tasks.workunit.client.1.smithi069.stdout:7/597: mknod d9/db/d1a/db1/d57/d75/d7a/d96/da0/cbf 0 2024-03-22T07:16:05.657 INFO:tasks.workunit.client.0.smithi063.stdout:7/689: dwrite d5/da/fe2 [0,4194304] 0 2024-03-22T07:16:05.657 INFO:tasks.workunit.client.0.smithi063.stdout:7/690: fsync d5/da/d5f/d33/f53 0 2024-03-22T07:16:05.659 INFO:tasks.workunit.client.1.smithi069.stdout:1/589: dwrite d1/d2a/d50/d4c/d55/f97 [0,4194304] 0 2024-03-22T07:16:05.659 INFO:tasks.workunit.client.1.smithi069.stdout:7/598: dread d9/db/f24 [0,4194304] 0 2024-03-22T07:16:05.660 INFO:tasks.workunit.client.0.smithi063.stdout:5/743: dread d8/d13/d24/d62/fd3 [4194304,4194304] 0 2024-03-22T07:16:05.660 INFO:tasks.workunit.client.0.smithi063.stdout:3/922: read da/d63/d65/d7b/fd9 [269954,3171] 0 2024-03-22T07:16:05.660 INFO:tasks.workunit.client.0.smithi063.stdout:5/744: write d8/d13/d1e/d31/d64/fc2 [304102,29598] 0 2024-03-22T07:16:05.660 INFO:tasks.workunit.client.1.smithi069.stdout:8/740: dwrite d1/d1c/d3e/d88/dcd/f1a [0,4194304] 0 2024-03-22T07:16:05.660 INFO:tasks.workunit.client.1.smithi069.stdout:4/735: dread d6/d4b/f97 [0,4194304] 0 2024-03-22T07:16:05.662 INFO:tasks.workunit.client.1.smithi069.stdout:7/599: rename d9/db/d1a/db1/d57/d44/d53/d74/c5a to d9/db/d1a/db1/d57/d44/d53/d74/cc0 0 2024-03-22T07:16:05.664 INFO:tasks.workunit.client.1.smithi069.stdout:5/499: write d1/d6/d86/d99/f24 [1706652,25837] 0 2024-03-22T07:16:05.665 INFO:tasks.workunit.client.1.smithi069.stdout:8/741: creat d1/d1c/d3e/fee x:0 0 0 2024-03-22T07:16:05.668 INFO:tasks.workunit.client.1.smithi069.stdout:4/736: creat d6/dc4/ffc x:0 0 0 2024-03-22T07:16:05.668 INFO:tasks.workunit.client.1.smithi069.stdout:4/737: truncate d6/d16/d83/d9a/fa3 1138275 0 2024-03-22T07:16:05.669 INFO:tasks.workunit.client.1.smithi069.stdout:5/500: dread d1/d6/d21/fb1 [0,4194304] 0 2024-03-22T07:16:05.669 INFO:tasks.workunit.client.1.smithi069.stdout:5/501: write d1/d10/d60/f83 [322697,72917] 0 2024-03-22T07:16:05.670 INFO:tasks.workunit.client.1.smithi069.stdout:5/502: write d1/d6/d86/d99/f90 [593897,77228] 0 2024-03-22T07:16:05.670 INFO:tasks.workunit.client.1.smithi069.stdout:5/503: write d1/d6/d26/d33/d5d/fa4 [992318,81768] 0 2024-03-22T07:16:05.670 INFO:tasks.workunit.client.1.smithi069.stdout:5/504: dread - d1/d6/fa9 zero size 2024-03-22T07:16:05.675 INFO:tasks.workunit.client.1.smithi069.stdout:1/590: read d1/d2a/d50/d88/f75 [1628418,129424] 0 2024-03-22T07:16:05.676 INFO:tasks.workunit.client.0.smithi063.stdout:1/996: dwrite d4/df/d13/d2b/d3a/f81 [4194304,4194304] 0 2024-03-22T07:16:05.676 INFO:tasks.workunit.client.0.smithi063.stdout:1/997: chown d4/df/d13/d39/f5e 41485 1 2024-03-22T07:16:05.676 INFO:tasks.workunit.client.0.smithi063.stdout:4/792: dwrite d1/d5/dc/dda/d79/daa/fb6 [0,4194304] 0 2024-03-22T07:16:05.678 INFO:tasks.workunit.client.0.smithi063.stdout:3/923: dread da/d16/d3f/f67 [0,4194304] 0 2024-03-22T07:16:05.681 INFO:tasks.workunit.client.1.smithi069.stdout:7/600: creat d9/db/d1a/db1/d57/d44/fc1 x:0 0 0 2024-03-22T07:16:05.683 INFO:tasks.workunit.client.0.smithi063.stdout:1/998: dread f2 [0,4194304] 0 2024-03-22T07:16:05.685 INFO:tasks.workunit.client.1.smithi069.stdout:8/742: unlink d1/l7 0 2024-03-22T07:16:05.687 INFO:tasks.workunit.client.1.smithi069.stdout:4/738: mkdir d6/daa/dfd 0 2024-03-22T07:16:05.688 INFO:tasks.workunit.client.1.smithi069.stdout:7/601: dread d9/db/d1a/db1/d57/d44/d53/d74/fa4 [0,4194304] 0 2024-03-22T07:16:05.688 INFO:tasks.workunit.client.1.smithi069.stdout:7/602: write d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/fae [280783,34331] 0 2024-03-22T07:16:05.690 INFO:tasks.workunit.client.0.smithi063.stdout:4/793: dread d1/d5/dc/d95/d59/faf [0,4194304] 0 2024-03-22T07:16:05.690 INFO:tasks.workunit.client.0.smithi063.stdout:4/794: fsync d1/d5/dc/d95/d59/d9c/dac/dc9/ff9 0 2024-03-22T07:16:05.690 INFO:tasks.workunit.client.1.smithi069.stdout:1/591: rename d1/d2a/d50/d88/d41/d1e/l65 to d1/d13/lbb 0 2024-03-22T07:16:05.690 INFO:tasks.workunit.client.1.smithi069.stdout:5/505: dwrite d1/d6/d26/d3e/d9c/d9d/fa6 [0,4194304] 0 2024-03-22T07:16:05.690 INFO:tasks.workunit.client.1.smithi069.stdout:5/506: creat d1/d28/fb8 x:0 0 0 2024-03-22T07:16:05.691 INFO:tasks.workunit.client.0.smithi063.stdout:4/795: read d1/d5/d3c/d65/f69 [389933,115356] 0 2024-03-22T07:16:05.696 INFO:tasks.workunit.client.0.smithi063.stdout:1/999: dread d4/df/d13/f3c [0,4194304] 0 2024-03-22T07:16:05.699 INFO:tasks.workunit.client.0.smithi063.stdout:7/691: link d5/da/d28/d4b/da8/dbc/d62/d73/d1d/l69 d5/da/d28/d4b/da8/dbc/d62/d73/dc5/le5 0 2024-03-22T07:16:05.701 INFO:tasks.workunit.client.1.smithi069.stdout:5/507: rename d1/d10/d44/fb7 to d1/d6/d86/d99/d43/fb9 0 2024-03-22T07:16:05.703 INFO:tasks.workunit.client.1.smithi069.stdout:8/743: rename d1/d1c/f22 to d1/d1c/d3e/d88/dcd/de8/fef 0 2024-03-22T07:16:05.705 INFO:tasks.workunit.client.0.smithi063.stdout:5/745: symlink d8/dba/le5 0 2024-03-22T07:16:05.705 INFO:tasks.workunit.client.0.smithi063.stdout:5/746: fsync d8/d13/d1e/d31/fa6 0 2024-03-22T07:16:05.705 INFO:tasks.workunit.client.1.smithi069.stdout:4/739: rename d6/d9b/d1f/f7f to d6/d12/d5d/ffe 0 2024-03-22T07:16:05.708 INFO:tasks.workunit.client.1.smithi069.stdout:8/744: symlink d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/lf0 0 2024-03-22T07:16:05.708 INFO:tasks.workunit.client.1.smithi069.stdout:8/745: chown d1/d1c/d3e/f61 775618 1 2024-03-22T07:16:05.708 INFO:tasks.workunit.client.1.smithi069.stdout:8/746: dread - d1/d1c/d3e/d88/dcd/d6c/d4e/fc7 zero size 2024-03-22T07:16:05.708 INFO:tasks.workunit.client.1.smithi069.stdout:8/747: truncate d1/d1c/d3e/d88/da8/dbb/fd8 674344 0 2024-03-22T07:16:05.708 INFO:tasks.workunit.client.1.smithi069.stdout:8/748: read d1/d1c/d3e/d88/dcd/d6c/d73/f74 [38964,94337] 0 2024-03-22T07:16:05.712 INFO:tasks.workunit.client.1.smithi069.stdout:7/603: dwrite d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/f40 [0,4194304] 0 2024-03-22T07:16:05.712 INFO:tasks.workunit.client.1.smithi069.stdout:1/592: dwrite d1/d2a/d50/d88/d41/d64/f7e [0,4194304] 0 2024-03-22T07:16:05.715 INFO:tasks.workunit.client.1.smithi069.stdout:7/604: truncate d9/f6f 6812694 0 2024-03-22T07:16:05.717 INFO:tasks.workunit.client.1.smithi069.stdout:7/605: unlink d9/c10 0 2024-03-22T07:16:05.722 INFO:tasks.workunit.client.1.smithi069.stdout:7/606: truncate d9/db/d48/d4b/f4f 3497076 0 2024-03-22T07:16:05.723 INFO:tasks.workunit.client.1.smithi069.stdout:8/749: dread d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/fae [0,4194304] 0 2024-03-22T07:16:05.723 INFO:tasks.workunit.client.1.smithi069.stdout:8/750: readlink d1/d1c/d3e/d88/da8/lb1 0 2024-03-22T07:16:05.723 INFO:tasks.workunit.client.1.smithi069.stdout:8/751: chown d1/d1c/d3e/d88/dcd/f40 52092 1 2024-03-22T07:16:05.723 INFO:tasks.workunit.client.1.smithi069.stdout:8/752: fdatasync d1/d1c/d3e/fb7 0 2024-03-22T07:16:05.724 INFO:tasks.workunit.client.1.smithi069.stdout:4/740: dwrite d6/d16/d3a/db4/dbc/dc6/fdb [0,4194304] 0 2024-03-22T07:16:05.726 INFO:tasks.workunit.client.1.smithi069.stdout:7/607: rename d9/db/d1a/db1/d57/d44/d53/d74/l4a to d9/db/d1a/db1/d57/lc2 0 2024-03-22T07:16:05.726 INFO:tasks.workunit.client.1.smithi069.stdout:7/608: chown d9/db/d1a/db1/d57/d44/d53/d2e/f6a 118 1 2024-03-22T07:16:05.726 INFO:tasks.workunit.client.1.smithi069.stdout:7/609: fdatasync d9/db/f17 0 2024-03-22T07:16:05.730 INFO:tasks.workunit.client.1.smithi069.stdout:7/610: creat d9/db/d1a/fc3 x:0 0 0 2024-03-22T07:16:05.731 INFO:tasks.workunit.client.0.smithi063.stdout:9/831: truncate d8/d19/d20/fa0 7697038 0 2024-03-22T07:16:05.731 INFO:tasks.workunit.client.0.smithi063.stdout:9/832: fdatasync d8/d19/d20/d22/d29/d31/d53/d6a/da2/f6b 0 2024-03-22T07:16:05.731 INFO:tasks.workunit.client.1.smithi069.stdout:0/692: sync 2024-03-22T07:16:05.732 INFO:tasks.workunit.client.1.smithi069.stdout:4/741: getdents d6/dc4 0 2024-03-22T07:16:05.734 INFO:tasks.workunit.client.1.smithi069.stdout:1/593: dwrite d1/d13/d36/f54 [0,4194304] 0 2024-03-22T07:16:05.734 INFO:tasks.workunit.client.1.smithi069.stdout:7/611: getdents d9/db/d1a/db1/d57/d44/d53/d2e 0 2024-03-22T07:16:05.736 INFO:tasks.workunit.client.1.smithi069.stdout:8/753: dread d1/d1c/f38 [0,4194304] 0 2024-03-22T07:16:05.740 INFO:tasks.workunit.client.1.smithi069.stdout:1/594: mkdir d1/d2a/d50/d88/dae/db5/dbc 0 2024-03-22T07:16:05.741 INFO:tasks.workunit.client.1.smithi069.stdout:7/612: symlink d9/db/d1a/db1/d57/d44/d53/d74/d30/lc4 0 2024-03-22T07:16:05.742 INFO:tasks.workunit.client.1.smithi069.stdout:7/613: truncate d9/db/d1a/db1/d57/d44/d53/f6d 1321429 0 2024-03-22T07:16:05.742 INFO:tasks.workunit.client.1.smithi069.stdout:9/490: sync 2024-03-22T07:16:05.742 INFO:tasks.workunit.client.1.smithi069.stdout:9/491: write d2/dd/d3c/d2e/f6a [614010,101656] 0 2024-03-22T07:16:05.742 INFO:tasks.workunit.client.1.smithi069.stdout:0/693: dread d3/d7/f4a [0,4194304] 0 2024-03-22T07:16:05.742 INFO:tasks.workunit.client.1.smithi069.stdout:4/742: getdents d6/d16/d83 0 2024-03-22T07:16:05.743 INFO:tasks.workunit.client.1.smithi069.stdout:6/532: sync 2024-03-22T07:16:05.743 INFO:tasks.workunit.client.1.smithi069.stdout:3/639: sync 2024-03-22T07:16:05.743 INFO:tasks.workunit.client.1.smithi069.stdout:6/533: chown d4/dd/d12/d2b/d72/c7d 19949179 1 2024-03-22T07:16:05.743 INFO:tasks.workunit.client.1.smithi069.stdout:6/534: chown d4/dd/d41/f93 659 1 2024-03-22T07:16:05.743 INFO:tasks.workunit.client.1.smithi069.stdout:6/535: truncate d4/dd/d12/d44/f6f 552349 0 2024-03-22T07:16:05.744 INFO:tasks.workunit.client.0.smithi063.stdout:9/833: dread d8/f54 [0,4194304] 0 2024-03-22T07:16:05.745 INFO:tasks.workunit.client.1.smithi069.stdout:8/754: symlink d1/d1c/d3e/d66/d9e/lf1 0 2024-03-22T07:16:05.745 INFO:tasks.workunit.client.1.smithi069.stdout:8/755: write d1/d1c/d3e/fb7 [337462,120303] 0 2024-03-22T07:16:05.745 INFO:tasks.workunit.client.1.smithi069.stdout:8/756: write d1/d1c/d3e/d88/dcd/de8/fef [5189007,1203] 0 2024-03-22T07:16:05.745 INFO:tasks.workunit.client.1.smithi069.stdout:8/757: read - d1/d1c/d48/d5e/fad zero size 2024-03-22T07:16:05.745 INFO:tasks.workunit.client.1.smithi069.stdout:2/435: sync 2024-03-22T07:16:05.748 INFO:tasks.workunit.client.1.smithi069.stdout:0/694: rename d3/d7/d24/d6a/fcd to d3/d7/d17/fe4 0 2024-03-22T07:16:05.748 INFO:tasks.workunit.client.1.smithi069.stdout:0/695: fsync d3/d7/d17/f4c 0 2024-03-22T07:16:05.748 INFO:tasks.workunit.client.1.smithi069.stdout:0/696: creat d3/fe5 x:0 0 0 2024-03-22T07:16:05.748 INFO:tasks.workunit.client.1.smithi069.stdout:0/697: chown d3/d7/d15/d1f/f4e 59 1 2024-03-22T07:16:05.748 INFO:tasks.workunit.client.1.smithi069.stdout:0/698: readlink d3/d7/d15/lb8 0 2024-03-22T07:16:05.748 INFO:tasks.workunit.client.1.smithi069.stdout:0/699: chown d3/d7/d15/lb8 0 1 2024-03-22T07:16:05.748 INFO:tasks.workunit.client.1.smithi069.stdout:1/595: write d1/d2a/d50/d88/f4d [1671214,83203] 0 2024-03-22T07:16:05.748 INFO:tasks.workunit.client.1.smithi069.stdout:1/596: creat d1/d13/d36/d6e/d73/fbd x:0 0 0 2024-03-22T07:16:05.750 INFO:tasks.workunit.client.1.smithi069.stdout:4/743: truncate d6/d9b/f14 1099672 0 2024-03-22T07:16:05.753 INFO:tasks.workunit.client.1.smithi069.stdout:7/614: write d9/db/d1a/db1/d57/d44/d53/f38 [4156932,82339] 0 2024-03-22T07:16:05.753 INFO:tasks.workunit.client.1.smithi069.stdout:6/536: mkdir d4/dd/d41/d78/da3 0 2024-03-22T07:16:05.757 INFO:tasks.workunit.client.1.smithi069.stdout:8/758: mkdir d1/d1c/d70/df2 0 2024-03-22T07:16:05.758 INFO:tasks.workunit.client.1.smithi069.stdout:0/700: dread d3/d7/d15/d29/f69 [0,4194304] 0 2024-03-22T07:16:05.758 INFO:tasks.workunit.client.1.smithi069.stdout:0/701: write d3/d7/d15/f20 [192540,32240] 0 2024-03-22T07:16:05.758 INFO:tasks.workunit.client.1.smithi069.stdout:0/702: write d3/fd9 [313962,84923] 0 2024-03-22T07:16:05.758 INFO:tasks.workunit.client.1.smithi069.stdout:0/703: dread d3/d7/d90/fa5 [0,4194304] 0 2024-03-22T07:16:05.762 INFO:tasks.workunit.client.1.smithi069.stdout:1/597: truncate d1/d13/d36/f3e 1453101 0 2024-03-22T07:16:05.763 INFO:tasks.workunit.client.1.smithi069.stdout:4/744: rmdir d6/dc4 39 2024-03-22T07:16:05.763 INFO:tasks.workunit.client.1.smithi069.stdout:4/745: creat d6/d16/d3a/db4/fff x:0 0 0 2024-03-22T07:16:05.763 INFO:tasks.workunit.client.1.smithi069.stdout:4/746: write d6/d12/d18/d21/f79 [2103229,85534] 0 2024-03-22T07:16:05.763 INFO:tasks.workunit.client.1.smithi069.stdout:4/747: creat d6/d4b/f100 x:0 0 0 2024-03-22T07:16:05.766 INFO:tasks.workunit.client.1.smithi069.stdout:7/615: unlink d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/l78 0 2024-03-22T07:16:05.766 INFO:tasks.workunit.client.1.smithi069.stdout:6/537: symlink d4/dd/la4 0 2024-03-22T07:16:05.770 INFO:tasks.workunit.client.1.smithi069.stdout:8/759: write d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/fae [1388332,76100] 0 2024-03-22T07:16:05.770 INFO:tasks.workunit.client.1.smithi069.stdout:8/760: dread - d1/d1c/d48/d5e/f77 zero size 2024-03-22T07:16:05.770 INFO:tasks.workunit.client.1.smithi069.stdout:8/761: fsync d1/d1c/d3e/d88/dcd/d6c/d73/fea 0 2024-03-22T07:16:05.772 INFO:tasks.workunit.client.1.smithi069.stdout:0/704: rename d3/d7/d17/fc1 to d3/fe6 0 2024-03-22T07:16:05.774 INFO:tasks.workunit.client.1.smithi069.stdout:1/598: symlink d1/d2a/d50/d88/d41/d1e/lbe 0 2024-03-22T07:16:05.774 INFO:tasks.workunit.client.1.smithi069.stdout:1/599: fsync d1/d2a/d50/d4c/d55/f56 0 2024-03-22T07:16:05.780 INFO:tasks.workunit.client.1.smithi069.stdout:6/538: mkdir d4/dd/d12/d2b/da5 0 2024-03-22T07:16:05.782 INFO:tasks.workunit.client.1.smithi069.stdout:0/705: getdents d3/d7/d15/d29/d5a/d5d/d98 0 2024-03-22T07:16:05.784 INFO:tasks.workunit.client.1.smithi069.stdout:4/748: rename d6/d12/db2/ff5 to d6/d16/d83/dd0/f101 0 2024-03-22T07:16:05.786 INFO:tasks.workunit.client.1.smithi069.stdout:6/539: unlink d4/dd/d12/d2b/f37 0 2024-03-22T07:16:05.788 INFO:tasks.workunit.client.1.smithi069.stdout:0/706: creat d3/d7/d15/dad/fe7 x:0 0 0 2024-03-22T07:16:05.788 INFO:tasks.workunit.client.1.smithi069.stdout:0/707: chown d3/fe6 15020 1 2024-03-22T07:16:05.789 INFO:tasks.workunit.client.1.smithi069.stdout:4/749: creat d6/d9b/d1f/d58/f102 x:0 0 0 2024-03-22T07:16:05.790 INFO:tasks.workunit.client.1.smithi069.stdout:4/750: readlink d6/d12/d5d/lb3 0 2024-03-22T07:16:05.791 INFO:tasks.workunit.client.1.smithi069.stdout:6/540: mkdir d4/d7/da6 0 2024-03-22T07:16:05.791 INFO:tasks.workunit.client.1.smithi069.stdout:6/541: fsync d4/dd/d12/d44/d58/f62 0 2024-03-22T07:16:05.794 INFO:tasks.workunit.client.1.smithi069.stdout:6/542: mknod d4/dd/d12/d13/ca7 0 2024-03-22T07:16:05.796 INFO:tasks.workunit.client.1.smithi069.stdout:6/543: rename d4/d7/da/d3f/d76/c8b to d4/dd/d12/d2b/d72/d73/d74/ca8 0 2024-03-22T07:16:05.798 INFO:tasks.workunit.client.1.smithi069.stdout:6/544: creat d4/dd/d12/fa9 x:0 0 0 2024-03-22T07:16:05.798 INFO:tasks.workunit.client.1.smithi069.stdout:6/545: write d4/d7/da/f1b [326289,37293] 0 2024-03-22T07:16:05.801 INFO:tasks.workunit.client.1.smithi069.stdout:7/616: dwrite d9/db/d1a/db1/d57/d44/d53/d74/f98 [0,4194304] 0 2024-03-22T07:16:05.802 INFO:tasks.workunit.client.1.smithi069.stdout:8/762: dwrite d1/d1c/fd9 [0,4194304] 0 2024-03-22T07:16:05.803 INFO:tasks.workunit.client.1.smithi069.stdout:9/492: dwrite d2/dd/d3c/d48/f4d [4194304,4194304] 0 2024-03-22T07:16:05.803 INFO:tasks.workunit.client.1.smithi069.stdout:9/493: write d2/d51/f57 [4873430,30931] 0 2024-03-22T07:16:05.804 INFO:tasks.workunit.client.1.smithi069.stdout:3/640: dwrite d5/d9/d26/d3f/d35/f37 [0,4194304] 0 2024-03-22T07:16:05.804 INFO:tasks.workunit.client.1.smithi069.stdout:7/617: creat d9/d8e/fc5 x:0 0 0 2024-03-22T07:16:05.815 INFO:tasks.workunit.client.1.smithi069.stdout:3/641: rename d5/d9/f5e to d5/d9/d26/d3f/d56/d8a/fd1 0 2024-03-22T07:16:05.816 INFO:tasks.workunit.client.1.smithi069.stdout:7/618: symlink d9/db/d1a/db1/d57/d75/lc6 0 2024-03-22T07:16:05.816 INFO:tasks.workunit.client.1.smithi069.stdout:7/619: stat f8 0 2024-03-22T07:16:05.817 INFO:tasks.workunit.client.1.smithi069.stdout:9/494: rmdir d2/dd/d3c/d2e/d84 39 2024-03-22T07:16:05.817 INFO:tasks.workunit.client.1.smithi069.stdout:9/495: fdatasync d2/dd/d3c/d2e/d84/f96 0 2024-03-22T07:16:05.817 INFO:tasks.workunit.client.1.smithi069.stdout:9/496: dread - d2/d71/f8d zero size 2024-03-22T07:16:05.821 INFO:tasks.workunit.client.1.smithi069.stdout:9/497: creat d2/dd/d3c/d48/d7f/fa0 x:0 0 0 2024-03-22T07:16:05.824 INFO:tasks.workunit.client.1.smithi069.stdout:7/620: rename d9/d42/d8a to d9/db/d1a/db1/d57/d44/d53/dc7 0 2024-03-22T07:16:05.828 INFO:tasks.workunit.client.1.smithi069.stdout:0/708: dwrite d3/d7/f12 [0,4194304] 0 2024-03-22T07:16:05.831 INFO:tasks.workunit.client.1.smithi069.stdout:4/751: dwrite d6/d12/d18/d21/f49 [0,4194304] 0 2024-03-22T07:16:05.831 INFO:tasks.workunit.client.1.smithi069.stdout:4/752: write d6/d9b/f14 [444701,54928] 0 2024-03-22T07:16:05.831 INFO:tasks.workunit.client.1.smithi069.stdout:4/753: read d6/d12/d18/f4f [1030078,92477] 0 2024-03-22T07:16:05.831 INFO:tasks.workunit.client.1.smithi069.stdout:4/754: write d6/d12/d5d/ff6 [48175,518] 0 2024-03-22T07:16:05.831 INFO:tasks.workunit.client.1.smithi069.stdout:4/755: chown d6/d4b/la0 66469 1 2024-03-22T07:16:05.831 INFO:tasks.workunit.client.1.smithi069.stdout:8/763: dwrite d1/d1c/d3e/d88/da8/dbb/fd8 [0,4194304] 0 2024-03-22T07:16:05.832 INFO:tasks.workunit.client.1.smithi069.stdout:0/709: creat d3/d7/d15/d1f/fe8 x:0 0 0 2024-03-22T07:16:05.835 INFO:tasks.workunit.client.1.smithi069.stdout:9/498: dread d2/dd/d3c/d2e/f32 [0,4194304] 0 2024-03-22T07:16:05.835 INFO:tasks.workunit.client.1.smithi069.stdout:3/642: dwrite d5/d9/db/d3b/f95 [0,4194304] 0 2024-03-22T07:16:05.840 INFO:tasks.workunit.client.1.smithi069.stdout:0/710: mkdir d3/d7/d15/d29/dcf/de9 0 2024-03-22T07:16:05.840 INFO:tasks.workunit.client.1.smithi069.stdout:0/711: dread - d3/d7/d15/fdb zero size 2024-03-22T07:16:05.843 INFO:tasks.workunit.client.1.smithi069.stdout:0/712: dread d3/d7/d15/dad/fae [0,4194304] 0 2024-03-22T07:16:05.845 INFO:tasks.workunit.client.1.smithi069.stdout:9/499: truncate d2/d21/f72 1621759 0 2024-03-22T07:16:05.845 INFO:tasks.workunit.client.1.smithi069.stdout:9/500: readlink d2/l58 0 2024-03-22T07:16:05.845 INFO:tasks.workunit.client.1.smithi069.stdout:3/643: write d5/fa0 [950396,79325] 0 2024-03-22T07:16:05.845 INFO:tasks.workunit.client.1.smithi069.stdout:3/644: stat d5/d9/f68 0 2024-03-22T07:16:05.847 INFO:tasks.workunit.client.1.smithi069.stdout:8/764: rename d1/d1c/d3e/d88/dcd/d6c/c60 to d1/d93/cf3 0 2024-03-22T07:16:05.849 INFO:tasks.workunit.client.1.smithi069.stdout:0/713: mknod d3/d7/d15/dce/cea 0 2024-03-22T07:16:05.849 INFO:tasks.workunit.client.1.smithi069.stdout:4/756: dwrite d6/d16/d3a/db4/fde [0,4194304] 0 2024-03-22T07:16:05.849 INFO:tasks.workunit.client.1.smithi069.stdout:4/757: creat d6/d12/d18/d21/f103 x:0 0 0 2024-03-22T07:16:05.849 INFO:tasks.workunit.client.1.smithi069.stdout:4/758: creat d6/d9b/d1f/d58/d7c/f104 x:0 0 0 2024-03-22T07:16:05.850 INFO:tasks.workunit.client.1.smithi069.stdout:9/501: symlink d2/d21/la1 0 2024-03-22T07:16:05.850 INFO:tasks.workunit.client.1.smithi069.stdout:9/502: stat d2/dd/d3c/d53/d8b 0 2024-03-22T07:16:05.851 INFO:tasks.workunit.client.1.smithi069.stdout:3/645: creat d5/d9/db/d3b/dc6/fd2 x:0 0 0 2024-03-22T07:16:05.855 INFO:tasks.workunit.client.1.smithi069.stdout:8/765: mknod d1/d1c/d3e/d88/da8/cf4 0 2024-03-22T07:16:05.855 INFO:tasks.workunit.client.1.smithi069.stdout:8/766: creat d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/ff5 x:0 0 0 2024-03-22T07:16:05.855 INFO:tasks.workunit.client.1.smithi069.stdout:8/767: dread - d1/fcb zero size 2024-03-22T07:16:05.858 INFO:tasks.workunit.client.1.smithi069.stdout:0/714: rename d3/d7/d15/d29/d6e/caf to d3/d7/d15/d29/d5a/d5d/db4/ceb 0 2024-03-22T07:16:05.858 INFO:tasks.workunit.client.1.smithi069.stdout:0/715: chown d3/d7/d15/d29/d5a/d5d 187 1 2024-03-22T07:16:05.865 INFO:tasks.workunit.client.1.smithi069.stdout:9/503: mkdir d2/dd/d3c/d48/d7f/da2 0 2024-03-22T07:16:05.865 INFO:tasks.workunit.client.1.smithi069.stdout:9/504: creat d2/dd/d3c/d2e/fa3 x:0 0 0 2024-03-22T07:16:05.867 INFO:tasks.workunit.client.1.smithi069.stdout:8/768: dread d1/d1c/f52 [0,4194304] 0 2024-03-22T07:16:05.870 INFO:tasks.workunit.client.0.smithi063.stdout:5/747: unlink d8/d13/d1e/d31/d58/f68 0 2024-03-22T07:16:05.870 INFO:tasks.workunit.client.0.smithi063.stdout:5/748: chown d8/d13/d28/d57 21174 1 2024-03-22T07:16:05.872 INFO:tasks.workunit.client.0.smithi063.stdout:5/749: symlink d8/d13/d1e/d31/d64/d65/d86/le6 0 2024-03-22T07:16:05.874 INFO:tasks.workunit.client.1.smithi069.stdout:0/716: dread d3/d7/d17/f3f [0,4194304] 0 2024-03-22T07:16:05.874 INFO:tasks.workunit.client.1.smithi069.stdout:0/717: write d3/d7/d15/d29/d6e/f70 [737360,59979] 0 2024-03-22T07:16:05.875 INFO:tasks.workunit.client.1.smithi069.stdout:8/769: link d1/d1c/c34 d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/ddb/cf6 0 2024-03-22T07:16:05.880 INFO:tasks.workunit.client.1.smithi069.stdout:8/770: symlink d1/d1c/d3e/d66/d9e/dd1/lf7 0 2024-03-22T07:16:05.880 INFO:tasks.workunit.client.1.smithi069.stdout:8/771: write d1/d1c/d3e/d66/fe2 [1025354,67663] 0 2024-03-22T07:16:05.882 INFO:tasks.workunit.client.1.smithi069.stdout:8/772: unlink d1/d1c/d3e/d66/d9e/dd1/lf7 0 2024-03-22T07:16:05.884 INFO:tasks.workunit.client.1.smithi069.stdout:3/646: dwrite d5/d9/d26/d3f/d56/d8a/fd1 [0,4194304] 0 2024-03-22T07:16:05.884 INFO:tasks.workunit.client.1.smithi069.stdout:3/647: read d5/d9/d23/f6d [4172560,64555] 0 2024-03-22T07:16:05.885 INFO:tasks.workunit.client.1.smithi069.stdout:8/773: creat d1/d1c/d3e/db9/ff8 x:0 0 0 2024-03-22T07:16:05.885 INFO:tasks.workunit.client.0.smithi063.stdout:7/692: creat d5/fe6 x:0 0 0 2024-03-22T07:16:05.885 INFO:tasks.workunit.client.0.smithi063.stdout:7/693: chown d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/f36 3 1 2024-03-22T07:16:05.885 INFO:tasks.workunit.client.0.smithi063.stdout:9/834: dwrite d8/d19/d20/fa0 [0,4194304] 0 2024-03-22T07:16:05.885 INFO:tasks.workunit.client.0.smithi063.stdout:9/835: chown d8/d15/fc9 25001 1 2024-03-22T07:16:05.885 INFO:tasks.workunit.client.0.smithi063.stdout:7/694: stat d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d4f/c74 0 2024-03-22T07:16:05.885 INFO:tasks.workunit.client.0.smithi063.stdout:7/695: stat d5/d16/d1b 0 2024-03-22T07:16:05.886 INFO:tasks.workunit.client.0.smithi063.stdout:7/696: chown d5/d16/f2b 27 1 2024-03-22T07:16:05.886 INFO:tasks.workunit.client.0.smithi063.stdout:7/697: write d5/da/d28/d4b/da8/faa [59684,57713] 0 2024-03-22T07:16:05.886 INFO:tasks.workunit.client.1.smithi069.stdout:0/718: dread d3/d7/d17/f6b [8388608,4194304] 0 2024-03-22T07:16:05.886 INFO:tasks.workunit.client.1.smithi069.stdout:9/505: dwrite d2/dd/d3c/d48/d7c/f9e [0,4194304] 0 2024-03-22T07:16:05.887 INFO:tasks.workunit.client.1.smithi069.stdout:9/506: chown d2/dd/c17 147 1 2024-03-22T07:16:05.889 INFO:tasks.workunit.client.0.smithi063.stdout:9/836: creat d8/d19/f100 x:0 0 0 2024-03-22T07:16:05.889 INFO:tasks.workunit.client.0.smithi063.stdout:9/837: dread - d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/ff8 zero size 2024-03-22T07:16:05.889 INFO:tasks.workunit.client.0.smithi063.stdout:9/838: write d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/f7c [1164881,4296] 0 2024-03-22T07:16:05.890 INFO:tasks.workunit.client.1.smithi069.stdout:3/648: rename d5/d9/d26/d3f/d56/d7f/f96 to d5/d9/d26/db4/fd3 0 2024-03-22T07:16:05.891 INFO:tasks.workunit.client.0.smithi063.stdout:7/698: creat d5/da/fe7 x:0 0 0 2024-03-22T07:16:05.892 INFO:tasks.workunit.client.0.smithi063.stdout:7/699: mknod d5/da/d28/d4b/da8/dbc/d62/d8d/dbb/ce8 0 2024-03-22T07:16:05.893 INFO:tasks.workunit.client.0.smithi063.stdout:7/700: read - d5/da/d28/d4b/da8/dbc/d62/d73/dc5/fc9 zero size 2024-03-22T07:16:05.894 INFO:tasks.workunit.client.0.smithi063.stdout:5/750: dwrite d8/d13/d1e/d31/daf/fcd [0,4194304] 0 2024-03-22T07:16:05.895 INFO:tasks.workunit.client.0.smithi063.stdout:7/701: creat d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d4f/dd1/fe9 x:0 0 0 2024-03-22T07:16:05.896 INFO:tasks.workunit.client.1.smithi069.stdout:9/507: unlink d2/dd/d3c/d2e/f66 0 2024-03-22T07:16:05.897 INFO:tasks.workunit.client.1.smithi069.stdout:0/719: rename d3/d7/d24/d6a/fa2 to d3/d7/d15/d29/d6e/fec 0 2024-03-22T07:16:05.898 INFO:tasks.workunit.client.0.smithi063.stdout:5/751: mkdir d8/d13/d1e/d31/d64/de7 0 2024-03-22T07:16:05.898 INFO:tasks.workunit.client.1.smithi069.stdout:8/774: write d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/f76 [133682,45620] 0 2024-03-22T07:16:05.901 INFO:tasks.workunit.client.1.smithi069.stdout:9/508: creat d2/d21/d89/fa4 x:0 0 0 2024-03-22T07:16:05.902 INFO:tasks.workunit.client.0.smithi063.stdout:5/752: creat d8/d13/d28/d57/d5f/fe8 x:0 0 0 2024-03-22T07:16:05.903 INFO:tasks.workunit.client.1.smithi069.stdout:8/775: dread d1/d1c/d3e/fb0 [0,4194304] 0 2024-03-22T07:16:05.903 INFO:tasks.workunit.client.1.smithi069.stdout:8/776: fsync d1/d1c/d3e/fee 0 2024-03-22T07:16:05.903 INFO:tasks.workunit.client.1.smithi069.stdout:0/720: mkdir d3/d7/d15/d29/d5a/d5d/db4/ded 0 2024-03-22T07:16:05.904 INFO:tasks.workunit.client.1.smithi069.stdout:0/721: write d3/d7/d15/d29/d5a/f93 [79977,13800] 0 2024-03-22T07:16:05.904 INFO:tasks.workunit.client.1.smithi069.stdout:0/722: write d3/d7/d15/d29/d6e/fec [169821,94780] 0 2024-03-22T07:16:05.904 INFO:tasks.workunit.client.1.smithi069.stdout:0/723: creat d3/d7/d15/d29/fee x:0 0 0 2024-03-22T07:16:05.906 INFO:tasks.workunit.client.0.smithi063.stdout:7/702: rename d5/da/d5f/d33/d57 to d5/da/d28/d4b/da8/dd4/dea 0 2024-03-22T07:16:05.914 INFO:tasks.workunit.client.0.smithi063.stdout:5/753: rename d8/d13/d1e/d31/d58 to d8/de9 0 2024-03-22T07:16:05.914 INFO:tasks.workunit.client.1.smithi069.stdout:9/509: mkdir d2/dd/d3c/d2e/d84/da5 0 2024-03-22T07:16:05.915 INFO:tasks.workunit.client.0.smithi063.stdout:4/796: creat d1/d5/dc/f10f x:0 0 0 2024-03-22T07:16:05.917 INFO:tasks.workunit.client.0.smithi063.stdout:4/797: rename d1/d5/dc/d95/f60 to d1/d5/dc2/f110 0 2024-03-22T07:16:05.917 INFO:tasks.workunit.client.0.smithi063.stdout:4/798: chown d1/d5/dc/d95/d59/d9c/dac/fe6 1 1 2024-03-22T07:16:05.918 INFO:tasks.workunit.client.0.smithi063.stdout:4/799: symlink d1/d5/dc/dda/d79/l111 0 2024-03-22T07:16:05.919 INFO:tasks.workunit.client.0.smithi063.stdout:4/800: getdents d1/d5/d43/d27/d2c/d3d/d54/ddb 0 2024-03-22T07:16:05.925 INFO:tasks.workunit.client.1.smithi069.stdout:8/777: getdents d1/d93/ddf 0 2024-03-22T07:16:05.925 INFO:tasks.workunit.client.1.smithi069.stdout:8/778: truncate d1/d1c/d3e/d88/dcd/d6c/d73/fea 949195 0 2024-03-22T07:16:05.928 INFO:tasks.workunit.client.1.smithi069.stdout:8/779: link d1/d93/ddf/fe9 d1/d1c/d3e/db9/ff9 0 2024-03-22T07:16:05.929 INFO:tasks.workunit.client.0.smithi063.stdout:7/703: dread d5/da/d28/da7/fb0 [0,4194304] 0 2024-03-22T07:16:05.929 INFO:tasks.workunit.client.0.smithi063.stdout:7/704: read - d5/d16/fe3 zero size 2024-03-22T07:16:05.929 INFO:tasks.workunit.client.0.smithi063.stdout:7/705: truncate d5/da/d28/d4b/da8/dbc/d85/fb9 469090 0 2024-03-22T07:16:05.938 INFO:tasks.workunit.client.0.smithi063.stdout:4/801: dread d1/d5/f9 [0,4194304] 0 2024-03-22T07:16:05.942 INFO:tasks.workunit.client.0.smithi063.stdout:9/839: dwrite d8/d19/d20/d22/d29/d31/d53/d6a/da2/f4a [4194304,4194304] 0 2024-03-22T07:16:05.942 INFO:tasks.workunit.client.0.smithi063.stdout:9/840: symlink d8/d19/d20/d22/de1/db7/df9/l101 0 2024-03-22T07:16:05.943 INFO:tasks.workunit.client.0.smithi063.stdout:9/841: chown d8/d19/d20/fa0 348 1 2024-03-22T07:16:05.955 INFO:tasks.workunit.client.0.smithi063.stdout:7/706: dread d5/f9 [4194304,4194304] 0 2024-03-22T07:16:05.957 INFO:tasks.workunit.client.0.smithi063.stdout:7/707: rename d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d4f/lc1 to d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/leb 0 2024-03-22T07:16:05.957 INFO:tasks.workunit.client.0.smithi063.stdout:7/708: write d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d4f/fa4 [736383,57742] 0 2024-03-22T07:16:05.957 INFO:tasks.workunit.client.0.smithi063.stdout:7/709: symlink d5/da/d28/d4b/da8/dbc/d62/d73/d1d/lec 0 2024-03-22T07:16:05.958 INFO:tasks.workunit.client.0.smithi063.stdout:7/710: mkdir d5/da/d28/da7/ded 0 2024-03-22T07:16:05.961 INFO:tasks.workunit.client.1.smithi069.stdout:3/649: dwrite d5/d9/d23/f6d [0,4194304] 0 2024-03-22T07:16:05.961 INFO:tasks.workunit.client.1.smithi069.stdout:3/650: creat d5/d9/db/dae/fd4 x:0 0 0 2024-03-22T07:16:05.964 INFO:tasks.workunit.client.0.smithi063.stdout:9/842: dread d8/f14 [0,4194304] 0 2024-03-22T07:16:05.965 INFO:tasks.workunit.client.0.smithi063.stdout:9/843: rename d8/d19/d20/d22/d5d/d76/d95/le4 to d8/d19/d20/d22/d9b/l102 0 2024-03-22T07:16:05.965 INFO:tasks.workunit.client.0.smithi063.stdout:9/844: chown d8/d19/d20/d22/db2 421954907 1 2024-03-22T07:16:05.973 INFO:tasks.workunit.client.1.smithi069.stdout:3/651: dread d5/d9/d26/d3f/f3e [0,4194304] 0 2024-03-22T07:16:05.973 INFO:tasks.workunit.client.0.smithi063.stdout:7/711: write d5/da/d28/d4b/da8/dbc/fd9 [942547,85472] 0 2024-03-22T07:16:05.975 INFO:tasks.workunit.client.1.smithi069.stdout:3/652: creat d5/d9/d26/d3f/d56/fd5 x:0 0 0 2024-03-22T07:16:05.975 INFO:tasks.workunit.client.1.smithi069.stdout:3/653: fdatasync d5/d9/d23/f73 0 2024-03-22T07:16:05.975 INFO:tasks.workunit.client.1.smithi069.stdout:3/654: write d5/d9/d26/d3f/d54/d9d/fb0 [689096,3355] 0 2024-03-22T07:16:05.975 INFO:tasks.workunit.client.1.smithi069.stdout:3/655: fdatasync d5/d9/d26/d3f/d56/d7f/fa1 0 2024-03-22T07:16:05.975 INFO:tasks.workunit.client.0.smithi063.stdout:7/712: symlink d5/da/d4e/lee 0 2024-03-22T07:16:05.976 INFO:tasks.workunit.client.0.smithi063.stdout:9/845: dread d8/d19/d20/d22/f65 [0,4194304] 0 2024-03-22T07:16:05.977 INFO:tasks.workunit.client.0.smithi063.stdout:7/713: creat d5/d16/fef x:0 0 0 2024-03-22T07:16:05.977 INFO:tasks.workunit.client.0.smithi063.stdout:7/714: dread - d5/da/d28/d4b/da8/fdb zero size 2024-03-22T07:16:05.977 INFO:tasks.workunit.client.0.smithi063.stdout:7/715: write d5/da/d5f/d33/fa3 [762494,40202] 0 2024-03-22T07:16:05.977 INFO:tasks.workunit.client.0.smithi063.stdout:7/716: truncate d5/da/d28/d4b/da8/dbc/d85/f9f 381548 0 2024-03-22T07:16:05.978 INFO:tasks.workunit.client.1.smithi069.stdout:3/656: rmdir d5/d9/d26/d36 39 2024-03-22T07:16:05.979 INFO:tasks.workunit.client.1.smithi069.stdout:3/657: mkdir d5/dd6 0 2024-03-22T07:16:05.980 INFO:tasks.workunit.client.0.smithi063.stdout:9/846: rename d8/d17/d1e/c68 to d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/da5/dee/c103 0 2024-03-22T07:16:05.980 INFO:tasks.workunit.client.1.smithi069.stdout:3/658: rename d5/d9/d26/l44 to d5/d9/d26/d3f/dbe/ld7 0 2024-03-22T07:16:05.981 INFO:tasks.workunit.client.1.smithi069.stdout:3/659: write d5/d9/d26/d3f/d35/f50 [1989641,99400] 0 2024-03-22T07:16:05.981 INFO:tasks.workunit.client.1.smithi069.stdout:9/510: dwrite d2/d71/d90/f95 [0,4194304] 0 2024-03-22T07:16:05.982 INFO:tasks.workunit.client.1.smithi069.stdout:8/780: dwrite d1/d1c/d3e/f61 [0,4194304] 0 2024-03-22T07:16:05.982 INFO:tasks.workunit.client.0.smithi063.stdout:7/717: mkdir d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/df0 0 2024-03-22T07:16:05.982 INFO:tasks.workunit.client.0.smithi063.stdout:7/718: read - d5/da/d28/d4b/da8/dd4/dea/db3/fe0 zero size 2024-03-22T07:16:05.982 INFO:tasks.workunit.client.0.smithi063.stdout:4/802: dwrite d1/d5/dc/d95/fc3 [0,4194304] 0 2024-03-22T07:16:05.982 INFO:tasks.workunit.client.0.smithi063.stdout:5/754: dwrite d8/d13/d28/d57/dca/fe1 [0,4194304] 0 2024-03-22T07:16:05.983 INFO:tasks.workunit.client.0.smithi063.stdout:4/803: creat d1/d5/dc2/f112 x:0 0 0 2024-03-22T07:16:05.983 INFO:tasks.workunit.client.1.smithi069.stdout:0/724: dwrite d3/d7/f8e [0,4194304] 0 2024-03-22T07:16:05.983 INFO:tasks.workunit.client.0.smithi063.stdout:4/804: write d1/d5/d43/d27/d2c/d3d/d54/f8d [242812,53771] 0 2024-03-22T07:16:05.984 INFO:tasks.workunit.client.0.smithi063.stdout:7/719: dread d5/da/d28/d4b/da8/dbc/f86 [0,4194304] 0 2024-03-22T07:16:05.984 INFO:tasks.workunit.client.1.smithi069.stdout:9/511: mkdir d2/dd/d3c/d53/da6 0 2024-03-22T07:16:05.985 INFO:tasks.workunit.client.1.smithi069.stdout:9/512: chown d2/d71/f8d 73995867 1 2024-03-22T07:16:05.986 INFO:tasks.workunit.client.0.smithi063.stdout:5/755: chown d8/d13/d28/c3f 100 1 2024-03-22T07:16:05.989 INFO:tasks.workunit.client.0.smithi063.stdout:7/720: truncate d5/d38/f81 2017154 0 2024-03-22T07:16:05.989 INFO:tasks.workunit.client.0.smithi063.stdout:7/721: fdatasync d5/da/d28/da7/fdd 0 2024-03-22T07:16:05.993 INFO:tasks.workunit.client.1.smithi069.stdout:0/725: truncate d3/d7/d15/f32 3710734 0 2024-03-22T07:16:05.995 INFO:tasks.workunit.client.1.smithi069.stdout:8/781: getdents d1/d1c/d3e/d66 0 2024-03-22T07:16:05.995 INFO:tasks.workunit.client.1.smithi069.stdout:8/782: write d1/d1c/d3e/db9/ff9 [819617,67389] 0 2024-03-22T07:16:05.995 INFO:tasks.workunit.client.1.smithi069.stdout:3/660: dread d5/d9/d26/d3f/d35/f6b [4194304,4194304] 0 2024-03-22T07:16:05.996 INFO:tasks.workunit.client.1.smithi069.stdout:0/726: symlink d3/d7/d17/d1c/dd4/lef 0 2024-03-22T07:16:05.997 INFO:tasks.workunit.client.0.smithi063.stdout:4/805: read d1/d5/f97 [961056,23898] 0 2024-03-22T07:16:05.998 INFO:tasks.workunit.client.1.smithi069.stdout:8/783: mkdir d1/d1c/d3e/d88/dcd/d6c/d73/dfa 0 2024-03-22T07:16:05.999 INFO:tasks.workunit.client.0.smithi063.stdout:7/722: dread d5/da/d5f/d33/fa3 [0,4194304] 0 2024-03-22T07:16:05.999 INFO:tasks.workunit.client.0.smithi063.stdout:7/723: rename d5/da/d28/d4b/da8 to d5/da/d28/d4b/da8/dbc/d62/df1 22 2024-03-22T07:16:06.000 INFO:tasks.workunit.client.0.smithi063.stdout:5/756: write d8/d13/d1e/d31/daf/fcd [2843549,90234] 0 2024-03-22T07:16:06.001 INFO:tasks.workunit.client.1.smithi069.stdout:0/727: dread d3/d7/d17/f85 [0,4194304] 0 2024-03-22T07:16:06.001 INFO:tasks.workunit.client.1.smithi069.stdout:3/661: link d5/da2/lb1 d5/d9/d26/d36/d41/d51/ld8 0 2024-03-22T07:16:06.005 INFO:tasks.workunit.client.1.smithi069.stdout:0/728: mknod d3/d7/d15/d29/d5a/d5d/db4/ded/cf0 0 2024-03-22T07:16:06.005 INFO:tasks.workunit.client.1.smithi069.stdout:0/729: symlink d3/d7/d15/d29/d5a/d99/lf1 0 2024-03-22T07:16:06.005 INFO:tasks.workunit.client.1.smithi069.stdout:0/730: rename d3/d9a/fb7 to d3/d9a/ff2 0 2024-03-22T07:16:06.006 INFO:tasks.workunit.client.0.smithi063.stdout:6/888: sync 2024-03-22T07:16:06.006 INFO:tasks.workunit.client.0.smithi063.stdout:8/913: sync 2024-03-22T07:16:06.006 INFO:tasks.workunit.client.0.smithi063.stdout:3/924: sync 2024-03-22T07:16:06.006 INFO:tasks.workunit.client.0.smithi063.stdout:3/925: stat da/f52 0 2024-03-22T07:16:06.006 INFO:tasks.workunit.client.0.smithi063.stdout:3/926: chown da/d16/d3f/c5d 460712639 1 2024-03-22T07:16:06.006 INFO:tasks.workunit.client.0.smithi063.stdout:2/763: sync 2024-03-22T07:16:06.006 INFO:tasks.workunit.client.0.smithi063.stdout:3/927: fsync da/d16/d89/da9/dd3/f110 0 2024-03-22T07:16:06.006 INFO:tasks.workunit.client.0.smithi063.stdout:6/889: chown d3/d15/d16/d60/d7b/dd0 0 1 2024-03-22T07:16:06.006 INFO:tasks.workunit.client.0.smithi063.stdout:2/764: write d0/de/d1b/d32/ff3 [557553,47178] 0 2024-03-22T07:16:06.006 INFO:tasks.workunit.client.0.smithi063.stdout:8/914: truncate d0/d1b/d28/fa3 4654802 0 2024-03-22T07:16:06.008 INFO:tasks.workunit.client.0.smithi063.stdout:9/847: dwrite d8/d19/d20/d22/d29/faa [0,4194304] 0 2024-03-22T07:16:06.008 INFO:tasks.workunit.client.1.smithi069.stdout:0/731: dread d3/d7/d15/d29/d5a/f93 [0,4194304] 0 2024-03-22T07:16:06.008 INFO:tasks.workunit.client.1.smithi069.stdout:0/732: write d3/d7/d15/f20 [2513431,122150] 0 2024-03-22T07:16:06.010 INFO:tasks.workunit.client.1.smithi069.stdout:9/513: dwrite d2/f27 [0,4194304] 0 2024-03-22T07:16:06.011 INFO:tasks.workunit.client.0.smithi063.stdout:3/928: rename da/d63/d65/l47 to da/l12c 0 2024-03-22T07:16:06.011 INFO:tasks.workunit.client.0.smithi063.stdout:3/929: write da/d63/d65/d4b/fb2 [840229,36723] 0 2024-03-22T07:16:06.012 INFO:tasks.workunit.client.1.smithi069.stdout:0/733: creat d3/d7/d15/dce/ff3 x:0 0 0 2024-03-22T07:16:06.013 INFO:tasks.workunit.client.0.smithi063.stdout:7/724: write d5/da/d5f/d33/f53 [5173085,79683] 0 2024-03-22T07:16:06.015 INFO:tasks.workunit.client.0.smithi063.stdout:4/806: dread d1/d5/dc/d95/d59/f87 [0,4194304] 0 2024-03-22T07:16:06.015 INFO:tasks.workunit.client.0.smithi063.stdout:6/890: link d3/d15/d16/d60/fb3 d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/d10a/f129 0 2024-03-22T07:16:06.015 INFO:tasks.workunit.client.1.smithi069.stdout:9/514: rename d2/dd/d3c/d53/d8b/f9c to d2/dd/d3c/d48/d7f/d86/fa7 0 2024-03-22T07:16:06.016 INFO:tasks.workunit.client.1.smithi069.stdout:9/515: fsync d2/dd/d3c/d48/f4d 0 2024-03-22T07:16:06.017 INFO:tasks.workunit.client.1.smithi069.stdout:0/734: mkdir d3/d7/d15/d29/d5a/d84/df4 0 2024-03-22T07:16:06.023 INFO:tasks.workunit.client.1.smithi069.stdout:9/516: unlink d2/f7e 0 2024-03-22T07:16:06.023 INFO:tasks.workunit.client.1.smithi069.stdout:9/517: creat d2/dd/d3c/d2e/fa8 x:0 0 0 2024-03-22T07:16:06.023 INFO:tasks.workunit.client.1.smithi069.stdout:9/518: chown d2/d51/c60 5607575 1 2024-03-22T07:16:06.023 INFO:tasks.workunit.client.1.smithi069.stdout:0/735: rename d3/d7/d15/d29/d5a/f75 to d3/d7/d15/d29/d5a/d5d/db4/ff5 0 2024-03-22T07:16:06.024 INFO:tasks.workunit.client.0.smithi063.stdout:2/765: symlink d0/de/d12/dd1/lfc 0 2024-03-22T07:16:06.024 INFO:tasks.workunit.client.0.smithi063.stdout:8/915: mknod d0/d1b/d28/d3e/db2/dbd/dfd/c133 0 2024-03-22T07:16:06.024 INFO:tasks.workunit.client.1.smithi069.stdout:8/784: dwrite d1/d1c/d3e/f16 [0,4194304] 0 2024-03-22T07:16:06.026 INFO:tasks.workunit.client.0.smithi063.stdout:9/848: rename d8/d19/d20/d22/de1/dd7/dd8/lde to d8/d17/l104 0 2024-03-22T07:16:06.027 INFO:tasks.workunit.client.1.smithi069.stdout:9/519: symlink d2/dd/d3c/d48/d7f/d86/la9 0 2024-03-22T07:16:06.028 INFO:tasks.workunit.client.0.smithi063.stdout:3/930: dread da/d16/d3f/f6f [0,4194304] 0 2024-03-22T07:16:06.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:05 smithi069 ceph-mon[36441]: pgmap v5: 65 pgs: 65 active+clean; 2.4 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail 2024-03-22T07:16:06.032 INFO:tasks.workunit.client.0.smithi063.stdout:8/916: creat d0/d9e/f134 x:0 0 0 2024-03-22T07:16:06.033 INFO:tasks.workunit.client.1.smithi069.stdout:5/508: sync 2024-03-22T07:16:06.033 INFO:tasks.workunit.client.1.smithi069.stdout:5/509: getdents d1/d6/d26/d7d/d9f 0 2024-03-22T07:16:06.033 INFO:tasks.workunit.client.1.smithi069.stdout:5/510: truncate d1/d6/d86/d99/f15 1436144 0 2024-03-22T07:16:06.033 INFO:tasks.workunit.client.1.smithi069.stdout:5/511: readlink d1/d6/d26/d33/d5d/d72/d78/l7a 0 2024-03-22T07:16:06.033 INFO:tasks.workunit.client.1.smithi069.stdout:8/785: creat d1/d1c/d3e/de6/ffb x:0 0 0 2024-03-22T07:16:06.033 INFO:tasks.workunit.client.1.smithi069.stdout:8/786: truncate d1/d1c/d3e/d88/dcd/fca 1128766 0 2024-03-22T07:16:06.034 INFO:tasks.workunit.client.1.smithi069.stdout:8/787: chown d1/fd 327818 1 2024-03-22T07:16:06.034 INFO:tasks.workunit.client.1.smithi069.stdout:8/788: chown d1/d1c/d3e/d88/dcd/l15 94 1 2024-03-22T07:16:06.034 INFO:tasks.workunit.client.1.smithi069.stdout:8/789: write d1/d1c/f52 [4068746,117389] 0 2024-03-22T07:16:06.034 INFO:tasks.workunit.client.0.smithi063.stdout:9/849: unlink d8/d19/d20/d22/de1/cff 0 2024-03-22T07:16:06.036 INFO:tasks.workunit.client.0.smithi063.stdout:8/917: read d0/d1b/d28/d7b/fa4 [451971,66358] 0 2024-03-22T07:16:06.037 INFO:tasks.workunit.client.1.smithi069.stdout:9/520: unlink d2/dd/d3c/d2e/f3f 0 2024-03-22T07:16:06.038 INFO:tasks.workunit.client.0.smithi063.stdout:5/757: dwrite d8/d13/d1e/d3c/fae [0,4194304] 0 2024-03-22T07:16:06.041 INFO:tasks.workunit.client.1.smithi069.stdout:0/736: rmdir d3/d7/d15/d1f 39 2024-03-22T07:16:06.042 INFO:tasks.workunit.client.0.smithi063.stdout:2/766: write d0/de/d12/d20/f70 [224318,91399] 0 2024-03-22T07:16:06.043 INFO:tasks.workunit.client.0.smithi063.stdout:9/850: mknod d8/d19/d39/df4/c105 0 2024-03-22T07:16:06.045 INFO:tasks.workunit.client.1.smithi069.stdout:8/790: creat d1/d93/ffc x:0 0 0 2024-03-22T07:16:06.048 INFO:tasks.workunit.client.0.smithi063.stdout:2/767: unlink d0/de/d58/d7f/fa7 0 2024-03-22T07:16:06.048 INFO:tasks.workunit.client.0.smithi063.stdout:2/768: truncate d0/de/d74/fee 150081 0 2024-03-22T07:16:06.048 INFO:tasks.workunit.client.0.smithi063.stdout:2/769: stat d0/de/d74/dca/cce 0 2024-03-22T07:16:06.049 INFO:tasks.workunit.client.1.smithi069.stdout:8/791: link d1/d1c/d3e/d88/dcd/d20/l42 d1/d93/lfd 0 2024-03-22T07:16:06.051 INFO:tasks.workunit.client.0.smithi063.stdout:9/851: mkdir d8/d19/d20/d22/d9b/d106 0 2024-03-22T07:16:06.053 INFO:tasks.workunit.client.1.smithi069.stdout:8/792: rename d1/d1c/d3e/d88/dcd/d6c/cd4 to d1/d1c/d3e/d88/cfe 0 2024-03-22T07:16:06.053 INFO:tasks.workunit.client.1.smithi069.stdout:8/793: creat d1/d1c/fff x:0 0 0 2024-03-22T07:16:06.053 INFO:tasks.workunit.client.1.smithi069.stdout:8/794: chown d1/d1c/d3e/f32 102432 1 2024-03-22T07:16:06.053 INFO:tasks.workunit.client.0.smithi063.stdout:3/931: dread da/d16/d3f/f70 [0,4194304] 0 2024-03-22T07:16:06.054 INFO:tasks.workunit.client.0.smithi063.stdout:5/758: rename d8/d13/d28/d57/dca to d8/d13/d24/da4/d8b/db6/dea 0 2024-03-22T07:16:06.055 INFO:tasks.workunit.client.0.smithi063.stdout:9/852: truncate d8/d17/f1d 2307390 0 2024-03-22T07:16:06.055 INFO:tasks.workunit.client.0.smithi063.stdout:9/853: dread - d8/d19/d2b/ff6 zero size 2024-03-22T07:16:06.056 INFO:tasks.workunit.client.1.smithi069.stdout:8/795: mknod d1/d1c/d3e/d88/dd3/c100 0 2024-03-22T07:16:06.056 INFO:tasks.workunit.client.1.smithi069.stdout:8/796: fsync d1/d1c/d3e/fb0 0 2024-03-22T07:16:06.056 INFO:tasks.workunit.client.0.smithi063.stdout:3/932: rmdir da/d16/d26 39 2024-03-22T07:16:06.058 INFO:tasks.workunit.client.0.smithi063.stdout:3/933: read da/d63/d65/d81/fe8 [110337,97467] 0 2024-03-22T07:16:06.059 INFO:tasks.workunit.client.0.smithi063.stdout:2/770: rename d0/de/d12/d20/f82 to d0/de/d12/ffd 0 2024-03-22T07:16:06.059 INFO:tasks.workunit.client.1.smithi069.stdout:8/797: dread d1/d1c/f96 [0,4194304] 0 2024-03-22T07:16:06.060 INFO:tasks.workunit.client.1.smithi069.stdout:8/798: mkdir d1/d1c/d3e/d88/dcd/d6c/d4e/d101 0 2024-03-22T07:16:06.060 INFO:tasks.workunit.client.0.smithi063.stdout:5/759: mknod d8/d13/d1e/d31/ceb 0 2024-03-22T07:16:06.060 INFO:tasks.workunit.client.0.smithi063.stdout:5/760: truncate d8/d13/d1e/d31/d64/d65/f71 1524067 0 2024-03-22T07:16:06.062 INFO:tasks.workunit.client.1.smithi069.stdout:8/799: rename d1/d1c/d3e/d88/dcd/fa4 to d1/d1c/d3e/d88/dcd/d6c/d73/dfa/f102 0 2024-03-22T07:16:06.062 INFO:tasks.workunit.client.1.smithi069.stdout:8/800: truncate d1/f9 2719000 0 2024-03-22T07:16:06.063 INFO:tasks.workunit.client.0.smithi063.stdout:9/854: creat d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/da5/f107 x:0 0 0 2024-03-22T07:16:06.063 INFO:tasks.workunit.client.0.smithi063.stdout:3/934: mkdir da/d63/d65/d4b/d12d 0 2024-03-22T07:16:06.064 INFO:tasks.workunit.client.1.smithi069.stdout:8/801: creat d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/ddb/f103 x:0 0 0 2024-03-22T07:16:06.064 INFO:tasks.workunit.client.1.smithi069.stdout:8/802: fsync d1/d1c/d3e/d88/dcd/d6c/f72 0 2024-03-22T07:16:06.064 INFO:tasks.workunit.client.1.smithi069.stdout:8/803: write d1/d1c/d3e/db9/ff9 [1161977,130025] 0 2024-03-22T07:16:06.064 INFO:tasks.workunit.client.1.smithi069.stdout:8/804: write d1/fb3 [629926,121276] 0 2024-03-22T07:16:06.064 INFO:tasks.workunit.client.0.smithi063.stdout:2/771: creat d0/de/d35/ffe x:0 0 0 2024-03-22T07:16:06.065 INFO:tasks.workunit.client.0.smithi063.stdout:2/772: chown d0/de/d1b/d8b/fb7 3179 1 2024-03-22T07:16:06.066 INFO:tasks.workunit.client.0.smithi063.stdout:5/761: mknod d8/d13/d1e/d3c/cec 0 2024-03-22T07:16:06.067 INFO:tasks.workunit.client.1.smithi069.stdout:3/662: dwrite d5/fc7 [0,4194304] 0 2024-03-22T07:16:06.067 INFO:tasks.workunit.client.0.smithi063.stdout:9/855: symlink d8/d19/d20/d22/d29/d31/d53/d6a/da2/l108 0 2024-03-22T07:16:06.067 INFO:tasks.workunit.client.0.smithi063.stdout:9/856: chown d8/d19/d20/d22/d5d/d76/d95 221110 1 2024-03-22T07:16:06.068 INFO:tasks.workunit.client.1.smithi069.stdout:3/663: write d5/d9/d26/d3f/d56/d7f/fa1 [1054314,90055] 0 2024-03-22T07:16:06.068 INFO:tasks.workunit.client.1.smithi069.stdout:3/664: write d5/d9/d23/f2b [1102779,24417] 0 2024-03-22T07:16:06.068 INFO:tasks.workunit.client.1.smithi069.stdout:3/665: creat d5/d9/d26/d3f/d56/fd9 x:0 0 0 2024-03-22T07:16:06.068 INFO:tasks.workunit.client.0.smithi063.stdout:3/935: creat da/d16/d89/da9/f12e x:0 0 0 2024-03-22T07:16:06.074 INFO:tasks.workunit.client.0.smithi063.stdout:2/773: link d0/de/d58/f7d d0/de/d58/fff 0 2024-03-22T07:16:06.075 INFO:tasks.workunit.client.0.smithi063.stdout:2/774: chown d0/de/d12/l4a 1072150427 1 2024-03-22T07:16:06.075 INFO:tasks.workunit.client.0.smithi063.stdout:3/936: dread da/d63/d65/d4b/dfa/dfc/f106 [0,4194304] 0 2024-03-22T07:16:06.076 INFO:tasks.workunit.client.0.smithi063.stdout:5/762: rename d8/d13/d1e/d31/d64/fbb to d8/d13/d28/d57/fed 0 2024-03-22T07:16:06.077 INFO:tasks.workunit.client.0.smithi063.stdout:9/857: creat d8/d19/d20/d22/d5d/d76/dab/f109 x:0 0 0 2024-03-22T07:16:06.078 INFO:tasks.workunit.client.0.smithi063.stdout:9/858: stat d8/d19/d20/d22/d29/df0/ffa 0 2024-03-22T07:16:06.078 INFO:tasks.workunit.client.0.smithi063.stdout:9/859: write d8/d19/d20/d22/de1/f78 [92174,103081] 0 2024-03-22T07:16:06.080 INFO:tasks.workunit.client.0.smithi063.stdout:9/860: creat d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/da5/dee/f10a x:0 0 0 2024-03-22T07:16:06.081 INFO:tasks.workunit.client.0.smithi063.stdout:9/861: mknod d8/d19/d20/d22/d5d/c10b 0 2024-03-22T07:16:06.083 INFO:tasks.workunit.client.0.smithi063.stdout:5/763: dread d8/d13/d1e/d31/d64/fc2 [0,4194304] 0 2024-03-22T07:16:06.083 INFO:tasks.workunit.client.0.smithi063.stdout:9/862: mkdir d8/d19/d20/d22/de1/dd7/d10c 0 2024-03-22T07:16:06.083 INFO:tasks.workunit.client.0.smithi063.stdout:9/863: fdatasync d8/d19/d20/d22/d29/d31/d53/d6a/da2/f4a 0 2024-03-22T07:16:06.084 INFO:tasks.workunit.client.0.smithi063.stdout:3/937: dread da/d16/d3f/f50 [0,4194304] 0 2024-03-22T07:16:06.087 INFO:tasks.workunit.client.0.smithi063.stdout:3/938: rename da/d63/d65/d81/lde to da/d16/d89/d8c/dc1/dfe/l12f 0 2024-03-22T07:16:06.087 INFO:tasks.workunit.client.0.smithi063.stdout:4/807: dwrite d1/d5/dc/d95/f49 [4194304,4194304] 0 2024-03-22T07:16:06.087 INFO:tasks.workunit.client.0.smithi063.stdout:3/939: dread - da/d63/d65/d81/d9b/fba zero size 2024-03-22T07:16:06.088 INFO:tasks.workunit.client.0.smithi063.stdout:2/775: dread d0/de/d12/d20/fb3 [0,4194304] 0 2024-03-22T07:16:06.089 INFO:tasks.workunit.client.0.smithi063.stdout:4/808: mknod d1/d5/dc/c113 0 2024-03-22T07:16:06.089 INFO:tasks.workunit.client.0.smithi063.stdout:4/809: chown d1/d5/dc2/l10e 0 1 2024-03-22T07:16:06.091 INFO:tasks.workunit.client.0.smithi063.stdout:3/940: symlink da/d16/d89/da9/l130 0 2024-03-22T07:16:06.091 INFO:tasks.workunit.client.0.smithi063.stdout:3/941: write da/d63/d105/f126 [847373,49466] 0 2024-03-22T07:16:06.091 INFO:tasks.workunit.client.0.smithi063.stdout:3/942: readlink da/d16/l23 0 2024-03-22T07:16:06.092 INFO:tasks.workunit.client.0.smithi063.stdout:2/776: creat d0/de/d58/d7f/d97/da2/f100 x:0 0 0 2024-03-22T07:16:06.092 INFO:tasks.workunit.client.1.smithi069.stdout:0/737: dwrite d3/d7/d15/d1f/f26 [4194304,4194304] 0 2024-03-22T07:16:06.092 INFO:tasks.workunit.client.1.smithi069.stdout:5/512: dwrite d1/d6/d88/fa1 [0,4194304] 0 2024-03-22T07:16:06.093 INFO:tasks.workunit.client.0.smithi063.stdout:6/891: dwrite d3/f19 [0,4194304] 0 2024-03-22T07:16:06.093 INFO:tasks.workunit.client.1.smithi069.stdout:9/521: dwrite d2/d51/f7a [0,4194304] 0 2024-03-22T07:16:06.095 INFO:tasks.workunit.client.1.smithi069.stdout:8/805: dwrite d1/d1c/d3e/d88/dcd/d6c/d73/fea [0,4194304] 0 2024-03-22T07:16:06.095 INFO:tasks.workunit.client.1.smithi069.stdout:8/806: fdatasync d1/d1c/d48/fc3 0 2024-03-22T07:16:06.096 INFO:tasks.workunit.client.1.smithi069.stdout:8/807: truncate d1/d1c/d3e/f9c 400116 0 2024-03-22T07:16:06.096 INFO:tasks.workunit.client.0.smithi063.stdout:8/918: dwrite d0/d26/d2d/d59/dc0/d106/f12e [0,4194304] 0 2024-03-22T07:16:06.096 INFO:tasks.workunit.client.0.smithi063.stdout:8/919: fdatasync d0/d26/d2d/f4d 0 2024-03-22T07:16:06.096 INFO:tasks.workunit.client.0.smithi063.stdout:8/920: chown d0/d1b/d4f/dde/df5/c68 2 1 2024-03-22T07:16:06.096 INFO:tasks.workunit.client.0.smithi063.stdout:4/810: rename d1/d5/dc/dda to d1/d5/d3c/d8f/dbb/d114 0 2024-03-22T07:16:06.097 INFO:tasks.workunit.client.1.smithi069.stdout:3/666: dwrite d5/d9/d26/d3f/d56/d8a/da5/fc0 [0,4194304] 0 2024-03-22T07:16:06.098 INFO:tasks.workunit.client.1.smithi069.stdout:0/738: rename d3/d7/d15/l8c to d3/d7/d15/d29/d5a/d5d/db4/ded/lf6 0 2024-03-22T07:16:06.101 INFO:tasks.workunit.client.0.smithi063.stdout:5/764: dread d8/d13/d24/d81/fd4 [0,4194304] 0 2024-03-22T07:16:06.101 INFO:tasks.workunit.client.0.smithi063.stdout:5/765: rename d8/d13/d1e to d8/d13/d1e/d31/d64/d65/dee 22 2024-03-22T07:16:06.101 INFO:tasks.workunit.client.0.smithi063.stdout:5/766: fsync d8/d13/d24/f3b 0 2024-03-22T07:16:06.101 INFO:tasks.workunit.client.0.smithi063.stdout:3/943: unlink da/d63/d65/f5e 0 2024-03-22T07:16:06.101 INFO:tasks.workunit.client.0.smithi063.stdout:8/921: dread d0/d26/d2d/d59/f87 [0,4194304] 0 2024-03-22T07:16:06.101 INFO:tasks.workunit.client.0.smithi063.stdout:8/922: readlink d0/d26/d2d/d83/dbc/da1/lac 0 2024-03-22T07:16:06.102 INFO:tasks.workunit.client.0.smithi063.stdout:9/864: dread d8/d19/d20/d22/de1/f78 [0,4194304] 0 2024-03-22T07:16:06.107 INFO:tasks.workunit.client.0.smithi063.stdout:6/892: dread d3/d6/d2a/d38/f6c [0,4194304] 0 2024-03-22T07:16:06.107 INFO:tasks.workunit.client.0.smithi063.stdout:6/893: stat d3/d6/d35/l102 0 2024-03-22T07:16:06.108 INFO:tasks.workunit.client.0.smithi063.stdout:5/767: creat d8/d13/d1e/d31/d64/fef x:0 0 0 2024-03-22T07:16:06.109 INFO:tasks.workunit.client.0.smithi063.stdout:5/768: fdatasync d8/d13/d24/f50 0 2024-03-22T07:16:06.109 INFO:tasks.workunit.client.0.smithi063.stdout:5/769: write d8/fd9 [76801,125765] 0 2024-03-22T07:16:06.110 INFO:tasks.workunit.client.0.smithi063.stdout:3/944: symlink da/l131 0 2024-03-22T07:16:06.112 INFO:tasks.workunit.client.0.smithi063.stdout:8/923: truncate d0/d26/d2d/d59/dba/dc9/fff 1864541 0 2024-03-22T07:16:06.116 INFO:tasks.workunit.client.0.smithi063.stdout:5/770: symlink d8/d13/d24/da4/d8b/db6/dea/lf0 0 2024-03-22T07:16:06.117 INFO:tasks.workunit.client.1.smithi069.stdout:8/808: fdatasync d1/d1c/d48/f4b 0 2024-03-22T07:16:06.117 INFO:tasks.workunit.client.1.smithi069.stdout:8/809: chown d1/d1c/d3e/d88/dcd/l1e 420 1 2024-03-22T07:16:06.118 INFO:tasks.workunit.client.0.smithi063.stdout:8/924: getdents d0 0 2024-03-22T07:16:06.121 INFO:tasks.workunit.client.0.smithi063.stdout:5/771: creat d8/d13/d24/d81/ff1 x:0 0 0 2024-03-22T07:16:06.122 INFO:tasks.workunit.client.0.smithi063.stdout:8/925: rmdir d0/d1b/d28/d7b/d4a 39 2024-03-22T07:16:06.122 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:05 smithi063 ceph-mon[29200]: pgmap v5: 65 pgs: 65 active+clean; 2.4 GiB data, 8.3 GiB used, 528 GiB / 536 GiB avail 2024-03-22T07:16:06.123 INFO:tasks.workunit.client.1.smithi069.stdout:3/667: creat d5/dd6/fda x:0 0 0 2024-03-22T07:16:06.125 INFO:tasks.workunit.client.0.smithi063.stdout:5/772: creat d8/d13/d1e/d31/d64/d65/d86/ff2 x:0 0 0 2024-03-22T07:16:06.126 INFO:tasks.workunit.client.1.smithi069.stdout:0/739: rename d3/d7/d15/d29/d5a/d84/c23 to d3/d7/d90/cf7 0 2024-03-22T07:16:06.127 INFO:tasks.workunit.client.0.smithi063.stdout:8/926: unlink d0/cb 0 2024-03-22T07:16:06.127 INFO:tasks.workunit.client.1.smithi069.stdout:8/810: unlink d1/d1c/d3e/d88/dd3/c100 0 2024-03-22T07:16:06.128 INFO:tasks.workunit.client.0.smithi063.stdout:8/927: symlink d0/l135 0 2024-03-22T07:16:06.128 INFO:tasks.workunit.client.1.smithi069.stdout:3/668: creat d5/d9/d26/d3f/fdb x:0 0 0 2024-03-22T07:16:06.129 INFO:tasks.workunit.client.1.smithi069.stdout:0/740: mknod d3/d7/d15/d29/d5a/d99/dca/cf8 0 2024-03-22T07:16:06.129 INFO:tasks.workunit.client.1.smithi069.stdout:0/741: readlink d3/d7/d15/d1f/l78 0 2024-03-22T07:16:06.130 INFO:tasks.workunit.client.0.smithi063.stdout:8/928: creat d0/d24/d33/dd9/f136 x:0 0 0 2024-03-22T07:16:06.130 INFO:tasks.workunit.client.1.smithi069.stdout:8/811: creat d1/d1c/d3e/d88/dcd/d6c/d4e/f104 x:0 0 0 2024-03-22T07:16:06.131 INFO:tasks.workunit.client.0.smithi063.stdout:8/929: mknod d0/d1b/d28/de8/dc6/c137 0 2024-03-22T07:16:06.131 INFO:tasks.workunit.client.0.smithi063.stdout:8/930: stat d0/d26/d2d/d59/dc0 0 2024-03-22T07:16:06.131 INFO:tasks.workunit.client.0.smithi063.stdout:8/931: fdatasync d0/d24/f42 0 2024-03-22T07:16:06.132 INFO:tasks.workunit.client.1.smithi069.stdout:3/669: creat d5/da2/fdc x:0 0 0 2024-03-22T07:16:06.135 INFO:tasks.workunit.client.1.smithi069.stdout:0/742: mkdir d3/d7/d15/d29/d5a/d5d/d9d/df9 0 2024-03-22T07:16:06.135 INFO:tasks.workunit.client.1.smithi069.stdout:0/743: getdents d3/d7/d15/d29/d5a/d5d 0 2024-03-22T07:16:06.137 INFO:tasks.workunit.client.1.smithi069.stdout:8/812: creat d1/d1c/d3e/d88/dcd/d6c/d4e/d98/f105 x:0 0 0 2024-03-22T07:16:06.137 INFO:tasks.workunit.client.1.smithi069.stdout:3/670: dread d5/d9/db/fbb [0,4194304] 0 2024-03-22T07:16:06.137 INFO:tasks.workunit.client.1.smithi069.stdout:3/671: write d5/d9/d26/d3f/d35/f9e [180205,105042] 0 2024-03-22T07:16:06.137 INFO:tasks.workunit.client.1.smithi069.stdout:3/672: readlink d5/d9/l1d 0 2024-03-22T07:16:06.141 INFO:tasks.workunit.client.0.smithi063.stdout:5/773: write d8/d13/d1e/d3c/fa1 [822300,82395] 0 2024-03-22T07:16:06.141 INFO:tasks.workunit.client.0.smithi063.stdout:5/774: fdatasync d8/d13/d1e/d3c/f4c 0 2024-03-22T07:16:06.141 INFO:tasks.workunit.client.0.smithi063.stdout:5/775: stat d8/d13/d1e/d3c/f8c 0 2024-03-22T07:16:06.141 INFO:tasks.workunit.client.0.smithi063.stdout:5/776: chown d8/de9/l74 75871 1 2024-03-22T07:16:06.146 INFO:tasks.workunit.client.1.smithi069.stdout:5/513: dwrite d1/d6/d26/d33/d5d/fa4 [0,4194304] 0 2024-03-22T07:16:06.146 INFO:tasks.workunit.client.1.smithi069.stdout:5/514: dread - d1/d28/f91 zero size 2024-03-22T07:16:06.147 INFO:tasks.workunit.client.0.smithi063.stdout:9/865: getdents d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/da5/dee 0 2024-03-22T07:16:06.154 INFO:tasks.workunit.client.1.smithi069.stdout:9/522: dwrite d2/dd/f10 [4194304,4194304] 0 2024-03-22T07:16:06.155 INFO:tasks.workunit.client.0.smithi063.stdout:9/866: link d8/d19/d20/d22/d29/d31/d53/d6a/fe5 d8/d19/d20/d22/d29/f10d 0 2024-03-22T07:16:06.156 INFO:tasks.workunit.client.0.smithi063.stdout:2/777: dwrite d0/de/d1b/d32/f3b [4194304,4194304] 0 2024-03-22T07:16:06.157 INFO:tasks.workunit.client.1.smithi069.stdout:9/523: mknod d2/d83/caa 0 2024-03-22T07:16:06.157 INFO:tasks.workunit.client.0.smithi063.stdout:5/777: mkdir d8/d13/d28/df3 0 2024-03-22T07:16:06.158 INFO:tasks.workunit.client.0.smithi063.stdout:9/867: mkdir d8/d19/d39/db5/d10e 0 2024-03-22T07:16:06.159 INFO:tasks.workunit.client.1.smithi069.stdout:9/524: rmdir d2/d71 39 2024-03-22T07:16:06.160 INFO:tasks.workunit.client.1.smithi069.stdout:5/515: dread d1/d6/d86/d99/f58 [0,4194304] 0 2024-03-22T07:16:06.160 INFO:tasks.workunit.client.1.smithi069.stdout:5/516: dread - d1/d28/fb8 zero size 2024-03-22T07:16:06.161 INFO:tasks.workunit.client.0.smithi063.stdout:5/778: mknod d8/d13/d1e/d31/d64/cf4 0 2024-03-22T07:16:06.161 INFO:tasks.workunit.client.0.smithi063.stdout:5/779: dread - d8/d13/d1e/d3c/fb7 zero size 2024-03-22T07:16:06.165 INFO:tasks.workunit.client.0.smithi063.stdout:9/868: link d8/d19/d20/f34 d8/d19/d20/d22/d29/d31/d53/d6a/da2/df1/dfb/f10f 0 2024-03-22T07:16:06.166 INFO:tasks.workunit.client.0.smithi063.stdout:2/778: rmdir d0/de/d1b/d32/dac/dc2 39 2024-03-22T07:16:06.166 INFO:tasks.workunit.client.0.smithi063.stdout:5/780: truncate d8/f14 794901 0 2024-03-22T07:16:06.166 INFO:tasks.workunit.client.0.smithi063.stdout:5/781: stat d8/de9/fad 0 2024-03-22T07:16:06.169 INFO:tasks.workunit.client.1.smithi069.stdout:2/436: sync 2024-03-22T07:16:06.170 INFO:tasks.workunit.client.0.smithi063.stdout:2/779: creat d0/de/d12/d5a/f101 x:0 0 0 2024-03-22T07:16:06.174 INFO:tasks.workunit.client.1.smithi069.stdout:0/744: dwrite d3/d7/f12 [4194304,4194304] 0 2024-03-22T07:16:06.174 INFO:tasks.workunit.client.1.smithi069.stdout:0/745: stat d3/d7/d15/d29/d5a/d5d 0 2024-03-22T07:16:06.176 INFO:tasks.workunit.client.0.smithi063.stdout:5/782: dread d8/f10 [0,4194304] 0 2024-03-22T07:16:06.176 INFO:tasks.workunit.client.0.smithi063.stdout:5/783: truncate d8/d13/f4b 4203581 0 2024-03-22T07:16:06.176 INFO:tasks.workunit.client.0.smithi063.stdout:5/784: chown d8/d13/d1e/d31/d64/d65/d86/lb2 1 1 2024-03-22T07:16:06.176 INFO:tasks.workunit.client.0.smithi063.stdout:5/785: getdents d8/d13/d1e/d31/d64/d65/ddf 0 2024-03-22T07:16:06.176 INFO:tasks.workunit.client.1.smithi069.stdout:5/517: write d1/d6/d88/fa1 [3419795,114176] 0 2024-03-22T07:16:06.177 INFO:tasks.workunit.client.1.smithi069.stdout:9/525: dread d2/d21/f73 [0,4194304] 0 2024-03-22T07:16:06.178 INFO:tasks.workunit.client.1.smithi069.stdout:0/746: stat d3/d7/la8 0 2024-03-22T07:16:06.180 INFO:tasks.workunit.client.1.smithi069.stdout:0/747: symlink d3/d7/d15/d29/d5a/d5d/db4/lfa 0 2024-03-22T07:16:06.180 INFO:tasks.workunit.client.1.smithi069.stdout:0/748: chown d3/d7/d15/d29/l2d 12566233 1 2024-03-22T07:16:06.181 INFO:tasks.workunit.client.1.smithi069.stdout:2/437: write d18/d4d/d25/f5b [2909294,87682] 0 2024-03-22T07:16:06.181 INFO:tasks.workunit.client.1.smithi069.stdout:2/438: write d18/d4d/d3e/d40/f76 [788463,105544] 0 2024-03-22T07:16:06.183 INFO:tasks.workunit.client.1.smithi069.stdout:2/439: truncate d18/d4d/d34/f53 1715994 0 2024-03-22T07:16:06.184 INFO:tasks.workunit.client.1.smithi069.stdout:2/440: unlink d18/d4d/c21 0 2024-03-22T07:16:06.185 INFO:tasks.workunit.client.0.smithi063.stdout:6/894: dwrite d3/d15/d16/f111 [0,4194304] 0 2024-03-22T07:16:06.186 INFO:tasks.workunit.client.0.smithi063.stdout:3/945: dwrite da/f71 [0,4194304] 0 2024-03-22T07:16:06.186 INFO:tasks.workunit.client.0.smithi063.stdout:3/946: write da/d63/d65/d4b/d10a/f120 [699572,97165] 0 2024-03-22T07:16:06.186 INFO:tasks.workunit.client.0.smithi063.stdout:3/947: creat da/d63/d65/d83/dc6/f132 x:0 0 0 2024-03-22T07:16:06.186 INFO:tasks.workunit.client.0.smithi063.stdout:3/948: fdatasync da/d16/d3f/de2/ffd 0 2024-03-22T07:16:06.189 INFO:tasks.workunit.client.0.smithi063.stdout:3/949: symlink da/d63/d65/d7b/l133 0 2024-03-22T07:16:06.189 INFO:tasks.workunit.client.0.smithi063.stdout:3/950: truncate da/d16/d89/d8c/dc1/fee 4350518 0 2024-03-22T07:16:06.190 INFO:tasks.workunit.client.0.smithi063.stdout:3/951: chown da/d63/d65/d83/ff4 15 1 2024-03-22T07:16:06.190 INFO:tasks.workunit.client.0.smithi063.stdout:3/952: truncate da/d63/d65/d4b/fb2 1578003 0 2024-03-22T07:16:06.190 INFO:tasks.workunit.client.0.smithi063.stdout:3/953: stat da/d63/d65/d81/d9b/l9d 0 2024-03-22T07:16:06.190 INFO:tasks.workunit.client.0.smithi063.stdout:3/954: fdatasync da/d16/d26/f48 0 2024-03-22T07:16:06.190 INFO:tasks.workunit.client.0.smithi063.stdout:3/955: write da/d63/d65/d92/f124 [735847,79264] 0 2024-03-22T07:16:06.190 INFO:tasks.workunit.client.1.smithi069.stdout:5/518: read d1/d6/f2e [3211173,45684] 0 2024-03-22T07:16:06.192 INFO:tasks.workunit.client.0.smithi063.stdout:5/786: write d8/fa7 [3227554,27098] 0 2024-03-22T07:16:06.192 INFO:tasks.workunit.client.0.smithi063.stdout:5/787: fdatasync d8/d13/d1e/d31/d64/d65/d86/f92 0 2024-03-22T07:16:06.195 INFO:tasks.workunit.client.1.smithi069.stdout:3/673: dwrite d5/d9/d26/d36/fc8 [0,4194304] 0 2024-03-22T07:16:06.195 INFO:tasks.workunit.client.0.smithi063.stdout:5/788: rename d8/de9/d7e/ca2 to d8/de9/d7e/cf5 0 2024-03-22T07:16:06.195 INFO:tasks.workunit.client.0.smithi063.stdout:5/789: dread - d8/d13/d28/d57/d5f/fe8 zero size 2024-03-22T07:16:06.197 INFO:tasks.workunit.client.1.smithi069.stdout:6/546: sync 2024-03-22T07:16:06.197 INFO:tasks.workunit.client.1.smithi069.stdout:4/759: sync 2024-03-22T07:16:06.197 INFO:tasks.workunit.client.1.smithi069.stdout:7/621: sync 2024-03-22T07:16:06.197 INFO:tasks.workunit.client.1.smithi069.stdout:1/600: sync 2024-03-22T07:16:06.197 INFO:tasks.workunit.client.1.smithi069.stdout:1/601: fsync d1/d13/fa3 0 2024-03-22T07:16:06.198 INFO:tasks.workunit.client.0.smithi063.stdout:5/790: fsync d8/d13/d28/f73 0 2024-03-22T07:16:06.199 INFO:tasks.workunit.client.1.smithi069.stdout:6/547: dread d4/dd/d12/f1f [4194304,4194304] 0 2024-03-22T07:16:06.200 INFO:tasks.workunit.client.0.smithi063.stdout:8/932: dwrite d0/d9e/df0/ffe [0,4194304] 0 2024-03-22T07:16:06.200 INFO:tasks.workunit.client.1.smithi069.stdout:2/441: write d18/d4d/d27/d56/f62 [3881449,71079] 0 2024-03-22T07:16:06.200 INFO:tasks.workunit.client.0.smithi063.stdout:5/791: mknod d8/d13/d24/da4/d8b/cf6 0 2024-03-22T07:16:06.201 INFO:tasks.workunit.client.1.smithi069.stdout:8/813: dwrite d1/d1c/d48/d56/f57 [0,4194304] 0 2024-03-22T07:16:06.201 INFO:tasks.workunit.client.1.smithi069.stdout:8/814: dread - d1/fcb zero size 2024-03-22T07:16:06.202 INFO:tasks.workunit.client.1.smithi069.stdout:4/760: link d6/d16/d3a/db4/dfb/d23/f5c d6/d16/d68/df8/f105 0 2024-03-22T07:16:06.203 INFO:tasks.workunit.client.1.smithi069.stdout:4/761: read d6/d9b/d1f/d3f/f46 [395282,36268] 0 2024-03-22T07:16:06.203 INFO:tasks.workunit.client.0.smithi063.stdout:6/895: dread d3/d6/f28 [0,4194304] 0 2024-03-22T07:16:06.203 INFO:tasks.workunit.client.0.smithi063.stdout:6/896: write d3/d15/d16/d60/d6d/fe9 [141295,59868] 0 2024-03-22T07:16:06.203 INFO:tasks.workunit.client.0.smithi063.stdout:9/869: dwrite d8/f90 [0,4194304] 0 2024-03-22T07:16:06.203 INFO:tasks.workunit.client.0.smithi063.stdout:9/870: write d8/d19/d2b/f5f [1872062,90451] 0 2024-03-22T07:16:06.204 INFO:tasks.workunit.client.0.smithi063.stdout:9/871: chown d8/d15/d63/lb6 51638225 1 2024-03-22T07:16:06.204 INFO:tasks.workunit.client.1.smithi069.stdout:1/602: symlink d1/d2a/d50/d88/d41/d64/lbf 0 2024-03-22T07:16:06.206 INFO:tasks.workunit.client.1.smithi069.stdout:6/548: symlink d4/dd/d41/d78/d9a/laa 0 2024-03-22T07:16:06.207 INFO:tasks.workunit.client.1.smithi069.stdout:9/526: dread d2/dd/d3c/d48/f4a [0,4194304] 0 2024-03-22T07:16:06.209 INFO:tasks.workunit.client.1.smithi069.stdout:2/442: rename d18/d4d/d27/d56/f88 to d18/d4d/d27/d56/f94 0 2024-03-22T07:16:06.211 INFO:tasks.workunit.client.0.smithi063.stdout:8/933: symlink d0/d26/d2d/d83/dbc/da1/l138 0 2024-03-22T07:16:06.211 INFO:tasks.workunit.client.0.smithi063.stdout:8/934: fdatasync d0/d26/d2d/d83/d112/f101 0 2024-03-22T07:16:06.211 INFO:tasks.workunit.client.0.smithi063.stdout:8/935: read - d0/d1b/d28/de8/dc6/f12c zero size 2024-03-22T07:16:06.212 INFO:tasks.workunit.client.1.smithi069.stdout:8/815: truncate d1/d1c/d3e/d66/f75 71678 0 2024-03-22T07:16:06.216 INFO:tasks.workunit.client.1.smithi069.stdout:4/762: rename d6/d16/d3a/fcb to d6/dc4/f106 0 2024-03-22T07:16:06.216 INFO:tasks.workunit.client.1.smithi069.stdout:9/527: mknod d2/d21/cab 0 2024-03-22T07:16:06.218 INFO:tasks.workunit.client.1.smithi069.stdout:7/622: dread d9/db/d48/d4b/f4f [0,4194304] 0 2024-03-22T07:16:06.218 INFO:tasks.workunit.client.0.smithi063.stdout:5/792: write d8/fe [986588,3883] 0 2024-03-22T07:16:06.219 INFO:tasks.workunit.client.1.smithi069.stdout:2/443: creat d18/d4d/d27/d33/d60/d63/d80/f95 x:0 0 0 2024-03-22T07:16:06.219 INFO:tasks.workunit.client.1.smithi069.stdout:2/444: write d18/d4d/d27/f49 [2058250,4475] 0 2024-03-22T07:16:06.220 INFO:tasks.workunit.client.1.smithi069.stdout:2/445: write d18/d4d/d3e/d40/f6f [384246,128129] 0 2024-03-22T07:16:06.220 INFO:tasks.workunit.client.1.smithi069.stdout:2/446: stat d18/l23 0 2024-03-22T07:16:06.220 INFO:tasks.workunit.client.1.smithi069.stdout:2/447: fsync d18/d4d/d25/f47 0 2024-03-22T07:16:06.220 INFO:tasks.workunit.client.1.smithi069.stdout:2/448: dread - d18/d4d/d27/d33/d3d/f54 zero size 2024-03-22T07:16:06.221 INFO:tasks.workunit.client.1.smithi069.stdout:9/528: creat d2/dd/fac x:0 0 0 2024-03-22T07:16:06.224 INFO:tasks.workunit.client.1.smithi069.stdout:0/749: read d3/d7/d90/f94 [2400110,18872] 0 2024-03-22T07:16:06.224 INFO:tasks.workunit.client.1.smithi069.stdout:0/750: creat d3/d7/d24/d6a/ffb x:0 0 0 2024-03-22T07:16:06.224 INFO:tasks.workunit.client.1.smithi069.stdout:0/751: chown d3/d7/d17/f56 2812 1 2024-03-22T07:16:06.225 INFO:tasks.workunit.client.1.smithi069.stdout:0/752: write d3/d7/d90/fe0 [129506,79394] 0 2024-03-22T07:16:06.225 INFO:tasks.workunit.client.1.smithi069.stdout:2/449: mkdir d18/d4d/d3e/d40/d96 0 2024-03-22T07:16:06.229 INFO:tasks.workunit.client.1.smithi069.stdout:0/753: symlink d3/d7/d5e/lfc 0 2024-03-22T07:16:06.232 INFO:tasks.workunit.client.1.smithi069.stdout:0/754: creat d3/d7/d15/d29/d5a/d5d/ffd x:0 0 0 2024-03-22T07:16:06.232 INFO:tasks.workunit.client.1.smithi069.stdout:0/755: read - d3/d9a/ff2 zero size 2024-03-22T07:16:06.241 INFO:tasks.workunit.client.0.smithi063.stdout:3/956: dwrite da/d16/d26/fdc [0,4194304] 0 2024-03-22T07:16:06.241 INFO:tasks.workunit.client.0.smithi063.stdout:9/872: truncate d8/d19/d20/fa0 5646430 0 2024-03-22T07:16:06.243 INFO:tasks.workunit.client.1.smithi069.stdout:9/529: dread d2/dd/d3c/d48/f4d [4194304,4194304] 0 2024-03-22T07:16:06.244 INFO:tasks.workunit.client.0.smithi063.stdout:8/936: symlink d0/d1b/d4f/l139 0 2024-03-22T07:16:06.244 INFO:tasks.workunit.client.0.smithi063.stdout:8/937: chown d0/d24/d5b/cd7 6282 1 2024-03-22T07:16:06.244 INFO:tasks.workunit.client.0.smithi063.stdout:8/938: readlink d0/l66 0 2024-03-22T07:16:06.244 INFO:tasks.workunit.client.0.smithi063.stdout:8/939: dread - d0/d1b/d28/de8/dc6/f12c zero size 2024-03-22T07:16:06.247 INFO:tasks.workunit.client.0.smithi063.stdout:3/957: mknod da/d63/d65/d81/d9b/d9f/dcf/c134 0 2024-03-22T07:16:06.247 INFO:tasks.workunit.client.0.smithi063.stdout:3/958: creat da/d63/f135 x:0 0 0 2024-03-22T07:16:06.248 INFO:tasks.workunit.client.0.smithi063.stdout:5/793: symlink d8/lf7 0 2024-03-22T07:16:06.253 INFO:tasks.workunit.client.0.smithi063.stdout:5/794: symlink d8/d13/d24/da4/d8b/lf8 0 2024-03-22T07:16:06.254 INFO:tasks.workunit.client.0.smithi063.stdout:5/795: truncate d8/d13/d1e/d3c/da0/fb9 4540487 0 2024-03-22T07:16:06.255 INFO:tasks.workunit.client.0.smithi063.stdout:8/940: rename d0/d9e to d0/d26/d2d/d59/dc0/d106/d13a 0 2024-03-22T07:16:06.257 INFO:tasks.workunit.client.0.smithi063.stdout:8/941: creat d0/d1b/d28/d3e/f13b x:0 0 0 2024-03-22T07:16:06.257 INFO:tasks.workunit.client.0.smithi063.stdout:8/942: symlink d0/d1b/d4f/dde/l13c 0 2024-03-22T07:16:06.268 INFO:tasks.workunit.client.0.smithi063.stdout:5/796: dread d8/d13/d24/da4/d8b/db6/dea/fe1 [0,4194304] 0 2024-03-22T07:16:06.277 INFO:tasks.workunit.client.1.smithi069.stdout:3/674: dwrite d5/d9/db/d3b/dc6/fd2 [0,4194304] 0 2024-03-22T07:16:06.282 INFO:tasks.workunit.client.1.smithi069.stdout:3/675: mkdir d5/da2/ddd 0 2024-03-22T07:16:06.283 INFO:tasks.workunit.client.1.smithi069.stdout:3/676: getdents d5/d9/d26/d3f/d56/d7f/d90 0 2024-03-22T07:16:06.283 INFO:tasks.workunit.client.1.smithi069.stdout:3/677: chown d5/d9/d26/d3f/d35/f45 210898247 1 2024-03-22T07:16:06.284 INFO:tasks.workunit.client.1.smithi069.stdout:3/678: stat d5/da2/lb1 0 2024-03-22T07:16:06.288 INFO:tasks.workunit.client.1.smithi069.stdout:3/679: mkdir d5/d9/d26/d3f/d54/d9d/dde 0 2024-03-22T07:16:06.290 INFO:tasks.workunit.client.1.smithi069.stdout:1/603: dwrite d1/d13/f2d [0,4194304] 0 2024-03-22T07:16:06.290 INFO:tasks.workunit.client.1.smithi069.stdout:1/604: readlink d1/d13/lbb 0 2024-03-22T07:16:06.290 INFO:tasks.workunit.client.1.smithi069.stdout:1/605: fdatasync d1/f3b 0 2024-03-22T07:16:06.291 INFO:tasks.workunit.client.1.smithi069.stdout:1/606: symlink d1/d2a/d50/lc0 0 2024-03-22T07:16:06.294 INFO:tasks.workunit.client.1.smithi069.stdout:6/549: dwrite d4/dd/d12/d44/d58/f62 [4194304,4194304] 0 2024-03-22T07:16:06.294 INFO:tasks.workunit.client.1.smithi069.stdout:6/550: write d4/dd/d12/d2b/f9f [677039,124844] 0 2024-03-22T07:16:06.294 INFO:tasks.workunit.client.0.smithi063.stdout:6/897: dwrite d3/d15/da3/dc2/ff9 [0,4194304] 0 2024-03-22T07:16:06.295 INFO:tasks.workunit.client.0.smithi063.stdout:6/898: fdatasync d3/f14 0 2024-03-22T07:16:06.297 INFO:tasks.workunit.client.1.smithi069.stdout:5/519: dwrite d1/d7/f40 [0,4194304] 0 2024-03-22T07:16:06.299 INFO:tasks.workunit.client.0.smithi063.stdout:6/899: symlink d3/d15/d16/d60/d6d/d107/d10b/dde/deb/dfd/l12a 0 2024-03-22T07:16:06.299 INFO:tasks.workunit.client.0.smithi063.stdout:6/900: fsync d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/fa1 0 2024-03-22T07:16:06.299 INFO:tasks.workunit.client.1.smithi069.stdout:6/551: rmdir d4/dd/d12 39 2024-03-22T07:16:06.303 INFO:tasks.workunit.client.1.smithi069.stdout:2/450: dwrite d18/d4d/d3e/f86 [0,4194304] 0 2024-03-22T07:16:06.303 INFO:tasks.workunit.client.1.smithi069.stdout:4/763: dwrite d6/d9b/d1f/d58/d7c/d90/fa5 [0,4194304] 0 2024-03-22T07:16:06.303 INFO:tasks.workunit.client.1.smithi069.stdout:7/623: dwrite d9/db/d1a/fc3 [0,4194304] 0 2024-03-22T07:16:06.304 INFO:tasks.workunit.client.0.smithi063.stdout:9/873: dwrite d8/d19/d20/d22/d29/d31/d53/d6a/da2/df1/dfb/f10f [0,4194304] 0 2024-03-22T07:16:06.305 INFO:tasks.workunit.client.1.smithi069.stdout:5/520: unlink d1/d6/d86/d99/c87 0 2024-03-22T07:16:06.306 INFO:tasks.workunit.client.0.smithi063.stdout:3/959: dwrite da/d16/d89/d8c/dc1/fee [0,4194304] 0 2024-03-22T07:16:06.306 INFO:tasks.workunit.client.0.smithi063.stdout:9/874: mkdir d8/d19/d20/d22/d29/d31/d53/d6a/da2/df1/d110 0 2024-03-22T07:16:06.307 INFO:tasks.workunit.client.1.smithi069.stdout:9/530: dwrite d2/f5 [0,4194304] 0 2024-03-22T07:16:06.307 INFO:tasks.workunit.client.1.smithi069.stdout:9/531: write d2/dd/d3c/d53/d9d/f9b [86147,99262] 0 2024-03-22T07:16:06.309 INFO:tasks.workunit.client.1.smithi069.stdout:9/532: write d2/dd/f69 [4587590,68642] 0 2024-03-22T07:16:06.309 INFO:tasks.workunit.client.1.smithi069.stdout:9/533: write d2/dd/d3c/d2e/fa8 [599699,24443] 0 2024-03-22T07:16:06.310 INFO:tasks.workunit.client.0.smithi063.stdout:9/875: unlink d8/d19/d39/db5/fcd 0 2024-03-22T07:16:06.310 INFO:tasks.workunit.client.0.smithi063.stdout:9/876: write d8/d19/d20/d22/d29/d31/d37/fc5 [4742991,100021] 0 2024-03-22T07:16:06.316 INFO:tasks.workunit.client.0.smithi063.stdout:9/877: creat d8/d19/d20/d22/d29/f111 x:0 0 0 2024-03-22T07:16:06.320 INFO:tasks.workunit.client.1.smithi069.stdout:6/552: mkdir d4/d7/da/d3f/d76/dab 0 2024-03-22T07:16:06.330 INFO:tasks.workunit.client.1.smithi069.stdout:1/607: rmdir d1/d2a 39 2024-03-22T07:16:06.332 INFO:tasks.workunit.client.1.smithi069.stdout:5/521: unlink d1/d10/d44/d50/c6a 0 2024-03-22T07:16:06.335 INFO:tasks.workunit.client.1.smithi069.stdout:6/553: symlink d4/dd/d41/d78/lac 0 2024-03-22T07:16:06.336 INFO:tasks.workunit.client.1.smithi069.stdout:5/522: mkdir d1/d6/d26/d3e/da0/dba 0 2024-03-22T07:16:06.337 INFO:tasks.workunit.client.1.smithi069.stdout:6/554: mknod d4/dd/d12/cad 0 2024-03-22T07:16:06.339 INFO:tasks.workunit.client.1.smithi069.stdout:5/523: getdents d1/d10 0 2024-03-22T07:16:06.341 INFO:tasks.workunit.client.0.smithi063.stdout:6/901: dwrite d3/d6/d35/d3a/ff1 [0,4194304] 0 2024-03-22T07:16:06.341 INFO:tasks.workunit.client.0.smithi063.stdout:6/902: read - d3/d15/d16/d60/d6d/d4c/fd5 zero size 2024-03-22T07:16:06.345 INFO:tasks.workunit.client.1.smithi069.stdout:5/524: mkdir d1/d6/d26/d3e/dbb 0 2024-03-22T07:16:06.345 INFO:tasks.workunit.client.1.smithi069.stdout:5/525: chown d1/d28 1182 1 2024-03-22T07:16:06.346 INFO:tasks.workunit.client.1.smithi069.stdout:4/764: dwrite d6/d12/f73 [0,4194304] 0 2024-03-22T07:16:06.346 INFO:tasks.workunit.client.1.smithi069.stdout:4/765: chown d6/d16/d3a/db4/dfb/d23/d59/fca 47 1 2024-03-22T07:16:06.346 INFO:tasks.workunit.client.1.smithi069.stdout:5/526: mkdir d1/d6/d26/d3e/dbb/dbc 0 2024-03-22T07:16:06.346 INFO:tasks.workunit.client.0.smithi063.stdout:6/903: dread d3/d15/f85 [0,4194304] 0 2024-03-22T07:16:06.349 INFO:tasks.workunit.client.1.smithi069.stdout:4/766: link d6/d16/d3a/db4/dfb/d23/f3c d6/daa/dfd/f107 0 2024-03-22T07:16:06.350 INFO:tasks.workunit.client.0.smithi063.stdout:6/904: rename d3/d15/f22 to d3/d15/d16/d60/d6d/d107/d10b/dde/deb/f12b 0 2024-03-22T07:16:06.350 INFO:tasks.workunit.client.0.smithi063.stdout:6/905: dread - d3/d15/d16/d60/d6d/d4e/d5f/fb7 zero size 2024-03-22T07:16:06.356 INFO:tasks.workunit.client.1.smithi069.stdout:4/767: rmdir d6/dc4/de1 39 2024-03-22T07:16:06.356 INFO:tasks.workunit.client.1.smithi069.stdout:2/451: dwrite d18/d4d/d3e/d40/f76 [0,4194304] 0 2024-03-22T07:16:06.356 INFO:tasks.workunit.client.1.smithi069.stdout:2/452: creat d18/d4d/d3e/d40/f97 x:0 0 0 2024-03-22T07:16:06.356 INFO:tasks.workunit.client.1.smithi069.stdout:7/624: dwrite d9/db/d1a/db1/d57/d44/d53/d74/f8d [0,4194304] 0 2024-03-22T07:16:06.356 INFO:tasks.workunit.client.1.smithi069.stdout:4/768: rename d6/d16/d3a/db4/dfb/d1d/f96 to d6/d16/f108 0 2024-03-22T07:16:06.358 INFO:tasks.workunit.client.0.smithi063.stdout:6/906: rename d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/d10a/f129 to d3/d6/d2a/d38/da5/d32/f12c 0 2024-03-22T07:16:06.358 INFO:tasks.workunit.client.0.smithi063.stdout:6/907: chown d3/d6/d35/d3a/ld3 102 1 2024-03-22T07:16:06.358 INFO:tasks.workunit.client.0.smithi063.stdout:6/908: rename d3/d15/d16/d60/fb3 to d3/f12d 0 2024-03-22T07:16:06.358 INFO:tasks.workunit.client.1.smithi069.stdout:7/625: creat d9/db/d1a/db1/d57/d44/d53/d74/d30/fc8 x:0 0 0 2024-03-22T07:16:06.359 INFO:tasks.workunit.client.1.smithi069.stdout:9/534: dwrite d2/dd/d3c/d53/d9d/f9b [0,4194304] 0 2024-03-22T07:16:06.360 INFO:tasks.workunit.client.1.smithi069.stdout:4/769: rmdir d6/daa/dce 39 2024-03-22T07:16:06.360 INFO:tasks.workunit.client.1.smithi069.stdout:4/770: chown d6/d4b/lae 1 1 2024-03-22T07:16:06.360 INFO:tasks.workunit.client.1.smithi069.stdout:4/771: readlink d6/d12/l32 0 2024-03-22T07:16:06.361 INFO:tasks.workunit.client.0.smithi063.stdout:6/909: mknod d3/d15/d16/d69/c12e 0 2024-03-22T07:16:06.362 INFO:tasks.workunit.client.1.smithi069.stdout:7/626: unlink f8 0 2024-03-22T07:16:06.362 INFO:tasks.workunit.client.0.smithi063.stdout:9/878: dwrite d8/d19/d20/d22/d29/d31/f4e [0,4194304] 0 2024-03-22T07:16:06.363 INFO:tasks.workunit.client.0.smithi063.stdout:6/910: mkdir d3/d6/d2a/d12f 0 2024-03-22T07:16:06.363 INFO:tasks.workunit.client.0.smithi063.stdout:6/911: fdatasync d3/d15/d16/d60/d6d/d4e/d5a/dd4/ffa 0 2024-03-22T07:16:06.363 INFO:tasks.workunit.client.0.smithi063.stdout:6/912: chown d3/d15/d16/d60/d6d/d107/d10b/l70 26288 1 2024-03-22T07:16:06.364 INFO:tasks.workunit.client.1.smithi069.stdout:4/772: truncate d6/d9b/d1f/f39 4871 0 2024-03-22T07:16:06.365 INFO:tasks.workunit.client.1.smithi069.stdout:4/773: truncate d6/d12/d18/d21/f79 4266169 0 2024-03-22T07:16:06.367 INFO:tasks.workunit.client.1.smithi069.stdout:4/774: mknod d6/dc4/de1/c109 0 2024-03-22T07:16:06.377 INFO:tasks.workunit.client.1.smithi069.stdout:9/535: dread d2/d71/d90/f95 [0,4194304] 0 2024-03-22T07:16:06.380 INFO:tasks.workunit.client.1.smithi069.stdout:6/555: dwrite d4/dd/d12/d2b/d72/f95 [0,4194304] 0 2024-03-22T07:16:06.381 INFO:tasks.workunit.client.1.smithi069.stdout:9/536: truncate d2/dd/d3c/f3e 1096787 0 2024-03-22T07:16:06.383 INFO:tasks.workunit.client.1.smithi069.stdout:9/537: unlink d2/dd/d3c/l44 0 2024-03-22T07:16:06.383 INFO:tasks.workunit.client.1.smithi069.stdout:9/538: creat d2/dd/d3c/d2e/fad x:0 0 0 2024-03-22T07:16:06.384 INFO:tasks.workunit.client.1.smithi069.stdout:9/539: chown d2/dd/d3c/d48/d7f/d86/c9a 2291493 1 2024-03-22T07:16:06.389 INFO:tasks.workunit.client.1.smithi069.stdout:1/608: dwrite d1/d2a/d5e/d9b/fb7 [0,4194304] 0 2024-03-22T07:16:06.390 INFO:tasks.workunit.client.1.smithi069.stdout:1/609: chown d1/f48 1311 1 2024-03-22T07:16:06.390 INFO:tasks.workunit.client.1.smithi069.stdout:1/610: chown d1/d13/d36/f3e 198462530 1 2024-03-22T07:16:06.392 INFO:tasks.workunit.client.1.smithi069.stdout:1/611: rename d1/d2a/f57 to d1/d2a/d84/d9a/fc1 0 2024-03-22T07:16:06.392 INFO:tasks.workunit.client.1.smithi069.stdout:1/612: fsync d1/d13/f26 0 2024-03-22T07:16:06.392 INFO:tasks.workunit.client.1.smithi069.stdout:1/613: chown d1/d13/f45 129574 1 2024-03-22T07:16:06.394 INFO:tasks.workunit.client.1.smithi069.stdout:1/614: rename d1/d2a/d5e/da6 to d1/d13/d82/dc2 0 2024-03-22T07:16:06.394 INFO:tasks.workunit.client.1.smithi069.stdout:1/615: write d1/d2a/d50/f40 [2204018,118735] 0 2024-03-22T07:16:06.399 INFO:tasks.workunit.client.1.smithi069.stdout:5/527: dwrite d1/d6/d26/d33/f80 [0,4194304] 0 2024-03-22T07:16:06.400 INFO:tasks.workunit.client.1.smithi069.stdout:5/528: dread - d1/d6/d26/d33/d5d/d72/f9e zero size 2024-03-22T07:16:06.400 INFO:tasks.workunit.client.1.smithi069.stdout:5/529: mkdir d1/d6/d26/d3a/dbd 0 2024-03-22T07:16:06.406 INFO:tasks.workunit.client.1.smithi069.stdout:5/530: write d1/d6/d26/d33/f38 [248818,109140] 0 2024-03-22T07:16:06.406 INFO:tasks.workunit.client.1.smithi069.stdout:5/531: write d1/d6/d26/d33/f4e [577793,31117] 0 2024-03-22T07:16:06.406 INFO:tasks.workunit.client.1.smithi069.stdout:5/532: truncate d1/d7/fa5 762209 0 2024-03-22T07:16:06.406 INFO:tasks.workunit.client.1.smithi069.stdout:5/533: creat d1/d10/d60/fbe x:0 0 0 2024-03-22T07:16:06.411 INFO:tasks.workunit.client.1.smithi069.stdout:7/627: dwrite d9/db/d1a/db1/d57/d44/d53/dc7/f9d [0,4194304] 0 2024-03-22T07:16:06.412 INFO:tasks.workunit.client.1.smithi069.stdout:2/453: dwrite d18/d4d/d3e/d40/f61 [0,4194304] 0 2024-03-22T07:16:06.413 INFO:tasks.workunit.client.1.smithi069.stdout:6/556: dwrite d4/f6b [0,4194304] 0 2024-03-22T07:16:06.413 INFO:tasks.workunit.client.1.smithi069.stdout:1/616: dread d1/d13/d36/d5b/f71 [0,4194304] 0 2024-03-22T07:16:06.414 INFO:tasks.workunit.client.1.smithi069.stdout:6/557: readlink d4/dd/d41/l69 0 2024-03-22T07:16:06.414 INFO:tasks.workunit.client.1.smithi069.stdout:1/617: fdatasync d1/d13/f27 0 2024-03-22T07:16:06.414 INFO:tasks.workunit.client.0.smithi063.stdout:9/879: dwrite d8/d19/d20/d22/d29/f111 [0,4194304] 0 2024-03-22T07:16:06.415 INFO:tasks.workunit.client.1.smithi069.stdout:4/775: dwrite d6/d9b/d1f/d58/f69 [0,4194304] 0 2024-03-22T07:16:06.419 INFO:tasks.workunit.client.0.smithi063.stdout:9/880: rename d8/d19/d20/d22/d56/f71 to d8/d17/d1e/f112 0 2024-03-22T07:16:06.419 INFO:tasks.workunit.client.0.smithi063.stdout:9/881: fsync d8/d19/d20/d22/f4f 0 2024-03-22T07:16:06.419 INFO:tasks.workunit.client.0.smithi063.stdout:9/882: chown d8 513122 1 2024-03-22T07:16:06.422 INFO:tasks.workunit.client.0.smithi063.stdout:9/883: rmdir d8/d19/d20/d22/d29/df0 39 2024-03-22T07:16:06.422 INFO:tasks.workunit.client.1.smithi069.stdout:6/558: mknod d4/d7/d5b/cae 0 2024-03-22T07:16:06.422 INFO:tasks.workunit.client.0.smithi063.stdout:7/725: sync 2024-03-22T07:16:06.423 INFO:tasks.workunit.client.0.smithi063.stdout:4/811: sync 2024-03-22T07:16:06.423 INFO:tasks.workunit.client.0.smithi063.stdout:9/884: symlink d8/d19/d39/df4/l113 0 2024-03-22T07:16:06.424 INFO:tasks.workunit.client.1.smithi069.stdout:1/618: rename d1/d13/d36/f8f to d1/d2a/d50/d88/d41/d64/fc3 0 2024-03-22T07:16:06.425 INFO:tasks.workunit.client.0.smithi063.stdout:7/726: creat d5/da/d28/da7/ded/ff2 x:0 0 0 2024-03-22T07:16:06.425 INFO:tasks.workunit.client.0.smithi063.stdout:2/780: sync 2024-03-22T07:16:06.425 INFO:tasks.workunit.client.0.smithi063.stdout:2/781: dread - d0/de/d12/d20/f9b zero size 2024-03-22T07:16:06.427 INFO:tasks.workunit.client.0.smithi063.stdout:4/812: unlink d1/d5/dc/d95/d59/d9c/dac/dc9/cd8 0 2024-03-22T07:16:06.427 INFO:tasks.workunit.client.0.smithi063.stdout:9/885: rmdir d8/d19/d20/d22/d29/d31/d53/d6a 39 2024-03-22T07:16:06.428 INFO:tasks.workunit.client.0.smithi063.stdout:9/886: stat d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/da5/dee/c103 0 2024-03-22T07:16:06.428 INFO:tasks.workunit.client.0.smithi063.stdout:9/887: stat d8/d19/d20/d22/d29/d31/d37/fc5 0 2024-03-22T07:16:06.429 INFO:tasks.workunit.client.0.smithi063.stdout:7/727: getdents d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29 0 2024-03-22T07:16:06.429 INFO:tasks.workunit.client.0.smithi063.stdout:7/728: chown d5/d16/c24 0 1 2024-03-22T07:16:06.429 INFO:tasks.workunit.client.0.smithi063.stdout:7/729: chown d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/f5d 1121 1 2024-03-22T07:16:06.429 INFO:tasks.workunit.client.0.smithi063.stdout:7/730: fdatasync d5/fe6 0 2024-03-22T07:16:06.429 INFO:tasks.workunit.client.0.smithi063.stdout:2/782: symlink d0/de/d1b/d63/dd6/l102 0 2024-03-22T07:16:06.430 INFO:tasks.workunit.client.0.smithi063.stdout:2/783: chown d0/de/d12/d5a/f9d 12034 1 2024-03-22T07:16:06.431 INFO:tasks.workunit.client.0.smithi063.stdout:4/813: mkdir d1/d5/d115 0 2024-03-22T07:16:06.431 INFO:tasks.workunit.client.0.smithi063.stdout:4/814: stat d1/d5/dc/c6d 0 2024-03-22T07:16:06.432 INFO:tasks.workunit.client.0.smithi063.stdout:9/888: link d8/l4c d8/d19/de2/l114 0 2024-03-22T07:16:06.432 INFO:tasks.workunit.client.0.smithi063.stdout:2/784: symlink d0/de/d1b/d63/dc6/l103 0 2024-03-22T07:16:06.438 INFO:tasks.workunit.client.1.smithi069.stdout:4/776: symlink d6/d16/d3a/db4/dbc/dc6/l10a 0 2024-03-22T07:16:06.440 INFO:tasks.workunit.client.1.smithi069.stdout:6/559: symlink d4/d7/da/d3f/d76/dab/laf 0 2024-03-22T07:16:06.442 INFO:tasks.workunit.client.1.smithi069.stdout:1/619: rmdir d1/d2a 39 2024-03-22T07:16:06.442 INFO:tasks.workunit.client.1.smithi069.stdout:1/620: stat d1/d13/d82/dc2/faf 0 2024-03-22T07:16:06.443 INFO:tasks.workunit.client.0.smithi063.stdout:4/815: dread d1/d5/dc/d95/d59/faf [0,4194304] 0 2024-03-22T07:16:06.445 INFO:tasks.workunit.client.1.smithi069.stdout:2/454: dwrite d18/d4d/d27/d33/d3d/f54 [0,4194304] 0 2024-03-22T07:16:06.446 INFO:tasks.workunit.client.1.smithi069.stdout:6/560: getdents d4/dd/d12/d2b 0 2024-03-22T07:16:06.446 INFO:tasks.workunit.client.1.smithi069.stdout:7/628: dwrite d9/db/d1a/f2c [0,4194304] 0 2024-03-22T07:16:06.446 INFO:tasks.workunit.client.1.smithi069.stdout:7/629: fsync d9/db/d1a/db1/d57/d44/d53/dc7/db5/fbb 0 2024-03-22T07:16:06.447 INFO:tasks.workunit.client.1.smithi069.stdout:1/621: unlink d1/d2a/d84/d9a/lb9 0 2024-03-22T07:16:06.448 INFO:tasks.workunit.client.0.smithi063.stdout:7/731: write d5/f89 [3716634,111237] 0 2024-03-22T07:16:06.449 INFO:tasks.workunit.client.1.smithi069.stdout:5/534: dwrite d1/d6/d86/d99/f58 [4194304,4194304] 0 2024-03-22T07:16:06.449 INFO:tasks.workunit.client.1.smithi069.stdout:5/535: fsync d1/d7/f40 0 2024-03-22T07:16:06.450 INFO:tasks.workunit.client.1.smithi069.stdout:2/455: write d18/d4d/d34/f53 [816802,84235] 0 2024-03-22T07:16:06.451 INFO:tasks.workunit.client.0.smithi063.stdout:9/889: write d8/d15/d2f/f93 [861924,552] 0 2024-03-22T07:16:06.451 INFO:tasks.workunit.client.1.smithi069.stdout:7/630: mkdir d9/d42/dc9 0 2024-03-22T07:16:06.452 INFO:tasks.workunit.client.1.smithi069.stdout:4/777: dread d6/d12/d18/d21/f2f [4194304,4194304] 0 2024-03-22T07:16:06.452 INFO:tasks.workunit.client.1.smithi069.stdout:1/622: link d1/d2a/d50/d4c/d55/f56 d1/d2a/d5e/fc4 0 2024-03-22T07:16:06.454 INFO:tasks.workunit.client.1.smithi069.stdout:5/536: symlink d1/d6/d21/dae/lbf 0 2024-03-22T07:16:06.455 INFO:tasks.workunit.client.1.smithi069.stdout:2/456: truncate d18/f70 314864 0 2024-03-22T07:16:06.456 INFO:tasks.workunit.client.1.smithi069.stdout:7/631: symlink d9/db/d1a/db1/lca 0 2024-03-22T07:16:06.456 INFO:tasks.workunit.client.1.smithi069.stdout:7/632: chown d9/d42/dc9 114841 1 2024-03-22T07:16:06.457 INFO:tasks.workunit.client.1.smithi069.stdout:1/623: dread d1/d13/f45 [0,4194304] 0 2024-03-22T07:16:06.457 INFO:tasks.workunit.client.1.smithi069.stdout:4/778: link d6/d12/f33 d6/dc4/de1/f10b 0 2024-03-22T07:16:06.459 INFO:tasks.workunit.client.1.smithi069.stdout:2/457: getdents d18/d4d/d25 0 2024-03-22T07:16:06.459 INFO:tasks.workunit.client.1.smithi069.stdout:4/779: getdents d6/d16/d3a/db4/dfb/d23/d59 0 2024-03-22T07:16:06.460 INFO:tasks.workunit.client.1.smithi069.stdout:2/458: creat d18/d91/f98 x:0 0 0 2024-03-22T07:16:06.461 INFO:tasks.workunit.client.1.smithi069.stdout:7/633: symlink d9/db/d1a/db1/d57/d44/d53/lcb 0 2024-03-22T07:16:06.463 INFO:tasks.workunit.client.0.smithi063.stdout:2/785: dwrite d0/de/d74/fee [0,4194304] 0 2024-03-22T07:16:06.465 INFO:tasks.workunit.client.1.smithi069.stdout:6/561: write d4/d7/d5b/f70 [988858,103732] 0 2024-03-22T07:16:06.466 INFO:tasks.workunit.client.1.smithi069.stdout:2/459: creat d18/d4d/d27/d33/d60/d64/f99 x:0 0 0 2024-03-22T07:16:06.466 INFO:tasks.workunit.client.1.smithi069.stdout:7/634: truncate d9/db/f17 519853 0 2024-03-22T07:16:06.467 INFO:tasks.workunit.client.1.smithi069.stdout:7/635: chown d9/db/d1a/c22 0 1 2024-03-22T07:16:06.469 INFO:tasks.workunit.client.1.smithi069.stdout:5/537: dread d1/d6/d26/d33/f4c [0,4194304] 0 2024-03-22T07:16:06.469 INFO:tasks.workunit.client.1.smithi069.stdout:1/624: write d1/d2a/d50/d88/d41/d64/f7e [3673384,3880] 0 2024-03-22T07:16:06.469 INFO:tasks.workunit.client.1.smithi069.stdout:1/625: readlink d1/d13/l49 0 2024-03-22T07:16:06.471 INFO:tasks.workunit.client.1.smithi069.stdout:1/626: fdatasync d1/d2a/d50/f40 0 2024-03-22T07:16:06.471 INFO:tasks.workunit.client.1.smithi069.stdout:6/562: creat d4/dd/d12/d13/fb0 x:0 0 0 2024-03-22T07:16:06.474 INFO:tasks.workunit.client.1.smithi069.stdout:4/780: write d6/d16/d3a/db4/dfb/d23/f3c [3575687,36948] 0 2024-03-22T07:16:06.474 INFO:tasks.workunit.client.1.smithi069.stdout:4/781: truncate d6/d9b/f86 145825 0 2024-03-22T07:16:06.474 INFO:tasks.workunit.client.0.smithi063.stdout:2/786: dread d0/de/d12/f1f [0,4194304] 0 2024-03-22T07:16:06.477 INFO:tasks.workunit.client.0.smithi063.stdout:2/787: mkdir d0/de/d58/d7f/d97/da2/d104 0 2024-03-22T07:16:06.478 INFO:tasks.workunit.client.1.smithi069.stdout:8/816: sync 2024-03-22T07:16:06.478 INFO:tasks.workunit.client.1.smithi069.stdout:3/680: sync 2024-03-22T07:16:06.478 INFO:tasks.workunit.client.1.smithi069.stdout:0/756: sync 2024-03-22T07:16:06.478 INFO:tasks.workunit.client.1.smithi069.stdout:0/757: chown d3/d7/f12 302518 1 2024-03-22T07:16:06.478 INFO:tasks.workunit.client.1.smithi069.stdout:0/758: write d3/d7/d15/dad/fae [967617,28153] 0 2024-03-22T07:16:06.479 INFO:tasks.workunit.client.1.smithi069.stdout:7/636: creat d9/db/d1a/db1/d57/d44/d53/dc7/fcc x:0 0 0 2024-03-22T07:16:06.480 INFO:tasks.workunit.client.0.smithi063.stdout:2/788: unlink d0/de/d12/d5a/c93 0 2024-03-22T07:16:06.480 INFO:tasks.workunit.client.0.smithi063.stdout:2/789: truncate d0/de/d74/dca/fd2 1751803 0 2024-03-22T07:16:06.480 INFO:tasks.workunit.client.1.smithi069.stdout:2/460: creat d18/d4d/f9a x:0 0 0 2024-03-22T07:16:06.481 INFO:tasks.workunit.client.1.smithi069.stdout:5/538: dread d1/d7/f40 [0,4194304] 0 2024-03-22T07:16:06.481 INFO:tasks.workunit.client.0.smithi063.stdout:2/790: symlink d0/de/d12/dd1/l105 0 2024-03-22T07:16:06.481 INFO:tasks.workunit.client.0.smithi063.stdout:2/791: dread - d0/de/d35/d36/f8f zero size 2024-03-22T07:16:06.482 INFO:tasks.workunit.client.0.smithi063.stdout:2/792: mkdir d0/de/d35/d106 0 2024-03-22T07:16:06.484 INFO:tasks.workunit.client.0.smithi063.stdout:2/793: rename d0/de/d58/f91 to d0/de/d1b/d8b/f107 0 2024-03-22T07:16:06.484 INFO:tasks.workunit.client.0.smithi063.stdout:2/794: fsync d0/de/f14 0 2024-03-22T07:16:06.484 INFO:tasks.workunit.client.1.smithi069.stdout:4/782: rename d6/d16/d83/c89 to d6/d12/c10c 0 2024-03-22T07:16:06.486 INFO:tasks.workunit.client.0.smithi063.stdout:2/795: mknod d0/de/d1b/d63/d6e/ddb/c108 0 2024-03-22T07:16:06.486 INFO:tasks.workunit.client.0.smithi063.stdout:2/796: fdatasync d0/de/d58/d7f/d97/fef 0 2024-03-22T07:16:06.486 INFO:tasks.workunit.client.0.smithi063.stdout:2/797: write d0/de/d12/d5a/f101 [458264,70926] 0 2024-03-22T07:16:06.489 INFO:tasks.workunit.client.1.smithi069.stdout:3/681: creat d5/d9/d26/d3f/d56/d7f/d90/fdf x:0 0 0 2024-03-22T07:16:06.489 INFO:tasks.workunit.client.1.smithi069.stdout:3/682: write d5/d9/d26/d3f/d35/f9e [410167,36266] 0 2024-03-22T07:16:06.493 INFO:tasks.workunit.client.0.smithi063.stdout:9/890: dwrite d8/d19/d20/d22/d9b/f9e [0,4194304] 0 2024-03-22T07:16:06.493 INFO:tasks.workunit.client.1.smithi069.stdout:8/817: dread d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/f76 [0,4194304] 0 2024-03-22T07:16:06.494 INFO:tasks.workunit.client.1.smithi069.stdout:7/637: link d9/db/d1a/db1/d57/d44/d53/d2e/f4d d9/db/d1a/db1/db3/fcd 0 2024-03-22T07:16:06.499 INFO:tasks.workunit.client.1.smithi069.stdout:2/461: creat d18/f9b x:0 0 0 2024-03-22T07:16:06.499 INFO:tasks.workunit.client.1.smithi069.stdout:2/462: creat d18/d4d/d27/d33/f9c x:0 0 0 2024-03-22T07:16:06.499 INFO:tasks.workunit.client.1.smithi069.stdout:2/463: write d18/d4d/d27/f8c [626103,80187] 0 2024-03-22T07:16:06.501 INFO:tasks.workunit.client.1.smithi069.stdout:5/539: symlink d1/lc0 0 2024-03-22T07:16:06.505 INFO:tasks.workunit.client.1.smithi069.stdout:7/638: mkdir d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/dce 0 2024-03-22T07:16:06.509 INFO:tasks.workunit.client.1.smithi069.stdout:2/464: creat d18/d4d/d27/d33/d60/d64/d89/d8b/f9d x:0 0 0 2024-03-22T07:16:06.512 INFO:tasks.workunit.client.1.smithi069.stdout:7/639: rename d9/db/d1a/db1/d57/d75/d7a/d96/da0 to d9/db/d1a/db1/d57/d75/d7a/dcf 0 2024-03-22T07:16:06.512 INFO:tasks.workunit.client.1.smithi069.stdout:7/640: dread - d9/db/d1a/f82 zero size 2024-03-22T07:16:06.512 INFO:tasks.workunit.client.1.smithi069.stdout:7/641: chown d9/db/d1a/fc3 937608502 1 2024-03-22T07:16:06.516 INFO:tasks.workunit.client.1.smithi069.stdout:8/818: write d1/d1c/d3e/f32 [1695915,70342] 0 2024-03-22T07:16:06.516 INFO:tasks.workunit.client.1.smithi069.stdout:8/819: write d1/d93/ddf/fe9 [1585638,122166] 0 2024-03-22T07:16:06.518 INFO:tasks.workunit.client.1.smithi069.stdout:7/642: rmdir d9/d42/dc9 0 2024-03-22T07:16:06.521 INFO:tasks.workunit.client.1.smithi069.stdout:8/820: rmdir d1/d1c/d3e/d88/da8 39 2024-03-22T07:16:06.522 INFO:tasks.workunit.client.1.smithi069.stdout:7/643: creat d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/dce/fd0 x:0 0 0 2024-03-22T07:16:06.522 INFO:tasks.workunit.client.1.smithi069.stdout:7/644: write d9/db/d1a/db1/d57/d44/d53/dc7/fcc [596666,5552] 0 2024-03-22T07:16:06.525 INFO:tasks.workunit.client.1.smithi069.stdout:7/645: link d9/db/d1a/db1/d57/c58 d9/db/d1a/db1/d57/d44/d53/d74/d30/cd1 0 2024-03-22T07:16:06.526 INFO:tasks.workunit.client.1.smithi069.stdout:7/646: link d9/db/d1a/db1/d57/d75/d7a/d96/la5 d9/db/d48/d4b/ld2 0 2024-03-22T07:16:06.539 INFO:tasks.workunit.client.1.smithi069.stdout:2/465: write d18/f19 [1491663,69946] 0 2024-03-22T07:16:06.539 INFO:tasks.workunit.client.1.smithi069.stdout:2/466: read d18/d4d/f2d [1494030,706] 0 2024-03-22T07:16:06.539 INFO:tasks.workunit.client.1.smithi069.stdout:2/467: write d18/d4d/d3e/d40/f51 [688848,16918] 0 2024-03-22T07:16:06.541 INFO:tasks.workunit.client.1.smithi069.stdout:1/627: dwrite d1/d2a/d5e/fc4 [0,4194304] 0 2024-03-22T07:16:06.545 INFO:tasks.workunit.client.1.smithi069.stdout:1/628: dread d1/d2a/d84/fa7 [0,4194304] 0 2024-03-22T07:16:06.552 INFO:tasks.workunit.client.1.smithi069.stdout:0/759: dwrite d3/d7/d15/d29/d6e/f7a [0,4194304] 0 2024-03-22T07:16:06.552 INFO:tasks.workunit.client.1.smithi069.stdout:0/760: fdatasync d3/d7/d15/d29/d5a/f6c 0 2024-03-22T07:16:06.552 INFO:tasks.workunit.client.0.smithi063.stdout:2/798: dwrite d0/de/d1b/d63/f6c [4194304,4194304] 0 2024-03-22T07:16:06.552 INFO:tasks.workunit.client.1.smithi069.stdout:4/783: dwrite d6/daa/fe8 [0,4194304] 0 2024-03-22T07:16:06.553 INFO:tasks.workunit.client.0.smithi063.stdout:2/799: readlink d0/de/d1b/ldf 0 2024-03-22T07:16:06.555 INFO:tasks.workunit.client.1.smithi069.stdout:3/683: dwrite d5/d9/d26/d36/d41/f72 [0,4194304] 0 2024-03-22T07:16:06.557 INFO:tasks.workunit.client.0.smithi063.stdout:2/800: rename d0/de/d12/d20/fc7 to d0/de/d58/dbb/f109 0 2024-03-22T07:16:06.557 INFO:tasks.workunit.client.0.smithi063.stdout:2/801: fsync d0/de/d12/d20/f29 0 2024-03-22T07:16:06.557 INFO:tasks.workunit.client.0.smithi063.stdout:2/802: readlink d0/de/d1b/d63/dc6/l103 0 2024-03-22T07:16:06.557 INFO:tasks.workunit.client.0.smithi063.stdout:7/732: dwrite d5/d38/f81 [0,4194304] 0 2024-03-22T07:16:06.557 INFO:tasks.workunit.client.0.smithi063.stdout:9/891: dwrite d8/d15/f1c [0,4194304] 0 2024-03-22T07:16:06.557 INFO:tasks.workunit.client.0.smithi063.stdout:9/892: chown d8/d19/d20/c18 0 1 2024-03-22T07:16:06.557 INFO:tasks.workunit.client.0.smithi063.stdout:2/803: creat d0/de/d58/d7f/d97/f10a x:0 0 0 2024-03-22T07:16:06.557 INFO:tasks.workunit.client.0.smithi063.stdout:2/804: read d0/f13 [657523,29160] 0 2024-03-22T07:16:06.557 INFO:tasks.workunit.client.0.smithi063.stdout:2/805: stat d0/de/d58/d7f/d97/da2/lf2 0 2024-03-22T07:16:06.557 INFO:tasks.workunit.client.0.smithi063.stdout:2/806: fdatasync d0/de/d1b/d32/dac/dc2/fc8 0 2024-03-22T07:16:06.559 INFO:tasks.workunit.client.0.smithi063.stdout:7/733: creat d5/dcf/ff3 x:0 0 0 2024-03-22T07:16:06.560 INFO:tasks.workunit.client.1.smithi069.stdout:8/821: fsync d1/d1c/d3e/db9/ff9 0 2024-03-22T07:16:06.560 INFO:tasks.workunit.client.1.smithi069.stdout:0/761: rename d3/d7/d15/d29/d5a/d84/fa0 to d3/d7/d15/d29/d5a/d5d/ffe 0 2024-03-22T07:16:06.560 INFO:tasks.workunit.client.1.smithi069.stdout:5/540: dwrite d1/d6/d86/f77 [0,4194304] 0 2024-03-22T07:16:06.560 INFO:tasks.workunit.client.1.smithi069.stdout:5/541: creat d1/d10/d60/fc1 x:0 0 0 2024-03-22T07:16:06.560 INFO:tasks.workunit.client.1.smithi069.stdout:7/647: dwrite d9/db/d1a/db1/d57/d44/d53/f43 [0,4194304] 0 2024-03-22T07:16:06.560 INFO:tasks.workunit.client.1.smithi069.stdout:3/684: dread d5/d9/db/fbb [0,4194304] 0 2024-03-22T07:16:06.560 INFO:tasks.workunit.client.1.smithi069.stdout:3/685: write d5/d9/db/dae/fd4 [369539,34369] 0 2024-03-22T07:16:06.560 INFO:tasks.workunit.client.1.smithi069.stdout:3/686: chown d5/d9/l1d 27553885 1 2024-03-22T07:16:06.561 INFO:tasks.workunit.client.1.smithi069.stdout:4/784: mkdir d6/d16/d83/dd0/d10d 0 2024-03-22T07:16:06.562 INFO:tasks.workunit.client.0.smithi063.stdout:9/893: creat d8/d19/d20/d22/d29/d31/f115 x:0 0 0 2024-03-22T07:16:06.563 INFO:tasks.workunit.client.1.smithi069.stdout:2/468: fdatasync d18/f19 0 2024-03-22T07:16:06.564 INFO:tasks.workunit.client.0.smithi063.stdout:9/894: mknod d8/d19/d20/d22/d29/d31/d53/d6a/da2/c116 0 2024-03-22T07:16:06.565 INFO:tasks.workunit.client.0.smithi063.stdout:2/807: link d0/de/d1b/d63/fe0 d0/de/d1b/d32/dac/f10b 0 2024-03-22T07:16:06.567 INFO:tasks.workunit.client.0.smithi063.stdout:9/895: unlink d8/d19/d20/d22/d5d/f64 0 2024-03-22T07:16:06.567 INFO:tasks.workunit.client.0.smithi063.stdout:9/896: stat d8/d19/d20/d22/d29 0 2024-03-22T07:16:06.567 INFO:tasks.workunit.client.0.smithi063.stdout:2/808: mknod d0/de/d1b/d63/dc6/c10c 0 2024-03-22T07:16:06.568 INFO:tasks.workunit.client.1.smithi069.stdout:3/687: dread d5/d9/d26/d3f/d35/f37 [0,4194304] 0 2024-03-22T07:16:06.568 INFO:tasks.workunit.client.1.smithi069.stdout:3/688: chown d5/d9/d23/c80 489958118 1 2024-03-22T07:16:06.569 INFO:tasks.workunit.client.0.smithi063.stdout:2/809: link d0/de/d35/d36/ff6 d0/de/d1b/d8b/f10d 0 2024-03-22T07:16:06.573 INFO:tasks.workunit.client.1.smithi069.stdout:5/542: dread d1/d28/f4d [0,4194304] 0 2024-03-22T07:16:06.573 INFO:tasks.workunit.client.1.smithi069.stdout:5/543: stat d1/d6/d26/d33/d5d/d72/d78 0 2024-03-22T07:16:06.573 INFO:tasks.workunit.client.1.smithi069.stdout:8/822: mkdir d1/d1c/d3e/d88/dcd/dbd/d106 0 2024-03-22T07:16:06.573 INFO:tasks.workunit.client.1.smithi069.stdout:8/823: stat d1/d1c/d3e/d88/dcd/d6c/f92 0 2024-03-22T07:16:06.578 INFO:tasks.workunit.client.1.smithi069.stdout:5/544: read d1/d6/d26/d33/f4c [2944631,17540] 0 2024-03-22T07:16:06.578 INFO:tasks.workunit.client.1.smithi069.stdout:0/762: creat d3/d7/d15/d29/dcf/de9/fff x:0 0 0 2024-03-22T07:16:06.581 INFO:tasks.workunit.client.1.smithi069.stdout:7/648: creat d9/db/d48/dab/fd3 x:0 0 0 2024-03-22T07:16:06.582 INFO:tasks.workunit.client.0.smithi063.stdout:2/810: dread d0/de/d1b/d8b/fb7 [0,4194304] 0 2024-03-22T07:16:06.582 INFO:tasks.workunit.client.0.smithi063.stdout:7/734: dwrite d5/da/d28/da7/f72 [0,4194304] 0 2024-03-22T07:16:06.583 INFO:tasks.workunit.client.0.smithi063.stdout:2/811: write d0/de/d12/dd1/fdd [845407,128566] 0 2024-03-22T07:16:06.583 INFO:tasks.workunit.client.0.smithi063.stdout:9/897: read d8/d19/d20/d22/d29/d31/f51 [927464,30191] 0 2024-03-22T07:16:06.583 INFO:tasks.workunit.client.1.smithi069.stdout:4/785: creat d6/d16/d68/df8/f10e x:0 0 0 2024-03-22T07:16:06.583 INFO:tasks.workunit.client.0.smithi063.stdout:9/898: dread d8/d19/d20/d22/fd6 [0,4194304] 0 2024-03-22T07:16:06.584 INFO:tasks.workunit.client.1.smithi069.stdout:3/689: symlink d5/d9/d26/d3f/d54/d9d/dde/le0 0 2024-03-22T07:16:06.585 INFO:tasks.workunit.client.0.smithi063.stdout:7/735: truncate d5/da/f25 15000 0 2024-03-22T07:16:06.585 INFO:tasks.workunit.client.0.smithi063.stdout:7/736: write d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/d4a/f7e [136525,50334] 0 2024-03-22T07:16:06.587 INFO:tasks.workunit.client.0.smithi063.stdout:2/812: creat d0/de/d1b/d32/dfb/f10e x:0 0 0 2024-03-22T07:16:06.589 INFO:tasks.workunit.client.1.smithi069.stdout:1/629: dwrite d1/d13/d82/dc2/faf [0,4194304] 0 2024-03-22T07:16:06.589 INFO:tasks.workunit.client.0.smithi063.stdout:9/899: link d8/d19/d20/d22/d29/d31/d53/d6a/da2/f80 d8/d19/d20/d22/db2/f117 0 2024-03-22T07:16:06.590 INFO:tasks.workunit.client.1.smithi069.stdout:0/763: unlink d3/d7/d15/d29/d6e/fec 0 2024-03-22T07:16:06.591 INFO:tasks.workunit.client.0.smithi063.stdout:9/900: creat d8/d19/d20/d22/d29/d31/d37/f118 x:0 0 0 2024-03-22T07:16:06.591 INFO:tasks.workunit.client.0.smithi063.stdout:9/901: stat d8/d19/d20/d22/d29/c44 0 2024-03-22T07:16:06.591 INFO:tasks.workunit.client.0.smithi063.stdout:8/943: sync 2024-03-22T07:16:06.592 INFO:tasks.workunit.client.1.smithi069.stdout:5/545: mkdir d1/d6/d26/d33/d5d/dc2 0 2024-03-22T07:16:06.592 INFO:tasks.workunit.client.1.smithi069.stdout:5/546: write d1/d10/d60/f7b [891987,110159] 0 2024-03-22T07:16:06.592 INFO:tasks.workunit.client.1.smithi069.stdout:5/547: read - d1/d6/fa9 zero size 2024-03-22T07:16:06.592 INFO:tasks.workunit.client.0.smithi063.stdout:8/944: creat d0/d1b/d28/d5f/f13d x:0 0 0 2024-03-22T07:16:06.593 INFO:tasks.workunit.client.0.smithi063.stdout:8/945: chown d0/d1b/d28/d3e/db2/dbd/dfd/c123 3265 1 2024-03-22T07:16:06.595 INFO:tasks.workunit.client.1.smithi069.stdout:2/469: dwrite d18/f2b [4194304,4194304] 0 2024-03-22T07:16:06.595 INFO:tasks.workunit.client.1.smithi069.stdout:2/470: dread - d18/d4d/d27/d33/f9c zero size 2024-03-22T07:16:06.596 INFO:tasks.workunit.client.1.smithi069.stdout:2/471: write d18/d4d/d34/f5f [2795021,27323] 0 2024-03-22T07:16:06.596 INFO:tasks.workunit.client.1.smithi069.stdout:4/786: mknod d6/d16/d83/dd0/c10f 0 2024-03-22T07:16:06.597 INFO:tasks.workunit.client.1.smithi069.stdout:3/690: creat d5/d9/d26/d36/d41/fe1 x:0 0 0 2024-03-22T07:16:06.599 INFO:tasks.workunit.client.0.smithi063.stdout:2/813: dread d0/de/d12/f27 [0,4194304] 0 2024-03-22T07:16:06.601 INFO:tasks.workunit.client.0.smithi063.stdout:2/814: creat d0/de/d58/f10f x:0 0 0 2024-03-22T07:16:06.602 INFO:tasks.workunit.client.0.smithi063.stdout:9/902: dread d8/d19/d39/f8b [0,4194304] 0 2024-03-22T07:16:06.603 INFO:tasks.workunit.client.0.smithi063.stdout:9/903: symlink d8/d19/l119 0 2024-03-22T07:16:06.603 INFO:tasks.workunit.client.0.smithi063.stdout:9/904: chown d8/d19/d20/d22/d29/d31/d53/d6a/da2/df1/d110 2614 1 2024-03-22T07:16:06.603 INFO:tasks.workunit.client.0.smithi063.stdout:9/905: truncate d8/d19/d20/d22/fd4 715535 0 2024-03-22T07:16:06.605 INFO:tasks.workunit.client.0.smithi063.stdout:8/946: write d0/d1b/d28/d7b/d45/f46 [1759427,93278] 0 2024-03-22T07:16:06.605 INFO:tasks.workunit.client.1.smithi069.stdout:3/691: write d5/d9/d26/f4c [1384915,84435] 0 2024-03-22T07:16:06.605 INFO:tasks.workunit.client.0.smithi063.stdout:8/947: mknod d0/d26/d2d/d59/c13e 0 2024-03-22T07:16:06.607 INFO:tasks.workunit.client.1.smithi069.stdout:3/692: dread d5/d9/d26/f8b [0,4194304] 0 2024-03-22T07:16:06.608 INFO:tasks.workunit.client.1.smithi069.stdout:3/693: fdatasync d5/d9/d26/d36/fc8 0 2024-03-22T07:16:06.610 INFO:tasks.workunit.client.0.smithi063.stdout:2/815: write d0/de/d74/f7e [6718015,62369] 0 2024-03-22T07:16:06.611 INFO:tasks.workunit.client.1.smithi069.stdout:1/630: rename d1/d2a/d50/f40 to d1/d2a/d50/d88/d41/fc5 0 2024-03-22T07:16:06.612 INFO:tasks.workunit.client.0.smithi063.stdout:9/906: read d8/d19/d20/d22/f65 [1035285,22973] 0 2024-03-22T07:16:06.612 INFO:tasks.workunit.client.0.smithi063.stdout:2/816: truncate d0/de/d35/d36/fc4 2427808 0 2024-03-22T07:16:06.612 INFO:tasks.workunit.client.0.smithi063.stdout:2/817: readlink d0/de/d58/dbb/ld9 0 2024-03-22T07:16:06.612 INFO:tasks.workunit.client.0.smithi063.stdout:2/818: dread - d0/de/d12/d5a/fa6 zero size 2024-03-22T07:16:06.612 INFO:tasks.workunit.client.0.smithi063.stdout:2/819: chown d0/f23 9 1 2024-03-22T07:16:06.613 INFO:tasks.workunit.client.1.smithi069.stdout:0/764: mkdir d3/d7/d15/d29/d5a/d5d/db4/d100 0 2024-03-22T07:16:06.613 INFO:tasks.workunit.client.1.smithi069.stdout:0/765: stat d3/fd9 0 2024-03-22T07:16:06.613 INFO:tasks.workunit.client.1.smithi069.stdout:0/766: write d3/d7/d90/fe0 [823998,8058] 0 2024-03-22T07:16:06.613 INFO:tasks.workunit.client.0.smithi063.stdout:9/907: rmdir d8/d19/d20/d22/d5d 39 2024-03-22T07:16:06.614 INFO:tasks.workunit.client.0.smithi063.stdout:9/908: mkdir d8/d19/d20/d22/d29/d11a 0 2024-03-22T07:16:06.615 INFO:tasks.workunit.client.1.smithi069.stdout:5/548: mkdir d1/d6/d26/d3e/d9c/d9d/dc3 0 2024-03-22T07:16:06.617 INFO:tasks.workunit.client.1.smithi069.stdout:7/649: symlink d9/db/d1a/db1/d57/d44/d53/ld4 0 2024-03-22T07:16:06.617 INFO:tasks.workunit.client.1.smithi069.stdout:7/650: write d9/db/d1a/db1/d57/d44/d53/f6d [480526,79256] 0 2024-03-22T07:16:06.617 INFO:tasks.workunit.client.1.smithi069.stdout:7/651: write d9/f6f [6929569,100156] 0 2024-03-22T07:16:06.617 INFO:tasks.workunit.client.1.smithi069.stdout:7/652: fdatasync d9/db/d1a/db1/d57/d44/d53/f60 0 2024-03-22T07:16:06.618 INFO:tasks.workunit.client.1.smithi069.stdout:8/824: rmdir d1/d1c/d3e 39 2024-03-22T07:16:06.619 INFO:tasks.workunit.client.0.smithi063.stdout:9/909: truncate d8/d19/d20/d22/d29/d31/d53/d6a/da2/df1/dfb/f10f 3115349 0 2024-03-22T07:16:06.619 INFO:tasks.workunit.client.0.smithi063.stdout:9/910: truncate d8/d19/d20/d22/d29/df0/ffa 769835 0 2024-03-22T07:16:06.619 INFO:tasks.workunit.client.0.smithi063.stdout:9/911: stat d8/d17/d1e/fd9 0 2024-03-22T07:16:06.620 INFO:tasks.workunit.client.1.smithi069.stdout:2/472: symlink d18/d4d/d27/d33/d4f/l9e 0 2024-03-22T07:16:06.621 INFO:tasks.workunit.client.0.smithi063.stdout:9/912: unlink d8/d17/d1e/f5c 0 2024-03-22T07:16:06.621 INFO:tasks.workunit.client.0.smithi063.stdout:9/913: fsync d8/d19/d2b/f89 0 2024-03-22T07:16:06.626 INFO:tasks.workunit.client.1.smithi069.stdout:5/549: dread d1/f9 [4194304,4194304] 0 2024-03-22T07:16:06.629 INFO:tasks.workunit.client.1.smithi069.stdout:3/694: getdents d5/d9/d26/d3f/d56/d8a 0 2024-03-22T07:16:06.632 INFO:tasks.workunit.client.1.smithi069.stdout:4/787: rename d6/d9b/d1f/d3f/d51/db8/cc2 to d6/d12/d5d/c110 0 2024-03-22T07:16:06.632 INFO:tasks.workunit.client.1.smithi069.stdout:4/788: truncate d6/fbb 1106245 0 2024-03-22T07:16:06.633 INFO:tasks.workunit.client.0.smithi063.stdout:9/914: dread d8/d19/d20/d22/d29/faa [0,4194304] 0 2024-03-22T07:16:06.634 INFO:tasks.workunit.client.0.smithi063.stdout:9/915: mkdir d8/d15/d11b 0 2024-03-22T07:16:06.634 INFO:tasks.workunit.client.1.smithi069.stdout:7/653: creat d9/db/d48/d4b/fd5 x:0 0 0 2024-03-22T07:16:06.636 INFO:tasks.workunit.client.1.smithi069.stdout:2/473: creat d18/d4d/d27/d77/f9f x:0 0 0 2024-03-22T07:16:06.636 INFO:tasks.workunit.client.1.smithi069.stdout:5/550: rmdir d1/d6/d26/d33/d5d/dc2 0 2024-03-22T07:16:06.636 INFO:tasks.workunit.client.1.smithi069.stdout:5/551: chown d1/d7/cb 199170541 1 2024-03-22T07:16:06.637 INFO:tasks.workunit.client.1.smithi069.stdout:3/695: mknod d5/d9/d26/d36/d41/ce2 0 2024-03-22T07:16:06.640 INFO:tasks.workunit.client.1.smithi069.stdout:0/767: rename d3/d7/d15/d29/d5a/d5d/db4/ded/lf6 to d3/d7/d90/l101 0 2024-03-22T07:16:06.640 INFO:tasks.workunit.client.0.smithi063.stdout:2/820: dwrite d0/de/d1b/d32/ff3 [0,4194304] 0 2024-03-22T07:16:06.642 INFO:tasks.workunit.client.1.smithi069.stdout:1/631: rmdir d1/d13/d36/d5b/d95 39 2024-03-22T07:16:06.644 INFO:tasks.workunit.client.1.smithi069.stdout:5/552: truncate d1/d6/d86/f77 903943 0 2024-03-22T07:16:06.645 INFO:tasks.workunit.client.0.smithi063.stdout:2/821: rename d0/de/d1b/d32/dfb to d0/de/d110 0 2024-03-22T07:16:06.647 INFO:tasks.workunit.client.1.smithi069.stdout:3/696: unlink d5/d9/d26/d3f/d56/d7f/d90/c94 0 2024-03-22T07:16:06.647 INFO:tasks.workunit.client.1.smithi069.stdout:0/768: mknod d3/d7/d5e/c102 0 2024-03-22T07:16:06.647 INFO:tasks.workunit.client.1.smithi069.stdout:0/769: write d3/d7/d15/d29/d5a/d5d/fb2 [917700,23136] 0 2024-03-22T07:16:06.647 INFO:tasks.workunit.client.1.smithi069.stdout:0/770: creat d3/d7/d17/f103 x:0 0 0 2024-03-22T07:16:06.647 INFO:tasks.workunit.client.1.smithi069.stdout:4/789: getdents d6/d12/db2 0 2024-03-22T07:16:06.647 INFO:tasks.workunit.client.1.smithi069.stdout:4/790: creat d6/d9b/d1f/d58/d7c/f111 x:0 0 0 2024-03-22T07:16:06.648 INFO:tasks.workunit.client.1.smithi069.stdout:4/791: creat d6/d16/d3a/db4/f112 x:0 0 0 2024-03-22T07:16:06.648 INFO:tasks.workunit.client.1.smithi069.stdout:8/825: dwrite d1/d1c/d3e/fb7 [0,4194304] 0 2024-03-22T07:16:06.650 INFO:tasks.workunit.client.1.smithi069.stdout:2/474: link d18/d4d/d27/d33/d60/d64/f8f d18/d4d/d27/d77/fa0 0 2024-03-22T07:16:06.652 INFO:tasks.workunit.client.1.smithi069.stdout:3/697: read d5/d9/d26/d3f/f3e [3064699,91174] 0 2024-03-22T07:16:06.652 INFO:tasks.workunit.client.1.smithi069.stdout:3/698: write d5/d9/d26/fac [547857,26434] 0 2024-03-22T07:16:06.652 INFO:tasks.workunit.client.1.smithi069.stdout:3/699: creat d5/d9/d26/d3f/d56/fe3 x:0 0 0 2024-03-22T07:16:06.652 INFO:tasks.workunit.client.1.smithi069.stdout:3/700: creat d5/d9/d26/fe4 x:0 0 0 2024-03-22T07:16:06.652 INFO:tasks.workunit.client.1.smithi069.stdout:3/701: chown d5/d9/db/d3b/l86 69856280 1 2024-03-22T07:16:06.655 INFO:tasks.workunit.client.1.smithi069.stdout:4/792: creat d6/d16/d83/f113 x:0 0 0 2024-03-22T07:16:06.657 INFO:tasks.workunit.client.1.smithi069.stdout:8/826: read d1/d1c/d3e/d88/dcd/d20/f3b [3677276,130606] 0 2024-03-22T07:16:06.659 INFO:tasks.workunit.client.1.smithi069.stdout:5/553: read d1/d6/d88/f93 [3903191,109776] 0 2024-03-22T07:16:06.661 INFO:tasks.workunit.client.1.smithi069.stdout:2/475: dread d18/d4d/d25/f47 [0,4194304] 0 2024-03-22T07:16:06.661 INFO:tasks.workunit.client.1.smithi069.stdout:2/476: dread - d18/d4d/d27/d33/f9c zero size 2024-03-22T07:16:06.665 INFO:tasks.workunit.client.1.smithi069.stdout:0/771: dread d3/d7/d15/d1f/f26 [4194304,4194304] 0 2024-03-22T07:16:06.665 INFO:tasks.workunit.client.1.smithi069.stdout:0/772: truncate d3/d9a/ff2 868033 0 2024-03-22T07:16:06.666 INFO:tasks.workunit.client.1.smithi069.stdout:3/702: unlink d5/d9/d23/l59 0 2024-03-22T07:16:06.668 INFO:tasks.workunit.client.1.smithi069.stdout:4/793: symlink d6/d16/d3a/db4/dbc/l114 0 2024-03-22T07:16:06.668 INFO:tasks.workunit.client.1.smithi069.stdout:4/794: dread - d6/d16/d3a/db4/f112 zero size 2024-03-22T07:16:06.668 INFO:tasks.workunit.client.1.smithi069.stdout:4/795: fdatasync d6/d16/d83/d9a/fa3 0 2024-03-22T07:16:06.668 INFO:tasks.workunit.client.1.smithi069.stdout:4/796: readlink d6/d16/d3a/db4/dfb/l63 0 2024-03-22T07:16:06.668 INFO:tasks.workunit.client.1.smithi069.stdout:4/797: fsync d6/daa/fe8 0 2024-03-22T07:16:06.669 INFO:tasks.workunit.client.0.smithi063.stdout:2/822: read d0/f9 [1451957,114915] 0 2024-03-22T07:16:06.669 INFO:tasks.workunit.client.0.smithi063.stdout:2/823: fsync d0/f41 0 2024-03-22T07:16:06.670 INFO:tasks.workunit.client.0.smithi063.stdout:2/824: rmdir d0/de/d1b/d8b 39 2024-03-22T07:16:06.671 INFO:tasks.workunit.client.1.smithi069.stdout:3/703: dread d5/d9/d26/f5a [0,4194304] 0 2024-03-22T07:16:06.671 INFO:tasks.workunit.client.1.smithi069.stdout:3/704: fsync d5/d9/db/fab 0 2024-03-22T07:16:06.671 INFO:tasks.workunit.client.1.smithi069.stdout:2/477: mkdir d18/d4d/d27/d33/d60/d63/d80/da1 0 2024-03-22T07:16:06.672 INFO:tasks.workunit.client.1.smithi069.stdout:5/554: write d1/d6/d26/d33/f80 [3322928,5973] 0 2024-03-22T07:16:06.672 INFO:tasks.workunit.client.1.smithi069.stdout:5/555: chown d1/d6/d26/d3e/da0/dba 15301403 1 2024-03-22T07:16:06.672 INFO:tasks.workunit.client.1.smithi069.stdout:5/556: dread - d1/d6/fa8 zero size 2024-03-22T07:16:06.672 INFO:tasks.workunit.client.1.smithi069.stdout:5/557: write d1/d6/f2e [3659758,70649] 0 2024-03-22T07:16:06.672 INFO:tasks.workunit.client.1.smithi069.stdout:2/478: read d18/d4d/f57 [528281,93159] 0 2024-03-22T07:16:06.675 INFO:tasks.workunit.client.1.smithi069.stdout:1/632: dwrite d1/d13/d36/f3e [0,4194304] 0 2024-03-22T07:16:06.678 INFO:tasks.workunit.client.1.smithi069.stdout:3/705: unlink d5/d9/d26/d3f/d56/fd9 0 2024-03-22T07:16:06.678 INFO:tasks.workunit.client.1.smithi069.stdout:3/706: write d5/d9/d26/d3f/d35/f58 [4873213,36633] 0 2024-03-22T07:16:06.678 INFO:tasks.workunit.client.1.smithi069.stdout:0/773: read d3/d7/d15/d29/d5a/d5d/db4/ff5 [4172466,78759] 0 2024-03-22T07:16:06.678 INFO:tasks.workunit.client.1.smithi069.stdout:7/654: dwrite d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/f7b [4194304,4194304] 0 2024-03-22T07:16:06.678 INFO:tasks.workunit.client.0.smithi063.stdout:9/916: truncate d8/d19/d20/f34 3015057 0 2024-03-22T07:16:06.680 INFO:tasks.workunit.client.1.smithi069.stdout:5/558: rename d1/d10/d60/fbe to d1/d6/d26/d33/d5d/fc4 0 2024-03-22T07:16:06.680 INFO:tasks.workunit.client.1.smithi069.stdout:5/559: chown d1/d7/f22 5215811 1 2024-03-22T07:16:06.680 INFO:tasks.workunit.client.1.smithi069.stdout:9/540: sync 2024-03-22T07:16:06.680 INFO:tasks.workunit.client.1.smithi069.stdout:9/541: write d2/d21/f72 [2161747,11873] 0 2024-03-22T07:16:06.680 INFO:tasks.workunit.client.1.smithi069.stdout:9/542: write d2/dd/d3c/d2e/fad [472691,76516] 0 2024-03-22T07:16:06.681 INFO:tasks.workunit.client.1.smithi069.stdout:2/479: mknod d18/d91/ca2 0 2024-03-22T07:16:06.681 INFO:tasks.workunit.client.0.smithi063.stdout:2/825: write d0/de/d12/d20/f99 [478076,115986] 0 2024-03-22T07:16:06.681 INFO:tasks.workunit.client.0.smithi063.stdout:2/826: creat d0/de/d12/d20/f111 x:0 0 0 2024-03-22T07:16:06.682 INFO:tasks.workunit.client.0.smithi063.stdout:2/827: chown d0/de/d12/d20/f2c 13177927 1 2024-03-22T07:16:06.685 INFO:tasks.workunit.client.1.smithi069.stdout:1/633: symlink d1/d2a/d50/d88/d41/lc6 0 2024-03-22T07:16:06.686 INFO:tasks.workunit.client.1.smithi069.stdout:6/563: sync 2024-03-22T07:16:06.686 INFO:tasks.workunit.client.0.smithi063.stdout:2/828: rmdir d0/de/d1b/d63/dd6 39 2024-03-22T07:16:06.688 INFO:tasks.workunit.client.1.smithi069.stdout:8/827: dwrite d1/d1c/d3e/d88/dcd/de8/fef [0,4194304] 0 2024-03-22T07:16:06.689 INFO:tasks.workunit.client.0.smithi063.stdout:2/829: creat d0/de/d35/d36/f112 x:0 0 0 2024-03-22T07:16:06.690 INFO:tasks.workunit.client.1.smithi069.stdout:0/774: symlink d3/d7/d15/d29/d6e/l104 0 2024-03-22T07:16:06.693 INFO:tasks.workunit.client.0.smithi063.stdout:2/830: dread d0/de/d74/f96 [0,4194304] 0 2024-03-22T07:16:06.694 INFO:tasks.workunit.client.1.smithi069.stdout:7/655: creat d9/db/d1a/fd6 x:0 0 0 2024-03-22T07:16:06.695 INFO:tasks.workunit.client.1.smithi069.stdout:5/560: creat d1/d6/d26/d33/d5d/d72/da7/fc5 x:0 0 0 2024-03-22T07:16:06.695 INFO:tasks.workunit.client.1.smithi069.stdout:5/561: stat d1/d6/d26/d3a/c3c 0 2024-03-22T07:16:06.697 INFO:tasks.workunit.client.1.smithi069.stdout:2/480: mknod d18/d4d/ca3 0 2024-03-22T07:16:06.699 INFO:tasks.workunit.client.1.smithi069.stdout:2/481: read d18/f1e [896871,11509] 0 2024-03-22T07:16:06.700 INFO:tasks.workunit.client.1.smithi069.stdout:6/564: creat d4/dd/d12/d44/d7f/fb1 x:0 0 0 2024-03-22T07:16:06.704 INFO:tasks.workunit.client.1.smithi069.stdout:8/828: creat d1/d1c/d3e/d88/dcd/d6c/d73/f107 x:0 0 0 2024-03-22T07:16:06.704 INFO:tasks.workunit.client.1.smithi069.stdout:8/829: chown d1/d1c/d3e/d88/dcd/d6c/f92 507 1 2024-03-22T07:16:06.705 INFO:tasks.workunit.client.1.smithi069.stdout:3/707: dwrite d5/d9/d23/f2b [0,4194304] 0 2024-03-22T07:16:06.705 INFO:tasks.workunit.client.1.smithi069.stdout:1/634: write d1/d2a/d50/d88/d41/f19 [1002732,22188] 0 2024-03-22T07:16:06.705 INFO:tasks.workunit.client.1.smithi069.stdout:1/635: write d1/d13/d36/d6e/d73/fa8 [111585,93226] 0 2024-03-22T07:16:06.706 INFO:tasks.workunit.client.1.smithi069.stdout:5/562: mknod d1/d6/d26/d3e/d9c/d9d/dc3/cc6 0 2024-03-22T07:16:06.706 INFO:tasks.workunit.client.1.smithi069.stdout:5/563: write d1/d6/d26/d3a/f6e [391296,18751] 0 2024-03-22T07:16:06.707 INFO:tasks.workunit.client.1.smithi069.stdout:2/482: link d18/d4d/d3e/d40/f5c d18/d4d/d27/d33/d60/fa4 0 2024-03-22T07:16:06.709 INFO:tasks.workunit.client.1.smithi069.stdout:8/830: creat d1/d1c/d3e/d66/f108 x:0 0 0 2024-03-22T07:16:06.709 INFO:tasks.workunit.client.1.smithi069.stdout:8/831: stat d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/f67 0 2024-03-22T07:16:06.709 INFO:tasks.workunit.client.1.smithi069.stdout:8/832: chown d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/l89 398224754 1 2024-03-22T07:16:06.709 INFO:tasks.workunit.client.1.smithi069.stdout:8/833: readlink d1/d1c/l50 0 2024-03-22T07:16:06.711 INFO:tasks.workunit.client.1.smithi069.stdout:2/483: mknod d18/ca5 0 2024-03-22T07:16:06.711 INFO:tasks.workunit.client.0.smithi063.stdout:5/797: sync 2024-03-22T07:16:06.712 INFO:tasks.workunit.client.0.smithi063.stdout:2/831: write d0/de/d12/d20/f99 [5226341,110644] 0 2024-03-22T07:16:06.712 INFO:tasks.workunit.client.0.smithi063.stdout:9/917: dwrite d8/d19/d2b/f5f [4194304,4194304] 0 2024-03-22T07:16:06.713 INFO:tasks.workunit.client.1.smithi069.stdout:8/834: creat d1/d1c/d3e/d88/dcd/dbd/d106/f109 x:0 0 0 2024-03-22T07:16:06.713 INFO:tasks.workunit.client.0.smithi063.stdout:5/798: creat d8/d13/d1e/d31/daf/ff9 x:0 0 0 2024-03-22T07:16:06.714 INFO:tasks.workunit.client.0.smithi063.stdout:2/832: getdents d0/de/d110 0 2024-03-22T07:16:06.714 INFO:tasks.workunit.client.1.smithi069.stdout:5/564: getdents d1/d6/d26/d3a 0 2024-03-22T07:16:06.715 INFO:tasks.workunit.client.0.smithi063.stdout:2/833: creat d0/de/d12/f113 x:0 0 0 2024-03-22T07:16:06.716 INFO:tasks.workunit.client.0.smithi063.stdout:5/799: read d8/d13/d28/fbc [213305,65746] 0 2024-03-22T07:16:06.716 INFO:tasks.workunit.client.0.smithi063.stdout:2/834: symlink d0/de/d1b/d63/dea/l114 0 2024-03-22T07:16:06.717 INFO:tasks.workunit.client.1.smithi069.stdout:2/484: truncate d18/f1e 11997 0 2024-03-22T07:16:06.718 INFO:tasks.workunit.client.0.smithi063.stdout:5/800: mkdir d8/d13/d1e/d31/d64/de7/dfa 0 2024-03-22T07:16:06.720 INFO:tasks.workunit.client.1.smithi069.stdout:1/636: dread d1/d13/d36/f3e [0,4194304] 0 2024-03-22T07:16:06.720 INFO:tasks.workunit.client.1.smithi069.stdout:1/637: truncate d1/d13/d36/d5b/f87 1378572 0 2024-03-22T07:16:06.723 INFO:tasks.workunit.client.1.smithi069.stdout:5/565: mkdir d1/d6/d26/d33/d5d/d72/d95/dc7 0 2024-03-22T07:16:06.723 INFO:tasks.workunit.client.1.smithi069.stdout:5/566: write d1/d6/d86/d99/f52 [1909347,58116] 0 2024-03-22T07:16:06.723 INFO:tasks.workunit.client.1.smithi069.stdout:5/567: getdents d1/d6/d26/d33/d5d/d72/d95/dc7 0 2024-03-22T07:16:06.724 INFO:tasks.workunit.client.1.smithi069.stdout:2/485: rename d18 to d18/d4d/d27/d33/d60/d64/da6 22 2024-03-22T07:16:06.734 INFO:tasks.workunit.client.1.smithi069.stdout:1/638: dread d1/d13/d36/f54 [0,4194304] 0 2024-03-22T07:16:06.735 INFO:tasks.workunit.client.1.smithi069.stdout:1/639: mkdir d1/d13/dc7 0 2024-03-22T07:16:06.737 INFO:tasks.workunit.client.1.smithi069.stdout:1/640: unlink d1/d2a/d50/d4c/d55/fa5 0 2024-03-22T07:16:06.737 INFO:tasks.workunit.client.1.smithi069.stdout:5/568: dread d1/d6/d26/d3e/d9c/d9d/fa6 [0,4194304] 0 2024-03-22T07:16:06.737 INFO:tasks.workunit.client.1.smithi069.stdout:5/569: fdatasync d1/d6/d26/d33/f4e 0 2024-03-22T07:16:06.738 INFO:tasks.workunit.client.1.smithi069.stdout:1/641: mknod d1/d2a/d50/d88/dae/db5/cc8 0 2024-03-22T07:16:06.739 INFO:tasks.workunit.client.1.smithi069.stdout:5/570: symlink d1/d6/d26/d3a/lc8 0 2024-03-22T07:16:06.739 INFO:tasks.workunit.client.1.smithi069.stdout:5/571: readlink d1/d6/d21/l6c 0 2024-03-22T07:16:06.740 INFO:tasks.workunit.client.1.smithi069.stdout:1/642: rename d1/d2a/d50/d88/c2e to d1/d2a/d5e/d9b/cc9 0 2024-03-22T07:16:06.741 INFO:tasks.workunit.client.1.smithi069.stdout:5/572: stat d1/d7/c1d 0 2024-03-22T07:16:06.741 INFO:tasks.workunit.client.1.smithi069.stdout:5/573: write d1/d6/d88/f9b [4438748,118486] 0 2024-03-22T07:16:06.743 INFO:tasks.workunit.client.1.smithi069.stdout:5/574: rmdir d1/d6/d26/d3a 39 2024-03-22T07:16:06.743 INFO:tasks.workunit.client.1.smithi069.stdout:2/486: read d18/d4d/d27/d33/d60/f5a [4081788,43460] 0 2024-03-22T07:16:06.744 INFO:tasks.workunit.client.1.smithi069.stdout:2/487: dread d18/d4d/f57 [0,4194304] 0 2024-03-22T07:16:06.745 INFO:tasks.workunit.client.1.smithi069.stdout:2/488: link f1 d18/d4d/d27/d77/d7c/fa7 0 2024-03-22T07:16:06.746 INFO:tasks.workunit.client.1.smithi069.stdout:2/489: mknod d18/d4d/d3e/d40/ca8 0 2024-03-22T07:16:06.746 INFO:tasks.workunit.client.1.smithi069.stdout:2/490: fdatasync d18/d4d/d27/d33/f82 0 2024-03-22T07:16:06.759 INFO:tasks.workunit.client.1.smithi069.stdout:5/575: dread d1/d6/d26/d33/d5d/fa4 [0,4194304] 0 2024-03-22T07:16:06.759 INFO:tasks.workunit.client.1.smithi069.stdout:5/576: creat d1/d6/d88/fc9 x:0 0 0 2024-03-22T07:16:06.760 INFO:tasks.workunit.client.1.smithi069.stdout:5/577: read d1/d10/d60/f7b [982063,85689] 0 2024-03-22T07:16:06.760 INFO:tasks.workunit.client.1.smithi069.stdout:5/578: write d1/d10/d60/fac [958742,106520] 0 2024-03-22T07:16:06.760 INFO:tasks.workunit.client.1.smithi069.stdout:2/491: write d18/d4d/d25/f4b [3425076,24317] 0 2024-03-22T07:16:06.760 INFO:tasks.workunit.client.1.smithi069.stdout:6/565: dwrite d4/d7/da/f83 [0,4194304] 0 2024-03-22T07:16:06.761 INFO:tasks.workunit.client.0.smithi063.stdout:9/918: dwrite d8/d19/d20/d22/d29/d31/f115 [0,4194304] 0 2024-03-22T07:16:06.761 INFO:tasks.workunit.client.0.smithi063.stdout:5/801: dwrite d8/d13/d24/d81/fd4 [0,4194304] 0 2024-03-22T07:16:06.762 INFO:tasks.workunit.client.0.smithi063.stdout:9/919: rename d8/d19/l119 to d8/d19/d20/d22/d29/d31/l11c 0 2024-03-22T07:16:06.762 INFO:tasks.workunit.client.1.smithi069.stdout:5/579: creat d1/d7/d3d/fca x:0 0 0 2024-03-22T07:16:06.764 INFO:tasks.workunit.client.0.smithi063.stdout:9/920: symlink d8/d15/d63/l11d 0 2024-03-22T07:16:06.764 INFO:tasks.workunit.client.0.smithi063.stdout:9/921: readlink d8/d19/l52 0 2024-03-22T07:16:06.765 INFO:tasks.workunit.client.0.smithi063.stdout:9/922: chown d8/d15/d63/l79 1178676 1 2024-03-22T07:16:06.765 INFO:tasks.workunit.client.0.smithi063.stdout:9/923: mkdir d8/d19/d20/d22/d9b/d106/d11e 0 2024-03-22T07:16:06.766 INFO:tasks.workunit.client.0.smithi063.stdout:9/924: chown d8/d19/d20/d22/d29/d31/d53 1999844 1 2024-03-22T07:16:06.767 INFO:tasks.workunit.client.0.smithi063.stdout:9/925: mkdir d8/d19/d2b/d11f 0 2024-03-22T07:16:06.774 INFO:tasks.workunit.client.1.smithi069.stdout:5/580: write d1/d6/d86/d99/f12 [190229,124990] 0 2024-03-22T07:16:06.775 INFO:tasks.workunit.client.1.smithi069.stdout:8/835: rmdir d1/d1c/d3e/d66 39 2024-03-22T07:16:06.775 INFO:tasks.workunit.client.1.smithi069.stdout:8/836: stat d1/d1c/d3e/d66/l84 0 2024-03-22T07:16:06.776 INFO:tasks.workunit.client.1.smithi069.stdout:8/837: write d1/d1c/d3e/fee [590658,32494] 0 2024-03-22T07:16:06.778 INFO:tasks.workunit.client.1.smithi069.stdout:6/566: mkdir d4/dd/d12/d2b/da5/db2 0 2024-03-22T07:16:06.782 INFO:tasks.workunit.client.1.smithi069.stdout:8/838: mkdir d1/d1c/d3e/dde/d10a 0 2024-03-22T07:16:06.782 INFO:tasks.workunit.client.1.smithi069.stdout:8/839: truncate d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/ff5 373464 0 2024-03-22T07:16:06.782 INFO:tasks.workunit.client.1.smithi069.stdout:8/840: chown d1/d1c/d48/d5e/fad 744660 1 2024-03-22T07:16:06.790 INFO:tasks.workunit.client.1.smithi069.stdout:2/492: dwrite d18/d4d/d27/d33/f9c [0,4194304] 0 2024-03-22T07:16:06.790 INFO:tasks.workunit.client.1.smithi069.stdout:2/493: write d18/d4d/d27/d33/d60/d64/f99 [897307,72255] 0 2024-03-22T07:16:06.791 INFO:tasks.workunit.client.1.smithi069.stdout:2/494: rename d18/d4d to d18/d4d/d27/d77/d7c/da9 22 2024-03-22T07:16:06.793 INFO:tasks.workunit.client.1.smithi069.stdout:2/495: link d18/d4d/d27/d77/f92 d18/d4d/d27/d33/d60/d63/faa 0 2024-03-22T07:16:06.795 INFO:tasks.workunit.client.0.smithi063.stdout:6/913: sync 2024-03-22T07:16:06.795 INFO:tasks.workunit.client.0.smithi063.stdout:3/960: sync 2024-03-22T07:16:06.795 INFO:tasks.workunit.client.0.smithi063.stdout:6/914: chown d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/fa1 32 1 2024-03-22T07:16:06.805 INFO:tasks.workunit.client.0.smithi063.stdout:3/961: write da/d63/d65/d4b/d10a/f12a [3848911,36806] 0 2024-03-22T07:16:06.806 INFO:tasks.workunit.client.0.smithi063.stdout:3/962: creat da/d63/d65/f136 x:0 0 0 2024-03-22T07:16:06.813 INFO:tasks.workunit.client.0.smithi063.stdout:6/915: dread d3/d6/f2c [0,4194304] 0 2024-03-22T07:16:06.813 INFO:tasks.workunit.client.0.smithi063.stdout:6/916: truncate d3/d15/d16/d60/d6d/d4e/fa7 750935 0 2024-03-22T07:16:06.813 INFO:tasks.workunit.client.0.smithi063.stdout:6/917: chown d3/d15/d16/d60/d6d/d4c 0 1 2024-03-22T07:16:06.813 INFO:tasks.workunit.client.0.smithi063.stdout:6/918: creat d3/d6/f130 x:0 0 0 2024-03-22T07:16:06.819 INFO:tasks.workunit.client.0.smithi063.stdout:3/963: read da/d63/d65/d92/d96/f10f [2944962,13403] 0 2024-03-22T07:16:06.819 INFO:tasks.workunit.client.0.smithi063.stdout:3/964: stat da/d63/d65/d92/l125 0 2024-03-22T07:16:06.819 INFO:tasks.workunit.client.0.smithi063.stdout:3/965: dread - da/d63/f75 zero size 2024-03-22T07:16:06.820 INFO:tasks.workunit.client.0.smithi063.stdout:3/966: symlink da/d16/d3f/l137 0 2024-03-22T07:16:06.829 INFO:tasks.workunit.client.0.smithi063.stdout:5/802: dwrite d8/fa7 [0,4194304] 0 2024-03-22T07:16:06.829 INFO:tasks.workunit.client.1.smithi069.stdout:6/567: dwrite d4/dd/d12/d13/fa1 [0,4194304] 0 2024-03-22T07:16:06.829 INFO:tasks.workunit.client.1.smithi069.stdout:8/841: dwrite d1/d1c/d3e/d88/da8/dbb/fd8 [0,4194304] 0 2024-03-22T07:16:06.830 INFO:tasks.workunit.client.1.smithi069.stdout:8/842: chown d1/d1c/d48/d56/c85 2669 1 2024-03-22T07:16:06.831 INFO:tasks.workunit.client.0.smithi063.stdout:6/919: dread d3/d15/d16/d60/d6d/d4e/d5a/f103 [0,4194304] 0 2024-03-22T07:16:06.831 INFO:tasks.workunit.client.0.smithi063.stdout:9/926: dwrite d8/d19/d20/d22/d5d/d76/fc6 [0,4194304] 0 2024-03-22T07:16:06.831 INFO:tasks.workunit.client.0.smithi063.stdout:6/920: fsync d3/d6/d2a/d38/da5/f63 0 2024-03-22T07:16:06.831 INFO:tasks.workunit.client.0.smithi063.stdout:9/927: write d8/d19/d20/d22/d29/f10d [450294,65547] 0 2024-03-22T07:16:06.832 INFO:tasks.workunit.client.0.smithi063.stdout:5/803: mkdir d8/d13/d24/da4/d8b/db6/dfb 0 2024-03-22T07:16:06.835 INFO:tasks.workunit.client.0.smithi063.stdout:6/921: mknod d3/d15/d16/d60/d6d/d107/d10b/dde/deb/c131 0 2024-03-22T07:16:06.836 INFO:tasks.workunit.client.0.smithi063.stdout:5/804: symlink d8/d13/d24/da4/d8b/db6/dfb/lfc 0 2024-03-22T07:16:06.836 INFO:tasks.workunit.client.0.smithi063.stdout:5/805: write d8/d13/d1e/d31/f46 [8886508,105907] 0 2024-03-22T07:16:06.836 INFO:tasks.workunit.client.0.smithi063.stdout:6/922: getdents d3/d15/d16/d69 0 2024-03-22T07:16:06.836 INFO:tasks.workunit.client.0.smithi063.stdout:6/923: fdatasync d3/d15/d16/d60/d6d/d107/d10b/dde/f120 0 2024-03-22T07:16:06.836 INFO:tasks.workunit.client.0.smithi063.stdout:6/924: write d3/d15/f17 [165914,58406] 0 2024-03-22T07:16:06.837 INFO:tasks.workunit.client.1.smithi069.stdout:8/843: creat d1/d1c/d3e/d66/f10b x:0 0 0 2024-03-22T07:16:06.837 INFO:tasks.workunit.client.0.smithi063.stdout:5/806: mkdir d8/d13/d28/d57/dfd 0 2024-03-22T07:16:06.838 INFO:tasks.workunit.client.1.smithi069.stdout:6/568: getdents d4/d7/da 0 2024-03-22T07:16:06.839 INFO:tasks.workunit.client.0.smithi063.stdout:5/807: rename d8/d13/d28/l40 to d8/d13/d1e/lfe 0 2024-03-22T07:16:06.841 INFO:tasks.workunit.client.0.smithi063.stdout:5/808: rmdir d8/d13/d1e/d31/d64 39 2024-03-22T07:16:06.844 INFO:tasks.workunit.client.1.smithi069.stdout:2/496: rename d18/d4d/d27/d77/d7c to d18/d4d/d27/d33/d60/d64/d89/d8b/dab 0 2024-03-22T07:16:06.848 INFO:tasks.workunit.client.1.smithi069.stdout:8/844: rename d1/d1c/d70/df2 to d1/d1c/d3e/d88/dcd/d6c/d10c 0 2024-03-22T07:16:06.849 INFO:tasks.workunit.client.1.smithi069.stdout:2/497: creat d18/d4d/d27/d33/d60/d64/fac x:0 0 0 2024-03-22T07:16:06.851 INFO:tasks.workunit.client.1.smithi069.stdout:8/845: creat d1/d1c/d3e/d88/dcd/dbd/f10d x:0 0 0 2024-03-22T07:16:06.853 INFO:tasks.workunit.client.1.smithi069.stdout:2/498: rmdir d18/d4d/d27/d33/d60/d63/d80/da1 0 2024-03-22T07:16:06.853 INFO:tasks.workunit.client.0.smithi063.stdout:9/928: dread d8/d15/f1c [0,4194304] 0 2024-03-22T07:16:06.855 INFO:tasks.workunit.client.1.smithi069.stdout:2/499: truncate d18/d4d/f26 1895687 0 2024-03-22T07:16:06.855 INFO:tasks.workunit.client.1.smithi069.stdout:2/500: chown d18/d4d/d27/d72 21301 1 2024-03-22T07:16:06.855 INFO:tasks.workunit.client.1.smithi069.stdout:8/846: symlink d1/d1c/d48/l10e 0 2024-03-22T07:16:06.856 INFO:tasks.workunit.client.1.smithi069.stdout:2/501: link d18/ca5 d18/d4d/d3e/d40/d96/cad 0 2024-03-22T07:16:06.856 INFO:tasks.workunit.client.1.smithi069.stdout:8/847: creat d1/d1c/d3e/d88/f10f x:0 0 0 2024-03-22T07:16:06.884 INFO:tasks.workunit.client.0.smithi063.stdout:3/967: dwrite da/d63/d65/d4b/dfa/dfc/f106 [0,4194304] 0 2024-03-22T07:16:06.884 INFO:tasks.workunit.client.0.smithi063.stdout:9/929: dwrite d8/d19/d20/d22/d29/faa [0,4194304] 0 2024-03-22T07:16:06.888 INFO:tasks.workunit.client.1.smithi069.stdout:6/569: dwrite d4/dd/f54 [0,4194304] 0 2024-03-22T07:16:06.888 INFO:tasks.workunit.client.1.smithi069.stdout:6/570: chown d4/dd/d41/d78/d9a 574838867 1 2024-03-22T07:16:06.890 INFO:tasks.workunit.client.0.smithi063.stdout:6/925: dwrite d3/d15/d16/d60/d6d/d107/f118 [0,4194304] 0 2024-03-22T07:16:06.891 INFO:tasks.workunit.client.0.smithi063.stdout:6/926: fsync d3/d6/d2a/d38/da5/f4b 0 2024-03-22T07:16:06.891 INFO:tasks.workunit.client.0.smithi063.stdout:6/927: creat d3/d15/d16/d69/d8c/f132 x:0 0 0 2024-03-22T07:16:06.891 INFO:tasks.workunit.client.0.smithi063.stdout:3/968: link da/d16/d89/d8c/dc1/fcb da/d16/d3f/dc7/f138 0 2024-03-22T07:16:06.893 INFO:tasks.workunit.client.1.smithi069.stdout:5/581: truncate d1/d6/d86/d99/f12 468354 0 2024-03-22T07:16:06.893 INFO:tasks.workunit.client.1.smithi069.stdout:5/582: fsync d1/d6/d86/d99/f18 0 2024-03-22T07:16:06.902 INFO:tasks.workunit.client.1.smithi069.stdout:5/583: read d1/d10/d60/f83 [105423,96450] 0 2024-03-22T07:16:06.903 INFO:tasks.workunit.client.0.smithi063.stdout:8/948: sync 2024-03-22T07:16:06.903 INFO:tasks.workunit.client.0.smithi063.stdout:4/816: sync 2024-03-22T07:16:06.904 INFO:tasks.workunit.client.0.smithi063.stdout:2/835: sync 2024-03-22T07:16:06.904 INFO:tasks.workunit.client.0.smithi063.stdout:2/836: write d0/de/d35/ffe [260782,29888] 0 2024-03-22T07:16:06.904 INFO:tasks.workunit.client.0.smithi063.stdout:7/737: sync 2024-03-22T07:16:06.907 INFO:tasks.workunit.client.0.smithi063.stdout:6/928: write d3/d15/da3/dc2/ff9 [1598436,69078] 0 2024-03-22T07:16:06.907 INFO:tasks.workunit.client.0.smithi063.stdout:6/929: fdatasync d3/d15/d16/d60/d6d/d4e/d5f/fb7 0 2024-03-22T07:16:06.907 INFO:tasks.workunit.client.0.smithi063.stdout:4/817: creat d1/d5/d43/d27/d2c/df4/f116 x:0 0 0 2024-03-22T07:16:06.907 INFO:tasks.workunit.client.0.smithi063.stdout:4/818: chown d1/d5/d43/d27/d2c/d3d/d54/ddb/c10c 249 1 2024-03-22T07:16:06.907 INFO:tasks.workunit.client.0.smithi063.stdout:2/837: rmdir d0/ded/df9 0 2024-03-22T07:16:06.908 INFO:tasks.workunit.client.0.smithi063.stdout:7/738: unlink d5/da/d28/da7/fb0 0 2024-03-22T07:16:06.911 INFO:tasks.workunit.client.0.smithi063.stdout:2/838: symlink d0/de/d1b/d32/dac/l115 0 2024-03-22T07:16:06.912 INFO:tasks.workunit.client.0.smithi063.stdout:7/739: dread d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/f6a [0,4194304] 0 2024-03-22T07:16:06.914 INFO:tasks.workunit.client.0.smithi063.stdout:7/740: truncate d5/f9 1408550 0 2024-03-22T07:16:06.917 INFO:tasks.workunit.client.0.smithi063.stdout:8/949: dread d0/d26/f34 [0,4194304] 0 2024-03-22T07:16:06.917 INFO:tasks.workunit.client.0.smithi063.stdout:8/950: dread - d0/d1b/d28/d3e/db2/dbd/dfd/f114 zero size 2024-03-22T07:16:06.932 INFO:tasks.workunit.client.0.smithi063.stdout:2/839: write d0/de/d12/d5a/f81 [2023159,43199] 0 2024-03-22T07:16:06.932 INFO:tasks.workunit.client.0.smithi063.stdout:2/840: readlink d0/de/d12/d5a/l5f 0 2024-03-22T07:16:06.934 INFO:tasks.workunit.client.0.smithi063.stdout:7/741: write d5/da/d28/d4b/da8/dbc/d62/d73/f7a [5009180,38184] 0 2024-03-22T07:16:06.935 INFO:tasks.workunit.client.0.smithi063.stdout:7/742: write d5/da/d5f/d33/fbd [4795946,106208] 0 2024-03-22T07:16:06.935 INFO:tasks.workunit.client.0.smithi063.stdout:7/743: read - d5/da/d28/d4b/da8/fdb zero size 2024-03-22T07:16:06.935 INFO:tasks.workunit.client.0.smithi063.stdout:7/744: fsync d5/da/d28/d4b/da8/dbc/d62/d73/fd7 0 2024-03-22T07:16:06.941 INFO:tasks.workunit.client.1.smithi069.stdout:8/848: creat d1/d1c/d3e/d88/dcd/d20/f110 x:0 0 0 2024-03-22T07:16:06.942 INFO:tasks.workunit.client.1.smithi069.stdout:8/849: read d1/d1c/d3e/d88/dcd/fca [537373,92492] 0 2024-03-22T07:16:06.942 INFO:tasks.workunit.client.1.smithi069.stdout:8/850: write d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/fcf [288031,33123] 0 2024-03-22T07:16:06.946 INFO:tasks.workunit.client.1.smithi069.stdout:8/851: mkdir d1/d1c/d3e/d88/dcd/d6c/d4e/d101/d111 0 2024-03-22T07:16:06.947 INFO:tasks.workunit.client.1.smithi069.stdout:5/584: getdents d1/d6/d26/d33/d5d/d72 0 2024-03-22T07:16:06.949 INFO:tasks.workunit.client.1.smithi069.stdout:8/852: mkdir d1/d1c/d3e/db9/d112 0 2024-03-22T07:16:06.951 INFO:tasks.workunit.client.1.smithi069.stdout:5/585: rename d1/d6/d26/d33/d5d/d72/f9e to d1/d6/d86/dab/fcb 0 2024-03-22T07:16:06.951 INFO:tasks.workunit.client.1.smithi069.stdout:5/586: write d1/d10/d44/d50/f96 [1042337,5337] 0 2024-03-22T07:16:06.953 INFO:tasks.workunit.client.1.smithi069.stdout:8/853: mknod d1/d1c/d3e/d88/dcd/d6c/d8e/c113 0 2024-03-22T07:16:06.953 INFO:tasks.workunit.client.1.smithi069.stdout:8/854: write d1/d1c/d3e/d88/f8f [2923281,48429] 0 2024-03-22T07:16:06.953 INFO:tasks.workunit.client.1.smithi069.stdout:8/855: stat d1/d1c/d3e/d88/dcd/d6c/d4e/d101 0 2024-03-22T07:16:06.953 INFO:tasks.workunit.client.1.smithi069.stdout:8/856: write d1/d93/ffc [1007359,103751] 0 2024-03-22T07:16:06.954 INFO:tasks.workunit.client.1.smithi069.stdout:8/857: symlink d1/d93/ddf/de3/l114 0 2024-03-22T07:16:06.962 INFO:tasks.workunit.client.1.smithi069.stdout:5/587: write d1/d6/f6f [15890,1722] 0 2024-03-22T07:16:06.963 INFO:tasks.workunit.client.0.smithi063.stdout:2/841: dread d0/f51 [0,4194304] 0 2024-03-22T07:16:06.963 INFO:tasks.workunit.client.0.smithi063.stdout:2/842: chown d0/de/d1b/d63/d6e/cb1 4 1 2024-03-22T07:16:06.963 INFO:tasks.workunit.client.0.smithi063.stdout:2/843: write d0/de/d12/d5a/fa6 [437891,108764] 0 2024-03-22T07:16:06.964 INFO:tasks.workunit.client.1.smithi069.stdout:5/588: link d1/d7/d3d/c8f d1/d6/d21/dae/ccc 0 2024-03-22T07:16:06.964 INFO:tasks.workunit.client.1.smithi069.stdout:5/589: creat d1/d6/d86/d99/d64/fcd x:0 0 0 2024-03-22T07:16:06.965 INFO:tasks.workunit.client.0.smithi063.stdout:2/844: creat d0/de/d1b/f116 x:0 0 0 2024-03-22T07:16:06.966 INFO:tasks.workunit.client.1.smithi069.stdout:6/571: dwrite d4/dd/d12/d44/f6f [0,4194304] 0 2024-03-22T07:16:06.966 INFO:tasks.workunit.client.1.smithi069.stdout:6/572: write d4/d7/da/d3f/f99 [432697,27312] 0 2024-03-22T07:16:06.966 INFO:tasks.workunit.client.0.smithi063.stdout:2/845: symlink d0/de/d1b/d63/dc6/l117 0 2024-03-22T07:16:06.968 INFO:tasks.workunit.client.0.smithi063.stdout:2/846: link d0/de/d74/cb5 d0/de/d1b/d63/dea/c118 0 2024-03-22T07:16:06.968 INFO:tasks.workunit.client.0.smithi063.stdout:2/847: read - d0/de/d35/d36/ff1 zero size 2024-03-22T07:16:06.969 INFO:tasks.workunit.client.0.smithi063.stdout:2/848: write d0/de/d58/f73 [911717,101824] 0 2024-03-22T07:16:06.973 INFO:tasks.workunit.client.1.smithi069.stdout:6/573: creat d4/dd/d12/fb3 x:0 0 0 2024-03-22T07:16:06.973 INFO:tasks.workunit.client.0.smithi063.stdout:2/849: truncate d0/de/d74/fee 1987647 0 2024-03-22T07:16:06.974 INFO:tasks.workunit.client.1.smithi069.stdout:6/574: mkdir d4/dd/d12/d44/d58/db4 0 2024-03-22T07:16:06.974 INFO:tasks.workunit.client.1.smithi069.stdout:6/575: dread - d4/d7/f6e zero size 2024-03-22T07:16:06.974 INFO:tasks.workunit.client.1.smithi069.stdout:6/576: fdatasync d4/dd/d12/d44/d58/f62 0 2024-03-22T07:16:06.975 INFO:tasks.workunit.client.1.smithi069.stdout:6/577: truncate d4/d7/da/d3f/f99 886111 0 2024-03-22T07:16:06.975 INFO:tasks.workunit.client.1.smithi069.stdout:6/578: creat d4/dd/d12/fb5 x:0 0 0 2024-03-22T07:16:06.975 INFO:tasks.workunit.client.0.smithi063.stdout:2/850: rename d0/de/d74/dca/fd5 to d0/de/d1b/d8b/f119 0 2024-03-22T07:16:06.976 INFO:tasks.workunit.client.0.smithi063.stdout:2/851: rename d0/de/d12/d20/fe6 to d0/de/d35/f11a 0 2024-03-22T07:16:06.991 INFO:tasks.workunit.client.0.smithi063.stdout:4/819: dwrite d1/d5/f97 [0,4194304] 0 2024-03-22T07:16:06.991 INFO:tasks.workunit.client.0.smithi063.stdout:4/820: unlink d1/d5/dc2/f112 0 2024-03-22T07:16:06.994 INFO:tasks.workunit.client.0.smithi063.stdout:9/930: dwrite d8/d19/d20/d22/d29/d31/d53/ffe [0,4194304] 0 2024-03-22T07:16:06.994 INFO:tasks.workunit.client.0.smithi063.stdout:3/969: dwrite da/d63/d65/d4b/d10a/f120 [0,4194304] 0 2024-03-22T07:16:06.994 INFO:tasks.workunit.client.0.smithi063.stdout:3/970: chown da/d16/d89/da9/l130 34589 1 2024-03-22T07:16:06.994 INFO:tasks.workunit.client.0.smithi063.stdout:4/821: symlink d1/d5/dc/d95/d59/d9c/dac/dc9/l117 0 2024-03-22T07:16:06.995 INFO:tasks.workunit.client.0.smithi063.stdout:9/931: creat d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/f120 x:0 0 0 2024-03-22T07:16:06.997 INFO:tasks.workunit.client.0.smithi063.stdout:3/971: rmdir da/d63/d65/d4b/ddd 39 2024-03-22T07:16:06.997 INFO:tasks.workunit.client.0.smithi063.stdout:4/822: mknod d1/d5/dc/d95/d59/c118 0 2024-03-22T07:16:06.999 INFO:tasks.workunit.client.0.smithi063.stdout:3/972: link da/c53 da/d16/d89/da9/c139 0 2024-03-22T07:16:07.000 INFO:tasks.workunit.client.1.smithi069.stdout:2/502: creat d18/d4d/d3e/fae x:0 0 0 2024-03-22T07:16:07.000 INFO:tasks.workunit.client.0.smithi063.stdout:3/973: symlink da/d16/d3f/l13a 0 2024-03-22T07:16:07.001 INFO:tasks.workunit.client.0.smithi063.stdout:3/974: stat da/d63/d65/d81/da5/laa 0 2024-03-22T07:16:07.001 INFO:tasks.workunit.client.0.smithi063.stdout:3/975: fsync da/d63/d65/d92/fff 0 2024-03-22T07:16:07.005 INFO:tasks.workunit.client.0.smithi063.stdout:3/976: unlink da/d16/d3f/c5d 0 2024-03-22T07:16:07.005 INFO:tasks.workunit.client.0.smithi063.stdout:9/932: dread d8/fd [0,4194304] 0 2024-03-22T07:16:07.005 INFO:tasks.workunit.client.0.smithi063.stdout:3/977: mknod da/d16/c13b 0 2024-03-22T07:16:07.005 INFO:tasks.workunit.client.0.smithi063.stdout:3/978: fdatasync da/f57 0 2024-03-22T07:16:07.007 INFO:tasks.workunit.client.0.smithi063.stdout:9/933: truncate d8/d19/d20/d22/d29/d31/d53/d6a/da2/f6b 2318345 0 2024-03-22T07:16:07.008 INFO:tasks.workunit.client.0.smithi063.stdout:3/979: creat da/d16/d26/f13c x:0 0 0 2024-03-22T07:16:07.008 INFO:tasks.workunit.client.0.smithi063.stdout:3/980: truncate da/d16/d3f/ff7 982068 0 2024-03-22T07:16:07.009 INFO:tasks.workunit.client.0.smithi063.stdout:3/981: fsync da/d63/d65/d7b/f117 0 2024-03-22T07:16:07.009 INFO:tasks.workunit.client.0.smithi063.stdout:3/982: write da/d16/d26/f13c [503759,16743] 0 2024-03-22T07:16:07.009 INFO:tasks.workunit.client.0.smithi063.stdout:9/934: mknod d8/d19/d20/d22/d9b/d106/d11e/c121 0 2024-03-22T07:16:07.009 INFO:tasks.workunit.client.0.smithi063.stdout:9/935: chown d8/d19/d20/d22/db2/fb9 16045945 1 2024-03-22T07:16:07.010 INFO:tasks.workunit.client.0.smithi063.stdout:6/930: dwrite d3/d15/d16/f59 [0,4194304] 0 2024-03-22T07:16:07.010 INFO:tasks.workunit.client.0.smithi063.stdout:6/931: read - d3/d6/f11b zero size 2024-03-22T07:16:07.010 INFO:tasks.workunit.client.0.smithi063.stdout:3/983: link da/d63/d65/d81/da5/fad da/d16/d3f/f13d 0 2024-03-22T07:16:07.011 INFO:tasks.workunit.client.0.smithi063.stdout:9/936: symlink d8/d19/d2b/d11f/l122 0 2024-03-22T07:16:07.012 INFO:tasks.workunit.client.1.smithi069.stdout:2/503: dread fa [0,4194304] 0 2024-03-22T07:16:07.013 INFO:tasks.workunit.client.0.smithi063.stdout:6/932: rename d3/d15/d16/l37 to d3/d15/d16/d60/d6d/d4e/d5a/d88/df5/d114/d11c/l133 0 2024-03-22T07:16:07.013 INFO:tasks.workunit.client.0.smithi063.stdout:6/933: readlink d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/ld7 0 2024-03-22T07:16:07.015 INFO:tasks.workunit.client.0.smithi063.stdout:3/984: creat da/d16/d89/d8c/f13e x:0 0 0 2024-03-22T07:16:07.021 INFO:tasks.workunit.client.0.smithi063.stdout:6/934: dread d3/f89 [0,4194304] 0 2024-03-22T07:16:07.021 INFO:tasks.workunit.client.0.smithi063.stdout:4/823: dwrite d1/d5/dc/d95/d59/d9c/dac/dcd/fc1 [0,4194304] 0 2024-03-22T07:16:07.025 INFO:tasks.workunit.client.0.smithi063.stdout:3/985: dread da/f1c [0,4194304] 0 2024-03-22T07:16:07.028 INFO:tasks.workunit.client.1.smithi069.stdout:2/504: read d18/d4d/d3e/d40/f61 [2096258,63186] 0 2024-03-22T07:16:07.029 INFO:tasks.workunit.client.1.smithi069.stdout:2/505: symlink d18/d4d/d27/d33/d60/d63/d80/laf 0 2024-03-22T07:16:07.029 INFO:tasks.workunit.client.1.smithi069.stdout:2/506: chown d18/d4d/d27/d33/d60/d64/d89/f8d 5 1 2024-03-22T07:16:07.030 INFO:tasks.workunit.client.1.smithi069.stdout:2/507: symlink d18/d4d/d27/d33/d60/d64/d89/d8b/dab/lb0 0 2024-03-22T07:16:07.031 INFO:tasks.workunit.client.0.smithi063.stdout:6/935: dread d3/d6/f10 [0,4194304] 0 2024-03-22T07:16:07.033 INFO:tasks.workunit.client.0.smithi063.stdout:6/936: rename d3/d15/d16/d11e to d3/d15/d16/d60/d6d/d134 0 2024-03-22T07:16:07.033 INFO:tasks.workunit.client.0.smithi063.stdout:6/937: write d3/d15/d16/d60/d6d/d107/d10b/f97 [1078464,41343] 0 2024-03-22T07:16:07.035 INFO:tasks.workunit.client.0.smithi063.stdout:6/938: creat d3/d15/d16/d60/d6d/d134/f135 x:0 0 0 2024-03-22T07:16:07.037 INFO:tasks.workunit.client.1.smithi069.stdout:6/579: dwrite d4/dd/d41/d78/f85 [0,4194304] 0 2024-03-22T07:16:07.047 INFO:tasks.workunit.client.0.smithi063.stdout:8/951: dwrite d0/d24/d33/d4c/f7d [0,4194304] 0 2024-03-22T07:16:07.048 INFO:tasks.workunit.client.0.smithi063.stdout:2/852: dwrite d0/de/d12/d20/f52 [0,4194304] 0 2024-03-22T07:16:07.050 INFO:tasks.workunit.client.0.smithi063.stdout:8/952: read d0/d1b/d28/d3e/fc2 [373205,48694] 0 2024-03-22T07:16:07.050 INFO:tasks.workunit.client.0.smithi063.stdout:8/953: creat d0/d1b/d28/d3e/db2/dbd/dd0/f13f x:0 0 0 2024-03-22T07:16:07.058 INFO:tasks.workunit.client.0.smithi063.stdout:8/954: mknod d0/d26/d2d/d59/dba/dc9/c140 0 2024-03-22T07:16:07.066 INFO:tasks.workunit.client.1.smithi069.stdout:8/858: dwrite d1/d1c/d3e/d88/dcd/d6c/d8e/f91 [0,4194304] 0 2024-03-22T07:16:07.066 INFO:tasks.workunit.client.1.smithi069.stdout:8/859: dread - d1/d1c/d48/d5e/fad zero size 2024-03-22T07:16:07.066 INFO:tasks.workunit.client.1.smithi069.stdout:8/860: read - d1/d1c/d3e/d88/dcd/d20/f110 zero size 2024-03-22T07:16:07.066 INFO:tasks.workunit.client.1.smithi069.stdout:5/590: dwrite d1/d6/d86/d99/f90 [0,4194304] 0 2024-03-22T07:16:07.071 INFO:tasks.workunit.client.1.smithi069.stdout:8/861: rename d1/c31 to d1/d1c/d3e/d88/dcd/d6c/d73/dfa/c115 0 2024-03-22T07:16:07.073 INFO:tasks.workunit.client.0.smithi063.stdout:8/955: dread d0/fc [0,4194304] 0 2024-03-22T07:16:07.075 INFO:tasks.workunit.client.1.smithi069.stdout:8/862: unlink d1/d1c/d3e/d88/dcd/d20/f4f 0 2024-03-22T07:16:07.075 INFO:tasks.workunit.client.1.smithi069.stdout:8/863: chown d1/d1c/f96 374 1 2024-03-22T07:16:07.075 INFO:tasks.workunit.client.1.smithi069.stdout:8/864: fdatasync d1/d1c/d3e/d66/f6e 0 2024-03-22T07:16:07.076 INFO:tasks.workunit.client.1.smithi069.stdout:8/865: mknod d1/d1c/d3e/db9/d112/c116 0 2024-03-22T07:16:07.081 INFO:tasks.workunit.client.0.smithi063.stdout:9/937: dwrite d8/d19/d20/fa0 [0,4194304] 0 2024-03-22T07:16:07.082 INFO:tasks.workunit.client.0.smithi063.stdout:9/938: chown d8/d19/l49 2 1 2024-03-22T07:16:07.082 INFO:tasks.workunit.client.0.smithi063.stdout:9/939: read d8/fd [973488,7675] 0 2024-03-22T07:16:07.082 INFO:tasks.workunit.client.0.smithi063.stdout:4/824: dwrite d1/d5/d3c/d8f/dc0/fef [0,4194304] 0 2024-03-22T07:16:07.082 INFO:tasks.workunit.client.0.smithi063.stdout:4/825: write d1/f45 [56629,62637] 0 2024-03-22T07:16:07.089 INFO:tasks.workunit.client.0.smithi063.stdout:8/956: dread d0/d24/d5b/f44 [0,4194304] 0 2024-03-22T07:16:07.091 INFO:tasks.workunit.client.0.smithi063.stdout:8/957: mkdir d0/d26/d2d/d59/dc0/d106/d141 0 2024-03-22T07:16:07.092 INFO:tasks.workunit.client.1.smithi069.stdout:9/543: sync 2024-03-22T07:16:07.092 INFO:tasks.workunit.client.1.smithi069.stdout:3/708: sync 2024-03-22T07:16:07.092 INFO:tasks.workunit.client.1.smithi069.stdout:4/798: sync 2024-03-22T07:16:07.092 INFO:tasks.workunit.client.1.smithi069.stdout:7/656: sync 2024-03-22T07:16:07.092 INFO:tasks.workunit.client.1.smithi069.stdout:0/775: sync 2024-03-22T07:16:07.092 INFO:tasks.workunit.client.1.smithi069.stdout:1/643: sync 2024-03-22T07:16:07.093 INFO:tasks.workunit.client.1.smithi069.stdout:1/644: chown d1/d2a/d50/d88/fb8 213861 1 2024-03-22T07:16:07.093 INFO:tasks.workunit.client.1.smithi069.stdout:4/799: rmdir d6/d9b/d1f/d3f 39 2024-03-22T07:16:07.094 INFO:tasks.workunit.client.1.smithi069.stdout:0/776: symlink d3/d7/d15/d29/d6e/l105 0 2024-03-22T07:16:07.095 INFO:tasks.workunit.client.0.smithi063.stdout:4/826: dread d1/d5/d43/f39 [0,4194304] 0 2024-03-22T07:16:07.095 INFO:tasks.workunit.client.0.smithi063.stdout:4/827: readlink d1/d5/dc/d95/d59/la5 0 2024-03-22T07:16:07.098 INFO:tasks.workunit.client.1.smithi069.stdout:7/657: rename d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/f7b to d9/fd7 0 2024-03-22T07:16:07.100 INFO:tasks.workunit.client.1.smithi069.stdout:3/709: symlink d5/d9/d26/d36/le5 0 2024-03-22T07:16:07.101 INFO:tasks.workunit.client.0.smithi063.stdout:3/986: dwrite da/d63/d65/d92/f124 [0,4194304] 0 2024-03-22T07:16:07.103 INFO:tasks.workunit.client.1.smithi069.stdout:9/544: creat d2/dd/d3c/d53/fae x:0 0 0 2024-03-22T07:16:07.103 INFO:tasks.workunit.client.1.smithi069.stdout:9/545: truncate d2/dd/d3c/d2e/f65 608832 0 2024-03-22T07:16:07.103 INFO:tasks.workunit.client.1.smithi069.stdout:2/508: getdents d18/d4d/d27/d33/d60/d63/d80 0 2024-03-22T07:16:07.104 INFO:tasks.workunit.client.1.smithi069.stdout:4/800: mknod d6/d16/c115 0 2024-03-22T07:16:07.104 INFO:tasks.workunit.client.1.smithi069.stdout:4/801: fsync d6/d16/d3a/db4/dfb/d1d/d55/f5f 0 2024-03-22T07:16:07.104 INFO:tasks.workunit.client.0.smithi063.stdout:3/987: truncate da/d16/d89/da9/dd3/f110 2920841 0 2024-03-22T07:16:07.105 INFO:tasks.workunit.client.1.smithi069.stdout:7/658: mkdir d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/dd8 0 2024-03-22T07:16:07.106 INFO:tasks.workunit.client.1.smithi069.stdout:7/659: creat d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/dce/fd9 x:0 0 0 2024-03-22T07:16:07.106 INFO:tasks.workunit.client.1.smithi069.stdout:7/660: stat d9/db/d1a/db1/d57/d44/d53/f6d 0 2024-03-22T07:16:07.106 INFO:tasks.workunit.client.1.smithi069.stdout:3/710: link d5/d9/d26/d3f/d35/fb6 d5/d9/d26/d3f/d56/d8a/fe6 0 2024-03-22T07:16:07.107 INFO:tasks.workunit.client.1.smithi069.stdout:3/711: readlink d5/d9/d26/d3f/d35/l42 0 2024-03-22T07:16:07.107 INFO:tasks.workunit.client.0.smithi063.stdout:3/988: unlink da/d63/d65/d4b/c118 0 2024-03-22T07:16:07.107 INFO:tasks.workunit.client.0.smithi063.stdout:4/828: dread d1/d5/dc/d95/d59/f64 [0,4194304] 0 2024-03-22T07:16:07.108 INFO:tasks.workunit.client.0.smithi063.stdout:4/829: mkdir d1/d5/dc/d95/d59/d9c/d119 0 2024-03-22T07:16:07.111 INFO:tasks.workunit.client.1.smithi069.stdout:7/661: symlink d9/db/d1a/db1/d57/d75/d7a/lda 0 2024-03-22T07:16:07.111 INFO:tasks.workunit.client.0.smithi063.stdout:4/830: creat d1/d5/d3c/d8f/dbb/d114/d79/daa/f11a x:0 0 0 2024-03-22T07:16:07.113 INFO:tasks.workunit.client.1.smithi069.stdout:5/591: dwrite d1/d6/fa8 [0,4194304] 0 2024-03-22T07:16:07.113 INFO:tasks.workunit.client.1.smithi069.stdout:5/592: chown d1/d6/d86/c67 13357860 1 2024-03-22T07:16:07.113 INFO:tasks.workunit.client.1.smithi069.stdout:3/712: link d5/d9/db/d3b/l66 d5/d9/d26/d3f/d56/d8a/da5/d7b/le7 0 2024-03-22T07:16:07.113 INFO:tasks.workunit.client.1.smithi069.stdout:3/713: write d5/d9/db/f63 [2336565,20660] 0 2024-03-22T07:16:07.114 INFO:tasks.workunit.client.1.smithi069.stdout:9/546: rmdir d2/d83 39 2024-03-22T07:16:07.114 INFO:tasks.workunit.client.0.smithi063.stdout:3/989: dread da/d63/d65/d4b/d10a/f12a [0,4194304] 0 2024-03-22T07:16:07.114 INFO:tasks.workunit.client.0.smithi063.stdout:3/990: fdatasync da/d16/f38 0 2024-03-22T07:16:07.114 INFO:tasks.workunit.client.0.smithi063.stdout:3/991: write da/d63/d65/f136 [923335,7452] 0 2024-03-22T07:16:07.116 INFO:tasks.workunit.client.0.smithi063.stdout:3/992: link da/d63/d65/d4b/f85 da/d63/d65/d83/f13f 0 2024-03-22T07:16:07.116 INFO:tasks.workunit.client.0.smithi063.stdout:3/993: stat da/d63/d65/d4b/cf2 0 2024-03-22T07:16:07.117 INFO:tasks.workunit.client.0.smithi063.stdout:3/994: creat da/d16/d26/d127/f140 x:0 0 0 2024-03-22T07:16:07.118 INFO:tasks.workunit.client.0.smithi063.stdout:3/995: truncate da/d16/d3f/f79 1084736 0 2024-03-22T07:16:07.118 INFO:tasks.workunit.client.1.smithi069.stdout:3/714: link d5/da2/fdc d5/d9/d26/d36/d41/fe8 0 2024-03-22T07:16:07.118 INFO:tasks.workunit.client.1.smithi069.stdout:3/715: creat d5/d9/db/dae/fe9 x:0 0 0 2024-03-22T07:16:07.119 INFO:tasks.workunit.client.1.smithi069.stdout:9/547: rename d2/dd/d3c/c34 to d2/dd/d3c/d2e/d84/da5/caf 0 2024-03-22T07:16:07.120 INFO:tasks.workunit.client.1.smithi069.stdout:5/593: mkdir d1/d6/d26/d33/d5d/d72/d95/dc7/dce 0 2024-03-22T07:16:07.121 INFO:tasks.workunit.client.0.smithi063.stdout:3/996: read da/d16/d26/d69/f6b [2741,91578] 0 2024-03-22T07:16:07.121 INFO:tasks.workunit.client.0.smithi063.stdout:3/997: write da/d63/d65/d4b/d10a/f12a [2394200,34301] 0 2024-03-22T07:16:07.121 INFO:tasks.workunit.client.0.smithi063.stdout:3/998: fdatasync da/d16/d26/fbd 0 2024-03-22T07:16:07.121 INFO:tasks.workunit.client.0.smithi063.stdout:3/999: dread - da/d63/d65/d81/da5/fad zero size 2024-03-22T07:16:07.121 INFO:tasks.workunit.client.0.smithi063.stdout:6/939: dwrite d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/f5b [0,4194304] 0 2024-03-22T07:16:07.122 INFO:tasks.workunit.client.1.smithi069.stdout:6/580: dwrite d4/dd/d12/d44/d58/f75 [0,4194304] 0 2024-03-22T07:16:07.122 INFO:tasks.workunit.client.1.smithi069.stdout:6/581: fsync d4/dd/d12/d13/f8a 0 2024-03-22T07:16:07.122 INFO:tasks.workunit.client.1.smithi069.stdout:3/716: rmdir d5/d9/d26/d3f/dc1 39 2024-03-22T07:16:07.123 INFO:tasks.workunit.client.0.smithi063.stdout:2/853: dwrite d0/de/d35/d36/fb9 [0,4194304] 0 2024-03-22T07:16:07.125 INFO:tasks.workunit.client.1.smithi069.stdout:9/548: mkdir d2/dd/d3c/d48/d7f/db0 0 2024-03-22T07:16:07.125 INFO:tasks.workunit.client.1.smithi069.stdout:9/549: fsync d2/d83/f92 0 2024-03-22T07:16:07.125 INFO:tasks.workunit.client.0.smithi063.stdout:6/940: creat d3/d15/d16/d60/d6d/d107/d10b/dde/deb/f136 x:0 0 0 2024-03-22T07:16:07.127 INFO:tasks.workunit.client.1.smithi069.stdout:3/717: symlink d5/da2/ddd/lea 0 2024-03-22T07:16:07.127 INFO:tasks.workunit.client.0.smithi063.stdout:2/854: symlink d0/de/d1b/d8b/l11b 0 2024-03-22T07:16:07.129 INFO:tasks.workunit.client.1.smithi069.stdout:5/594: rename d1/d6/d86/d99/d64 to d1/d6/d86/dab/dcf 0 2024-03-22T07:16:07.132 INFO:tasks.workunit.client.1.smithi069.stdout:8/866: dwrite d1/d1c/d3e/d88/dcd/d6c/d4e/d98/f105 [0,4194304] 0 2024-03-22T07:16:07.138 INFO:tasks.workunit.client.1.smithi069.stdout:9/550: dread d2/d51/f57 [0,4194304] 0 2024-03-22T07:16:07.140 INFO:tasks.workunit.client.1.smithi069.stdout:9/551: write d2/dd/d3c/d2e/fad [269128,73960] 0 2024-03-22T07:16:07.142 INFO:tasks.workunit.client.0.smithi063.stdout:7/745: truncate d5/f9 55175 0 2024-03-22T07:16:07.146 INFO:tasks.workunit.client.0.smithi063.stdout:7/746: rename d5/da/d28/d4b/da8/dbc/fac to d5/da/d28/d4b/da8/dbc/d62/d73/dc5/ff4 0 2024-03-22T07:16:07.147 INFO:tasks.workunit.client.0.smithi063.stdout:7/747: mknod d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/df0/cf5 0 2024-03-22T07:16:07.147 INFO:tasks.workunit.client.0.smithi063.stdout:7/748: readlink d5/da/d5f/lc2 0 2024-03-22T07:16:07.147 INFO:tasks.workunit.client.0.smithi063.stdout:7/749: creat d5/da/d28/da7/ded/ff6 x:0 0 0 2024-03-22T07:16:07.147 INFO:tasks.workunit.client.0.smithi063.stdout:8/958: dwrite d0/d26/d2d/d59/ffc [0,4194304] 0 2024-03-22T07:16:07.148 INFO:tasks.workunit.client.1.smithi069.stdout:3/718: write d5/d9/d26/d3f/d35/f58 [1241687,118739] 0 2024-03-22T07:16:07.149 INFO:tasks.workunit.client.1.smithi069.stdout:6/582: write d4/dd/d12/d13/f5e [1337536,122080] 0 2024-03-22T07:16:07.149 INFO:tasks.workunit.client.1.smithi069.stdout:3/719: read d5/d9/db/dae/faf [17198,122896] 0 2024-03-22T07:16:07.149 INFO:tasks.workunit.client.1.smithi069.stdout:3/720: chown d5/dd6 6428 1 2024-03-22T07:16:07.152 INFO:tasks.workunit.client.0.smithi063.stdout:6/941: dread d3/d6/d2a/d38/f9f [0,4194304] 0 2024-03-22T07:16:07.153 INFO:tasks.workunit.client.0.smithi063.stdout:6/942: creat d3/d15/d16/d60/d6d/d4e/f137 x:0 0 0 2024-03-22T07:16:07.154 INFO:tasks.workunit.client.1.smithi069.stdout:6/583: symlink d4/d7/lb6 0 2024-03-22T07:16:07.156 INFO:tasks.workunit.client.1.smithi069.stdout:5/595: dread d1/d6/d88/fa1 [0,4194304] 0 2024-03-22T07:16:07.156 INFO:tasks.workunit.client.1.smithi069.stdout:5/596: chown d1/d28 78507177 1 2024-03-22T07:16:07.162 INFO:tasks.workunit.client.1.smithi069.stdout:6/584: symlink d4/dd/d12/d44/d58/db4/lb7 0 2024-03-22T07:16:07.162 INFO:tasks.workunit.client.1.smithi069.stdout:5/597: creat d1/d6/d86/d99/d43/d8a/fd0 x:0 0 0 2024-03-22T07:16:07.162 INFO:tasks.workunit.client.1.smithi069.stdout:6/585: write d4/dd/d12/d44/f9b [1563911,10369] 0 2024-03-22T07:16:07.162 INFO:tasks.workunit.client.1.smithi069.stdout:6/586: chown d4/d7/da/f94 13163 1 2024-03-22T07:16:07.162 INFO:tasks.workunit.client.1.smithi069.stdout:6/587: write d4/dd/d12/d44/d58/f75 [5916370,9024] 0 2024-03-22T07:16:07.163 INFO:tasks.workunit.client.1.smithi069.stdout:5/598: mknod d1/d6/d21/cd1 0 2024-03-22T07:16:07.166 INFO:tasks.workunit.client.1.smithi069.stdout:6/588: rename d4/dd/d12/l8c to d4/dd/d41/d78/da3/lb8 0 2024-03-22T07:16:07.168 INFO:tasks.workunit.client.1.smithi069.stdout:6/589: symlink d4/dd/d12/d44/d7f/lb9 0 2024-03-22T07:16:07.169 INFO:tasks.workunit.client.1.smithi069.stdout:5/599: rename d1/d7/c57 to d1/d10/d60/cd2 0 2024-03-22T07:16:07.169 INFO:tasks.workunit.client.1.smithi069.stdout:5/600: write d1/d6/d26/d3a/f6e [887417,45670] 0 2024-03-22T07:16:07.170 INFO:tasks.workunit.client.1.smithi069.stdout:6/590: link d4/fb d4/dd/d12/d44/d7f/fba 0 2024-03-22T07:16:07.170 INFO:tasks.workunit.client.1.smithi069.stdout:6/591: creat d4/dd/d12/d2b/da5/db2/fbb x:0 0 0 2024-03-22T07:16:07.171 INFO:tasks.workunit.client.0.smithi063.stdout:6/943: write d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/d10a/fb5 [1831887,68869] 0 2024-03-22T07:16:07.172 INFO:tasks.workunit.client.1.smithi069.stdout:5/601: rename d1/d6/d86/d99/f24 to d1/d6/d26/d33/d5d/d72/d78/fd3 0 2024-03-22T07:16:07.172 INFO:tasks.workunit.client.1.smithi069.stdout:5/602: read - d1/d6/d26/d3e/f6d zero size 2024-03-22T07:16:07.173 INFO:tasks.workunit.client.1.smithi069.stdout:5/603: mknod d1/d6/d26/d3a/dbd/cd4 0 2024-03-22T07:16:07.173 INFO:tasks.workunit.client.1.smithi069.stdout:5/604: readlink d1/d6/d21/l6c 0 2024-03-22T07:16:07.173 INFO:tasks.workunit.client.1.smithi069.stdout:5/605: mkdir d1/d10/d44/d50/dd5 0 2024-03-22T07:16:07.183 INFO:tasks.workunit.client.0.smithi063.stdout:6/944: write d3/d6/d2a/d38/f4d [2928183,83958] 0 2024-03-22T07:16:07.186 INFO:tasks.workunit.client.0.smithi063.stdout:6/945: rename d3/d15/d123/f128 to d3/d15/d16/d60/d6d/f138 0 2024-03-22T07:16:07.187 INFO:tasks.workunit.client.0.smithi063.stdout:6/946: symlink d3/d6/d2a/d38/l139 0 2024-03-22T07:16:07.191 INFO:tasks.workunit.client.0.smithi063.stdout:6/947: link d3/d6/d35/d3a/fbe d3/d15/d16/d60/d6d/d4e/d5a/d88/df5/d114/d11c/f13a 0 2024-03-22T07:16:07.192 INFO:tasks.workunit.client.0.smithi063.stdout:6/948: chown d3/f12d 10207 1 2024-03-22T07:16:07.192 INFO:tasks.workunit.client.0.smithi063.stdout:6/949: creat d3/d15/d16/d60/f13b x:0 0 0 2024-03-22T07:16:07.193 INFO:tasks.workunit.client.1.smithi069.stdout:6/592: dread d4/dd/d12/d44/d7f/fba [4194304,4194304] 0 2024-03-22T07:16:07.194 INFO:tasks.workunit.client.1.smithi069.stdout:6/593: fsync d4/dd/d12/d13/f8a 0 2024-03-22T07:16:07.194 INFO:tasks.workunit.client.1.smithi069.stdout:6/594: unlink d4/dd/d12/fa9 0 2024-03-22T07:16:07.205 INFO:tasks.workunit.client.1.smithi069.stdout:9/552: dwrite d2/dd/d3c/d48/d7c/f9e [0,4194304] 0 2024-03-22T07:16:07.206 INFO:tasks.workunit.client.1.smithi069.stdout:9/553: symlink d2/dd/d3c/d2e/d84/da5/lb1 0 2024-03-22T07:16:07.214 INFO:tasks.workunit.client.0.smithi063.stdout:6/950: dread d3/d15/d16/d60/d6d/d107/f118 [0,4194304] 0 2024-03-22T07:16:07.215 INFO:tasks.workunit.client.0.smithi063.stdout:6/951: unlink d3/d6/d2a/d38/da5/d32/c46 0 2024-03-22T07:16:07.216 INFO:tasks.workunit.client.0.smithi063.stdout:6/952: creat d3/f13c x:0 0 0 2024-03-22T07:16:07.218 INFO:tasks.workunit.client.0.smithi063.stdout:6/953: unlink d3/d6/f28 0 2024-03-22T07:16:07.218 INFO:tasks.workunit.client.0.smithi063.stdout:6/954: creat d3/d6/d2a/f13d x:0 0 0 2024-03-22T07:16:07.219 INFO:tasks.workunit.client.1.smithi069.stdout:0/777: dwrite d3/d7/d15/d29/d5a/d5d/d9d/fbd [0,4194304] 0 2024-03-22T07:16:07.220 INFO:tasks.workunit.client.1.smithi069.stdout:7/662: dwrite d9/db/f24 [0,4194304] 0 2024-03-22T07:16:07.221 INFO:tasks.workunit.client.1.smithi069.stdout:2/509: dwrite f1 [0,4194304] 0 2024-03-22T07:16:07.221 INFO:tasks.workunit.client.1.smithi069.stdout:1/645: dwrite d1/d2a/d50/d4c/d55/f97 [0,4194304] 0 2024-03-22T07:16:07.221 INFO:tasks.workunit.client.1.smithi069.stdout:1/646: write d1/d13/f39 [1122378,70553] 0 2024-03-22T07:16:07.221 INFO:tasks.workunit.client.1.smithi069.stdout:1/647: truncate d1/d13/d36/d5b/d95/fad 408263 0 2024-03-22T07:16:07.222 INFO:tasks.workunit.client.1.smithi069.stdout:4/802: write d6/d9b/f86 [1038994,45676] 0 2024-03-22T07:16:07.223 INFO:tasks.workunit.client.1.smithi069.stdout:8/867: dwrite d1/d1c/d3e/db9/ff8 [0,4194304] 0 2024-03-22T07:16:07.223 INFO:tasks.workunit.client.0.smithi063.stdout:7/750: dwrite d5/f13 [4194304,4194304] 0 2024-03-22T07:16:07.223 INFO:tasks.workunit.client.0.smithi063.stdout:7/751: write d5/da/d28/d4b/da8/dbc/d62/d73/dc5/ff4 [88993,83974] 0 2024-03-22T07:16:07.223 INFO:tasks.workunit.client.0.smithi063.stdout:7/752: chown d5/d16/f32 15 1 2024-03-22T07:16:07.224 INFO:tasks.workunit.client.0.smithi063.stdout:8/959: dwrite d0/d24/d33/f5e [0,4194304] 0 2024-03-22T07:16:07.224 INFO:tasks.workunit.client.0.smithi063.stdout:8/960: creat d0/d24/d33/dd9/f142 x:0 0 0 2024-03-22T07:16:07.224 INFO:tasks.workunit.client.1.smithi069.stdout:3/721: dwrite d5/d9/db/fbb [0,4194304] 0 2024-03-22T07:16:07.224 INFO:tasks.workunit.client.1.smithi069.stdout:3/722: fdatasync d5/d9/db/d3b/f95 0 2024-03-22T07:16:07.224 INFO:tasks.workunit.client.1.smithi069.stdout:5/606: dwrite d1/d6/d21/fb1 [0,4194304] 0 2024-03-22T07:16:07.224 INFO:tasks.workunit.client.1.smithi069.stdout:5/607: chown d1/d6/d26/d3e/c6b 36296 1 2024-03-22T07:16:07.225 INFO:tasks.workunit.client.1.smithi069.stdout:0/778: creat d3/d7/f106 x:0 0 0 2024-03-22T07:16:07.225 INFO:tasks.workunit.client.0.smithi063.stdout:7/753: symlink d5/da/d5f/lf7 0 2024-03-22T07:16:07.228 INFO:tasks.workunit.client.0.smithi063.stdout:8/961: mknod d0/d26/d2d/d59/dba/dc9/ded/d11e/c143 0 2024-03-22T07:16:07.230 INFO:tasks.workunit.client.1.smithi069.stdout:7/663: creat d9/db/d48/fdb x:0 0 0 2024-03-22T07:16:07.230 INFO:tasks.workunit.client.0.smithi063.stdout:7/754: rmdir d5/da/d28/d4b/da8/dbc/d62 39 2024-03-22T07:16:07.230 INFO:tasks.workunit.client.0.smithi063.stdout:6/955: write d3/d15/d16/d60/d6d/fae [647864,98219] 0 2024-03-22T07:16:07.231 INFO:tasks.workunit.client.0.smithi063.stdout:6/956: truncate d3/d15/d16/d60/d6d/d107/d10b/dde/f120 494711 0 2024-03-22T07:16:07.232 INFO:tasks.workunit.client.0.smithi063.stdout:8/962: mknod d0/d1b/d4f/c144 0 2024-03-22T07:16:07.234 INFO:tasks.workunit.client.0.smithi063.stdout:7/755: mkdir d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d4f/df8 0 2024-03-22T07:16:07.234 INFO:tasks.workunit.client.0.smithi063.stdout:7/756: write d5/da/d5f/fb8 [5033056,22392] 0 2024-03-22T07:16:07.238 INFO:tasks.workunit.client.1.smithi069.stdout:4/803: rename d6/d9b/d1f/d58/cad to d6/d4b/de9/c116 0 2024-03-22T07:16:07.238 INFO:tasks.workunit.client.1.smithi069.stdout:4/804: stat d6/d16/d83/ff4 0 2024-03-22T07:16:07.238 INFO:tasks.workunit.client.1.smithi069.stdout:1/648: read d1/d13/d82/f83 [2057351,55588] 0 2024-03-22T07:16:07.239 INFO:tasks.workunit.client.0.smithi063.stdout:6/957: mknod d3/d15/d16/d60/d6d/c13e 0 2024-03-22T07:16:07.239 INFO:tasks.workunit.client.0.smithi063.stdout:7/757: symlink d5/da/d28/d4b/da8/dbc/d85/lf9 0 2024-03-22T07:16:07.239 INFO:tasks.workunit.client.0.smithi063.stdout:7/758: read - d5/d16/fef zero size 2024-03-22T07:16:07.240 INFO:tasks.workunit.client.1.smithi069.stdout:6/595: dwrite d4/dd/d12/d13/fb0 [0,4194304] 0 2024-03-22T07:16:07.240 INFO:tasks.workunit.client.0.smithi063.stdout:6/958: mknod d3/d15/d16/d60/d6d/d134/c13f 0 2024-03-22T07:16:07.241 INFO:tasks.workunit.client.0.smithi063.stdout:6/959: fdatasync d3/d6/d2a/d38/da5/d32/dbf/fe5 0 2024-03-22T07:16:07.241 INFO:tasks.workunit.client.0.smithi063.stdout:6/960: dread - d3/d6/d35/f101 zero size 2024-03-22T07:16:07.241 INFO:tasks.workunit.client.0.smithi063.stdout:6/961: write d3/f9 [631075,39068] 0 2024-03-22T07:16:07.243 INFO:tasks.workunit.client.0.smithi063.stdout:7/759: creat d5/da/d28/d4b/da8/dbc/d62/d8d/dbb/ffa x:0 0 0 2024-03-22T07:16:07.244 INFO:tasks.workunit.client.1.smithi069.stdout:8/868: link d1/d1c/fff d1/d1c/d3e/d66/d9e/dd1/f117 0 2024-03-22T07:16:07.246 INFO:tasks.workunit.client.1.smithi069.stdout:9/554: dwrite d2/f8f [0,4194304] 0 2024-03-22T07:16:07.246 INFO:tasks.workunit.client.1.smithi069.stdout:3/723: dread d5/d9/d26/d3f/d56/d8a/da5/f39 [0,4194304] 0 2024-03-22T07:16:07.246 INFO:tasks.workunit.client.1.smithi069.stdout:3/724: chown d5/d9/d26/d36/d41/d8e/f9f 47294688 1 2024-03-22T07:16:07.247 INFO:tasks.workunit.client.0.smithi063.stdout:5/809: sync 2024-03-22T07:16:07.254 INFO:tasks.workunit.client.1.smithi069.stdout:4/805: rename d6/dc4/db5/cef to d6/dc4/de1/c117 0 2024-03-22T07:16:07.254 INFO:tasks.workunit.client.1.smithi069.stdout:4/806: write d6/d16/d3a/db4/dfb/d1d/d55/f5f [851383,28961] 0 2024-03-22T07:16:07.256 INFO:tasks.workunit.client.0.smithi063.stdout:6/962: unlink d3/d15/d16/d60/d6d/d4e/d5a/d88/f10e 0 2024-03-22T07:16:07.256 INFO:tasks.workunit.client.0.smithi063.stdout:6/963: fdatasync d3/f19 0 2024-03-22T07:16:07.259 INFO:tasks.workunit.client.1.smithi069.stdout:8/869: rmdir d1/d1c/d3e/d88/dcd/d6c/d10c 0 2024-03-22T07:16:07.259 INFO:tasks.workunit.client.0.smithi063.stdout:7/760: mkdir d5/da/d28/d4b/da8/dbc/dfb 0 2024-03-22T07:16:07.259 INFO:tasks.workunit.client.0.smithi063.stdout:7/761: chown d5/da/d5f/d33/f53 1 1 2024-03-22T07:16:07.260 INFO:tasks.workunit.client.0.smithi063.stdout:7/762: write d5/da/d28/d4b/da8/dd4/dea/f9e [323543,121790] 0 2024-03-22T07:16:07.260 INFO:tasks.workunit.client.0.smithi063.stdout:7/763: creat d5/da/d28/da7/ffc x:0 0 0 2024-03-22T07:16:07.260 INFO:tasks.workunit.client.0.smithi063.stdout:7/764: chown d5/d16/fef 193891896 1 2024-03-22T07:16:07.260 INFO:tasks.workunit.client.0.smithi063.stdout:7/765: stat d5/d16/f2b 0 2024-03-22T07:16:07.260 INFO:tasks.workunit.client.0.smithi063.stdout:5/810: mkdir d8/de9/dff 0 2024-03-22T07:16:07.260 INFO:tasks.workunit.client.0.smithi063.stdout:5/811: chown d8/d13/d24 28 1 2024-03-22T07:16:07.260 INFO:tasks.workunit.client.0.smithi063.stdout:5/812: chown d8 12 1 2024-03-22T07:16:07.261 INFO:tasks.workunit.client.0.smithi063.stdout:6/964: mkdir d3/d6/d140 0 2024-03-22T07:16:07.263 INFO:tasks.workunit.client.1.smithi069.stdout:9/555: rmdir d2/dd/d3c/d53/d9d 39 2024-03-22T07:16:07.264 INFO:tasks.workunit.client.0.smithi063.stdout:5/813: dread d8/fd9 [0,4194304] 0 2024-03-22T07:16:07.264 INFO:tasks.workunit.client.0.smithi063.stdout:7/766: link d5/d16/d43/laf d5/da/d5f/lfd 0 2024-03-22T07:16:07.265 INFO:tasks.workunit.client.1.smithi069.stdout:0/779: mknod d3/d7/d15/c107 0 2024-03-22T07:16:07.266 INFO:tasks.workunit.client.0.smithi063.stdout:7/767: dread d5/d16/f90 [0,4194304] 0 2024-03-22T07:16:07.266 INFO:tasks.workunit.client.0.smithi063.stdout:5/814: mkdir d8/d13/d24/da4/d8b/db6/dea/d100 0 2024-03-22T07:16:07.267 INFO:tasks.workunit.client.0.smithi063.stdout:7/768: read d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/f6a [663324,50580] 0 2024-03-22T07:16:07.267 INFO:tasks.workunit.client.1.smithi069.stdout:8/870: mkdir d1/d1c/d3e/d88/dcd/d6c/d118 0 2024-03-22T07:16:07.268 INFO:tasks.workunit.client.1.smithi069.stdout:9/556: stat d2/d21/c6f 0 2024-03-22T07:16:07.269 INFO:tasks.workunit.client.0.smithi063.stdout:5/815: symlink d8/dba/l101 0 2024-03-22T07:16:07.269 INFO:tasks.workunit.client.0.smithi063.stdout:5/816: chown d8/d13/d1e/d31/d64/d65/d86/le6 4304 1 2024-03-22T07:16:07.270 INFO:tasks.workunit.client.0.smithi063.stdout:7/769: link d5/da/d28/da7/l3d d5/da/d28/d4b/da8/dd4/lfe 0 2024-03-22T07:16:07.272 INFO:tasks.workunit.client.1.smithi069.stdout:8/871: unlink d1/d1c/d3e/d66/f95 0 2024-03-22T07:16:07.273 INFO:tasks.workunit.client.0.smithi063.stdout:7/770: truncate d5/da/d5f/f8a 1217743 0 2024-03-22T07:16:07.273 INFO:tasks.workunit.client.0.smithi063.stdout:7/771: write d5/d16/d1b/f7c [184982,52992] 0 2024-03-22T07:16:07.275 INFO:tasks.workunit.client.1.smithi069.stdout:0/780: rename d3/d7/d15/c107 to d3/d7/d15/c108 0 2024-03-22T07:16:07.275 INFO:tasks.workunit.client.0.smithi063.stdout:5/817: rename d8/d13/c54 to d8/d13/d24/da4/d8b/c102 0 2024-03-22T07:16:07.276 INFO:tasks.workunit.client.1.smithi069.stdout:8/872: creat d1/d1c/d70/f119 x:0 0 0 2024-03-22T07:16:07.277 INFO:tasks.workunit.client.1.smithi069.stdout:8/873: chown d1/d1c/f52 123 1 2024-03-22T07:16:07.277 INFO:tasks.workunit.client.1.smithi069.stdout:0/781: mknod d3/d7/d15/d29/d6e/c109 0 2024-03-22T07:16:07.278 INFO:tasks.workunit.client.1.smithi069.stdout:0/782: fsync d3/d7/d15/fe1 0 2024-03-22T07:16:07.278 INFO:tasks.workunit.client.0.smithi063.stdout:7/772: link d5/d16/f2b d5/da/d28/d4b/da8/dd4/fff 0 2024-03-22T07:16:07.278 INFO:tasks.workunit.client.0.smithi063.stdout:7/773: chown d5/f17 179605 1 2024-03-22T07:16:07.278 INFO:tasks.workunit.client.0.smithi063.stdout:7/774: chown d5/da/d28/d4b/da8/dbc/d62/d73/f7a 121 1 2024-03-22T07:16:07.279 INFO:tasks.workunit.client.1.smithi069.stdout:8/874: rename d1/d1c/d3e/d88/dcd/d20/ca2 to d1/d1c/d3e/d88/dcd/d6c/d4e/d98/c11a 0 2024-03-22T07:16:07.280 INFO:tasks.workunit.client.0.smithi063.stdout:5/818: creat d8/d13/d1e/d31/d64/d65/ddf/f103 x:0 0 0 2024-03-22T07:16:07.284 INFO:tasks.workunit.client.1.smithi069.stdout:7/664: dwrite d9/db/d1a/db1/d57/d75/d7a/fad [0,4194304] 0 2024-03-22T07:16:07.295 INFO:tasks.workunit.client.1.smithi069.stdout:0/783: write d3/d7/fb [4633854,2364] 0 2024-03-22T07:16:07.298 INFO:tasks.workunit.client.1.smithi069.stdout:7/665: write d9/db/d1a/db1/d57/d44/d53/d74/f91 [3808293,91435] 0 2024-03-22T07:16:07.299 INFO:tasks.workunit.client.1.smithi069.stdout:2/510: dwrite fa [0,4194304] 0 2024-03-22T07:16:07.299 INFO:tasks.workunit.client.1.smithi069.stdout:2/511: write d18/f3a [2293679,89901] 0 2024-03-22T07:16:07.299 INFO:tasks.workunit.client.1.smithi069.stdout:5/608: dwrite d1/d10/d60/f5f [0,4194304] 0 2024-03-22T07:16:07.299 INFO:tasks.workunit.client.1.smithi069.stdout:5/609: fdatasync d1/d6/d86/d99/f12 0 2024-03-22T07:16:07.299 INFO:tasks.workunit.client.1.smithi069.stdout:0/784: rename d3/d7/d15/dce/cea to d3/d7/d17/d1c/dd6/c10a 0 2024-03-22T07:16:07.299 INFO:tasks.workunit.client.1.smithi069.stdout:0/785: write d3/d7/d17/d1c/f91 [2772182,79885] 0 2024-03-22T07:16:07.309 INFO:tasks.workunit.client.1.smithi069.stdout:5/610: mkdir d1/d6/d26/d33/dd6 0 2024-03-22T07:16:07.309 INFO:tasks.workunit.client.1.smithi069.stdout:5/611: creat d1/d6/d88/fd7 x:0 0 0 2024-03-22T07:16:07.310 INFO:tasks.workunit.client.1.smithi069.stdout:0/786: mknod d3/d7/d15/d29/d5a/d5d/d9d/df9/c10b 0 2024-03-22T07:16:07.310 INFO:tasks.workunit.client.1.smithi069.stdout:0/787: write d3/d7/d15/fe1 [677774,66495] 0 2024-03-22T07:16:07.312 INFO:tasks.workunit.client.1.smithi069.stdout:5/612: mkdir d1/d10/d44/dd8 0 2024-03-22T07:16:07.313 INFO:tasks.workunit.client.1.smithi069.stdout:0/788: truncate d3/d7/d17/f56 2759579 0 2024-03-22T07:16:07.313 INFO:tasks.workunit.client.1.smithi069.stdout:0/789: chown d3/d7/f106 53086 1 2024-03-22T07:16:07.313 INFO:tasks.workunit.client.1.smithi069.stdout:0/790: fsync d3/d7/d17/f6b 0 2024-03-22T07:16:07.314 INFO:tasks.workunit.client.1.smithi069.stdout:5/613: mknod d1/d6/d21/dae/cd9 0 2024-03-22T07:16:07.314 INFO:tasks.workunit.client.1.smithi069.stdout:5/614: getdents d1/d10/d44/d50/dd5 0 2024-03-22T07:16:07.315 INFO:tasks.workunit.client.1.smithi069.stdout:5/615: creat d1/d6/d26/d7d/fda x:0 0 0 2024-03-22T07:16:07.315 INFO:tasks.workunit.client.1.smithi069.stdout:5/616: mknod d1/d6/d26/d3e/d9c/cdb 0 2024-03-22T07:16:07.315 INFO:tasks.workunit.client.1.smithi069.stdout:5/617: readlink d1/d6/d86/d99/lc 0 2024-03-22T07:16:07.338 INFO:tasks.workunit.client.1.smithi069.stdout:3/725: dwrite d5/d9/d26/d3f/d35/f8d [0,4194304] 0 2024-03-22T07:16:07.338 INFO:tasks.workunit.client.1.smithi069.stdout:6/596: dwrite d4/d7/da/f7c [0,4194304] 0 2024-03-22T07:16:07.338 INFO:tasks.workunit.client.1.smithi069.stdout:9/557: dwrite d2/dd/d3c/d2e/f32 [4194304,4194304] 0 2024-03-22T07:16:07.343 INFO:tasks.workunit.client.0.smithi063.stdout:6/965: truncate d3/d15/d16/d60/d6d/d107/d10b/dde/f120 276456 0 2024-03-22T07:16:07.345 INFO:tasks.workunit.client.1.smithi069.stdout:3/726: link d5/d9/d26/d36/d41/d51/f7c d5/d9/d26/d3f/d54/feb 0 2024-03-22T07:16:07.345 INFO:tasks.workunit.client.1.smithi069.stdout:3/727: read - d5/da2/fdc zero size 2024-03-22T07:16:07.345 INFO:tasks.workunit.client.1.smithi069.stdout:3/728: chown d5/d9/d26/d3f/d35/f50 5752637 1 2024-03-22T07:16:07.345 INFO:tasks.workunit.client.1.smithi069.stdout:0/791: dwrite d3/d7/d15/d29/d5a/d84/f60 [0,4194304] 0 2024-03-22T07:16:07.345 INFO:tasks.workunit.client.1.smithi069.stdout:0/792: chown d3/d7/d15/d29/d5a/d5d/db4/d100 11569227 1 2024-03-22T07:16:07.346 INFO:tasks.workunit.client.1.smithi069.stdout:0/793: chown d3/d7/d15/d29/d5a/d5d/l9c 9 1 2024-03-22T07:16:07.346 INFO:tasks.workunit.client.1.smithi069.stdout:0/794: dread - d3/d7/d15/d29/d5a/d99/fbf zero size 2024-03-22T07:16:07.346 INFO:tasks.workunit.client.1.smithi069.stdout:0/795: creat d3/d7/d24/d6a/f10c x:0 0 0 2024-03-22T07:16:07.348 INFO:tasks.workunit.client.1.smithi069.stdout:3/729: mkdir d5/d9/d26/d3f/dec 0 2024-03-22T07:16:07.348 INFO:tasks.workunit.client.1.smithi069.stdout:3/730: write d5/d9/d26/d3f/d35/f6b [9161394,104504] 0 2024-03-22T07:16:07.349 INFO:tasks.workunit.client.1.smithi069.stdout:8/875: dwrite d1/d1c/d70/f119 [0,4194304] 0 2024-03-22T07:16:07.350 INFO:tasks.workunit.client.0.smithi063.stdout:5/819: dwrite d8/d13/fdb [0,4194304] 0 2024-03-22T07:16:07.351 INFO:tasks.workunit.client.1.smithi069.stdout:3/731: dread d5/d9/d26/fac [0,4194304] 0 2024-03-22T07:16:07.351 INFO:tasks.workunit.client.1.smithi069.stdout:3/732: chown d5/d9/d26/d3f/d54/l85 23 1 2024-03-22T07:16:07.351 INFO:tasks.workunit.client.1.smithi069.stdout:0/796: rmdir d3/d7/d15/d29/d5a/d5d/db4/ded 39 2024-03-22T07:16:07.352 INFO:tasks.workunit.client.0.smithi063.stdout:5/820: link d8/d13/d1e/d31/ceb d8/dba/c104 0 2024-03-22T07:16:07.353 INFO:tasks.workunit.client.1.smithi069.stdout:8/876: mknod d1/d1c/d3e/dde/d10a/c11b 0 2024-03-22T07:16:07.354 INFO:tasks.workunit.client.0.smithi063.stdout:5/821: mknod d8/d13/d24/da4/d8b/db6/dde/c105 0 2024-03-22T07:16:07.356 INFO:tasks.workunit.client.0.smithi063.stdout:5/822: link d8/d13/d24/da4/d8b/ce0 d8/d13/c106 0 2024-03-22T07:16:07.356 INFO:tasks.workunit.client.1.smithi069.stdout:3/733: mkdir d5/d9/d26/d3f/d54/d9d/ded 0 2024-03-22T07:16:07.356 INFO:tasks.workunit.client.1.smithi069.stdout:3/734: write d5/d9/d23/f6d [5116911,100688] 0 2024-03-22T07:16:07.356 INFO:tasks.workunit.client.0.smithi063.stdout:5/823: symlink d8/d13/d24/da4/l107 0 2024-03-22T07:16:07.358 INFO:tasks.workunit.client.0.smithi063.stdout:7/775: dwrite d5/f13 [4194304,4194304] 0 2024-03-22T07:16:07.358 INFO:tasks.workunit.client.0.smithi063.stdout:5/824: truncate d8/d13/d28/fbc 51061 0 2024-03-22T07:16:07.358 INFO:tasks.workunit.client.0.smithi063.stdout:7/776: chown d5/d16/d43/d78/lcb 10 1 2024-03-22T07:16:07.358 INFO:tasks.workunit.client.0.smithi063.stdout:7/777: chown d5/da/d28/d4b/da8/dbc/d62/d73/d1d/lec 3409 1 2024-03-22T07:16:07.358 INFO:tasks.workunit.client.0.smithi063.stdout:7/778: getdents d5/d38 0 2024-03-22T07:16:07.359 INFO:tasks.workunit.client.1.smithi069.stdout:3/735: mknod d5/d9/d26/d3f/dc1/cee 0 2024-03-22T07:16:07.359 INFO:tasks.workunit.client.1.smithi069.stdout:7/666: dwrite d9/db/d48/d4b/f65 [0,4194304] 0 2024-03-22T07:16:07.360 INFO:tasks.workunit.client.1.smithi069.stdout:3/736: mknod d5/d9/d23/cef 0 2024-03-22T07:16:07.362 INFO:tasks.workunit.client.0.smithi063.stdout:6/966: dread d3/d15/d16/d60/d6d/d4e/d5a/d88/dbb/f5b [0,4194304] 0 2024-03-22T07:16:07.364 INFO:tasks.workunit.client.0.smithi063.stdout:5/825: mkdir d8/d13/d28/d57/dfd/d108 0 2024-03-22T07:16:07.364 INFO:tasks.workunit.client.0.smithi063.stdout:5/826: readlink d8/d13/d1e/d31/d64/d65/d86/lc1 0 2024-03-22T07:16:07.364 INFO:tasks.workunit.client.0.smithi063.stdout:7/779: rmdir d5/da/d28/d4b/da8/dd4 39 2024-03-22T07:16:07.365 INFO:tasks.workunit.client.0.smithi063.stdout:7/780: chown d5/da/d5f/c6b 838902 1 2024-03-22T07:16:07.365 INFO:tasks.workunit.client.0.smithi063.stdout:7/781: fsync d5/da/d28/d4b/da8/fc8 0 2024-03-22T07:16:07.365 INFO:tasks.workunit.client.1.smithi069.stdout:0/797: read d3/d7/d15/f32 [1423568,111190] 0 2024-03-22T07:16:07.367 INFO:tasks.workunit.client.0.smithi063.stdout:7/782: dread d5/f9 [0,4194304] 0 2024-03-22T07:16:07.367 INFO:tasks.workunit.client.0.smithi063.stdout:7/783: fsync d5/d16/d1b/f52 0 2024-03-22T07:16:07.368 INFO:tasks.workunit.client.0.smithi063.stdout:5/827: link d8/d13/d28/fbd d8/d13/d1e/d3c/da0/f109 0 2024-03-22T07:16:07.369 INFO:tasks.workunit.client.1.smithi069.stdout:7/667: rename d9/c15 to d9/db/d1a/cdc 0 2024-03-22T07:16:07.371 INFO:tasks.workunit.client.1.smithi069.stdout:0/798: creat d3/d7/d15/d29/d5a/d5d/db4/f10d x:0 0 0 2024-03-22T07:16:07.373 INFO:tasks.workunit.client.1.smithi069.stdout:7/668: symlink d9/db/d48/d4b/dbd/ldd 0 2024-03-22T07:16:07.374 INFO:tasks.workunit.client.0.smithi063.stdout:6/967: write d3/d15/d16/d60/d6d/d107/d10b/fa9 [4066933,20992] 0 2024-03-22T07:16:07.374 INFO:tasks.workunit.client.1.smithi069.stdout:7/669: mknod d9/db/d1a/db1/d57/d75/cde 0 2024-03-22T07:16:07.376 INFO:tasks.workunit.client.1.smithi069.stdout:7/670: rename d9/d8e/fc5 to d9/db/d1a/db1/d57/d44/d53/d1d/fdf 0 2024-03-22T07:16:07.376 INFO:tasks.workunit.client.1.smithi069.stdout:7/671: fdatasync d9/db/d1a/db1/d57/d44/f47 0 2024-03-22T07:16:07.378 INFO:tasks.workunit.client.0.smithi063.stdout:6/968: rename d3/d15/d16/d60/d6d/d4c/fd5 to d3/d15/d16/d60/d6d/d4e/d5a/f141 0 2024-03-22T07:16:07.378 INFO:tasks.workunit.client.1.smithi069.stdout:7/672: creat d9/db/d1a/db1/d57/d44/d53/d74/fe0 x:0 0 0 2024-03-22T07:16:07.381 INFO:tasks.workunit.client.0.smithi063.stdout:6/969: creat d3/d15/d16/d60/d7b/dd0/f142 x:0 0 0 2024-03-22T07:16:07.381 INFO:tasks.workunit.client.0.smithi063.stdout:6/970: chown d3/d6/d2a/d38/da5/d32/d51/l6e 10525 1 2024-03-22T07:16:07.386 INFO:tasks.workunit.client.1.smithi069.stdout:5/618: dwrite d1/d6/d26/d3e/f8d [0,4194304] 0 2024-03-22T07:16:07.386 INFO:tasks.workunit.client.1.smithi069.stdout:5/619: fdatasync d1/d6/d86/d99/f52 0 2024-03-22T07:16:07.389 INFO:tasks.workunit.client.0.smithi063.stdout:6/971: dread d3/d15/d16/d60/d6d/d4e/d5a/d88/fc1 [0,4194304] 0 2024-03-22T07:16:07.391 INFO:tasks.workunit.client.0.smithi063.stdout:6/972: unlink d3/d15/d16/d69/d8c/f132 0 2024-03-22T07:16:07.394 INFO:tasks.workunit.client.0.smithi063.stdout:6/973: mknod d3/d15/d16/d60/d6d/d4e/d5a/c143 0 2024-03-22T07:16:07.394 INFO:tasks.workunit.client.0.smithi063.stdout:6/974: creat d3/d15/d16/f144 x:0 0 0 2024-03-22T07:16:07.398 INFO:tasks.workunit.client.1.smithi069.stdout:5/620: dread d1/d6/d26/d33/d5d/d72/d78/fd3 [0,4194304] 0 2024-03-22T07:16:07.408 INFO:tasks.workunit.client.0.smithi063.stdout:6/975: write d3/d6/d2a/d38/da5/fdc [810444,56508] 0 2024-03-22T07:16:07.409 INFO:tasks.workunit.client.0.smithi063.stdout:6/976: read d3/d15/d16/d60/d6d/d4e/fa7 [586222,10467] 0 2024-03-22T07:16:07.409 INFO:tasks.workunit.client.0.smithi063.stdout:6/977: write d3/d6/dcd/fce [942904,99283] 0 2024-03-22T07:16:07.410 INFO:tasks.workunit.client.0.smithi063.stdout:6/978: readlink d3/d6/ld 0 2024-03-22T07:16:07.411 INFO:tasks.workunit.client.0.smithi063.stdout:6/979: dread - d3/d15/d16/d60/d6d/d4e/fd2 zero size 2024-03-22T07:16:07.411 INFO:tasks.workunit.client.0.smithi063.stdout:6/980: creat d3/d15/d16/d60/d6d/d107/d10b/dde/f145 x:0 0 0 2024-03-22T07:16:07.417 INFO:tasks.workunit.client.1.smithi069.stdout:9/558: dwrite d2/f55 [0,4194304] 0 2024-03-22T07:16:07.418 INFO:tasks.workunit.client.1.smithi069.stdout:6/597: dwrite d4/dd/d12/d2b/f30 [0,4194304] 0 2024-03-22T07:16:07.423 INFO:tasks.workunit.client.1.smithi069.stdout:6/598: creat d4/dd/d12/d44/d58/db4/fbc x:0 0 0 2024-03-22T07:16:07.423 INFO:tasks.workunit.client.1.smithi069.stdout:6/599: write d4/f14 [4068774,124015] 0 2024-03-22T07:16:07.423 INFO:tasks.workunit.client.1.smithi069.stdout:3/737: dwrite d5/d9/d23/f78 [0,4194304] 0 2024-03-22T07:16:07.424 INFO:tasks.workunit.client.0.smithi063.stdout:6/981: dread d3/d6/d2a/d38/da5/d32/dbf/fe5 [0,4194304] 0 2024-03-22T07:16:07.424 INFO:tasks.workunit.client.0.smithi063.stdout:6/982: write d3/d15/d16/d60/d6d/d4e/d5a/d88/df5/d114/d11c/f13a [287715,10349] 0 2024-03-22T07:16:07.424 INFO:tasks.workunit.client.0.smithi063.stdout:6/983: dread - d3/d6/d2a/d38/da5/d32/f12c zero size 2024-03-22T07:16:07.424 INFO:tasks.workunit.client.0.smithi063.stdout:8/963: sync 2024-03-22T07:16:07.424 INFO:tasks.workunit.client.0.smithi063.stdout:9/940: sync 2024-03-22T07:16:07.424 INFO:tasks.workunit.client.0.smithi063.stdout:4/831: sync 2024-03-22T07:16:07.424 INFO:tasks.workunit.client.0.smithi063.stdout:9/941: creat d8/d19/d20/d22/d56/f123 x:0 0 0 2024-03-22T07:16:07.424 INFO:tasks.workunit.client.0.smithi063.stdout:2/855: sync 2024-03-22T07:16:07.424 INFO:tasks.workunit.client.0.smithi063.stdout:2/856: write d0/de/d12/d20/f70 [344271,56910] 0 2024-03-22T07:16:07.424 INFO:tasks.workunit.client.0.smithi063.stdout:9/942: write d8/d19/d2b/ff6 [489226,77667] 0 2024-03-22T07:16:07.425 INFO:tasks.workunit.client.0.smithi063.stdout:2/857: write d0/f51 [3598170,18350] 0 2024-03-22T07:16:07.425 INFO:tasks.workunit.client.0.smithi063.stdout:9/943: chown d8/d19/d20/d22/db2/led 8895 1 2024-03-22T07:16:07.426 INFO:tasks.workunit.client.0.smithi063.stdout:7/784: dwrite d5/da/d5f/d33/fa3 [0,4194304] 0 2024-03-22T07:16:07.426 INFO:tasks.workunit.client.0.smithi063.stdout:6/984: rename d3/d15/f23 to d3/d6/d2a/d38/da5/d32/f146 0 2024-03-22T07:16:07.426 INFO:tasks.workunit.client.0.smithi063.stdout:5/828: dwrite d8/f12 [4194304,4194304] 0 2024-03-22T07:16:07.426 INFO:tasks.workunit.client.0.smithi063.stdout:6/985: write d3/d15/d16/d60/d6d/d107/d10b/dde/f145 [714712,90763] 0 2024-03-22T07:16:07.426 INFO:tasks.workunit.client.0.smithi063.stdout:6/986: dread - d3/d15/d16/f144 zero size 2024-03-22T07:16:07.427 INFO:tasks.workunit.client.1.smithi069.stdout:0/799: dwrite d3/d7/d15/fa3 [0,4194304] 0 2024-03-22T07:16:07.427 INFO:tasks.workunit.client.0.smithi063.stdout:8/964: creat d0/d26/d2d/d59/dc0/d106/d13a/f145 x:0 0 0 2024-03-22T07:16:07.428 INFO:tasks.workunit.client.1.smithi069.stdout:5/621: dwrite d1/d6/f6f [0,4194304] 0 2024-03-22T07:16:07.431 INFO:tasks.workunit.client.0.smithi063.stdout:4/832: symlink d1/d5/d43/d27/l11b 0 2024-03-22T07:16:07.439 INFO:tasks.workunit.client.0.smithi063.stdout:2/858: link d0/l7 d0/de/d58/d7f/d97/da2/d104/l11c 0 2024-03-22T07:16:07.439 INFO:tasks.workunit.client.0.smithi063.stdout:9/944: creat d8/d19/d20/d22/de1/dd7/dd8/f124 x:0 0 0 2024-03-22T07:16:07.442 INFO:tasks.workunit.client.1.smithi069.stdout:5/622: dread d1/d6/fa8 [0,4194304] 0 2024-03-22T07:16:07.443 INFO:tasks.workunit.client.1.smithi069.stdout:5/623: readlink d1/d28/l82 0 2024-03-22T07:16:07.444 INFO:tasks.workunit.client.0.smithi063.stdout:7/785: creat d5/da/d28/d4b/da8/dbc/d62/d73/f100 x:0 0 0 2024-03-22T07:16:07.445 INFO:tasks.workunit.client.0.smithi063.stdout:7/786: write d5/da/d28/d4b/da8/fc8 [86528,22920] 0 2024-03-22T07:16:07.445 INFO:tasks.workunit.client.0.smithi063.stdout:5/829: creat d8/d5e/f10a x:0 0 0 2024-03-22T07:16:07.445 INFO:tasks.workunit.client.0.smithi063.stdout:5/830: chown d8/d13/d24/da4/d8b/fb5 44267 1 2024-03-22T07:16:07.446 INFO:tasks.workunit.client.1.smithi069.stdout:0/800: creat d3/d7/d17/d1c/dd4/f10e x:0 0 0 2024-03-22T07:16:07.447 INFO:tasks.workunit.client.0.smithi063.stdout:4/833: creat d1/d5/d3c/d8f/dbb/d114/f11c x:0 0 0 2024-03-22T07:16:07.449 INFO:tasks.workunit.client.0.smithi063.stdout:2/859: unlink d0/de/d1b/d32/dac/l115 0 2024-03-22T07:16:07.450 INFO:tasks.workunit.client.1.smithi069.stdout:0/801: mkdir d3/d7/d15/d29/d5a/d84/d10f 0 2024-03-22T07:16:07.450 INFO:tasks.workunit.client.1.smithi069.stdout:0/802: fdatasync d3/d7/d17/d1c/fb5 0 2024-03-22T07:16:07.450 INFO:tasks.workunit.client.0.smithi063.stdout:7/787: mkdir d5/da/d5f/d101 0 2024-03-22T07:16:07.450 INFO:tasks.workunit.client.0.smithi063.stdout:7/788: truncate d5/da/d4e/fd5 89816 0 2024-03-22T07:16:07.450 INFO:tasks.workunit.client.0.smithi063.stdout:7/789: read - d5/da/d5f/f97 zero size 2024-03-22T07:16:07.452 INFO:tasks.workunit.client.0.smithi063.stdout:5/831: link d8/lc d8/d13/d1e/d31/d64/de7/l10b 0 2024-03-22T07:16:07.453 INFO:tasks.workunit.client.0.smithi063.stdout:5/832: write d8/d13/d28/f73 [4026577,15130] 0 2024-03-22T07:16:07.453 INFO:tasks.workunit.client.1.smithi069.stdout:5/624: rmdir d1/d6/d26/d33 39 2024-03-22T07:16:07.454 INFO:tasks.workunit.client.0.smithi063.stdout:4/834: creat d1/d5/d43/d27/d2c/df4/d10b/f11d x:0 0 0 2024-03-22T07:16:07.454 INFO:tasks.workunit.client.1.smithi069.stdout:5/625: unlink d1/d6/d26/d3e/c6b 0 2024-03-22T07:16:07.455 INFO:tasks.workunit.client.0.smithi063.stdout:2/860: symlink d0/de/d58/d7f/d97/da2/d104/l11d 0 2024-03-22T07:16:07.456 INFO:tasks.workunit.client.0.smithi063.stdout:7/790: symlink d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/l102 0 2024-03-22T07:16:07.456 INFO:tasks.workunit.client.0.smithi063.stdout:7/791: stat d5/da/d28/da7/fb2 0 2024-03-22T07:16:07.458 INFO:tasks.workunit.client.0.smithi063.stdout:5/833: mknod d8/de9/c10c 0 2024-03-22T07:16:07.461 INFO:tasks.workunit.client.0.smithi063.stdout:5/834: mknod d8/d13/d28/df3/c10d 0 2024-03-22T07:16:07.461 INFO:tasks.workunit.client.0.smithi063.stdout:5/835: truncate d8/d13/d28/d57/d5f/fe8 927888 0 2024-03-22T07:16:07.461 INFO:tasks.workunit.client.0.smithi063.stdout:5/836: read - d8/d13/d1e/d3c/da0/fcf zero size 2024-03-22T07:16:07.461 INFO:tasks.workunit.client.0.smithi063.stdout:5/837: chown d8/d13/d1e/d31/d64/fef 63926 1 2024-03-22T07:16:07.462 INFO:tasks.workunit.client.0.smithi063.stdout:5/838: symlink d8/d13/d28/df3/l10e 0 2024-03-22T07:16:07.465 INFO:tasks.workunit.client.0.smithi063.stdout:5/839: rename d8/d13/d28/df3/c10d to d8/d13/d1e/d31/d64/d65/d86/c10f 0 2024-03-22T07:16:07.467 INFO:tasks.workunit.client.0.smithi063.stdout:5/840: link d8/dba/le5 d8/d13/d1e/d3c/l110 0 2024-03-22T07:16:07.467 INFO:tasks.workunit.client.0.smithi063.stdout:5/841: rename d8/d13/d1e/d31 to d8/d13/d1e/d31/d111 22 2024-03-22T07:16:07.471 INFO:tasks.workunit.client.0.smithi063.stdout:4/835: read d1/d5/d43/d27/d2c/d4a/f9d [2779227,53900] 0 2024-03-22T07:16:07.473 INFO:tasks.workunit.client.0.smithi063.stdout:4/836: mkdir d1/d5/dc/d95/da4/d11e 0 2024-03-22T07:16:07.473 INFO:tasks.workunit.client.0.smithi063.stdout:4/837: creat d1/d5/d43/d27/d2c/d4a/d80/f11f x:0 0 0 2024-03-22T07:16:07.476 INFO:tasks.workunit.client.0.smithi063.stdout:4/838: creat d1/d5/dc/d95/da4/d104/f120 x:0 0 0 2024-03-22T07:16:07.476 INFO:tasks.workunit.client.0.smithi063.stdout:4/839: readlink d1/d5/d3c/d8f/l9e 0 2024-03-22T07:16:07.477 INFO:tasks.workunit.client.0.smithi063.stdout:4/840: mknod d1/d5/d3c/d8f/dbb/d114/d79/daa/c121 0 2024-03-22T07:16:07.511 INFO:tasks.workunit.client.1.smithi069.stdout:3/738: dwrite d5/d9/d26/d36/d41/f72 [4194304,4194304] 0 2024-03-22T07:16:07.513 INFO:tasks.workunit.client.1.smithi069.stdout:3/739: rename d5/d9/d26/d3f/d56/d8a/da5/d7b/dbc/fc2 to d5/d9/d26/d3f/d56/d7f/d90/ff0 0 2024-03-22T07:16:07.515 INFO:tasks.workunit.client.1.smithi069.stdout:3/740: rename d5/d9/d26/l29 to d5/d9/d26/d3f/d54/d9d/lf1 0 2024-03-22T07:16:07.515 INFO:tasks.workunit.client.1.smithi069.stdout:3/741: fsync d5/d9/d26/d3f/d35/fb6 0 2024-03-22T07:16:07.515 INFO:tasks.workunit.client.1.smithi069.stdout:3/742: write d5/d9/d26/d36/f52 [2201236,39666] 0 2024-03-22T07:16:07.515 INFO:tasks.workunit.client.1.smithi069.stdout:3/743: chown d5/d9/db/dae/fe9 0 1 2024-03-22T07:16:07.515 INFO:tasks.workunit.client.1.smithi069.stdout:3/744: symlink d5/da2/ddd/lf2 0 2024-03-22T07:16:07.547 INFO:tasks.workunit.client.1.smithi069.stdout:6/600: dwrite d4/f60 [0,4194304] 0 2024-03-22T07:16:07.548 INFO:tasks.workunit.client.1.smithi069.stdout:6/601: chown d4/dd/d12/d13/fa1 1 1 2024-03-22T07:16:07.549 INFO:tasks.workunit.client.0.smithi063.stdout:6/987: dwrite d3/d15/da3/dc2/f112 [0,4194304] 0 2024-03-22T07:16:07.549 INFO:tasks.workunit.client.0.smithi063.stdout:9/945: dwrite d8/f90 [0,4194304] 0 2024-03-22T07:16:07.549 INFO:tasks.workunit.client.0.smithi063.stdout:8/965: dwrite d0/d1b/d28/d3e/f13b [0,4194304] 0 2024-03-22T07:16:07.551 INFO:tasks.workunit.client.0.smithi063.stdout:6/988: mkdir d3/d15/d16/d60/d6d/d4c/de8/d147 0 2024-03-22T07:16:07.557 INFO:tasks.workunit.client.1.smithi069.stdout:6/602: unlink d4/d7/l4f 0 2024-03-22T07:16:07.557 INFO:tasks.workunit.client.1.smithi069.stdout:6/603: mkdir d4/dd/dbd 0 2024-03-22T07:16:07.557 INFO:tasks.workunit.client.1.smithi069.stdout:6/604: write d4/d7/f6e [607960,46496] 0 2024-03-22T07:16:07.558 INFO:tasks.workunit.client.0.smithi063.stdout:9/946: symlink d8/d19/d39/l125 0 2024-03-22T07:16:07.558 INFO:tasks.workunit.client.0.smithi063.stdout:8/966: mkdir d0/d24/d33/d146 0 2024-03-22T07:16:07.558 INFO:tasks.workunit.client.0.smithi063.stdout:8/967: chown d0/d1b/d28/d5f/fe7 334606 1 2024-03-22T07:16:07.558 INFO:tasks.workunit.client.0.smithi063.stdout:8/968: truncate d0/d1b/d28/d3e/db2/dbd/dd0/f10c 373592 0 2024-03-22T07:16:07.558 INFO:tasks.workunit.client.0.smithi063.stdout:8/969: dread - d0/d24/d33/dd9/f142 zero size 2024-03-22T07:16:07.558 INFO:tasks.workunit.client.0.smithi063.stdout:8/970: truncate d0/d24/d33/dd9/f136 118291 0 2024-03-22T07:16:07.558 INFO:tasks.workunit.client.0.smithi063.stdout:6/989: creat d3/d6/dcd/f148 x:0 0 0 2024-03-22T07:16:07.558 INFO:tasks.workunit.client.0.smithi063.stdout:9/947: symlink d8/d19/d20/d22/de1/db7/df9/l126 0 2024-03-22T07:16:07.560 INFO:tasks.workunit.client.0.smithi063.stdout:8/971: unlink d0/d1b/d4f/dde/df5/c68 0 2024-03-22T07:16:07.560 INFO:tasks.workunit.client.0.smithi063.stdout:8/972: write d0/f89 [9278896,98112] 0 2024-03-22T07:16:07.561 INFO:tasks.workunit.client.1.smithi069.stdout:6/605: dread d4/dd/d12/d2b/f36 [4194304,4194304] 0 2024-03-22T07:16:07.561 INFO:tasks.workunit.client.1.smithi069.stdout:6/606: dread - d4/dd/d41/d78/f9c zero size 2024-03-22T07:16:07.570 INFO:tasks.workunit.client.1.smithi069.stdout:6/607: mknod d4/dd/d41/d78/cbe 0 2024-03-22T07:16:07.570 INFO:tasks.workunit.client.1.smithi069.stdout:6/608: chown d4/d7/da/d3f/f99 3948696 1 2024-03-22T07:16:07.571 INFO:tasks.workunit.client.0.smithi063.stdout:2/861: dwrite d0/de/d1b/d8b/f119 [0,4194304] 0 2024-03-22T07:16:07.571 INFO:tasks.workunit.client.0.smithi063.stdout:2/862: write d0/de/d1b/d63/d6e/f95 [866021,89253] 0 2024-03-22T07:16:07.572 INFO:tasks.workunit.client.1.smithi069.stdout:5/626: dwrite d1/d6/d88/f9b [0,4194304] 0 2024-03-22T07:16:07.572 INFO:tasks.workunit.client.1.smithi069.stdout:5/627: stat d1/d6/d26/d3e/d9c/d9d/fa6 0 2024-03-22T07:16:07.572 INFO:tasks.workunit.client.1.smithi069.stdout:6/609: symlink d4/dd/d12/d2b/da5/lbf 0 2024-03-22T07:16:07.572 INFO:tasks.workunit.client.1.smithi069.stdout:6/610: symlink d4/dd/dbd/lc0 0 2024-03-22T07:16:07.575 INFO:tasks.workunit.client.0.smithi063.stdout:2/863: dread d0/de/d35/d36/fb9 [0,4194304] 0 2024-03-22T07:16:07.575 INFO:tasks.workunit.client.0.smithi063.stdout:2/864: dread - d0/de/d12/d5a/f9d zero size 2024-03-22T07:16:07.575 INFO:tasks.workunit.client.0.smithi063.stdout:5/842: dwrite d8/d13/f48 [0,4194304] 0 2024-03-22T07:16:07.575 INFO:tasks.workunit.client.0.smithi063.stdout:5/843: creat d8/d13/d28/d57/d5f/f112 x:0 0 0 2024-03-22T07:16:07.575 INFO:tasks.workunit.client.0.smithi063.stdout:4/841: dwrite d1/d5/d3c/d65/f73 [0,4194304] 0 2024-03-22T07:16:07.576 INFO:tasks.workunit.client.1.smithi069.stdout:3/745: dwrite d5/da2/fdc [0,4194304] 0 2024-03-22T07:16:07.576 INFO:tasks.workunit.client.1.smithi069.stdout:3/746: chown d5/d9/d23/laa 19506900 1 2024-03-22T07:16:07.577 INFO:tasks.workunit.client.0.smithi063.stdout:7/792: dwrite d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/d4a/f7e [0,4194304] 0 2024-03-22T07:16:07.577 INFO:tasks.workunit.client.1.smithi069.stdout:5/628: write d1/d6/d21/fb1 [355836,40748] 0 2024-03-22T07:16:07.588 INFO:tasks.workunit.client.0.smithi063.stdout:5/844: creat d8/d13/d1e/d3c/f113 x:0 0 0 2024-03-22T07:16:07.593 INFO:tasks.workunit.client.0.smithi063.stdout:2/865: rename d0/de/d58 to d0/de/d74/dca/d11e 0 2024-03-22T07:16:07.601 INFO:tasks.workunit.client.0.smithi063.stdout:4/842: rename d1/d5/dc/d95/d59/f87 to d1/d5/dc2/dee/f122 0 2024-03-22T07:16:07.602 INFO:tasks.workunit.client.0.smithi063.stdout:5/845: read d8/d13/d1e/d31/daf/fb0 [2275196,100618] 0 2024-03-22T07:16:07.602 INFO:tasks.workunit.client.0.smithi063.stdout:2/866: link d0/de/d35/d36/laa d0/de/d1b/l11f 0 2024-03-22T07:16:07.603 INFO:tasks.workunit.client.0.smithi063.stdout:5/846: mkdir d8/d13/d28/df3/d114 0 2024-03-22T07:16:07.603 INFO:tasks.workunit.client.0.smithi063.stdout:5/847: chown d8/d13/d1e/d31/d64/d65/ddf 1 1 2024-03-22T07:16:07.605 INFO:tasks.workunit.client.0.smithi063.stdout:2/867: rename d0/de/d74/dca/d11e/dbb/f109 to d0/de/d12/f120 0 2024-03-22T07:16:07.608 INFO:tasks.workunit.client.0.smithi063.stdout:2/868: mkdir d0/de/d74/d121 0 2024-03-22T07:16:07.609 INFO:tasks.workunit.client.0.smithi063.stdout:2/869: chown d0/de/d12/d5a/l64 221878876 1 2024-03-22T07:16:07.610 INFO:tasks.workunit.client.0.smithi063.stdout:4/843: read d1/d5/dc/d95/d59/f64 [1635834,13663] 0 2024-03-22T07:16:07.610 INFO:tasks.workunit.client.0.smithi063.stdout:4/844: readlink d1/d5/d43/d27/l11b 0 2024-03-22T07:16:07.610 INFO:tasks.workunit.client.1.smithi069.stdout:6/611: dwrite d4/dd/d12/d44/f6f [0,4194304] 0 2024-03-22T07:16:07.610 INFO:tasks.workunit.client.1.smithi069.stdout:6/612: readlink d4/d7/da/d3f/l8d 0 2024-03-22T07:16:07.611 INFO:tasks.workunit.client.0.smithi063.stdout:9/948: dwrite d8/d19/d39/f6f [4194304,4194304] 0 2024-03-22T07:16:07.611 INFO:tasks.workunit.client.0.smithi063.stdout:2/870: truncate d0/de/d12/d20/fb3 2218440 0 2024-03-22T07:16:07.612 INFO:tasks.workunit.client.0.smithi063.stdout:7/793: dwrite d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/fca [0,4194304] 0 2024-03-22T07:16:07.612 INFO:tasks.workunit.client.0.smithi063.stdout:7/794: write d5/da/d28/d4b/da8/dbc/d62/d8d/dbb/ffa [851773,128359] 0 2024-03-22T07:16:07.612 INFO:tasks.workunit.client.1.smithi069.stdout:6/613: symlink d4/dd/d12/d44/d58/lc1 0 2024-03-22T07:16:07.615 INFO:tasks.workunit.client.0.smithi063.stdout:9/949: link d8/d17/f43 d8/d19/d20/d22/de1/dd7/d10c/f127 0 2024-03-22T07:16:07.615 INFO:tasks.workunit.client.0.smithi063.stdout:9/950: chown d8/d19/d20/d22/de1/dd7 317404 1 2024-03-22T07:16:07.615 INFO:tasks.workunit.client.0.smithi063.stdout:9/951: write d8/fd [935193,109054] 0 2024-03-22T07:16:07.615 INFO:tasks.workunit.client.0.smithi063.stdout:9/952: write d8/d19/f100 [944054,30685] 0 2024-03-22T07:16:07.616 INFO:tasks.workunit.client.1.smithi069.stdout:6/614: creat d4/dd/d41/d78/da3/fc2 x:0 0 0 2024-03-22T07:16:07.617 INFO:tasks.workunit.client.1.smithi069.stdout:6/615: unlink d4/d7/da/f83 0 2024-03-22T07:16:07.618 INFO:tasks.workunit.client.0.smithi063.stdout:7/795: symlink d5/da/d28/d4b/da8/dbc/dfb/l103 0 2024-03-22T07:16:07.618 INFO:tasks.workunit.client.0.smithi063.stdout:7/796: write d5/da/d5f/f8a [1711989,247] 0 2024-03-22T07:16:07.618 INFO:tasks.workunit.client.0.smithi063.stdout:7/797: chown d5/da/d28/d4b/da8/dd4/dea/ld0 1772919 1 2024-03-22T07:16:07.618 INFO:tasks.workunit.client.1.smithi069.stdout:3/747: dwrite d5/d9/d26/d36/d41/fd0 [0,4194304] 0 2024-03-22T07:16:07.619 INFO:tasks.workunit.client.1.smithi069.stdout:6/616: truncate d4/dd/d12/d13/f86 3790973 0 2024-03-22T07:16:07.619 INFO:tasks.workunit.client.0.smithi063.stdout:9/953: symlink d8/d19/d20/d22/d29/d11a/l128 0 2024-03-22T07:16:07.620 INFO:tasks.workunit.client.0.smithi063.stdout:7/798: dread d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d4f/fa4 [0,4194304] 0 2024-03-22T07:16:07.622 INFO:tasks.workunit.client.0.smithi063.stdout:7/799: getdents d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d4f 0 2024-03-22T07:16:07.622 INFO:tasks.workunit.client.1.smithi069.stdout:3/748: dread d5/fa0 [0,4194304] 0 2024-03-22T07:16:07.623 INFO:tasks.workunit.client.1.smithi069.stdout:3/749: rmdir d5/d9/db/d3b/dc6 39 2024-03-22T07:16:07.624 INFO:tasks.workunit.client.0.smithi063.stdout:7/800: mkdir d5/da/d28/d4b/da8/dbc/d62/d104 0 2024-03-22T07:16:07.624 INFO:tasks.workunit.client.0.smithi063.stdout:7/801: creat d5/da/d28/da7/ded/f105 x:0 0 0 2024-03-22T07:16:07.624 INFO:tasks.workunit.client.1.smithi069.stdout:4/807: sync 2024-03-22T07:16:07.624 INFO:tasks.workunit.client.1.smithi069.stdout:4/808: write d6/d9b/d1f/d58/d7c/d90/f95 [694897,18297] 0 2024-03-22T07:16:07.624 INFO:tasks.workunit.client.1.smithi069.stdout:1/649: sync 2024-03-22T07:16:07.625 INFO:tasks.workunit.client.1.smithi069.stdout:7/673: sync 2024-03-22T07:16:07.625 INFO:tasks.workunit.client.1.smithi069.stdout:9/559: sync 2024-03-22T07:16:07.625 INFO:tasks.workunit.client.1.smithi069.stdout:2/512: sync 2024-03-22T07:16:07.625 INFO:tasks.workunit.client.1.smithi069.stdout:2/513: chown d18/d4d/d27/d33/d4f/c78 9874723 1 2024-03-22T07:16:07.625 INFO:tasks.workunit.client.1.smithi069.stdout:9/560: creat d2/dd/d3c/d53/d8b/fb2 x:0 0 0 2024-03-22T07:16:07.625 INFO:tasks.workunit.client.1.smithi069.stdout:0/803: sync 2024-03-22T07:16:07.625 INFO:tasks.workunit.client.1.smithi069.stdout:0/804: write d3/d7/d90/fa5 [587014,27733] 0 2024-03-22T07:16:07.625 INFO:tasks.workunit.client.1.smithi069.stdout:8/877: sync 2024-03-22T07:16:07.625 INFO:tasks.workunit.client.1.smithi069.stdout:8/878: chown d1/d1c/d3e/d66/d9e/dd1/f117 1640 1 2024-03-22T07:16:07.626 INFO:tasks.workunit.client.0.smithi063.stdout:2/871: dread d0/de/d12/d20/f5d [0,4194304] 0 2024-03-22T07:16:07.626 INFO:tasks.workunit.client.0.smithi063.stdout:7/802: unlink d5/da/d28/da7/c94 0 2024-03-22T07:16:07.626 INFO:tasks.workunit.client.0.smithi063.stdout:7/803: fdatasync d5/da/d28/da7/ded/ff6 0 2024-03-22T07:16:07.627 INFO:tasks.workunit.client.0.smithi063.stdout:7/804: truncate d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d4f/f95 806235 0 2024-03-22T07:16:07.627 INFO:tasks.workunit.client.1.smithi069.stdout:3/750: unlink d5/d9/d23/f6d 0 2024-03-22T07:16:07.627 INFO:tasks.workunit.client.1.smithi069.stdout:3/751: fsync d5/d9/d26/d36/f52 0 2024-03-22T07:16:07.627 INFO:tasks.workunit.client.1.smithi069.stdout:4/809: creat d6/d16/f118 x:0 0 0 2024-03-22T07:16:07.627 INFO:tasks.workunit.client.1.smithi069.stdout:4/810: chown d6/d9b/d1f/d58/f85 0 1 2024-03-22T07:16:07.627 INFO:tasks.workunit.client.1.smithi069.stdout:4/811: fsync d6/d4b/f97 0 2024-03-22T07:16:07.627 INFO:tasks.workunit.client.1.smithi069.stdout:4/812: dread - d6/d12/f80 zero size 2024-03-22T07:16:07.628 INFO:tasks.workunit.client.1.smithi069.stdout:1/650: symlink d1/d2a/d84/d9a/lca 0 2024-03-22T07:16:07.628 INFO:tasks.workunit.client.1.smithi069.stdout:1/651: fsync d1/d13/f26 0 2024-03-22T07:16:07.628 INFO:tasks.workunit.client.1.smithi069.stdout:1/652: fsync d1/d2a/d50/d88/d41/d64/f9d 0 2024-03-22T07:16:07.628 INFO:tasks.workunit.client.1.smithi069.stdout:1/653: stat d1/d2a/d50/d88/f4d 0 2024-03-22T07:16:07.629 INFO:tasks.workunit.client.1.smithi069.stdout:1/654: stat d1/d13/d36/d6e/d73/lb3 0 2024-03-22T07:16:07.629 INFO:tasks.workunit.client.1.smithi069.stdout:1/655: dread - d1/d2a/d50/d88/d41/d64/fc3 zero size 2024-03-22T07:16:07.629 INFO:tasks.workunit.client.1.smithi069.stdout:1/656: write d1/d2a/d50/d88/d41/d64/f99 [4569581,60209] 0 2024-03-22T07:16:07.629 INFO:tasks.workunit.client.0.smithi063.stdout:2/872: unlink d0/de/d74/dca/d11e/d7f/d97/da2/fcd 0 2024-03-22T07:16:07.629 INFO:tasks.workunit.client.0.smithi063.stdout:2/873: rename d0 to d0/de/d74/dca/d11e/dbb/d122 22 2024-03-22T07:16:07.629 INFO:tasks.workunit.client.0.smithi063.stdout:2/874: write d0/f8 [181366,13461] 0 2024-03-22T07:16:07.630 INFO:tasks.workunit.client.1.smithi069.stdout:2/514: symlink d18/d4d/d25/lb1 0 2024-03-22T07:16:07.630 INFO:tasks.workunit.client.0.smithi063.stdout:7/805: truncate d5/da/d28/d4b/da8/dd4/fff 4537921 0 2024-03-22T07:16:07.635 INFO:tasks.workunit.client.0.smithi063.stdout:9/954: dread d8/d19/d20/d22/de1/f4d [0,4194304] 0 2024-03-22T07:16:07.635 INFO:tasks.workunit.client.0.smithi063.stdout:9/955: fsync d8/d19/d20/d22/db2/fb9 0 2024-03-22T07:16:07.635 INFO:tasks.workunit.client.0.smithi063.stdout:9/956: fsync d8/d19/d20/d22/f65 0 2024-03-22T07:16:07.635 INFO:tasks.workunit.client.1.smithi069.stdout:0/805: symlink d3/d7/d15/d29/d6e/l110 0 2024-03-22T07:16:07.635 INFO:tasks.workunit.client.1.smithi069.stdout:3/752: creat d5/d9/d26/d3f/dc1/ff3 x:0 0 0 2024-03-22T07:16:07.636 INFO:tasks.workunit.client.1.smithi069.stdout:4/813: rmdir d6/d16/d3a/db4/dfb/d1d 39 2024-03-22T07:16:07.636 INFO:tasks.workunit.client.1.smithi069.stdout:4/814: chown d6/d12/f41 32593 1 2024-03-22T07:16:07.636 INFO:tasks.workunit.client.0.smithi063.stdout:5/848: dwrite d8/d13/d1e/d3c/da0/f88 [0,4194304] 0 2024-03-22T07:16:07.639 INFO:tasks.workunit.client.1.smithi069.stdout:1/657: creat d1/d13/d82/dc2/fcb x:0 0 0 2024-03-22T07:16:07.639 INFO:tasks.workunit.client.1.smithi069.stdout:1/658: stat d1/d13/d36/d6e 0 2024-03-22T07:16:07.640 INFO:tasks.workunit.client.1.smithi069.stdout:9/561: getdents d2/dd/d3c/d48/d7f 0 2024-03-22T07:16:07.644 INFO:tasks.workunit.client.1.smithi069.stdout:7/674: write d9/db/d1a/db1/d57/d44/d53/d74/fa4 [3773328,28447] 0 2024-03-22T07:16:07.644 INFO:tasks.workunit.client.1.smithi069.stdout:7/675: stat d9/db/d1a/db1/d57/d44/d53/d2e 0 2024-03-22T07:16:07.644 INFO:tasks.workunit.client.1.smithi069.stdout:0/806: rename d3/d7/d90/fe0 to d3/d7/d24/d6a/f111 0 2024-03-22T07:16:07.644 INFO:tasks.workunit.client.0.smithi063.stdout:9/957: dread d8/d19/d20/d22/d56/f82 [0,4194304] 0 2024-03-22T07:16:07.645 INFO:tasks.workunit.client.0.smithi063.stdout:9/958: chown d8/d19/d20/d22/d29/d31/d53/d6a/da2/fc7 5276 1 2024-03-22T07:16:07.645 INFO:tasks.workunit.client.0.smithi063.stdout:9/959: truncate d8/d19/d20/d22/d29/d31/d53/d6a/da2/f80 357130 0 2024-03-22T07:16:07.646 INFO:tasks.workunit.client.0.smithi063.stdout:9/960: mkdir d8/d19/d20/d22/d29/d31/d53/d6a/d129 0 2024-03-22T07:16:07.648 INFO:tasks.workunit.client.0.smithi063.stdout:9/961: getdents d8/d19/d20/d22/d5d/d76/d95 0 2024-03-22T07:16:07.648 INFO:tasks.workunit.client.1.smithi069.stdout:4/815: mknod d6/d9b/c119 0 2024-03-22T07:16:07.648 INFO:tasks.workunit.client.1.smithi069.stdout:1/659: symlink d1/d13/dc7/lcc 0 2024-03-22T07:16:07.651 INFO:tasks.workunit.client.0.smithi063.stdout:9/962: link d8/d19/d20/d22/de1/f78 d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/da5/f12a 0 2024-03-22T07:16:07.662 INFO:tasks.workunit.client.1.smithi069.stdout:1/660: rmdir d1/d13/d36/da1 0 2024-03-22T07:16:07.662 INFO:tasks.workunit.client.1.smithi069.stdout:1/661: write d1/d13/f45 [2565483,61519] 0 2024-03-22T07:16:07.662 INFO:tasks.workunit.client.1.smithi069.stdout:7/676: mknod d9/db/d1a/ce1 0 2024-03-22T07:16:07.662 INFO:tasks.workunit.client.1.smithi069.stdout:3/753: rename d5/d9/db/dae to d5/d9/d26/d3f/d54/d9d/dde/df4 0 2024-03-22T07:16:07.662 INFO:tasks.workunit.client.1.smithi069.stdout:3/754: write d5/d9/d26/d3f/d35/f58 [2241244,17091] 0 2024-03-22T07:16:07.662 INFO:tasks.workunit.client.1.smithi069.stdout:7/677: mkdir d9/db/d1a/db1/d57/d75/de2 0 2024-03-22T07:16:07.663 INFO:tasks.workunit.client.1.smithi069.stdout:0/807: dread d3/d7/d17/f4c [0,4194304] 0 2024-03-22T07:16:07.668 INFO:tasks.workunit.client.1.smithi069.stdout:3/755: creat d5/d9/d26/d3f/d56/d8a/ff5 x:0 0 0 2024-03-22T07:16:07.668 INFO:tasks.workunit.client.1.smithi069.stdout:3/756: chown d5/d9/d23/c80 5657424 1 2024-03-22T07:16:07.668 INFO:tasks.workunit.client.1.smithi069.stdout:0/808: mknod d3/d7/d15/d29/d5a/d5d/db4/d100/c112 0 2024-03-22T07:16:07.668 INFO:tasks.workunit.client.1.smithi069.stdout:4/816: write d6/d16/d3a/db4/dfb/f37 [509387,130844] 0 2024-03-22T07:16:07.673 INFO:tasks.workunit.client.0.smithi063.stdout:9/963: dread d8/d19/d20/d22/d29/d31/d53/d6a/da2/f4a [4194304,4194304] 0 2024-03-22T07:16:07.673 INFO:tasks.workunit.client.1.smithi069.stdout:3/757: dread d5/d9/d23/fa9 [4194304,4194304] 0 2024-03-22T07:16:07.674 INFO:tasks.workunit.client.0.smithi063.stdout:9/964: creat d8/d19/d20/d22/d5d/f12b x:0 0 0 2024-03-22T07:16:07.675 INFO:tasks.workunit.client.1.smithi069.stdout:3/758: creat d5/d9/d26/d3f/dbe/ff6 x:0 0 0 2024-03-22T07:16:07.675 INFO:tasks.workunit.client.1.smithi069.stdout:3/759: write d5/d9/d26/d3f/d35/f45 [3721213,5083] 0 2024-03-22T07:16:07.675 INFO:tasks.workunit.client.0.smithi063.stdout:9/965: symlink d8/d19/d20/d22/d9b/l12c 0 2024-03-22T07:16:07.676 INFO:tasks.workunit.client.1.smithi069.stdout:3/760: mknod d5/d9/d26/d3f/d54/d9d/cf7 0 2024-03-22T07:16:07.676 INFO:tasks.workunit.client.1.smithi069.stdout:3/761: write d5/d9/d26/d3f/d54/feb [419048,74529] 0 2024-03-22T07:16:07.676 INFO:tasks.workunit.client.0.smithi063.stdout:9/966: truncate d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/fa3 3947817 0 2024-03-22T07:16:07.677 INFO:tasks.workunit.client.1.smithi069.stdout:1/662: read d1/d2a/d50/d88/d41/d7a/f70 [3950662,18330] 0 2024-03-22T07:16:07.679 INFO:tasks.workunit.client.0.smithi063.stdout:9/967: creat d8/d19/d20/d22/d29/d31/d53/d6a/da2/df1/d110/f12d x:0 0 0 2024-03-22T07:16:07.680 INFO:tasks.workunit.client.0.smithi063.stdout:9/968: unlink d8/d15/d2f/cf2 0 2024-03-22T07:16:07.681 INFO:tasks.workunit.client.0.smithi063.stdout:9/969: mknod d8/d19/d20/d22/d5d/d76/dab/c12e 0 2024-03-22T07:16:07.681 INFO:tasks.workunit.client.0.smithi063.stdout:9/970: readlink d8/d19/d39/db5/ld3 0 2024-03-22T07:16:07.681 INFO:tasks.workunit.client.0.smithi063.stdout:9/971: dread - d8/d19/d2b/fbb zero size 2024-03-22T07:16:07.684 INFO:tasks.workunit.client.0.smithi063.stdout:9/972: rename d8/d19/d2b to d8/d19/d20/d22/d29/d31/d53/d6a/d12f 0 2024-03-22T07:16:07.684 INFO:tasks.workunit.client.0.smithi063.stdout:9/973: truncate d8/d19/d20/d22/d29/d31/f115 4263834 0 2024-03-22T07:16:07.686 INFO:tasks.workunit.client.0.smithi063.stdout:9/974: unlink d8/d15/d63/fb3 0 2024-03-22T07:16:07.688 INFO:tasks.workunit.client.0.smithi063.stdout:9/975: unlink d8/d19/f35 0 2024-03-22T07:16:07.688 INFO:tasks.workunit.client.0.smithi063.stdout:9/976: chown d8/d15/l5e 12438209 1 2024-03-22T07:16:07.688 INFO:tasks.workunit.client.1.smithi069.stdout:0/809: dread d3/d7/d15/d29/d6e/f95 [0,4194304] 0 2024-03-22T07:16:07.689 INFO:tasks.workunit.client.1.smithi069.stdout:7/678: write d9/db/d1a/f21 [1300439,33561] 0 2024-03-22T07:16:07.689 INFO:tasks.workunit.client.0.smithi063.stdout:9/977: truncate d8/d19/d20/d22/d29/d31/d53/ffe 1517765 0 2024-03-22T07:16:07.690 INFO:tasks.workunit.client.1.smithi069.stdout:2/515: dwrite d18/d4d/d27/d33/d60/d64/d89/d8b/f9d [0,4194304] 0 2024-03-22T07:16:07.691 INFO:tasks.workunit.client.1.smithi069.stdout:7/679: creat d9/db/fe3 x:0 0 0 2024-03-22T07:16:07.695 INFO:tasks.workunit.client.1.smithi069.stdout:2/516: rmdir d18/d4d/d27/d33/d60/d64/d89 39 2024-03-22T07:16:07.697 INFO:tasks.workunit.client.1.smithi069.stdout:0/810: dread d3/d7/d15/fa3 [0,4194304] 0 2024-03-22T07:16:07.697 INFO:tasks.workunit.client.1.smithi069.stdout:0/811: stat d3/d7/d15/d29/d5a/f6c 0 2024-03-22T07:16:07.698 INFO:tasks.workunit.client.1.smithi069.stdout:0/812: readlink d3/d7/l19 0 2024-03-22T07:16:07.698 INFO:tasks.workunit.client.1.smithi069.stdout:0/813: stat d3/d7/d15/d29/d5a/d5d 0 2024-03-22T07:16:07.698 INFO:tasks.workunit.client.1.smithi069.stdout:2/517: unlink d18/d4d/d3e/d40/f97 0 2024-03-22T07:16:07.698 INFO:tasks.workunit.client.1.smithi069.stdout:2/518: chown d18/d91 0 1 2024-03-22T07:16:07.698 INFO:tasks.workunit.client.1.smithi069.stdout:0/814: mknod d3/d7/d15/d29/d5a/d84/df4/c113 0 2024-03-22T07:16:07.709 INFO:tasks.workunit.client.1.smithi069.stdout:3/762: dread d5/d9/d26/d36/d41/d51/f7c [0,4194304] 0 2024-03-22T07:16:07.714 INFO:tasks.workunit.client.1.smithi069.stdout:7/680: read d9/db/d1a/db1/d57/d44/d53/d74/d30/f3a [3722208,124738] 0 2024-03-22T07:16:07.715 INFO:tasks.workunit.client.1.smithi069.stdout:7/681: write d9/db/d1a/db1/d57/d75/d7a/f8f [259329,116836] 0 2024-03-22T07:16:07.716 INFO:tasks.workunit.client.0.smithi063.stdout:9/978: dread d8/d19/d20/d22/d29/d31/d53/d6a/da2/f4a [0,4194304] 0 2024-03-22T07:16:07.717 INFO:tasks.workunit.client.0.smithi063.stdout:9/979: mknod d8/d19/d20/d22/db2/c130 0 2024-03-22T07:16:07.717 INFO:tasks.workunit.client.0.smithi063.stdout:9/980: stat d8/d15/d63/lb6 0 2024-03-22T07:16:07.722 INFO:tasks.workunit.client.1.smithi069.stdout:3/763: dread d5/d9/d26/d36/d41/f72 [0,4194304] 0 2024-03-22T07:16:07.725 INFO:tasks.workunit.client.1.smithi069.stdout:7/682: dread d9/db/d1a/db1/d57/d44/f9b [0,4194304] 0 2024-03-22T07:16:07.725 INFO:tasks.workunit.client.1.smithi069.stdout:3/764: dread d5/f4e [0,4194304] 0 2024-03-22T07:16:07.725 INFO:tasks.workunit.client.1.smithi069.stdout:7/683: getdents d9/db/d1a/db1/d57/d44/d53/dc7/db5 0 2024-03-22T07:16:07.730 INFO:tasks.workunit.client.0.smithi063.stdout:9/981: dread d8/d19/d20/d22/de1/dd7/d10c/f127 [0,4194304] 0 2024-03-22T07:16:07.731 INFO:tasks.workunit.client.0.smithi063.stdout:9/982: truncate d8/d15/d2f/f93 3769398 0 2024-03-22T07:16:07.731 INFO:tasks.workunit.client.0.smithi063.stdout:9/983: write d8/d19/d20/d22/de1/f4d [3350594,8819] 0 2024-03-22T07:16:07.732 INFO:tasks.workunit.client.0.smithi063.stdout:9/984: mknod d8/d19/d20/d22/d29/d31/d53/d6a/da2/df1/dfb/c131 0 2024-03-22T07:16:07.732 INFO:tasks.workunit.client.0.smithi063.stdout:9/985: creat d8/d19/d20/d22/d56/f132 x:0 0 0 2024-03-22T07:16:07.733 INFO:tasks.workunit.client.0.smithi063.stdout:9/986: symlink d8/d19/d20/d22/d29/d31/d53/d6a/d129/l133 0 2024-03-22T07:16:07.734 INFO:tasks.workunit.client.0.smithi063.stdout:9/987: fdatasync d8/d19/d20/d22/f65 0 2024-03-22T07:16:07.734 INFO:tasks.workunit.client.0.smithi063.stdout:9/988: creat d8/d19/d20/d22/f134 x:0 0 0 2024-03-22T07:16:07.735 INFO:tasks.workunit.client.0.smithi063.stdout:9/989: symlink d8/d19/d20/d22/d9b/l135 0 2024-03-22T07:16:07.736 INFO:tasks.workunit.client.0.smithi063.stdout:9/990: rename d8/d19/l52 to d8/d19/d20/d22/db2/l136 0 2024-03-22T07:16:07.736 INFO:tasks.workunit.client.0.smithi063.stdout:9/991: readlink d8/d15/l8e 0 2024-03-22T07:16:07.736 INFO:tasks.workunit.client.0.smithi063.stdout:9/992: write d8/d19/d20/d22/d29/d31/f115 [4930692,46823] 0 2024-03-22T07:16:07.737 INFO:tasks.workunit.client.1.smithi069.stdout:3/765: dread d5/d9/d26/f6c [0,4194304] 0 2024-03-22T07:16:07.739 INFO:tasks.workunit.client.0.smithi063.stdout:9/993: dread d8/d15/fbf [0,4194304] 0 2024-03-22T07:16:07.739 INFO:tasks.workunit.client.0.smithi063.stdout:9/994: readlink d8/d15/l88 0 2024-03-22T07:16:07.739 INFO:tasks.workunit.client.0.smithi063.stdout:9/995: write d8/d19/d20/d22/d29/d31/d37/f59 [901181,71248] 0 2024-03-22T07:16:07.739 INFO:tasks.workunit.client.0.smithi063.stdout:9/996: fdatasync d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/f85 0 2024-03-22T07:16:07.739 INFO:tasks.workunit.client.0.smithi063.stdout:9/997: chown d8/d17 664 1 2024-03-22T07:16:07.739 INFO:tasks.workunit.client.0.smithi063.stdout:9/998: creat d8/d19/d20/d22/d29/d31/d53/d6a/da2/d48/f137 x:0 0 0 2024-03-22T07:16:07.739 INFO:tasks.workunit.client.0.smithi063.stdout:9/999: stat d8/d19/d20/d22/lae 0 2024-03-22T07:16:07.739 INFO:tasks.workunit.client.0.smithi063.stdout:5/849: dwrite d8/f14 [0,4194304] 0 2024-03-22T07:16:07.739 INFO:tasks.workunit.client.0.smithi063.stdout:7/806: dwrite d5/da/d28/d4b/da8/dbc/d62/d8d/d98/fb7 [0,4194304] 0 2024-03-22T07:16:07.742 INFO:tasks.workunit.client.0.smithi063.stdout:5/850: rename d8/d5e/l9c to d8/d13/d1e/d31/d64/l115 0 2024-03-22T07:16:07.743 INFO:tasks.workunit.client.1.smithi069.stdout:9/562: dwrite d2/d83/f92 [0,4194304] 0 2024-03-22T07:16:07.743 INFO:tasks.workunit.client.1.smithi069.stdout:2/519: dwrite d18/d4d/f22 [8388608,4194304] 0 2024-03-22T07:16:07.743 INFO:tasks.workunit.client.1.smithi069.stdout:4/817: dwrite d6/d16/d3a/db4/dfb/d23/d59/fca [4194304,4194304] 0 2024-03-22T07:16:07.746 INFO:tasks.workunit.client.0.smithi063.stdout:7/807: mkdir d5/da/d5f/d33/d106 0 2024-03-22T07:16:07.746 INFO:tasks.workunit.client.0.smithi063.stdout:7/808: chown d5/d38/f81 323854563 1 2024-03-22T07:16:07.746 INFO:tasks.workunit.client.0.smithi063.stdout:7/809: fsync d5/d16/d43/f56 0 2024-03-22T07:16:07.748 INFO:tasks.workunit.client.0.smithi063.stdout:5/851: link d8/dba/c104 d8/d13/d24/da4/c116 0 2024-03-22T07:16:07.753 INFO:tasks.workunit.client.0.smithi063.stdout:7/810: creat d5/da/d5f/d33/f107 x:0 0 0 2024-03-22T07:16:07.754 INFO:tasks.workunit.client.1.smithi069.stdout:2/520: readlink d18/d4d/d27/d33/d60/d64/l81 0 2024-03-22T07:16:07.754 INFO:tasks.workunit.client.1.smithi069.stdout:2/521: read d18/d4d/d25/f47 [1394630,77392] 0 2024-03-22T07:16:07.760 INFO:tasks.workunit.client.1.smithi069.stdout:4/818: mkdir d6/d16/d68/df8/d11a 0 2024-03-22T07:16:07.761 INFO:tasks.workunit.client.1.smithi069.stdout:3/766: dwrite d5/d9/d26/db4/fd3 [0,4194304] 0 2024-03-22T07:16:07.763 INFO:tasks.workunit.client.1.smithi069.stdout:0/815: rename d3/d7/d24/d6a to d3/d7/d15/d29/dcf/d114 0 2024-03-22T07:16:07.769 INFO:tasks.workunit.client.0.smithi063.stdout:7/811: read d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/fb4 [1124263,66202] 0 2024-03-22T07:16:07.771 INFO:tasks.workunit.client.0.smithi063.stdout:7/812: truncate d5/d16/d1b/f80 1121969 0 2024-03-22T07:16:07.772 INFO:tasks.workunit.client.0.smithi063.stdout:7/813: stat d5/da/d28/da7 0 2024-03-22T07:16:07.772 INFO:tasks.workunit.client.1.smithi069.stdout:9/563: rmdir d2/dd/d3c 39 2024-03-22T07:16:07.772 INFO:tasks.workunit.client.1.smithi069.stdout:9/564: write d2/d21/f3a [877087,66309] 0 2024-03-22T07:16:07.772 INFO:tasks.workunit.client.1.smithi069.stdout:9/565: fsync d2/dd/d3c/d2e/d84/f88 0 2024-03-22T07:16:07.773 INFO:tasks.workunit.client.1.smithi069.stdout:2/522: symlink d18/d4d/d27/d33/d60/d63/d73/lb2 0 2024-03-22T07:16:07.776 INFO:tasks.workunit.client.0.smithi063.stdout:7/814: rename d5/da/d28/da7/c50 to d5/da/d28/d4b/da8/dbc/d62/d73/c108 0 2024-03-22T07:16:07.776 INFO:tasks.workunit.client.0.smithi063.stdout:7/815: read - d5/da/d28/da7/ded/ff2 zero size 2024-03-22T07:16:07.776 INFO:tasks.workunit.client.0.smithi063.stdout:7/816: stat d5/da/f35 0 2024-03-22T07:16:07.776 INFO:tasks.workunit.client.0.smithi063.stdout:7/817: getdents d5/da/d5f/d101 0 2024-03-22T07:16:07.777 INFO:tasks.workunit.client.1.smithi069.stdout:4/819: symlink d6/d16/d3a/db4/dfb/d23/d59/d8f/l11b 0 2024-03-22T07:16:07.779 INFO:tasks.workunit.client.0.smithi063.stdout:7/818: rename d5/d16/f22 to d5/da/f109 0 2024-03-22T07:16:07.780 INFO:tasks.workunit.client.1.smithi069.stdout:2/523: dread - d18/d4d/d27/d33/d60/d64/d89/f8d zero size 2024-03-22T07:16:07.782 INFO:tasks.workunit.client.0.smithi063.stdout:7/819: symlink d5/da/d28/d4b/da8/dbc/d62/d104/l10a 0 2024-03-22T07:16:07.783 INFO:tasks.workunit.client.1.smithi069.stdout:4/820: mkdir d6/dc4/d11c 0 2024-03-22T07:16:07.783 INFO:tasks.workunit.client.1.smithi069.stdout:2/524: truncate d18/d4d/d27/d33/f6b 447310 0 2024-03-22T07:16:07.783 INFO:tasks.workunit.client.1.smithi069.stdout:2/525: fsync d18/d4d/d34/f7e 0 2024-03-22T07:16:07.784 INFO:tasks.workunit.client.0.smithi063.stdout:7/820: symlink d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/df0/l10b 0 2024-03-22T07:16:07.786 INFO:tasks.workunit.client.0.smithi063.stdout:7/821: rename d5/f39 to d5/da/d28/d4b/da8/dbc/d62/d73/f10c 0 2024-03-22T07:16:07.787 INFO:tasks.workunit.client.1.smithi069.stdout:2/526: mkdir d18/d4d/d25/db3 0 2024-03-22T07:16:07.787 INFO:tasks.workunit.client.1.smithi069.stdout:7/684: dwrite d9/db/d1a/db1/d57/d75/d7a/f8f [0,4194304] 0 2024-03-22T07:16:07.788 INFO:tasks.workunit.client.0.smithi063.stdout:7/822: mknod d5/d16/d43/d9d/dd8/c10d 0 2024-03-22T07:16:07.788 INFO:tasks.workunit.client.0.smithi063.stdout:7/823: chown d5/da/d4e/fd5 69968 1 2024-03-22T07:16:07.789 INFO:tasks.workunit.client.1.smithi069.stdout:2/527: creat d18/d4d/d27/d33/d60/d63/d73/fb4 x:0 0 0 2024-03-22T07:16:07.789 INFO:tasks.workunit.client.1.smithi069.stdout:2/528: write d18/d4d/d25/f47 [1929328,22214] 0 2024-03-22T07:16:07.793 INFO:tasks.workunit.client.1.smithi069.stdout:2/529: symlink d18/d4d/d3e/lb5 0 2024-03-22T07:16:07.794 INFO:tasks.workunit.client.1.smithi069.stdout:2/530: mknod d18/d4d/d27/d33/d60/d64/d89/cb6 0 2024-03-22T07:16:07.795 INFO:tasks.workunit.client.1.smithi069.stdout:2/531: creat d18/d4d/d27/d72/fb7 x:0 0 0 2024-03-22T07:16:07.798 INFO:tasks.workunit.client.0.smithi063.stdout:5/852: dwrite d8/d13/d1e/d31/fa6 [0,4194304] 0 2024-03-22T07:16:07.798 INFO:tasks.workunit.client.1.smithi069.stdout:3/767: dwrite d5/d9/f76 [0,4194304] 0 2024-03-22T07:16:07.799 INFO:tasks.workunit.client.1.smithi069.stdout:0/816: dwrite d3/d7/d15/d1f/f41 [0,4194304] 0 2024-03-22T07:16:07.799 INFO:tasks.workunit.client.1.smithi069.stdout:0/817: creat d3/d7/d15/dce/f115 x:0 0 0 2024-03-22T07:16:07.799 INFO:tasks.workunit.client.1.smithi069.stdout:0/818: truncate d3/d7/d15/d29/d5a/d99/fab 885516 0 2024-03-22T07:16:07.800 INFO:tasks.workunit.client.1.smithi069.stdout:3/768: dread d5/d9/d26/d3f/d54/d9d/dde/df4/faf [0,4194304] 0 2024-03-22T07:16:07.800 INFO:tasks.workunit.client.1.smithi069.stdout:3/769: write d5/d9/db/fbb [5158186,115710] 0 2024-03-22T07:16:07.800 INFO:tasks.workunit.client.1.smithi069.stdout:9/566: dwrite d2/f55 [4194304,4194304] 0 2024-03-22T07:16:07.806 INFO:tasks.workunit.client.1.smithi069.stdout:5/629: sync 2024-03-22T07:16:07.807 INFO:tasks.workunit.client.1.smithi069.stdout:3/770: dread d5/d9/f68 [0,4194304] 0 2024-03-22T07:16:07.810 INFO:tasks.workunit.client.0.smithi063.stdout:5/853: unlink d8/d13/d1e/d3c/cc3 0 2024-03-22T07:16:07.810 INFO:tasks.workunit.client.0.smithi063.stdout:5/854: fdatasync d8/d13/d24/f3b 0 2024-03-22T07:16:07.810 INFO:tasks.workunit.client.0.smithi063.stdout:5/855: dread - d8/d13/d1e/d31/d64/d65/fa3 zero size 2024-03-22T07:16:07.810 INFO:tasks.workunit.client.0.smithi063.stdout:5/856: fsync d8/d13/d1e/d31/d64/fc2 0 2024-03-22T07:16:07.811 INFO:tasks.workunit.client.0.smithi063.stdout:5/857: read - d8/d13/d28/d57/d5f/f112 zero size 2024-03-22T07:16:07.811 INFO:tasks.workunit.client.0.smithi063.stdout:5/858: link d8/d13/d1e/f79 d8/d13/d28/df3/d114/f117 0 2024-03-22T07:16:07.812 INFO:tasks.workunit.client.1.smithi069.stdout:4/821: dwrite d6/d16/d3a/db4/fde [0,4194304] 0 2024-03-22T07:16:07.812 INFO:tasks.workunit.client.1.smithi069.stdout:4/822: chown d6/d9b/d1f/d3f/d51 44290 1 2024-03-22T07:16:07.812 INFO:tasks.workunit.client.1.smithi069.stdout:2/532: write d18/d4d/d27/d33/d3d/f46 [1312854,15565] 0 2024-03-22T07:16:07.812 INFO:tasks.workunit.client.1.smithi069.stdout:2/533: creat d18/d91/fb8 x:0 0 0 2024-03-22T07:16:07.812 INFO:tasks.workunit.client.1.smithi069.stdout:3/771: mknod d5/d9/d26/d36/cf8 0 2024-03-22T07:16:07.812 INFO:tasks.workunit.client.1.smithi069.stdout:3/772: truncate d5/d9/d26/d3f/d54/d9d/dde/df4/fd4 724324 0 2024-03-22T07:16:07.812 INFO:tasks.workunit.client.1.smithi069.stdout:3/773: readlink d5/d9/l1f 0 2024-03-22T07:16:07.812 INFO:tasks.workunit.client.1.smithi069.stdout:0/819: dread d3/d7/d15/d29/dcf/d114/fc9 [4194304,4194304] 0 2024-03-22T07:16:07.814 INFO:tasks.workunit.client.1.smithi069.stdout:5/630: dread d1/d6/d26/d33/f80 [0,4194304] 0 2024-03-22T07:16:07.814 INFO:tasks.workunit.client.1.smithi069.stdout:5/631: chown d1/d6/d26/d33/d5d/d72 24621 1 2024-03-22T07:16:07.815 INFO:tasks.workunit.client.1.smithi069.stdout:2/534: write d18/d4d/d27/d33/d60/d64/f99 [439003,16563] 0 2024-03-22T07:16:07.816 INFO:tasks.workunit.client.1.smithi069.stdout:4/823: dread d6/f9 [0,4194304] 0 2024-03-22T07:16:07.816 INFO:tasks.workunit.client.1.smithi069.stdout:4/824: stat d6/d16/d3a/db4/dbc/dc6/l10a 0 2024-03-22T07:16:07.816 INFO:tasks.workunit.client.1.smithi069.stdout:9/567: read d2/f5 [4587079,17426] 0 2024-03-22T07:16:07.818 INFO:tasks.workunit.client.1.smithi069.stdout:3/774: mkdir d5/d9/d26/d36/d41/df9 0 2024-03-22T07:16:07.819 INFO:tasks.workunit.client.1.smithi069.stdout:4/825: dread d6/d16/d83/fdd [0,4194304] 0 2024-03-22T07:16:07.824 INFO:tasks.workunit.client.1.smithi069.stdout:0/820: mknod d3/d7/d15/d29/d5a/d84/d10f/c116 0 2024-03-22T07:16:07.824 INFO:tasks.workunit.client.1.smithi069.stdout:0/821: write d3/d7/fb [5464014,119689] 0 2024-03-22T07:16:07.825 INFO:tasks.workunit.client.1.smithi069.stdout:2/535: write fa [1423140,111518] 0 2024-03-22T07:16:07.825 INFO:tasks.workunit.client.0.smithi063.stdout:4/845: sync 2024-03-22T07:16:07.826 INFO:tasks.workunit.client.0.smithi063.stdout:6/990: sync 2024-03-22T07:16:07.826 INFO:tasks.workunit.client.0.smithi063.stdout:8/973: sync 2024-03-22T07:16:07.826 INFO:tasks.workunit.client.0.smithi063.stdout:2/875: sync 2024-03-22T07:16:07.826 INFO:tasks.workunit.client.0.smithi063.stdout:2/876: chown d0/de/d1b/d63/dc6/l117 21651351 1 2024-03-22T07:16:07.826 INFO:tasks.workunit.client.1.smithi069.stdout:5/632: unlink d1/d7/l70 0 2024-03-22T07:16:07.827 INFO:tasks.workunit.client.0.smithi063.stdout:8/974: truncate d0/d24/d33/d4c/f7d 2599996 0 2024-03-22T07:16:07.827 INFO:tasks.workunit.client.1.smithi069.stdout:9/568: symlink d2/d71/d90/lb3 0 2024-03-22T07:16:07.828 INFO:tasks.workunit.client.0.smithi063.stdout:6/991: truncate d3/d15/d16/d60/d6d/d4e/d5f/f93 762614 0 2024-03-22T07:16:07.831 INFO:tasks.workunit.client.0.smithi063.stdout:4/846: mkdir d1/d5/d43/d27/d2c/df4/d10b/d123 0 2024-03-22T07:16:07.832 INFO:tasks.workunit.client.0.smithi063.stdout:2/877: rmdir d0/de/d74/dca/d11e/d7f/d97/da2/d104 39 2024-03-22T07:16:07.832 INFO:tasks.workunit.client.0.smithi063.stdout:2/878: fdatasync d0/de/d1b/d32/dac/faf 0 2024-03-22T07:16:07.832 INFO:tasks.workunit.client.0.smithi063.stdout:2/879: chown d0/de/d74/dca/d11e 24 1 2024-03-22T07:16:07.832 INFO:tasks.workunit.client.0.smithi063.stdout:2/880: write d0/de/d74/dca/d11e/f73 [507125,86989] 0 2024-03-22T07:16:07.832 INFO:tasks.workunit.client.0.smithi063.stdout:2/881: dread - d0/de/d12/fa9 zero size 2024-03-22T07:16:07.832 INFO:tasks.workunit.client.0.smithi063.stdout:2/882: readlink d0/de/d35/l3c 0 2024-03-22T07:16:07.832 INFO:tasks.workunit.client.0.smithi063.stdout:8/975: symlink d0/d26/d2d/d59/dc0/l147 0 2024-03-22T07:16:07.833 INFO:tasks.workunit.client.0.smithi063.stdout:4/847: symlink d1/d5/dc2/l124 0 2024-03-22T07:16:07.833 INFO:tasks.workunit.client.0.smithi063.stdout:6/992: dread d3/d6/d2a/d38/da5/d32/fb1 [0,4194304] 0 2024-03-22T07:16:07.834 INFO:tasks.workunit.client.1.smithi069.stdout:4/826: rename d6/d16/d3a/db4/dfb/d1d/d55/f5f to d6/d12/d18/f11d 0 2024-03-22T07:16:07.834 INFO:tasks.workunit.client.1.smithi069.stdout:3/775: dread d5/d9/d26/db4/fd3 [4194304,4194304] 0 2024-03-22T07:16:07.834 INFO:tasks.workunit.client.1.smithi069.stdout:3/776: fsync d5/d9/d26/d3f/d35/f37 0 2024-03-22T07:16:07.834 INFO:tasks.workunit.client.1.smithi069.stdout:2/536: creat d18/d4d/d27/d33/d60/fb9 x:0 0 0 2024-03-22T07:16:07.834 INFO:tasks.workunit.client.1.smithi069.stdout:4/827: dread d6/d12/db2/fbd [0,4194304] 0 2024-03-22T07:16:07.834 INFO:tasks.workunit.client.1.smithi069.stdout:5/633: dread - d1/d6/d26/d33/d5d/d72/da7/fc5 zero size 2024-03-22T07:16:07.836 INFO:tasks.workunit.client.0.smithi063.stdout:2/883: link d0/de/d74/dca/d11e/f10f d0/de/d74/dca/d11e/dbb/f123 0 2024-03-22T07:16:07.836 INFO:tasks.workunit.client.0.smithi063.stdout:7/824: write d5/d16/f2b [3423398,5433] 0 2024-03-22T07:16:07.836 INFO:tasks.workunit.client.0.smithi063.stdout:5/859: dwrite d8/d13/d24/d81/fd4 [0,4194304] 0 2024-03-22T07:16:07.836 INFO:tasks.workunit.client.0.smithi063.stdout:7/825: stat d5/da/d5f/d33/fbd 0 2024-03-22T07:16:07.837 INFO:tasks.workunit.client.0.smithi063.stdout:6/993: truncate d3/d15/d16/d60/d6d/fae 2218017 0 2024-03-22T07:16:07.837 INFO:tasks.workunit.client.1.smithi069.stdout:0/822: dread d3/d7/d15/d29/d5a/d5d/d9d/fbd [0,4194304] 0 2024-03-22T07:16:07.837 INFO:tasks.workunit.client.1.smithi069.stdout:4/828: rmdir d6/d9b/d1f/d58 39 2024-03-22T07:16:07.837 INFO:tasks.workunit.client.1.smithi069.stdout:4/829: truncate d6/d9b/d1f/d58/d7c/f111 161848 0 2024-03-22T07:16:07.837 INFO:tasks.workunit.client.1.smithi069.stdout:0/823: truncate d3/d7/d17/fe4 943712 0 2024-03-22T07:16:07.837 INFO:tasks.workunit.client.1.smithi069.stdout:4/830: creat d6/daa/f11e x:0 0 0 2024-03-22T07:16:07.837 INFO:tasks.workunit.client.1.smithi069.stdout:4/831: fsync d6/d16/d3a/db4/dfb/feb 0 2024-03-22T07:16:07.841 INFO:tasks.workunit.client.1.smithi069.stdout:0/824: creat d3/d7/d15/d29/d5a/d84/df4/f117 x:0 0 0 2024-03-22T07:16:07.841 INFO:tasks.workunit.client.1.smithi069.stdout:0/825: creat d3/f118 x:0 0 0 2024-03-22T07:16:07.841 INFO:tasks.workunit.client.1.smithi069.stdout:0/826: write d3/d7/d15/d29/f30 [899346,121179] 0 2024-03-22T07:16:07.841 INFO:tasks.workunit.client.1.smithi069.stdout:0/827: creat d3/d7/d15/d29/d5a/d5d/f119 x:0 0 0 2024-03-22T07:16:07.843 INFO:tasks.workunit.client.0.smithi063.stdout:2/884: creat d0/f124 x:0 0 0 2024-03-22T07:16:07.846 INFO:tasks.workunit.client.0.smithi063.stdout:7/826: unlink d5/da/d28/d4b/da8/dbc/f70 0 2024-03-22T07:16:07.846 INFO:tasks.workunit.client.0.smithi063.stdout:7/827: creat d5/f10e x:0 0 0 2024-03-22T07:16:07.846 INFO:tasks.workunit.client.0.smithi063.stdout:7/828: write d5/d16/f90 [289364,47454] 0 2024-03-22T07:16:07.847 INFO:tasks.workunit.client.1.smithi069.stdout:9/569: write d2/dd/d3c/d53/d9d/f9b [1063609,86493] 0 2024-03-22T07:16:07.849 INFO:tasks.workunit.client.0.smithi063.stdout:5/860: rename d8/d13/d28/d57/d5f/fe8 to d8/de9/f118 0 2024-03-22T07:16:07.849 INFO:tasks.workunit.client.0.smithi063.stdout:5/861: fsync d8/d13/d24/d81/ff1 0 2024-03-22T07:16:07.849 INFO:tasks.workunit.client.0.smithi063.stdout:5/862: chown d8/d5e 1312764921 1 2024-03-22T07:16:07.850 INFO:tasks.workunit.client.1.smithi069.stdout:9/570: link d2/dd/d3c/c6c d2/dd/d3c/d53/d8b/cb4 0 2024-03-22T07:16:07.850 INFO:tasks.workunit.client.1.smithi069.stdout:9/571: fsync d2/d51/f7a 0 2024-03-22T07:16:07.850 INFO:tasks.workunit.client.1.smithi069.stdout:9/572: chown d2/dd/d3c/d2e/f80 639554 1 2024-03-22T07:16:07.851 INFO:tasks.workunit.client.1.smithi069.stdout:9/573: mknod d2/dd/d3c/d48/d7c/cb5 0 2024-03-22T07:16:07.852 INFO:tasks.workunit.client.0.smithi063.stdout:6/994: mkdir d3/d15/d16/d60/d6d/d4e/d5a/d149 0 2024-03-22T07:16:07.852 INFO:tasks.workunit.client.1.smithi069.stdout:9/574: mkdir d2/db6 0 2024-03-22T07:16:07.853 INFO:tasks.workunit.client.0.smithi063.stdout:4/848: write d1/d5/f20 [211021,52371] 0 2024-03-22T07:16:07.853 INFO:tasks.workunit.client.0.smithi063.stdout:2/885: creat d0/de/d1b/d63/d6e/f125 x:0 0 0 2024-03-22T07:16:07.853 INFO:tasks.workunit.client.0.smithi063.stdout:2/886: truncate d0/de/d12/f1f 4713549 0 2024-03-22T07:16:07.856 INFO:tasks.workunit.client.0.smithi063.stdout:7/829: creat d5/d16/d43/f10f x:0 0 0 2024-03-22T07:16:07.856 INFO:tasks.workunit.client.0.smithi063.stdout:7/830: creat d5/da/d28/d4b/da8/dbc/d62/d8d/dbb/f110 x:0 0 0 2024-03-22T07:16:07.858 INFO:tasks.workunit.client.0.smithi063.stdout:7/831: write d5/d16/d43/f10f [802806,72130] 0 2024-03-22T07:16:07.858 INFO:tasks.workunit.client.0.smithi063.stdout:5/863: mkdir d8/d13/d1e/d31/d64/d65/ddf/d119 0 2024-03-22T07:16:07.859 INFO:tasks.workunit.client.1.smithi069.stdout:2/537: dread d18/d4d/d25/f47 [0,4194304] 0 2024-03-22T07:16:07.860 INFO:tasks.workunit.client.1.smithi069.stdout:2/538: write d18/f1e [357190,4691] 0 2024-03-22T07:16:07.860 INFO:tasks.workunit.client.1.smithi069.stdout:9/575: dread d2/dd/d3c/d2e/f80 [0,4194304] 0 2024-03-22T07:16:07.861 INFO:tasks.workunit.client.0.smithi063.stdout:6/995: mkdir d3/d15/d16/d60/dd9/d14a 0 2024-03-22T07:16:07.861 INFO:tasks.workunit.client.0.smithi063.stdout:6/996: read d3/d6/fc [1087955,49392] 0 2024-03-22T07:16:07.861 INFO:tasks.workunit.client.0.smithi063.stdout:6/997: creat d3/d15/d16/d60/d6d/f14b x:0 0 0 2024-03-22T07:16:07.862 INFO:tasks.workunit.client.0.smithi063.stdout:4/849: mknod d1/d5/d3c/c125 0 2024-03-22T07:16:07.864 INFO:tasks.workunit.client.1.smithi069.stdout:9/576: mknod d2/dd/d3c/d48/d7c/cb7 0 2024-03-22T07:16:07.866 INFO:tasks.workunit.client.0.smithi063.stdout:7/832: truncate d5/d38/f7b 788803 0 2024-03-22T07:16:07.866 INFO:tasks.workunit.client.0.smithi063.stdout:7/833: read d5/da/d5f/f8a [1113507,919] 0 2024-03-22T07:16:07.871 INFO:tasks.workunit.client.0.smithi063.stdout:6/998: mknod d3/d15/d16/d60/d6d/d4e/d5a/c14c 0 2024-03-22T07:16:07.871 INFO:tasks.workunit.client.0.smithi063.stdout:6/999: truncate d3/d6/d2a/d38/da5/d32/f12c 126940 0 2024-03-22T07:16:07.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:07 smithi063 ceph-mon[29200]: pgmap v6: 65 pgs: 65 active+clean; 2.6 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 57 MiB/s rd, 75 MiB/s wr, 57 op/s 2024-03-22T07:16:07.874 INFO:tasks.workunit.client.0.smithi063.stdout:8/976: dwrite d0/d1b/d28/d3e/f13b [4194304,4194304] 0 2024-03-22T07:16:07.877 INFO:tasks.workunit.client.0.smithi063.stdout:8/977: symlink d0/d26/d2d/d83/d11c/d121/l148 0 2024-03-22T07:16:07.877 INFO:tasks.workunit.client.0.smithi063.stdout:8/978: read d0/d1b/d28/d7b/f7f [665177,28197] 0 2024-03-22T07:16:07.878 INFO:tasks.workunit.client.0.smithi063.stdout:8/979: dread - d0/d1b/d28/d7b/d4a/f116 zero size 2024-03-22T07:16:07.879 INFO:tasks.workunit.client.1.smithi069.stdout:9/577: dread d2/dd/f26 [0,4194304] 0 2024-03-22T07:16:07.879 INFO:tasks.workunit.client.1.smithi069.stdout:9/578: write d2/dd/d3c/d2e/f80 [1166754,82211] 0 2024-03-22T07:16:07.879 INFO:tasks.workunit.client.1.smithi069.stdout:9/579: stat d2/l58 0 2024-03-22T07:16:07.879 INFO:tasks.workunit.client.1.smithi069.stdout:9/580: stat d2/dd/d3c/d48/d7f/d86 0 2024-03-22T07:16:07.879 INFO:tasks.workunit.client.1.smithi069.stdout:9/581: chown d2/dd/d3c 224588147 1 2024-03-22T07:16:07.879 INFO:tasks.workunit.client.1.smithi069.stdout:9/582: chown d2/dd/d3c/f43 1 1 2024-03-22T07:16:07.879 INFO:tasks.workunit.client.1.smithi069.stdout:9/583: chown d2/d71/d90/f95 33 1 2024-03-22T07:16:07.879 INFO:tasks.workunit.client.1.smithi069.stdout:9/584: readlink d2/dd/d3c/l33 0 2024-03-22T07:16:07.880 INFO:tasks.workunit.client.0.smithi063.stdout:7/834: dread d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/fb4 [0,4194304] 0 2024-03-22T07:16:07.880 INFO:tasks.workunit.client.0.smithi063.stdout:7/835: stat d5/f9 0 2024-03-22T07:16:07.881 INFO:tasks.workunit.client.1.smithi069.stdout:9/585: dread d2/dd/d3c/d2e/f65 [0,4194304] 0 2024-03-22T07:16:07.881 INFO:tasks.workunit.client.0.smithi063.stdout:7/836: getdents d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d40 0 2024-03-22T07:16:07.881 INFO:tasks.workunit.client.0.smithi063.stdout:7/837: read d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/fb4 [495184,23906] 0 2024-03-22T07:16:07.882 INFO:tasks.workunit.client.0.smithi063.stdout:7/838: rmdir d5/da/d5f/d33/d106 0 2024-03-22T07:16:07.882 INFO:tasks.workunit.client.0.smithi063.stdout:7/839: readlink d5/d16/l4c 0 2024-03-22T07:16:07.889 INFO:tasks.workunit.client.1.smithi069.stdout:3/777: dwrite d5/d9/d26/d3f/d54/feb [0,4194304] 0 2024-03-22T07:16:07.893 INFO:tasks.workunit.client.1.smithi069.stdout:3/778: mkdir d5/d9/d26/d3f/dec/dfa 0 2024-03-22T07:16:07.893 INFO:tasks.workunit.client.1.smithi069.stdout:3/779: truncate d5/d9/db/d3b/fcf 160637 0 2024-03-22T07:16:07.894 INFO:tasks.workunit.client.1.smithi069.stdout:3/780: creat d5/d9/d26/d36/d41/df9/ffb x:0 0 0 2024-03-22T07:16:07.895 INFO:tasks.workunit.client.0.smithi063.stdout:8/980: write d0/d26/d2d/d8f/ff1 [1237532,30348] 0 2024-03-22T07:16:07.897 INFO:tasks.workunit.client.0.smithi063.stdout:8/981: mkdir d0/d1b/d28/d3e/db2/dbd/d149 0 2024-03-22T07:16:07.898 INFO:tasks.workunit.client.1.smithi069.stdout:0/828: dwrite d3/d7/d15/d29/dcf/d114/fc0 [0,4194304] 0 2024-03-22T07:16:07.899 INFO:tasks.workunit.client.1.smithi069.stdout:0/829: chown d3/d7/d15/d29/d5a/d84/f60 24 1 2024-03-22T07:16:07.900 INFO:tasks.workunit.client.1.smithi069.stdout:5/634: dwrite d1/d6/d26/d33/f4c [0,4194304] 0 2024-03-22T07:16:07.900 INFO:tasks.workunit.client.1.smithi069.stdout:4/832: dwrite d6/dc4/ff7 [0,4194304] 0 2024-03-22T07:16:07.900 INFO:tasks.workunit.client.1.smithi069.stdout:4/833: chown d6/l20 230590337 1 2024-03-22T07:16:07.900 INFO:tasks.workunit.client.1.smithi069.stdout:4/834: fdatasync d6/d4b/f4d 0 2024-03-22T07:16:07.904 INFO:tasks.workunit.client.1.smithi069.stdout:5/635: dread d1/d6/d26/d33/f4e [0,4194304] 0 2024-03-22T07:16:07.905 INFO:tasks.workunit.client.1.smithi069.stdout:4/835: mknod d6/daa/dce/c11f 0 2024-03-22T07:16:07.906 INFO:tasks.workunit.client.1.smithi069.stdout:4/836: mkdir d6/d16/d3a/db4/dfb/d23/d59/d120 0 2024-03-22T07:16:07.906 INFO:tasks.workunit.client.1.smithi069.stdout:4/837: dread - d6/d12/d5d/ff0 zero size 2024-03-22T07:16:07.906 INFO:tasks.workunit.client.1.smithi069.stdout:4/838: write d6/d16/d3a/db4/dfb/d1d/d55/fd5 [1063760,37527] 0 2024-03-22T07:16:07.908 INFO:tasks.workunit.client.1.smithi069.stdout:4/839: link d6/d12/f73 d6/d16/d83/d9a/f121 0 2024-03-22T07:16:07.909 INFO:tasks.workunit.client.0.smithi063.stdout:7/840: getdents d5/da/d5f/d33 0 2024-03-22T07:16:07.910 INFO:tasks.workunit.client.1.smithi069.stdout:4/840: rename d6/d16/d68/cc5 to d6/d12/d5d/c122 0 2024-03-22T07:16:07.911 INFO:tasks.workunit.client.0.smithi063.stdout:7/841: mkdir d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/d111 0 2024-03-22T07:16:07.913 INFO:tasks.workunit.client.0.smithi063.stdout:8/982: read d0/f7c [118617,106412] 0 2024-03-22T07:16:07.913 INFO:tasks.workunit.client.0.smithi063.stdout:8/983: read d0/f22 [500198,55313] 0 2024-03-22T07:16:07.915 INFO:tasks.workunit.client.0.smithi063.stdout:7/842: link d5/d16/d1b/c1c d5/da/d28/d4b/da8/dd4/c112 0 2024-03-22T07:16:07.915 INFO:tasks.workunit.client.0.smithi063.stdout:7/843: truncate d5/da/d28/d4b/da8/dbc/d62/d8d/dbb/f110 479155 0 2024-03-22T07:16:07.915 INFO:tasks.workunit.client.1.smithi069.stdout:0/830: write d3/d7/d90/fa6 [228889,13157] 0 2024-03-22T07:16:07.916 INFO:tasks.workunit.client.0.smithi063.stdout:8/984: symlink d0/d26/l14a 0 2024-03-22T07:16:07.918 INFO:tasks.workunit.client.0.smithi063.stdout:7/844: creat d5/da/d28/d4b/da8/dd4/dea/f113 x:0 0 0 2024-03-22T07:16:07.918 INFO:tasks.workunit.client.1.smithi069.stdout:0/831: dread d3/d7/d15/d29/d5a/d5d/fb2 [0,4194304] 0 2024-03-22T07:16:07.921 INFO:tasks.workunit.client.0.smithi063.stdout:7/845: creat d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/df0/f114 x:0 0 0 2024-03-22T07:16:07.921 INFO:tasks.workunit.client.0.smithi063.stdout:7/846: stat d5/dcf/ff3 0 2024-03-22T07:16:07.921 INFO:tasks.workunit.client.0.smithi063.stdout:4/850: dwrite d1/d5/dc/d95/d59/f64 [0,4194304] 0 2024-03-22T07:16:07.921 INFO:tasks.workunit.client.1.smithi069.stdout:0/832: creat d3/d7/d15/d1f/f11a x:0 0 0 2024-03-22T07:16:07.921 INFO:tasks.workunit.client.1.smithi069.stdout:0/833: chown d3/d7/d17/d1c/c68 133515 1 2024-03-22T07:16:07.922 INFO:tasks.workunit.client.1.smithi069.stdout:0/834: write d3/d7/d15/d29/dcf/d114/f10c [855622,26139] 0 2024-03-22T07:16:07.922 INFO:tasks.workunit.client.0.smithi063.stdout:8/985: dread d0/d26/d2d/f4d [0,4194304] 0 2024-03-22T07:16:07.922 INFO:tasks.workunit.client.0.smithi063.stdout:5/864: dwrite d8/d13/d1e/d31/d64/d65/d86/f92 [0,4194304] 0 2024-03-22T07:16:07.922 INFO:tasks.workunit.client.0.smithi063.stdout:8/986: chown d0/d24/d5b/f44 5104 1 2024-03-22T07:16:07.923 INFO:tasks.workunit.client.0.smithi063.stdout:7/847: creat d5/da/d28/d4b/da8/dbc/dfb/f115 x:0 0 0 2024-03-22T07:16:07.923 INFO:tasks.workunit.client.0.smithi063.stdout:8/987: write d0/d26/d2d/d59/f87 [1956674,51234] 0 2024-03-22T07:16:07.923 INFO:tasks.workunit.client.0.smithi063.stdout:8/988: chown d0/d1b/c11f 22 1 2024-03-22T07:16:07.924 INFO:tasks.workunit.client.1.smithi069.stdout:4/841: write d6/d9b/d1f/d3f/d51/f81 [1688428,22898] 0 2024-03-22T07:16:07.924 INFO:tasks.workunit.client.1.smithi069.stdout:2/539: dwrite d18/d4d/f22 [8388608,4194304] 0 2024-03-22T07:16:07.924 INFO:tasks.workunit.client.1.smithi069.stdout:2/540: readlink d18/d4d/d3e/lb5 0 2024-03-22T07:16:07.924 INFO:tasks.workunit.client.1.smithi069.stdout:2/541: creat d18/d91/fba x:0 0 0 2024-03-22T07:16:07.924 INFO:tasks.workunit.client.1.smithi069.stdout:2/542: truncate d18/d4d/d27/d77/f9f 391303 0 2024-03-22T07:16:07.924 INFO:tasks.workunit.client.1.smithi069.stdout:2/543: creat d18/d4d/d27/d72/fbb x:0 0 0 2024-03-22T07:16:07.924 INFO:tasks.workunit.client.1.smithi069.stdout:2/544: fsync d18/d4d/d27/f8c 0 2024-03-22T07:16:07.924 INFO:tasks.workunit.client.1.smithi069.stdout:0/835: symlink d3/d7/d15/l11b 0 2024-03-22T07:16:07.925 INFO:tasks.workunit.client.1.smithi069.stdout:0/836: readlink d3/d7/d90/l101 0 2024-03-22T07:16:07.925 INFO:tasks.workunit.client.1.smithi069.stdout:0/837: creat d3/d7/d17/d1c/f11c x:0 0 0 2024-03-22T07:16:07.925 INFO:tasks.workunit.client.1.smithi069.stdout:0/838: readlink d3/d7/d15/dad/ldd 0 2024-03-22T07:16:07.926 INFO:tasks.workunit.client.0.smithi063.stdout:4/851: dread d1/d5/d3c/fe9 [0,4194304] 0 2024-03-22T07:16:07.926 INFO:tasks.workunit.client.0.smithi063.stdout:4/852: fsync d1/d5/dc/d95/d59/d9c/dac/dcd/fc1 0 2024-03-22T07:16:07.928 INFO:tasks.workunit.client.1.smithi069.stdout:3/781: dwrite d5/d9/d26/fac [0,4194304] 0 2024-03-22T07:16:07.928 INFO:tasks.workunit.client.1.smithi069.stdout:3/782: write d5/d9/d26/d3f/d35/f6b [276391,118066] 0 2024-03-22T07:16:07.928 INFO:tasks.workunit.client.1.smithi069.stdout:4/842: unlink d6/d12/c60 0 2024-03-22T07:16:07.929 INFO:tasks.workunit.client.0.smithi063.stdout:7/848: rename d5/da/d28/d4b/da8/dd4/c112 to d5/da/d5f/d33/c116 0 2024-03-22T07:16:07.929 INFO:tasks.workunit.client.0.smithi063.stdout:7/849: write d5/da/d5f/d33/f107 [1036992,39826] 0 2024-03-22T07:16:07.931 INFO:tasks.workunit.client.1.smithi069.stdout:9/586: dwrite d2/d21/f73 [0,4194304] 0 2024-03-22T07:16:07.931 INFO:tasks.workunit.client.1.smithi069.stdout:9/587: write d2/d51/f59 [470741,69058] 0 2024-03-22T07:16:07.931 INFO:tasks.workunit.client.1.smithi069.stdout:9/588: readlink d2/dd/d3c/d2e/l7d 0 2024-03-22T07:16:07.935 INFO:tasks.workunit.client.0.smithi063.stdout:4/853: unlink d1/d5/d43/d27/d2c/d3d/cea 0 2024-03-22T07:16:07.935 INFO:tasks.workunit.client.0.smithi063.stdout:4/854: write d1/d5/dc/f5a [231832,7769] 0 2024-03-22T07:16:07.936 INFO:tasks.workunit.client.0.smithi063.stdout:7/850: mkdir d5/da/d5f/d101/d117 0 2024-03-22T07:16:07.937 INFO:tasks.workunit.client.0.smithi063.stdout:7/851: symlink d5/da/d28/d4b/da8/l118 0 2024-03-22T07:16:07.940 INFO:tasks.workunit.client.0.smithi063.stdout:5/865: write d8/d13/d1e/d3c/da0/fb9 [2842121,88988] 0 2024-03-22T07:16:07.940 INFO:tasks.workunit.client.0.smithi063.stdout:5/866: chown d8/d13/d1e/d31/d64/d65/d86/le6 30599126 1 2024-03-22T07:16:07.941 INFO:tasks.workunit.client.0.smithi063.stdout:5/867: mknod d8/d13/d28/d7d/c11a 0 2024-03-22T07:16:07.942 INFO:tasks.workunit.client.0.smithi063.stdout:5/868: mknod d8/d13/d24/da4/d8b/db6/dea/c11b 0 2024-03-22T07:16:07.946 INFO:tasks.workunit.client.1.smithi069.stdout:9/589: dread d2/f55 [4194304,4194304] 0 2024-03-22T07:16:07.951 INFO:tasks.workunit.client.0.smithi063.stdout:4/855: dread d1/d5/d3c/d65/f73 [0,4194304] 0 2024-03-22T07:16:07.953 INFO:tasks.workunit.client.1.smithi069.stdout:5/636: dwrite d1/d6/d26/d33/f4c [0,4194304] 0 2024-03-22T07:16:07.953 INFO:tasks.workunit.client.0.smithi063.stdout:4/856: truncate d1/d5/dc/d95/f49 6392036 0 2024-03-22T07:16:07.958 INFO:tasks.workunit.client.1.smithi069.stdout:2/545: truncate d18/f2b 3553740 0 2024-03-22T07:16:07.959 INFO:tasks.workunit.client.0.smithi063.stdout:5/869: dread d8/d13/d1e/f38 [0,4194304] 0 2024-03-22T07:16:07.959 INFO:tasks.workunit.client.0.smithi063.stdout:5/870: chown d8/f44 542998721 1 2024-03-22T07:16:07.959 INFO:tasks.workunit.client.0.smithi063.stdout:5/871: fsync d8/d13/f4b 0 2024-03-22T07:16:07.961 INFO:tasks.workunit.client.0.smithi063.stdout:5/872: link d8/d13/d1e/d31/d64/l115 d8/d13/d1e/d31/d64/d65/ddf/d119/l11c 0 2024-03-22T07:16:07.962 INFO:tasks.workunit.client.1.smithi069.stdout:3/783: creat d5/d9/d26/d36/d41/d8e/ffc x:0 0 0 2024-03-22T07:16:07.962 INFO:tasks.workunit.client.1.smithi069.stdout:3/784: stat d5/d9/d26 0 2024-03-22T07:16:07.962 INFO:tasks.workunit.client.1.smithi069.stdout:4/843: rename d6/daa/dce/c11f to d6/d12/d18/c123 0 2024-03-22T07:16:07.963 INFO:tasks.workunit.client.1.smithi069.stdout:4/844: read - d6/d16/d3a/db4/fc1 zero size 2024-03-22T07:16:07.963 INFO:tasks.workunit.client.1.smithi069.stdout:4/845: truncate d6/d9b/d1f/d58/d7c/f104 606369 0 2024-03-22T07:16:07.963 INFO:tasks.workunit.client.1.smithi069.stdout:3/785: read d5/fa0 [820214,31753] 0 2024-03-22T07:16:07.963 INFO:tasks.workunit.client.1.smithi069.stdout:3/786: readlink d5/d9/d26/d3f/d35/la7 0 2024-03-22T07:16:07.964 INFO:tasks.workunit.client.0.smithi063.stdout:5/873: truncate d8/d13/fdb 1793180 0 2024-03-22T07:16:07.965 INFO:tasks.workunit.client.0.smithi063.stdout:4/857: dread d1/f21 [0,4194304] 0 2024-03-22T07:16:07.968 INFO:tasks.workunit.client.0.smithi063.stdout:4/858: truncate d1/d5/d3c/d8f/fbc 331217 0 2024-03-22T07:16:07.968 INFO:tasks.workunit.client.0.smithi063.stdout:8/989: dwrite d0/f7c [4194304,4194304] 0 2024-03-22T07:16:07.969 INFO:tasks.workunit.client.1.smithi069.stdout:0/839: dwrite d3/d9a/fc3 [0,4194304] 0 2024-03-22T07:16:07.969 INFO:tasks.workunit.client.1.smithi069.stdout:5/637: write d1/d6/d26/d33/f4c [3543373,118401] 0 2024-03-22T07:16:07.974 INFO:tasks.workunit.client.0.smithi063.stdout:7/852: dwrite d5/d16/f90 [0,4194304] 0 2024-03-22T07:16:07.976 INFO:tasks.workunit.client.1.smithi069.stdout:5/638: dread d1/d10/d44/d50/f96 [0,4194304] 0 2024-03-22T07:16:07.976 INFO:tasks.workunit.client.1.smithi069.stdout:9/590: dwrite d2/dd/d3c/d2e/f32 [0,4194304] 0 2024-03-22T07:16:07.978 INFO:tasks.workunit.client.1.smithi069.stdout:2/546: dread d18/d4d/f2d [0,4194304] 0 2024-03-22T07:16:07.979 INFO:tasks.workunit.client.1.smithi069.stdout:0/840: read d3/d7/d15/d29/d5a/d84/f52 [5029183,23115] 0 2024-03-22T07:16:07.980 INFO:tasks.workunit.client.0.smithi063.stdout:7/853: mkdir d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d40/d119 0 2024-03-22T07:16:07.980 INFO:tasks.workunit.client.1.smithi069.stdout:5/639: mknod d1/d6/d88/cdc 0 2024-03-22T07:16:07.981 INFO:tasks.workunit.client.0.smithi063.stdout:5/874: dread d8/d13/f39 [0,4194304] 0 2024-03-22T07:16:07.982 INFO:tasks.workunit.client.1.smithi069.stdout:5/640: dread d1/d6/d26/d33/f4e [0,4194304] 0 2024-03-22T07:16:07.982 INFO:tasks.workunit.client.1.smithi069.stdout:5/641: fdatasync d1/d6/d86/d99/d43/d8a/fd0 0 2024-03-22T07:16:07.983 INFO:tasks.workunit.client.0.smithi063.stdout:5/875: rmdir d8/d13/d1e/d31/d64/d65 39 2024-03-22T07:16:07.983 INFO:tasks.workunit.client.0.smithi063.stdout:5/876: dread - d8/d13/d1e/d31/d64/d65/fd5 zero size 2024-03-22T07:16:07.983 INFO:tasks.workunit.client.1.smithi069.stdout:9/591: mkdir d2/d76/db8 0 2024-03-22T07:16:07.986 INFO:tasks.workunit.client.1.smithi069.stdout:2/547: symlink d18/d91/lbc 0 2024-03-22T07:16:07.986 INFO:tasks.workunit.client.1.smithi069.stdout:2/548: write d18/d4d/f28 [4762569,116852] 0 2024-03-22T07:16:07.987 INFO:tasks.workunit.client.1.smithi069.stdout:0/841: truncate d3/d7/f4a 3440761 0 2024-03-22T07:16:07.988 INFO:tasks.workunit.client.1.smithi069.stdout:0/842: chown d3/d7/d15/d29/dcf/d114/fdf 7531 1 2024-03-22T07:16:07.989 INFO:tasks.workunit.client.0.smithi063.stdout:4/859: read d1/f5b [2558010,9448] 0 2024-03-22T07:16:07.990 INFO:tasks.workunit.client.1.smithi069.stdout:5/642: rename d1/d28/f4d to d1/d6/d26/d3e/dbb/fdd 0 2024-03-22T07:16:07.992 INFO:tasks.workunit.client.1.smithi069.stdout:4/846: dwrite d6/d12/f33 [0,4194304] 0 2024-03-22T07:16:07.992 INFO:tasks.workunit.client.1.smithi069.stdout:9/592: creat d2/dd/d3c/d53/d9d/fb9 x:0 0 0 2024-03-22T07:16:07.993 INFO:tasks.workunit.client.1.smithi069.stdout:9/593: chown d2/dd/d3c 184563 1 2024-03-22T07:16:07.996 INFO:tasks.workunit.client.1.smithi069.stdout:5/643: mkdir d1/d6/d86/dab/dcf/dde 0 2024-03-22T07:16:07.996 INFO:tasks.workunit.client.1.smithi069.stdout:5/644: chown d1/d6/d21/dae/cb6 1498 1 2024-03-22T07:16:08.002 INFO:tasks.workunit.client.1.smithi069.stdout:4/847: unlink d6/d16/l72 0 2024-03-22T07:16:08.003 INFO:tasks.workunit.client.1.smithi069.stdout:4/848: creat d6/d16/d3a/db4/dbc/dc6/f124 x:0 0 0 2024-03-22T07:16:08.003 INFO:tasks.workunit.client.1.smithi069.stdout:6/617: sync 2024-03-22T07:16:08.003 INFO:tasks.workunit.client.1.smithi069.stdout:9/594: mknod d2/d76/cba 0 2024-03-22T07:16:08.005 INFO:tasks.workunit.client.1.smithi069.stdout:5/645: truncate d1/d6/d86/f77 583421 0 2024-03-22T07:16:08.005 INFO:tasks.workunit.client.1.smithi069.stdout:5/646: dread - d1/d10/d60/fc1 zero size 2024-03-22T07:16:08.005 INFO:tasks.workunit.client.1.smithi069.stdout:5/647: readlink d1/d6/d26/d3a/lc8 0 2024-03-22T07:16:08.005 INFO:tasks.workunit.client.1.smithi069.stdout:2/549: dread d18/d4d/d27/d33/d60/d64/d89/d8b/dab/fa7 [0,4194304] 0 2024-03-22T07:16:08.007 INFO:tasks.workunit.client.1.smithi069.stdout:4/849: write d6/d16/f71 [193289,76070] 0 2024-03-22T07:16:08.009 INFO:tasks.workunit.client.0.smithi063.stdout:4/860: read d1/d5/f97 [108059,28432] 0 2024-03-22T07:16:08.010 INFO:tasks.workunit.client.1.smithi069.stdout:9/595: creat d2/dd/d3c/d53/d75/fbb x:0 0 0 2024-03-22T07:16:08.011 INFO:tasks.workunit.client.1.smithi069.stdout:9/596: fsync d2/dd/d3c/d2e/f65 0 2024-03-22T07:16:08.012 INFO:tasks.workunit.client.1.smithi069.stdout:4/850: link d6/d16/d3a/db4/dfb/d1d/d55/l87 d6/d16/d3a/db4/dbc/l125 0 2024-03-22T07:16:08.015 INFO:tasks.workunit.client.1.smithi069.stdout:9/597: mknod d2/dd/d6e/cbc 0 2024-03-22T07:16:08.016 INFO:tasks.workunit.client.0.smithi063.stdout:7/854: dwrite d5/da/d28/d4b/da8/dbc/d85/fce [0,4194304] 0 2024-03-22T07:16:08.016 INFO:tasks.workunit.client.0.smithi063.stdout:8/990: dwrite d0/f22 [0,4194304] 0 2024-03-22T07:16:08.016 INFO:tasks.workunit.client.1.smithi069.stdout:2/550: getdents d18/d4d 0 2024-03-22T07:16:08.017 INFO:tasks.workunit.client.1.smithi069.stdout:6/618: dread d4/dd/d41/f51 [0,4194304] 0 2024-03-22T07:16:08.018 INFO:tasks.workunit.client.1.smithi069.stdout:2/551: mkdir d18/d4d/d27/d33/d60/dbd 0 2024-03-22T07:16:08.021 INFO:tasks.workunit.client.0.smithi063.stdout:7/855: rename c2 to d5/d16/d43/c11a 0 2024-03-22T07:16:08.021 INFO:tasks.workunit.client.0.smithi063.stdout:7/856: truncate d5/da/d28/d4b/da8/dbc/fd9 5002059 0 2024-03-22T07:16:08.021 INFO:tasks.workunit.client.1.smithi069.stdout:6/619: mkdir d4/dd/d12/d2b/d72/d73/d9d/dc3 0 2024-03-22T07:16:08.021 INFO:tasks.workunit.client.1.smithi069.stdout:6/620: creat d4/dd/d12/d44/d58/db4/fc4 x:0 0 0 2024-03-22T07:16:08.022 INFO:tasks.workunit.client.1.smithi069.stdout:1/663: sync 2024-03-22T07:16:08.022 INFO:tasks.workunit.client.1.smithi069.stdout:7/685: sync 2024-03-22T07:16:08.022 INFO:tasks.workunit.client.1.smithi069.stdout:1/664: fdatasync d1/d13/f39 0 2024-03-22T07:16:08.023 INFO:tasks.workunit.client.1.smithi069.stdout:7/686: fsync d9/db/fe3 0 2024-03-22T07:16:08.023 INFO:tasks.workunit.client.1.smithi069.stdout:8/879: sync 2024-03-22T07:16:08.023 INFO:tasks.workunit.client.1.smithi069.stdout:2/552: mknod d18/d4d/d3e/d40/d96/cbe 0 2024-03-22T07:16:08.023 INFO:tasks.workunit.client.0.smithi063.stdout:7/857: creat d5/da/d28/d4b/da8/dbc/d62/d8d/f11b x:0 0 0 2024-03-22T07:16:08.023 INFO:tasks.workunit.client.0.smithi063.stdout:7/858: link d5/da/d28/d4b/da8/dbc/d85/c91 d5/da/d28/d4b/da8/dbc/d62/d73/d1d/d29/d4a/c11c 0 2024-03-22T07:16:08.024 INFO:tasks.workunit.client.0.smithi063.stdout:7/859: getdents d5/da/d28/d4b/da8/dbc/d62/d73/d1d 0 2024-03-22T07:16:08.024 INFO:tasks.workunit.client.0.smithi063.stdout:7/860: chown d5/da/d5f/d33/fa3 15969 1 2024-03-22T07:16:08.024 INFO:tasks.workunit.client.0.smithi063.stdout:7/861: creat d5/da/d28/d4b/da8/dbc/d62/d73/dc5/f11d x:0 0 0 2024-03-22T07:16:08.024 INFO:tasks.workunit.client.0.smithi063.stdout:7/862: chown d5/da/d4e 1 1 2024-03-22T07:16:08.025 INFO:tasks.workunit.client.0.smithi063.stdout:7/863: getdents d5/da/d28/d4b 0 2024-03-22T07:16:08.025 INFO:tasks.workunit.client.1.smithi069.stdout:4/851: write d6/d9b/d1f/d58/f69 [2411977,79503] 0 2024-03-22T07:16:08.026 INFO:tasks.workunit.client.1.smithi069.stdout:4/852: readlink d6/d12/d18/l19 0 2024-03-22T07:16:08.027 INFO:tasks.workunit.client.1.smithi069.stdout:6/621: mknod d4/dd/d41/d78/da3/cc5 0 2024-03-22T07:16:08.027 INFO:tasks.workunit.client.1.smithi069.stdout:6/622: chown d4/dd/d12/d44/l4b 1074544256 1 2024-03-22T07:16:08.028 INFO:tasks.workunit.client.1.smithi069.stdout:6/623: fsync d4/f6b 0 2024-03-22T07:16:08.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:07 smithi069 ceph-mon[36441]: pgmap v6: 65 pgs: 65 active+clean; 2.6 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 57 MiB/s rd, 75 MiB/s wr, 57 op/s 2024-03-22T07:16:08.029 INFO:tasks.workunit.client.1.smithi069.stdout:7/687: mkdir d9/de4 0 2024-03-22T07:16:08.031 INFO:tasks.workunit.client.1.smithi069.stdout:8/880: rename d1/d1c/d3e/d88/cfe to d1/d1c/d3e/c11c 0 2024-03-22T07:16:08.031 INFO:tasks.workunit.client.1.smithi069.stdout:8/881: dread - d1/d1c/fff zero size 2024-03-22T07:16:08.032 INFO:tasks.workunit.client.1.smithi069.stdout:2/553: creat d18/d4d/d27/d33/d60/dbd/fbf x:0 0 0 2024-03-22T07:16:08.032 INFO:tasks.workunit.client.0.smithi063.stdout:5/877: dwrite d8/d13/fdb [0,4194304] 0 2024-03-22T07:16:08.037 INFO:tasks.workunit.client.0.smithi063.stdout:5/878: write d8/d13/d24/da4/d8b/fb5 [777677,114920] 0 2024-03-22T07:16:08.037 INFO:tasks.workunit.client.0.smithi063.stdout:5/879: truncate d8/d13/d1e/d31/d64/d65/f95 896695 0 2024-03-22T07:16:08.038 INFO:tasks.workunit.client.1.smithi069.stdout:9/598: write d2/dd/d3c/d48/f4a [3653161,94945] 0 2024-03-22T07:16:08.039 INFO:tasks.workunit.client.1.smithi069.stdout:0/843: dwrite d3/d7/d15/fe1 [0,4194304] 0 2024-03-22T07:16:08.045 INFO:tasks.workunit.client.1.smithi069.stdout:9/599: mkdir d2/d51/dbd 0 2024-03-22T07:16:08.047 INFO:tasks.workunit.client.1.smithi069.stdout:0/844: rename d3/d7/d15/d29/d5a/d99/dca/cf8 to d3/d7/d17/d1c/dd6/c11d 0 2024-03-22T07:16:08.054 INFO:tasks.workunit.client.1.smithi069.stdout:9/600: symlink d2/dd/d3c/d48/lbe 0 2024-03-22T07:16:08.054 INFO:tasks.workunit.client.1.smithi069.stdout:9/601: chown d2/d21/d89 89 1 2024-03-22T07:16:08.055 INFO:tasks.workunit.client.0.smithi063.stdout:8/991: dread d0/d1b/d4f/dde/df5/f5c [0,4194304] 0 2024-03-22T07:16:08.055 INFO:tasks.workunit.client.1.smithi069.stdout:0/845: creat d3/d7/d5e/f11e x:0 0 0 2024-03-22T07:16:08.058 INFO:tasks.workunit.client.1.smithi069.stdout:9/602: creat d2/fbf x:0 0 0 2024-03-22T07:16:08.059 INFO:tasks.workunit.client.1.smithi069.stdout:0/846: creat d3/d7/d15/d29/d5a/d5d/d98/f11f x:0 0 0 2024-03-22T07:16:08.061 INFO:tasks.workunit.client.1.smithi069.stdout:9/603: creat d2/dd/d3c/d48/d7f/d86/fc0 x:0 0 0 2024-03-22T07:16:08.074 INFO:tasks.workunit.client.0.smithi063.stdout:4/861: dwrite d1/d5/d43/f31 [0,4194304] 0 2024-03-22T07:16:08.074 INFO:tasks.workunit.client.0.smithi063.stdout:4/862: creat d1/d5/dc/d95/d59/d9c/dac/dc9/f126 x:0 0 0 2024-03-22T07:16:08.074 INFO:tasks.workunit.client.0.smithi063.stdout:4/863: creat d1/d5/dc/d95/d59/d9c/dac/dc9/f127 x:0 0 0 2024-03-22T07:16:08.074 INFO:tasks.workunit.client.0.smithi063.stdout:4/864: chown d1/d5/d43/d27/d2c/df4/f116 3614857 1 2024-03-22T07:16:08.074 INFO:tasks.workunit.client.1.smithi069.stdout:5/648: dwrite d1/d6/d86/f73 [4194304,4194304] 0 2024-03-22T07:16:08.079 INFO:tasks.workunit.client.0.smithi063.stdout:4/865: mkdir d1/d5/d43/d27/d2c/df4/d128 0 2024-03-22T07:16:08.082 INFO:tasks.workunit.client.0.smithi063.stdout:4/866: unlink d1/d5/dc2/ce0 0 2024-03-22T07:16:08.082 INFO:tasks.workunit.client.0.smithi063.stdout:4/867: dread - d1/d5/dc/d95/d59/d9c/ffa zero size 2024-03-22T07:16:08.083 INFO:tasks.workunit.client.0.smithi063.stdout:4/868: mknod d1/d5/d43/d27/d2c/df4/d128/c129 0 2024-03-22T07:16:08.083 INFO:tasks.workunit.client.0.smithi063.stdout:4/869: creat d1/d5/d43/d107/f12a x:0 0 0 2024-03-22T07:16:08.085 INFO:tasks.workunit.client.0.smithi063.stdout:4/870: rename d1/c4 to d1/d5/d43/d27/d2c/df4/d128/c12b 0 2024-03-22T07:16:08.085 INFO:tasks.workunit.client.0.smithi063.stdout:4/871: write d1/d5/d43/d27/d2c/d4a/d80/ff6 [327800,2499] 0 2024-03-22T07:16:08.086 INFO:tasks.workunit.client.0.smithi063.stdout:4/872: symlink d1/d5/d3c/d8f/dbb/d114/d79/l12c 0 2024-03-22T07:16:08.086 INFO:tasks.workunit.client.0.smithi063.stdout:4/873: chown d1/d5/d43/d27/d2c/d4a/d78/ce2 568 1 2024-03-22T07:16:08.087 INFO:tasks.workunit.client.0.smithi063.stdout:4/874: rename d1/f21 to d1/d5/d43/d27/d2c/d4a/d78/f12d 0 2024-03-22T07:16:08.087 INFO:tasks.workunit.client.0.smithi063.stdout:4/875: chown d1/d5/d43/d27/d2c/d4a/d78 449429444 1 2024-03-22T07:16:08.088 INFO:tasks.workunit.client.0.smithi063.stdout:4/876: creat d1/d5/d3c/d8f/deb/f12e x:0 0 0 2024-03-22T07:16:08.088 INFO:tasks.workunit.client.0.smithi063.stdout:4/877: creat d1/d5/d43/d27/d2c/d4a/d78/f12f x:0 0 0 2024-03-22T07:16:08.093 INFO:tasks.workunit.client.1.smithi069.stdout:4/853: dwrite d6/d16/d68/fa9 [0,4194304] 0 2024-03-22T07:16:08.094 INFO:tasks.workunit.client.1.smithi069.stdout:4/854: stat d6/d9b/d1f/d58/d7c/d90/fa5 0 2024-03-22T07:16:08.097 INFO:tasks.workunit.client.1.smithi069.stdout:2/554: dwrite d18/f1b [0,4194304] 0 2024-03-22T07:16:08.097 INFO:tasks.workunit.client.1.smithi069.stdout:2/555: write d18/d4d/d27/d33/d60/d64/f99 [333666,116873] 0 2024-03-22T07:16:08.097 INFO:tasks.workunit.client.1.smithi069.stdout:2/556: fsync d18/d4d/d27/d72/fbb 0 2024-03-22T07:16:08.099 INFO:tasks.workunit.client.1.smithi069.stdout:5/649: dread d1/d6/d26/d3e/d9c/d9d/fa6 [0,4194304] 0 2024-03-22T07:16:08.100 INFO:tasks.workunit.client.1.smithi069.stdout:8/882: dwrite d1/f29 [0,4194304] 0 2024-03-22T07:16:08.100 INFO:tasks.workunit.client.1.smithi069.stdout:4/855: rename d6/d12/db2/fe2 to d6/d9b/d1f/d58/d7c/f126 0 2024-03-22T07:16:08.100 INFO:tasks.workunit.client.0.smithi063.stdout:5/880: dwrite d8/f2b [0,4194304] 0 2024-03-22T07:16:08.101 INFO:tasks.workunit.client.1.smithi069.stdout:6/624: dwrite d4/d7/da/d3f/f61 [4194304,4194304] 0 2024-03-22T07:16:08.101 INFO:tasks.workunit.client.0.smithi063.stdout:8/992: dwrite d0/f1a [4194304,4194304] 0 2024-03-22T07:16:08.101 INFO:tasks.workunit.client.0.smithi063.stdout:8/993: creat d0/d26/f14b x:0 0 0 2024-03-22T07:16:08.102 INFO:tasks.workunit.client.0.smithi063.stdout:8/994: dread - d0/d24/d5b/f95 zero size 2024-03-22T07:16:08.102 INFO:tasks.workunit.client.1.smithi069.stdout:0/847: dwrite d3/d7/d15/d29/d5a/d84/f38 [4194304,4194304] 0 2024-03-22T07:16:08.105 INFO:tasks.workunit.client.1.smithi069.stdout:2/557: creat d18/d4d/d25/fc0 x:0 0 0 2024-03-22T07:16:08.107 INFO:tasks.workunit.client.1.smithi069.stdout:5/650: creat d1/d6/fdf x:0 0 0 2024-03-22T07:16:08.108 INFO:tasks.workunit.client.1.smithi069.stdout:8/883: rmdir d1/d1c/d3e/d88/da8/dbb 39 2024-03-22T07:16:08.108 INFO:tasks.workunit.client.1.smithi069.stdout:8/884: creat d1/d1c/d3e/d88/dcd/dbd/d106/f11d x:0 0 0 2024-03-22T07:16:08.109 INFO:tasks.workunit.client.1.smithi069.stdout:9/604: dwrite d2/d31/f67 [0,4194304] 0 2024-03-22T07:16:08.109 INFO:tasks.workunit.client.1.smithi069.stdout:9/605: chown d2/dd/d3c/d48/d7f/d86/c9a 1322 1 2024-03-22T07:16:08.110 INFO:tasks.workunit.client.1.smithi069.stdout:4/856: creat d6/dc4/db5/f127 x:0 0 0 2024-03-22T07:16:08.110 INFO:tasks.workunit.client.0.smithi063.stdout:8/995: dread d0/d24/d5b/f60 [0,4194304] 0 2024-03-22T07:16:08.110 INFO:tasks.workunit.client.0.smithi063.stdout:8/996: chown d0/d24/d33/d4c 122 1 2024-03-22T07:16:08.110 INFO:tasks.workunit.client.0.smithi063.stdout:8/997: dread - d0/d26/f14b zero size 2024-03-22T07:16:08.111 INFO:tasks.workunit.client.0.smithi063.stdout:5/881: read d8/d13/d28/df3/d114/f117 [3836922,126953] 0 2024-03-22T07:16:08.111 INFO:tasks.workunit.client.0.smithi063.stdout:4/878: dwrite d1/f3 [0,4194304] 0 2024-03-22T07:16:08.111 INFO:tasks.workunit.client.0.smithi063.stdout:8/998: mkdir d0/d1b/d4f/d14c 0 2024-03-22T07:16:08.112 INFO:tasks.workunit.client.0.smithi063.stdout:8/999: getdents d0/d26/d2d/d83/d11c/d121/d12d 0 2024-03-22T07:16:08.113 INFO:tasks.workunit.client.0.smithi063.stdout:5/882: truncate d8/d13/d28/f73 2594529 0 2024-03-22T07:16:08.113 INFO:tasks.workunit.client.1.smithi069.stdout:0/848: creat d3/d7/d15/d29/d5a/f120 x:0 0 0 2024-03-22T07:16:08.114 INFO:tasks.workunit.client.0.smithi063.stdout:2/887: sync 2024-03-22T07:16:08.114 INFO:tasks.workunit.client.0.smithi063.stdout:2/888: dread - d0/de/d12/d5a/f9d zero size 2024-03-22T07:16:08.114 INFO:tasks.workunit.client.0.smithi063.stdout:2/889: write d0/de/d1b/fae [758626,58435] 0 2024-03-22T07:16:08.114 INFO:tasks.workunit.client.0.smithi063.stdout:2/890: chown d0/de/d1b/d63/d6e/ce7 437916 1 2024-03-22T07:16:08.114 INFO:tasks.workunit.client.0.smithi063.stdout:4/879: symlink d1/d5/d3c/d65/l130 0 2024-03-22T07:16:08.116 INFO:tasks.workunit.client.1.smithi069.stdout:5/651: rename d1/d10/d60/f7b to d1/d6/d86/dab/dcf/fe0 0 2024-03-22T07:16:08.116 INFO:tasks.workunit.client.1.smithi069.stdout:5/652: write d1/d28/f7e [672561,104624] 0 2024-03-22T07:16:08.117 INFO:tasks.workunit.client.0.smithi063.stdout:5/883: unlink d8/d13/f4b 0 2024-03-22T07:16:08.117 INFO:tasks.workunit.client.1.smithi069.stdout:2/558: write d18/d4d/d3e/f86 [2687141,23899] 0 2024-03-22T07:16:08.119 INFO:tasks.workunit.client.0.smithi063.stdout:2/891: link d0/de/d1b/d63/c9f d0/de/d1b/d32/dac/dc2/c126 0 2024-03-22T07:16:08.120 INFO:tasks.workunit.client.0.smithi063.stdout:4/880: symlink d1/d5/d43/d27/d2c/d4a/d80/l131 0 2024-03-22T07:16:08.120 INFO:tasks.workunit.client.0.smithi063.stdout:4/881: chown d1/d5/d3c/d65/ld5 17657 1 2024-03-22T07:16:08.121 INFO:tasks.workunit.client.1.smithi069.stdout:8/885: unlink d1/d1c/l33 0 2024-03-22T07:16:08.124 INFO:tasks.workunit.client.1.smithi069.stdout:6/625: dread d4/d7/da/f7c [0,4194304] 0 2024-03-22T07:16:08.124 INFO:tasks.workunit.client.0.smithi063.stdout:5/884: unlink d8/d13/d1e/d3c/cec 0 2024-03-22T07:16:08.125 INFO:tasks.workunit.client.1.smithi069.stdout:0/849: dread d3/d7/d90/fa6 [0,4194304] 0 2024-03-22T07:16:08.126 INFO:tasks.workunit.client.1.smithi069.stdout:9/606: mknod d2/dd/d3c/d53/d75/cc1 0 2024-03-22T07:16:08.127 INFO:tasks.workunit.client.1.smithi069.stdout:4/857: mknod d6/d12/d18/d21/c128 0 2024-03-22T07:16:08.128 INFO:tasks.workunit.client.1.smithi069.stdout:4/858: creat d6/d12/d18/d21/f129 x:0 0 0 2024-03-22T07:16:08.128 INFO:tasks.workunit.client.1.smithi069.stdout:4/859: dread - d6/d16/d3a/db4/f112 zero size 2024-03-22T07:16:08.128 INFO:tasks.workunit.client.1.smithi069.stdout:4/860: fdatasync d6/d16/d3a/db4/f112 0 2024-03-22T07:16:08.130 INFO:tasks.workunit.client.1.smithi069.stdout:5/653: mknod d1/d10/d44/d50/ce1 0 2024-03-22T07:16:08.133 INFO:tasks.workunit.client.1.smithi069.stdout:2/559: symlink d18/d4d/d27/lc1 0 2024-03-22T07:16:08.133 INFO:tasks.workunit.client.1.smithi069.stdout:8/886: creat d1/d1c/d3e/d88/dcd/d6c/f11e x:0 0 0 2024-03-22T07:16:08.133 INFO:tasks.workunit.client.1.smithi069.stdout:8/887: fsync d1/d1c/d3e/d66/d9e/dd1/f117 0 2024-03-22T07:16:08.138 INFO:tasks.workunit.client.1.smithi069.stdout:9/607: unlink d2/dd/d3c/d53/d9d/fb9 0 2024-03-22T07:16:08.138 INFO:tasks.workunit.client.1.smithi069.stdout:9/608: write d2/d51/f57 [1638748,125911] 0 2024-03-22T07:16:08.138 INFO:tasks.workunit.client.1.smithi069.stdout:4/861: mknod d6/d4b/de9/c12a 0 2024-03-22T07:16:08.139 INFO:tasks.workunit.client.0.smithi063.stdout:5/885: dread d8/f12 [0,4194304] 0 2024-03-22T07:16:08.139 INFO:tasks.workunit.client.0.smithi063.stdout:5/886: chown d8/d13/d24/da4/c116 9 1 2024-03-22T07:16:08.139 INFO:tasks.workunit.client.0.smithi063.stdout:5/887: fdatasync d8/f25 0 2024-03-22T07:16:08.140 INFO:tasks.workunit.client.0.smithi063.stdout:5/888: creat d8/d13/d1e/d31/d64/de7/f11d x:0 0 0 2024-03-22T07:16:08.141 INFO:tasks.workunit.client.1.smithi069.stdout:2/560: creat d18/d4d/fc2 x:0 0 0 2024-03-22T07:16:08.141 INFO:tasks.workunit.client.1.smithi069.stdout:2/561: dread - d18/d4d/d27/d33/d60/d64/fac zero size 2024-03-22T07:16:08.141 INFO:tasks.workunit.client.1.smithi069.stdout:2/562: stat d18/d4d/d27/d33/d3d/f48 0 2024-03-22T07:16:08.141 INFO:tasks.workunit.client.1.smithi069.stdout:2/563: write d18/d4d/f2d [1656496,9215] 0 2024-03-22T07:16:08.142 INFO:tasks.workunit.client.1.smithi069.stdout:8/888: symlink d1/d1c/d3e/d88/dcd/d6c/d118/l11f 0 2024-03-22T07:16:08.142 INFO:tasks.workunit.client.1.smithi069.stdout:5/654: dread d1/d6/d86/d99/f90 [0,4194304] 0 2024-03-22T07:16:08.144 INFO:tasks.workunit.client.1.smithi069.stdout:0/850: getdents d3/d7/d15/d29/dcf/de9 0 2024-03-22T07:16:08.144 INFO:tasks.workunit.client.1.smithi069.stdout:0/851: fdatasync d3/d7/d17/f103 0 2024-03-22T07:16:08.144 INFO:tasks.workunit.client.1.smithi069.stdout:0/852: write d3/d7/d15/d1f/f41 [5858932,111159] 0 2024-03-22T07:16:08.145 INFO:tasks.workunit.client.1.smithi069.stdout:9/609: rmdir d2/d71 39 2024-03-22T07:16:08.145 INFO:tasks.workunit.client.1.smithi069.stdout:9/610: truncate d2/dd/d3c/d53/d75/fbb 645323 0 2024-03-22T07:16:08.148 INFO:tasks.workunit.client.1.smithi069.stdout:4/862: symlink d6/d9b/d1f/d58/l12b 0 2024-03-22T07:16:08.148 INFO:tasks.workunit.client.1.smithi069.stdout:4/863: chown d6/daa 58790 1 2024-03-22T07:16:08.148 INFO:tasks.workunit.client.1.smithi069.stdout:4/864: dread - d6/d16/d83/fe0 zero size 2024-03-22T07:16:08.149 INFO:tasks.workunit.client.1.smithi069.stdout:2/564: creat d18/d4d/d27/d33/d60/d63/d80/fc3 x:0 0 0 2024-03-22T07:16:08.149 INFO:tasks.workunit.client.0.smithi063.stdout:2/892: dwrite d0/de/d1b/d8b/fb7 [0,4194304] 0 2024-03-22T07:16:08.150 INFO:tasks.workunit.client.1.smithi069.stdout:8/889: mknod d1/d1c/d3e/d66/d9e/c120 0 2024-03-22T07:16:08.152 INFO:tasks.workunit.client.0.smithi063.stdout:2/893: rename d0/de/d74/dca/d11e/c59 to d0/de/d1b/d32/dac/c127 0 2024-03-22T07:16:08.153 INFO:tasks.workunit.client.1.smithi069.stdout:5/655: rename d1/d10/d60/f5f to d1/d10/d44/fe2 0 2024-03-22T07:16:08.153 INFO:tasks.workunit.client.1.smithi069.stdout:5/656: creat d1/d10/d60/fe3 x:0 0 0 2024-03-22T07:16:08.153 INFO:tasks.workunit.client.1.smithi069.stdout:5/657: write d1/d6/d26/d3e/f6d [16333,30768] 0 2024-03-22T07:16:08.154 INFO:tasks.workunit.client.1.smithi069.stdout:0/853: getdents d3/d7/d15/d29/d5a/d5d/d9d/df9 0 2024-03-22T07:16:08.154 INFO:tasks.workunit.client.1.smithi069.stdout:0/854: readlink d3/d7/d15/d29/d5a/d5d/d98/lcc 0 2024-03-22T07:16:08.154 INFO:tasks.workunit.client.1.smithi069.stdout:0/855: chown d3/d7/d17/d1c/dd4/f10e 281261 1 2024-03-22T07:16:08.154 INFO:tasks.workunit.client.1.smithi069.stdout:6/626: dwrite d4/dd/d12/d13/f86 [0,4194304] 0 2024-03-22T07:16:08.154 INFO:tasks.workunit.client.1.smithi069.stdout:6/627: fdatasync d4/dd/d12/d13/fa1 0 2024-03-22T07:16:08.155 INFO:tasks.workunit.client.1.smithi069.stdout:9/611: unlink d2/dd/d3c/d53/d9d/f9b 0 2024-03-22T07:16:08.156 INFO:tasks.workunit.client.1.smithi069.stdout:2/565: stat d18/d4d/d27/l6e 0 2024-03-22T07:16:08.156 INFO:tasks.workunit.client.1.smithi069.stdout:2/566: write d18/d4d/d27/d33/d60/d63/d80/fc3 [5799,112475] 0 2024-03-22T07:16:08.156 INFO:tasks.workunit.client.1.smithi069.stdout:2/567: fdatasync d18/f19 0 2024-03-22T07:16:08.158 INFO:tasks.workunit.client.1.smithi069.stdout:5/658: creat d1/d6/d26/d33/d5d/d72/d95/fe4 x:0 0 0 2024-03-22T07:16:08.158 INFO:tasks.workunit.client.1.smithi069.stdout:0/856: mkdir d3/d7/d15/d29/d5a/d5d/db4/d100/d121 0 2024-03-22T07:16:08.159 INFO:tasks.workunit.client.1.smithi069.stdout:0/857: readlink d3/d7/d15/d29/d5a/d5d/l9e 0 2024-03-22T07:16:08.160 INFO:tasks.workunit.client.1.smithi069.stdout:4/865: dread d6/d16/f108 [0,4194304] 0 2024-03-22T07:16:08.160 INFO:tasks.workunit.client.1.smithi069.stdout:4/866: creat d6/daa/dce/de4/f12c x:0 0 0 2024-03-22T07:16:08.160 INFO:tasks.workunit.client.1.smithi069.stdout:4/867: creat d6/d16/d68/df8/f12d x:0 0 0 2024-03-22T07:16:08.160 INFO:tasks.workunit.client.1.smithi069.stdout:4/868: chown d6/d9b/d56/la6 244016 1 2024-03-22T07:16:08.163 INFO:tasks.workunit.client.0.smithi063.stdout:5/889: dwrite d8/dba/fcc [0,4194304] 0 2024-03-22T07:16:08.166 INFO:tasks.workunit.client.1.smithi069.stdout:8/890: dread d1/fd [0,4194304] 0 2024-03-22T07:16:08.167 INFO:tasks.workunit.client.0.smithi063.stdout:5/890: rename d8/fd9 to d8/d13/d24/d62/f11e 0 2024-03-22T07:16:08.168 INFO:tasks.workunit.client.1.smithi069.stdout:6/628: write d4/dd/d12/d13/f86 [2822762,13775] 0 2024-03-22T07:16:08.168 INFO:tasks.workunit.client.1.smithi069.stdout:6/629: dread - d4/dd/d12/d44/d58/db4/fbc zero size 2024-03-22T07:16:08.168 INFO:tasks.workunit.client.1.smithi069.stdout:6/630: chown d4/d7/da/l84 176 1 2024-03-22T07:16:08.168 INFO:tasks.workunit.client.1.smithi069.stdout:6/631: readlink d4/dd/d12/l71 0 2024-03-22T07:16:08.168 INFO:tasks.workunit.client.1.smithi069.stdout:6/632: chown d4/dd/d12/d44/d7f/lb9 3951 1 2024-03-22T07:16:08.169 INFO:tasks.workunit.client.1.smithi069.stdout:2/568: dread d18/d4d/d25/f2e [0,4194304] 0 2024-03-22T07:16:08.172 INFO:tasks.workunit.client.1.smithi069.stdout:5/659: link d1/d6/l20 d1/d6/d26/d3a/le5 0 2024-03-22T07:16:08.172 INFO:tasks.workunit.client.1.smithi069.stdout:5/660: stat d1/d6/d86/d99/lc 0 2024-03-22T07:16:08.173 INFO:tasks.workunit.client.0.smithi063.stdout:5/891: dread d8/d13/f39 [0,4194304] 0 2024-03-22T07:16:08.173 INFO:tasks.workunit.client.0.smithi063.stdout:5/892: write d8/d13/d1e/d3c/f8c [1048746,77737] 0 2024-03-22T07:16:08.174 INFO:tasks.workunit.client.1.smithi069.stdout:0/858: creat d3/d7/d15/d29/dcf/d114/f122 x:0 0 0 2024-03-22T07:16:08.175 INFO:tasks.workunit.client.0.smithi063.stdout:5/893: symlink d8/d13/d24/d81/l11f 0 2024-03-22T07:16:08.175 INFO:tasks.workunit.client.0.smithi063.stdout:5/894: write d8/d13/d1e/f2f [2756645,8978] 0 2024-03-22T07:16:08.175 INFO:tasks.workunit.client.0.smithi063.stdout:5/895: chown d8/de9 6382189 1 2024-03-22T07:16:08.176 INFO:tasks.workunit.client.0.smithi063.stdout:5/896: rename d8/d13/f48 to d8/dba/dd7/f120 0 2024-03-22T07:16:08.177 INFO:tasks.workunit.client.0.smithi063.stdout:2/894: dwrite d0/de/d1b/f48 [0,4194304] 0 2024-03-22T07:16:08.177 INFO:tasks.workunit.client.1.smithi069.stdout:4/869: mkdir d6/d16/d3a/db4/dfb/d23/d59/d120/d12e 0 2024-03-22T07:16:08.178 INFO:tasks.workunit.client.1.smithi069.stdout:4/870: truncate d6/dc4/de1/f10b 4621113 0 2024-03-22T07:16:08.178 INFO:tasks.workunit.client.1.smithi069.stdout:8/891: creat d1/d1c/d3e/db9/f121 x:0 0 0 2024-03-22T07:16:08.178 INFO:tasks.workunit.client.1.smithi069.stdout:8/892: readlink d1/d1c/d3e/d66/d9e/lcc 0 2024-03-22T07:16:08.179 INFO:tasks.workunit.client.1.smithi069.stdout:2/569: symlink d18/d4d/d27/d33/d4f/lc4 0 2024-03-22T07:16:08.179 INFO:tasks.workunit.client.1.smithi069.stdout:2/570: fsync d18/d4d/d3e/d40/f6f 0 2024-03-22T07:16:08.180 INFO:tasks.workunit.client.1.smithi069.stdout:9/612: rmdir d2/d21/d89 39 2024-03-22T07:16:08.185 INFO:tasks.workunit.client.1.smithi069.stdout:4/871: unlink d6/d16/d68/df8/f105 0 2024-03-22T07:16:08.188 INFO:tasks.workunit.client.0.smithi063.stdout:5/897: write d8/d13/d28/f73 [1029022,45720] 0 2024-03-22T07:16:08.190 INFO:tasks.workunit.client.1.smithi069.stdout:2/571: creat d18/d4d/d27/d33/d60/d64/fc5 x:0 0 0 2024-03-22T07:16:08.191 INFO:tasks.workunit.client.1.smithi069.stdout:9/613: write d2/d71/d90/f95 [3395452,102529] 0 2024-03-22T07:16:08.191 INFO:tasks.workunit.client.1.smithi069.stdout:9/614: creat d2/d51/fc2 x:0 0 0 2024-03-22T07:16:08.191 INFO:tasks.workunit.client.1.smithi069.stdout:6/633: dwrite d4/dd/d12/d13/f40 [0,4194304] 0 2024-03-22T07:16:08.192 INFO:tasks.workunit.client.0.smithi063.stdout:2/895: dread d0/fa [4194304,4194304] 0 2024-03-22T07:16:08.192 INFO:tasks.workunit.client.1.smithi069.stdout:5/661: unlink d1/d6/d26/d33/d5d/d72/l97 0 2024-03-22T07:16:08.192 INFO:tasks.workunit.client.1.smithi069.stdout:5/662: stat d1/d6/d26/d3e/c62 0 2024-03-22T07:16:08.193 INFO:tasks.workunit.client.1.smithi069.stdout:4/872: dread d6/d16/d68/fa9 [0,4194304] 0 2024-03-22T07:16:08.194 INFO:tasks.workunit.client.1.smithi069.stdout:0/859: getdents d3/d7/d17/d1c 0 2024-03-22T07:16:08.194 INFO:tasks.workunit.client.0.smithi063.stdout:2/896: mkdir d0/de/d74/d121/d128 0 2024-03-22T07:16:08.195 INFO:tasks.workunit.client.0.smithi063.stdout:2/897: read - d0/de/d1b/d32/dac/fbf zero size 2024-03-22T07:16:08.197 INFO:tasks.workunit.client.0.smithi063.stdout:2/898: truncate d0/de/d12/ffd 1688731 0 2024-03-22T07:16:08.197 INFO:tasks.workunit.client.0.smithi063.stdout:2/899: getdents d0/de/d74/d121/d128 0 2024-03-22T07:16:08.200 INFO:tasks.workunit.client.1.smithi069.stdout:8/893: rmdir d1/d1c/d3e/d88/dcd/dbd/d106 39 2024-03-22T07:16:08.200 INFO:tasks.workunit.client.1.smithi069.stdout:8/894: write d1/d1c/d48/d5e/f8b [1404478,56469] 0 2024-03-22T07:16:08.201 INFO:tasks.workunit.client.1.smithi069.stdout:2/572: truncate d18/d4d/d27/d33/d60/d64/d89/d8b/f9d 200300 0 2024-03-22T07:16:08.202 INFO:tasks.workunit.client.1.smithi069.stdout:2/573: getdents d18/d4d/d25/db3 0 2024-03-22T07:16:08.206 INFO:tasks.workunit.client.0.smithi063.stdout:2/900: dread d0/de/d1b/d8b/fb7 [0,4194304] 0 2024-03-22T07:16:08.207 INFO:tasks.workunit.client.1.smithi069.stdout:5/663: mkdir d1/d10/d44/dd8/de6 0 2024-03-22T07:16:08.207 INFO:tasks.workunit.client.1.smithi069.stdout:5/664: dread - d1/d6/d26/d7d/fda zero size 2024-03-22T07:16:08.208 INFO:tasks.workunit.client.0.smithi063.stdout:2/901: truncate d0/de/d12/d20/f99 11404409 0 2024-03-22T07:16:08.208 INFO:tasks.workunit.client.0.smithi063.stdout:2/902: chown d0/de/d12/d20/c45 12874 1 2024-03-22T07:16:08.215 INFO:tasks.workunit.client.1.smithi069.stdout:8/895: mknod d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/d82/c122 0 2024-03-22T07:16:08.215 INFO:tasks.workunit.client.1.smithi069.stdout:6/634: dwrite d4/dd/f25 [0,4194304] 0 2024-03-22T07:16:08.215 INFO:tasks.workunit.client.1.smithi069.stdout:2/574: unlink d18/d4d/d25/f2e 0 2024-03-22T07:16:08.217 INFO:tasks.workunit.client.1.smithi069.stdout:1/665: sync 2024-03-22T07:16:08.223 INFO:tasks.workunit.client.1.smithi069.stdout:7/688: sync 2024-03-22T07:16:08.223 INFO:tasks.workunit.client.1.smithi069.stdout:3/787: sync 2024-03-22T07:16:08.223 INFO:tasks.workunit.client.1.smithi069.stdout:7/689: readlink d9/db/d1a/db1/lca 0 2024-03-22T07:16:08.223 INFO:tasks.workunit.client.1.smithi069.stdout:7/690: write d9/db/fe3 [702774,125847] 0 2024-03-22T07:16:08.223 INFO:tasks.workunit.client.1.smithi069.stdout:3/788: write d5/d9/f68 [2096454,102849] 0 2024-03-22T07:16:08.223 INFO:tasks.workunit.client.1.smithi069.stdout:3/789: truncate d5/d9/d26/d3f/d56/d7f/d90/ff0 993549 0 2024-03-22T07:16:08.225 INFO:tasks.workunit.client.1.smithi069.stdout:7/691: write d9/db/fe3 [637121,85698] 0 2024-03-22T07:16:08.225 INFO:tasks.workunit.client.1.smithi069.stdout:8/896: mkdir d1/d93/ddf/de3/d123 0 2024-03-22T07:16:08.226 INFO:tasks.workunit.client.1.smithi069.stdout:5/665: dwrite d1/f9 [0,4194304] 0 2024-03-22T07:16:08.226 INFO:tasks.workunit.client.1.smithi069.stdout:5/666: fsync d1/d6/d26/d3a/f6e 0 2024-03-22T07:16:08.226 INFO:tasks.workunit.client.1.smithi069.stdout:4/873: rmdir d6/d16 39 2024-03-22T07:16:08.226 INFO:tasks.workunit.client.1.smithi069.stdout:1/666: dread d1/f22 [0,4194304] 0 2024-03-22T07:16:08.230 INFO:tasks.workunit.client.1.smithi069.stdout:6/635: mkdir d4/d7/da/dc6 0 2024-03-22T07:16:08.230 INFO:tasks.workunit.client.1.smithi069.stdout:6/636: stat d4/dd/d12/d44/d58/db4 0 2024-03-22T07:16:08.230 INFO:tasks.workunit.client.1.smithi069.stdout:6/637: fsync d4/dd/d12/fb3 0 2024-03-22T07:16:08.230 INFO:tasks.workunit.client.1.smithi069.stdout:6/638: readlink d4/d7/l91 0 2024-03-22T07:16:08.230 INFO:tasks.workunit.client.1.smithi069.stdout:4/874: write d6/d16/f108 [2248262,50356] 0 2024-03-22T07:16:08.232 INFO:tasks.workunit.client.1.smithi069.stdout:3/790: mkdir d5/d9/d26/d36/dfd 0 2024-03-22T07:16:08.236 INFO:tasks.workunit.client.1.smithi069.stdout:6/639: mkdir d4/d7/da/dc6/dc7 0 2024-03-22T07:16:08.236 INFO:tasks.workunit.client.1.smithi069.stdout:7/692: dread d9/db/d1a/db1/d57/d44/d53/f38 [0,4194304] 0 2024-03-22T07:16:08.237 INFO:tasks.workunit.client.1.smithi069.stdout:7/693: fdatasync d9/db/fe3 0 2024-03-22T07:16:08.238 INFO:tasks.workunit.client.1.smithi069.stdout:2/575: getdents d18/d4d/d27 0 2024-03-22T07:16:08.238 INFO:tasks.workunit.client.1.smithi069.stdout:2/576: fsync d18/f3a 0 2024-03-22T07:16:08.239 INFO:tasks.workunit.client.1.smithi069.stdout:2/577: chown d18/d4d/d27/d56 84823 1 2024-03-22T07:16:08.239 INFO:tasks.workunit.client.1.smithi069.stdout:8/897: dread d1/d1c/d48/d5e/f8b [0,4194304] 0 2024-03-22T07:16:08.239 INFO:tasks.workunit.client.1.smithi069.stdout:8/898: write d1/d1c/d3e/d88/dcd/d6c/f92 [812701,19919] 0 2024-03-22T07:16:08.239 INFO:tasks.workunit.client.1.smithi069.stdout:8/899: dread - d1/d1c/d3e/db9/f121 zero size 2024-03-22T07:16:08.240 INFO:tasks.workunit.client.1.smithi069.stdout:5/667: dread d1/d6/d26/d3a/f6e [0,4194304] 0 2024-03-22T07:16:08.240 INFO:tasks.workunit.client.1.smithi069.stdout:5/668: chown d1/d6/d26/d33/d5d/d72 335694792 1 2024-03-22T07:16:08.241 INFO:tasks.workunit.client.1.smithi069.stdout:5/669: write d1/d10/d60/f83 [379057,40605] 0 2024-03-22T07:16:08.241 INFO:tasks.workunit.client.1.smithi069.stdout:9/615: getdents d2/d51 0 2024-03-22T07:16:08.244 INFO:tasks.workunit.client.1.smithi069.stdout:4/875: mknod d6/d12/db2/c12f 0 2024-03-22T07:16:08.244 INFO:tasks.workunit.client.1.smithi069.stdout:4/876: readlink d6/lb 0 2024-03-22T07:16:08.244 INFO:tasks.workunit.client.1.smithi069.stdout:4/877: chown d6/d9b/l75 32773 1 2024-03-22T07:16:08.249 INFO:tasks.workunit.client.1.smithi069.stdout:3/791: symlink d5/d9/lfe 0 2024-03-22T07:16:08.250 INFO:tasks.workunit.client.1.smithi069.stdout:2/578: read d18/d4d/d27/d72/f7d [643336,111812] 0 2024-03-22T07:16:08.250 INFO:tasks.workunit.client.1.smithi069.stdout:6/640: creat d4/dd/d12/d2b/d72/d73/d9d/fc8 x:0 0 0 2024-03-22T07:16:08.252 INFO:tasks.workunit.client.1.smithi069.stdout:6/641: read d4/d7/da/d3f/f99 [20135,48213] 0 2024-03-22T07:16:08.255 INFO:tasks.workunit.client.1.smithi069.stdout:5/670: symlink d1/d6/d86/d99/le7 0 2024-03-22T07:16:08.255 INFO:tasks.workunit.client.1.smithi069.stdout:5/671: truncate d1/d10/d44/fb4 716580 0 2024-03-22T07:16:08.256 INFO:tasks.workunit.client.1.smithi069.stdout:9/616: mknod d2/d83/cc3 0 2024-03-22T07:16:08.258 INFO:tasks.workunit.client.1.smithi069.stdout:4/878: getdents d6/d9b/d1f 0 2024-03-22T07:16:08.258 INFO:tasks.workunit.client.1.smithi069.stdout:4/879: chown d6/d16/d3a/db4/dfb/d23/d59/fe6 4 1 2024-03-22T07:16:08.258 INFO:tasks.workunit.client.1.smithi069.stdout:4/880: chown d6/d16/d3a/db4/dfb/d1d/c35 96540 1 2024-03-22T07:16:08.259 INFO:tasks.workunit.client.1.smithi069.stdout:8/900: dwrite d1/d1c/d3e/f46 [0,4194304] 0 2024-03-22T07:16:08.260 INFO:tasks.workunit.client.0.smithi063.stdout:7/864: sync 2024-03-22T07:16:08.261 INFO:tasks.workunit.client.1.smithi069.stdout:3/792: rename d5/d9/d26/d36/d41/d51 to d5/d9/db/dff 0 2024-03-22T07:16:08.262 INFO:tasks.workunit.client.1.smithi069.stdout:2/579: unlink d18/c32 0 2024-03-22T07:16:08.265 INFO:tasks.workunit.client.1.smithi069.stdout:8/901: dread d1/d1c/d3e/db9/ff9 [0,4194304] 0 2024-03-22T07:16:08.265 INFO:tasks.workunit.client.1.smithi069.stdout:8/902: fdatasync d1/d1c/d3e/db9/f121 0 2024-03-22T07:16:08.266 INFO:tasks.workunit.client.1.smithi069.stdout:7/694: rmdir d9/db/d1a/db1/d57/d44/d53/d74 39 2024-03-22T07:16:08.267 INFO:tasks.workunit.client.0.smithi063.stdout:7/865: rename d5/da/d28/d4b to d5/da/d28/d11e 0 2024-03-22T07:16:08.267 INFO:tasks.workunit.client.0.smithi063.stdout:7/866: chown d5/da/c5b 1956342 1 2024-03-22T07:16:08.268 INFO:tasks.workunit.client.1.smithi069.stdout:5/672: link d1/d6/d26/d3a/c3c d1/d6/d88/ce8 0 2024-03-22T07:16:08.269 INFO:tasks.workunit.client.0.smithi063.stdout:7/867: mkdir d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d40/d119/d11f 0 2024-03-22T07:16:08.270 INFO:tasks.workunit.client.0.smithi063.stdout:7/868: creat d5/d16/d43/d78/f120 x:0 0 0 2024-03-22T07:16:08.271 INFO:tasks.workunit.client.0.smithi063.stdout:7/869: readlink d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d29/df0/l10b 0 2024-03-22T07:16:08.271 INFO:tasks.workunit.client.1.smithi069.stdout:4/881: mknod d6/d9b/d1f/d58/d7c/c130 0 2024-03-22T07:16:08.271 INFO:tasks.workunit.client.1.smithi069.stdout:4/882: write d6/d12/d18/d21/f2f [5064900,49412] 0 2024-03-22T07:16:08.271 INFO:tasks.workunit.client.1.smithi069.stdout:9/617: dread d2/f5 [0,4194304] 0 2024-03-22T07:16:08.271 INFO:tasks.workunit.client.1.smithi069.stdout:3/793: symlink d5/d9/d26/d3f/d56/d8a/da5/d7b/dbc/l100 0 2024-03-22T07:16:08.274 INFO:tasks.workunit.client.1.smithi069.stdout:6/642: truncate d4/dd/f25 3558656 0 2024-03-22T07:16:08.277 INFO:tasks.workunit.client.1.smithi069.stdout:8/903: symlink d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/l124 0 2024-03-22T07:16:08.278 INFO:tasks.workunit.client.1.smithi069.stdout:8/904: chown d1/d1c/lc0 58 1 2024-03-22T07:16:08.278 INFO:tasks.workunit.client.1.smithi069.stdout:8/905: chown d1/d1c/d3e/dde/d10a/c11b 2218352 1 2024-03-22T07:16:08.279 INFO:tasks.workunit.client.1.smithi069.stdout:7/695: unlink d9/db/d48/d4b/f77 0 2024-03-22T07:16:08.282 INFO:tasks.workunit.client.1.smithi069.stdout:4/883: link d6/d16/d83/f113 d6/d16/d3a/db4/dbc/f131 0 2024-03-22T07:16:08.290 INFO:tasks.workunit.client.1.smithi069.stdout:3/794: link d5/d9/d23/laa d5/d9/d26/l101 0 2024-03-22T07:16:08.292 INFO:tasks.workunit.client.1.smithi069.stdout:9/618: dwrite d2/dd/d3c/d2e/f4b [0,4194304] 0 2024-03-22T07:16:08.292 INFO:tasks.workunit.client.1.smithi069.stdout:9/619: fdatasync d2/dd/d3c/d2e/f4e 0 2024-03-22T07:16:08.293 INFO:tasks.workunit.client.1.smithi069.stdout:3/795: dread d5/d9/d26/d3f/d56/d7f/d90/ff0 [0,4194304] 0 2024-03-22T07:16:08.296 INFO:tasks.workunit.client.1.smithi069.stdout:2/580: rmdir d18 39 2024-03-22T07:16:08.296 INFO:tasks.workunit.client.1.smithi069.stdout:2/581: write d18/d91/fba [176400,43911] 0 2024-03-22T07:16:08.297 INFO:tasks.workunit.client.1.smithi069.stdout:4/884: dwrite d6/d12/f6f [0,4194304] 0 2024-03-22T07:16:08.298 INFO:tasks.workunit.client.1.smithi069.stdout:6/643: truncate d4/f77 4039212 0 2024-03-22T07:16:08.300 INFO:tasks.workunit.client.1.smithi069.stdout:8/906: creat d1/d93/ddf/de3/d123/f125 x:0 0 0 2024-03-22T07:16:08.300 INFO:tasks.workunit.client.1.smithi069.stdout:7/696: dread - d9/db/d1a/db1/d57/d44/d53/dc7/f9a zero size 2024-03-22T07:16:08.304 INFO:tasks.workunit.client.1.smithi069.stdout:9/620: unlink d2/dd/d3c/d2e/c52 0 2024-03-22T07:16:08.304 INFO:tasks.workunit.client.1.smithi069.stdout:9/621: readlink d2/dd/d3c/d2e/l7d 0 2024-03-22T07:16:08.309 INFO:tasks.workunit.client.1.smithi069.stdout:3/796: link d5/d9/d26/d3f/d35/f8d d5/d9/d26/d3f/d54/d9d/ded/f102 0 2024-03-22T07:16:08.315 INFO:tasks.workunit.client.1.smithi069.stdout:4/885: creat d6/d16/d3a/db4/f132 x:0 0 0 2024-03-22T07:16:08.315 INFO:tasks.workunit.client.1.smithi069.stdout:4/886: truncate d6/d9b/d1f/d58/d7c/d90/fa5 5060739 0 2024-03-22T07:16:08.318 INFO:tasks.workunit.client.1.smithi069.stdout:6/644: rename d4/dd/d12/d44/d58/f62 to d4/d7/da/d3f/fc9 0 2024-03-22T07:16:08.319 INFO:tasks.workunit.client.1.smithi069.stdout:6/645: truncate d4/dd/d12/d44/d58/f63 1574779 0 2024-03-22T07:16:08.320 INFO:tasks.workunit.client.1.smithi069.stdout:8/907: creat d1/d1c/d3e/dde/d10a/f126 x:0 0 0 2024-03-22T07:16:08.320 INFO:tasks.workunit.client.1.smithi069.stdout:8/908: chown d1/d1c/d48/d56 273563198 1 2024-03-22T07:16:08.321 INFO:tasks.workunit.client.1.smithi069.stdout:3/797: creat d5/d9/d26/d3f/f103 x:0 0 0 2024-03-22T07:16:08.321 INFO:tasks.workunit.client.1.smithi069.stdout:3/798: creat d5/d9/d26/d36/d41/df9/f104 x:0 0 0 2024-03-22T07:16:08.322 INFO:tasks.workunit.client.1.smithi069.stdout:7/697: dwrite d9/db/d1a/db1/d57/d44/f47 [0,4194304] 0 2024-03-22T07:16:08.323 INFO:tasks.workunit.client.1.smithi069.stdout:9/622: dwrite d2/dd/d3c/d48/f5d [0,4194304] 0 2024-03-22T07:16:08.323 INFO:tasks.workunit.client.1.smithi069.stdout:9/623: creat d2/d31/fc4 x:0 0 0 2024-03-22T07:16:08.323 INFO:tasks.workunit.client.1.smithi069.stdout:9/624: read d2/d51/f57 [4307985,63753] 0 2024-03-22T07:16:08.323 INFO:tasks.workunit.client.1.smithi069.stdout:6/646: rename d4/dd/d12/d44/d7f/l81 to d4/dd/lca 0 2024-03-22T07:16:08.325 INFO:tasks.workunit.client.1.smithi069.stdout:8/909: symlink d1/d1c/d3e/d66/d9e/dd1/l127 0 2024-03-22T07:16:08.327 INFO:tasks.workunit.client.1.smithi069.stdout:3/799: rmdir d5/d9/d26/d36 39 2024-03-22T07:16:08.333 INFO:tasks.workunit.client.1.smithi069.stdout:7/698: link d9/da1/la3 d9/db/d1a/db1/d57/d75/de2/le5 0 2024-03-22T07:16:08.338 INFO:tasks.workunit.client.1.smithi069.stdout:9/625: truncate d2/dd/d3c/d48/f4d 1764923 0 2024-03-22T07:16:08.340 INFO:tasks.workunit.client.1.smithi069.stdout:6/647: creat d4/dd/d12/d44/fcb x:0 0 0 2024-03-22T07:16:08.340 INFO:tasks.workunit.client.1.smithi069.stdout:6/648: chown d4/dd/c3d 63 1 2024-03-22T07:16:08.343 INFO:tasks.workunit.client.1.smithi069.stdout:8/910: creat d1/d1c/d3e/d88/dcd/d6c/d73/dfa/f128 x:0 0 0 2024-03-22T07:16:08.344 INFO:tasks.workunit.client.1.smithi069.stdout:3/800: getdents d5/d9/d26 0 2024-03-22T07:16:08.344 INFO:tasks.workunit.client.0.smithi063.stdout:4/882: sync 2024-03-22T07:16:08.346 INFO:tasks.workunit.client.0.smithi063.stdout:4/883: mknod d1/d5/dc/d95/da4/c132 0 2024-03-22T07:16:08.346 INFO:tasks.workunit.client.0.smithi063.stdout:4/884: creat d1/d5/d43/d27/d2c/f133 x:0 0 0 2024-03-22T07:16:08.346 INFO:tasks.workunit.client.0.smithi063.stdout:4/885: fsync d1/d5/f9 0 2024-03-22T07:16:08.346 INFO:tasks.workunit.client.0.smithi063.stdout:4/886: write d1/d5/f97 [3797675,88121] 0 2024-03-22T07:16:08.346 INFO:tasks.workunit.client.0.smithi063.stdout:4/887: creat d1/d5/dc/d95/da4/d104/f134 x:0 0 0 2024-03-22T07:16:08.346 INFO:tasks.workunit.client.0.smithi063.stdout:4/888: write d1/d5/d43/d27/d2c/d4a/d78/f12d [5214691,5002] 0 2024-03-22T07:16:08.347 INFO:tasks.workunit.client.1.smithi069.stdout:9/626: creat d2/d83/fc5 x:0 0 0 2024-03-22T07:16:08.348 INFO:tasks.workunit.client.0.smithi063.stdout:4/889: creat d1/d5/f135 x:0 0 0 2024-03-22T07:16:08.348 INFO:tasks.workunit.client.1.smithi069.stdout:8/911: stat d1/d1c/d70/lb2 0 2024-03-22T07:16:08.348 INFO:tasks.workunit.client.1.smithi069.stdout:8/912: chown d1/d1c/d3e/d88/dcd/d20/l42 472 1 2024-03-22T07:16:08.349 INFO:tasks.workunit.client.0.smithi063.stdout:4/890: mkdir d1/d5/dc/d95/da4/d136 0 2024-03-22T07:16:08.349 INFO:tasks.workunit.client.1.smithi069.stdout:6/649: write d4/dd/d12/d13/f40 [3916199,22909] 0 2024-03-22T07:16:08.349 INFO:tasks.workunit.client.1.smithi069.stdout:6/650: write d4/dd/d12/d13/fa1 [4609083,2495] 0 2024-03-22T07:16:08.349 INFO:tasks.workunit.client.0.smithi063.stdout:4/891: mknod d1/d5/d43/d27/d2c/d3d/d54/ddb/c137 0 2024-03-22T07:16:08.349 INFO:tasks.workunit.client.0.smithi063.stdout:4/892: chown d1/d5/dc/d95/da4/d104/f120 0 1 2024-03-22T07:16:08.350 INFO:tasks.workunit.client.1.smithi069.stdout:3/801: truncate d5/f8f 946652 0 2024-03-22T07:16:08.350 INFO:tasks.workunit.client.1.smithi069.stdout:3/802: write d5/d9/d26/d3f/f21 [1252166,58510] 0 2024-03-22T07:16:08.350 INFO:tasks.workunit.client.1.smithi069.stdout:3/803: write d5/f5d [2029552,121901] 0 2024-03-22T07:16:08.352 INFO:tasks.workunit.client.1.smithi069.stdout:9/627: unlink d2/dd/d3c/d2e/l7d 0 2024-03-22T07:16:08.352 INFO:tasks.workunit.client.1.smithi069.stdout:7/699: dwrite d9/db/d1a/f39 [0,4194304] 0 2024-03-22T07:16:08.353 INFO:tasks.workunit.client.1.smithi069.stdout:7/700: read d9/db/d1a/f85 [286637,1709] 0 2024-03-22T07:16:08.353 INFO:tasks.workunit.client.1.smithi069.stdout:7/701: readlink d9/db/d1a/db1/lca 0 2024-03-22T07:16:08.357 INFO:tasks.workunit.client.1.smithi069.stdout:8/913: mkdir d1/d1c/d3e/db9/d112/d129 0 2024-03-22T07:16:08.365 INFO:tasks.workunit.client.1.smithi069.stdout:6/651: creat d4/dd/d12/d2b/da5/fcc x:0 0 0 2024-03-22T07:16:08.365 INFO:tasks.workunit.client.1.smithi069.stdout:6/652: chown d4/d7/da/d3f/l8d 289 1 2024-03-22T07:16:08.367 INFO:tasks.workunit.client.1.smithi069.stdout:9/628: mkdir d2/d31/dc6 0 2024-03-22T07:16:08.367 INFO:tasks.workunit.client.1.smithi069.stdout:9/629: dread - d2/dd/d3c/d2e/d84/f96 zero size 2024-03-22T07:16:08.369 INFO:tasks.workunit.client.1.smithi069.stdout:8/914: mknod d1/d1c/d3e/c12a 0 2024-03-22T07:16:08.369 INFO:tasks.workunit.client.1.smithi069.stdout:8/915: chown d1/d1c/d3e/d88/da8/dbb/fd8 661 1 2024-03-22T07:16:08.371 INFO:tasks.workunit.client.1.smithi069.stdout:9/630: mknod d2/d21/cc7 0 2024-03-22T07:16:08.371 INFO:tasks.workunit.client.1.smithi069.stdout:9/631: dread - d2/dd/fac zero size 2024-03-22T07:16:08.372 INFO:tasks.workunit.client.1.smithi069.stdout:8/916: symlink d1/d1c/d3e/d88/dcd/d6c/d118/l12b 0 2024-03-22T07:16:08.372 INFO:tasks.workunit.client.1.smithi069.stdout:8/917: fsync d1/d1c/d3e/d66/f10b 0 2024-03-22T07:16:08.373 INFO:tasks.workunit.client.1.smithi069.stdout:9/632: creat d2/dd/d3c/d2e/d84/da5/fc8 x:0 0 0 2024-03-22T07:16:08.384 INFO:tasks.workunit.client.1.smithi069.stdout:8/918: dread d1/d1c/d3e/d88/dcd/d20/f3c [0,4194304] 0 2024-03-22T07:16:08.390 INFO:tasks.workunit.client.1.smithi069.stdout:7/702: dwrite d9/db/d48/d4b/faf [0,4194304] 0 2024-03-22T07:16:08.390 INFO:tasks.workunit.client.1.smithi069.stdout:7/703: write d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/dce/fd0 [249639,59138] 0 2024-03-22T07:16:08.390 INFO:tasks.workunit.client.1.smithi069.stdout:7/704: fdatasync d9/db/d48/f5b 0 2024-03-22T07:16:08.393 INFO:tasks.workunit.client.0.smithi063.stdout:4/893: dwrite d1/d5/dc/d95/d59/d9c/ffa [0,4194304] 0 2024-03-22T07:16:08.393 INFO:tasks.workunit.client.0.smithi063.stdout:4/894: chown d1/d5/d43/d27/d2c/d4a/f4d 3 1 2024-03-22T07:16:08.394 INFO:tasks.workunit.client.1.smithi069.stdout:3/804: dwrite d5/d9/d26/d3f/d54/feb [0,4194304] 0 2024-03-22T07:16:08.394 INFO:tasks.workunit.client.1.smithi069.stdout:9/633: dwrite d2/d31/f67 [4194304,4194304] 0 2024-03-22T07:16:08.404 INFO:tasks.workunit.client.1.smithi069.stdout:8/919: write d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/f76 [1480197,112615] 0 2024-03-22T07:16:08.404 INFO:tasks.workunit.client.1.smithi069.stdout:6/653: truncate d4/dd/d12/d13/f40 1338580 0 2024-03-22T07:16:08.404 INFO:tasks.workunit.client.1.smithi069.stdout:6/654: symlink d4/dd/d41/d78/d9a/lcd 0 2024-03-22T07:16:08.404 INFO:tasks.workunit.client.1.smithi069.stdout:6/655: truncate d4/dd/d41/d78/f85 1894276 0 2024-03-22T07:16:08.405 INFO:tasks.workunit.client.1.smithi069.stdout:7/705: dread d9/db/d1a/fc3 [0,4194304] 0 2024-03-22T07:16:08.406 INFO:tasks.workunit.client.1.smithi069.stdout:8/920: read d1/d93/ffc [883712,75641] 0 2024-03-22T07:16:08.406 INFO:tasks.workunit.client.1.smithi069.stdout:6/656: creat d4/dd/d12/d2b/d72/d73/fce x:0 0 0 2024-03-22T07:16:08.407 INFO:tasks.workunit.client.0.smithi063.stdout:2/903: sync 2024-03-22T07:16:08.407 INFO:tasks.workunit.client.0.smithi063.stdout:5/898: sync 2024-03-22T07:16:08.408 INFO:tasks.workunit.client.0.smithi063.stdout:2/904: truncate d0/de/d1b/d63/f86 896433 0 2024-03-22T07:16:08.408 INFO:tasks.workunit.client.1.smithi069.stdout:8/921: creat d1/d1c/d3e/d88/dcd/d6c/d4e/f12c x:0 0 0 2024-03-22T07:16:08.408 INFO:tasks.workunit.client.1.smithi069.stdout:8/922: fdatasync d1/d1c/f68 0 2024-03-22T07:16:08.408 INFO:tasks.workunit.client.1.smithi069.stdout:8/923: read - d1/d1c/d3e/d88/dcd/d6c/d4e/f12c zero size 2024-03-22T07:16:08.408 INFO:tasks.workunit.client.1.smithi069.stdout:3/805: write d5/d9/d26/fac [1707384,116700] 0 2024-03-22T07:16:08.409 INFO:tasks.workunit.client.1.smithi069.stdout:6/657: mkdir d4/dd/d12/d44/d7f/dcf 0 2024-03-22T07:16:08.410 INFO:tasks.workunit.client.0.smithi063.stdout:2/905: mkdir d0/de/d74/dca/d11e/d7f/d129 0 2024-03-22T07:16:08.410 INFO:tasks.workunit.client.0.smithi063.stdout:2/906: creat d0/de/d1b/d32/dac/dc2/f12a x:0 0 0 2024-03-22T07:16:08.410 INFO:tasks.workunit.client.0.smithi063.stdout:2/907: write d0/de/d1b/d32/dac/dc2/fc8 [536418,113262] 0 2024-03-22T07:16:08.410 INFO:tasks.workunit.client.0.smithi063.stdout:2/908: dread - d0/de/d1b/d63/dd6/fe4 zero size 2024-03-22T07:16:08.410 INFO:tasks.workunit.client.0.smithi063.stdout:2/909: creat d0/de/d1b/d63/f12b x:0 0 0 2024-03-22T07:16:08.410 INFO:tasks.workunit.client.0.smithi063.stdout:2/910: readlink d0/de/d74/dca/d11e/d7f/d97/da2/la8 0 2024-03-22T07:16:08.411 INFO:tasks.workunit.client.1.smithi069.stdout:9/634: read d2/d51/f56 [898659,95583] 0 2024-03-22T07:16:08.411 INFO:tasks.workunit.client.1.smithi069.stdout:9/635: write d2/dd/d3c/d48/d7f/d86/f8e [212706,14193] 0 2024-03-22T07:16:08.411 INFO:tasks.workunit.client.1.smithi069.stdout:9/636: write d2/dd/d3c/d2e/f6a [1487148,45769] 0 2024-03-22T07:16:08.411 INFO:tasks.workunit.client.1.smithi069.stdout:9/637: chown d2/d21/c82 218 1 2024-03-22T07:16:08.413 INFO:tasks.workunit.client.1.smithi069.stdout:3/806: link d5/d9/cb2 d5/d9/d26/d3f/d56/c105 0 2024-03-22T07:16:08.414 INFO:tasks.workunit.client.1.smithi069.stdout:6/658: mknod d4/dd/d12/d2b/d72/d73/d9d/cd0 0 2024-03-22T07:16:08.414 INFO:tasks.workunit.client.1.smithi069.stdout:6/659: fsync d4/d7/da/d3f/f61 0 2024-03-22T07:16:08.414 INFO:tasks.workunit.client.1.smithi069.stdout:6/660: chown d4/d7/f35 299598716 1 2024-03-22T07:16:08.415 INFO:tasks.workunit.client.1.smithi069.stdout:7/706: dread d9/db/d1a/db1/d57/d44/d53/d74/f98 [0,4194304] 0 2024-03-22T07:16:08.415 INFO:tasks.workunit.client.1.smithi069.stdout:8/924: getdents d1/d1c/d70 0 2024-03-22T07:16:08.417 INFO:tasks.workunit.client.1.smithi069.stdout:9/638: link d2/l79 d2/dd/d3c/d2e/lc9 0 2024-03-22T07:16:08.417 INFO:tasks.workunit.client.1.smithi069.stdout:9/639: fdatasync d2/dd/fac 0 2024-03-22T07:16:08.418 INFO:tasks.workunit.client.1.smithi069.stdout:3/807: unlink d5/d9/d26/d36/fcc 0 2024-03-22T07:16:08.420 INFO:tasks.workunit.client.1.smithi069.stdout:8/925: rename d1/d1c/d48/d56/fc9 to d1/d1c/d70/f12d 0 2024-03-22T07:16:08.420 INFO:tasks.workunit.client.1.smithi069.stdout:8/926: fsync d1/d1c/d3e/fee 0 2024-03-22T07:16:08.420 INFO:tasks.workunit.client.1.smithi069.stdout:8/927: write d1/f9 [294466,85255] 0 2024-03-22T07:16:08.421 INFO:tasks.workunit.client.1.smithi069.stdout:7/707: dread d9/db/d1a/db1/d57/d44/d53/f38 [0,4194304] 0 2024-03-22T07:16:08.421 INFO:tasks.workunit.client.1.smithi069.stdout:9/640: creat d2/dd/d3c/d48/fca x:0 0 0 2024-03-22T07:16:08.421 INFO:tasks.workunit.client.1.smithi069.stdout:9/641: write d2/d71/d90/f95 [2723866,73620] 0 2024-03-22T07:16:08.421 INFO:tasks.workunit.client.1.smithi069.stdout:9/642: readlink d2/dd/l2c 0 2024-03-22T07:16:08.426 INFO:tasks.workunit.client.1.smithi069.stdout:3/808: creat d5/d9/d26/d3f/d56/f106 x:0 0 0 2024-03-22T07:16:08.426 INFO:tasks.workunit.client.1.smithi069.stdout:3/809: chown d5/d9/d26/f5a 0 1 2024-03-22T07:16:08.426 INFO:tasks.workunit.client.1.smithi069.stdout:7/708: truncate d9/db/d1a/db1/d57/d75/d7a/fad 307985 0 2024-03-22T07:16:08.426 INFO:tasks.workunit.client.1.smithi069.stdout:9/643: mknod d2/dd/d3c/d2e/d84/ccb 0 2024-03-22T07:16:08.426 INFO:tasks.workunit.client.1.smithi069.stdout:9/644: dread - d2/d21/d89/fa4 zero size 2024-03-22T07:16:08.426 INFO:tasks.workunit.client.1.smithi069.stdout:9/645: truncate d2/dd/d3c/d48/f5d 4223437 0 2024-03-22T07:16:08.426 INFO:tasks.workunit.client.1.smithi069.stdout:9/646: chown d2/dd/d3c/d48/d7c/c91 15 1 2024-03-22T07:16:08.426 INFO:tasks.workunit.client.1.smithi069.stdout:3/810: truncate d5/d9/d26/d36/d41/f75 1270174 0 2024-03-22T07:16:08.427 INFO:tasks.workunit.client.1.smithi069.stdout:3/811: chown d5/f5d 69931 1 2024-03-22T07:16:08.427 INFO:tasks.workunit.client.1.smithi069.stdout:7/709: getdents d9/db/d1a/db1/d57/d75 0 2024-03-22T07:16:08.428 INFO:tasks.workunit.client.1.smithi069.stdout:9/647: symlink d2/dd/d3c/d48/d7f/da2/lcc 0 2024-03-22T07:16:08.429 INFO:tasks.workunit.client.1.smithi069.stdout:6/661: dread d4/fb [0,4194304] 0 2024-03-22T07:16:08.429 INFO:tasks.workunit.client.1.smithi069.stdout:6/662: chown d4/d7/da/d3f/l8d 219503 1 2024-03-22T07:16:08.432 INFO:tasks.workunit.client.1.smithi069.stdout:7/710: creat d9/db/d1a/db1/d57/d44/d53/d74/d6e/fe6 x:0 0 0 2024-03-22T07:16:08.432 INFO:tasks.workunit.client.0.smithi063.stdout:2/911: dwrite d0/de/d35/ffe [0,4194304] 0 2024-03-22T07:16:08.435 INFO:tasks.workunit.client.1.smithi069.stdout:9/648: rename d2/dd/d3c/d48/f5d to d2/d71/fcd 0 2024-03-22T07:16:08.435 INFO:tasks.workunit.client.0.smithi063.stdout:2/912: unlink d0/de/d12/d20/de9/leb 0 2024-03-22T07:16:08.435 INFO:tasks.workunit.client.0.smithi063.stdout:2/913: truncate d0/de/d12/d20/fd3 4983230 0 2024-03-22T07:16:08.436 INFO:tasks.workunit.client.1.smithi069.stdout:8/928: dread d1/d1c/d3e/d88/dcd/d6c/d73/fea [0,4194304] 0 2024-03-22T07:16:08.436 INFO:tasks.workunit.client.0.smithi063.stdout:2/914: symlink d0/de/d74/dca/d11e/d7f/d97/da2/dfa/l12c 0 2024-03-22T07:16:08.437 INFO:tasks.workunit.client.1.smithi069.stdout:3/812: getdents d5/d9/d26/d3f/d56/d8a/da5 0 2024-03-22T07:16:08.437 INFO:tasks.workunit.client.1.smithi069.stdout:3/813: chown d5/d9/d26/d36/d41/df9/ffb 587228499 1 2024-03-22T07:16:08.438 INFO:tasks.workunit.client.1.smithi069.stdout:7/711: truncate f4 3395147 0 2024-03-22T07:16:08.443 INFO:tasks.workunit.client.1.smithi069.stdout:9/649: unlink d2/d51/c60 0 2024-03-22T07:16:08.443 INFO:tasks.workunit.client.1.smithi069.stdout:9/650: chown d2/d71/d90/lb3 20 1 2024-03-22T07:16:08.446 INFO:tasks.workunit.client.1.smithi069.stdout:8/929: creat d1/d1c/d3e/d88/dcd/d6c/d4e/d98/f12e x:0 0 0 2024-03-22T07:16:08.447 INFO:tasks.workunit.client.1.smithi069.stdout:9/651: dread d2/f55 [4194304,4194304] 0 2024-03-22T07:16:08.448 INFO:tasks.workunit.client.1.smithi069.stdout:3/814: symlink d5/d9/d26/d3f/d56/d7f/l107 0 2024-03-22T07:16:08.449 INFO:tasks.workunit.client.1.smithi069.stdout:7/712: mknod d9/db/d1a/db1/ce7 0 2024-03-22T07:16:08.450 INFO:tasks.workunit.client.1.smithi069.stdout:7/713: stat d9/db/d1a/f82 0 2024-03-22T07:16:08.451 INFO:tasks.workunit.client.1.smithi069.stdout:8/930: dread d1/d1c/d3e/d66/d9e/dd1/fda [0,4194304] 0 2024-03-22T07:16:08.451 INFO:tasks.workunit.client.1.smithi069.stdout:8/931: chown d1/d1c/lbf 11518 1 2024-03-22T07:16:08.451 INFO:tasks.workunit.client.1.smithi069.stdout:8/932: chown d1/d1c/d3e/d66/f78 3215 1 2024-03-22T07:16:08.455 INFO:tasks.workunit.client.1.smithi069.stdout:9/652: rename d2/d21/d77/f87 to d2/dd/d3c/d53/d8b/fce 0 2024-03-22T07:16:08.456 INFO:tasks.workunit.client.0.smithi063.stdout:2/915: unlink d0/de/f4c 0 2024-03-22T07:16:08.457 INFO:tasks.workunit.client.1.smithi069.stdout:3/815: mknod d5/d9/c108 0 2024-03-22T07:16:08.457 INFO:tasks.workunit.client.1.smithi069.stdout:3/816: write d5/d9/d26/d36/d41/fe1 [379086,37530] 0 2024-03-22T07:16:08.457 INFO:tasks.workunit.client.1.smithi069.stdout:3/817: stat d5/d9/db/d3b/fcf 0 2024-03-22T07:16:08.458 INFO:tasks.workunit.client.1.smithi069.stdout:7/714: truncate d9/db/d48/d4b/f4f 2453293 0 2024-03-22T07:16:08.459 INFO:tasks.workunit.client.1.smithi069.stdout:8/933: mknod d1/d1c/d3e/db9/d112/d129/c12f 0 2024-03-22T07:16:08.460 INFO:tasks.workunit.client.1.smithi069.stdout:8/934: creat d1/d1c/d3e/d88/dcd/d6c/d73/f130 x:0 0 0 2024-03-22T07:16:08.462 INFO:tasks.workunit.client.1.smithi069.stdout:9/653: unlink d2/dd/d3c/c6c 0 2024-03-22T07:16:08.462 INFO:tasks.workunit.client.1.smithi069.stdout:9/654: readlink d2/dd/d3c/d48/d7f/d86/la9 0 2024-03-22T07:16:08.463 INFO:tasks.workunit.client.1.smithi069.stdout:3/818: mkdir d5/da2/ddd/d109 0 2024-03-22T07:16:08.466 INFO:tasks.workunit.client.1.smithi069.stdout:8/935: mknod d1/d1c/d3e/de6/c131 0 2024-03-22T07:16:08.466 INFO:tasks.workunit.client.1.smithi069.stdout:8/936: fdatasync d1/d1c/d3e/f61 0 2024-03-22T07:16:08.468 INFO:tasks.workunit.client.1.smithi069.stdout:9/655: mknod d2/d21/d77/ccf 0 2024-03-22T07:16:08.468 INFO:tasks.workunit.client.1.smithi069.stdout:7/715: write d9/fd7 [385088,106686] 0 2024-03-22T07:16:08.469 INFO:tasks.workunit.client.0.smithi063.stdout:2/916: write d0/de/d12/f1f [273193,95281] 0 2024-03-22T07:16:08.469 INFO:tasks.workunit.client.0.smithi063.stdout:2/917: creat d0/de/d74/f12d x:0 0 0 2024-03-22T07:16:08.469 INFO:tasks.workunit.client.1.smithi069.stdout:3/819: creat d5/f10a x:0 0 0 2024-03-22T07:16:08.470 INFO:tasks.workunit.client.1.smithi069.stdout:3/820: creat d5/da2/f10b x:0 0 0 2024-03-22T07:16:08.470 INFO:tasks.workunit.client.1.smithi069.stdout:9/656: write d2/dd/d3c/f35 [13869,54997] 0 2024-03-22T07:16:08.470 INFO:tasks.workunit.client.1.smithi069.stdout:9/657: write d2/dd/d3c/f3e [2099890,113557] 0 2024-03-22T07:16:08.470 INFO:tasks.workunit.client.1.smithi069.stdout:9/658: readlink d2/d21/l6b 0 2024-03-22T07:16:08.470 INFO:tasks.workunit.client.1.smithi069.stdout:8/937: symlink d1/l132 0 2024-03-22T07:16:08.471 INFO:tasks.workunit.client.1.smithi069.stdout:8/938: chown d1/d1c/d3e/d88/dcd/d6c/d4e/d98/f12e 180 1 2024-03-22T07:16:08.474 INFO:tasks.workunit.client.1.smithi069.stdout:8/939: getdents d1/d1c/d3e/d88/dd3 0 2024-03-22T07:16:08.479 INFO:tasks.workunit.client.1.smithi069.stdout:9/659: rmdir d2/dd/d3c/d48 39 2024-03-22T07:16:08.482 INFO:tasks.workunit.client.1.smithi069.stdout:9/660: symlink d2/dd/d3c/d53/ld0 0 2024-03-22T07:16:08.484 INFO:tasks.workunit.client.1.smithi069.stdout:8/940: dread d1/d1c/f38 [0,4194304] 0 2024-03-22T07:16:08.484 INFO:tasks.workunit.client.1.smithi069.stdout:8/941: creat d1/d1c/d3e/d88/dcd/d6c/d73/dfa/f133 x:0 0 0 2024-03-22T07:16:08.484 INFO:tasks.workunit.client.1.smithi069.stdout:8/942: write d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/fae [1273729,18482] 0 2024-03-22T07:16:08.484 INFO:tasks.workunit.client.1.smithi069.stdout:9/661: rename d2/dd/d3c/d48/d7c/cb7 to d2/dd/d6e/cd1 0 2024-03-22T07:16:08.489 INFO:tasks.workunit.client.1.smithi069.stdout:9/662: mknod d2/d21/cd2 0 2024-03-22T07:16:08.490 INFO:tasks.workunit.client.1.smithi069.stdout:9/663: symlink d2/d31/ld3 0 2024-03-22T07:16:08.490 INFO:tasks.workunit.client.0.smithi063.stdout:2/918: dwrite d0/de/d35/ffe [0,4194304] 0 2024-03-22T07:16:08.494 INFO:tasks.workunit.client.0.smithi063.stdout:2/919: mkdir d0/de/d74/dca/d11e/d7f/d97/da2/d12e 0 2024-03-22T07:16:08.494 INFO:tasks.workunit.client.0.smithi063.stdout:2/920: write d0/de/d12/fa9 [47010,44961] 0 2024-03-22T07:16:08.494 INFO:tasks.workunit.client.0.smithi063.stdout:2/921: chown d0/de/d12/d20 2 1 2024-03-22T07:16:08.495 INFO:tasks.workunit.client.1.smithi069.stdout:3/821: dwrite d5/d9/db/f30 [0,4194304] 0 2024-03-22T07:16:08.498 INFO:tasks.workunit.client.1.smithi069.stdout:3/822: mknod d5/d9/d26/c10c 0 2024-03-22T07:16:08.499 INFO:tasks.workunit.client.1.smithi069.stdout:3/823: creat d5/d9/d26/d3f/dc1/f10d x:0 0 0 2024-03-22T07:16:08.499 INFO:tasks.workunit.client.1.smithi069.stdout:3/824: creat d5/d9/d26/d3f/dbe/f10e x:0 0 0 2024-03-22T07:16:08.499 INFO:tasks.workunit.client.1.smithi069.stdout:3/825: read - d5/d9/d26/d3f/fdb zero size 2024-03-22T07:16:08.499 INFO:tasks.workunit.client.1.smithi069.stdout:3/826: write d5/d9/db/f63 [1107702,41532] 0 2024-03-22T07:16:08.504 INFO:tasks.workunit.client.1.smithi069.stdout:3/827: truncate d5/d9/d26/d3f/d56/d8a/da5/f47 30308 0 2024-03-22T07:16:08.504 INFO:tasks.workunit.client.1.smithi069.stdout:3/828: chown d5/d9/d26/d3f/d54 11179 1 2024-03-22T07:16:08.506 INFO:tasks.workunit.client.1.smithi069.stdout:8/943: dwrite d1/d1c/d3e/d88/dcd/d6c/d4e/d98/fc5 [0,4194304] 0 2024-03-22T07:16:08.506 INFO:tasks.workunit.client.1.smithi069.stdout:8/944: fdatasync d1/d1c/d3e/db9/ff8 0 2024-03-22T07:16:08.506 INFO:tasks.workunit.client.1.smithi069.stdout:8/945: fsync d1/d1c/f38 0 2024-03-22T07:16:08.506 INFO:tasks.workunit.client.1.smithi069.stdout:8/946: dread - d1/d1c/d3e/d66/f108 zero size 2024-03-22T07:16:08.507 INFO:tasks.workunit.client.1.smithi069.stdout:8/947: chown d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/lb6 253 1 2024-03-22T07:16:08.508 INFO:tasks.workunit.client.1.smithi069.stdout:3/829: symlink d5/d9/d26/d3f/d54/d9d/l10f 0 2024-03-22T07:16:08.512 INFO:tasks.workunit.client.0.smithi063.stdout:7/870: sync 2024-03-22T07:16:08.512 INFO:tasks.workunit.client.0.smithi063.stdout:7/871: stat d5/da/d28/d11e/da8/l118 0 2024-03-22T07:16:08.523 INFO:tasks.workunit.client.0.smithi063.stdout:7/872: dread d5/da/d5f/fb8 [4194304,4194304] 0 2024-03-22T07:16:08.524 INFO:tasks.workunit.client.0.smithi063.stdout:7/873: dread - d5/da/d28/d11e/da8/dbc/d62/d73/fe4 zero size 2024-03-22T07:16:08.524 INFO:tasks.workunit.client.0.smithi063.stdout:7/874: symlink d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d29/d4a/l121 0 2024-03-22T07:16:08.525 INFO:tasks.workunit.client.0.smithi063.stdout:7/875: write d5/da/fe2 [4202701,10020] 0 2024-03-22T07:16:08.541 INFO:tasks.workunit.client.1.smithi069.stdout:9/664: dwrite d2/dd/f10 [0,4194304] 0 2024-03-22T07:16:08.541 INFO:tasks.workunit.client.1.smithi069.stdout:9/665: fsync d2/d51/f56 0 2024-03-22T07:16:08.542 INFO:tasks.workunit.client.1.smithi069.stdout:9/666: creat d2/d71/fd4 x:0 0 0 2024-03-22T07:16:08.547 INFO:tasks.workunit.client.0.smithi063.stdout:7/876: dwrite d5/da/d28/d11e/da8/dbc/f59 [0,4194304] 0 2024-03-22T07:16:08.547 INFO:tasks.workunit.client.0.smithi063.stdout:2/922: dwrite d0/de/d1b/d63/f6c [4194304,4194304] 0 2024-03-22T07:16:08.547 INFO:tasks.workunit.client.0.smithi063.stdout:2/923: fsync d0/de/f65 0 2024-03-22T07:16:08.548 INFO:tasks.workunit.client.1.smithi069.stdout:8/948: dwrite d1/f14 [4194304,4194304] 0 2024-03-22T07:16:08.550 INFO:tasks.workunit.client.0.smithi063.stdout:7/877: creat d5/da/d28/d11e/da8/dd4/dea/db3/f122 x:0 0 0 2024-03-22T07:16:08.550 INFO:tasks.workunit.client.0.smithi063.stdout:7/878: write d5/d16/f63 [4419759,73935] 0 2024-03-22T07:16:08.553 INFO:tasks.workunit.client.1.smithi069.stdout:3/830: rename d5/d9/d26/d3f/d56/d8a/da5/f47 to d5/d9/d26/d3f/dc1/f110 0 2024-03-22T07:16:08.553 INFO:tasks.workunit.client.0.smithi063.stdout:7/879: rename d5/d16/fe3 to d5/da/d28/d11e/da8/f123 0 2024-03-22T07:16:08.554 INFO:tasks.workunit.client.1.smithi069.stdout:8/949: unlink d1/d1c/d48/d5e/fad 0 2024-03-22T07:16:08.556 INFO:tasks.workunit.client.1.smithi069.stdout:3/831: creat d5/d9/db/f111 x:0 0 0 2024-03-22T07:16:08.556 INFO:tasks.workunit.client.1.smithi069.stdout:3/832: read d5/d9/d26/d3f/d54/d9d/dde/df4/fd4 [387992,90749] 0 2024-03-22T07:16:08.557 INFO:tasks.workunit.client.1.smithi069.stdout:8/950: creat d1/d1c/d3e/dde/f134 x:0 0 0 2024-03-22T07:16:08.558 INFO:tasks.workunit.client.1.smithi069.stdout:3/833: creat d5/d9/f112 x:0 0 0 2024-03-22T07:16:08.559 INFO:tasks.workunit.client.1.smithi069.stdout:8/951: mkdir d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/d82/d135 0 2024-03-22T07:16:08.561 INFO:tasks.workunit.client.1.smithi069.stdout:3/834: truncate d5/d9/d26/d36/fc8 2267835 0 2024-03-22T07:16:08.561 INFO:tasks.workunit.client.1.smithi069.stdout:3/835: stat d5/d9/d26/d36/d41/cb8 0 2024-03-22T07:16:08.561 INFO:tasks.workunit.client.1.smithi069.stdout:9/667: dwrite d2/d51/f64 [0,4194304] 0 2024-03-22T07:16:08.562 INFO:tasks.workunit.client.1.smithi069.stdout:8/952: unlink d1/d1c/d3e/d66/d9e/dd1/fda 0 2024-03-22T07:16:08.562 INFO:tasks.workunit.client.1.smithi069.stdout:8/953: truncate d1/d1c/d3e/d88/dcd/d6c/d73/dfa/f133 859627 0 2024-03-22T07:16:08.563 INFO:tasks.workunit.client.1.smithi069.stdout:3/836: symlink d5/d9/d26/d3f/dec/dfa/l113 0 2024-03-22T07:16:08.566 INFO:tasks.workunit.client.0.smithi063.stdout:7/880: dread d5/d38/f81 [0,4194304] 0 2024-03-22T07:16:08.566 INFO:tasks.workunit.client.0.smithi063.stdout:7/881: truncate d5/d16/f1e 4569072 0 2024-03-22T07:16:08.566 INFO:tasks.workunit.client.0.smithi063.stdout:7/882: stat d5/da/d5f/d101/d117 0 2024-03-22T07:16:08.567 INFO:tasks.workunit.client.0.smithi063.stdout:7/883: mknod d5/da/d28/d11e/da8/dd4/dea/c124 0 2024-03-22T07:16:08.567 INFO:tasks.workunit.client.0.smithi063.stdout:7/884: chown d5/f13 1 1 2024-03-22T07:16:08.567 INFO:tasks.workunit.client.1.smithi069.stdout:9/668: rmdir d2/dd/d3c/d53/d8b 39 2024-03-22T07:16:08.568 INFO:tasks.workunit.client.0.smithi063.stdout:7/885: rename d5/d16/f63 to d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d4f/df8/f125 0 2024-03-22T07:16:08.569 INFO:tasks.workunit.client.0.smithi063.stdout:7/886: rmdir d5/da/d28/d11e 39 2024-03-22T07:16:08.569 INFO:tasks.workunit.client.1.smithi069.stdout:8/954: rename d1/d1c/c34 to d1/d1c/d3e/de6/c136 0 2024-03-22T07:16:08.569 INFO:tasks.workunit.client.1.smithi069.stdout:8/955: read d1/f1d [2017383,73006] 0 2024-03-22T07:16:08.570 INFO:tasks.workunit.client.1.smithi069.stdout:3/837: link d5/d9/d26/d36/d41/fe8 d5/da2/f114 0 2024-03-22T07:16:08.570 INFO:tasks.workunit.client.0.smithi063.stdout:7/887: rename d5/da/c5b to d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d29/df0/c126 0 2024-03-22T07:16:08.571 INFO:tasks.workunit.client.0.smithi063.stdout:7/888: truncate d5/d16/d43/f10f 418200 0 2024-03-22T07:16:08.571 INFO:tasks.workunit.client.1.smithi069.stdout:9/669: rename d2/d31/f45 to d2/d21/d89/fd5 0 2024-03-22T07:16:08.572 INFO:tasks.workunit.client.0.smithi063.stdout:7/889: rename d5/da/d5f/fb8 to d5/da/f127 0 2024-03-22T07:16:08.572 INFO:tasks.workunit.client.0.smithi063.stdout:7/890: write d5/d16/f2b [373256,130191] 0 2024-03-22T07:16:08.572 INFO:tasks.workunit.client.0.smithi063.stdout:7/891: stat d5/da/d28/d11e/da8/dbc/d62/d73/c108 0 2024-03-22T07:16:08.573 INFO:tasks.workunit.client.1.smithi069.stdout:8/956: creat d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/d82/f137 x:0 0 0 2024-03-22T07:16:08.574 INFO:tasks.workunit.client.1.smithi069.stdout:3/838: dread d5/d9/d26/d36/d41/f75 [0,4194304] 0 2024-03-22T07:16:08.574 INFO:tasks.workunit.client.1.smithi069.stdout:3/839: stat d5/d9/d26/d3f/fdb 0 2024-03-22T07:16:08.576 INFO:tasks.workunit.client.1.smithi069.stdout:9/670: creat d2/dd/d3c/fd6 x:0 0 0 2024-03-22T07:16:08.579 INFO:tasks.workunit.client.1.smithi069.stdout:5/673: sync 2024-03-22T07:16:08.580 INFO:tasks.workunit.client.1.smithi069.stdout:1/667: sync 2024-03-22T07:16:08.580 INFO:tasks.workunit.client.1.smithi069.stdout:0/860: sync 2024-03-22T07:16:08.580 INFO:tasks.workunit.client.1.smithi069.stdout:8/957: truncate d1/d1c/d3e/d88/dcd/d6c/f72 3938650 0 2024-03-22T07:16:08.580 INFO:tasks.workunit.client.1.smithi069.stdout:8/958: chown d1/d1c/d48/d5e/fc4 319 1 2024-03-22T07:16:08.580 INFO:tasks.workunit.client.1.smithi069.stdout:8/959: read - d1/d1c/d48/d5e/fc4 zero size 2024-03-22T07:16:08.584 INFO:tasks.workunit.client.1.smithi069.stdout:3/840: creat d5/d9/d26/d3f/d56/d8a/da5/f115 x:0 0 0 2024-03-22T07:16:08.586 INFO:tasks.workunit.client.1.smithi069.stdout:5/674: rename d1/d6/d26/d3a/le5 to d1/d10/d44/dd8/le9 0 2024-03-22T07:16:08.587 INFO:tasks.workunit.client.1.smithi069.stdout:1/668: truncate d1/d13/f39 1419601 0 2024-03-22T07:16:08.590 INFO:tasks.workunit.client.1.smithi069.stdout:8/960: truncate d1/d1c/d3e/d88/dcd/d6c/d73/dfa/f128 266874 0 2024-03-22T07:16:08.590 INFO:tasks.workunit.client.1.smithi069.stdout:8/961: stat d1/d1c/d3e/f61 0 2024-03-22T07:16:08.590 INFO:tasks.workunit.client.1.smithi069.stdout:8/962: truncate d1/d1c/d3e/d88/dcd/d6c/d4e/f104 583917 0 2024-03-22T07:16:08.591 INFO:tasks.workunit.client.1.smithi069.stdout:1/669: getdents d1/d2a/d5e/d9b 0 2024-03-22T07:16:08.591 INFO:tasks.workunit.client.1.smithi069.stdout:1/670: chown d1/d13/d82/dc2 0 1 2024-03-22T07:16:08.592 INFO:tasks.workunit.client.1.smithi069.stdout:3/841: rename d5/ca4 to d5/d9/d26/db4/c116 0 2024-03-22T07:16:08.595 INFO:tasks.workunit.client.1.smithi069.stdout:8/963: dread d1/d1c/d3e/d88/dcd/d20/f65 [0,4194304] 0 2024-03-22T07:16:08.595 INFO:tasks.workunit.client.1.smithi069.stdout:8/964: creat d1/d1c/d3e/d88/dcd/d6c/d4e/d98/f138 x:0 0 0 2024-03-22T07:16:08.596 INFO:tasks.workunit.client.1.smithi069.stdout:8/965: creat d1/d1c/d3e/d88/dd3/f139 x:0 0 0 2024-03-22T07:16:08.597 INFO:tasks.workunit.client.1.smithi069.stdout:8/966: getdents d1/d93/ddf/de3/d123 0 2024-03-22T07:16:08.599 INFO:tasks.workunit.client.1.smithi069.stdout:8/967: creat d1/d1c/d3e/d88/da8/de7/f13a x:0 0 0 2024-03-22T07:16:08.603 INFO:tasks.workunit.client.0.smithi063.stdout:7/892: dwrite d5/da/d28/d11e/da8/dbc/d62/d73/f100 [0,4194304] 0 2024-03-22T07:16:08.603 INFO:tasks.workunit.client.1.smithi069.stdout:9/671: dwrite d2/d31/f40 [0,4194304] 0 2024-03-22T07:16:08.605 INFO:tasks.workunit.client.0.smithi063.stdout:7/893: symlink d5/l128 0 2024-03-22T07:16:08.605 INFO:tasks.workunit.client.1.smithi069.stdout:3/842: write d5/d9/f76 [330229,46714] 0 2024-03-22T07:16:08.606 INFO:tasks.workunit.client.0.smithi063.stdout:7/894: creat d5/da/d28/d11e/da8/dbc/dfb/f129 x:0 0 0 2024-03-22T07:16:08.614 INFO:tasks.workunit.client.1.smithi069.stdout:8/968: write d1/d1c/d3e/d88/da8/dbb/fd8 [3000636,93876] 0 2024-03-22T07:16:08.615 INFO:tasks.workunit.client.1.smithi069.stdout:8/969: read d1/d1c/d3e/fb0 [109531,76729] 0 2024-03-22T07:16:08.615 INFO:tasks.workunit.client.1.smithi069.stdout:8/970: chown d1/d1c/d3e/d88/dcd/l1b 85763 1 2024-03-22T07:16:08.616 INFO:tasks.workunit.client.1.smithi069.stdout:3/843: dread d5/d9/db/d3b/f95 [0,4194304] 0 2024-03-22T07:16:08.617 INFO:tasks.workunit.client.1.smithi069.stdout:8/971: creat d1/d1c/d3e/d66/d9e/f13b x:0 0 0 2024-03-22T07:16:08.620 INFO:tasks.workunit.client.1.smithi069.stdout:3/844: symlink d5/d9/db/d3b/dc6/l117 0 2024-03-22T07:16:08.622 INFO:tasks.workunit.client.1.smithi069.stdout:9/672: dwrite d2/dd/d3c/d2e/fad [0,4194304] 0 2024-03-22T07:16:08.622 INFO:tasks.workunit.client.1.smithi069.stdout:8/972: mkdir d1/d1c/d3e/d66/d13c 0 2024-03-22T07:16:08.622 INFO:tasks.workunit.client.1.smithi069.stdout:3/845: dread d5/d9/d26/d36/d41/f75 [0,4194304] 0 2024-03-22T07:16:08.624 INFO:tasks.workunit.client.1.smithi069.stdout:1/671: dread d1/d13/f39 [0,4194304] 0 2024-03-22T07:16:08.627 INFO:tasks.workunit.client.1.smithi069.stdout:8/973: dread d1/d1c/d3e/d66/fe2 [0,4194304] 0 2024-03-22T07:16:08.627 INFO:tasks.workunit.client.1.smithi069.stdout:9/673: mkdir d2/d21/dd7 0 2024-03-22T07:16:08.627 INFO:tasks.workunit.client.1.smithi069.stdout:9/674: truncate d2/dd/d3c/d2e/d84/da5/fc8 288789 0 2024-03-22T07:16:08.627 INFO:tasks.workunit.client.1.smithi069.stdout:9/675: chown d2/d76/db8 364710813 1 2024-03-22T07:16:08.628 INFO:tasks.workunit.client.1.smithi069.stdout:9/676: truncate d2/dd/d3c/d2e/fa3 827874 0 2024-03-22T07:16:08.628 INFO:tasks.workunit.client.1.smithi069.stdout:4/887: sync 2024-03-22T07:16:08.628 INFO:tasks.workunit.client.1.smithi069.stdout:3/846: symlink d5/d9/d26/d3f/dec/l118 0 2024-03-22T07:16:08.630 INFO:tasks.workunit.client.1.smithi069.stdout:2/582: sync 2024-03-22T07:16:08.630 INFO:tasks.workunit.client.1.smithi069.stdout:7/716: sync 2024-03-22T07:16:08.630 INFO:tasks.workunit.client.1.smithi069.stdout:6/663: sync 2024-03-22T07:16:08.634 INFO:tasks.workunit.client.1.smithi069.stdout:1/672: rename d1/d2a/d50/d88/c30 to d1/d2a/d84/ccd 0 2024-03-22T07:16:08.634 INFO:tasks.workunit.client.0.smithi063.stdout:4/895: sync 2024-03-22T07:16:08.635 INFO:tasks.workunit.client.1.smithi069.stdout:2/583: fdatasync d18/d4d/d27/d33/d60/f5a 0 2024-03-22T07:16:08.638 INFO:tasks.workunit.client.1.smithi069.stdout:7/717: link f4 d9/db/d1a/db1/fe8 0 2024-03-22T07:16:08.639 INFO:tasks.workunit.client.1.smithi069.stdout:8/974: dread d1/d1c/d3e/d88/dcd/de8/fef [0,4194304] 0 2024-03-22T07:16:08.640 INFO:tasks.workunit.client.1.smithi069.stdout:1/673: creat d1/d2a/d50/d88/dae/db5/dbc/fce x:0 0 0 2024-03-22T07:16:08.642 INFO:tasks.workunit.client.1.smithi069.stdout:2/584: mkdir d18/d4d/d27/d33/d3d/dc6 0 2024-03-22T07:16:08.643 INFO:tasks.workunit.client.1.smithi069.stdout:6/664: write d4/dd/d12/d13/fb0 [903014,67539] 0 2024-03-22T07:16:08.648 INFO:tasks.workunit.client.1.smithi069.stdout:3/847: read d5/d9/d23/f78 [491866,62684] 0 2024-03-22T07:16:08.648 INFO:tasks.workunit.client.1.smithi069.stdout:7/718: creat d9/db/d1a/db1/d57/d44/d53/d74/d30/fe9 x:0 0 0 2024-03-22T07:16:08.651 INFO:tasks.workunit.client.1.smithi069.stdout:8/975: mknod d1/d1c/d3e/d66/d9e/dd1/c13d 0 2024-03-22T07:16:08.651 INFO:tasks.workunit.client.1.smithi069.stdout:8/976: fsync d1/f1d 0 2024-03-22T07:16:08.651 INFO:tasks.workunit.client.1.smithi069.stdout:8/977: write d1/d1c/d3e/d88/dcd/d6c/f92 [1516163,48176] 0 2024-03-22T07:16:08.652 INFO:tasks.workunit.client.1.smithi069.stdout:3/848: mknod d5/d9/d26/d36/d41/d8e/c119 0 2024-03-22T07:16:08.653 INFO:tasks.workunit.client.1.smithi069.stdout:6/665: dread d4/f6b [4194304,4194304] 0 2024-03-22T07:16:08.654 INFO:tasks.workunit.client.1.smithi069.stdout:1/674: write d1/f42 [8293858,67021] 0 2024-03-22T07:16:08.654 INFO:tasks.workunit.client.1.smithi069.stdout:8/978: link d1/d1c/d48/d5e/l6b d1/d1c/l13e 0 2024-03-22T07:16:08.655 INFO:tasks.workunit.client.1.smithi069.stdout:3/849: rmdir d5/d9/db/dca 0 2024-03-22T07:16:08.656 INFO:tasks.workunit.client.1.smithi069.stdout:8/979: symlink d1/d1c/d3e/d88/dcd/d6c/d4e/l13f 0 2024-03-22T07:16:08.657 INFO:tasks.workunit.client.1.smithi069.stdout:8/980: symlink d1/d1c/d3e/d88/dcd/d6c/d4e/d98/l140 0 2024-03-22T07:16:08.658 INFO:tasks.workunit.client.1.smithi069.stdout:2/585: dread d18/d4d/d3e/d40/f51 [0,4194304] 0 2024-03-22T07:16:08.658 INFO:tasks.workunit.client.1.smithi069.stdout:1/675: getdents d1 0 2024-03-22T07:16:08.658 INFO:tasks.workunit.client.1.smithi069.stdout:2/586: write d18/d4d/d3e/d40/f6f [990425,50093] 0 2024-03-22T07:16:08.658 INFO:tasks.workunit.client.1.smithi069.stdout:1/676: write d1/d13/d36/d6e/d73/fa8 [944116,19569] 0 2024-03-22T07:16:08.661 INFO:tasks.workunit.client.1.smithi069.stdout:7/719: write d9/db/d1a/db1/d57/d44/f9b [3129251,110360] 0 2024-03-22T07:16:08.661 INFO:tasks.workunit.client.1.smithi069.stdout:1/677: rename d1/d2a/la9 to d1/d2a/d50/d88/d41/d64/lcf 0 2024-03-22T07:16:08.662 INFO:tasks.workunit.client.1.smithi069.stdout:2/587: creat d18/d4d/d27/d77/fc7 x:0 0 0 2024-03-22T07:16:08.664 INFO:tasks.workunit.client.1.smithi069.stdout:7/720: mknod d9/db/d1a/db1/d57/d75/cea 0 2024-03-22T07:16:08.664 INFO:tasks.workunit.client.1.smithi069.stdout:7/721: write d9/db/d1a/db1/d57/d44/fac [864855,26375] 0 2024-03-22T07:16:08.665 INFO:tasks.workunit.client.1.smithi069.stdout:1/678: creat d1/d13/d36/d6e/fd0 x:0 0 0 2024-03-22T07:16:08.666 INFO:tasks.workunit.client.1.smithi069.stdout:2/588: rmdir d18/d4d/d27/d33/d60/d63/d73 39 2024-03-22T07:16:08.669 INFO:tasks.workunit.client.1.smithi069.stdout:1/679: mkdir d1/d2a/d84/dd1 0 2024-03-22T07:16:08.670 INFO:tasks.workunit.client.1.smithi069.stdout:1/680: mkdir d1/d13/dc7/dd2 0 2024-03-22T07:16:08.673 INFO:tasks.workunit.client.1.smithi069.stdout:1/681: symlink d1/d13/d36/d5b/ld3 0 2024-03-22T07:16:08.676 INFO:tasks.workunit.client.1.smithi069.stdout:4/888: dwrite d6/d16/d3a/db4/fff [0,4194304] 0 2024-03-22T07:16:08.676 INFO:tasks.workunit.client.1.smithi069.stdout:9/677: dwrite d2/dd/d3c/d2e/d84/da5/fc8 [0,4194304] 0 2024-03-22T07:16:08.676 INFO:tasks.workunit.client.1.smithi069.stdout:9/678: chown d2/dd/d3c/d48/d7f/d86/la9 1 1 2024-03-22T07:16:08.676 INFO:tasks.workunit.client.1.smithi069.stdout:9/679: stat d2/f8f 0 2024-03-22T07:16:08.676 INFO:tasks.workunit.client.1.smithi069.stdout:2/589: dread d18/d4d/f22 [8388608,4194304] 0 2024-03-22T07:16:08.677 INFO:tasks.workunit.client.1.smithi069.stdout:6/666: write d4/f6b [4027889,20591] 0 2024-03-22T07:16:08.682 INFO:tasks.workunit.client.1.smithi069.stdout:6/667: unlink d4/dd/d12/d2b/d72/d73/d74/ca8 0 2024-03-22T07:16:08.682 INFO:tasks.workunit.client.1.smithi069.stdout:6/668: readlink d4/d7/da/d3f/d76/dab/laf 0 2024-03-22T07:16:08.682 INFO:tasks.workunit.client.1.smithi069.stdout:6/669: chown d4/d7/da/l4c 93848 1 2024-03-22T07:16:08.684 INFO:tasks.workunit.client.0.smithi063.stdout:4/896: dwrite d1/d5/d43/d27/d2c/d3d/d54/f8d [0,4194304] 0 2024-03-22T07:16:08.689 INFO:tasks.workunit.client.1.smithi069.stdout:1/682: dread d1/d2a/f32 [0,4194304] 0 2024-03-22T07:16:08.690 INFO:tasks.workunit.client.1.smithi069.stdout:1/683: chown d1/d2a/d50/d88/dae 36626169 1 2024-03-22T07:16:08.690 INFO:tasks.workunit.client.1.smithi069.stdout:1/684: fsync d1/d2a/d50/fac 0 2024-03-22T07:16:08.690 INFO:tasks.workunit.client.1.smithi069.stdout:1/685: write d1/d2a/d50/d88/d41/f4b [607353,84985] 0 2024-03-22T07:16:08.691 INFO:tasks.workunit.client.1.smithi069.stdout:9/680: read d2/d31/f40 [2502901,109191] 0 2024-03-22T07:16:08.693 INFO:tasks.workunit.client.1.smithi069.stdout:2/590: dread d18/d4d/d34/f7e [0,4194304] 0 2024-03-22T07:16:08.693 INFO:tasks.workunit.client.1.smithi069.stdout:9/681: mknod d2/d51/cd8 0 2024-03-22T07:16:08.694 INFO:tasks.workunit.client.0.smithi063.stdout:4/897: write d1/d5/dc2/dee/f122 [253109,51417] 0 2024-03-22T07:16:08.695 INFO:tasks.workunit.client.1.smithi069.stdout:2/591: truncate d18/d4d/d27/d77/f92 1571744 0 2024-03-22T07:16:08.696 INFO:tasks.workunit.client.1.smithi069.stdout:9/682: getdents d2/d76 0 2024-03-22T07:16:08.697 INFO:tasks.workunit.client.0.smithi063.stdout:4/898: creat d1/d5/dc/d95/d59/f138 x:0 0 0 2024-03-22T07:16:08.697 INFO:tasks.workunit.client.0.smithi063.stdout:4/899: creat d1/d5/dc2/f139 x:0 0 0 2024-03-22T07:16:08.697 INFO:tasks.workunit.client.1.smithi069.stdout:7/722: dwrite d9/db/f69 [0,4194304] 0 2024-03-22T07:16:08.698 INFO:tasks.workunit.client.1.smithi069.stdout:2/592: creat d18/d4d/d3e/fc8 x:0 0 0 2024-03-22T07:16:08.698 INFO:tasks.workunit.client.1.smithi069.stdout:2/593: chown d18/d4d/d27/d33/d60/d63/d80 8085393 1 2024-03-22T07:16:08.702 INFO:tasks.workunit.client.1.smithi069.stdout:2/594: getdents d18/d4d/d27/d33/d60/d64 0 2024-03-22T07:16:08.702 INFO:tasks.workunit.client.1.smithi069.stdout:2/595: write d18/d4d/d27/d33/d60/d64/fac [913342,3287] 0 2024-03-22T07:16:08.702 INFO:tasks.workunit.client.1.smithi069.stdout:3/850: dwrite d5/d9/d26/d3f/fdb [0,4194304] 0 2024-03-22T07:16:08.703 INFO:tasks.workunit.client.1.smithi069.stdout:3/851: mknod d5/d9/d26/d36/d41/c11a 0 2024-03-22T07:16:08.705 INFO:tasks.workunit.client.1.smithi069.stdout:2/596: mkdir d18/d4d/d27/d33/d60/d63/d73/dc9 0 2024-03-22T07:16:08.706 INFO:tasks.workunit.client.1.smithi069.stdout:2/597: dread d18/d4d/d27/d77/f9f [0,4194304] 0 2024-03-22T07:16:08.707 INFO:tasks.workunit.client.1.smithi069.stdout:2/598: rmdir d18/d4d/d27/d33 39 2024-03-22T07:16:08.710 INFO:tasks.workunit.client.1.smithi069.stdout:9/683: dread d2/d71/fcd [0,4194304] 0 2024-03-22T07:16:08.714 INFO:tasks.workunit.client.1.smithi069.stdout:2/599: rename d18/d4d/d27/c4e to d18/d4d/d3e/d40/d96/cca 0 2024-03-22T07:16:08.714 INFO:tasks.workunit.client.1.smithi069.stdout:2/600: fdatasync d18/d4d/d3e/d40/f44 0 2024-03-22T07:16:08.715 INFO:tasks.workunit.client.1.smithi069.stdout:2/601: write d18/d91/fb8 [428733,107683] 0 2024-03-22T07:16:08.715 INFO:tasks.workunit.client.1.smithi069.stdout:2/602: write d18/d91/f98 [1010460,104268] 0 2024-03-22T07:16:08.717 INFO:tasks.workunit.client.1.smithi069.stdout:4/889: dwrite d6/d16/d3a/db4/dfb/f1a [0,4194304] 0 2024-03-22T07:16:08.718 INFO:tasks.workunit.client.0.smithi063.stdout:5/899: sync 2024-03-22T07:16:08.719 INFO:tasks.workunit.client.0.smithi063.stdout:2/924: sync 2024-03-22T07:16:08.719 INFO:tasks.workunit.client.0.smithi063.stdout:7/895: sync 2024-03-22T07:16:08.721 INFO:tasks.workunit.client.0.smithi063.stdout:4/900: write d1/d5/d43/f35 [1550518,3518] 0 2024-03-22T07:16:08.721 INFO:tasks.workunit.client.0.smithi063.stdout:7/896: truncate d5/da/d28/d11e/da8/dbc/d62/d73/fd7 430997 0 2024-03-22T07:16:08.721 INFO:tasks.workunit.client.0.smithi063.stdout:4/901: readlink d1/d5/dc2/led 0 2024-03-22T07:16:08.721 INFO:tasks.workunit.client.0.smithi063.stdout:7/897: fdatasync d5/da/d28/da7/f64 0 2024-03-22T07:16:08.723 INFO:tasks.workunit.client.0.smithi063.stdout:4/902: creat d1/f13a x:0 0 0 2024-03-22T07:16:08.725 INFO:tasks.workunit.client.0.smithi063.stdout:7/898: creat d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d29/d4a/f12a x:0 0 0 2024-03-22T07:16:08.725 INFO:tasks.workunit.client.0.smithi063.stdout:7/899: fdatasync d5/d16/d43/fa5 0 2024-03-22T07:16:08.725 INFO:tasks.workunit.client.0.smithi063.stdout:7/900: stat d5/da/f25 0 2024-03-22T07:16:08.726 INFO:tasks.workunit.client.0.smithi063.stdout:7/901: symlink d5/da/d28/d11e/da8/dbc/d62/d73/dc5/l12b 0 2024-03-22T07:16:08.729 INFO:tasks.workunit.client.1.smithi069.stdout:6/670: dwrite d4/dd/d41/d78/da3/fc2 [0,4194304] 0 2024-03-22T07:16:08.731 INFO:tasks.workunit.client.1.smithi069.stdout:2/603: dread d18/d4d/d25/f5b [0,4194304] 0 2024-03-22T07:16:08.737 INFO:tasks.workunit.client.0.smithi063.stdout:4/903: write d1/d5/d43/d27/d2c/d4a/f4d [3927843,72999] 0 2024-03-22T07:16:08.737 INFO:tasks.workunit.client.0.smithi063.stdout:4/904: write d1/d5/dc/d95/d59/d9c/dac/fe6 [1285956,42190] 0 2024-03-22T07:16:08.740 INFO:tasks.workunit.client.0.smithi063.stdout:7/902: write d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d29/fca [93406,28651] 0 2024-03-22T07:16:08.740 INFO:tasks.workunit.client.0.smithi063.stdout:4/905: mkdir d1/d5/d43/d27/d2c/d13b 0 2024-03-22T07:16:08.740 INFO:tasks.workunit.client.1.smithi069.stdout:6/671: write d4/dd/d12/d2b/f36 [5375241,15772] 0 2024-03-22T07:16:08.741 INFO:tasks.workunit.client.0.smithi063.stdout:2/925: write d0/de/d12/d5a/f9d [287244,850] 0 2024-03-22T07:16:08.743 INFO:tasks.workunit.client.0.smithi063.stdout:7/903: mknod d5/dcf/c12c 0 2024-03-22T07:16:08.744 INFO:tasks.workunit.client.0.smithi063.stdout:4/906: creat d1/d5/d43/d27/d2c/d3d/d54/ddb/f13c x:0 0 0 2024-03-22T07:16:08.746 INFO:tasks.workunit.client.1.smithi069.stdout:1/686: dwrite d1/d2a/d5e/d93/fa0 [0,4194304] 0 2024-03-22T07:16:08.746 INFO:tasks.workunit.client.1.smithi069.stdout:1/687: fsync d1/d2a/d84/f8b 0 2024-03-22T07:16:08.746 INFO:tasks.workunit.client.1.smithi069.stdout:1/688: fsync d1/d13/d82/dc2/faf 0 2024-03-22T07:16:08.746 INFO:tasks.workunit.client.1.smithi069.stdout:1/689: fdatasync d1/d2a/d50/d88/d41/fc5 0 2024-03-22T07:16:08.746 INFO:tasks.workunit.client.1.smithi069.stdout:7/723: dwrite d9/db/d1a/db1/d57/d44/d53/f43 [4194304,4194304] 0 2024-03-22T07:16:08.746 INFO:tasks.workunit.client.0.smithi063.stdout:2/926: rename d0/de/d12/f27 to d0/de/d110/f12f 0 2024-03-22T07:16:08.747 INFO:tasks.workunit.client.0.smithi063.stdout:2/927: dread - d0/de/d1b/d63/dd6/fe4 zero size 2024-03-22T07:16:08.747 INFO:tasks.workunit.client.0.smithi063.stdout:7/904: mknod d5/d16/d43/d78/c12d 0 2024-03-22T07:16:08.747 INFO:tasks.workunit.client.0.smithi063.stdout:7/905: chown d5/f17 219061933 1 2024-03-22T07:16:08.747 INFO:tasks.workunit.client.0.smithi063.stdout:7/906: write d5/da/d5f/f5c [626545,106428] 0 2024-03-22T07:16:08.748 INFO:tasks.workunit.client.0.smithi063.stdout:4/907: symlink d1/d5/dc/d95/da4/l13d 0 2024-03-22T07:16:08.749 INFO:tasks.workunit.client.0.smithi063.stdout:7/907: mkdir d5/da/d28/d11e/da8/dbc/d62/d73/d12e 0 2024-03-22T07:16:08.749 INFO:tasks.workunit.client.1.smithi069.stdout:1/690: creat d1/d2a/db6/fd4 x:0 0 0 2024-03-22T07:16:08.750 INFO:tasks.workunit.client.0.smithi063.stdout:7/908: rmdir d5/da/d28/d11e/da8/dbc/d62/d8d/dbb 39 2024-03-22T07:16:08.750 INFO:tasks.workunit.client.0.smithi063.stdout:5/900: dwrite d8/d13/d1e/d3c/da0/f109 [0,4194304] 0 2024-03-22T07:16:08.751 INFO:tasks.workunit.client.0.smithi063.stdout:4/908: getdents d1/d5/dc/d95/d59/d9c/dac/dc9 0 2024-03-22T07:16:08.752 INFO:tasks.workunit.client.0.smithi063.stdout:7/909: creat d5/da/d28/d11e/da8/dbc/d62/f12f x:0 0 0 2024-03-22T07:16:08.754 INFO:tasks.workunit.client.0.smithi063.stdout:5/901: rename d8/fc6 to d8/de9/d7e/d9e/f121 0 2024-03-22T07:16:08.754 INFO:tasks.workunit.client.1.smithi069.stdout:1/691: unlink d1/c6f 0 2024-03-22T07:16:08.755 INFO:tasks.workunit.client.0.smithi063.stdout:4/909: mkdir d1/d5/d43/d27/d2c/d13b/d13e 0 2024-03-22T07:16:08.755 INFO:tasks.workunit.client.0.smithi063.stdout:4/910: creat d1/d5/dc/f13f x:0 0 0 2024-03-22T07:16:08.756 INFO:tasks.workunit.client.0.smithi063.stdout:7/910: creat d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d4f/df8/f130 x:0 0 0 2024-03-22T07:16:08.756 INFO:tasks.workunit.client.0.smithi063.stdout:7/911: fsync d5/d16/f32 0 2024-03-22T07:16:08.756 INFO:tasks.workunit.client.0.smithi063.stdout:7/912: fdatasync d5/da/d4e/fd5 0 2024-03-22T07:16:08.756 INFO:tasks.workunit.client.0.smithi063.stdout:7/913: dread - d5/dcf/ff3 zero size 2024-03-22T07:16:08.756 INFO:tasks.workunit.client.0.smithi063.stdout:7/914: fdatasync d5/fe6 0 2024-03-22T07:16:08.756 INFO:tasks.workunit.client.1.smithi069.stdout:6/672: write d4/dd/d41/d78/da3/fc2 [2713529,69567] 0 2024-03-22T07:16:08.756 INFO:tasks.workunit.client.1.smithi069.stdout:6/673: readlink d4/dd/d12/d44/l4b 0 2024-03-22T07:16:08.756 INFO:tasks.workunit.client.1.smithi069.stdout:3/852: dwrite d5/d9/d26/d3f/d56/d7f/d90/ff0 [0,4194304] 0 2024-03-22T07:16:08.757 INFO:tasks.workunit.client.1.smithi069.stdout:1/692: creat d1/d2a/d50/d88/d41/d1e/fd5 x:0 0 0 2024-03-22T07:16:08.758 INFO:tasks.workunit.client.0.smithi063.stdout:4/911: rename d1/d5/dc/f4c to d1/d5/d43/d27/d2c/d3d/d54/ddb/f140 0 2024-03-22T07:16:08.761 INFO:tasks.workunit.client.1.smithi069.stdout:9/684: dwrite d2/dd/d3c/d48/d7f/d86/fc0 [0,4194304] 0 2024-03-22T07:16:08.765 INFO:tasks.workunit.client.1.smithi069.stdout:1/693: mkdir d1/d13/dd6 0 2024-03-22T07:16:08.766 INFO:tasks.workunit.client.1.smithi069.stdout:3/853: dread d5/d9/d26/d3f/d35/f50 [4194304,4194304] 0 2024-03-22T07:16:08.766 INFO:tasks.workunit.client.1.smithi069.stdout:3/854: write d5/d9/d26/d3f/d56/d8a/da5/f39 [1021546,106865] 0 2024-03-22T07:16:08.770 INFO:tasks.workunit.client.1.smithi069.stdout:9/685: creat d2/dd/d3c/d2e/d84/fd9 x:0 0 0 2024-03-22T07:16:08.771 INFO:tasks.workunit.client.1.smithi069.stdout:9/686: readlink d2/l79 0 2024-03-22T07:16:08.772 INFO:tasks.workunit.client.1.smithi069.stdout:1/694: symlink d1/d2a/d5e/d93/ld7 0 2024-03-22T07:16:08.774 INFO:tasks.workunit.client.1.smithi069.stdout:3/855: link d5/d9/db/c53 d5/d9/d26/d3f/dc1/c11b 0 2024-03-22T07:16:08.775 INFO:tasks.workunit.client.1.smithi069.stdout:3/856: mkdir d5/d9/d26/d36/d41/d8e/d11c 0 2024-03-22T07:16:08.776 INFO:tasks.workunit.client.0.smithi063.stdout:5/902: dread d8/d13/d1e/d3c/da0/f109 [0,4194304] 0 2024-03-22T07:16:08.777 INFO:tasks.workunit.client.0.smithi063.stdout:7/915: read d5/d16/f2e [7885571,77299] 0 2024-03-22T07:16:08.777 INFO:tasks.workunit.client.0.smithi063.stdout:7/916: chown d5/d16/f1e 0 1 2024-03-22T07:16:08.777 INFO:tasks.workunit.client.1.smithi069.stdout:2/604: dwrite d18/d4d/d25/f5b [0,4194304] 0 2024-03-22T07:16:08.778 INFO:tasks.workunit.client.1.smithi069.stdout:3/857: symlink d5/l11d 0 2024-03-22T07:16:08.778 INFO:tasks.workunit.client.1.smithi069.stdout:6/674: dwrite d4/d7/f35 [4194304,4194304] 0 2024-03-22T07:16:08.780 INFO:tasks.workunit.client.0.smithi063.stdout:5/903: mknod d8/d13/d24/da4/d8b/db6/dfb/c122 0 2024-03-22T07:16:08.780 INFO:tasks.workunit.client.0.smithi063.stdout:5/904: fsync d8/de9/fad 0 2024-03-22T07:16:08.781 INFO:tasks.workunit.client.1.smithi069.stdout:2/605: mkdir d18/d4d/d27/d33/d60/d64/d89/dcb 0 2024-03-22T07:16:08.781 INFO:tasks.workunit.client.1.smithi069.stdout:2/606: chown d18/d4d/d27/d33/d3d/f46 625440 1 2024-03-22T07:16:08.781 INFO:tasks.workunit.client.1.smithi069.stdout:2/607: readlink d18/d4d/d27/d33/d4f/lc4 0 2024-03-22T07:16:08.782 INFO:tasks.workunit.client.0.smithi063.stdout:7/917: mkdir d5/d16/d131 0 2024-03-22T07:16:08.782 INFO:tasks.workunit.client.0.smithi063.stdout:7/918: readlink d5/da/d5f/la2 0 2024-03-22T07:16:08.783 INFO:tasks.workunit.client.1.smithi069.stdout:3/858: mkdir d5/da2/ddd/d11e 0 2024-03-22T07:16:08.786 INFO:tasks.workunit.client.0.smithi063.stdout:7/919: mkdir d5/da/d28/d11e/da8/dd4/d132 0 2024-03-22T07:16:08.787 INFO:tasks.workunit.client.1.smithi069.stdout:9/687: dread d2/f5 [0,4194304] 0 2024-03-22T07:16:08.787 INFO:tasks.workunit.client.1.smithi069.stdout:6/675: mknod d4/dd/d12/d2b/d72/d73/d9d/dc3/cd1 0 2024-03-22T07:16:08.787 INFO:tasks.workunit.client.1.smithi069.stdout:6/676: chown d4/dd/d41/d78/f9c 3 1 2024-03-22T07:16:08.787 INFO:tasks.workunit.client.0.smithi063.stdout:7/920: mknod d5/c133 0 2024-03-22T07:16:08.790 INFO:tasks.workunit.client.1.smithi069.stdout:2/608: link d18/d4d/d3e/d40/l75 d18/d4d/d27/d33/d4f/lcc 0 2024-03-22T07:16:08.790 INFO:tasks.workunit.client.1.smithi069.stdout:2/609: getdents d18/d4d/d27/d33/d60/d63/d93 0 2024-03-22T07:16:08.793 INFO:tasks.workunit.client.1.smithi069.stdout:3/859: unlink d5/d9/d26/d3f/d56/c105 0 2024-03-22T07:16:08.795 INFO:tasks.workunit.client.0.smithi063.stdout:4/912: rmdir d1/d5/d43/d27/d2c 39 2024-03-22T07:16:08.795 INFO:tasks.workunit.client.0.smithi063.stdout:4/913: write d1/d5/dc/f10f [220328,16929] 0 2024-03-22T07:16:08.795 INFO:tasks.workunit.client.1.smithi069.stdout:9/688: truncate d2/dd/f10 5549463 0 2024-03-22T07:16:08.796 INFO:tasks.workunit.client.1.smithi069.stdout:3/860: symlink d5/d9/d26/d36/dfd/l11f 0 2024-03-22T07:16:08.799 INFO:tasks.workunit.client.1.smithi069.stdout:9/689: creat d2/db6/fda x:0 0 0 2024-03-22T07:16:08.800 INFO:tasks.workunit.client.1.smithi069.stdout:3/861: link d5/d9/d26/d3f/d56/c79 d5/d9/d26/d3f/d54/d9d/dde/c120 0 2024-03-22T07:16:08.802 INFO:tasks.workunit.client.1.smithi069.stdout:9/690: truncate d2/dd/d3c/d2e/f6a 1029204 0 2024-03-22T07:16:08.804 INFO:tasks.workunit.client.1.smithi069.stdout:3/862: mknod d5/d9/c121 0 2024-03-22T07:16:08.804 INFO:tasks.workunit.client.1.smithi069.stdout:3/863: creat d5/dd6/f122 x:0 0 0 2024-03-22T07:16:08.806 INFO:tasks.workunit.client.0.smithi063.stdout:4/914: dread d1/d5/d3c/f47 [0,4194304] 0 2024-03-22T07:16:08.806 INFO:tasks.workunit.client.0.smithi063.stdout:4/915: write d1/d5/dc/d95/d59/f138 [42293,105495] 0 2024-03-22T07:16:08.807 INFO:tasks.workunit.client.0.smithi063.stdout:4/916: mknod d1/d5/dc/d95/d59/c141 0 2024-03-22T07:16:08.809 INFO:tasks.workunit.client.1.smithi069.stdout:9/691: getdents d2/dd/d3c/d53/d9d 0 2024-03-22T07:16:08.810 INFO:tasks.workunit.client.1.smithi069.stdout:9/692: write d2/d21/f3a [44320,57730] 0 2024-03-22T07:16:08.811 INFO:tasks.workunit.client.1.smithi069.stdout:9/693: chown d2/dd/d3c/d53/d8b/fce 97023 1 2024-03-22T07:16:08.811 INFO:tasks.workunit.client.1.smithi069.stdout:9/694: write d2/dd/d3c/d48/d7f/d86/f8e [752780,35116] 0 2024-03-22T07:16:08.811 INFO:tasks.workunit.client.1.smithi069.stdout:9/695: stat d2/dd/d6e/c93 0 2024-03-22T07:16:08.812 INFO:tasks.workunit.client.1.smithi069.stdout:1/695: dwrite d1/d2a/db6/fd4 [0,4194304] 0 2024-03-22T07:16:08.816 INFO:tasks.workunit.client.1.smithi069.stdout:1/696: rename d1/d2a/d50/d88/dae/db5/cc8 to d1/d13/d36/d6e/cd8 0 2024-03-22T07:16:08.818 INFO:tasks.workunit.client.1.smithi069.stdout:1/697: link d1/d2a/d50/d88/l3d d1/d2a/d50/d88/d41/ld9 0 2024-03-22T07:16:08.818 INFO:tasks.workunit.client.1.smithi069.stdout:1/698: creat d1/d2a/d50/d88/d41/d1e/fda x:0 0 0 2024-03-22T07:16:08.818 INFO:tasks.workunit.client.1.smithi069.stdout:1/699: write d1/d2a/f4e [449324,82712] 0 2024-03-22T07:16:08.819 INFO:tasks.workunit.client.0.smithi063.stdout:4/917: dread d1/d5/d3c/f52 [0,4194304] 0 2024-03-22T07:16:08.819 INFO:tasks.workunit.client.0.smithi063.stdout:4/918: read d1/d5/dc/ff [108321,44698] 0 2024-03-22T07:16:08.833 INFO:tasks.workunit.client.1.smithi069.stdout:1/700: dread d1/d2a/d50/d88/d41/d64/f9d [0,4194304] 0 2024-03-22T07:16:08.833 INFO:tasks.workunit.client.1.smithi069.stdout:1/701: fsync d1/d2a/d5e/fba 0 2024-03-22T07:16:08.833 INFO:tasks.workunit.client.1.smithi069.stdout:1/702: mkdir d1/d2a/d84/dd1/ddb 0 2024-03-22T07:16:08.834 INFO:tasks.workunit.client.0.smithi063.stdout:4/919: dread d1/d5/dc/d95/fb5 [0,4194304] 0 2024-03-22T07:16:08.835 INFO:tasks.workunit.client.0.smithi063.stdout:4/920: rename d1/d5/dc/d95/d59/f138 to d1/d5/d43/d27/d2c/df3/df7/f142 0 2024-03-22T07:16:08.836 INFO:tasks.workunit.client.0.smithi063.stdout:4/921: mkdir d1/d5/dc/d95/d143 0 2024-03-22T07:16:08.837 INFO:tasks.workunit.client.0.smithi063.stdout:4/922: dread d1/d5/d3c/d8f/dbb/d114/d79/fb0 [0,4194304] 0 2024-03-22T07:16:08.838 INFO:tasks.workunit.client.0.smithi063.stdout:4/923: chown d1/d5/d3c/d65/l130 1584723532 1 2024-03-22T07:16:08.838 INFO:tasks.workunit.client.0.smithi063.stdout:4/924: fsync d1/d5/f28 0 2024-03-22T07:16:08.839 INFO:tasks.workunit.client.0.smithi063.stdout:4/925: creat d1/d5/dc/d95/f144 x:0 0 0 2024-03-22T07:16:08.841 INFO:tasks.workunit.client.0.smithi063.stdout:4/926: creat d1/d5/d43/d27/d2c/df4/d128/f145 x:0 0 0 2024-03-22T07:16:08.842 INFO:tasks.workunit.client.0.smithi063.stdout:4/927: unlink d1/d5/d43/d27/d2c/d3d/d54/ddb/c137 0 2024-03-22T07:16:08.850 INFO:tasks.workunit.client.0.smithi063.stdout:5/905: dwrite d8/d13/d24/da4/d8b/fb5 [0,4194304] 0 2024-03-22T07:16:08.850 INFO:tasks.workunit.client.0.smithi063.stdout:7/921: dwrite d5/da/d28/d11e/da8/fc8 [0,4194304] 0 2024-03-22T07:16:08.851 INFO:tasks.workunit.client.1.smithi069.stdout:2/610: dwrite d18/f19 [0,4194304] 0 2024-03-22T07:16:08.854 INFO:tasks.workunit.client.1.smithi069.stdout:1/703: dwrite d1/d13/fa3 [0,4194304] 0 2024-03-22T07:16:08.854 INFO:tasks.workunit.client.1.smithi069.stdout:3/864: dwrite d5/d9/d26/db4/fd3 [4194304,4194304] 0 2024-03-22T07:16:08.854 INFO:tasks.workunit.client.1.smithi069.stdout:3/865: chown d5 772765 1 2024-03-22T07:16:08.854 INFO:tasks.workunit.client.1.smithi069.stdout:3/866: chown d5/f10a 84 1 2024-03-22T07:16:08.854 INFO:tasks.workunit.client.1.smithi069.stdout:3/867: fsync d5/d9/d26/d3f/f22 0 2024-03-22T07:16:08.854 INFO:tasks.workunit.client.1.smithi069.stdout:3/868: dread - d5/d9/d26/d3f/dc1/ff3 zero size 2024-03-22T07:16:08.855 INFO:tasks.workunit.client.1.smithi069.stdout:3/869: chown d5/d9/d26/d3f/d35 85073 1 2024-03-22T07:16:08.855 INFO:tasks.workunit.client.1.smithi069.stdout:3/870: dread - d5/d9/d26/d3f/f103 zero size 2024-03-22T07:16:08.855 INFO:tasks.workunit.client.1.smithi069.stdout:3/871: write d5/d9/d26/d3f/d35/f58 [1177256,115175] 0 2024-03-22T07:16:08.855 INFO:tasks.workunit.client.1.smithi069.stdout:3/872: dread - d5/d9/f112 zero size 2024-03-22T07:16:08.855 INFO:tasks.workunit.client.0.smithi063.stdout:4/928: write d1/d5/d43/d27/d2c/d4a/d80/fa8 [1409088,97866] 0 2024-03-22T07:16:08.855 INFO:tasks.workunit.client.0.smithi063.stdout:5/906: write d8/d13/d28/d57/d5f/fb4 [964797,74248] 0 2024-03-22T07:16:08.855 INFO:tasks.workunit.client.0.smithi063.stdout:5/907: truncate d8/d13/d1e/d3c/fb7 16560 0 2024-03-22T07:16:08.855 INFO:tasks.workunit.client.0.smithi063.stdout:5/908: write d8/f10 [3398945,112543] 0 2024-03-22T07:16:08.861 INFO:tasks.workunit.client.0.smithi063.stdout:5/909: mknod d8/d13/d24/da4/d8b/db6/dea/c123 0 2024-03-22T07:16:08.861 INFO:tasks.workunit.client.0.smithi063.stdout:4/929: rmdir d1/d5/dc/d95/da4 39 2024-03-22T07:16:08.861 INFO:tasks.workunit.client.1.smithi069.stdout:3/873: getdents d5/d9/db/d3b 0 2024-03-22T07:16:08.867 INFO:tasks.workunit.client.0.smithi063.stdout:5/910: symlink d8/d13/d28/d57/dfd/d108/l124 0 2024-03-22T07:16:08.868 INFO:tasks.workunit.client.0.smithi063.stdout:5/911: creat d8/d5e/f125 x:0 0 0 2024-03-22T07:16:08.868 INFO:tasks.workunit.client.1.smithi069.stdout:2/611: dread d18/d4d/d27/d77/fa0 [0,4194304] 0 2024-03-22T07:16:08.868 INFO:tasks.workunit.client.1.smithi069.stdout:3/874: rename d5/d9/d23/c2a to d5/d9/d26/d3f/d56/d8a/da5/d7b/dbc/c123 0 2024-03-22T07:16:08.868 INFO:tasks.workunit.client.1.smithi069.stdout:3/875: rename d5/d9/d26/d3f/d56/d8a/da5/d7b/c9b to d5/d9/d26/d36/d41/df9/c124 0 2024-03-22T07:16:08.868 INFO:tasks.workunit.client.1.smithi069.stdout:3/876: symlink d5/d9/d26/d3f/d35/l125 0 2024-03-22T07:16:08.870 INFO:tasks.workunit.client.1.smithi069.stdout:2/612: rmdir d18/d4d/d25/db3 0 2024-03-22T07:16:08.870 INFO:tasks.workunit.client.1.smithi069.stdout:2/613: dread - d18/d4d/d3e/fc8 zero size 2024-03-22T07:16:08.871 INFO:tasks.workunit.client.1.smithi069.stdout:3/877: truncate d5/d9/d26/d3f/d35/fb6 367766 0 2024-03-22T07:16:08.872 INFO:tasks.workunit.client.1.smithi069.stdout:3/878: write d5/d9/d26/f6c [3561011,97777] 0 2024-03-22T07:16:08.872 INFO:tasks.workunit.client.1.smithi069.stdout:3/879: creat d5/d9/d26/d36/d41/d8e/f126 x:0 0 0 2024-03-22T07:16:08.872 INFO:tasks.workunit.client.1.smithi069.stdout:3/880: creat d5/d9/d26/d3f/dbe/f127 x:0 0 0 2024-03-22T07:16:08.872 INFO:tasks.workunit.client.1.smithi069.stdout:3/881: chown d5/d9/d26/d36/d41/c69 3070496 1 2024-03-22T07:16:08.872 INFO:tasks.workunit.client.1.smithi069.stdout:2/614: getdents d18/d4d/d27/d33/d60 0 2024-03-22T07:16:08.872 INFO:tasks.workunit.client.1.smithi069.stdout:2/615: write d18/d4d/d27/d33/d60/f5a [1881917,48391] 0 2024-03-22T07:16:08.872 INFO:tasks.workunit.client.0.smithi063.stdout:4/930: rename d1/d5/d43/d27/d2c/d3d to d1/d5/d43/d27/d2c/d4a/d146 0 2024-03-22T07:16:08.873 INFO:tasks.workunit.client.0.smithi063.stdout:4/931: dread - d1/d5/d43/d27/d2c/d4a/ff5 zero size 2024-03-22T07:16:08.873 INFO:tasks.workunit.client.0.smithi063.stdout:4/932: getdents d1/d5/dc2/d102 0 2024-03-22T07:16:08.874 INFO:tasks.workunit.client.0.smithi063.stdout:4/933: mkdir d1/d5/d43/d107/d147 0 2024-03-22T07:16:08.884 INFO:tasks.workunit.client.0.smithi063.stdout:4/934: read d1/d5/dc/d95/d59/d9c/ffa [448491,55607] 0 2024-03-22T07:16:08.885 INFO:tasks.workunit.client.0.smithi063.stdout:4/935: chown d1/d5/dc/d95/d59/c63 2123356638 1 2024-03-22T07:16:08.885 INFO:tasks.workunit.client.0.smithi063.stdout:4/936: stat d1/d5/dc/d95/d59/la5 0 2024-03-22T07:16:08.885 INFO:tasks.workunit.client.0.smithi063.stdout:2/928: sync 2024-03-22T07:16:08.885 INFO:tasks.workunit.client.0.smithi063.stdout:2/929: creat d0/de/d12/d20/f130 x:0 0 0 2024-03-22T07:16:08.886 INFO:tasks.workunit.client.0.smithi063.stdout:4/937: unlink d1/d5/dc/d95/d59/d9c/lc8 0 2024-03-22T07:16:08.886 INFO:tasks.workunit.client.0.smithi063.stdout:4/938: fsync d1/d5/d3c/f83 0 2024-03-22T07:16:08.887 INFO:tasks.workunit.client.0.smithi063.stdout:2/930: mkdir d0/de/d35/d131 0 2024-03-22T07:16:08.890 INFO:tasks.workunit.client.1.smithi069.stdout:1/704: dwrite d1/fb [0,4194304] 0 2024-03-22T07:16:08.900 INFO:tasks.workunit.client.0.smithi063.stdout:4/939: write d1/d5/f67 [4170229,72026] 0 2024-03-22T07:16:08.903 INFO:tasks.workunit.client.0.smithi063.stdout:4/940: dread d1/d5/d43/d27/f4b [0,4194304] 0 2024-03-22T07:16:08.904 INFO:tasks.workunit.client.0.smithi063.stdout:4/941: mkdir d1/d5/dc/d95/d59/d9c/d119/d148 0 2024-03-22T07:16:08.906 INFO:tasks.workunit.client.1.smithi069.stdout:1/705: write d1/d2a/d50/d88/f38 [458375,17549] 0 2024-03-22T07:16:08.906 INFO:tasks.workunit.client.1.smithi069.stdout:1/706: chown d1/d2a/d50/d88/d41/d1e/fd5 305695132 1 2024-03-22T07:16:08.907 INFO:tasks.workunit.client.1.smithi069.stdout:1/707: creat d1/d2a/d84/fdc x:0 0 0 2024-03-22T07:16:08.915 INFO:tasks.workunit.client.1.smithi069.stdout:1/708: dread d1/d2a/d50/d88/d41/d64/f7e [0,4194304] 0 2024-03-22T07:16:08.915 INFO:tasks.workunit.client.0.smithi063.stdout:7/922: dwrite d5/da/d28/d11e/da8/dbc/d62/d73/dc5/fc9 [0,4194304] 0 2024-03-22T07:16:08.919 INFO:tasks.workunit.client.0.smithi063.stdout:7/923: unlink d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d29/ccd 0 2024-03-22T07:16:08.919 INFO:tasks.workunit.client.0.smithi063.stdout:7/924: read - d5/da/d28/d11e/da8/dbc/dfb/f115 zero size 2024-03-22T07:16:08.919 INFO:tasks.workunit.client.0.smithi063.stdout:7/925: creat d5/da/d4e/f134 x:0 0 0 2024-03-22T07:16:08.919 INFO:tasks.workunit.client.0.smithi063.stdout:7/926: dread - d5/da/d28/d11e/da8/dbc/d62/d73/fe4 zero size 2024-03-22T07:16:08.919 INFO:tasks.workunit.client.0.smithi063.stdout:7/927: symlink d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d40/d119/d11f/l135 0 2024-03-22T07:16:08.919 INFO:tasks.workunit.client.0.smithi063.stdout:7/928: readlink d5/d16/d43/laf 0 2024-03-22T07:16:08.921 INFO:tasks.workunit.client.0.smithi063.stdout:7/929: creat d5/da/d28/d11e/da8/dbc/d62/d73/dc5/f136 x:0 0 0 2024-03-22T07:16:08.921 INFO:tasks.workunit.client.0.smithi063.stdout:7/930: read - d5/da/d28/d11e/da8/dd4/dea/f113 zero size 2024-03-22T07:16:08.921 INFO:tasks.workunit.client.0.smithi063.stdout:7/931: readlink d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d4f/dd1/ldc 0 2024-03-22T07:16:08.921 INFO:tasks.workunit.client.0.smithi063.stdout:7/932: fdatasync d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d29/fb4 0 2024-03-22T07:16:08.922 INFO:tasks.workunit.client.0.smithi063.stdout:7/933: mknod d5/d16/d43/d9d/c137 0 2024-03-22T07:16:08.923 INFO:tasks.workunit.client.0.smithi063.stdout:7/934: unlink d5/da/d28/d11e/da8/dbc/d62/d73/fd7 0 2024-03-22T07:16:08.933 INFO:tasks.workunit.client.1.smithi069.stdout:1/709: write d1/d13/d36/d5b/f8e [2895631,53713] 0 2024-03-22T07:16:08.935 INFO:tasks.workunit.client.1.smithi069.stdout:1/710: rename d1/d13/d36/d6e/caa to d1/d2a/d84/dd1/cdd 0 2024-03-22T07:16:08.939 INFO:tasks.workunit.client.1.smithi069.stdout:3/882: dwrite d5/d9/d26/f8b [0,4194304] 0 2024-03-22T07:16:08.940 INFO:tasks.workunit.client.0.smithi063.stdout:7/935: write d5/da/d28/d11e/f9c [3243439,53817] 0 2024-03-22T07:16:08.940 INFO:tasks.workunit.client.0.smithi063.stdout:4/942: dwrite d1/d5/d43/d27/d2c/d4a/d78/fb9 [0,4194304] 0 2024-03-22T07:16:08.940 INFO:tasks.workunit.client.0.smithi063.stdout:5/912: dwrite d8/d13/d24/da4/d8b/fb5 [0,4194304] 0 2024-03-22T07:16:08.940 INFO:tasks.workunit.client.1.smithi069.stdout:2/616: dwrite d18/d4d/d27/d33/d60/d64/d89/d8b/f9d [0,4194304] 0 2024-03-22T07:16:08.942 INFO:tasks.workunit.client.1.smithi069.stdout:1/711: rmdir d1/d13 39 2024-03-22T07:16:08.943 INFO:tasks.workunit.client.0.smithi063.stdout:5/913: dread d8/d13/d28/d57/d5f/fb4 [0,4194304] 0 2024-03-22T07:16:08.945 INFO:tasks.workunit.client.0.smithi063.stdout:5/914: symlink d8/d13/d28/d57/dfd/l126 0 2024-03-22T07:16:08.945 INFO:tasks.workunit.client.0.smithi063.stdout:5/915: chown d8/d13/d1e/d3c/c70 4684961 1 2024-03-22T07:16:08.946 INFO:tasks.workunit.client.1.smithi069.stdout:3/883: mkdir d5/d9/d23/d128 0 2024-03-22T07:16:08.946 INFO:tasks.workunit.client.1.smithi069.stdout:3/884: chown d5/d9/d26/d3f/d56/d7f 1 1 2024-03-22T07:16:08.947 INFO:tasks.workunit.client.0.smithi063.stdout:4/943: write d1/d5/dc/d95/fb5 [2393985,98132] 0 2024-03-22T07:16:08.948 INFO:tasks.workunit.client.0.smithi063.stdout:5/916: rename d8/d13/d1e/d31/d64/d65/ddf/d119 to d8/d13/d28/d7d/d127 0 2024-03-22T07:16:08.948 INFO:tasks.workunit.client.0.smithi063.stdout:5/917: truncate d8/d13/d28/d57/d5f/f94 1727974 0 2024-03-22T07:16:08.948 INFO:tasks.workunit.client.0.smithi063.stdout:5/918: write d8/d13/d24/f3b [4599471,38322] 0 2024-03-22T07:16:08.950 INFO:tasks.workunit.client.1.smithi069.stdout:2/617: dread d18/d4d/d27/d33/d60/fa4 [0,4194304] 0 2024-03-22T07:16:08.951 INFO:tasks.workunit.client.0.smithi063.stdout:4/944: rmdir d1/d5/dc/d95/d59/d9c/d119/d148 0 2024-03-22T07:16:08.954 INFO:tasks.workunit.client.0.smithi063.stdout:5/919: symlink d8/dba/l128 0 2024-03-22T07:16:08.954 INFO:tasks.workunit.client.1.smithi069.stdout:2/618: unlink d18/d4d/d27/d33/d60/dbd/fbf 0 2024-03-22T07:16:08.954 INFO:tasks.workunit.client.1.smithi069.stdout:2/619: getdents d18/d4d/d27/d33/d60/d64/d89/d8b/dab 0 2024-03-22T07:16:08.955 INFO:tasks.workunit.client.0.smithi063.stdout:4/945: creat d1/d5/d43/d27/d2c/df4/d10b/d123/f149 x:0 0 0 2024-03-22T07:16:08.961 INFO:tasks.workunit.client.0.smithi063.stdout:5/920: truncate d8/d13/d24/fd8 2315455 0 2024-03-22T07:16:08.961 INFO:tasks.workunit.client.0.smithi063.stdout:5/921: truncate d8/d13/d24/d81/ff1 370734 0 2024-03-22T07:16:08.961 INFO:tasks.workunit.client.0.smithi063.stdout:5/922: fsync d8/d13/d24/d62/fd3 0 2024-03-22T07:16:08.961 INFO:tasks.workunit.client.0.smithi063.stdout:5/923: chown d8/d13/d28/d57/l69 1 1 2024-03-22T07:16:08.961 INFO:tasks.workunit.client.0.smithi063.stdout:5/924: truncate d8/d13/d28/f73 3617005 0 2024-03-22T07:16:08.961 INFO:tasks.workunit.client.0.smithi063.stdout:5/925: dread - d8/d13/d28/d57/fed zero size 2024-03-22T07:16:08.961 INFO:tasks.workunit.client.0.smithi063.stdout:7/936: dread d5/da/f127 [0,4194304] 0 2024-03-22T07:16:08.961 INFO:tasks.workunit.client.0.smithi063.stdout:7/937: chown d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d29/f36 135712271 1 2024-03-22T07:16:08.967 INFO:tasks.workunit.client.0.smithi063.stdout:7/938: rmdir d5/da/d28/d11e 39 2024-03-22T07:16:08.967 INFO:tasks.workunit.client.0.smithi063.stdout:4/946: mknod d1/d5/d43/d27/d2c/d4a/d146/c14a 0 2024-03-22T07:16:08.967 INFO:tasks.workunit.client.0.smithi063.stdout:4/947: truncate d1/d5/d3c/d8f/dbb/d114/f11c 372321 0 2024-03-22T07:16:08.969 INFO:tasks.workunit.client.0.smithi063.stdout:4/948: dread d1/d5/d43/d27/d2c/d4a/d80/ff6 [0,4194304] 0 2024-03-22T07:16:08.972 INFO:tasks.workunit.client.0.smithi063.stdout:4/949: unlink d1/d5/dc/d95/da4/d104/f134 0 2024-03-22T07:16:08.977 INFO:tasks.workunit.client.1.smithi069.stdout:2/620: dread d18/d4d/d27/d33/d60/d64/f8f [4194304,4194304] 0 2024-03-22T07:16:08.978 INFO:tasks.workunit.client.1.smithi069.stdout:2/621: readlink d18/d4d/l45 0 2024-03-22T07:16:08.978 INFO:tasks.workunit.client.1.smithi069.stdout:2/622: chown d18/d4d/d27/d33/d60/d64/d89/d8b 155434403 1 2024-03-22T07:16:08.980 INFO:tasks.workunit.client.1.smithi069.stdout:2/623: mkdir d18/d4d/d27/d33/d60/d64/dcd 0 2024-03-22T07:16:08.980 INFO:tasks.workunit.client.1.smithi069.stdout:5/675: sync 2024-03-22T07:16:08.980 INFO:tasks.workunit.client.1.smithi069.stdout:0/861: sync 2024-03-22T07:16:08.980 INFO:tasks.workunit.client.1.smithi069.stdout:0/862: write d3/d7/d15/d1f/fe8 [812489,65355] 0 2024-03-22T07:16:08.980 INFO:tasks.workunit.client.1.smithi069.stdout:2/624: mknod d18/d4d/d27/d33/d3d/dc6/cce 0 2024-03-22T07:16:08.980 INFO:tasks.workunit.client.1.smithi069.stdout:2/625: readlink l10 0 2024-03-22T07:16:08.981 INFO:tasks.workunit.client.1.smithi069.stdout:0/863: mknod d3/d7/d15/d29/d6e/c123 0 2024-03-22T07:16:08.981 INFO:tasks.workunit.client.1.smithi069.stdout:0/864: chown d3/d7/d15/d29/d5a/d5d/d9d/fbd 16285687 1 2024-03-22T07:16:08.981 INFO:tasks.workunit.client.1.smithi069.stdout:0/865: stat d3/d7/l27 0 2024-03-22T07:16:08.981 INFO:tasks.workunit.client.1.smithi069.stdout:0/866: truncate d3/d7/d15/d29/d5a/d5d/ffd 522514 0 2024-03-22T07:16:08.982 INFO:tasks.workunit.client.1.smithi069.stdout:2/626: write d18/d4d/d27/d56/f62 [5035133,29775] 0 2024-03-22T07:16:08.984 INFO:tasks.workunit.client.1.smithi069.stdout:0/867: mknod d3/d7/d5e/c124 0 2024-03-22T07:16:08.986 INFO:tasks.workunit.client.1.smithi069.stdout:2/627: creat d18/d4d/d27/d33/d60/d63/d93/fcf x:0 0 0 2024-03-22T07:16:08.987 INFO:tasks.workunit.client.0.smithi063.stdout:4/950: dread d1/d5/dc/f1d [0,4194304] 0 2024-03-22T07:16:08.988 INFO:tasks.workunit.client.0.smithi063.stdout:4/951: getdents d1/d5/dc/d95/d59/d9c/dac/dcd 0 2024-03-22T07:16:08.988 INFO:tasks.workunit.client.1.smithi069.stdout:0/868: creat d3/d7/d15/d29/d5a/d5d/d9d/df9/f125 x:0 0 0 2024-03-22T07:16:08.989 INFO:tasks.workunit.client.0.smithi063.stdout:4/952: symlink d1/l14b 0 2024-03-22T07:16:08.990 INFO:tasks.workunit.client.0.smithi063.stdout:4/953: creat d1/d5/d43/f14c x:0 0 0 2024-03-22T07:16:08.990 INFO:tasks.workunit.client.0.smithi063.stdout:4/954: creat d1/d5/dc2/f14d x:0 0 0 2024-03-22T07:16:08.990 INFO:tasks.workunit.client.1.smithi069.stdout:0/869: truncate d3/d7/f49 3164710 0 2024-03-22T07:16:08.991 INFO:tasks.workunit.client.0.smithi063.stdout:4/955: symlink d1/d5/d43/d27/d2c/df4/d10b/l14e 0 2024-03-22T07:16:08.991 INFO:tasks.workunit.client.0.smithi063.stdout:4/956: write d1/d5/f97 [5020632,121164] 0 2024-03-22T07:16:08.997 INFO:tasks.workunit.client.0.smithi063.stdout:4/957: dread d1/d5/dc/f41 [0,4194304] 0 2024-03-22T07:16:08.997 INFO:tasks.workunit.client.0.smithi063.stdout:4/958: readlink d1/d5/dc/d95/d59/d9c/dac/dcd/lc4 0 2024-03-22T07:16:08.998 INFO:tasks.workunit.client.1.smithi069.stdout:2/628: write d18/d4d/d27/d33/d3d/f54 [1272266,43262] 0 2024-03-22T07:16:08.998 INFO:tasks.workunit.client.0.smithi063.stdout:4/959: link d1/d5/dc/d95/d59/d9c/dac/dcd/lc4 d1/d5/d43/d27/d2c/df4/d10b/l14f 0 2024-03-22T07:16:09.007 INFO:tasks.workunit.client.1.smithi069.stdout:0/870: write d3/d9a/fc3 [3362404,117561] 0 2024-03-22T07:16:09.007 INFO:tasks.workunit.client.1.smithi069.stdout:0/871: readlink d3/d7/d24/l4d 0 2024-03-22T07:16:09.009 INFO:tasks.workunit.client.1.smithi069.stdout:0/872: getdents d3/d9a 0 2024-03-22T07:16:09.009 INFO:tasks.workunit.client.1.smithi069.stdout:0/873: stat d3/d7/d15/d29/d5a/d5d/d9d/fbd 0 2024-03-22T07:16:09.010 INFO:tasks.workunit.client.0.smithi063.stdout:4/960: write d1/d5/d43/f29 [2870786,89975] 0 2024-03-22T07:16:09.010 INFO:tasks.workunit.client.0.smithi063.stdout:4/961: chown d1/d5 472 1 2024-03-22T07:16:09.010 INFO:tasks.workunit.client.0.smithi063.stdout:4/962: write d1/d5/d43/d27/d2c/d4a/ff5 [571429,121973] 0 2024-03-22T07:16:09.011 INFO:tasks.workunit.client.1.smithi069.stdout:0/874: truncate d3/d7/d15/d29/d5a/f6c 619966 0 2024-03-22T07:16:09.011 INFO:tasks.workunit.client.0.smithi063.stdout:4/963: rename d1/d5/d3c/d8f/dc0/fef to d1/d5/d3c/d8f/dbb/d114/d109/f150 0 2024-03-22T07:16:09.011 INFO:tasks.workunit.client.0.smithi063.stdout:4/964: chown d1/d5/dc/d95/d59/d9c/dac 123499808 1 2024-03-22T07:16:09.012 INFO:tasks.workunit.client.0.smithi063.stdout:4/965: dread - d1/d5/d43/d27/d2c/d4a/d78/f12f zero size 2024-03-22T07:16:09.012 INFO:tasks.workunit.client.0.smithi063.stdout:4/966: symlink d1/d5/d43/d27/d2c/d4a/d78/dfd/l151 0 2024-03-22T07:16:09.012 INFO:tasks.workunit.client.0.smithi063.stdout:4/967: fdatasync d1/d5/f20 0 2024-03-22T07:16:09.019 INFO:tasks.workunit.client.1.smithi069.stdout:1/712: dwrite d1/d13/d36/d5b/f8e [0,4194304] 0 2024-03-22T07:16:09.020 INFO:tasks.workunit.client.1.smithi069.stdout:1/713: read - d1/d2a/d84/f8b zero size 2024-03-22T07:16:09.021 INFO:tasks.workunit.client.0.smithi063.stdout:5/926: dwrite d8/d13/d1e/d31/f46 [0,4194304] 0 2024-03-22T07:16:09.021 INFO:tasks.workunit.client.1.smithi069.stdout:3/885: dwrite d5/d9/d26/d3f/dc1/f110 [0,4194304] 0 2024-03-22T07:16:09.021 INFO:tasks.workunit.client.1.smithi069.stdout:3/886: chown d5/d9/d26/d3f/dc1/cee 2 1 2024-03-22T07:16:09.022 INFO:tasks.workunit.client.1.smithi069.stdout:3/887: fsync d5/f10a 0 2024-03-22T07:16:09.022 INFO:tasks.workunit.client.1.smithi069.stdout:1/714: truncate d1/d13/f2d 4051579 0 2024-03-22T07:16:09.022 INFO:tasks.workunit.client.1.smithi069.stdout:1/715: creat d1/d2a/d84/fde x:0 0 0 2024-03-22T07:16:09.022 INFO:tasks.workunit.client.1.smithi069.stdout:1/716: write d1/d2a/d5e/fba [663630,4058] 0 2024-03-22T07:16:09.022 INFO:tasks.workunit.client.0.smithi063.stdout:5/927: mkdir d8/d13/d28/d57/d5f/d129 0 2024-03-22T07:16:09.022 INFO:tasks.workunit.client.0.smithi063.stdout:5/928: mknod d8/d13/d1e/c12a 0 2024-03-22T07:16:09.024 INFO:tasks.workunit.client.1.smithi069.stdout:3/888: rename d5/d9/d26/d3f/d56/d8a/da5/lcd to d5/d9/d26/d3f/dbe/l129 0 2024-03-22T07:16:09.024 INFO:tasks.workunit.client.1.smithi069.stdout:3/889: truncate d5/d9/d26/d3f/d56/d7f/d90/fdf 653848 0 2024-03-22T07:16:09.025 INFO:tasks.workunit.client.1.smithi069.stdout:5/676: dwrite d1/d6/d88/fc9 [0,4194304] 0 2024-03-22T07:16:09.025 INFO:tasks.workunit.client.1.smithi069.stdout:5/677: chown d1/d6/d26/d3e/cad 7 1 2024-03-22T07:16:09.026 INFO:tasks.workunit.client.0.smithi063.stdout:7/939: dwrite d5/da/d28/d11e/da8/fc8 [0,4194304] 0 2024-03-22T07:16:09.026 INFO:tasks.workunit.client.1.smithi069.stdout:2/629: dwrite d18/d4d/d3e/d40/f61 [0,4194304] 0 2024-03-22T07:16:09.027 INFO:tasks.workunit.client.1.smithi069.stdout:1/717: unlink d1/d2a/d84/fa7 0 2024-03-22T07:16:09.027 INFO:tasks.workunit.client.0.smithi063.stdout:7/940: truncate d5/da/d28/d11e/da8/dd4/fff 1261065 0 2024-03-22T07:16:09.027 INFO:tasks.workunit.client.0.smithi063.stdout:7/941: read - d5/da/d28/da7/fdd zero size 2024-03-22T07:16:09.028 INFO:tasks.workunit.client.1.smithi069.stdout:3/890: creat d5/f12a x:0 0 0 2024-03-22T07:16:09.029 INFO:tasks.workunit.client.0.smithi063.stdout:7/942: rename d5/da/d28/d11e/da8/dbc/d85/c8f to d5/da/d28/d11e/da8/dbc/d62/d8d/d98/c138 0 2024-03-22T07:16:09.029 INFO:tasks.workunit.client.0.smithi063.stdout:7/943: write d5/da/d28/d11e/da8/dbc/f2a [3597773,42276] 0 2024-03-22T07:16:09.030 INFO:tasks.workunit.client.0.smithi063.stdout:7/944: truncate d5/d38/f7b 738285 0 2024-03-22T07:16:09.032 INFO:tasks.workunit.client.1.smithi069.stdout:5/678: rename d1/d6/d26/d3e/da0 to d1/d6/d21/dae/dea 0 2024-03-22T07:16:09.032 INFO:tasks.workunit.client.1.smithi069.stdout:0/875: dwrite d3/d7/d15/d29/d5a/d5d/fb2 [0,4194304] 0 2024-03-22T07:16:09.032 INFO:tasks.workunit.client.1.smithi069.stdout:5/679: write d1/d6/d26/d33/f4e [1294195,55290] 0 2024-03-22T07:16:09.033 INFO:tasks.workunit.client.0.smithi063.stdout:4/968: rmdir d1/d5/dc2 39 2024-03-22T07:16:09.036 INFO:tasks.workunit.client.0.smithi063.stdout:5/929: dread d8/d13/d1e/d31/daf/fb0 [0,4194304] 0 2024-03-22T07:16:09.040 INFO:tasks.workunit.client.1.smithi069.stdout:2/630: getdents d18/d4d/d27/d33/d60/d63 0 2024-03-22T07:16:09.045 INFO:tasks.workunit.client.0.smithi063.stdout:5/930: rename d8/lc to d8/d13/d1e/d31/daf/l12b 0 2024-03-22T07:16:09.045 INFO:tasks.workunit.client.0.smithi063.stdout:5/931: chown d8/d13/d28/f73 373247329 1 2024-03-22T07:16:09.046 INFO:tasks.workunit.client.0.smithi063.stdout:5/932: creat d8/d13/d24/d81/f12c x:0 0 0 2024-03-22T07:16:09.046 INFO:tasks.workunit.client.1.smithi069.stdout:1/718: symlink d1/d2a/d50/d88/dae/db5/dbc/ldf 0 2024-03-22T07:16:09.047 INFO:tasks.workunit.client.0.smithi063.stdout:5/933: rename d8/d13/d24/d81/l83 to d8/d13/d24/da4/l12d 0 2024-03-22T07:16:09.048 INFO:tasks.workunit.client.1.smithi069.stdout:5/680: link d1/d6/d21/dae/cd9 d1/d6/d26/d3a/ceb 0 2024-03-22T07:16:09.049 INFO:tasks.workunit.client.0.smithi063.stdout:5/934: unlink d8/d13/d1e/d31/d64/d65/d86/cd2 0 2024-03-22T07:16:09.049 INFO:tasks.workunit.client.1.smithi069.stdout:0/876: truncate d3/d7/d15/d29/dcf/d114/fc9 5289654 0 2024-03-22T07:16:09.049 INFO:tasks.workunit.client.1.smithi069.stdout:0/877: readlink d3/d7/d17/d1c/l1d 0 2024-03-22T07:16:09.050 INFO:tasks.workunit.client.1.smithi069.stdout:8/981: sync 2024-03-22T07:16:09.050 INFO:tasks.workunit.client.1.smithi069.stdout:4/890: sync 2024-03-22T07:16:09.050 INFO:tasks.workunit.client.1.smithi069.stdout:4/891: chown d6/daa/f11e 15 1 2024-03-22T07:16:09.050 INFO:tasks.workunit.client.1.smithi069.stdout:4/892: readlink d6/d9b/l75 0 2024-03-22T07:16:09.050 INFO:tasks.workunit.client.1.smithi069.stdout:4/893: truncate d6/d12/db2/fbd 481973 0 2024-03-22T07:16:09.051 INFO:tasks.workunit.client.1.smithi069.stdout:1/719: creat d1/fe0 x:0 0 0 2024-03-22T07:16:09.051 INFO:tasks.workunit.client.1.smithi069.stdout:3/891: getdents d5/d9/d26/d3f/d56/d8a 0 2024-03-22T07:16:09.051 INFO:tasks.workunit.client.1.smithi069.stdout:3/892: fdatasync d5/f5d 0 2024-03-22T07:16:09.051 INFO:tasks.workunit.client.1.smithi069.stdout:3/893: chown d5/d9/d26/d3f/d35/f37 361117 1 2024-03-22T07:16:09.052 INFO:tasks.workunit.client.1.smithi069.stdout:0/878: dread d3/d7/d17/f85 [0,4194304] 0 2024-03-22T07:16:09.054 INFO:tasks.workunit.client.1.smithi069.stdout:1/720: dread d1/d2a/d50/d88/d41/d1e/f51 [0,4194304] 0 2024-03-22T07:16:09.054 INFO:tasks.workunit.client.1.smithi069.stdout:1/721: fsync d1/d13/d82/dc2/fcb 0 2024-03-22T07:16:09.055 INFO:tasks.workunit.client.1.smithi069.stdout:1/722: fdatasync d1/fe0 0 2024-03-22T07:16:09.055 INFO:tasks.workunit.client.1.smithi069.stdout:8/982: truncate d1/d1c/fe4 809694 0 2024-03-22T07:16:09.055 INFO:tasks.workunit.client.1.smithi069.stdout:8/983: chown d1/d1c/d3e/d88/dcd/d6c/d4e/l13f 13 1 2024-03-22T07:16:09.055 INFO:tasks.workunit.client.1.smithi069.stdout:8/984: truncate d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/d82/f9d 679654 0 2024-03-22T07:16:09.057 INFO:tasks.workunit.client.1.smithi069.stdout:4/894: symlink d6/d9b/d1f/d58/d7c/d90/l133 0 2024-03-22T07:16:09.059 INFO:tasks.workunit.client.0.smithi063.stdout:5/935: read d8/dba/fcc [1781666,10253] 0 2024-03-22T07:16:09.060 INFO:tasks.workunit.client.1.smithi069.stdout:3/894: link d5/d9/d26/d3f/d56/d8a/da5/d7b/dbc/c123 d5/d9/d26/d3f/d56/d7f/c12b 0 2024-03-22T07:16:09.062 INFO:tasks.workunit.client.0.smithi063.stdout:5/936: creat d8/d13/d1e/d31/d64/d65/f12e x:0 0 0 2024-03-22T07:16:09.063 INFO:tasks.workunit.client.1.smithi069.stdout:0/879: rmdir d3/d7/d15/d29/d5a/d5d/db4/d100/d121 0 2024-03-22T07:16:09.066 INFO:tasks.workunit.client.1.smithi069.stdout:1/723: creat d1/d2a/d84/dd1/fe1 x:0 0 0 2024-03-22T07:16:09.066 INFO:tasks.workunit.client.1.smithi069.stdout:1/724: stat d1/d13/d36/f3e 0 2024-03-22T07:16:09.067 INFO:tasks.workunit.client.1.smithi069.stdout:8/985: mkdir d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/d141 0 2024-03-22T07:16:09.067 INFO:tasks.workunit.client.1.smithi069.stdout:8/986: write d1/d1c/d48/f4b [2306561,72194] 0 2024-03-22T07:16:09.067 INFO:tasks.workunit.client.1.smithi069.stdout:7/724: sync 2024-03-22T07:16:09.068 INFO:tasks.workunit.client.1.smithi069.stdout:9/696: sync 2024-03-22T07:16:09.068 INFO:tasks.workunit.client.1.smithi069.stdout:6/677: sync 2024-03-22T07:16:09.069 INFO:tasks.workunit.client.1.smithi069.stdout:0/880: mknod d3/d7/d15/d29/dcf/c126 0 2024-03-22T07:16:09.069 INFO:tasks.workunit.client.1.smithi069.stdout:0/881: truncate d3/d7/d90/fa5 1662920 0 2024-03-22T07:16:09.069 INFO:tasks.workunit.client.1.smithi069.stdout:0/882: dread - d3/d7/d15/dce/f115 zero size 2024-03-22T07:16:09.070 INFO:tasks.workunit.client.0.smithi063.stdout:4/969: dwrite d1/d5/dc/d95/d59/faf [4194304,4194304] 0 2024-03-22T07:16:09.070 INFO:tasks.workunit.client.0.smithi063.stdout:4/970: chown d1/l14b 0 1 2024-03-22T07:16:09.070 INFO:tasks.workunit.client.0.smithi063.stdout:4/971: chown d1/l36 1730709 1 2024-03-22T07:16:09.071 INFO:tasks.workunit.client.1.smithi069.stdout:1/725: creat d1/d13/d82/dc2/fe2 x:0 0 0 2024-03-22T07:16:09.071 INFO:tasks.workunit.client.1.smithi069.stdout:1/726: read - d1/d2a/d50/d4c/d55/d5f/f9c zero size 2024-03-22T07:16:09.071 INFO:tasks.workunit.client.1.smithi069.stdout:1/727: write d1/d2a/d50/d88/d41/fc5 [2946716,71658] 0 2024-03-22T07:16:09.071 INFO:tasks.workunit.client.1.smithi069.stdout:1/728: write d1/d2a/d84/f8b [827893,22667] 0 2024-03-22T07:16:09.071 INFO:tasks.workunit.client.1.smithi069.stdout:1/729: write d1/d2a/d84/fde [986899,24247] 0 2024-03-22T07:16:09.071 INFO:tasks.workunit.client.1.smithi069.stdout:8/987: stat d1/d1c/l50 0 2024-03-22T07:16:09.072 INFO:tasks.workunit.client.1.smithi069.stdout:8/988: read - d1/d1c/d3e/dde/f134 zero size 2024-03-22T07:16:09.072 INFO:tasks.workunit.client.1.smithi069.stdout:8/989: chown d1/d1c/d3e/d88/da8/dbb/fd8 3284 1 2024-03-22T07:16:09.072 INFO:tasks.workunit.client.1.smithi069.stdout:8/990: dread - d1/d1c/d3e/de6/ffb zero size 2024-03-22T07:16:09.073 INFO:tasks.workunit.client.1.smithi069.stdout:6/678: mkdir d4/dd/d12/d44/d58/db4/dd2 0 2024-03-22T07:16:09.076 INFO:tasks.workunit.client.0.smithi063.stdout:5/937: dread d8/fe [0,4194304] 0 2024-03-22T07:16:09.076 INFO:tasks.workunit.client.0.smithi063.stdout:4/972: symlink d1/d5/l152 0 2024-03-22T07:16:09.079 INFO:tasks.workunit.client.1.smithi069.stdout:1/730: creat d1/d13/d36/d6e/fe3 x:0 0 0 2024-03-22T07:16:09.079 INFO:tasks.workunit.client.1.smithi069.stdout:1/731: chown d1/d2a/d84/fdc 845 1 2024-03-22T07:16:09.079 INFO:tasks.workunit.client.1.smithi069.stdout:1/732: fsync d1/d13/d36/f8d 0 2024-03-22T07:16:09.080 INFO:tasks.workunit.client.1.smithi069.stdout:1/733: readlink d1/d13/l5a 0 2024-03-22T07:16:09.080 INFO:tasks.workunit.client.1.smithi069.stdout:6/679: symlink d4/dd/ld3 0 2024-03-22T07:16:09.081 INFO:tasks.workunit.client.1.smithi069.stdout:9/697: dread d2/dd/d3c/f43 [0,4194304] 0 2024-03-22T07:16:09.085 INFO:tasks.workunit.client.1.smithi069.stdout:1/734: getdents d1/d13/d36/d6e/d73 0 2024-03-22T07:16:09.085 INFO:tasks.workunit.client.1.smithi069.stdout:1/735: write d1/d2a/d50/d88/dae/db5/dbc/fce [370229,6808] 0 2024-03-22T07:16:09.085 INFO:tasks.workunit.client.1.smithi069.stdout:1/736: stat d1/d13/d36/d6e/d73/fa8 0 2024-03-22T07:16:09.086 INFO:tasks.workunit.client.0.smithi063.stdout:5/938: dread d8/d13/f39 [0,4194304] 0 2024-03-22T07:16:09.087 INFO:tasks.workunit.client.1.smithi069.stdout:6/680: symlink d4/dd/d12/d2b/ld4 0 2024-03-22T07:16:09.088 INFO:tasks.workunit.client.0.smithi063.stdout:5/939: unlink d8/d13/d24/f50 0 2024-03-22T07:16:09.089 INFO:tasks.workunit.client.0.smithi063.stdout:5/940: truncate d8/d13/d24/d62/fd3 4669064 0 2024-03-22T07:16:09.089 INFO:tasks.workunit.client.0.smithi063.stdout:5/941: fsync d8/d13/d1e/d3c/da0/fcf 0 2024-03-22T07:16:09.089 INFO:tasks.workunit.client.0.smithi063.stdout:5/942: stat d8/dba/dd7 0 2024-03-22T07:16:09.090 INFO:tasks.workunit.client.1.smithi069.stdout:9/698: mknod d2/dd/d3c/d53/d9d/cdb 0 2024-03-22T07:16:09.092 INFO:tasks.workunit.client.0.smithi063.stdout:5/943: rename d8/dba to d8/d13/d28/d57/d5f/d12f 0 2024-03-22T07:16:09.092 INFO:tasks.workunit.client.1.smithi069.stdout:0/883: rename d3/d7/d15/d29/d5a/d99 to d3/d7/d15/dce/d127 0 2024-03-22T07:16:09.095 INFO:tasks.workunit.client.1.smithi069.stdout:1/737: mknod d1/d2a/d50/d4c/d55/d5f/ce4 0 2024-03-22T07:16:09.099 INFO:tasks.workunit.client.1.smithi069.stdout:9/699: mknod d2/dd/d6e/cdc 0 2024-03-22T07:16:09.101 INFO:tasks.workunit.client.1.smithi069.stdout:9/700: symlink d2/dd/d3c/d2e/d84/da5/ldd 0 2024-03-22T07:16:09.101 INFO:tasks.workunit.client.1.smithi069.stdout:9/701: fsync d2/d21/f72 0 2024-03-22T07:16:09.101 INFO:tasks.workunit.client.1.smithi069.stdout:9/702: write d2/d21/f3a [470038,93988] 0 2024-03-22T07:16:09.101 INFO:tasks.workunit.client.1.smithi069.stdout:9/703: chown d2/dd/d3c/d48 25088244 1 2024-03-22T07:16:09.102 INFO:tasks.workunit.client.1.smithi069.stdout:9/704: mknod d2/dd/d3c/d2e/d84/da5/cde 0 2024-03-22T07:16:09.103 INFO:tasks.workunit.client.1.smithi069.stdout:9/705: chown d2/d51/f59 1451753 1 2024-03-22T07:16:09.105 INFO:tasks.workunit.client.0.smithi063.stdout:2/931: sync 2024-03-22T07:16:09.106 INFO:tasks.workunit.client.0.smithi063.stdout:2/932: write d0/de/d12/d20/f99 [981941,60115] 0 2024-03-22T07:16:09.107 INFO:tasks.workunit.client.0.smithi063.stdout:2/933: creat d0/de/d1b/d32/dac/f132 x:0 0 0 2024-03-22T07:16:09.108 INFO:tasks.workunit.client.0.smithi063.stdout:2/934: mknod d0/de/d35/d106/c133 0 2024-03-22T07:16:09.108 INFO:tasks.workunit.client.1.smithi069.stdout:1/738: dread d1/f48 [0,4194304] 0 2024-03-22T07:16:09.109 INFO:tasks.workunit.client.0.smithi063.stdout:2/935: unlink d0/de/d1b/d32/dac/cc0 0 2024-03-22T07:16:09.109 INFO:tasks.workunit.client.0.smithi063.stdout:2/936: chown d0/de/d74/dca/d11e/cf0 1553 1 2024-03-22T07:16:09.109 INFO:tasks.workunit.client.0.smithi063.stdout:2/937: creat d0/de/d12/d20/f134 x:0 0 0 2024-03-22T07:16:09.110 INFO:tasks.workunit.client.0.smithi063.stdout:2/938: truncate d0/de/d12/d20/f70 1513856 0 2024-03-22T07:16:09.112 INFO:tasks.workunit.client.0.smithi063.stdout:2/939: rename d0/de/d1b/d63/dd6/fe4 to d0/de/d74/d121/f135 0 2024-03-22T07:16:09.112 INFO:tasks.workunit.client.0.smithi063.stdout:2/940: chown d0/de/d35/d36/cad 116 1 2024-03-22T07:16:09.114 INFO:tasks.workunit.client.0.smithi063.stdout:2/941: link d0/f41 d0/de/d74/dca/d11e/f136 0 2024-03-22T07:16:09.121 INFO:tasks.workunit.client.1.smithi069.stdout:1/739: dread d1/d13/d36/f54 [0,4194304] 0 2024-03-22T07:16:09.132 INFO:tasks.workunit.client.1.smithi069.stdout:8/991: dwrite d1/d1c/f47 [0,4194304] 0 2024-03-22T07:16:09.132 INFO:tasks.workunit.client.1.smithi069.stdout:8/992: chown d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/lb6 129036 1 2024-03-22T07:16:09.132 INFO:tasks.workunit.client.1.smithi069.stdout:8/993: rmdir d1/d1c/d3e 39 2024-03-22T07:16:09.132 INFO:tasks.workunit.client.1.smithi069.stdout:8/994: symlink d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/ddb/l142 0 2024-03-22T07:16:09.133 INFO:tasks.workunit.client.1.smithi069.stdout:8/995: chown d1/d1c/d3e/d88/dcd/d6c/d4e/d5b/d82/d135 14832 1 2024-03-22T07:16:09.146 INFO:tasks.workunit.client.1.smithi069.stdout:7/725: dwrite d9/db/d1a/db1/d57/d44/d53/dc7/fcc [0,4194304] 0 2024-03-22T07:16:09.155 INFO:tasks.workunit.client.1.smithi069.stdout:4/895: dwrite d6/d12/d18/d21/f6a [0,4194304] 0 2024-03-22T07:16:09.156 INFO:tasks.workunit.client.1.smithi069.stdout:4/896: chown d6/d12/db2/c12f 6354 1 2024-03-22T07:16:09.156 INFO:tasks.workunit.client.1.smithi069.stdout:4/897: write d6/d12/d18/d21/f2f [2231690,29658] 0 2024-03-22T07:16:09.157 INFO:tasks.workunit.client.1.smithi069.stdout:4/898: creat d6/d16/d83/dd0/f134 x:0 0 0 2024-03-22T07:16:09.157 INFO:tasks.workunit.client.1.smithi069.stdout:4/899: dread - d6/dc4/ffc zero size 2024-03-22T07:16:09.157 INFO:tasks.workunit.client.1.smithi069.stdout:4/900: chown d6/d16/d3a/db4/dfb/d23/d59/l5a 1768228 1 2024-03-22T07:16:09.159 INFO:tasks.workunit.client.1.smithi069.stdout:9/706: dwrite d2/dd/d3c/f3e [0,4194304] 0 2024-03-22T07:16:09.161 INFO:tasks.workunit.client.0.smithi063.stdout:5/944: dwrite d8/d13/d1e/d31/daf/fb0 [0,4194304] 0 2024-03-22T07:16:09.161 INFO:tasks.workunit.client.0.smithi063.stdout:2/942: dwrite d0/de/d35/d36/ff6 [0,4194304] 0 2024-03-22T07:16:09.163 INFO:tasks.workunit.client.1.smithi069.stdout:9/707: dread d2/dd/d3c/f35 [0,4194304] 0 2024-03-22T07:16:09.163 INFO:tasks.workunit.client.1.smithi069.stdout:4/901: creat d6/d16/d3a/db4/dfb/d1d/d55/f135 x:0 0 0 2024-03-22T07:16:09.163 INFO:tasks.workunit.client.0.smithi063.stdout:5/945: symlink d8/d13/d24/d81/l130 0 2024-03-22T07:16:09.163 INFO:tasks.workunit.client.0.smithi063.stdout:5/946: chown d8/d13/d1e/d31/d64/l115 447980821 1 2024-03-22T07:16:09.163 INFO:tasks.workunit.client.0.smithi063.stdout:5/947: fsync d8/d13/d28/f73 0 2024-03-22T07:16:09.163 INFO:tasks.workunit.client.0.smithi063.stdout:5/948: truncate d8/de9/d7e/d9e/f121 232333 0 2024-03-22T07:16:09.164 INFO:tasks.workunit.client.0.smithi063.stdout:2/943: creat d0/de/d74/dca/d11e/d7f/d97/da2/f137 x:0 0 0 2024-03-22T07:16:09.164 INFO:tasks.workunit.client.1.smithi069.stdout:6/681: dwrite d4/fb [4194304,4194304] 0 2024-03-22T07:16:09.164 INFO:tasks.workunit.client.1.smithi069.stdout:0/884: dwrite d3/d7/d17/d1c/dd4/f10e [0,4194304] 0 2024-03-22T07:16:09.164 INFO:tasks.workunit.client.1.smithi069.stdout:0/885: chown d3/d7/d90/f94 75 1 2024-03-22T07:16:09.164 INFO:tasks.workunit.client.1.smithi069.stdout:7/726: write d9/db/f17 [6687583,120205] 0 2024-03-22T07:16:09.165 INFO:tasks.workunit.client.0.smithi063.stdout:2/944: rename d0/de/d1b/d63/d6e/f72 to d0/de/d74/dca/d11e/d7f/d97/da2/f138 0 2024-03-22T07:16:09.166 INFO:tasks.workunit.client.1.smithi069.stdout:4/902: creat d6/d12/f136 x:0 0 0 2024-03-22T07:16:09.166 INFO:tasks.workunit.client.1.smithi069.stdout:7/727: dread d9/db/f94 [0,4194304] 0 2024-03-22T07:16:09.173 INFO:tasks.workunit.client.1.smithi069.stdout:0/886: unlink d3/d7/d15/dad/fe7 0 2024-03-22T07:16:09.174 INFO:tasks.workunit.client.1.smithi069.stdout:4/903: mknod d6/dc4/db5/c137 0 2024-03-22T07:16:09.174 INFO:tasks.workunit.client.1.smithi069.stdout:4/904: readlink d6/d9b/d1f/l36 0 2024-03-22T07:16:09.175 INFO:tasks.workunit.client.1.smithi069.stdout:0/887: mkdir d3/d9a/d128 0 2024-03-22T07:16:09.177 INFO:tasks.workunit.client.1.smithi069.stdout:0/888: mkdir d3/d7/d15/d29/d5a/d84/df4/d129 0 2024-03-22T07:16:09.177 INFO:tasks.workunit.client.1.smithi069.stdout:0/889: write d3/d7/d15/d1f/fe8 [1397966,121497] 0 2024-03-22T07:16:09.179 INFO:tasks.workunit.client.0.smithi063.stdout:2/945: read d0/de/d74/f7e [2873391,62593] 0 2024-03-22T07:16:09.180 INFO:tasks.workunit.client.1.smithi069.stdout:0/890: unlink d3/d7/d5e/c124 0 2024-03-22T07:16:09.180 INFO:tasks.workunit.client.1.smithi069.stdout:0/891: write d3/d7/d15/d29/d5a/d5d/ffe [1977588,1774] 0 2024-03-22T07:16:09.181 INFO:tasks.workunit.client.0.smithi063.stdout:2/946: link d0/de/d12/d20/f130 d0/de/d74/dca/d11e/d7f/d129/f139 0 2024-03-22T07:16:09.181 INFO:tasks.workunit.client.0.smithi063.stdout:2/947: write d0/de/d12/d20/f9b [890136,90230] 0 2024-03-22T07:16:09.182 INFO:tasks.workunit.client.0.smithi063.stdout:2/948: read d0/de/d12/d5a/f9d [72066,222] 0 2024-03-22T07:16:09.182 INFO:tasks.workunit.client.1.smithi069.stdout:1/740: rename d1/d2a/d50 to d1/d13/d36/de5 0 2024-03-22T07:16:09.182 INFO:tasks.workunit.client.1.smithi069.stdout:1/741: fsync d1/d13/d36/de5/d88/d41/f4b 0 2024-03-22T07:16:09.184 INFO:tasks.workunit.client.1.smithi069.stdout:9/708: rename d2/dd/c23 to d2/d21/dd7/cdf 0 2024-03-22T07:16:09.184 INFO:tasks.workunit.client.1.smithi069.stdout:9/709: getdents d2/dd/d3c/d53/da6 0 2024-03-22T07:16:09.185 INFO:tasks.workunit.client.1.smithi069.stdout:1/742: mknod d1/d13/d36/de5/d88/d41/d64/ce6 0 2024-03-22T07:16:09.188 INFO:tasks.workunit.client.1.smithi069.stdout:4/905: rename d6/dc4/cfa to d6/d9b/d1f/d3f/d51/db8/c138 0 2024-03-22T07:16:09.189 INFO:tasks.workunit.client.1.smithi069.stdout:1/743: mkdir d1/d13/d36/de5/d88/dae/de7 0 2024-03-22T07:16:09.191 INFO:tasks.workunit.client.1.smithi069.stdout:4/906: rename d6/d9b/d1f/d58/d7c/f104 to d6/d16/d3a/db4/f139 0 2024-03-22T07:16:09.191 INFO:tasks.workunit.client.1.smithi069.stdout:4/907: fdatasync d6/d12/d5d/ff6 0 2024-03-22T07:16:09.191 INFO:tasks.workunit.client.1.smithi069.stdout:4/908: chown d6/d12/d18/d21/l40 9 1 2024-03-22T07:16:09.191 INFO:tasks.workunit.client.1.smithi069.stdout:1/744: mknod d1/d2a/d5e/ce8 0 2024-03-22T07:16:09.192 INFO:tasks.workunit.client.1.smithi069.stdout:4/909: getdents d6/d16/d3a 0 2024-03-22T07:16:09.193 INFO:tasks.workunit.client.1.smithi069.stdout:1/745: getdents d1/d2a/d5e/d93 0 2024-03-22T07:16:09.195 INFO:tasks.workunit.client.1.smithi069.stdout:1/746: rename d1/d13/d36/de5/l94 to d1/d13/d36/de5/d88/d41/le9 0 2024-03-22T07:16:09.197 INFO:tasks.workunit.client.0.smithi063.stdout:7/945: sync 2024-03-22T07:16:09.197 INFO:tasks.workunit.client.0.smithi063.stdout:7/946: stat d5/f9 0 2024-03-22T07:16:09.200 INFO:tasks.workunit.client.1.smithi069.stdout:9/710: write d2/dd/d3c/d2e/fad [2635024,110965] 0 2024-03-22T07:16:09.200 INFO:tasks.workunit.client.1.smithi069.stdout:6/682: dwrite d4/dd/d12/d13/f8a [0,4194304] 0 2024-03-22T07:16:09.201 INFO:tasks.workunit.client.0.smithi063.stdout:7/947: write d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d29/f5d [2449675,131007] 0 2024-03-22T07:16:09.201 INFO:tasks.workunit.client.0.smithi063.stdout:7/948: creat d5/da/d28/d11e/da8/dbc/d62/d73/dc5/f139 x:0 0 0 2024-03-22T07:16:09.203 INFO:tasks.workunit.client.1.smithi069.stdout:7/728: dwrite d9/db/fe3 [0,4194304] 0 2024-03-22T07:16:09.204 INFO:tasks.workunit.client.0.smithi063.stdout:7/949: symlink d5/da/d28/da7/l13a 0 2024-03-22T07:16:09.205 INFO:tasks.workunit.client.1.smithi069.stdout:9/711: mkdir d2/dd/d3c/d2e/d84/da5/de0 0 2024-03-22T07:16:09.206 INFO:tasks.workunit.client.1.smithi069.stdout:6/683: mknod d4/dd/d41/d78/d9a/cd5 0 2024-03-22T07:16:09.206 INFO:tasks.workunit.client.1.smithi069.stdout:6/684: creat d4/dd/d12/fd6 x:0 0 0 2024-03-22T07:16:09.207 INFO:tasks.workunit.client.0.smithi063.stdout:7/950: rmdir d5/da/d4e 39 2024-03-22T07:16:09.208 INFO:tasks.workunit.client.0.smithi063.stdout:7/951: symlink d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d4f/df8/l13b 0 2024-03-22T07:16:09.210 INFO:tasks.workunit.client.1.smithi069.stdout:7/729: unlink d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/dce/fd9 0 2024-03-22T07:16:09.210 INFO:tasks.workunit.client.1.smithi069.stdout:7/730: chown d9/db/d48 401006 1 2024-03-22T07:16:09.211 INFO:tasks.workunit.client.0.smithi063.stdout:7/952: creat d5/da/d5f/d101/d117/f13c x:0 0 0 2024-03-22T07:16:09.211 INFO:tasks.workunit.client.0.smithi063.stdout:7/953: stat d5/d16/d43/c11a 0 2024-03-22T07:16:09.212 INFO:tasks.workunit.client.1.smithi069.stdout:6/685: link d4/dd/l15 d4/d7/da/d3f/ld7 0 2024-03-22T07:16:09.213 INFO:tasks.workunit.client.1.smithi069.stdout:7/731: rename d9/db/d1a/db1/d57/d44/d53/l93 to d9/db/d1a/db1/d57/d44/d53/d2e/leb 0 2024-03-22T07:16:09.214 INFO:tasks.workunit.client.1.smithi069.stdout:7/732: dread - d9/db/d1a/db1/d57/d44/d53/dc7/db5/fbb zero size 2024-03-22T07:16:09.216 INFO:tasks.workunit.client.1.smithi069.stdout:7/733: rmdir d9/db/d1a/db1/d57/d75/d7a/d96 39 2024-03-22T07:16:09.217 INFO:tasks.workunit.client.1.smithi069.stdout:7/734: unlink d9/db/d48/fdb 0 2024-03-22T07:16:09.217 INFO:tasks.workunit.client.1.smithi069.stdout:1/747: dwrite d1/d2a/d84/d9a/fc1 [0,4194304] 0 2024-03-22T07:16:09.217 INFO:tasks.workunit.client.1.smithi069.stdout:1/748: fdatasync d1/d13/d36/d5b/d95/fad 0 2024-03-22T07:16:09.218 INFO:tasks.workunit.client.1.smithi069.stdout:6/686: dread d4/dd/d12/d44/f9b [0,4194304] 0 2024-03-22T07:16:09.218 INFO:tasks.workunit.client.1.smithi069.stdout:6/687: stat d4/d7/d5b/f97 0 2024-03-22T07:16:09.219 INFO:tasks.workunit.client.1.smithi069.stdout:7/735: unlink d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/f40 0 2024-03-22T07:16:09.221 INFO:tasks.workunit.client.1.smithi069.stdout:0/892: dread d3/d7/d15/d29/d5a/d5d/ffe [0,4194304] 0 2024-03-22T07:16:09.221 INFO:tasks.workunit.client.1.smithi069.stdout:0/893: stat d3/d7/d15/d29/d5a/d5d/lc8 0 2024-03-22T07:16:09.229 INFO:tasks.workunit.client.1.smithi069.stdout:6/688: dread d4/f14 [0,4194304] 0 2024-03-22T07:16:09.235 INFO:tasks.workunit.client.1.smithi069.stdout:7/736: rename d9/da1 to d9/db/d1a/db1/d57/d75/dec 0 2024-03-22T07:16:09.235 INFO:tasks.workunit.client.1.smithi069.stdout:7/737: fdatasync d9/db/d1a/db1/d57/d44/d53/d74/d30/fc8 0 2024-03-22T07:16:09.237 INFO:tasks.workunit.client.1.smithi069.stdout:7/738: creat d9/db/d1a/db1/d57/d44/d53/d74/fed x:0 0 0 2024-03-22T07:16:09.249 INFO:tasks.workunit.client.1.smithi069.stdout:9/712: dwrite d2/dd/d3c/d2e/fa3 [0,4194304] 0 2024-03-22T07:16:09.250 INFO:tasks.workunit.client.1.smithi069.stdout:9/713: dread - d2/d71/fd4 zero size 2024-03-22T07:16:09.250 INFO:tasks.workunit.client.1.smithi069.stdout:9/714: write d2/d21/d89/fd5 [158641,89291] 0 2024-03-22T07:16:09.250 INFO:tasks.workunit.client.1.smithi069.stdout:9/715: chown d2/dd/c17 62935 1 2024-03-22T07:16:09.252 INFO:tasks.workunit.client.1.smithi069.stdout:9/716: getdents d2/dd/d3c/d48/d7f/d86 0 2024-03-22T07:16:09.252 INFO:tasks.workunit.client.1.smithi069.stdout:9/717: creat d2/dd/d6e/fe1 x:0 0 0 2024-03-22T07:16:09.256 INFO:tasks.workunit.client.1.smithi069.stdout:9/718: mknod d2/d21/dd7/ce2 0 2024-03-22T07:16:09.257 INFO:tasks.workunit.client.1.smithi069.stdout:9/719: mknod d2/dd/d3c/d2e/d84/da5/de0/ce3 0 2024-03-22T07:16:09.259 INFO:tasks.workunit.client.1.smithi069.stdout:9/720: mkdir d2/d51/dbd/de4 0 2024-03-22T07:16:09.260 INFO:tasks.workunit.client.1.smithi069.stdout:9/721: mknod d2/d51/ce5 0 2024-03-22T07:16:09.260 INFO:tasks.workunit.client.1.smithi069.stdout:9/722: fdatasync d2/d51/f59 0 2024-03-22T07:16:09.269 INFO:tasks.workunit.client.1.smithi069.stdout:1/749: dwrite d1/f3b [0,4194304] 0 2024-03-22T07:16:09.269 INFO:tasks.workunit.client.0.smithi063.stdout:7/954: dwrite d5/da/d28/da7/f72 [4194304,4194304] 0 2024-03-22T07:16:09.269 INFO:tasks.workunit.client.0.smithi063.stdout:7/955: symlink d5/da/d5f/d101/l13d 0 2024-03-22T07:16:09.270 INFO:tasks.workunit.client.1.smithi069.stdout:0/894: dwrite d3/d7/d15/d29/d5a/d5d/f119 [0,4194304] 0 2024-03-22T07:16:09.271 INFO:tasks.workunit.client.1.smithi069.stdout:6/689: dwrite d4/dd/d12/d2b/f3a [0,4194304] 0 2024-03-22T07:16:09.272 INFO:tasks.workunit.client.1.smithi069.stdout:6/690: write d4/dd/d12/d2b/da5/fcc [718967,86426] 0 2024-03-22T07:16:09.272 INFO:tasks.workunit.client.1.smithi069.stdout:0/895: symlink d3/d7/d15/d29/d6e/l12a 0 2024-03-22T07:16:09.273 INFO:tasks.workunit.client.1.smithi069.stdout:9/723: write d2/f55 [4412610,113672] 0 2024-03-22T07:16:09.273 INFO:tasks.workunit.client.1.smithi069.stdout:9/724: chown d2/d21/l6b 66205508 1 2024-03-22T07:16:09.275 INFO:tasks.workunit.client.1.smithi069.stdout:6/691: creat d4/dd/d41/d78/d9a/fd8 x:0 0 0 2024-03-22T07:16:09.281 INFO:tasks.workunit.client.0.smithi063.stdout:4/973: sync 2024-03-22T07:16:09.284 INFO:tasks.workunit.client.1.smithi069.stdout:9/725: write d2/d21/f73 [1977768,64411] 0 2024-03-22T07:16:09.285 INFO:tasks.workunit.client.0.smithi063.stdout:4/974: creat d1/d5/d43/d27/d2c/df4/d10b/f153 x:0 0 0 2024-03-22T07:16:09.285 INFO:tasks.workunit.client.0.smithi063.stdout:4/975: chown d1/d5/d43/d27/d2c/d4a/d78/dfd/c101 2034 1 2024-03-22T07:16:09.287 INFO:tasks.workunit.client.0.smithi063.stdout:4/976: unlink d1/d5/d43/d27/d2c/df4/d10b/f11d 0 2024-03-22T07:16:09.287 INFO:tasks.workunit.client.0.smithi063.stdout:4/977: read - d1/d5/d43/d27/d2c/d4a/d146/d54/ddb/f13c zero size 2024-03-22T07:16:09.291 INFO:tasks.workunit.client.1.smithi069.stdout:7/739: dwrite d9/db/d1a/db1/d57/d44/d53/d74/f91 [0,4194304] 0 2024-03-22T07:16:09.293 INFO:tasks.workunit.client.1.smithi069.stdout:7/740: unlink d9/db/d1a/db1/d57/d44/d53/d74/d30/l4e 0 2024-03-22T07:16:09.294 INFO:tasks.workunit.client.1.smithi069.stdout:7/741: creat d9/db/d1a/fee x:0 0 0 2024-03-22T07:16:09.295 INFO:tasks.workunit.client.1.smithi069.stdout:7/742: creat d9/db/d1a/db1/fef x:0 0 0 2024-03-22T07:16:09.296 INFO:tasks.workunit.client.1.smithi069.stdout:7/743: creat d9/db/d1a/db1/d57/d44/d53/ff0 x:0 0 0 2024-03-22T07:16:09.299 INFO:tasks.workunit.client.1.smithi069.stdout:7/744: rename d9/db/f19 to d9/db/d1a/db1/d57/d44/d53/d74/ff1 0 2024-03-22T07:16:09.300 INFO:tasks.workunit.client.1.smithi069.stdout:7/745: mknod d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/cf2 0 2024-03-22T07:16:09.301 INFO:tasks.workunit.client.1.smithi069.stdout:7/746: creat d9/db/d1a/db1/d57/d75/d7a/dcf/ff3 x:0 0 0 2024-03-22T07:16:09.301 INFO:tasks.workunit.client.1.smithi069.stdout:7/747: chown d9/db/d1a/db1/d57/d44/d53/dc7/f9d 57 1 2024-03-22T07:16:09.301 INFO:tasks.workunit.client.1.smithi069.stdout:7/748: creat d9/db/d1a/db1/d57/d75/ff4 x:0 0 0 2024-03-22T07:16:09.302 INFO:tasks.workunit.client.1.smithi069.stdout:7/749: mkdir d9/de4/df5 0 2024-03-22T07:16:09.303 INFO:tasks.workunit.client.1.smithi069.stdout:7/750: read d9/db/d1a/f21 [1549750,79370] 0 2024-03-22T07:16:09.312 INFO:tasks.workunit.client.1.smithi069.stdout:9/726: dread d2/d51/f64 [0,4194304] 0 2024-03-22T07:16:09.312 INFO:tasks.workunit.client.1.smithi069.stdout:9/727: stat d2/dd/l15 0 2024-03-22T07:16:09.313 INFO:tasks.workunit.client.1.smithi069.stdout:9/728: symlink d2/dd/d3c/d48/d7f/da2/le6 0 2024-03-22T07:16:09.313 INFO:tasks.workunit.client.1.smithi069.stdout:9/729: readlink d2/dd/d3c/l61 0 2024-03-22T07:16:09.313 INFO:tasks.workunit.client.1.smithi069.stdout:9/730: symlink d2/dd/d3c/d53/da6/le7 0 2024-03-22T07:16:09.315 INFO:tasks.workunit.client.1.smithi069.stdout:9/731: creat d2/dd/d3c/d48/fe8 x:0 0 0 2024-03-22T07:16:09.315 INFO:tasks.workunit.client.1.smithi069.stdout:9/732: mknod d2/d76/ce9 0 2024-03-22T07:16:09.317 INFO:tasks.workunit.client.1.smithi069.stdout:9/733: symlink d2/dd/d3c/d2e/d84/da5/de0/lea 0 2024-03-22T07:16:09.317 INFO:tasks.workunit.client.1.smithi069.stdout:9/734: mkdir d2/dd/d3c/d48/d7f/deb 0 2024-03-22T07:16:09.319 INFO:tasks.workunit.client.1.smithi069.stdout:9/735: mknod d2/d31/cec 0 2024-03-22T07:16:09.319 INFO:tasks.workunit.client.1.smithi069.stdout:9/736: chown d2/d21/f3a 0 1 2024-03-22T07:16:09.319 INFO:tasks.workunit.client.1.smithi069.stdout:9/737: stat d2/d71/f8d 0 2024-03-22T07:16:09.319 INFO:tasks.workunit.client.1.smithi069.stdout:9/738: read d2/dd/d3c/d48/f4d [353789,76894] 0 2024-03-22T07:16:09.319 INFO:tasks.workunit.client.1.smithi069.stdout:9/739: read d2/d71/d90/f95 [4010577,61409] 0 2024-03-22T07:16:09.319 INFO:tasks.workunit.client.1.smithi069.stdout:9/740: creat d2/dd/d3c/d48/d7f/deb/fed x:0 0 0 2024-03-22T07:16:09.328 INFO:tasks.workunit.client.0.smithi063.stdout:7/956: dwrite d5/da/d28/d11e/da8/f123 [0,4194304] 0 2024-03-22T07:16:09.328 INFO:tasks.workunit.client.0.smithi063.stdout:7/957: read - d5/da/d4e/f134 zero size 2024-03-22T07:16:09.328 INFO:tasks.workunit.client.1.smithi069.stdout:9/741: dread d2/dd/d3c/d48/d7c/f9e [0,4194304] 0 2024-03-22T07:16:09.328 INFO:tasks.workunit.client.1.smithi069.stdout:9/742: fdatasync d2/dd/d3c/d2e/f4b 0 2024-03-22T07:16:09.330 INFO:tasks.workunit.client.1.smithi069.stdout:9/743: rmdir d2/d21/d77 39 2024-03-22T07:16:09.331 INFO:tasks.workunit.client.1.smithi069.stdout:9/744: truncate d2/dd/d3c/d48/d7f/d86/fc0 1556951 0 2024-03-22T07:16:09.332 INFO:tasks.workunit.client.1.smithi069.stdout:9/745: symlink d2/dd/d3c/d48/lee 0 2024-03-22T07:16:09.334 INFO:tasks.workunit.client.1.smithi069.stdout:9/746: mkdir d2/d21/d77/def 0 2024-03-22T07:16:09.334 INFO:tasks.workunit.client.1.smithi069.stdout:6/692: dwrite d4/dd/d12/d2b/f3a [0,4194304] 0 2024-03-22T07:16:09.336 INFO:tasks.workunit.client.1.smithi069.stdout:9/747: creat d2/dd/d3c/d2e/d84/da5/de0/ff0 x:0 0 0 2024-03-22T07:16:09.337 INFO:tasks.workunit.client.1.smithi069.stdout:6/693: symlink d4/dd/d12/d44/ld9 0 2024-03-22T07:16:09.337 INFO:tasks.workunit.client.1.smithi069.stdout:6/694: chown d4/dd/d12/d2b/d72/d73/fce 0 1 2024-03-22T07:16:09.337 INFO:tasks.workunit.client.0.smithi063.stdout:7/958: write d5/da/d28/d11e/da8/dbc/d62/d73/f100 [2791060,65814] 0 2024-03-22T07:16:09.338 INFO:tasks.workunit.client.1.smithi069.stdout:9/748: symlink d2/d21/dd7/lf1 0 2024-03-22T07:16:09.339 INFO:tasks.workunit.client.1.smithi069.stdout:6/695: creat d4/dd/dbd/fda x:0 0 0 2024-03-22T07:16:09.339 INFO:tasks.workunit.client.1.smithi069.stdout:6/696: fdatasync d4/dd/d12/d44/d7f/fba 0 2024-03-22T07:16:09.339 INFO:tasks.workunit.client.1.smithi069.stdout:6/697: getdents d4/d7/da6 0 2024-03-22T07:16:09.340 INFO:tasks.workunit.client.1.smithi069.stdout:9/749: mknod d2/d21/d77/cf2 0 2024-03-22T07:16:09.342 INFO:tasks.workunit.client.1.smithi069.stdout:6/698: symlink d4/dd/ldb 0 2024-03-22T07:16:09.344 INFO:tasks.workunit.client.1.smithi069.stdout:0/896: dwrite d3/d7/d15/d29/f30 [0,4194304] 0 2024-03-22T07:16:09.344 INFO:tasks.workunit.client.1.smithi069.stdout:1/750: dwrite d1/d2a/d84/dd1/fe1 [0,4194304] 0 2024-03-22T07:16:09.345 INFO:tasks.workunit.client.1.smithi069.stdout:7/751: dwrite d9/db/d1a/db1/d57/d44/d53/d2e/f4d [0,4194304] 0 2024-03-22T07:16:09.345 INFO:tasks.workunit.client.1.smithi069.stdout:7/752: readlink d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/l3f 0 2024-03-22T07:16:09.345 INFO:tasks.workunit.client.1.smithi069.stdout:7/753: write d9/db/d1a/db1/d57/d44/d53/d74/d6e/fe6 [170254,90949] 0 2024-03-22T07:16:09.345 INFO:tasks.workunit.client.1.smithi069.stdout:6/699: symlink d4/dd/d12/d2b/d72/d73/d9d/dc3/ldc 0 2024-03-22T07:16:09.346 INFO:tasks.workunit.client.1.smithi069.stdout:1/751: read d1/d2a/d84/fde [554339,111334] 0 2024-03-22T07:16:09.353 INFO:tasks.workunit.client.1.smithi069.stdout:0/897: getdents d3/d7/d15/dce/d127/dca 0 2024-03-22T07:16:09.357 INFO:tasks.workunit.client.1.smithi069.stdout:7/754: creat d9/de4/ff6 x:0 0 0 2024-03-22T07:16:09.357 INFO:tasks.workunit.client.1.smithi069.stdout:7/755: dread - d9/db/d1a/db1/d57/d44/d53/d74/fe0 zero size 2024-03-22T07:16:09.360 INFO:tasks.workunit.client.1.smithi069.stdout:1/752: link d1/d2a/d84/fde d1/d13/d36/de5/d4c/d55/fea 0 2024-03-22T07:16:09.361 INFO:tasks.workunit.client.1.smithi069.stdout:0/898: unlink d3/d7/d5e/c102 0 2024-03-22T07:16:09.361 INFO:tasks.workunit.client.1.smithi069.stdout:7/756: mkdir d9/d8e/df7 0 2024-03-22T07:16:09.362 INFO:tasks.workunit.client.1.smithi069.stdout:6/700: getdents d4/dd/d12/d2b/da5 0 2024-03-22T07:16:09.362 INFO:tasks.workunit.client.1.smithi069.stdout:6/701: getdents d4/dd/d12/d44/d58/db4/dd2 0 2024-03-22T07:16:09.364 INFO:tasks.workunit.client.1.smithi069.stdout:1/753: creat d1/d2a/d5e/d9b/feb x:0 0 0 2024-03-22T07:16:09.364 INFO:tasks.workunit.client.1.smithi069.stdout:1/754: dread - d1/d13/d36/d6e/fd0 zero size 2024-03-22T07:16:09.365 INFO:tasks.workunit.client.1.smithi069.stdout:9/750: dwrite d2/db6/fda [0,4194304] 0 2024-03-22T07:16:09.365 INFO:tasks.workunit.client.1.smithi069.stdout:9/751: dread - d2/dd/d3c/d48/fe8 zero size 2024-03-22T07:16:09.366 INFO:tasks.workunit.client.0.smithi063.stdout:7/959: dwrite d5/da/d5f/f97 [0,4194304] 0 2024-03-22T07:16:09.366 INFO:tasks.workunit.client.0.smithi063.stdout:7/960: write d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d4f/df8/f125 [5076907,119834] 0 2024-03-22T07:16:09.366 INFO:tasks.workunit.client.1.smithi069.stdout:6/702: mknod d4/dd/d12/d2b/d72/d73/d74/cdd 0 2024-03-22T07:16:09.367 INFO:tasks.workunit.client.0.smithi063.stdout:4/978: sync 2024-03-22T07:16:09.367 INFO:tasks.workunit.client.0.smithi063.stdout:2/949: sync 2024-03-22T07:16:09.367 INFO:tasks.workunit.client.0.smithi063.stdout:5/949: sync 2024-03-22T07:16:09.367 INFO:tasks.workunit.client.0.smithi063.stdout:2/950: dread - d0/de/d74/dca/d11e/d7f/d129/f139 zero size 2024-03-22T07:16:09.367 INFO:tasks.workunit.client.1.smithi069.stdout:1/755: mkdir d1/d13/d36/de5/d88/dec 0 2024-03-22T07:16:09.368 INFO:tasks.workunit.client.1.smithi069.stdout:0/899: dread d3/d7/d15/d29/dcf/d114/f111 [0,4194304] 0 2024-03-22T07:16:09.369 INFO:tasks.workunit.client.0.smithi063.stdout:7/961: creat d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d4f/dd1/f13e x:0 0 0 2024-03-22T07:16:09.372 INFO:tasks.workunit.client.0.smithi063.stdout:4/979: link d1/d5/dc/d95/d59/d9c/dac/dcd/lc4 d1/d5/dc/d95/d143/l154 0 2024-03-22T07:16:09.372 INFO:tasks.workunit.client.0.smithi063.stdout:4/980: chown d1/d5/d3c/d8f/dbb/d114/d79/l111 22791790 1 2024-03-22T07:16:09.372 INFO:tasks.workunit.client.0.smithi063.stdout:4/981: readlink d1/d5/dc/d95/d59/d9c/dac/dcd/lc4 0 2024-03-22T07:16:09.372 INFO:tasks.workunit.client.0.smithi063.stdout:4/982: creat d1/f155 x:0 0 0 2024-03-22T07:16:09.373 INFO:tasks.workunit.client.0.smithi063.stdout:5/950: creat d8/d13/d28/d7d/f131 x:0 0 0 2024-03-22T07:16:09.375 INFO:tasks.workunit.client.0.smithi063.stdout:7/962: symlink d5/da/d28/d11e/da8/dbc/d62/d8d/d98/l13f 0 2024-03-22T07:16:09.378 INFO:tasks.workunit.client.1.smithi069.stdout:2/631: sync 2024-03-22T07:16:09.379 INFO:tasks.workunit.client.1.smithi069.stdout:5/681: sync 2024-03-22T07:16:09.379 INFO:tasks.workunit.client.1.smithi069.stdout:8/996: sync 2024-03-22T07:16:09.379 INFO:tasks.workunit.client.1.smithi069.stdout:8/997: dread - d1/d1c/d3e/d88/dcd/d6c/d4e/f12c zero size 2024-03-22T07:16:09.379 INFO:tasks.workunit.client.1.smithi069.stdout:4/910: sync 2024-03-22T07:16:09.379 INFO:tasks.workunit.client.1.smithi069.stdout:8/998: read d1/d1c/d3e/d88/dcd/f2d [3784879,51893] 0 2024-03-22T07:16:09.379 INFO:tasks.workunit.client.1.smithi069.stdout:3/895: sync 2024-03-22T07:16:09.379 INFO:tasks.workunit.client.1.smithi069.stdout:3/896: write d5/d9/d26/d36/d41/f75 [2030511,113085] 0 2024-03-22T07:16:09.379 INFO:tasks.workunit.client.1.smithi069.stdout:3/897: creat d5/d9/d26/d3f/d56/d7f/d90/f12c x:0 0 0 2024-03-22T07:16:09.381 INFO:tasks.workunit.client.0.smithi063.stdout:4/983: symlink d1/d5/dc/d95/l156 0 2024-03-22T07:16:09.382 INFO:tasks.workunit.client.0.smithi063.stdout:4/984: dread d1/d5/d3c/d8f/fbc [0,4194304] 0 2024-03-22T07:16:09.383 INFO:tasks.workunit.client.1.smithi069.stdout:6/703: dread d4/dd/d12/f32 [0,4194304] 0 2024-03-22T07:16:09.383 INFO:tasks.workunit.client.0.smithi063.stdout:5/951: getdents d8/d13/d24/da4 0 2024-03-22T07:16:09.385 INFO:tasks.workunit.client.1.smithi069.stdout:9/752: dread d2/d31/f40 [4194304,4194304] 0 2024-03-22T07:16:09.387 INFO:tasks.workunit.client.0.smithi063.stdout:7/963: symlink d5/da/d28/d11e/l140 0 2024-03-22T07:16:09.389 INFO:tasks.workunit.client.0.smithi063.stdout:7/964: dread d5/da/d28/d11e/da8/dbc/d62/d8d/dbb/f110 [0,4194304] 0 2024-03-22T07:16:09.389 INFO:tasks.workunit.client.1.smithi069.stdout:9/753: mkdir d2/d51/dbd/de4/df3 0 2024-03-22T07:16:09.389 INFO:tasks.workunit.client.0.smithi063.stdout:4/985: rename d1/d5/d43/d107/d147 to d1/d5/d43/d27/d2c/d4a/d80/d157 0 2024-03-22T07:16:09.391 INFO:tasks.workunit.client.1.smithi069.stdout:9/754: link d2/d31/f67 d2/dd/d3c/d53/d9d/ff4 0 2024-03-22T07:16:09.394 INFO:tasks.workunit.client.1.smithi069.stdout:9/755: unlink d2/dd/d3c/l33 0 2024-03-22T07:16:09.394 INFO:tasks.workunit.client.1.smithi069.stdout:8/999: write d1/d1c/d3e/d88/dcd/d6c/d4e/d98/f105 [2845786,47457] 0 2024-03-22T07:16:09.395 INFO:tasks.workunit.client.0.smithi063.stdout:5/952: truncate d8/d13/f39 2318927 0 2024-03-22T07:16:09.395 INFO:tasks.workunit.client.0.smithi063.stdout:7/965: mkdir d5/da/d28/d11e/da8/dbc/d62/d8d/d98/d141 0 2024-03-22T07:16:09.395 INFO:tasks.workunit.client.0.smithi063.stdout:7/966: chown d5/d16/l27 89551 1 2024-03-22T07:16:09.395 INFO:tasks.workunit.client.0.smithi063.stdout:7/967: write d5/da/f127 [6046637,78616] 0 2024-03-22T07:16:09.395 INFO:tasks.workunit.client.0.smithi063.stdout:4/986: symlink d1/d5/d3c/d8f/dc0/dcb/l158 0 2024-03-22T07:16:09.397 INFO:tasks.workunit.client.0.smithi063.stdout:7/968: symlink d5/da/d28/d11e/da8/dbc/d62/d8d/d98/l142 0 2024-03-22T07:16:09.400 INFO:tasks.workunit.client.0.smithi063.stdout:4/987: rename d1/d5/d43/d27/d2c/d4a/d78/f7d to d1/d5/d43/d27/d2c/df4/d128/f159 0 2024-03-22T07:16:09.404 INFO:tasks.workunit.client.0.smithi063.stdout:4/988: rename d1/d5/dc/d95/da4/ff0 to d1/d5/d43/d27/d2c/d4a/d78/f15a 0 2024-03-22T07:16:09.406 INFO:tasks.workunit.client.0.smithi063.stdout:7/969: rename d5/da/d28/d11e/da8/l118 to d5/da/d28/d11e/l143 0 2024-03-22T07:16:09.406 INFO:tasks.workunit.client.0.smithi063.stdout:7/970: dread - d5/da/d28/d11e/da8/dbc/d62/d73/dc5/f11d zero size 2024-03-22T07:16:09.406 INFO:tasks.workunit.client.0.smithi063.stdout:7/971: readlink d5/da/d5f/lc2 0 2024-03-22T07:16:09.406 INFO:tasks.workunit.client.0.smithi063.stdout:7/972: fsync d5/da/d28/d11e/da8/dbc/d62/d73/dc5/f139 0 2024-03-22T07:16:09.406 INFO:tasks.workunit.client.1.smithi069.stdout:9/756: dread d2/f8f [0,4194304] 0 2024-03-22T07:16:09.407 INFO:tasks.workunit.client.0.smithi063.stdout:4/989: creat d1/d5/d43/d27/d2c/d4a/d146/d54/f15b x:0 0 0 2024-03-22T07:16:09.408 INFO:tasks.workunit.client.0.smithi063.stdout:7/973: mknod d5/d16/d43/d78/c144 0 2024-03-22T07:16:09.409 INFO:tasks.workunit.client.0.smithi063.stdout:7/974: chown d5/d38 2 1 2024-03-22T07:16:09.409 INFO:tasks.workunit.client.0.smithi063.stdout:7/975: dread - d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d4f/dd1/fe9 zero size 2024-03-22T07:16:09.409 INFO:tasks.workunit.client.0.smithi063.stdout:7/976: write d5/d16/d1b/f7c [1282665,8313] 0 2024-03-22T07:16:09.409 INFO:tasks.workunit.client.0.smithi063.stdout:4/990: mknod d1/d5/d43/d27/d2c/d4a/d78/dfd/c15c 0 2024-03-22T07:16:09.411 INFO:tasks.workunit.client.0.smithi063.stdout:4/991: unlink d1/d5/d43/d27/d2c/d4a/d146/d54/ddb/f140 0 2024-03-22T07:16:09.412 INFO:tasks.workunit.client.0.smithi063.stdout:4/992: mknod d1/d5/dc/d95/d59/d9c/dac/dc9/c15d 0 2024-03-22T07:16:09.413 INFO:tasks.workunit.client.0.smithi063.stdout:4/993: mkdir d1/d5/d43/d27/d2c/df3/df7/d15e 0 2024-03-22T07:16:09.413 INFO:tasks.workunit.client.0.smithi063.stdout:4/994: write d1/d5/d43/f14 [3349358,35722] 0 2024-03-22T07:16:09.413 INFO:tasks.workunit.client.0.smithi063.stdout:4/995: dread - d1/d5/d3c/d8f/dbb/fd1 zero size 2024-03-22T07:16:09.415 INFO:tasks.workunit.client.0.smithi063.stdout:5/953: dread d8/f10 [0,4194304] 0 2024-03-22T07:16:09.416 INFO:tasks.workunit.client.0.smithi063.stdout:5/954: write d8/f44 [2321977,44766] 0 2024-03-22T07:16:09.417 INFO:tasks.workunit.client.0.smithi063.stdout:5/955: symlink d8/d13/d24/da4/d8b/db6/dfb/l132 0 2024-03-22T07:16:09.433 INFO:tasks.workunit.client.0.smithi063.stdout:7/977: dread d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d29/fb4 [0,4194304] 0 2024-03-22T07:16:09.435 INFO:tasks.workunit.client.0.smithi063.stdout:7/978: symlink d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d4f/df8/l145 0 2024-03-22T07:16:09.435 INFO:tasks.workunit.client.0.smithi063.stdout:7/979: write d5/fe6 [122354,114451] 0 2024-03-22T07:16:09.436 INFO:tasks.workunit.client.0.smithi063.stdout:7/980: truncate d5/d16/f32 1281213 0 2024-03-22T07:16:09.440 INFO:tasks.workunit.client.1.smithi069.stdout:7/757: dwrite d9/db/d1a/db1/d57/d44/d53/f38 [0,4194304] 0 2024-03-22T07:16:09.440 INFO:tasks.workunit.client.1.smithi069.stdout:7/758: stat d9/db/d48/d4b/f97 0 2024-03-22T07:16:09.443 INFO:tasks.workunit.client.1.smithi069.stdout:7/759: rename d9/db/d1a/db1/d57/c58 to d9/db/d1a/db1/db3/cf8 0 2024-03-22T07:16:09.468 INFO:tasks.workunit.client.1.smithi069.stdout:7/760: write d9/db/d1a/f39 [3215110,90174] 0 2024-03-22T07:16:09.468 INFO:tasks.workunit.client.1.smithi069.stdout:7/761: readlink d9/db/d1a/db1/d57/d44/d53/lcb 0 2024-03-22T07:16:09.491 INFO:tasks.workunit.client.1.smithi069.stdout:0/900: dwrite d3/d7/d15/d29/d5a/d84/f60 [0,4194304] 0 2024-03-22T07:16:09.491 INFO:tasks.workunit.client.1.smithi069.stdout:5/682: dwrite d1/d6/d26/d3e/f8d [0,4194304] 0 2024-03-22T07:16:09.491 INFO:tasks.workunit.client.1.smithi069.stdout:3/898: dwrite d5/d9/d23/f73 [0,4194304] 0 2024-03-22T07:16:09.491 INFO:tasks.workunit.client.1.smithi069.stdout:5/683: fsync d1/d7/f59 0 2024-03-22T07:16:09.492 INFO:tasks.workunit.client.1.smithi069.stdout:4/911: dwrite d6/d16/d83/dd0/f101 [0,4194304] 0 2024-03-22T07:16:09.492 INFO:tasks.workunit.client.0.smithi063.stdout:7/981: dwrite d5/da/d28/d11e/da8/dd4/dea/f67 [0,4194304] 0 2024-03-22T07:16:09.492 INFO:tasks.workunit.client.0.smithi063.stdout:5/956: dwrite d8/d13/d1e/d31/d64/d65/d86/ff2 [0,4194304] 0 2024-03-22T07:16:09.492 INFO:tasks.workunit.client.0.smithi063.stdout:7/982: creat d5/d16/d43/f146 x:0 0 0 2024-03-22T07:16:09.492 INFO:tasks.workunit.client.0.smithi063.stdout:7/983: dread - d5/f10e zero size 2024-03-22T07:16:09.492 INFO:tasks.workunit.client.0.smithi063.stdout:7/984: chown d5/dcf/c12c 40108252 1 2024-03-22T07:16:09.493 INFO:tasks.workunit.client.0.smithi063.stdout:2/951: dwrite d0/de/d35/d36/f4b [0,4194304] 0 2024-03-22T07:16:09.493 INFO:tasks.workunit.client.1.smithi069.stdout:2/632: dwrite d18/d4d/d27/d33/d60/f5a [0,4194304] 0 2024-03-22T07:16:09.493 INFO:tasks.workunit.client.0.smithi063.stdout:4/996: dwrite d1/d5/d3c/d8f/dbb/d114/d79/fb0 [0,4194304] 0 2024-03-22T07:16:09.493 INFO:tasks.workunit.client.0.smithi063.stdout:4/997: write d1/d5/d85/f10a [377777,100974] 0 2024-03-22T07:16:09.493 INFO:tasks.workunit.client.0.smithi063.stdout:4/998: dread - d1/d5/d43/d27/d2c/df4/fff zero size 2024-03-22T07:16:09.494 INFO:tasks.workunit.client.1.smithi069.stdout:7/762: fdatasync d9/db/d1a/f39 0 2024-03-22T07:16:09.495 INFO:tasks.workunit.client.1.smithi069.stdout:9/757: dwrite d2/d51/f64 [0,4194304] 0 2024-03-22T07:16:09.495 INFO:tasks.workunit.client.1.smithi069.stdout:9/758: fsync d2/dd/d3c/d48/f4d 0 2024-03-22T07:16:09.495 INFO:tasks.workunit.client.1.smithi069.stdout:9/759: chown d2/dd/d6e/cbc 145041951 1 2024-03-22T07:16:09.496 INFO:tasks.workunit.client.1.smithi069.stdout:3/899: symlink d5/da2/l12d 0 2024-03-22T07:16:09.496 INFO:tasks.workunit.client.1.smithi069.stdout:3/900: readlink d5/d9/db/dff/ld8 0 2024-03-22T07:16:09.496 INFO:tasks.workunit.client.0.smithi063.stdout:5/957: mkdir d8/d13/d28/df3/d133 0 2024-03-22T07:16:09.498 INFO:tasks.workunit.client.1.smithi069.stdout:5/684: mkdir d1/d6/d86/d99/dec 0 2024-03-22T07:16:09.498 INFO:tasks.workunit.client.1.smithi069.stdout:5/685: write d1/d6/d26/d33/f4e [1378552,34292] 0 2024-03-22T07:16:09.503 INFO:tasks.workunit.client.0.smithi063.stdout:4/999: symlink d1/l15f 0 2024-03-22T07:16:09.504 INFO:tasks.workunit.client.1.smithi069.stdout:4/912: dread d6/d16/d3a/db4/fde [0,4194304] 0 2024-03-22T07:16:09.504 INFO:tasks.workunit.client.1.smithi069.stdout:4/913: readlink d6/d9b/d56/lb1 0 2024-03-22T07:16:09.505 INFO:tasks.workunit.client.1.smithi069.stdout:2/633: creat d18/d4d/d27/d33/fd0 x:0 0 0 2024-03-22T07:16:09.508 INFO:tasks.workunit.client.0.smithi063.stdout:5/958: dread d8/d13/d28/df3/d114/f117 [0,4194304] 0 2024-03-22T07:16:09.509 INFO:tasks.workunit.client.1.smithi069.stdout:9/760: mkdir d2/d51/dbd/de4/df3/df5 0 2024-03-22T07:16:09.509 INFO:tasks.workunit.client.1.smithi069.stdout:9/761: creat d2/ff6 x:0 0 0 2024-03-22T07:16:09.509 INFO:tasks.workunit.client.0.smithi063.stdout:2/952: symlink d0/de/d35/d36/l13a 0 2024-03-22T07:16:09.510 INFO:tasks.workunit.client.1.smithi069.stdout:3/901: creat d5/d9/d23/f12e x:0 0 0 2024-03-22T07:16:09.510 INFO:tasks.workunit.client.1.smithi069.stdout:3/902: fdatasync d5/d9/d26/f5a 0 2024-03-22T07:16:09.510 INFO:tasks.workunit.client.1.smithi069.stdout:3/903: write d5/d9/d26/f6c [2047381,54614] 0 2024-03-22T07:16:09.510 INFO:tasks.workunit.client.1.smithi069.stdout:3/904: read d5/f8f [730001,89787] 0 2024-03-22T07:16:09.511 INFO:tasks.workunit.client.0.smithi063.stdout:5/959: mkdir d8/d13/d28/d57/d134 0 2024-03-22T07:16:09.511 INFO:tasks.workunit.client.0.smithi063.stdout:5/960: chown d8/de9/f118 34 1 2024-03-22T07:16:09.511 INFO:tasks.workunit.client.0.smithi063.stdout:5/961: chown d8/f2b 23055 1 2024-03-22T07:16:09.512 INFO:tasks.workunit.client.0.smithi063.stdout:2/953: getdents d0/de/d110 0 2024-03-22T07:16:09.512 INFO:tasks.workunit.client.0.smithi063.stdout:2/954: fsync d0/de/d35/f44 0 2024-03-22T07:16:09.512 INFO:tasks.workunit.client.0.smithi063.stdout:2/955: truncate d0/de/d1b/d32/dac/fab 679805 0 2024-03-22T07:16:09.512 INFO:tasks.workunit.client.1.smithi069.stdout:2/634: mkdir d18/d4d/d27/d33/d3d/dc6/dd1 0 2024-03-22T07:16:09.512 INFO:tasks.workunit.client.1.smithi069.stdout:2/635: write d18/d4d/d25/f4b [1717762,46262] 0 2024-03-22T07:16:09.517 INFO:tasks.workunit.client.0.smithi063.stdout:2/956: dread d0/de/d1b/d63/f86 [0,4194304] 0 2024-03-22T07:16:09.518 INFO:tasks.workunit.client.1.smithi069.stdout:3/905: symlink d5/d9/d26/d3f/d54/d9d/dde/l12f 0 2024-03-22T07:16:09.519 INFO:tasks.workunit.client.1.smithi069.stdout:5/686: mknod d1/d6/d26/d33/d5d/d72/ced 0 2024-03-22T07:16:09.519 INFO:tasks.workunit.client.1.smithi069.stdout:5/687: stat d1/d10/d44/l84 0 2024-03-22T07:16:09.519 INFO:tasks.workunit.client.1.smithi069.stdout:5/688: chown d1/d6/d21/dae/dea/dba 1313177 1 2024-03-22T07:16:09.519 INFO:tasks.workunit.client.0.smithi063.stdout:2/957: write d0/de/d12/d20/f9b [491778,64550] 0 2024-03-22T07:16:09.519 INFO:tasks.workunit.client.0.smithi063.stdout:2/958: write d0/de/d1b/d63/d6e/f95 [1678806,33743] 0 2024-03-22T07:16:09.519 INFO:tasks.workunit.client.0.smithi063.stdout:2/959: readlink d0/de/d1b/d63/lb2 0 2024-03-22T07:16:09.520 INFO:tasks.workunit.client.1.smithi069.stdout:7/763: rename d9/db/d48/d4b to d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/df9 0 2024-03-22T07:16:09.520 INFO:tasks.workunit.client.0.smithi063.stdout:2/960: creat d0/de/d1b/d8b/f13b x:0 0 0 2024-03-22T07:16:09.521 INFO:tasks.workunit.client.0.smithi063.stdout:5/962: dread d8/f12 [0,4194304] 0 2024-03-22T07:16:09.522 INFO:tasks.workunit.client.1.smithi069.stdout:3/906: mkdir d5/d9/d23/d130 0 2024-03-22T07:16:09.522 INFO:tasks.workunit.client.1.smithi069.stdout:3/907: fsync d5/d9/db/f30 0 2024-03-22T07:16:09.525 INFO:tasks.workunit.client.1.smithi069.stdout:5/689: link d1/d6/d21/dae/ccc d1/d6/d86/dab/dcf/cee 0 2024-03-22T07:16:09.529 INFO:tasks.workunit.client.1.smithi069.stdout:4/914: rename d6/d12/d18/d21/ce3 to d6/dc4/c13a 0 2024-03-22T07:16:09.530 INFO:tasks.workunit.client.1.smithi069.stdout:4/915: chown d6/d16/c115 456591439 1 2024-03-22T07:16:09.530 INFO:tasks.workunit.client.1.smithi069.stdout:4/916: creat d6/d12/f13b x:0 0 0 2024-03-22T07:16:09.530 INFO:tasks.workunit.client.1.smithi069.stdout:4/917: chown d6/d16/d3a/db4/dfb/d23/d59/d8f 11086 1 2024-03-22T07:16:09.530 INFO:tasks.workunit.client.1.smithi069.stdout:3/908: creat d5/da2/ddd/d109/f131 x:0 0 0 2024-03-22T07:16:09.530 INFO:tasks.workunit.client.0.smithi063.stdout:5/963: write d8/d13/d24/f3b [256700,108344] 0 2024-03-22T07:16:09.531 INFO:tasks.workunit.client.1.smithi069.stdout:9/762: read d2/d51/f7a [1893665,3350] 0 2024-03-22T07:16:09.532 INFO:tasks.workunit.client.1.smithi069.stdout:3/909: unlink d5/d9/d26/d3f/d56/f106 0 2024-03-22T07:16:09.535 INFO:tasks.workunit.client.1.smithi069.stdout:5/690: getdents d1/d10/d44/d50 0 2024-03-22T07:16:09.535 INFO:tasks.workunit.client.1.smithi069.stdout:9/763: symlink d2/d21/d89/lf7 0 2024-03-22T07:16:09.538 INFO:tasks.workunit.client.1.smithi069.stdout:3/910: symlink d5/da2/l132 0 2024-03-22T07:16:09.538 INFO:tasks.workunit.client.1.smithi069.stdout:3/911: creat d5/d9/d26/d3f/d56/d8a/da5/d7b/dbc/f133 x:0 0 0 2024-03-22T07:16:09.538 INFO:tasks.workunit.client.1.smithi069.stdout:3/912: chown d5/d9/d26/d3f/d56/d8a/ff5 11978245 1 2024-03-22T07:16:09.539 INFO:tasks.workunit.client.1.smithi069.stdout:5/691: mknod d1/d10/d44/d50/dd5/cef 0 2024-03-22T07:16:09.539 INFO:tasks.workunit.client.1.smithi069.stdout:5/692: write d1/d28/fb8 [536560,111331] 0 2024-03-22T07:16:09.540 INFO:tasks.workunit.client.1.smithi069.stdout:4/918: dread d6/d16/f108 [0,4194304] 0 2024-03-22T07:16:09.540 INFO:tasks.workunit.client.1.smithi069.stdout:4/919: stat d6/daa/dce/de4/ff9 0 2024-03-22T07:16:09.545 INFO:tasks.workunit.client.0.smithi063.stdout:5/964: dread d8/d13/d28/f43 [0,4194304] 0 2024-03-22T07:16:09.545 INFO:tasks.workunit.client.0.smithi063.stdout:5/965: chown d8/d13/d1e/d31/c89 20 1 2024-03-22T07:16:09.548 INFO:tasks.workunit.client.1.smithi069.stdout:7/764: dwrite d9/db/d1a/db1/fe8 [0,4194304] 0 2024-03-22T07:16:09.548 INFO:tasks.workunit.client.0.smithi063.stdout:2/961: dwrite d0/de/d1b/d63/f86 [0,4194304] 0 2024-03-22T07:16:09.548 INFO:tasks.workunit.client.0.smithi063.stdout:2/962: dread - d0/de/d12/d20/f111 zero size 2024-03-22T07:16:09.549 INFO:tasks.workunit.client.0.smithi063.stdout:5/966: creat d8/d13/d28/d57/dfd/f135 x:0 0 0 2024-03-22T07:16:09.550 INFO:tasks.workunit.client.1.smithi069.stdout:9/764: dread d2/dd/f69 [0,4194304] 0 2024-03-22T07:16:09.551 INFO:tasks.workunit.client.1.smithi069.stdout:9/765: mknod d2/d31/dc6/cf8 0 2024-03-22T07:16:09.552 INFO:tasks.workunit.client.1.smithi069.stdout:9/766: read - d2/d31/fc4 zero size 2024-03-22T07:16:09.552 INFO:tasks.workunit.client.0.smithi063.stdout:2/963: mknod d0/de/d74/dca/c13c 0 2024-03-22T07:16:09.552 INFO:tasks.workunit.client.0.smithi063.stdout:5/967: mknod d8/d13/d1e/d3c/da0/c136 0 2024-03-22T07:16:09.552 INFO:tasks.workunit.client.0.smithi063.stdout:5/968: creat d8/d13/f137 x:0 0 0 2024-03-22T07:16:09.553 INFO:tasks.workunit.client.1.smithi069.stdout:9/767: creat d2/d21/dd7/ff9 x:0 0 0 2024-03-22T07:16:09.553 INFO:tasks.workunit.client.1.smithi069.stdout:9/768: truncate d2/d51/f59 1362511 0 2024-03-22T07:16:09.553 INFO:tasks.workunit.client.1.smithi069.stdout:9/769: read d2/dd/f69 [1533669,80932] 0 2024-03-22T07:16:09.553 INFO:tasks.workunit.client.1.smithi069.stdout:9/770: mkdir d2/d71/d90/dfa 0 2024-03-22T07:16:09.553 INFO:tasks.workunit.client.1.smithi069.stdout:9/771: truncate d2/d83/f92 5193250 0 2024-03-22T07:16:09.554 INFO:tasks.workunit.client.1.smithi069.stdout:9/772: getdents d2/dd/d3c/d48/d7c 0 2024-03-22T07:16:09.554 INFO:tasks.workunit.client.1.smithi069.stdout:9/773: fdatasync d2/d51/fc2 0 2024-03-22T07:16:09.554 INFO:tasks.workunit.client.1.smithi069.stdout:9/774: write d2/dd/d3c/d2e/f85 [399217,93823] 0 2024-03-22T07:16:09.555 INFO:tasks.workunit.client.0.smithi063.stdout:5/969: mknod d8/d13/c138 0 2024-03-22T07:16:09.555 INFO:tasks.workunit.client.1.smithi069.stdout:9/775: link d2/dd/d3c/d53/c70 d2/d21/d77/cfb 0 2024-03-22T07:16:09.560 INFO:tasks.workunit.client.1.smithi069.stdout:4/920: dwrite d6/d4b/f4d [0,4194304] 0 2024-03-22T07:16:09.560 INFO:tasks.workunit.client.1.smithi069.stdout:4/921: chown d6/d9b/d56/fa8 395965 1 2024-03-22T07:16:09.560 INFO:tasks.workunit.client.1.smithi069.stdout:4/922: chown d6/d9b/d1f/d58/f69 6775612 1 2024-03-22T07:16:09.563 INFO:tasks.workunit.client.1.smithi069.stdout:7/765: dread d9/db/d1a/db1/d57/d44/f9b [0,4194304] 0 2024-03-22T07:16:09.564 INFO:tasks.workunit.client.1.smithi069.stdout:7/766: mknod d9/db/d48/dab/cfa 0 2024-03-22T07:16:09.569 INFO:tasks.workunit.client.1.smithi069.stdout:4/923: dread d6/d4b/f4d [0,4194304] 0 2024-03-22T07:16:09.570 INFO:tasks.workunit.client.1.smithi069.stdout:4/924: symlink d6/d9b/d1f/d3f/l13c 0 2024-03-22T07:16:09.571 INFO:tasks.workunit.client.1.smithi069.stdout:4/925: truncate d6/daa/fe8 1892767 0 2024-03-22T07:16:09.571 INFO:tasks.workunit.client.1.smithi069.stdout:4/926: chown d6/d16/d3a/db4/dbc/fd4 18765832 1 2024-03-22T07:16:09.573 INFO:tasks.workunit.client.1.smithi069.stdout:7/767: dread d9/db/d1a/db1/d57/d44/d53/d74/ff1 [0,4194304] 0 2024-03-22T07:16:09.575 INFO:tasks.workunit.client.1.smithi069.stdout:4/927: symlink d6/d16/d3a/db4/dfb/d23/d59/l13d 0 2024-03-22T07:16:09.576 INFO:tasks.workunit.client.1.smithi069.stdout:7/768: mknod d9/db/d1a/db1/d57/cfb 0 2024-03-22T07:16:09.576 INFO:tasks.workunit.client.0.smithi063.stdout:2/964: dwrite d0/f23 [0,4194304] 0 2024-03-22T07:16:09.576 INFO:tasks.workunit.client.0.smithi063.stdout:2/965: chown d0/de/d74/dca/d11e/d7f/d97/fa1 473 1 2024-03-22T07:16:09.577 INFO:tasks.workunit.client.0.smithi063.stdout:2/966: chown d0/de/d1b/d63/dc6 1015154203 1 2024-03-22T07:16:09.580 INFO:tasks.workunit.client.0.smithi063.stdout:5/970: dwrite d8/d13/d1e/d31/d64/d65/f12e [0,4194304] 0 2024-03-22T07:16:09.580 INFO:tasks.workunit.client.0.smithi063.stdout:5/971: read d8/d13/d28/d57/f5c [4007591,55113] 0 2024-03-22T07:16:09.580 INFO:tasks.workunit.client.0.smithi063.stdout:5/972: chown d8/d13/d24/da4/d8b/db6/dea/d100 1344424 1 2024-03-22T07:16:09.580 INFO:tasks.workunit.client.0.smithi063.stdout:5/973: truncate d8/d5e/f125 762650 0 2024-03-22T07:16:09.580 INFO:tasks.workunit.client.0.smithi063.stdout:2/967: rename d0/de/d35/d36/fc4 to d0/de/d12/d5a/f13d 0 2024-03-22T07:16:09.586 INFO:tasks.workunit.client.0.smithi063.stdout:2/968: dread d0/de/d12/d5a/f101 [0,4194304] 0 2024-03-22T07:16:09.588 INFO:tasks.workunit.client.0.smithi063.stdout:5/974: rename d8/de9/d7e/cf5 to d8/d13/d28/d57/d5f/c139 0 2024-03-22T07:16:09.589 INFO:tasks.workunit.client.1.smithi069.stdout:4/928: dread d6/d16/d3a/db4/fff [0,4194304] 0 2024-03-22T07:16:09.590 INFO:tasks.workunit.client.1.smithi069.stdout:4/929: stat d6/d16/d3a/db4/dbc/l114 0 2024-03-22T07:16:09.592 INFO:tasks.workunit.client.1.smithi069.stdout:4/930: mkdir d6/d16/d68/df8/d11a/d13e 0 2024-03-22T07:16:09.595 INFO:tasks.workunit.client.0.smithi063.stdout:7/985: sync 2024-03-22T07:16:09.596 INFO:tasks.workunit.client.1.smithi069.stdout:7/769: dwrite d9/db/d1a/db1/d57/d44/d53/d74/f91 [0,4194304] 0 2024-03-22T07:16:09.597 INFO:tasks.workunit.client.0.smithi063.stdout:2/969: read d0/de/d1b/d32/dac/faf [2772369,1332] 0 2024-03-22T07:16:09.597 INFO:tasks.workunit.client.0.smithi063.stdout:2/970: stat d0/de/d74/dca/d11e/d7f/d97/da2/dfa/l12c 0 2024-03-22T07:16:09.597 INFO:tasks.workunit.client.0.smithi063.stdout:7/986: unlink d5/da/d28/da7/l3d 0 2024-03-22T07:16:09.597 INFO:tasks.workunit.client.0.smithi063.stdout:7/987: readlink d5/da/d28/d11e/da8/lae 0 2024-03-22T07:16:09.598 INFO:tasks.workunit.client.0.smithi063.stdout:7/988: mkdir d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d29/d4a/d147 0 2024-03-22T07:16:09.599 INFO:tasks.workunit.client.0.smithi063.stdout:2/971: dread d0/de/d12/d5a/fa6 [0,4194304] 0 2024-03-22T07:16:09.600 INFO:tasks.workunit.client.1.smithi069.stdout:7/770: link d9/db/d1a/db1/d57/d44/fc1 d9/db/d1a/db1/d57/d44/d53/d74/d30/ffc 0 2024-03-22T07:16:09.600 INFO:tasks.workunit.client.1.smithi069.stdout:7/771: fsync d9/db/f17 0 2024-03-22T07:16:09.600 INFO:tasks.workunit.client.1.smithi069.stdout:7/772: dread - d9/db/d1a/db1/d57/d44/fc1 zero size 2024-03-22T07:16:09.600 INFO:tasks.workunit.client.1.smithi069.stdout:7/773: chown d9/db/d1a/db1/d57/d44/d53/d1d/fdf 134 1 2024-03-22T07:16:09.600 INFO:tasks.workunit.client.1.smithi069.stdout:7/774: stat d9/fd7 0 2024-03-22T07:16:09.600 INFO:tasks.workunit.client.1.smithi069.stdout:7/775: readlink d9/db/d1a/db1/d57/d44/d53/d74/l90 0 2024-03-22T07:16:09.601 INFO:tasks.workunit.client.1.smithi069.stdout:7/776: stat d9/db/d1a/db1/d57/d44/d53/ld4 0 2024-03-22T07:16:09.601 INFO:tasks.workunit.client.1.smithi069.stdout:7/777: write d9/db/d1a/db1/d57/d44/fac [1830156,32972] 0 2024-03-22T07:16:09.601 INFO:tasks.workunit.client.1.smithi069.stdout:7/778: creat d9/db/d1a/db1/ffd x:0 0 0 2024-03-22T07:16:09.601 INFO:tasks.workunit.client.0.smithi063.stdout:7/989: mkdir d5/da/d28/d11e/d148 0 2024-03-22T07:16:09.601 INFO:tasks.workunit.client.0.smithi063.stdout:7/990: write d5/d16/fef [662856,110682] 0 2024-03-22T07:16:09.602 INFO:tasks.workunit.client.0.smithi063.stdout:7/991: write d5/d16/d43/f146 [504923,73994] 0 2024-03-22T07:16:09.602 INFO:tasks.workunit.client.0.smithi063.stdout:2/972: symlink d0/de/d74/dca/d11e/dbb/l13e 0 2024-03-22T07:16:09.602 INFO:tasks.workunit.client.1.smithi069.stdout:7/779: truncate d9/db/d1a/db1/db3/fb8 359967 0 2024-03-22T07:16:09.603 INFO:tasks.workunit.client.0.smithi063.stdout:2/973: mkdir d0/de/d1b/d63/d6e/d13f 0 2024-03-22T07:16:09.603 INFO:tasks.workunit.client.0.smithi063.stdout:2/974: write d0/de/d110/f10e [581911,128262] 0 2024-03-22T07:16:09.607 INFO:tasks.workunit.client.0.smithi063.stdout:2/975: dread d0/de/d12/d20/f47 [0,4194304] 0 2024-03-22T07:16:09.611 INFO:tasks.workunit.client.0.smithi063.stdout:2/976: link d0/de/d1b/d32/dac/f10b d0/de/d35/f140 0 2024-03-22T07:16:09.611 INFO:tasks.workunit.client.0.smithi063.stdout:2/977: creat d0/de/d1b/d63/f141 x:0 0 0 2024-03-22T07:16:09.611 INFO:tasks.workunit.client.0.smithi063.stdout:2/978: readlink d0/l16 0 2024-03-22T07:16:09.611 INFO:tasks.workunit.client.0.smithi063.stdout:2/979: chown d0/de/d74/fee 15 1 2024-03-22T07:16:09.611 INFO:tasks.workunit.client.0.smithi063.stdout:2/980: dread - d0/de/d1b/d63/f141 zero size 2024-03-22T07:16:09.613 INFO:tasks.workunit.client.1.smithi069.stdout:4/931: dwrite d6/d12/f41 [0,4194304] 0 2024-03-22T07:16:09.614 INFO:tasks.workunit.client.0.smithi063.stdout:2/981: read d0/de/d74/dca/fd2 [981713,78585] 0 2024-03-22T07:16:09.616 INFO:tasks.workunit.client.0.smithi063.stdout:2/982: rename d0/de/d74/dca/cce to d0/de/d1b/d63/dd6/c142 0 2024-03-22T07:16:09.618 INFO:tasks.workunit.client.0.smithi063.stdout:2/983: mknod d0/de/d1b/d32/dac/c143 0 2024-03-22T07:16:09.618 INFO:tasks.workunit.client.0.smithi063.stdout:2/984: write d0/de/d74/d121/f135 [304422,122208] 0 2024-03-22T07:16:09.632 INFO:tasks.workunit.client.0.smithi063.stdout:5/975: dwrite d8/d13/d1e/d3c/da0/f109 [0,4194304] 0 2024-03-22T07:16:09.633 INFO:tasks.workunit.client.0.smithi063.stdout:5/976: rmdir d8/d13/d24/da4/d8b/db6/dfb 39 2024-03-22T07:16:09.633 INFO:tasks.workunit.client.0.smithi063.stdout:5/977: readlink d8/d13/d1e/d3c/da0/lc7 0 2024-03-22T07:16:09.633 INFO:tasks.workunit.client.0.smithi063.stdout:5/978: fsync d8/d13/fdb 0 2024-03-22T07:16:09.637 INFO:tasks.workunit.client.0.smithi063.stdout:7/992: dwrite d5/da/d28/d11e/da8/dbc/d62/d8d/f11b [0,4194304] 0 2024-03-22T07:16:09.639 INFO:tasks.workunit.client.1.smithi069.stdout:7/780: dwrite d9/db/d1a/f21 [0,4194304] 0 2024-03-22T07:16:09.640 INFO:tasks.workunit.client.1.smithi069.stdout:7/781: dread - d9/db/d1a/db1/d57/d44/d53/ff0 zero size 2024-03-22T07:16:09.640 INFO:tasks.workunit.client.1.smithi069.stdout:4/932: dwrite d6/dc4/f106 [0,4194304] 0 2024-03-22T07:16:09.640 INFO:tasks.workunit.client.0.smithi063.stdout:7/993: creat d5/da/d28/d11e/da8/dd4/d132/f149 x:0 0 0 2024-03-22T07:16:09.641 INFO:tasks.workunit.client.0.smithi063.stdout:2/985: dwrite d0/de/d12/f1f [0,4194304] 0 2024-03-22T07:16:09.641 INFO:tasks.workunit.client.0.smithi063.stdout:2/986: truncate d0/de/d1b/d8b/f13b 60189 0 2024-03-22T07:16:09.641 INFO:tasks.workunit.client.0.smithi063.stdout:2/987: fsync d0/de/d12/d20/f2a 0 2024-03-22T07:16:09.642 INFO:tasks.workunit.client.0.smithi063.stdout:7/994: creat d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d29/f14a x:0 0 0 2024-03-22T07:16:09.643 INFO:tasks.workunit.client.1.smithi069.stdout:4/933: getdents d6/d9b/d1f/d3f/d51/db8 0 2024-03-22T07:16:09.645 INFO:tasks.workunit.client.1.smithi069.stdout:4/934: rename d6/d9b/d1f/d3f/f9d to d6/d9b/d1f/d3f/d51/f13f 0 2024-03-22T07:16:09.647 INFO:tasks.workunit.client.1.smithi069.stdout:4/935: link d6/d4b/de9/lee d6/d16/d3a/db4/dfb/d23/d59/d120/d12e/l140 0 2024-03-22T07:16:09.648 INFO:tasks.workunit.client.0.smithi063.stdout:2/988: mkdir d0/de/d12/d20/de9/d144 0 2024-03-22T07:16:09.649 INFO:tasks.workunit.client.1.smithi069.stdout:7/782: write d9/db/d1a/db1/d57/d44/d53/d2e/f6a [4934746,130274] 0 2024-03-22T07:16:09.649 INFO:tasks.workunit.client.1.smithi069.stdout:7/783: getdents d9/d42/da8 0 2024-03-22T07:16:09.649 INFO:tasks.workunit.client.1.smithi069.stdout:4/936: truncate d6/d9b/d1f/f39 190096 0 2024-03-22T07:16:09.649 INFO:tasks.workunit.client.1.smithi069.stdout:4/937: dread - d6/d16/d3a/db4/dfb/d1d/d55/f135 zero size 2024-03-22T07:16:09.650 INFO:tasks.workunit.client.1.smithi069.stdout:4/938: truncate d6/d16/d3a/db4/fd3 993691 0 2024-03-22T07:16:09.650 INFO:tasks.workunit.client.0.smithi063.stdout:2/989: symlink d0/de/d1b/d8b/l145 0 2024-03-22T07:16:09.652 INFO:tasks.workunit.client.0.smithi063.stdout:2/990: creat d0/de/d74/dca/d11e/d7f/d97/da2/d12e/f146 x:0 0 0 2024-03-22T07:16:09.652 INFO:tasks.workunit.client.1.smithi069.stdout:4/939: mknod d6/d16/d83/d9a/c141 0 2024-03-22T07:16:09.652 INFO:tasks.workunit.client.1.smithi069.stdout:4/940: chown d6/d4b/la0 158 1 2024-03-22T07:16:09.662 INFO:tasks.workunit.client.1.smithi069.stdout:7/784: write d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/df9/faf [1386476,29409] 0 2024-03-22T07:16:09.662 INFO:tasks.workunit.client.1.smithi069.stdout:7/785: read - d9/db/d1a/f82 zero size 2024-03-22T07:16:09.663 INFO:tasks.workunit.client.1.smithi069.stdout:7/786: dread - d9/db/d1a/db1/fef zero size 2024-03-22T07:16:09.663 INFO:tasks.workunit.client.1.smithi069.stdout:7/787: dread - d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/df9/f97 zero size 2024-03-22T07:16:09.666 INFO:tasks.workunit.client.1.smithi069.stdout:7/788: truncate d9/f6f 4291766 0 2024-03-22T07:16:09.668 INFO:tasks.workunit.client.1.smithi069.stdout:7/789: mkdir d9/db/d1a/db1/d57/d44/d53/dc7/dfe 0 2024-03-22T07:16:09.668 INFO:tasks.workunit.client.1.smithi069.stdout:7/790: chown d9/db/d1a/db1/d57/d75/d7a/lda 0 1 2024-03-22T07:16:09.669 INFO:tasks.workunit.client.0.smithi063.stdout:7/995: dwrite d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d29/f47 [0,4194304] 0 2024-03-22T07:16:09.671 INFO:tasks.workunit.client.0.smithi063.stdout:7/996: mknod d5/d16/c14b 0 2024-03-22T07:16:09.671 INFO:tasks.workunit.client.0.smithi063.stdout:7/997: readlink d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d29/l79 0 2024-03-22T07:16:09.672 INFO:tasks.workunit.client.0.smithi063.stdout:7/998: link d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d29/lde d5/d16/d43/d78/l14c 0 2024-03-22T07:16:09.673 INFO:tasks.workunit.client.0.smithi063.stdout:2/991: dwrite d0/de/d74/dca/d11e/f10f [0,4194304] 0 2024-03-22T07:16:09.673 INFO:tasks.workunit.client.0.smithi063.stdout:7/999: creat d5/da/d28/d11e/da8/dbc/d62/d73/d1d/d29/df0/f14d x:0 0 0 2024-03-22T07:16:09.680 INFO:tasks.workunit.client.0.smithi063.stdout:2/992: mknod d0/de/d12/d5a/c147 0 2024-03-22T07:16:09.688 INFO:tasks.workunit.client.1.smithi069.stdout:7/791: dwrite d9/db/d1a/f39 [0,4194304] 0 2024-03-22T07:16:09.688 INFO:tasks.workunit.client.1.smithi069.stdout:7/792: fsync d9/db/d1a/db1/d57/d44/f9b 0 2024-03-22T07:16:09.689 INFO:tasks.workunit.client.1.smithi069.stdout:7/793: chown d9/db/d1a/db1/d57/d44/d53/d2e 7923040 1 2024-03-22T07:16:09.697 INFO:tasks.workunit.client.1.smithi069.stdout:2/636: sync 2024-03-22T07:16:09.697 INFO:tasks.workunit.client.1.smithi069.stdout:1/756: sync 2024-03-22T07:16:09.697 INFO:tasks.workunit.client.1.smithi069.stdout:1/757: fdatasync d1/d13/d36/de5/d4c/d55/fea 0 2024-03-22T07:16:09.697 INFO:tasks.workunit.client.1.smithi069.stdout:1/758: chown d1/d2a/d84/dd1/ddb 52659 1 2024-03-22T07:16:09.697 INFO:tasks.workunit.client.1.smithi069.stdout:6/704: sync 2024-03-22T07:16:09.697 INFO:tasks.workunit.client.1.smithi069.stdout:0/901: sync 2024-03-22T07:16:09.698 INFO:tasks.workunit.client.1.smithi069.stdout:2/637: chown d18/d4d/d27/c3c 1 1 2024-03-22T07:16:09.698 INFO:tasks.workunit.client.1.smithi069.stdout:2/638: readlink d18/d4d/d27/d33/d4f/l87 0 2024-03-22T07:16:09.698 INFO:tasks.workunit.client.1.smithi069.stdout:2/639: chown d18/d4d/d27/d33/d60 1334 1 2024-03-22T07:16:09.699 INFO:tasks.workunit.client.1.smithi069.stdout:1/759: creat d1/d2a/d84/d9a/fed x:0 0 0 2024-03-22T07:16:09.699 INFO:tasks.workunit.client.1.smithi069.stdout:6/705: symlink d4/d7/lde 0 2024-03-22T07:16:09.700 INFO:tasks.workunit.client.1.smithi069.stdout:0/902: mknod d3/d7/d15/d29/d5a/d84/df4/c12b 0 2024-03-22T07:16:09.700 INFO:tasks.workunit.client.1.smithi069.stdout:0/903: creat d3/d7/f12c x:0 0 0 2024-03-22T07:16:09.701 INFO:tasks.workunit.client.1.smithi069.stdout:2/640: unlink d18/l2f 0 2024-03-22T07:16:09.702 INFO:tasks.workunit.client.1.smithi069.stdout:1/760: getdents d1/d13 0 2024-03-22T07:16:09.704 INFO:tasks.workunit.client.1.smithi069.stdout:6/706: unlink d4/dd/d41/f93 0 2024-03-22T07:16:09.707 INFO:tasks.workunit.client.1.smithi069.stdout:0/904: mkdir d3/d12d 0 2024-03-22T07:16:09.707 INFO:tasks.workunit.client.1.smithi069.stdout:1/761: dread d1/d13/f39 [0,4194304] 0 2024-03-22T07:16:09.709 INFO:tasks.workunit.client.1.smithi069.stdout:0/905: symlink d3/d7/d17/d1c/l12e 0 2024-03-22T07:16:09.709 INFO:tasks.workunit.client.1.smithi069.stdout:0/906: write d3/d7/d17/f83 [548650,123751] 0 2024-03-22T07:16:09.709 INFO:tasks.workunit.client.1.smithi069.stdout:0/907: chown d3/d7/d17/d1c/f91 518137 1 2024-03-22T07:16:09.709 INFO:tasks.workunit.client.1.smithi069.stdout:1/762: mknod d1/d13/d36/de5/d88/d41/d1e/cee 0 2024-03-22T07:16:09.710 INFO:tasks.workunit.client.0.smithi063.stdout:2/993: dwrite d0/de/d12/ffd [0,4194304] 0 2024-03-22T07:16:09.710 INFO:tasks.workunit.client.0.smithi063.stdout:2/994: dread - d0/f124 zero size 2024-03-22T07:16:09.711 INFO:tasks.workunit.client.1.smithi069.stdout:1/763: symlink d1/d13/d82/dc2/lef 0 2024-03-22T07:16:09.713 INFO:tasks.workunit.client.0.smithi063.stdout:2/995: rename d0/de/d1b/d63/dd6/l102 to d0/de/d74/dca/d11e/d7f/d97/da2/dfa/l148 0 2024-03-22T07:16:09.713 INFO:tasks.workunit.client.0.smithi063.stdout:2/996: creat d0/de/d74/f149 x:0 0 0 2024-03-22T07:16:09.715 INFO:tasks.workunit.client.0.smithi063.stdout:2/997: creat d0/de/d74/dca/d11e/d7f/d97/da2/d104/f14a x:0 0 0 2024-03-22T07:16:09.717 INFO:tasks.workunit.client.0.smithi063.stdout:2/998: mkdir d0/de/d74/dca/d11e/d7f/d97/d14b 0 2024-03-22T07:16:09.717 INFO:tasks.workunit.client.0.smithi063.stdout:2/999: read - d0/de/d12/d20/f130 zero size 2024-03-22T07:16:09.725 INFO:tasks.workunit.client.1.smithi069.stdout:1/764: dread d1/f3b [0,4194304] 0 2024-03-22T07:16:09.726 INFO:tasks.workunit.client.1.smithi069.stdout:2/641: dwrite d18/d91/f98 [0,4194304] 0 2024-03-22T07:16:09.726 INFO:tasks.workunit.client.1.smithi069.stdout:2/642: chown d18/d4d/d3e/d40 973826662 1 2024-03-22T07:16:09.726 INFO:tasks.workunit.client.1.smithi069.stdout:2/643: chown d18/d4d/f26 1 1 2024-03-22T07:16:09.731 INFO:tasks.workunit.client.1.smithi069.stdout:6/707: dwrite d4/dd/d41/d78/f9c [0,4194304] 0 2024-03-22T07:16:09.731 INFO:tasks.workunit.client.1.smithi069.stdout:6/708: creat d4/dd/d12/d44/d58/db4/fdf x:0 0 0 2024-03-22T07:16:09.731 INFO:tasks.workunit.client.1.smithi069.stdout:2/644: dread d18/d4d/d27/d33/d60/d63/d80/fc3 [0,4194304] 0 2024-03-22T07:16:09.732 INFO:tasks.workunit.client.1.smithi069.stdout:1/765: rename d1/d13/d36/de5/d88/dae/db5 to d1/d13/d36/d6e/d73/df0 0 2024-03-22T07:16:09.732 INFO:tasks.workunit.client.1.smithi069.stdout:2/645: write d18/d4d/d34/f7e [2221630,69465] 0 2024-03-22T07:16:09.737 INFO:tasks.workunit.client.1.smithi069.stdout:6/709: mknod d4/d7/da/d3f/d76/dab/ce0 0 2024-03-22T07:16:09.737 INFO:tasks.workunit.client.1.smithi069.stdout:6/710: read - d4/dd/dbd/fda zero size 2024-03-22T07:16:09.738 INFO:tasks.workunit.client.1.smithi069.stdout:1/766: mknod d1/d13/d36/d5b/d95/cf1 0 2024-03-22T07:16:09.738 INFO:tasks.workunit.client.1.smithi069.stdout:1/767: write d1/d13/d36/de5/d88/d41/d1e/fda [574481,13501] 0 2024-03-22T07:16:09.743 INFO:tasks.workunit.client.1.smithi069.stdout:1/768: mknod d1/d2a/d84/dd1/cf2 0 2024-03-22T07:16:09.747 INFO:tasks.workunit.client.1.smithi069.stdout:2/646: dwrite d18/d4d/d27/d33/d60/d63/d73/fb4 [0,4194304] 0 2024-03-22T07:16:09.747 INFO:tasks.workunit.client.1.smithi069.stdout:6/711: getdents d4/dd/d12/d2b/d72/d73/d9d 0 2024-03-22T07:16:09.751 INFO:tasks.workunit.client.1.smithi069.stdout:1/769: mknod d1/d13/d36/de5/d4c/cf3 0 2024-03-22T07:16:09.757 INFO:tasks.workunit.client.1.smithi069.stdout:2/647: rename d18/d4d/d27/d33/d60/d64/l67 to d18/ld2 0 2024-03-22T07:16:09.759 INFO:tasks.workunit.client.1.smithi069.stdout:2/648: rmdir d18/d4d/d27/d33/d60/d64 39 2024-03-22T07:16:09.760 INFO:tasks.workunit.client.0.smithi063.stdout:5/979: sync 2024-03-22T07:16:09.760 INFO:tasks.workunit.client.1.smithi069.stdout:1/770: write d1/d2a/f32 [1961040,32621] 0 2024-03-22T07:16:09.761 INFO:tasks.workunit.client.0.smithi063.stdout:5/980: symlink d8/d13/d28/l13a 0 2024-03-22T07:16:09.764 INFO:tasks.workunit.client.0.smithi063.stdout:5/981: creat d8/d13/d24/da4/d8b/db6/dea/f13b x:0 0 0 2024-03-22T07:16:09.764 INFO:tasks.workunit.client.0.smithi063.stdout:5/982: chown d8/d13/d28/d57/d5f/d12f/dd7/f120 19553346 1 2024-03-22T07:16:09.764 INFO:tasks.workunit.client.1.smithi069.stdout:1/771: rmdir d1/d13/dd6 0 2024-03-22T07:16:09.764 INFO:tasks.workunit.client.0.smithi063.stdout:5/983: creat d8/d13/d24/da4/f13c x:0 0 0 2024-03-22T07:16:09.766 INFO:tasks.workunit.client.1.smithi069.stdout:1/772: symlink d1/d13/d36/de5/d88/d41/d64/lf4 0 2024-03-22T07:16:09.766 INFO:tasks.workunit.client.1.smithi069.stdout:1/773: dread - d1/d13/d36/de5/d88/d41/d64/fc3 zero size 2024-03-22T07:16:09.766 INFO:tasks.workunit.client.1.smithi069.stdout:1/774: readlink d1/d13/d36/de5/d88/d41/le9 0 2024-03-22T07:16:09.767 INFO:tasks.workunit.client.0.smithi063.stdout:5/984: link d8/d13/d1e/l97 d8/d13/d1e/d31/d64/de7/dfa/l13d 0 2024-03-22T07:16:09.768 INFO:tasks.workunit.client.1.smithi069.stdout:1/775: link d1/l2c d1/d13/d82/dc2/lf5 0 2024-03-22T07:16:09.769 INFO:tasks.workunit.client.1.smithi069.stdout:1/776: truncate d1/d2a/d84/f8b 358929 0 2024-03-22T07:16:09.770 INFO:tasks.workunit.client.1.smithi069.stdout:1/777: dread - d1/d13/d36/de5/d4c/f74 zero size 2024-03-22T07:16:09.770 INFO:tasks.workunit.client.1.smithi069.stdout:1/778: chown d1/d13/d36/de5/c6a 12364184 1 2024-03-22T07:16:09.777 INFO:tasks.workunit.client.0.smithi063.stdout:5/985: write d8/d13/d1e/d3c/f4c [1340043,6088] 0 2024-03-22T07:16:09.777 INFO:tasks.workunit.client.1.smithi069.stdout:2/649: dwrite d18/f1e [0,4194304] 0 2024-03-22T07:16:09.779 INFO:tasks.workunit.client.0.smithi063.stdout:5/986: unlink d8/d13/d1e/l29 0 2024-03-22T07:16:09.779 INFO:tasks.workunit.client.0.smithi063.stdout:5/987: symlink d8/d13/d24/l13e 0 2024-03-22T07:16:09.781 INFO:tasks.workunit.client.0.smithi063.stdout:5/988: rename d8/d13/d28/d57/dfd/d108/l124 to d8/d13/d1e/l13f 0 2024-03-22T07:16:09.782 INFO:tasks.workunit.client.0.smithi063.stdout:5/989: creat d8/d13/d28/d7d/f140 x:0 0 0 2024-03-22T07:16:09.782 INFO:tasks.workunit.client.0.smithi063.stdout:5/990: chown d8/d13/d1e/d31/daf/fcd 5372 1 2024-03-22T07:16:09.782 INFO:tasks.workunit.client.0.smithi063.stdout:5/991: fdatasync d8/d13/d24/da4/d8b/db6/fe2 0 2024-03-22T07:16:09.786 INFO:tasks.workunit.client.0.smithi063.stdout:5/992: truncate d8/d13/d1e/d3c/f55 1321803 0 2024-03-22T07:16:09.786 INFO:tasks.workunit.client.0.smithi063.stdout:5/993: stat d8/de9/d7e/d9e 0 2024-03-22T07:16:09.786 INFO:tasks.workunit.client.0.smithi063.stdout:5/994: readlink d8/d13/d1e/d31/d64/d65/d86/lc1 0 2024-03-22T07:16:09.786 INFO:tasks.workunit.client.0.smithi063.stdout:5/995: write d8/d13/d1e/f7b [674835,85131] 0 2024-03-22T07:16:09.786 INFO:tasks.workunit.client.0.smithi063.stdout:5/996: write d8/d13/f18 [2806303,29332] 0 2024-03-22T07:16:09.787 INFO:tasks.workunit.client.0.smithi063.stdout:5/997: rename d8/d13/d24/ce3 to d8/d13/d1e/d31/d64/d65/ddf/c141 0 2024-03-22T07:16:09.788 INFO:tasks.workunit.client.0.smithi063.stdout:5/998: creat d8/d13/d28/d7d/d127/f142 x:0 0 0 2024-03-22T07:16:09.789 INFO:tasks.workunit.client.1.smithi069.stdout:1/779: dread d1/f22 [0,4194304] 0 2024-03-22T07:16:09.789 INFO:tasks.workunit.client.1.smithi069.stdout:1/780: chown d1/d13/d36/de5/d88/d41/ld9 32449 1 2024-03-22T07:16:09.789 INFO:tasks.workunit.client.1.smithi069.stdout:1/781: creat d1/d2a/db6/ff6 x:0 0 0 2024-03-22T07:16:09.790 INFO:tasks.workunit.client.1.smithi069.stdout:1/782: symlink d1/d13/d36/de5/lf7 0 2024-03-22T07:16:09.799 INFO:tasks.workunit.client.1.smithi069.stdout:1/783: write d1/d13/d36/de5/d4c/d55/f97 [4038714,74296] 0 2024-03-22T07:16:09.799 INFO:tasks.workunit.client.1.smithi069.stdout:2/650: dwrite d18/d4d/d27/d33/d3d/f54 [0,4194304] 0 2024-03-22T07:16:09.799 INFO:tasks.workunit.client.1.smithi069.stdout:2/651: stat d18/d4d/d27/d33/d60/d63/d73/fb4 0 2024-03-22T07:16:09.799 INFO:tasks.workunit.client.0.smithi063.stdout:5/999: read d8/d13/d1e/d31/d64/d65/d86/f92 [3384094,91501] 0 2024-03-22T07:16:09.802 INFO:tasks.workunit.client.0.smithi063.stderr:+ rm -rf -- ./tmp.4YiehetO4A 2024-03-22T07:16:09.802 INFO:tasks.workunit.client.1.smithi069.stdout:2/652: mkdir d18/d4d/d27/d33/d60/d64/d89/dcb/dd3 0 2024-03-22T07:16:09.804 INFO:tasks.workunit.client.1.smithi069.stdout:1/784: rename d1/d13/d36/d6e to d1/d2a/db6/df8 0 2024-03-22T07:16:09.804 INFO:tasks.workunit.client.1.smithi069.stdout:2/653: dread d18/d4d/d3e/d40/f44 [0,4194304] 0 2024-03-22T07:16:09.804 INFO:tasks.workunit.client.1.smithi069.stdout:2/654: fdatasync d18/d4d/d3e/fc8 0 2024-03-22T07:16:09.805 INFO:tasks.workunit.client.1.smithi069.stdout:1/785: symlink d1/d2a/db6/df8/d73/lf9 0 2024-03-22T07:16:09.806 INFO:tasks.workunit.client.1.smithi069.stdout:2/655: readlink d18/d4d/d27/d72/l8a 0 2024-03-22T07:16:09.806 INFO:tasks.workunit.client.1.smithi069.stdout:2/656: truncate d18/d4d/d3e/d40/f51 5727652 0 2024-03-22T07:16:09.815 INFO:tasks.workunit.client.1.smithi069.stdout:1/786: write d1/d2a/d5e/d9b/fb1 [2306037,88118] 0 2024-03-22T07:16:09.817 INFO:tasks.workunit.client.1.smithi069.stdout:1/787: rename d1/d13/lbb to d1/d13/d36/de5/d88/lfa 0 2024-03-22T07:16:09.817 INFO:tasks.workunit.client.1.smithi069.stdout:1/788: fsync d1/d13/d36/de5/d88/f38 0 2024-03-22T07:16:09.818 INFO:tasks.workunit.client.1.smithi069.stdout:1/789: chown d1/d2a/d5e/d9b/cc9 0 1 2024-03-22T07:16:09.818 INFO:tasks.workunit.client.1.smithi069.stdout:1/790: fsync d1/d13/d36/de5/d4c/d55/f56 0 2024-03-22T07:16:09.823 INFO:tasks.workunit.client.1.smithi069.stdout:2/657: dwrite d18/d4d/d27/d33/d60/fa4 [0,4194304] 0 2024-03-22T07:16:09.823 INFO:tasks.workunit.client.1.smithi069.stdout:2/658: write d18/d4d/d27/d33/d60/d64/f8f [6897150,29355] 0 2024-03-22T07:16:09.825 INFO:tasks.workunit.client.1.smithi069.stdout:5/693: sync 2024-03-22T07:16:09.826 INFO:tasks.workunit.client.1.smithi069.stdout:3/913: sync 2024-03-22T07:16:09.826 INFO:tasks.workunit.client.1.smithi069.stdout:5/694: truncate d1/d6/d86/d99/d43/d8a/fd0 672313 0 2024-03-22T07:16:09.826 INFO:tasks.workunit.client.1.smithi069.stdout:3/914: read d5/d9/d26/f4c [1588462,103704] 0 2024-03-22T07:16:09.828 INFO:tasks.workunit.client.1.smithi069.stdout:2/659: rename d18/d4d/d27/f5e to d18/d4d/d27/d33/d60/d64/dcd/fd4 0 2024-03-22T07:16:09.828 INFO:tasks.workunit.client.1.smithi069.stdout:1/791: getdents d1/d2a/db6/df8/d73 0 2024-03-22T07:16:09.831 INFO:tasks.workunit.client.1.smithi069.stdout:5/695: creat d1/d6/d86/dab/ff0 x:0 0 0 2024-03-22T07:16:09.833 INFO:tasks.workunit.client.1.smithi069.stdout:3/915: mkdir d5/da2/d134 0 2024-03-22T07:16:09.833 INFO:tasks.workunit.client.1.smithi069.stdout:3/916: creat d5/d9/db/f135 x:0 0 0 2024-03-22T07:16:09.833 INFO:tasks.workunit.client.1.smithi069.stdout:3/917: creat d5/d9/d26/d3f/d56/d8a/da5/d7b/dbc/f136 x:0 0 0 2024-03-22T07:16:09.834 INFO:tasks.workunit.client.1.smithi069.stdout:3/918: write d5/d9/db/f135 [407719,24433] 0 2024-03-22T07:16:09.835 INFO:tasks.workunit.client.1.smithi069.stdout:1/792: symlink d1/d13/d36/de5/d88/dae/de7/lfb 0 2024-03-22T07:16:09.835 INFO:tasks.workunit.client.1.smithi069.stdout:5/696: mkdir d1/d10/df1 0 2024-03-22T07:16:09.836 INFO:tasks.workunit.client.1.smithi069.stdout:5/697: chown d1/d10/d44/d50/ce1 1817 1 2024-03-22T07:16:09.836 INFO:tasks.workunit.client.1.smithi069.stdout:5/698: creat d1/d28/ff2 x:0 0 0 2024-03-22T07:16:09.838 INFO:tasks.workunit.client.1.smithi069.stdout:3/919: mknod d5/d9/d26/d3f/d56/d7f/d90/c137 0 2024-03-22T07:16:09.838 INFO:tasks.workunit.client.1.smithi069.stdout:3/920: read - d5/d9/d26/d3f/dbe/f10e zero size 2024-03-22T07:16:09.839 INFO:tasks.workunit.client.1.smithi069.stdout:1/793: creat d1/d13/d36/de5/d88/ffc x:0 0 0 2024-03-22T07:16:09.839 INFO:tasks.workunit.client.1.smithi069.stdout:1/794: readlink d1/d13/d36/de5/lc0 0 2024-03-22T07:16:09.841 INFO:tasks.workunit.client.1.smithi069.stdout:5/699: creat d1/d6/d26/d33/d5d/d72/da7/ff3 x:0 0 0 2024-03-22T07:16:09.843 INFO:tasks.workunit.client.1.smithi069.stdout:5/700: unlink d1/d6/d88/f9b 0 2024-03-22T07:16:09.845 INFO:tasks.workunit.client.1.smithi069.stdout:5/701: creat d1/d6/d26/d3e/dbb/dbc/ff4 x:0 0 0 2024-03-22T07:16:09.846 INFO:tasks.workunit.client.1.smithi069.stdout:2/660: dwrite d18/f2b [0,4194304] 0 2024-03-22T07:16:09.847 INFO:tasks.workunit.client.1.smithi069.stdout:5/702: link d1/d6/d21/dae/cd9 d1/d10/d44/dd8/de6/cf5 0 2024-03-22T07:16:09.849 INFO:tasks.workunit.client.1.smithi069.stdout:2/661: symlink d18/d4d/d3e/d40/d96/ld5 0 2024-03-22T07:16:09.850 INFO:tasks.workunit.client.1.smithi069.stdout:3/921: dread d5/d9/d26/db4/fd3 [4194304,4194304] 0 2024-03-22T07:16:09.851 INFO:tasks.workunit.client.1.smithi069.stdout:3/922: rmdir d5/da2/ddd 39 2024-03-22T07:16:09.851 INFO:tasks.workunit.client.1.smithi069.stdout:3/923: creat d5/d9/d26/d3f/dbe/f138 x:0 0 0 2024-03-22T07:16:09.854 INFO:tasks.workunit.client.1.smithi069.stdout:1/795: dread d1/d2a/d84/d9a/fc1 [0,4194304] 0 2024-03-22T07:16:09.854 INFO:tasks.workunit.client.1.smithi069.stdout:2/662: write d18/d4d/d3e/f65 [1396090,121144] 0 2024-03-22T07:16:09.855 INFO:tasks.workunit.client.1.smithi069.stdout:3/924: creat d5/d9/d26/d3f/dec/f139 x:0 0 0 2024-03-22T07:16:09.856 INFO:tasks.workunit.client.1.smithi069.stdout:3/925: fsync d5/d9/d26/d3f/f22 0 2024-03-22T07:16:09.856 INFO:tasks.workunit.client.1.smithi069.stdout:3/926: creat d5/d9/d26/d3f/dec/f13a x:0 0 0 2024-03-22T07:16:09.856 INFO:tasks.workunit.client.1.smithi069.stdout:1/796: dread d1/d13/d36/de5/d88/d41/d1e/f51 [0,4194304] 0 2024-03-22T07:16:09.856 INFO:tasks.workunit.client.1.smithi069.stdout:1/797: dread - d1/d13/d82/dc2/fe2 zero size 2024-03-22T07:16:09.862 INFO:tasks.workunit.client.1.smithi069.stdout:2/663: rename d18/d4d/d27/d33/d3d/f48 to d18/fd6 0 2024-03-22T07:16:09.862 INFO:tasks.workunit.client.1.smithi069.stdout:2/664: write d18/d4d/d3e/d40/f6f [472757,38274] 0 2024-03-22T07:16:09.863 INFO:tasks.workunit.client.1.smithi069.stdout:3/927: symlink d5/da2/d134/l13b 0 2024-03-22T07:16:09.863 INFO:tasks.workunit.client.1.smithi069.stdout:3/928: read - d5/d9/d26/d3f/dbe/ff6 zero size 2024-03-22T07:16:09.865 INFO:tasks.workunit.client.1.smithi069.stdout:1/798: creat d1/d2a/db6/df8/d73/df0/ffd x:0 0 0 2024-03-22T07:16:09.865 INFO:tasks.workunit.client.1.smithi069.stdout:1/799: write d1/d2a/d84/fde [1661502,38154] 0 2024-03-22T07:16:09.869 INFO:tasks.workunit.client.1.smithi069.stdout:5/703: rmdir d1/d28 39 2024-03-22T07:16:09.873 INFO:tasks.workunit.client.1.smithi069.stdout:2/665: link d18/d4d/d3e/fc8 d18/d4d/d27/d33/d60/fd7 0 2024-03-22T07:16:09.873 INFO:tasks.workunit.client.1.smithi069.stdout:2/666: getdents d18/d4d/d27/d33/d60/d63 0 2024-03-22T07:16:09.876 INFO:tasks.workunit.client.1.smithi069.stdout:3/929: getdents d5/d9/d26/d3f/dec/dfa 0 2024-03-22T07:16:09.878 INFO:tasks.workunit.client.1.smithi069.stdout:3/930: dread d5/f8f [0,4194304] 0 2024-03-22T07:16:09.878 INFO:tasks.workunit.client.1.smithi069.stdout:3/931: write d5/d9/d26/d3f/d56/d7f/d90/f12c [922897,129842] 0 2024-03-22T07:16:09.878 INFO:tasks.workunit.client.1.smithi069.stdout:3/932: chown d5/da2/f10b 33271372 1 2024-03-22T07:16:09.879 INFO:tasks.workunit.client.1.smithi069.stdout:1/800: creat d1/d13/d36/ffe x:0 0 0 2024-03-22T07:16:09.880 INFO:tasks.workunit.client.1.smithi069.stdout:5/704: getdents d1/d6/d26 0 2024-03-22T07:16:09.882 INFO:tasks.workunit.client.1.smithi069.stdout:2/667: mknod d18/d91/cd8 0 2024-03-22T07:16:09.890 INFO:tasks.workunit.client.1.smithi069.stdout:5/705: dread d1/d6/fa8 [0,4194304] 0 2024-03-22T07:16:09.891 INFO:tasks.workunit.client.1.smithi069.stdout:5/706: write d1/d10/f5b [990388,128774] 0 2024-03-22T07:16:09.891 INFO:tasks.workunit.client.1.smithi069.stdout:3/933: write d5/d9/d26/d3f/d56/d8a/da5/fc0 [2528403,37103] 0 2024-03-22T07:16:09.892 INFO:tasks.workunit.client.1.smithi069.stdout:3/934: unlink d5/d9/d26/f6c 0 2024-03-22T07:16:09.892 INFO:tasks.workunit.client.1.smithi069.stdout:3/935: write d5/d9/d26/d3f/dbe/ff6 [41843,97345] 0 2024-03-22T07:16:09.895 INFO:tasks.workunit.client.1.smithi069.stdout:3/936: truncate d5/f64 1869412 0 2024-03-22T07:16:09.897 INFO:tasks.workunit.client.1.smithi069.stdout:2/668: dread d18/f2b [0,4194304] 0 2024-03-22T07:16:09.897 INFO:tasks.workunit.client.1.smithi069.stdout:2/669: write d18/d4d/d27/d72/f7d [3888547,21457] 0 2024-03-22T07:16:09.897 INFO:tasks.workunit.client.1.smithi069.stdout:2/670: chown d18/d4d/d34/c4c 1940 1 2024-03-22T07:16:09.901 INFO:tasks.workunit.client.1.smithi069.stdout:5/707: write d1/d6/d26/d33/d5d/fa4 [237519,12312] 0 2024-03-22T07:16:09.901 INFO:tasks.workunit.client.1.smithi069.stdout:5/708: fdatasync d1/d6/fa8 0 2024-03-22T07:16:09.904 INFO:tasks.workunit.client.1.smithi069.stdout:3/937: rename d5/dd6/f122 to d5/d9/d26/d3f/dec/dfa/f13c 0 2024-03-22T07:16:09.904 INFO:tasks.workunit.client.1.smithi069.stdout:3/938: chown d5/d9/db/f30 5 1 2024-03-22T07:16:09.910 INFO:tasks.workunit.client.1.smithi069.stdout:2/671: dread d18/d4d/d25/f47 [0,4194304] 0 2024-03-22T07:16:09.913 INFO:tasks.workunit.client.1.smithi069.stdout:5/709: rename d1/d6/d86/d99/f18 to d1/d6/d86/d99/d43/ff6 0 2024-03-22T07:16:09.934 INFO:tasks.workunit.client.1.smithi069.stdout:5/710: dread d1/d6/d21/fb1 [0,4194304] 0 2024-03-22T07:16:09.934 INFO:tasks.workunit.client.1.smithi069.stdout:5/711: dread - d1/d6/d26/d33/d5d/d72/d95/faf zero size 2024-03-22T07:16:09.934 INFO:tasks.workunit.client.1.smithi069.stdout:5/712: fsync d1/d6/d86/d99/f52 0 2024-03-22T07:16:09.935 INFO:tasks.workunit.client.1.smithi069.stdout:3/939: dwrite d5/d9/d26/d3f/d56/d8a/fd1 [4194304,4194304] 0 2024-03-22T07:16:09.940 INFO:tasks.workunit.client.1.smithi069.stdout:3/940: dread d5/d9/d26/d3f/d35/f9e [0,4194304] 0 2024-03-22T07:16:09.941 INFO:tasks.workunit.client.1.smithi069.stdout:5/713: dread d1/d6/d26/d33/f4e [0,4194304] 0 2024-03-22T07:16:09.941 INFO:tasks.workunit.client.1.smithi069.stdout:5/714: creat d1/d6/d26/d3e/d9c/d9d/ff7 x:0 0 0 2024-03-22T07:16:09.941 INFO:tasks.workunit.client.1.smithi069.stdout:5/715: readlink d1/d6/d26/l2d 0 2024-03-22T07:16:09.942 INFO:tasks.workunit.client.1.smithi069.stdout:2/672: dwrite d18/d4d/d25/f36 [0,4194304] 0 2024-03-22T07:16:09.944 INFO:tasks.workunit.client.1.smithi069.stdout:5/716: mknod d1/d6/d26/d33/d5d/d72/d95/cf8 0 2024-03-22T07:16:09.953 INFO:tasks.workunit.client.1.smithi069.stdout:2/673: dread d18/d4d/d27/d33/d60/f5a [0,4194304] 0 2024-03-22T07:16:09.956 INFO:tasks.workunit.client.1.smithi069.stdout:2/674: symlink d18/d4d/d27/d72/ld9 0 2024-03-22T07:16:09.956 INFO:tasks.workunit.client.1.smithi069.stdout:2/675: write d18/d4d/d27/d56/f62 [1632212,89795] 0 2024-03-22T07:16:09.956 INFO:tasks.workunit.client.1.smithi069.stdout:2/676: dread - d18/d4d/d27/d33/d60/d64/dcd/fd4 zero size 2024-03-22T07:16:09.956 INFO:tasks.workunit.client.1.smithi069.stdout:2/677: readlink d18/d4d/d27/d33/d4f/l9e 0 2024-03-22T07:16:09.956 INFO:tasks.workunit.client.1.smithi069.stdout:2/678: fsync d18/d4d/d27/d33/d60/d63/faa 0 2024-03-22T07:16:09.956 INFO:tasks.workunit.client.1.smithi069.stdout:2/679: write d18/d4d/d27/d33/d60/d64/fc5 [904848,63001] 0 2024-03-22T07:16:09.958 INFO:tasks.workunit.client.1.smithi069.stdout:2/680: link d18/d4d/d27/d72/fbb d18/d4d/d27/d33/d60/d64/fda 0 2024-03-22T07:16:09.959 INFO:tasks.workunit.client.1.smithi069.stdout:3/941: dwrite d5/d9/d26/d3f/d56/d8a/da5/d7b/dbc/f133 [0,4194304] 0 2024-03-22T07:16:09.959 INFO:tasks.workunit.client.1.smithi069.stdout:3/942: chown d5/d9/db/la3 36 1 2024-03-22T07:16:09.961 INFO:tasks.workunit.client.1.smithi069.stdout:2/681: unlink d18/ca5 0 2024-03-22T07:16:09.964 INFO:tasks.workunit.client.1.smithi069.stdout:2/682: mkdir d18/d4d/d27/d33/d60/d63/d73/dc9/ddb 0 2024-03-22T07:16:09.970 INFO:tasks.workunit.client.1.smithi069.stdout:2/683: dread d18/d4d/d25/f47 [0,4194304] 0 2024-03-22T07:16:09.970 INFO:tasks.workunit.client.1.smithi069.stdout:2/684: write d18/f9b [758640,70728] 0 2024-03-22T07:16:09.970 INFO:tasks.workunit.client.1.smithi069.stdout:2/685: readlink d18/d4d/d3e/d40/d96/ld5 0 2024-03-22T07:16:09.972 INFO:tasks.workunit.client.1.smithi069.stdout:2/686: write d18/d4d/d3e/d40/f44 [311171,92310] 0 2024-03-22T07:16:09.973 INFO:tasks.workunit.client.1.smithi069.stdout:3/943: read d5/d9/db/f30 [386108,11211] 0 2024-03-22T07:16:09.974 INFO:tasks.workunit.client.1.smithi069.stdout:3/944: write d5/f10a [17802,43455] 0 2024-03-22T07:16:09.974 INFO:tasks.workunit.client.1.smithi069.stdout:5/717: dwrite d1/d6/d26/d33/d5d/fa4 [0,4194304] 0 2024-03-22T07:16:09.974 INFO:tasks.workunit.client.1.smithi069.stdout:2/687: unlink d18/d4d/d3e/d40/f76 0 2024-03-22T07:16:09.980 INFO:tasks.workunit.client.1.smithi069.stdout:2/688: mkdir d18/d4d/d27/d33/d60/dbd/ddc 0 2024-03-22T07:16:09.981 INFO:tasks.workunit.client.1.smithi069.stdout:2/689: unlink d18/d4d/d25/c6a 0 2024-03-22T07:16:09.999 INFO:tasks.workunit.client.1.smithi069.stdout:5/718: dwrite d1/d6/fdf [0,4194304] 0 2024-03-22T07:16:09.999 INFO:tasks.workunit.client.1.smithi069.stdout:5/719: creat d1/d6/d26/d3a/db2/ff9 x:0 0 0 2024-03-22T07:16:10.002 INFO:tasks.workunit.client.1.smithi069.stdout:5/720: mkdir d1/d7/d3d/dfa 0 2024-03-22T07:16:10.003 INFO:tasks.workunit.client.1.smithi069.stdout:5/721: truncate d1/d7/d3d/fca 8815 0 2024-03-22T07:16:10.003 INFO:tasks.workunit.client.1.smithi069.stdout:5/722: fdatasync d1/d6/d86/dab/ff0 0 2024-03-22T07:16:10.003 INFO:tasks.workunit.client.1.smithi069.stdout:5/723: truncate d1/d6/d26/d3e/dbb/fdd 4532013 0 2024-03-22T07:16:10.003 INFO:tasks.workunit.client.1.smithi069.stdout:5/724: mkdir d1/d6/d88/dfb 0 2024-03-22T07:16:10.003 INFO:tasks.workunit.client.1.smithi069.stdout:5/725: symlink d1/d6/d86/lfc 0 2024-03-22T07:16:10.003 INFO:tasks.workunit.client.1.smithi069.stdout:4/941: sync 2024-03-22T07:16:10.004 INFO:tasks.workunit.client.1.smithi069.stdout:7/794: sync 2024-03-22T07:16:10.004 INFO:tasks.workunit.client.1.smithi069.stdout:9/776: sync 2024-03-22T07:16:10.004 INFO:tasks.workunit.client.1.smithi069.stdout:9/777: dread - d2/d71/fd4 zero size 2024-03-22T07:16:10.006 INFO:tasks.workunit.client.1.smithi069.stdout:2/690: dwrite d18/d4d/d27/d33/d3d/f54 [0,4194304] 0 2024-03-22T07:16:10.006 INFO:tasks.workunit.client.1.smithi069.stdout:2/691: chown d18/d4d/d27/d33/d60/d64/d89/f8d 3676315 1 2024-03-22T07:16:10.007 INFO:tasks.workunit.client.1.smithi069.stdout:4/942: symlink d6/d16/d83/d9a/l142 0 2024-03-22T07:16:10.007 INFO:tasks.workunit.client.1.smithi069.stdout:7/795: stat d9/d42/l86 0 2024-03-22T07:16:10.007 INFO:tasks.workunit.client.1.smithi069.stdout:3/945: dwrite d5/d9/d26/d3f/d56/d8a/da5/f39 [0,4194304] 0 2024-03-22T07:16:10.009 INFO:tasks.workunit.client.1.smithi069.stdout:2/692: truncate d18/d4d/d27/d72/f7d 3456229 0 2024-03-22T07:16:10.010 INFO:tasks.workunit.client.1.smithi069.stdout:7/796: mkdir d9/dff 0 2024-03-22T07:16:10.010 INFO:tasks.workunit.client.1.smithi069.stdout:7/797: write d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/fae [1338702,12927] 0 2024-03-22T07:16:10.011 INFO:tasks.workunit.client.1.smithi069.stdout:7/798: write d9/fd7 [2005505,839] 0 2024-03-22T07:16:10.011 INFO:tasks.workunit.client.1.smithi069.stdout:7/799: stat d9/ld 0 2024-03-22T07:16:10.011 INFO:tasks.workunit.client.1.smithi069.stdout:7/800: dread - d9/db/d1a/f82 zero size 2024-03-22T07:16:10.011 INFO:tasks.workunit.client.1.smithi069.stdout:7/801: readlink d9/db/d1a/db1/d57/d44/d53/d2e/l9f 0 2024-03-22T07:16:10.011 INFO:tasks.workunit.client.1.smithi069.stdout:3/946: symlink d5/da2/ddd/d109/l13d 0 2024-03-22T07:16:10.011 INFO:tasks.workunit.client.1.smithi069.stdout:3/947: creat d5/d9/d26/d36/d41/d8e/f13e x:0 0 0 2024-03-22T07:16:10.013 INFO:tasks.workunit.client.1.smithi069.stdout:9/778: rename d2/dd/d3c/d53/d8b/fce to d2/dd/d3c/d53/d8b/ffc 0 2024-03-22T07:16:10.013 INFO:tasks.workunit.client.1.smithi069.stdout:2/693: readlink d18/d4d/d27/l6e 0 2024-03-22T07:16:10.018 INFO:tasks.workunit.client.1.smithi069.stdout:3/948: symlink d5/d9/db/d3b/dc6/l13f 0 2024-03-22T07:16:10.018 INFO:tasks.workunit.client.1.smithi069.stdout:3/949: getdents d5/d9/d26/d3f/d56/d8a 0 2024-03-22T07:16:10.020 INFO:tasks.workunit.client.1.smithi069.stdout:3/950: dread d5/d9/d26/d3f/d56/d7f/d90/fdf [0,4194304] 0 2024-03-22T07:16:10.020 INFO:tasks.workunit.client.1.smithi069.stdout:2/694: mkdir d18/d4d/d27/d33/d4f/ddd 0 2024-03-22T07:16:10.020 INFO:tasks.workunit.client.1.smithi069.stdout:2/695: chown d18/l1a 33689256 1 2024-03-22T07:16:10.022 INFO:tasks.workunit.client.1.smithi069.stdout:4/943: dread d6/d16/d3a/db4/dfb/d23/f3c [0,4194304] 0 2024-03-22T07:16:10.022 INFO:tasks.workunit.client.1.smithi069.stdout:4/944: chown d6/d9b/d1f/d58 186449 1 2024-03-22T07:16:10.025 INFO:tasks.workunit.client.1.smithi069.stdout:2/696: creat d18/d4d/d27/d33/d60/d63/d80/fde x:0 0 0 2024-03-22T07:16:10.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:09 smithi069 ceph-mon[36441]: pgmap v7: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 260 MiB/s rd, 292 MiB/s wr, 431 op/s 2024-03-22T07:16:10.032 INFO:tasks.workunit.client.1.smithi069.stdout:3/951: dread d5/d9/d26/d3f/d56/d8a/da5/f39 [0,4194304] 0 2024-03-22T07:16:10.032 INFO:tasks.workunit.client.1.smithi069.stdout:3/952: readlink d5/d9/d26/d3f/d56/d8a/da5/d7b/db9/lcb 0 2024-03-22T07:16:10.034 INFO:tasks.workunit.client.1.smithi069.stdout:3/953: mkdir d5/d9/d26/d3f/d56/d8a/da5/d140 0 2024-03-22T07:16:10.034 INFO:tasks.workunit.client.1.smithi069.stdout:3/954: dread - d5/d9/d23/f12e zero size 2024-03-22T07:16:10.034 INFO:tasks.workunit.client.1.smithi069.stdout:3/955: creat d5/d9/d26/d3f/d54/d9d/ded/f141 x:0 0 0 2024-03-22T07:16:10.035 INFO:tasks.workunit.client.1.smithi069.stdout:3/956: creat d5/d9/d26/d36/d41/f142 x:0 0 0 2024-03-22T07:16:10.037 INFO:tasks.workunit.client.1.smithi069.stdout:2/697: dread d18/d4d/d3e/d40/f61 [0,4194304] 0 2024-03-22T07:16:10.043 INFO:tasks.workunit.client.1.smithi069.stdout:7/802: dwrite d9/db/f94 [0,4194304] 0 2024-03-22T07:16:10.044 INFO:tasks.workunit.client.1.smithi069.stdout:5/726: dwrite d1/d6/d86/d99/d43/d8a/fd0 [0,4194304] 0 2024-03-22T07:16:10.044 INFO:tasks.workunit.client.1.smithi069.stdout:5/727: chown d1/d7/c37 49421259 1 2024-03-22T07:16:10.044 INFO:tasks.workunit.client.1.smithi069.stdout:9/779: dwrite d2/d51/f57 [0,4194304] 0 2024-03-22T07:16:10.044 INFO:tasks.workunit.client.1.smithi069.stdout:9/780: dread - d2/dd/d3c/d53/d9d/f97 zero size 2024-03-22T07:16:10.053 INFO:tasks.workunit.client.1.smithi069.stdout:2/698: dread fa [0,4194304] 0 2024-03-22T07:16:10.053 INFO:tasks.workunit.client.1.smithi069.stdout:9/781: getdents d2/d51/dbd/de4 0 2024-03-22T07:16:10.053 INFO:tasks.workunit.client.1.smithi069.stdout:3/957: write d5/d9/d26/d3f/d54/feb [2608899,105530] 0 2024-03-22T07:16:10.053 INFO:tasks.workunit.client.1.smithi069.stdout:9/782: write d2/dd/d3c/d48/d7f/d86/f8e [1094611,93766] 0 2024-03-22T07:16:10.053 INFO:tasks.workunit.client.1.smithi069.stdout:9/783: read - d2/f9f zero size 2024-03-22T07:16:10.053 INFO:tasks.workunit.client.1.smithi069.stdout:3/958: stat d5/da2/ddd/lf2 0 2024-03-22T07:16:10.053 INFO:tasks.workunit.client.1.smithi069.stdout:9/784: chown d2/d21/d89/lf7 0 1 2024-03-22T07:16:10.058 INFO:tasks.workunit.client.1.smithi069.stdout:9/785: rename d2/f55 to d2/dd/d3c/d53/ffd 0 2024-03-22T07:16:10.058 INFO:tasks.workunit.client.1.smithi069.stdout:9/786: chown d2/dd/f69 157 1 2024-03-22T07:16:10.060 INFO:tasks.workunit.client.1.smithi069.stdout:3/959: mkdir d5/d9/d23/d143 0 2024-03-22T07:16:10.066 INFO:tasks.workunit.client.1.smithi069.stdout:9/787: truncate d2/d71/fcd 3987881 0 2024-03-22T07:16:10.066 INFO:tasks.workunit.client.1.smithi069.stdout:9/788: chown d2/dd/d3c/d2e/f32 48609 1 2024-03-22T07:16:10.066 INFO:tasks.workunit.client.1.smithi069.stdout:9/789: fsync d2/d83/fc5 0 2024-03-22T07:16:10.066 INFO:tasks.workunit.client.1.smithi069.stdout:9/790: fdatasync d2/dd/d3c/d53/d8b/ffc 0 2024-03-22T07:16:10.066 INFO:tasks.workunit.client.1.smithi069.stdout:9/791: fdatasync d2/dd/d3c/d2e/fa8 0 2024-03-22T07:16:10.067 INFO:tasks.workunit.client.1.smithi069.stdout:3/960: link d5/d9/d26/d3f/d35/f58 d5/da2/ddd/d11e/f144 0 2024-03-22T07:16:10.068 INFO:tasks.workunit.client.1.smithi069.stdout:3/961: link d5/d9/d26/d3f/d35/lc3 d5/d9/d26/d3f/d56/d8a/l145 0 2024-03-22T07:16:10.069 INFO:tasks.workunit.client.1.smithi069.stdout:3/962: truncate d5/d9/f48 4919494 0 2024-03-22T07:16:10.069 INFO:tasks.workunit.client.1.smithi069.stdout:3/963: fsync d5/d9/d26/d3f/d56/d8a/da5/f99 0 2024-03-22T07:16:10.071 INFO:tasks.workunit.client.1.smithi069.stdout:3/964: unlink d5/d9/d26/d3f/dc1/f110 0 2024-03-22T07:16:10.072 INFO:tasks.workunit.client.1.smithi069.stdout:3/965: mkdir d5/da2/d146 0 2024-03-22T07:16:10.072 INFO:tasks.workunit.client.1.smithi069.stdout:3/966: stat d5/d9/db/dff/f7c 0 2024-03-22T07:16:10.072 INFO:tasks.workunit.client.1.smithi069.stdout:3/967: fsync d5/d9/d26/d36/d41/fe8 0 2024-03-22T07:16:10.077 INFO:tasks.workunit.client.1.smithi069.stdout:2/699: dwrite d18/d4d/d27/d72/fbb [0,4194304] 0 2024-03-22T07:16:10.078 INFO:tasks.workunit.client.1.smithi069.stdout:5/728: dwrite d1/d6/d26/d33/d5d/fc4 [0,4194304] 0 2024-03-22T07:16:10.081 INFO:tasks.workunit.client.1.smithi069.stdout:2/700: truncate d18/f1e 686770 0 2024-03-22T07:16:10.082 INFO:tasks.workunit.client.1.smithi069.stdout:9/792: read d2/dd/d3c/d53/d9d/ff4 [1333588,12954] 0 2024-03-22T07:16:10.082 INFO:tasks.workunit.client.1.smithi069.stdout:9/793: fdatasync d2/dd/d3c/f43 0 2024-03-22T07:16:10.087 INFO:tasks.workunit.client.1.smithi069.stdout:2/701: mknod d18/d4d/d27/cdf 0 2024-03-22T07:16:10.089 INFO:tasks.workunit.client.1.smithi069.stdout:3/968: dwrite d5/d9/db/f135 [0,4194304] 0 2024-03-22T07:16:10.090 INFO:tasks.workunit.client.1.smithi069.stdout:2/702: mknod d18/d4d/d27/d33/d60/d64/d89/d8b/dab/ce0 0 2024-03-22T07:16:10.090 INFO:tasks.workunit.client.1.smithi069.stdout:3/969: write d5/f4e [1552094,129064] 0 2024-03-22T07:16:10.092 INFO:tasks.workunit.client.1.smithi069.stdout:2/703: rmdir d18/d4d/d27/d33/d60/d64/d89 39 2024-03-22T07:16:10.092 INFO:tasks.workunit.client.1.smithi069.stdout:2/704: dread - d18/d4d/d27/d33/d60/d63/d80/f95 zero size 2024-03-22T07:16:10.098 INFO:tasks.workunit.client.1.smithi069.stdout:3/970: mknod d5/d9/d26/d3f/d56/d7f/d90/c147 0 2024-03-22T07:16:10.102 INFO:tasks.workunit.client.1.smithi069.stdout:2/705: rename d18/d4d/d27/l6e to d18/d4d/d27/d33/d60/d63/d73/dc9/ddb/le1 0 2024-03-22T07:16:10.103 INFO:tasks.workunit.client.1.smithi069.stdout:2/706: dread - d18/d4d/d27/d33/f82 zero size 2024-03-22T07:16:10.106 INFO:tasks.workunit.client.1.smithi069.stdout:5/729: dwrite d1/d7/f40 [0,4194304] 0 2024-03-22T07:16:10.106 INFO:tasks.workunit.client.1.smithi069.stdout:5/730: fdatasync d1/d6/d86/dab/fcb 0 2024-03-22T07:16:10.108 INFO:tasks.workunit.client.1.smithi069.stdout:5/731: mknod d1/d6/d26/d3a/cfd 0 2024-03-22T07:16:10.108 INFO:tasks.workunit.client.1.smithi069.stdout:5/732: write d1/d7/d3d/f9a [869704,27600] 0 2024-03-22T07:16:10.110 INFO:tasks.workunit.client.1.smithi069.stdout:9/794: dwrite d2/dd/d3c/d53/fae [0,4194304] 0 2024-03-22T07:16:10.110 INFO:tasks.workunit.client.1.smithi069.stdout:9/795: fdatasync d2/f5 0 2024-03-22T07:16:10.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:09 smithi063 ceph-mon[29200]: pgmap v7: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 260 MiB/s rd, 292 MiB/s wr, 431 op/s 2024-03-22T07:16:10.132 INFO:tasks.workunit.client.1.smithi069.stdout:6/712: sync 2024-03-22T07:16:10.132 INFO:tasks.workunit.client.1.smithi069.stdout:0/908: sync 2024-03-22T07:16:10.144 INFO:tasks.workunit.client.1.smithi069.stdout:2/707: dwrite d18/d4d/d27/d33/d60/d64/dcd/fd4 [0,4194304] 0 2024-03-22T07:16:10.145 INFO:tasks.workunit.client.1.smithi069.stdout:5/733: dwrite d1/d6/d26/d33/d5d/d72/da7/ff3 [0,4194304] 0 2024-03-22T07:16:10.145 INFO:tasks.workunit.client.1.smithi069.stdout:9/796: dwrite d2/d51/f59 [0,4194304] 0 2024-03-22T07:16:10.147 INFO:tasks.workunit.client.1.smithi069.stdout:3/971: dwrite d5/d9/d26/f8b [0,4194304] 0 2024-03-22T07:16:10.153 INFO:tasks.workunit.client.1.smithi069.stdout:0/909: dwrite d3/d7/d15/d29/d5a/d5d/f119 [4194304,4194304] 0 2024-03-22T07:16:10.157 INFO:tasks.workunit.client.1.smithi069.stdout:3/972: link d5/d9/lfe d5/d9/d26/d36/l148 0 2024-03-22T07:16:10.157 INFO:tasks.workunit.client.1.smithi069.stdout:3/973: chown d5/d9/d26/d3f/dbe/f127 12 1 2024-03-22T07:16:10.159 INFO:tasks.workunit.client.1.smithi069.stdout:3/974: symlink d5/d9/d23/d130/l149 0 2024-03-22T07:16:10.161 INFO:tasks.workunit.client.1.smithi069.stdout:0/910: getdents d3/d7/d17/d1c/dd4 0 2024-03-22T07:16:10.161 INFO:tasks.workunit.client.1.smithi069.stdout:0/911: chown d3/d7/d15/dad/fd0 1318344578 1 2024-03-22T07:16:10.162 INFO:tasks.workunit.client.1.smithi069.stdout:3/975: mkdir d5/d9/d26/d3f/d56/d8a/d14a 0 2024-03-22T07:16:10.162 INFO:tasks.workunit.client.1.smithi069.stdout:3/976: fdatasync d5/f5d 0 2024-03-22T07:16:10.163 INFO:tasks.workunit.client.1.smithi069.stdout:0/912: mknod d3/d7/d15/d29/d5a/d84/df4/c12f 0 2024-03-22T07:16:10.166 INFO:tasks.workunit.client.1.smithi069.stdout:0/913: truncate d3/d7/d5e/f61 2839913 0 2024-03-22T07:16:10.166 INFO:tasks.workunit.client.1.smithi069.stdout:0/914: readlink d3/d7/d15/d29/l2d 0 2024-03-22T07:16:10.166 INFO:tasks.workunit.client.1.smithi069.stdout:0/915: write d3/d7/d15/d29/d5a/d84/f3b [1124046,28574] 0 2024-03-22T07:16:10.166 INFO:tasks.workunit.client.1.smithi069.stdout:0/916: chown d3/d7/d15/d29/c96 95706224 1 2024-03-22T07:16:10.166 INFO:tasks.workunit.client.1.smithi069.stdout:3/977: rmdir d5/d9/d26/d36/d41/d8e 39 2024-03-22T07:16:10.167 INFO:tasks.workunit.client.1.smithi069.stdout:2/708: dwrite d18/d4d/d3e/fc8 [0,4194304] 0 2024-03-22T07:16:10.167 INFO:tasks.workunit.client.1.smithi069.stdout:2/709: readlink d18/d4d/d27/d56/l68 0 2024-03-22T07:16:10.167 INFO:tasks.workunit.client.1.smithi069.stdout:5/734: dwrite d1/d6/d26/d33/d5d/d72/d78/fd3 [0,4194304] 0 2024-03-22T07:16:10.167 INFO:tasks.workunit.client.1.smithi069.stdout:5/735: chown d1/d6/d26/d3a/db2/ff9 0 1 2024-03-22T07:16:10.169 INFO:tasks.workunit.client.1.smithi069.stdout:0/917: creat d3/d7/d15/d29/dcf/f130 x:0 0 0 2024-03-22T07:16:10.169 INFO:tasks.workunit.client.1.smithi069.stdout:0/918: write d3/d7/d90/f94 [1603472,60582] 0 2024-03-22T07:16:10.169 INFO:tasks.workunit.client.1.smithi069.stdout:0/919: write d3/d7/d15/d1f/fe8 [2155668,74692] 0 2024-03-22T07:16:10.171 INFO:tasks.workunit.client.1.smithi069.stdout:3/978: rename d5/d9/d23/c80 to d5/d9/d26/d3f/d54/c14b 0 2024-03-22T07:16:10.171 INFO:tasks.workunit.client.1.smithi069.stdout:3/979: readlink d5/da2/l12d 0 2024-03-22T07:16:10.174 INFO:tasks.workunit.client.1.smithi069.stdout:2/710: creat d18/d4d/d27/d33/d3d/dc6/fe2 x:0 0 0 2024-03-22T07:16:10.174 INFO:tasks.workunit.client.1.smithi069.stdout:2/711: write d18/d4d/f2d [121172,129978] 0 2024-03-22T07:16:10.174 INFO:tasks.workunit.client.1.smithi069.stdout:0/920: dread d3/d7/d17/f56 [0,4194304] 0 2024-03-22T07:16:10.176 INFO:tasks.workunit.client.1.smithi069.stdout:5/736: unlink d1/d6/d26/d3a/dbd/cd4 0 2024-03-22T07:16:10.176 INFO:tasks.workunit.client.1.smithi069.stdout:5/737: chown d1/d10/c16 21 1 2024-03-22T07:16:10.180 INFO:tasks.workunit.client.1.smithi069.stdout:3/980: mknod d5/d9/d26/d3f/d56/d8a/da5/d140/c14c 0 2024-03-22T07:16:10.180 INFO:tasks.workunit.client.1.smithi069.stdout:3/981: write d5/d9/db/dff/f7c [4739994,618] 0 2024-03-22T07:16:10.181 INFO:tasks.workunit.client.1.smithi069.stdout:2/712: creat d18/d4d/d27/d33/d60/d63/fe3 x:0 0 0 2024-03-22T07:16:10.182 INFO:tasks.workunit.client.1.smithi069.stdout:0/921: mkdir d3/d7/d5e/d131 0 2024-03-22T07:16:10.184 INFO:tasks.workunit.client.1.smithi069.stdout:5/738: creat d1/d10/d44/dd8/de6/ffe x:0 0 0 2024-03-22T07:16:10.184 INFO:tasks.workunit.client.1.smithi069.stdout:5/739: write d1/d6/d26/d33/d5d/d72/da7/ff3 [4836208,44631] 0 2024-03-22T07:16:10.186 INFO:tasks.workunit.client.1.smithi069.stdout:3/982: creat d5/da2/ddd/d11e/f14d x:0 0 0 2024-03-22T07:16:10.186 INFO:tasks.workunit.client.1.smithi069.stdout:3/983: chown d5/d9/db/fab 2688003 1 2024-03-22T07:16:10.189 INFO:tasks.workunit.client.1.smithi069.stdout:2/713: truncate d18/d4d/d27/d33/d60/d63/d73/fb4 3865081 0 2024-03-22T07:16:10.196 INFO:tasks.workunit.client.1.smithi069.stdout:2/714: creat d18/d4d/d27/d33/d4f/ddd/fe4 x:0 0 0 2024-03-22T07:16:10.197 INFO:tasks.workunit.client.1.smithi069.stdout:5/740: rmdir d1/d6/d86/d99/d43/d8a 39 2024-03-22T07:16:10.198 INFO:tasks.workunit.client.1.smithi069.stdout:2/715: unlink d18/d4d/d3e/f65 0 2024-03-22T07:16:10.198 INFO:tasks.workunit.client.1.smithi069.stdout:2/716: write d18/d4d/d27/d77/fc7 [83547,64383] 0 2024-03-22T07:16:10.200 INFO:tasks.workunit.client.1.smithi069.stdout:2/717: symlink d18/d4d/d27/d33/d60/d63/d73/dc9/le5 0 2024-03-22T07:16:10.203 INFO:tasks.workunit.client.1.smithi069.stdout:2/718: dread d18/d4d/d27/d33/d60/d64/fac [0,4194304] 0 2024-03-22T07:16:10.205 INFO:tasks.workunit.client.1.smithi069.stdout:2/719: link d18/f3a d18/d4d/d27/d33/d60/dbd/fe6 0 2024-03-22T07:16:10.207 INFO:tasks.workunit.client.1.smithi069.stdout:0/922: dwrite d3/d7/d15/d29/d5a/d5d/d9d/fa9 [0,4194304] 0 2024-03-22T07:16:10.207 INFO:tasks.workunit.client.1.smithi069.stdout:0/923: truncate d3/d7/d5e/f8a 158143 0 2024-03-22T07:16:10.209 INFO:tasks.workunit.client.1.smithi069.stdout:3/984: dwrite d5/d9/d26/d3f/dc1/ff3 [0,4194304] 0 2024-03-22T07:16:10.210 INFO:tasks.workunit.client.1.smithi069.stdout:0/924: mknod d3/d7/d5e/c132 0 2024-03-22T07:16:10.212 INFO:tasks.workunit.client.1.smithi069.stdout:3/985: truncate d5/d9/d26/d3f/d56/d8a/da5/f9c 107685 0 2024-03-22T07:16:10.213 INFO:tasks.workunit.client.1.smithi069.stdout:0/925: mkdir d3/d9a/d133 0 2024-03-22T07:16:10.216 INFO:tasks.workunit.client.1.smithi069.stdout:5/741: dwrite d1/d6/d88/fc9 [0,4194304] 0 2024-03-22T07:16:10.216 INFO:tasks.workunit.client.1.smithi069.stdout:5/742: read d1/d6/d86/dab/dcf/fe0 [221961,113707] 0 2024-03-22T07:16:10.217 INFO:tasks.workunit.client.1.smithi069.stdout:5/743: write d1/d28/f7e [456249,9832] 0 2024-03-22T07:16:10.219 INFO:tasks.workunit.client.1.smithi069.stdout:2/720: getdents d18/d4d/d3e 0 2024-03-22T07:16:10.228 INFO:tasks.workunit.client.1.smithi069.stdout:0/926: write d3/d7/d17/f3f [2184157,80767] 0 2024-03-22T07:16:10.228 INFO:tasks.workunit.client.1.smithi069.stdout:0/927: stat d3/d7/d15/dce/f115 0 2024-03-22T07:16:10.228 INFO:tasks.workunit.client.1.smithi069.stdout:5/744: link d1/d6/fdf d1/d6/d86/d99/d43/d8a/fff 0 2024-03-22T07:16:10.228 INFO:tasks.workunit.client.1.smithi069.stdout:2/721: chown d18/d4d/d27/f8c 1 1 2024-03-22T07:16:10.228 INFO:tasks.workunit.client.1.smithi069.stdout:2/722: creat d18/d4d/d27/d33/d3d/dc6/fe7 x:0 0 0 2024-03-22T07:16:10.231 INFO:tasks.workunit.client.1.smithi069.stdout:0/928: rename d3/d7/d15/d29/l80 to d3/d7/d15/dad/l134 0 2024-03-22T07:16:10.233 INFO:tasks.workunit.client.1.smithi069.stdout:5/745: mkdir d1/d7/d3d/d100 0 2024-03-22T07:16:10.233 INFO:tasks.workunit.client.1.smithi069.stdout:2/723: fsync d18/d4d/d27/d33/d60/d63/d80/f95 0 2024-03-22T07:16:10.235 INFO:tasks.workunit.client.1.smithi069.stdout:5/746: mkdir d1/d10/d44/dd8/d101 0 2024-03-22T07:16:10.236 INFO:tasks.workunit.client.1.smithi069.stdout:5/747: rename d1/d6/d21/dae to d1/d6/d26/d33/d5d/d72/d78/d102 0 2024-03-22T07:16:10.243 INFO:tasks.workunit.client.1.smithi069.stdout:5/748: dread d1/d10/d44/fe2 [0,4194304] 0 2024-03-22T07:16:10.245 INFO:tasks.workunit.client.1.smithi069.stdout:5/749: mkdir d1/d6/d26/d33/d5d/d72/d95/d103 0 2024-03-22T07:16:10.247 INFO:tasks.workunit.client.1.smithi069.stdout:5/750: creat d1/d6/d26/d7d/d9f/f104 x:0 0 0 2024-03-22T07:16:10.248 INFO:tasks.workunit.client.1.smithi069.stdout:5/751: link d1/d10/d44/fe2 d1/d10/d44/f105 0 2024-03-22T07:16:10.252 INFO:tasks.workunit.client.1.smithi069.stdout:3/986: dwrite d5/d9/d26/d3f/d56/d7f/d90/fdf [0,4194304] 0 2024-03-22T07:16:10.253 INFO:tasks.workunit.client.1.smithi069.stdout:3/987: write d5/d9/d26/d3f/d35/f50 [506406,29668] 0 2024-03-22T07:16:10.253 INFO:tasks.workunit.client.1.smithi069.stdout:3/988: write d5/da2/f10b [715283,27159] 0 2024-03-22T07:16:10.253 INFO:tasks.workunit.client.1.smithi069.stdout:3/989: readlink d5/da2/l132 0 2024-03-22T07:16:10.255 INFO:tasks.workunit.client.1.smithi069.stdout:2/724: dwrite d18/d4d/d25/f4b [0,4194304] 0 2024-03-22T07:16:10.255 INFO:tasks.workunit.client.1.smithi069.stdout:2/725: truncate d18/f1e 736718 0 2024-03-22T07:16:10.259 INFO:tasks.workunit.client.1.smithi069.stdout:2/726: read d18/f2b [3285886,103457] 0 2024-03-22T07:16:10.339 INFO:tasks.workunit.client.1.smithi069.stdout:3/990: mknod d5/d9/d23/d128/c14e 0 2024-03-22T07:16:10.339 INFO:tasks.workunit.client.1.smithi069.stdout:3/991: readlink d5/d9/d26/d3f/d56/d8a/da5/d7b/db9/lcb 0 2024-03-22T07:16:10.339 INFO:tasks.workunit.client.1.smithi069.stdout:3/992: stat d5/d9/d26/d3f/f3e 0 2024-03-22T07:16:10.339 INFO:tasks.workunit.client.1.smithi069.stdout:3/993: stat d5/da2/ddd/lea 0 2024-03-22T07:16:10.343 INFO:tasks.workunit.client.1.smithi069.stdout:3/994: rename d5/d9/d26/d3f/d35/f8d to d5/d9/d26/d36/d41/d8e/d11c/f14f 0 2024-03-22T07:16:10.343 INFO:tasks.workunit.client.1.smithi069.stdout:1/801: sync 2024-03-22T07:16:10.343 INFO:tasks.workunit.client.1.smithi069.stdout:1/802: fdatasync d1/d13/d36/f67 0 2024-03-22T07:16:10.343 INFO:tasks.workunit.client.1.smithi069.stdout:1/803: readlink d1/d13/l49 0 2024-03-22T07:16:10.343 INFO:tasks.workunit.client.1.smithi069.stdout:1/804: fdatasync d1/d13/d36/f54 0 2024-03-22T07:16:10.351 INFO:tasks.workunit.client.1.smithi069.stdout:1/805: read d1/d13/f2d [1957189,127992] 0 2024-03-22T07:16:10.352 INFO:tasks.workunit.client.1.smithi069.stdout:1/806: creat d1/d13/dc7/fff x:0 0 0 2024-03-22T07:16:10.353 INFO:tasks.workunit.client.1.smithi069.stdout:1/807: mkdir d1/d13/dc7/d100 0 2024-03-22T07:16:10.354 INFO:tasks.workunit.client.1.smithi069.stdout:1/808: symlink d1/d13/d36/de5/d88/l101 0 2024-03-22T07:16:10.355 INFO:tasks.workunit.client.1.smithi069.stdout:1/809: symlink d1/d13/d36/de5/d88/l102 0 2024-03-22T07:16:10.355 INFO:tasks.workunit.client.1.smithi069.stdout:1/810: write d1/d2a/d84/d9a/fc1 [4575152,16962] 0 2024-03-22T07:16:10.356 INFO:tasks.workunit.client.1.smithi069.stdout:3/995: dwrite d5/d9/d26/d36/d41/d8e/f13e [0,4194304] 0 2024-03-22T07:16:10.356 INFO:tasks.workunit.client.1.smithi069.stdout:3/996: dread - d5/d9/d26/fe4 zero size 2024-03-22T07:16:10.357 INFO:tasks.workunit.client.1.smithi069.stdout:1/811: mknod d1/d2a/db6/df8/d73/df0/dbc/c103 0 2024-03-22T07:16:10.360 INFO:tasks.workunit.client.1.smithi069.stdout:1/812: link d1/d13/d36/de5/d88/l101 d1/d13/d36/d5b/d95/l104 0 2024-03-22T07:16:10.361 INFO:tasks.workunit.client.1.smithi069.stdout:1/813: mknod d1/d13/d36/d5b/d95/c105 0 2024-03-22T07:16:10.363 INFO:tasks.workunit.client.1.smithi069.stdout:1/814: rename d1/d13/d36/de5/d88/l101 to d1/d2a/d5e/d9b/l106 0 2024-03-22T07:16:10.364 INFO:tasks.workunit.client.1.smithi069.stdout:1/815: mknod d1/d13/d36/d5b/d95/c107 0 2024-03-22T07:16:10.365 INFO:tasks.workunit.client.1.smithi069.stdout:1/816: write d1/d13/f2d [1648805,81947] 0 2024-03-22T07:16:10.365 INFO:tasks.workunit.client.1.smithi069.stdout:1/817: fdatasync d1/d13/d36/de5/d88/d41/fc5 0 2024-03-22T07:16:10.366 INFO:tasks.workunit.client.1.smithi069.stdout:3/997: dread d5/d9/d26/d36/d41/fd0 [0,4194304] 0 2024-03-22T07:16:10.367 INFO:tasks.workunit.client.1.smithi069.stdout:1/818: rename d1/f3b to d1/d13/d36/de5/d4c/d55/d5f/f108 0 2024-03-22T07:16:10.368 INFO:tasks.workunit.client.1.smithi069.stdout:3/998: mkdir d5/d9/d26/d36/dfd/d150 0 2024-03-22T07:16:10.369 INFO:tasks.workunit.client.1.smithi069.stdout:1/819: creat d1/d13/d36/de5/d4c/d55/f109 x:0 0 0 2024-03-22T07:16:10.370 INFO:tasks.workunit.client.1.smithi069.stdout:1/820: symlink d1/d13/d36/de5/d88/d41/d1e/l10a 0 2024-03-22T07:16:10.370 INFO:tasks.workunit.client.1.smithi069.stdout:1/821: write d1/d2a/f4e [1073235,53224] 0 2024-03-22T07:16:10.370 INFO:tasks.workunit.client.1.smithi069.stdout:1/822: creat d1/d13/d36/de5/d4c/d55/f10b x:0 0 0 2024-03-22T07:16:10.371 INFO:tasks.workunit.client.1.smithi069.stdout:3/999: dread d5/da2/f10b [0,4194304] 0 2024-03-22T07:16:10.372 INFO:tasks.workunit.client.1.smithi069.stdout:1/823: rename d1/d13/d36/de5/d88/d41/d64/fc3 to d1/d13/dc7/dd2/f10c 0 2024-03-22T07:16:10.372 INFO:tasks.workunit.client.1.smithi069.stdout:1/824: chown d1/d2a/db6/df8/d73/df0/dbc/c103 8689 1 2024-03-22T07:16:10.373 INFO:tasks.workunit.client.1.smithi069.stdout:1/825: creat d1/d13/dc7/dd2/f10d x:0 0 0 2024-03-22T07:16:10.374 INFO:tasks.workunit.client.1.smithi069.stdout:1/826: link d1/d13/d36/de5/d88/d41/lc6 d1/d2a/d5e/d93/l10e 0 2024-03-22T07:16:10.375 INFO:tasks.workunit.client.1.smithi069.stdout:1/827: rename d1/d13/d82/f83 to d1/d2a/d5e/f10f 0 2024-03-22T07:16:10.423 INFO:tasks.workunit.client.1.smithi069.stdout:1/828: dwrite d1/d13/d36/de5/fac [0,4194304] 0 2024-03-22T07:16:10.424 INFO:tasks.workunit.client.1.smithi069.stdout:1/829: chown d1/d13/d36/de5/d4c/d55/d5f/f108 472 1 2024-03-22T07:16:10.438 INFO:tasks.workunit.client.1.smithi069.stdout:1/830: dwrite d1/d13/d36/de5/d4c/d55/f10b [0,4194304] 0 2024-03-22T07:16:10.438 INFO:tasks.workunit.client.1.smithi069.stdout:1/831: write d1/d13/d36/f81 [644412,69351] 0 2024-03-22T07:16:10.438 INFO:tasks.workunit.client.1.smithi069.stdout:1/832: stat d1/d13/d36/d5b/f8e 0 2024-03-22T07:16:10.439 INFO:tasks.workunit.client.1.smithi069.stdout:1/833: write d1/d13/d36/de5/d4c/d55/fea [1646002,57730] 0 2024-03-22T07:16:10.440 INFO:tasks.workunit.client.1.smithi069.stdout:1/834: rename d1/d2a/d5e/ce8 to d1/d13/c110 0 2024-03-22T07:16:10.440 INFO:tasks.workunit.client.1.smithi069.stdout:1/835: fsync d1/d13/d36/ffe 0 2024-03-22T07:16:10.441 INFO:tasks.workunit.client.1.smithi069.stdout:1/836: rename d1/d2a/d84/cb2 to d1/d13/d36/de5/d88/d41/d64/c111 0 2024-03-22T07:16:10.441 INFO:tasks.workunit.client.1.smithi069.stdout:1/837: chown d1/d13/d36/de5/d88/d41/lc6 2613 1 2024-03-22T07:16:10.441 INFO:tasks.workunit.client.1.smithi069.stdout:1/838: chown d1/d2a/d5e/d9b/l106 1767 1 2024-03-22T07:16:10.442 INFO:tasks.workunit.client.1.smithi069.stdout:1/839: mkdir d1/d2a/db6/df8/d73/df0/d112 0 2024-03-22T07:16:10.442 INFO:tasks.workunit.client.1.smithi069.stdout:1/840: write d1/d13/d36/de5/d4c/d55/f62 [979927,76471] 0 2024-03-22T07:16:10.451 INFO:tasks.workunit.client.1.smithi069.stdout:1/841: write d1/d13/d36/f68 [6678933,91501] 0 2024-03-22T07:16:10.451 INFO:tasks.workunit.client.1.smithi069.stdout:1/842: chown d1/d2a/d84/fdc 30820077 1 2024-03-22T07:16:10.452 INFO:tasks.workunit.client.1.smithi069.stdout:1/843: mknod d1/d13/d36/de5/d88/d41/c113 0 2024-03-22T07:16:10.453 INFO:tasks.workunit.client.1.smithi069.stdout:1/844: unlink d1/d13/d36/de5/d88/d41/d64/ce6 0 2024-03-22T07:16:10.454 INFO:tasks.workunit.client.1.smithi069.stdout:1/845: mkdir d1/d13/d36/de5/d88/dae/de7/d114 0 2024-03-22T07:16:10.455 INFO:tasks.workunit.client.1.smithi069.stdout:1/846: getdents d1 0 2024-03-22T07:16:10.457 INFO:tasks.workunit.client.1.smithi069.stdout:1/847: rename d1/d13/d36/de5/d88/d41/d7a/c69 to d1/d2a/db6/df8/d73/df0/c115 0 2024-03-22T07:16:10.457 INFO:tasks.workunit.client.1.smithi069.stdout:1/848: stat d1/d2a/d5e/d93/l10e 0 2024-03-22T07:16:10.463 INFO:tasks.workunit.client.1.smithi069.stdout:1/849: dread d1/d2a/f32 [0,4194304] 0 2024-03-22T07:16:10.470 INFO:tasks.workunit.client.1.smithi069.stdout:1/850: dread d1/d13/d36/de5/d4c/d55/f56 [0,4194304] 0 2024-03-22T07:16:10.479 INFO:tasks.workunit.client.1.smithi069.stdout:7/803: sync 2024-03-22T07:16:10.479 INFO:tasks.workunit.client.1.smithi069.stdout:4/945: sync 2024-03-22T07:16:10.479 INFO:tasks.workunit.client.1.smithi069.stdout:4/946: fdatasync d6/d12/d18/d21/f6a 0 2024-03-22T07:16:10.492 INFO:tasks.workunit.client.1.smithi069.stdout:7/804: write d9/db/f17 [2516933,80802] 0 2024-03-22T07:16:10.494 INFO:tasks.workunit.client.1.smithi069.stdout:1/851: dwrite d1/d13/d36/fb4 [0,4194304] 0 2024-03-22T07:16:10.494 INFO:tasks.workunit.client.1.smithi069.stdout:1/852: write d1/d2a/d84/fdc [741719,23228] 0 2024-03-22T07:16:10.498 INFO:tasks.workunit.client.1.smithi069.stdout:1/853: getdents d1/d13/d36/de5/d88/d41/d64 0 2024-03-22T07:16:10.498 INFO:tasks.workunit.client.1.smithi069.stdout:1/854: chown d1/d13 96 1 2024-03-22T07:16:10.498 INFO:tasks.workunit.client.1.smithi069.stdout:1/855: write d1/d13/f2d [2670685,117767] 0 2024-03-22T07:16:10.509 INFO:tasks.workunit.client.1.smithi069.stdout:7/805: dwrite d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/dce/fd0 [0,4194304] 0 2024-03-22T07:16:10.510 INFO:tasks.workunit.client.1.smithi069.stdout:7/806: creat d9/db/d1a/db1/d57/d44/d53/dc7/f100 x:0 0 0 2024-03-22T07:16:10.510 INFO:tasks.workunit.client.1.smithi069.stdout:7/807: readlink d9/db/d1a/l1b 0 2024-03-22T07:16:10.510 INFO:tasks.workunit.client.1.smithi069.stdout:7/808: write d9/db/f16 [4711286,84818] 0 2024-03-22T07:16:10.510 INFO:tasks.workunit.client.1.smithi069.stdout:7/809: write d9/db/d1a/db1/d57/d44/d53/d1d/f87 [1838090,97018] 0 2024-03-22T07:16:10.511 INFO:tasks.workunit.client.1.smithi069.stdout:7/810: symlink d9/de4/df5/l101 0 2024-03-22T07:16:10.517 INFO:tasks.workunit.client.1.smithi069.stdout:1/856: dwrite d1/d13/d36/de5/d88/d41/d1e/fd5 [0,4194304] 0 2024-03-22T07:16:10.520 INFO:tasks.workunit.client.1.smithi069.stdout:1/857: mknod d1/d2a/d84/dd1/ddb/c116 0 2024-03-22T07:16:10.520 INFO:tasks.workunit.client.1.smithi069.stdout:1/858: mknod d1/d13/d36/de5/c117 0 2024-03-22T07:16:10.522 INFO:tasks.workunit.client.1.smithi069.stdout:1/859: link d1/c4f d1/d13/d36/de5/d88/dec/c118 0 2024-03-22T07:16:10.524 INFO:tasks.workunit.client.1.smithi069.stdout:7/811: write d9/db/d1a/fc3 [1543648,35419] 0 2024-03-22T07:16:10.524 INFO:tasks.workunit.client.1.smithi069.stdout:7/812: creat d9/d8e/f102 x:0 0 0 2024-03-22T07:16:10.525 INFO:tasks.workunit.client.1.smithi069.stdout:7/813: mkdir d9/d8e/df7/d103 0 2024-03-22T07:16:10.529 INFO:tasks.workunit.client.1.smithi069.stdout:1/860: dread d1/d13/d36/f3e [0,4194304] 0 2024-03-22T07:16:10.532 INFO:tasks.workunit.client.1.smithi069.stdout:7/814: dread d9/db/d1a/db1/d57/d44/d53/dc7/fcc [0,4194304] 0 2024-03-22T07:16:10.538 INFO:tasks.workunit.client.1.smithi069.stdout:1/861: dread d1/d13/d36/f67 [0,4194304] 0 2024-03-22T07:16:10.538 INFO:tasks.workunit.client.1.smithi069.stdout:1/862: creat d1/d2a/db6/df8/d73/df0/f119 x:0 0 0 2024-03-22T07:16:10.550 INFO:tasks.workunit.client.1.smithi069.stdout:7/815: dwrite f4 [0,4194304] 0 2024-03-22T07:16:10.550 INFO:tasks.workunit.client.1.smithi069.stdout:7/816: dread - d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/df9/fd5 zero size 2024-03-22T07:16:10.551 INFO:tasks.workunit.client.1.smithi069.stdout:7/817: creat d9/d8e/df7/d103/f104 x:0 0 0 2024-03-22T07:16:10.553 INFO:tasks.workunit.client.1.smithi069.stdout:7/818: mkdir d9/dff/d105 0 2024-03-22T07:16:10.554 INFO:tasks.workunit.client.1.smithi069.stdout:7/819: getdents d9/db/d1a/db1/d57/d44/d53/d2e 0 2024-03-22T07:16:10.560 INFO:tasks.workunit.client.1.smithi069.stdout:9/797: sync 2024-03-22T07:16:10.560 INFO:tasks.workunit.client.1.smithi069.stdout:6/713: sync 2024-03-22T07:16:10.560 INFO:tasks.workunit.client.1.smithi069.stdout:6/714: fsync d4/d7/d5b/f97 0 2024-03-22T07:16:10.562 INFO:tasks.workunit.client.1.smithi069.stdout:9/798: creat d2/dd/d3c/d48/d7c/ffe x:0 0 0 2024-03-22T07:16:10.563 INFO:tasks.workunit.client.1.smithi069.stdout:7/820: write d9/db/d1a/db1/d57/d44/f47 [3929500,72401] 0 2024-03-22T07:16:10.563 INFO:tasks.workunit.client.1.smithi069.stdout:7/821: truncate d9/db/d1a/db1/d57/d75/d7a/fa2 655443 0 2024-03-22T07:16:10.563 INFO:tasks.workunit.client.1.smithi069.stdout:7/822: write d9/db/d1a/db1/d57/d44/d53/dc7/f100 [839774,56035] 0 2024-03-22T07:16:10.564 INFO:tasks.workunit.client.1.smithi069.stdout:7/823: stat d9/db/d1a/fc3 0 2024-03-22T07:16:10.564 INFO:tasks.workunit.client.1.smithi069.stdout:7/824: creat d9/db/d1a/db1/d57/d44/d53/d74/d30/f106 x:0 0 0 2024-03-22T07:16:10.564 INFO:tasks.workunit.client.1.smithi069.stdout:7/825: chown d9/db/d1a/db1/d57/d75/de2 185 1 2024-03-22T07:16:10.564 INFO:tasks.workunit.client.1.smithi069.stdout:6/715: rename d4/l66 to d4/dd/d41/d78/le1 0 2024-03-22T07:16:10.564 INFO:tasks.workunit.client.1.smithi069.stdout:6/716: write d4/d7/d5b/f97 [338035,59652] 0 2024-03-22T07:16:10.564 INFO:tasks.workunit.client.1.smithi069.stdout:9/799: truncate d2/d83/f92 4857279 0 2024-03-22T07:16:10.565 INFO:tasks.workunit.client.1.smithi069.stdout:7/826: symlink d9/d42/l107 0 2024-03-22T07:16:10.567 INFO:tasks.workunit.client.1.smithi069.stdout:6/717: creat d4/d7/da/d3f/fe2 x:0 0 0 2024-03-22T07:16:10.568 INFO:tasks.workunit.client.1.smithi069.stdout:9/800: mkdir d2/dd/d3c/d48/d7c/dff 0 2024-03-22T07:16:10.570 INFO:tasks.workunit.client.1.smithi069.stdout:6/718: creat d4/fe3 x:0 0 0 2024-03-22T07:16:10.570 INFO:tasks.workunit.client.1.smithi069.stdout:6/719: fdatasync d4/f60 0 2024-03-22T07:16:10.572 INFO:tasks.workunit.client.1.smithi069.stdout:9/801: link d2/ff6 d2/dd/d3c/d2e/d84/f100 0 2024-03-22T07:16:10.572 INFO:tasks.workunit.client.1.smithi069.stdout:9/802: chown d2/d21/f73 30 1 2024-03-22T07:16:10.572 INFO:tasks.workunit.client.1.smithi069.stdout:9/803: fdatasync d2/dd/d3c/d2e/f65 0 2024-03-22T07:16:10.576 INFO:tasks.workunit.client.1.smithi069.stdout:7/827: mknod d9/db/d1a/db1/d57/d44/d53/dc7/c108 0 2024-03-22T07:16:10.576 INFO:tasks.workunit.client.1.smithi069.stdout:7/828: write d9/db/d1a/db1/d57/d44/d53/d74/fb2 [756051,108425] 0 2024-03-22T07:16:10.577 INFO:tasks.workunit.client.1.smithi069.stdout:6/720: mknod d4/dd/d12/d44/d58/db4/dd2/ce4 0 2024-03-22T07:16:10.582 INFO:tasks.workunit.client.1.smithi069.stdout:7/829: dread d9/db/d1a/db1/d57/d44/d53/d74/fb2 [0,4194304] 0 2024-03-22T07:16:10.583 INFO:tasks.workunit.client.1.smithi069.stdout:7/830: write d9/db/d1a/db1/fef [182302,76012] 0 2024-03-22T07:16:10.583 INFO:tasks.workunit.client.1.smithi069.stdout:7/831: write d9/db/d1a/db1/d57/d44/d53/d74/fe0 [665259,4770] 0 2024-03-22T07:16:10.595 INFO:tasks.workunit.client.1.smithi069.stdout:6/721: dwrite d4/dd/d12/d13/f86 [0,4194304] 0 2024-03-22T07:16:10.596 INFO:tasks.workunit.client.1.smithi069.stdout:6/722: getdents d4/dd/d12/d2b/d72/d73/d9d/dc3 0 2024-03-22T07:16:10.596 INFO:tasks.workunit.client.1.smithi069.stdout:9/804: dwrite d2/d71/fd4 [0,4194304] 0 2024-03-22T07:16:10.600 INFO:tasks.workunit.client.1.smithi069.stdout:6/723: unlink d4/d7/da/d3f/f61 0 2024-03-22T07:16:10.600 INFO:tasks.workunit.client.1.smithi069.stdout:6/724: getdents d4/dd/d12/d44/d7f/dcf 0 2024-03-22T07:16:10.600 INFO:tasks.workunit.client.1.smithi069.stdout:6/725: mknod d4/dd/d41/ce5 0 2024-03-22T07:16:10.604 INFO:tasks.workunit.client.1.smithi069.stdout:6/726: dread d4/f6c [0,4194304] 0 2024-03-22T07:16:10.605 INFO:tasks.workunit.client.1.smithi069.stdout:6/727: creat d4/dd/d12/d13/fe6 x:0 0 0 2024-03-22T07:16:10.605 INFO:tasks.workunit.client.1.smithi069.stdout:6/728: write d4/dd/d12/d13/f59 [1797884,73819] 0 2024-03-22T07:16:10.605 INFO:tasks.workunit.client.1.smithi069.stdout:7/832: dwrite d9/db/d1a/fc3 [0,4194304] 0 2024-03-22T07:16:10.605 INFO:tasks.workunit.client.1.smithi069.stdout:7/833: write d9/f6f [4233427,12367] 0 2024-03-22T07:16:10.619 INFO:tasks.workunit.client.1.smithi069.stdout:7/834: dread d9/db/d1a/db1/d57/d44/d53/d74/d6e/f83 [0,4194304] 0 2024-03-22T07:16:10.620 INFO:tasks.workunit.client.1.smithi069.stdout:6/729: dwrite d4/dd/d12/d44/d7f/fba [8388608,4194304] 0 2024-03-22T07:16:10.620 INFO:tasks.workunit.client.1.smithi069.stdout:7/835: creat d9/db/d1a/db1/db3/f109 x:0 0 0 2024-03-22T07:16:10.620 INFO:tasks.workunit.client.1.smithi069.stdout:7/836: write d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/dce/fd0 [4501049,85731] 0 2024-03-22T07:16:10.620 INFO:tasks.workunit.client.1.smithi069.stdout:7/837: truncate d9/db/d1a/db1/d57/d75/d7a/dcf/ff3 760089 0 2024-03-22T07:16:10.621 INFO:tasks.workunit.client.1.smithi069.stdout:7/838: fsync d9/db/d1a/db1/db3/fcd 0 2024-03-22T07:16:10.621 INFO:tasks.workunit.client.1.smithi069.stdout:6/730: creat d4/d7/da/dc6/dc7/fe7 x:0 0 0 2024-03-22T07:16:10.621 INFO:tasks.workunit.client.1.smithi069.stdout:6/731: rename d4/dd/d12/d2b/ld4 to d4/dd/d12/d44/le8 0 2024-03-22T07:16:10.641 INFO:tasks.workunit.client.1.smithi069.stdout:7/839: dwrite f4 [0,4194304] 0 2024-03-22T07:16:10.641 INFO:tasks.workunit.client.1.smithi069.stdout:7/840: fdatasync d9/db/d1a/db1/d57/d44/d53/d74/f8d 0 2024-03-22T07:16:10.643 INFO:tasks.workunit.client.1.smithi069.stdout:7/841: creat d9/db/d1a/db1/d57/d44/f10a x:0 0 0 2024-03-22T07:16:10.643 INFO:tasks.workunit.client.1.smithi069.stdout:7/842: write d9/db/d48/f5b [72029,29844] 0 2024-03-22T07:16:10.647 INFO:tasks.workunit.client.1.smithi069.stdout:6/732: dwrite d4/dd/d12/fb5 [0,4194304] 0 2024-03-22T07:16:10.648 INFO:tasks.workunit.client.1.smithi069.stdout:6/733: truncate d4/dd/d12/f32 4414430 0 2024-03-22T07:16:10.655 INFO:tasks.workunit.client.1.smithi069.stdout:6/734: write d4/dd/d12/d2b/f3a [2412140,82523] 0 2024-03-22T07:16:10.726 INFO:tasks.workunit.client.1.smithi069.stdout:2/727: sync 2024-03-22T07:16:10.726 INFO:tasks.workunit.client.1.smithi069.stdout:5/752: sync 2024-03-22T07:16:10.726 INFO:tasks.workunit.client.1.smithi069.stdout:2/728: chown d18/d4d/d27/d33/d60/d64/d89/d8b 2143 1 2024-03-22T07:16:10.726 INFO:tasks.workunit.client.1.smithi069.stdout:5/753: chown d1/d10/d44/dd8/d101 1 1 2024-03-22T07:16:10.726 INFO:tasks.workunit.client.1.smithi069.stdout:2/729: chown d18/d4d/d3e/d40/d96 229 1 2024-03-22T07:16:10.726 INFO:tasks.workunit.client.1.smithi069.stdout:0/929: sync 2024-03-22T07:16:10.726 INFO:tasks.workunit.client.1.smithi069.stdout:0/930: dread - d3/d7/d15/dad/fd0 zero size 2024-03-22T07:16:10.726 INFO:tasks.workunit.client.1.smithi069.stdout:0/931: readlink d3/d7/d24/l44 0 2024-03-22T07:16:10.726 INFO:tasks.workunit.client.1.smithi069.stdout:0/932: stat d3/d7/d90/fa6 0 2024-03-22T07:16:10.735 INFO:tasks.workunit.client.1.smithi069.stdout:5/754: write d1/d6/d26/d33/d5d/d72/d78/fd3 [1640235,7919] 0 2024-03-22T07:16:10.738 INFO:tasks.workunit.client.1.smithi069.stdout:5/755: link d1/d7/c2b d1/d6/d26/d3a/dbd/c106 0 2024-03-22T07:16:10.738 INFO:tasks.workunit.client.1.smithi069.stdout:5/756: chown d1/d6/d26/d33/f38 197738 1 2024-03-22T07:16:10.740 INFO:tasks.workunit.client.1.smithi069.stdout:2/730: write f1 [3607611,98535] 0 2024-03-22T07:16:10.742 INFO:tasks.workunit.client.1.smithi069.stdout:2/731: symlink d18/d4d/d27/d33/d60/d64/d89/dcb/le8 0 2024-03-22T07:16:10.742 INFO:tasks.workunit.client.1.smithi069.stdout:2/732: write d18/d4d/d27/d77/f9f [462524,104433] 0 2024-03-22T07:16:10.745 INFO:tasks.workunit.client.1.smithi069.stdout:2/733: symlink d18/d4d/d27/d33/d4f/le9 0 2024-03-22T07:16:10.745 INFO:tasks.workunit.client.1.smithi069.stdout:2/734: chown d18/d4d/d25/l8e 49052 1 2024-03-22T07:16:10.745 INFO:tasks.workunit.client.1.smithi069.stdout:2/735: write d18/d4d/f57 [3155,62500] 0 2024-03-22T07:16:10.746 INFO:tasks.workunit.client.1.smithi069.stdout:2/736: rename d18/d4d/d27/d33/d60/d63/d80/f95 to d18/d4d/d27/d33/d60/d64/d89/d8b/dab/fea 0 2024-03-22T07:16:10.749 INFO:tasks.workunit.client.1.smithi069.stdout:5/757: dread d1/d6/d21/fb1 [0,4194304] 0 2024-03-22T07:16:10.755 INFO:tasks.workunit.client.1.smithi069.stdout:0/933: dwrite d3/d7/d15/d29/d5a/f6c [0,4194304] 0 2024-03-22T07:16:10.760 INFO:tasks.workunit.client.1.smithi069.stdout:5/758: dread d1/d6/fdf [0,4194304] 0 2024-03-22T07:16:10.761 INFO:tasks.workunit.client.1.smithi069.stdout:0/934: rename d3/d7/d15/dad/fd0 to d3/d7/d5e/f135 0 2024-03-22T07:16:10.763 INFO:tasks.workunit.client.1.smithi069.stdout:5/759: truncate d1/d6/d86/d99/f4b 354294 0 2024-03-22T07:16:10.768 INFO:tasks.workunit.client.1.smithi069.stdout:2/737: dwrite d18/f1b [0,4194304] 0 2024-03-22T07:16:10.768 INFO:tasks.workunit.client.1.smithi069.stdout:2/738: write d18/d4d/d27/d56/f94 [798451,9390] 0 2024-03-22T07:16:10.768 INFO:tasks.workunit.client.1.smithi069.stdout:2/739: fdatasync d18/d4d/d27/d33/d60/dbd/fe6 0 2024-03-22T07:16:10.772 INFO:tasks.workunit.client.1.smithi069.stdout:2/740: rename d18/d91/lbc to d18/d4d/d27/d33/d60/d63/d73/leb 0 2024-03-22T07:16:10.774 INFO:tasks.workunit.client.1.smithi069.stdout:2/741: rmdir d18/d4d/d34 39 2024-03-22T07:16:10.775 INFO:tasks.workunit.client.1.smithi069.stdout:2/742: mkdir d18/d4d/dec 0 2024-03-22T07:16:10.775 INFO:tasks.workunit.client.1.smithi069.stdout:2/743: write d18/d4d/d3e/d40/f6f [1167565,46561] 0 2024-03-22T07:16:10.776 INFO:tasks.workunit.client.1.smithi069.stdout:2/744: dread - d18/d4d/d27/d72/fb7 zero size 2024-03-22T07:16:10.776 INFO:tasks.workunit.client.1.smithi069.stdout:2/745: write d18/d4d/d27/d33/d60/d63/fe3 [416822,15309] 0 2024-03-22T07:16:10.778 INFO:tasks.workunit.client.1.smithi069.stdout:2/746: creat d18/d4d/d3e/d40/d96/fed x:0 0 0 2024-03-22T07:16:10.781 INFO:tasks.workunit.client.1.smithi069.stdout:0/935: dwrite d3/d7/d15/dad/fae [0,4194304] 0 2024-03-22T07:16:10.783 INFO:tasks.workunit.client.1.smithi069.stdout:2/747: dread d18/d4d/d27/d77/f9f [0,4194304] 0 2024-03-22T07:16:10.792 INFO:tasks.workunit.client.1.smithi069.stdout:0/936: write d3/d9a/fda [935527,45869] 0 2024-03-22T07:16:10.792 INFO:tasks.workunit.client.1.smithi069.stdout:0/937: readlink d3/d7/d15/dad/l134 0 2024-03-22T07:16:10.794 INFO:tasks.workunit.client.1.smithi069.stdout:0/938: truncate d3/d7/d15/fe1 2672877 0 2024-03-22T07:16:10.795 INFO:tasks.workunit.client.1.smithi069.stdout:0/939: creat d3/d7/d15/d29/dcf/f136 x:0 0 0 2024-03-22T07:16:10.797 INFO:tasks.workunit.client.1.smithi069.stdout:0/940: mknod d3/d7/d24/c137 0 2024-03-22T07:16:10.799 INFO:tasks.workunit.client.1.smithi069.stdout:2/748: dwrite d18/d4d/d3e/d40/d96/fed [0,4194304] 0 2024-03-22T07:16:10.802 INFO:tasks.workunit.client.1.smithi069.stdout:2/749: truncate d18/f1b 425927 0 2024-03-22T07:16:10.805 INFO:tasks.workunit.client.1.smithi069.stdout:2/750: mkdir d18/d4d/d27/d33/d60/d64/d89/d8b/dab/dee 0 2024-03-22T07:16:10.805 INFO:tasks.workunit.client.1.smithi069.stdout:2/751: fsync d18/d4d/d3e/d40/f61 0 2024-03-22T07:16:10.805 INFO:tasks.workunit.client.1.smithi069.stdout:2/752: read d18/d4d/d27/d33/d60/d64/d89/f90 [25574,16613] 0 2024-03-22T07:16:10.805 INFO:tasks.workunit.client.1.smithi069.stdout:2/753: fsync d18/f70 0 2024-03-22T07:16:10.807 INFO:tasks.workunit.client.1.smithi069.stdout:0/941: write d3/d7/d15/d29/f30 [468381,44556] 0 2024-03-22T07:16:10.807 INFO:tasks.workunit.client.1.smithi069.stdout:2/754: dread d18/f70 [0,4194304] 0 2024-03-22T07:16:10.809 INFO:tasks.workunit.client.1.smithi069.stdout:0/942: symlink d3/l138 0 2024-03-22T07:16:10.810 INFO:tasks.workunit.client.1.smithi069.stdout:0/943: getdents d3/d9a/d133 0 2024-03-22T07:16:10.811 INFO:tasks.workunit.client.1.smithi069.stdout:0/944: creat d3/d7/d15/d29/d5a/d84/d10f/f139 x:0 0 0 2024-03-22T07:16:10.811 INFO:tasks.workunit.client.1.smithi069.stdout:0/945: write d3/d7/d15/d29/dcf/d114/ffb [173331,122168] 0 2024-03-22T07:16:10.812 INFO:tasks.workunit.client.1.smithi069.stdout:0/946: read d3/d7/d15/d29/d5a/d84/f52 [5581415,88715] 0 2024-03-22T07:16:10.812 INFO:tasks.workunit.client.1.smithi069.stdout:2/755: mkdir d18/d4d/d27/d33/d60/d63/d93/def 0 2024-03-22T07:16:10.821 INFO:tasks.workunit.client.1.smithi069.stdout:2/756: read f6 [8245673,20853] 0 2024-03-22T07:16:10.839 INFO:tasks.workunit.client.1.smithi069.stdout:0/947: dwrite d3/d7/d15/dce/f115 [0,4194304] 0 2024-03-22T07:16:10.839 INFO:tasks.workunit.client.1.smithi069.stdout:2/757: dwrite d18/d4d/d27/d33/d60/d64/d89/f90 [0,4194304] 0 2024-03-22T07:16:10.839 INFO:tasks.workunit.client.1.smithi069.stdout:2/758: readlink d18/d4d/d27/d33/d60/d64/l71 0 2024-03-22T07:16:10.842 INFO:tasks.workunit.client.1.smithi069.stdout:0/948: creat d3/d7/d15/d29/d5a/d5d/db4/ded/f13a x:0 0 0 2024-03-22T07:16:10.843 INFO:tasks.workunit.client.1.smithi069.stdout:2/759: creat d18/d4d/d27/d33/d60/d63/d73/dc9/ff0 x:0 0 0 2024-03-22T07:16:10.843 INFO:tasks.workunit.client.1.smithi069.stdout:2/760: write d18/d4d/d27/d77/f9f [1085382,106719] 0 2024-03-22T07:16:10.843 INFO:tasks.workunit.client.1.smithi069.stdout:2/761: dread - d18/d4d/d27/d33/d60/d63/d73/dc9/ff0 zero size 2024-03-22T07:16:10.843 INFO:tasks.workunit.client.1.smithi069.stdout:2/762: stat d18/d4d/d27/d33/d3d/dc6/fe7 0 2024-03-22T07:16:10.844 INFO:tasks.workunit.client.1.smithi069.stdout:2/763: chown d18/d4d/d27/d33/d60/d64/fda 134 1 2024-03-22T07:16:10.845 INFO:tasks.workunit.client.1.smithi069.stdout:2/764: mkdir d18/d4d/d3e/d40/df1 0 2024-03-22T07:16:10.846 INFO:tasks.workunit.client.1.smithi069.stdout:2/765: creat d18/d4d/d27/d33/d60/d64/dcd/ff2 x:0 0 0 2024-03-22T07:16:10.846 INFO:tasks.workunit.client.1.smithi069.stdout:2/766: creat d18/d4d/d27/d33/ff3 x:0 0 0 2024-03-22T07:16:10.848 INFO:tasks.workunit.client.1.smithi069.stdout:2/767: truncate d18/d4d/d27/d33/d60/f5a 3809445 0 2024-03-22T07:16:10.849 INFO:tasks.workunit.client.1.smithi069.stdout:2/768: truncate d18/d4d/f24 594626 0 2024-03-22T07:16:10.849 INFO:tasks.workunit.client.1.smithi069.stdout:0/949: dread d3/d7/d15/d29/d6e/f7a [0,4194304] 0 2024-03-22T07:16:10.849 INFO:tasks.workunit.client.1.smithi069.stdout:0/950: fdatasync d3/d7/d15/fe1 0 2024-03-22T07:16:10.850 INFO:tasks.workunit.client.1.smithi069.stdout:2/769: mkdir d18/d4d/d27/d33/d4f/ddd/df4 0 2024-03-22T07:16:10.850 INFO:tasks.workunit.client.1.smithi069.stdout:2/770: read f6 [6033844,81163] 0 2024-03-22T07:16:10.851 INFO:tasks.workunit.client.1.smithi069.stdout:0/951: symlink d3/d7/d24/l13b 0 2024-03-22T07:16:10.851 INFO:tasks.workunit.client.1.smithi069.stdout:0/952: write d3/d7/d15/f20 [2700226,86935] 0 2024-03-22T07:16:10.852 INFO:tasks.workunit.client.1.smithi069.stdout:2/771: symlink d18/d4d/d27/d33/d3d/dc6/dd1/lf5 0 2024-03-22T07:16:10.852 INFO:tasks.workunit.client.1.smithi069.stdout:2/772: write d18/d4d/f22 [7616430,83694] 0 2024-03-22T07:16:10.855 INFO:tasks.workunit.client.1.smithi069.stdout:2/773: truncate d18/d4d/d27/d77/f9f 472842 0 2024-03-22T07:16:10.855 INFO:tasks.workunit.client.1.smithi069.stdout:2/774: fsync d18/d4d/d27/d33/d60/d63/d93/fcf 0 2024-03-22T07:16:10.862 INFO:tasks.workunit.client.1.smithi069.stdout:0/953: read d3/d7/d17/d1c/fb3 [5880985,20835] 0 2024-03-22T07:16:10.868 INFO:tasks.workunit.client.1.smithi069.stdout:4/947: sync 2024-03-22T07:16:10.868 INFO:tasks.workunit.client.1.smithi069.stdout:4/948: stat d6/dc4/de1/f10b 0 2024-03-22T07:16:10.869 INFO:tasks.workunit.client.1.smithi069.stdout:4/949: creat d6/d16/d68/df8/f143 x:0 0 0 2024-03-22T07:16:10.871 INFO:tasks.workunit.client.1.smithi069.stdout:4/950: rename d6/d16/f88 to d6/d9b/d1f/d3f/f144 0 2024-03-22T07:16:10.872 INFO:tasks.workunit.client.1.smithi069.stdout:4/951: rmdir d6/daa/dfd 39 2024-03-22T07:16:10.877 INFO:tasks.workunit.client.1.smithi069.stdout:2/775: dwrite d18/d4d/d3e/d40/f61 [0,4194304] 0 2024-03-22T07:16:10.878 INFO:tasks.workunit.client.1.smithi069.stdout:0/954: dwrite d3/d7/d15/fdb [0,4194304] 0 2024-03-22T07:16:10.878 INFO:tasks.workunit.client.1.smithi069.stdout:0/955: getdents d3/d7/d15/d29/d5a/d84/df4/d129 0 2024-03-22T07:16:10.879 INFO:tasks.workunit.client.1.smithi069.stdout:2/776: read d18/d4d/d27/f8c [601014,112459] 0 2024-03-22T07:16:10.879 INFO:tasks.workunit.client.1.smithi069.stdout:2/777: stat d18/d4d/d3e/d40 0 2024-03-22T07:16:10.879 INFO:tasks.workunit.client.1.smithi069.stdout:2/778: chown d18/d4d/d27/d33/d60/d63/d73/leb 132 1 2024-03-22T07:16:10.881 INFO:tasks.workunit.client.1.smithi069.stdout:4/952: write d6/d12/f6f [3230792,110716] 0 2024-03-22T07:16:10.884 INFO:tasks.workunit.client.1.smithi069.stdout:4/953: mknod d6/daa/dfd/c145 0 2024-03-22T07:16:10.888 INFO:tasks.workunit.client.1.smithi069.stdout:4/954: rename d6/d9b/d1f/f62 to d6/d16/d68/df8/d11a/d13e/f146 0 2024-03-22T07:16:10.888 INFO:tasks.workunit.client.1.smithi069.stdout:4/955: write d6/d9b/d1f/d58/d7c/d90/f95 [918875,47387] 0 2024-03-22T07:16:10.890 INFO:tasks.workunit.client.1.smithi069.stdout:4/956: creat d6/d9b/d1f/d58/f147 x:0 0 0 2024-03-22T07:16:10.890 INFO:tasks.workunit.client.1.smithi069.stdout:4/957: stat d6/d16/d83/d9a/f121 0 2024-03-22T07:16:10.890 INFO:tasks.workunit.client.1.smithi069.stdout:4/958: write d6/d12/d18/d21/f103 [860701,38347] 0 2024-03-22T07:16:10.890 INFO:tasks.workunit.client.1.smithi069.stdout:4/959: stat d6/d9b/d1f/f7a 0 2024-03-22T07:16:10.890 INFO:tasks.workunit.client.1.smithi069.stdout:4/960: write d6/d12/d18/d21/f129 [470506,27678] 0 2024-03-22T07:16:10.891 INFO:tasks.workunit.client.1.smithi069.stdout:4/961: dread d6/fbb [0,4194304] 0 2024-03-22T07:16:10.893 INFO:tasks.workunit.client.1.smithi069.stdout:4/962: truncate d6/d16/d3a/db4/dfb/ff 830746 0 2024-03-22T07:16:10.893 INFO:tasks.workunit.client.1.smithi069.stdout:4/963: dread - d6/d16/d83/ff4 zero size 2024-03-22T07:16:10.895 INFO:tasks.workunit.client.1.smithi069.stdout:4/964: rmdir d6/d9b/d1f/d3f/d51 39 2024-03-22T07:16:10.931 INFO:tasks.workunit.client.1.smithi069.stdout:2/779: dwrite d18/f19 [0,4194304] 0 2024-03-22T07:16:10.932 INFO:tasks.workunit.client.1.smithi069.stdout:4/965: dwrite d6/d16/d3a/db4/fff [0,4194304] 0 2024-03-22T07:16:10.935 INFO:tasks.workunit.client.1.smithi069.stdout:2/780: mkdir d18/d4d/d27/d33/d60/dbd/ddc/df6 0 2024-03-22T07:16:10.937 INFO:tasks.workunit.client.1.smithi069.stdout:4/966: mkdir d6/daa/dfd/d148 0 2024-03-22T07:16:10.938 INFO:tasks.workunit.client.1.smithi069.stdout:4/967: mknod d6/daa/dfd/c149 0 2024-03-22T07:16:10.938 INFO:tasks.workunit.client.1.smithi069.stdout:4/968: chown d6/d12/d18/d21/f79 2045454 1 2024-03-22T07:16:10.939 INFO:tasks.workunit.client.1.smithi069.stdout:4/969: truncate d6/d16/d68/df8/f10e 421024 0 2024-03-22T07:16:10.939 INFO:tasks.workunit.client.1.smithi069.stdout:4/970: dread - d6/d12/f80 zero size 2024-03-22T07:16:10.942 INFO:tasks.workunit.client.1.smithi069.stdout:2/781: write d18/d4d/f22 [3358751,94950] 0 2024-03-22T07:16:10.943 INFO:tasks.workunit.client.1.smithi069.stdout:4/971: dread d6/d12/d18/d21/f103 [0,4194304] 0 2024-03-22T07:16:10.943 INFO:tasks.workunit.client.1.smithi069.stdout:2/782: symlink d18/d4d/d27/d33/d60/d63/d93/lf7 0 2024-03-22T07:16:10.945 INFO:tasks.workunit.client.1.smithi069.stdout:2/783: symlink d18/d4d/d27/d33/d60/d63/d80/lf8 0 2024-03-22T07:16:10.946 INFO:tasks.workunit.client.1.smithi069.stdout:2/784: creat d18/d4d/d27/d33/d60/d63/d73/ff9 x:0 0 0 2024-03-22T07:16:10.958 INFO:tasks.workunit.client.1.smithi069.stdout:2/785: write d18/d4d/f28 [322868,118866] 0 2024-03-22T07:16:10.958 INFO:tasks.workunit.client.1.smithi069.stdout:2/786: write d18/d4d/d27/d33/d60/d64/d89/f8d [841816,6969] 0 2024-03-22T07:16:10.958 INFO:tasks.workunit.client.1.smithi069.stdout:2/787: stat d18/d4d/d27/c29 0 2024-03-22T07:16:10.958 INFO:tasks.workunit.client.1.smithi069.stdout:2/788: dread - d18/d4d/f9a zero size 2024-03-22T07:16:10.958 INFO:tasks.workunit.client.1.smithi069.stdout:4/972: dwrite d6/d16/f57 [0,4194304] 0 2024-03-22T07:16:10.958 INFO:tasks.workunit.client.1.smithi069.stdout:4/973: chown d6/d16/d3a/db4/dfb/d23/d59/l13d 838111 1 2024-03-22T07:16:10.959 INFO:tasks.workunit.client.1.smithi069.stdout:2/789: rename d18/d4d/d27/l83 to d18/d4d/d27/d33/d60/d64/d89/d8b/dab/lfa 0 2024-03-22T07:16:10.960 INFO:tasks.workunit.client.1.smithi069.stdout:4/974: creat d6/d16/d3a/db4/dbc/dc6/f14a x:0 0 0 2024-03-22T07:16:10.961 INFO:tasks.workunit.client.1.smithi069.stdout:2/790: link d18/d4d/d27/d72/c7b d18/d4d/d27/d33/d3d/dc6/dd1/cfb 0 2024-03-22T07:16:10.961 INFO:tasks.workunit.client.1.smithi069.stdout:4/975: dread d6/d9b/d1f/f39 [0,4194304] 0 2024-03-22T07:16:10.962 INFO:tasks.workunit.client.1.smithi069.stdout:2/791: mknod d18/d4d/d27/d56/cfc 0 2024-03-22T07:16:10.962 INFO:tasks.workunit.client.1.smithi069.stdout:2/792: dread - d18/d4d/d27/d33/d60/d63/d73/ff9 zero size 2024-03-22T07:16:10.963 INFO:tasks.workunit.client.1.smithi069.stdout:4/976: rename d6/d9b/d1f/d3f/d51/l9f to d6/d16/d3a/db4/dfb/d1d/l14b 0 2024-03-22T07:16:10.964 INFO:tasks.workunit.client.1.smithi069.stdout:2/793: truncate d18/d4d/d25/f36 2279051 0 2024-03-22T07:16:10.986 INFO:tasks.workunit.client.1.smithi069.stdout:9/805: sync 2024-03-22T07:16:10.986 INFO:tasks.workunit.client.1.smithi069.stdout:7/843: sync 2024-03-22T07:16:10.986 INFO:tasks.workunit.client.1.smithi069.stdout:1/863: sync 2024-03-22T07:16:10.986 INFO:tasks.workunit.client.1.smithi069.stdout:1/864: stat d1/d2a/d84/d9a/fed 0 2024-03-22T07:16:10.987 INFO:tasks.workunit.client.1.smithi069.stdout:7/844: chown d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/l3f 27 1 2024-03-22T07:16:10.987 INFO:tasks.workunit.client.1.smithi069.stdout:9/806: truncate d2/dd/fac 84775 0 2024-03-22T07:16:10.987 INFO:tasks.workunit.client.1.smithi069.stdout:7/845: creat d9/db/d48/dab/f10b x:0 0 0 2024-03-22T07:16:10.987 INFO:tasks.workunit.client.1.smithi069.stdout:1/865: stat d1/d2a/d84/dd1/cdd 0 2024-03-22T07:16:10.988 INFO:tasks.workunit.client.1.smithi069.stdout:1/866: fdatasync d1/d13/d36/de5/d88/d41/d1e/f72 0 2024-03-22T07:16:10.990 INFO:tasks.workunit.client.1.smithi069.stdout:1/867: mknod d1/d2a/db6/df8/d73/df0/c11a 0 2024-03-22T07:16:10.990 INFO:tasks.workunit.client.1.smithi069.stdout:1/868: fdatasync d1/d2a/d5e/fba 0 2024-03-22T07:16:10.990 INFO:tasks.workunit.client.1.smithi069.stdout:1/869: dread - d1/d13/dc7/fff zero size 2024-03-22T07:16:10.991 INFO:tasks.workunit.client.1.smithi069.stdout:1/870: dread - d1/d13/d36/de5/d4c/f74 zero size 2024-03-22T07:16:10.991 INFO:tasks.workunit.client.1.smithi069.stdout:4/977: dwrite d6/d16/d3a/db4/dfb/f37 [4194304,4194304] 0 2024-03-22T07:16:10.991 INFO:tasks.workunit.client.1.smithi069.stdout:2/794: dwrite d18/d4d/d34/f7e [0,4194304] 0 2024-03-22T07:16:10.999 INFO:tasks.workunit.client.1.smithi069.stdout:7/846: write d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/df9/f4f [1459999,84068] 0 2024-03-22T07:16:10.999 INFO:tasks.workunit.client.1.smithi069.stdout:7/847: truncate d9/db/d1a/db1/d57/d44/d53/dc7/db5/fbb 1014781 0 2024-03-22T07:16:11.001 INFO:tasks.workunit.client.1.smithi069.stdout:7/848: symlink d9/db/d1a/db1/d57/d75/l10c 0 2024-03-22T07:16:11.003 INFO:tasks.workunit.client.1.smithi069.stdout:4/978: dread d6/d16/d68/df8/d11a/d13e/f146 [0,4194304] 0 2024-03-22T07:16:11.004 INFO:tasks.workunit.client.1.smithi069.stdout:7/849: unlink d9/db/d1a/db1/d57/d44/d53/dc7/f100 0 2024-03-22T07:16:11.004 INFO:tasks.workunit.client.1.smithi069.stdout:1/871: write d1/d2a/db6/fd4 [2893999,19542] 0 2024-03-22T07:16:11.010 INFO:tasks.workunit.client.1.smithi069.stdout:1/872: dread d1/d13/d36/f3e [0,4194304] 0 2024-03-22T07:16:11.010 INFO:tasks.workunit.client.1.smithi069.stdout:2/795: dwrite d18/d4d/d27/d33/d3d/dc6/fe7 [0,4194304] 0 2024-03-22T07:16:11.010 INFO:tasks.workunit.client.1.smithi069.stdout:2/796: readlink d18/d4d/l2a 0 2024-03-22T07:16:11.010 INFO:tasks.workunit.client.1.smithi069.stdout:2/797: stat d18/d4d/l84 0 2024-03-22T07:16:11.013 INFO:tasks.workunit.client.1.smithi069.stdout:1/873: link d1/d13/d36/d5b/d95/c105 d1/d13/d36/de5/d88/d41/c11b 0 2024-03-22T07:16:11.013 INFO:tasks.workunit.client.1.smithi069.stdout:2/798: dread d18/d4d/d34/f53 [0,4194304] 0 2024-03-22T07:16:11.014 INFO:tasks.workunit.client.1.smithi069.stdout:4/979: dread d6/f30 [0,4194304] 0 2024-03-22T07:16:11.014 INFO:tasks.workunit.client.1.smithi069.stdout:9/807: dwrite d2/dd/d3c/d53/d8b/ffc [0,4194304] 0 2024-03-22T07:16:11.015 INFO:tasks.workunit.client.1.smithi069.stdout:1/874: read d1/d13/d36/f81 [197817,111675] 0 2024-03-22T07:16:11.016 INFO:tasks.workunit.client.1.smithi069.stdout:2/799: creat d18/d4d/d27/d33/d60/d64/d89/d8b/dab/dee/ffd x:0 0 0 2024-03-22T07:16:11.016 INFO:tasks.workunit.client.1.smithi069.stdout:4/980: getdents d6/d16/d83/d9a 0 2024-03-22T07:16:11.019 INFO:tasks.workunit.client.1.smithi069.stdout:9/808: mknod d2/dd/d3c/d53/c101 0 2024-03-22T07:16:11.020 INFO:tasks.workunit.client.1.smithi069.stdout:1/875: creat d1/d13/d36/de5/d88/d41/f11c x:0 0 0 2024-03-22T07:16:11.020 INFO:tasks.workunit.client.1.smithi069.stdout:1/876: readlink d1/lf 0 2024-03-22T07:16:11.021 INFO:tasks.workunit.client.1.smithi069.stdout:2/800: creat d18/d4d/d27/d33/d3d/ffe x:0 0 0 2024-03-22T07:16:11.022 INFO:tasks.workunit.client.1.smithi069.stdout:4/981: link d6/d12/l2b d6/daa/dfd/l14c 0 2024-03-22T07:16:11.022 INFO:tasks.workunit.client.1.smithi069.stdout:4/982: chown d6/d12/d18/d21/l43 37 1 2024-03-22T07:16:11.023 INFO:tasks.workunit.client.1.smithi069.stdout:7/850: dwrite d9/db/d1a/db1/d57/d44/d53/d74/d6e/f83 [0,4194304] 0 2024-03-22T07:16:11.023 INFO:tasks.workunit.client.1.smithi069.stdout:7/851: creat d9/d8e/f10d x:0 0 0 2024-03-22T07:16:11.023 INFO:tasks.workunit.client.1.smithi069.stdout:7/852: creat d9/db/d1a/db1/d57/d75/d7a/dcf/f10e x:0 0 0 2024-03-22T07:16:11.023 INFO:tasks.workunit.client.1.smithi069.stdout:1/877: creat d1/d13/dc7/d100/f11d x:0 0 0 2024-03-22T07:16:11.025 INFO:tasks.workunit.client.1.smithi069.stdout:4/983: mknod d6/d9b/d1f/d3f/d51/db8/c14d 0 2024-03-22T07:16:11.025 INFO:tasks.workunit.client.1.smithi069.stdout:4/984: stat d6/d16/d3a/db4/dfb/d23/f3c 0 2024-03-22T07:16:11.033 INFO:tasks.workunit.client.1.smithi069.stdout:4/985: symlink d6/daa/dce/de4/l14e 0 2024-03-22T07:16:11.033 INFO:tasks.workunit.client.1.smithi069.stdout:4/986: chown d6/d12/d18/f11d 2360072 1 2024-03-22T07:16:11.034 INFO:tasks.workunit.client.1.smithi069.stdout:4/987: mkdir d6/d16/d3a/db4/dfb/d23/d59/d120/d12e/d14f 0 2024-03-22T07:16:11.034 INFO:tasks.workunit.client.1.smithi069.stdout:4/988: read d6/d9b/d1f/d3f/f46 [448069,12324] 0 2024-03-22T07:16:11.034 INFO:tasks.workunit.client.1.smithi069.stdout:4/989: read - d6/d12/d5d/ffe zero size 2024-03-22T07:16:11.036 INFO:tasks.workunit.client.1.smithi069.stdout:4/990: dread d6/d9b/d1f/d3f/f46 [0,4194304] 0 2024-03-22T07:16:11.036 INFO:tasks.workunit.client.1.smithi069.stdout:4/991: chown d6/d16/d3a/db4/dfb/d23/d59/l67 24 1 2024-03-22T07:16:11.037 INFO:tasks.workunit.client.1.smithi069.stdout:4/992: symlink d6/d16/d3a/db4/dfb/d1d/d55/l150 0 2024-03-22T07:16:11.038 INFO:tasks.workunit.client.1.smithi069.stdout:4/993: mknod d6/d9b/d1f/d58/d7c/d90/c151 0 2024-03-22T07:16:11.040 INFO:tasks.workunit.client.1.smithi069.stdout:4/994: mkdir d6/dc4/db5/d152 0 2024-03-22T07:16:11.040 INFO:tasks.workunit.client.1.smithi069.stdout:4/995: getdents d6/d12/db2 0 2024-03-22T07:16:11.040 INFO:tasks.workunit.client.1.smithi069.stdout:4/996: chown d6/d9b/d1f/d3f/d51 4385 1 2024-03-22T07:16:11.045 INFO:tasks.workunit.client.1.smithi069.stdout:6/735: sync 2024-03-22T07:16:11.045 INFO:tasks.workunit.client.1.smithi069.stdout:5/760: sync 2024-03-22T07:16:11.045 INFO:tasks.workunit.client.1.smithi069.stdout:6/736: readlink d4/dd/d12/d44/le8 0 2024-03-22T07:16:11.050 INFO:tasks.workunit.client.1.smithi069.stdout:5/761: dread d1/d28/fb8 [0,4194304] 0 2024-03-22T07:16:11.051 INFO:tasks.workunit.client.1.smithi069.stdout:1/878: dread d1/d2a/d5e/d9b/fb7 [0,4194304] 0 2024-03-22T07:16:11.051 INFO:tasks.workunit.client.1.smithi069.stdout:1/879: write d1/d2a/d5e/f10f [4283144,124489] 0 2024-03-22T07:16:11.051 INFO:tasks.workunit.client.1.smithi069.stdout:1/880: dread - d1/d2a/db6/df8/d73/df0/f119 zero size 2024-03-22T07:16:11.051 INFO:tasks.workunit.client.1.smithi069.stdout:5/762: symlink d1/d6/d26/d3e/d9c/d9d/dc3/l107 0 2024-03-22T07:16:11.052 INFO:tasks.workunit.client.1.smithi069.stdout:5/763: dread d1/d7/d3d/fca [0,4194304] 0 2024-03-22T07:16:11.053 INFO:tasks.workunit.client.1.smithi069.stdout:9/809: dwrite d2/f27 [0,4194304] 0 2024-03-22T07:16:11.054 INFO:tasks.workunit.client.1.smithi069.stdout:1/881: link d1/f48 d1/d2a/db6/df8/f11e 0 2024-03-22T07:16:11.055 INFO:tasks.workunit.client.1.smithi069.stdout:9/810: rename d2/d51/f64 to d2/dd/d3c/d53/d75/f102 0 2024-03-22T07:16:11.055 INFO:tasks.workunit.client.1.smithi069.stdout:7/853: dwrite d9/db/f16 [0,4194304] 0 2024-03-22T07:16:11.056 INFO:tasks.workunit.client.1.smithi069.stdout:1/882: rename d1/d13/d36/de5/d88/d41/d1e/f96 to d1/d13/d36/de5/d88/dec/f11f 0 2024-03-22T07:16:11.056 INFO:tasks.workunit.client.1.smithi069.stdout:2/801: dwrite d18/d4d/d27/d33/d3d/dc6/fe7 [0,4194304] 0 2024-03-22T07:16:11.057 INFO:tasks.workunit.client.1.smithi069.stdout:4/997: dwrite d6/d16/d83/d9a/fa3 [0,4194304] 0 2024-03-22T07:16:11.057 INFO:tasks.workunit.client.1.smithi069.stdout:4/998: truncate d6/dc4/ff7 4347674 0 2024-03-22T07:16:11.057 INFO:tasks.workunit.client.1.smithi069.stdout:4/999: readlink d6/d4b/de9/lee 0 2024-03-22T07:16:11.057 INFO:tasks.workunit.client.1.smithi069.stdout:9/811: rename d2/dd/d3c/d53/d8b/ffc to d2/dd/d3c/d53/da6/f103 0 2024-03-22T07:16:11.058 INFO:tasks.workunit.client.1.smithi069.stdout:9/812: chown d2/dd/d3c/d53/d8b 207870 1 2024-03-22T07:16:11.059 INFO:tasks.workunit.client.1.smithi069.stdout:2/802: mknod d18/d4d/d27/d33/d60/d64/d89/cff 0 2024-03-22T07:16:11.062 INFO:tasks.workunit.client.1.smithi069.stdout:2/803: mkdir d18/d4d/d27/d33/d60/d63/d80/d100 0 2024-03-22T07:16:11.063 INFO:tasks.workunit.client.1.smithi069.stdout:9/813: getdents d2/d21/dd7 0 2024-03-22T07:16:11.066 INFO:tasks.workunit.client.1.smithi069.stdout:1/883: write d1/fb [2705870,971] 0 2024-03-22T07:16:11.067 INFO:tasks.workunit.client.1.smithi069.stdout:7/854: dread d9/db/f69 [0,4194304] 0 2024-03-22T07:16:11.067 INFO:tasks.workunit.client.1.smithi069.stdout:2/804: rename d18/d4d/d3e/d40/df1 to d18/d4d/d27/d33/d60/d64/dcd/d101 0 2024-03-22T07:16:11.067 INFO:tasks.workunit.client.1.smithi069.stdout:2/805: write d18/d4d/d27/d72/f7d [3827521,118839] 0 2024-03-22T07:16:11.068 INFO:tasks.workunit.client.1.smithi069.stdout:9/814: symlink d2/dd/d3c/d53/d75/l104 0 2024-03-22T07:16:11.068 INFO:tasks.workunit.client.1.smithi069.stdout:9/815: readlink d2/d31/ld3 0 2024-03-22T07:16:11.070 INFO:tasks.workunit.client.1.smithi069.stdout:2/806: dread d18/d4d/d27/d56/f94 [0,4194304] 0 2024-03-22T07:16:11.070 INFO:tasks.workunit.client.1.smithi069.stdout:2/807: read - d18/d4d/d27/d33/d60/d64/dcd/ff2 zero size 2024-03-22T07:16:11.070 INFO:tasks.workunit.client.1.smithi069.stdout:2/808: write d18/f1e [1068422,6444] 0 2024-03-22T07:16:11.073 INFO:tasks.workunit.client.1.smithi069.stdout:7/855: rename d9/db/d1a/db1/d57/d44/l72 to d9/db/d1a/db1/d57/d75/d7a/dcf/l10f 0 2024-03-22T07:16:11.073 INFO:tasks.workunit.client.1.smithi069.stdout:1/884: read d1/d13/d36/f54 [659260,113609] 0 2024-03-22T07:16:11.073 INFO:tasks.workunit.client.1.smithi069.stdout:1/885: write d1/d13/d36/de5/d88/d41/f4b [1267897,81969] 0 2024-03-22T07:16:11.073 INFO:tasks.workunit.client.1.smithi069.stdout:1/886: chown d1/d13/d36/de5/d88/d41/lc6 173 1 2024-03-22T07:16:11.074 INFO:tasks.workunit.client.1.smithi069.stdout:9/816: symlink d2/dd/d3c/d48/d7f/d86/l105 0 2024-03-22T07:16:11.074 INFO:tasks.workunit.client.1.smithi069.stdout:9/817: write d2/dd/d3c/d2e/fa8 [1672324,77843] 0 2024-03-22T07:16:11.075 INFO:tasks.workunit.client.1.smithi069.stdout:2/809: symlink d18/d4d/dec/l102 0 2024-03-22T07:16:11.075 INFO:tasks.workunit.client.1.smithi069.stdout:2/810: creat d18/d4d/d27/d33/f103 x:0 0 0 2024-03-22T07:16:11.077 INFO:tasks.workunit.client.1.smithi069.stdout:7/856: link d9/db/l5f d9/db/d1a/db1/db3/l110 0 2024-03-22T07:16:11.078 INFO:tasks.workunit.client.1.smithi069.stdout:9/818: creat d2/d76/f106 x:0 0 0 2024-03-22T07:16:11.078 INFO:tasks.workunit.client.1.smithi069.stdout:9/819: chown d2/dd/d3c/d48/d7f/d86/c9a 552 1 2024-03-22T07:16:11.085 INFO:tasks.workunit.client.1.smithi069.stdout:2/811: dread d18/d4d/d3e/d40/d96/fed [0,4194304] 0 2024-03-22T07:16:11.085 INFO:tasks.workunit.client.1.smithi069.stdout:2/812: fsync d18/d4d/d27/d33/d60/d63/d73/fb4 0 2024-03-22T07:16:11.085 INFO:tasks.workunit.client.1.smithi069.stdout:2/813: chown d18/d91/ca2 379 1 2024-03-22T07:16:11.086 INFO:tasks.workunit.client.1.smithi069.stdout:7/857: rmdir d9/db/d1a/db1/d57/d75/de2 39 2024-03-22T07:16:11.088 INFO:tasks.workunit.client.1.smithi069.stdout:7/858: symlink d9/d42/da8/l111 0 2024-03-22T07:16:11.088 INFO:tasks.workunit.client.1.smithi069.stdout:7/859: truncate d9/db/d1a/db1/d57/d75/f9c 3584190 0 2024-03-22T07:16:11.088 INFO:tasks.workunit.client.1.smithi069.stdout:7/860: write d9/db/d1a/db1/d57/d44/f9b [2161589,449] 0 2024-03-22T07:16:11.090 INFO:tasks.workunit.client.1.smithi069.stdout:7/861: mknod d9/db/d1a/db1/d57/d44/d53/d2e/c112 0 2024-03-22T07:16:11.096 INFO:tasks.workunit.client.1.smithi069.stdout:2/814: read d18/d4d/d27/d33/d60/d64/d89/d8b/f9d [1046229,8787] 0 2024-03-22T07:16:11.097 INFO:tasks.workunit.client.1.smithi069.stdout:2/815: write d18/d4d/d25/f4b [5075350,9089] 0 2024-03-22T07:16:11.097 INFO:tasks.workunit.client.1.smithi069.stdout:2/816: chown d18/l1a 304497 1 2024-03-22T07:16:11.097 INFO:tasks.workunit.client.1.smithi069.stdout:9/820: dwrite d2/dd/d3c/d2e/d84/f100 [0,4194304] 0 2024-03-22T07:16:11.100 INFO:tasks.workunit.client.1.smithi069.stdout:7/862: read d9/db/d1a/db1/d57/d44/d53/d2e/fa6 [2155162,81172] 0 2024-03-22T07:16:11.100 INFO:tasks.workunit.client.1.smithi069.stdout:2/817: creat d18/d4d/dec/f104 x:0 0 0 2024-03-22T07:16:11.100 INFO:tasks.workunit.client.1.smithi069.stdout:7/863: chown d9/db/d1a/db1/d57/d44/d53/d74/d6e 20 1 2024-03-22T07:16:11.100 INFO:tasks.workunit.client.1.smithi069.stdout:7/864: creat d9/db/d1a/db1/d57/d44/d53/d74/d6e/f113 x:0 0 0 2024-03-22T07:16:11.101 INFO:tasks.workunit.client.1.smithi069.stdout:9/821: rename d2/dd/d3c/d2e/fad to d2/dd/d6e/f107 0 2024-03-22T07:16:11.104 INFO:tasks.workunit.client.1.smithi069.stdout:7/865: creat d9/db/d1a/db1/d57/d44/d53/d1d/f114 x:0 0 0 2024-03-22T07:16:11.106 INFO:tasks.workunit.client.1.smithi069.stdout:9/822: truncate d2/dd/d3c/d53/ffd 7570107 0 2024-03-22T07:16:11.106 INFO:tasks.workunit.client.1.smithi069.stdout:9/823: stat d2/dd/f26 0 2024-03-22T07:16:11.106 INFO:tasks.workunit.client.1.smithi069.stdout:9/824: creat d2/d83/f108 x:0 0 0 2024-03-22T07:16:11.107 INFO:tasks.workunit.client.1.smithi069.stdout:2/818: getdents d18/d4d/d27/d33/d60/d63/d93 0 2024-03-22T07:16:11.107 INFO:tasks.workunit.client.1.smithi069.stdout:2/819: creat d18/d4d/d27/d33/d3d/f105 x:0 0 0 2024-03-22T07:16:11.107 INFO:tasks.workunit.client.1.smithi069.stdout:2/820: write d18/d4d/d27/d33/d60/d64/f99 [1165771,45898] 0 2024-03-22T07:16:11.110 INFO:tasks.workunit.client.1.smithi069.stdout:9/825: creat d2/dd/d3c/d53/f109 x:0 0 0 2024-03-22T07:16:11.113 INFO:tasks.workunit.client.1.smithi069.stdout:2/821: rename d18/d91/ca2 to d18/d4d/d27/d33/d60/d63/d73/dc9/c106 0 2024-03-22T07:16:11.114 INFO:tasks.workunit.client.1.smithi069.stdout:2/822: fdatasync d18/d4d/d27/d33/d60/d64/d89/f90 0 2024-03-22T07:16:11.115 INFO:tasks.workunit.client.1.smithi069.stdout:9/826: creat d2/d51/dbd/de4/df3/df5/f10a x:0 0 0 2024-03-22T07:16:11.115 INFO:tasks.workunit.client.1.smithi069.stdout:9/827: truncate d2/dd/d3c/d2e/d84/da5/de0/ff0 938193 0 2024-03-22T07:16:11.132 INFO:tasks.workunit.client.1.smithi069.stdout:7/866: dwrite d9/db/d1a/db1/d57/d44/d53/d74/d30/ffc [0,4194304] 0 2024-03-22T07:16:11.132 INFO:tasks.workunit.client.1.smithi069.stdout:9/828: dwrite d2/d31/fc4 [0,4194304] 0 2024-03-22T07:16:11.132 INFO:tasks.workunit.client.1.smithi069.stdout:9/829: write d2/dd/d3c/d53/d75/fbb [504112,64643] 0 2024-03-22T07:16:11.149 INFO:tasks.workunit.client.1.smithi069.stdout:7/867: dwrite d9/db/d1a/db1/d57/d75/d7a/fad [0,4194304] 0 2024-03-22T07:16:11.152 INFO:tasks.workunit.client.1.smithi069.stdout:7/868: rename d9/db/d1a/db1/fe8 to d9/db/d1a/db1/d57/d44/d53/d74/f115 0 2024-03-22T07:16:11.161 INFO:tasks.workunit.client.1.smithi069.stdout:7/869: dread f4 [0,4194304] 0 2024-03-22T07:16:11.161 INFO:tasks.workunit.client.1.smithi069.stdout:9/830: dwrite d2/dd/d3c/d2e/fa3 [0,4194304] 0 2024-03-22T07:16:11.161 INFO:tasks.workunit.client.1.smithi069.stdout:9/831: creat d2/dd/d3c/d53/d75/f10b x:0 0 0 2024-03-22T07:16:11.163 INFO:tasks.workunit.client.1.smithi069.stdout:9/832: dread d2/d51/f5e [0,4194304] 0 2024-03-22T07:16:11.165 INFO:tasks.workunit.client.1.smithi069.stdout:9/833: mkdir d2/d21/d77/d10c 0 2024-03-22T07:16:11.167 INFO:tasks.workunit.client.1.smithi069.stdout:9/834: rmdir d2/d76 39 2024-03-22T07:16:11.167 INFO:tasks.workunit.client.1.smithi069.stdout:9/835: chown d2/dd/d3c/d2e/f4b 2759715 1 2024-03-22T07:16:11.168 INFO:tasks.workunit.client.1.smithi069.stdout:9/836: dread d2/dd/d3c/d2e/f65 [0,4194304] 0 2024-03-22T07:16:11.171 INFO:tasks.workunit.client.1.smithi069.stdout:9/837: rename d2/d21/d77/cfb to d2/d21/d77/def/c10d 0 2024-03-22T07:16:11.176 INFO:tasks.workunit.client.1.smithi069.stdout:7/870: dwrite d9/db/d1a/db1/d57/d75/d7a/f80 [0,4194304] 0 2024-03-22T07:16:11.177 INFO:tasks.workunit.client.1.smithi069.stdout:7/871: rename d9/db/d1a/db1/d57/d44/d53/d1d/l29 to d9/db/d1a/db1/d57/d44/d53/d74/d6e/l116 0 2024-03-22T07:16:11.180 INFO:tasks.workunit.client.1.smithi069.stdout:9/838: write d2/dd/d3c/d48/d7c/f9e [4068290,76168] 0 2024-03-22T07:16:11.181 INFO:tasks.workunit.client.1.smithi069.stdout:9/839: creat d2/dd/d3c/d48/d7f/da2/f10e x:0 0 0 2024-03-22T07:16:11.183 INFO:tasks.workunit.client.1.smithi069.stdout:9/840: symlink d2/d51/l10f 0 2024-03-22T07:16:11.183 INFO:tasks.workunit.client.1.smithi069.stdout:9/841: chown d2/d21/d77/ccf 8 1 2024-03-22T07:16:11.183 INFO:tasks.workunit.client.1.smithi069.stdout:9/842: write d2/dd/d3c/d48/f4d [2589648,31703] 0 2024-03-22T07:16:11.183 INFO:tasks.workunit.client.1.smithi069.stdout:9/843: readlink d2/l7b 0 2024-03-22T07:16:11.183 INFO:tasks.workunit.client.1.smithi069.stdout:9/844: chown d2/dd/d3c/d2e/f80 992976 1 2024-03-22T07:16:11.183 INFO:tasks.workunit.client.1.smithi069.stdout:9/845: readlink d2/dd/d3c/d2e/lc9 0 2024-03-22T07:16:11.183 INFO:tasks.workunit.client.1.smithi069.stdout:9/846: mknod d2/d21/dd7/c110 0 2024-03-22T07:16:11.184 INFO:tasks.workunit.client.1.smithi069.stdout:9/847: creat d2/d76/f111 x:0 0 0 2024-03-22T07:16:11.184 INFO:tasks.workunit.client.1.smithi069.stdout:9/848: chown d2/dd/d3c/d2e 4 1 2024-03-22T07:16:11.184 INFO:tasks.workunit.client.1.smithi069.stdout:9/849: symlink d2/dd/d3c/d2e/d84/da5/de0/l112 0 2024-03-22T07:16:11.185 INFO:tasks.workunit.client.1.smithi069.stdout:9/850: mkdir d2/d21/d77/d10c/d113 0 2024-03-22T07:16:11.197 INFO:tasks.workunit.client.1.smithi069.stdout:0/956: sync 2024-03-22T07:16:11.197 INFO:tasks.workunit.client.1.smithi069.stdout:0/957: readlink d3/d7/la8 0 2024-03-22T07:16:11.197 INFO:tasks.workunit.client.1.smithi069.stdout:0/958: creat d3/d7/d17/f13c x:0 0 0 2024-03-22T07:16:11.197 INFO:tasks.workunit.client.1.smithi069.stdout:0/959: mknod d3/d9a/c13d 0 2024-03-22T07:16:11.198 INFO:tasks.workunit.client.1.smithi069.stdout:0/960: creat d3/d7/d17/d1c/dd4/f13e x:0 0 0 2024-03-22T07:16:11.204 INFO:tasks.workunit.client.1.smithi069.stdout:9/851: dwrite d2/d51/f59 [4194304,4194304] 0 2024-03-22T07:16:11.207 INFO:tasks.workunit.client.1.smithi069.stdout:7/872: dwrite d9/db/d1a/db1/d57/d44/d53/dc7/f9a [0,4194304] 0 2024-03-22T07:16:11.213 INFO:tasks.workunit.client.1.smithi069.stdout:9/852: write d2/dd/d3c/f43 [1600317,4344] 0 2024-03-22T07:16:11.215 INFO:tasks.workunit.client.1.smithi069.stdout:9/853: dread d2/d51/f5e [0,4194304] 0 2024-03-22T07:16:11.215 INFO:tasks.workunit.client.1.smithi069.stdout:9/854: fdatasync d2/dd/d3c/d2e/f85 0 2024-03-22T07:16:11.217 INFO:tasks.workunit.client.1.smithi069.stdout:9/855: creat d2/d51/dbd/de4/df3/f114 x:0 0 0 2024-03-22T07:16:11.218 INFO:tasks.workunit.client.1.smithi069.stdout:9/856: dread - d2/dd/d3c/d2e/d84/fd9 zero size 2024-03-22T07:16:11.218 INFO:tasks.workunit.client.1.smithi069.stdout:9/857: fsync d2/dd/fac 0 2024-03-22T07:16:11.220 INFO:tasks.workunit.client.1.smithi069.stdout:9/858: mknod d2/dd/d3c/d48/d7f/c115 0 2024-03-22T07:16:11.220 INFO:tasks.workunit.client.1.smithi069.stdout:7/873: write d9/db/d1a/db1/db3/fcd [3134326,58831] 0 2024-03-22T07:16:11.221 INFO:tasks.workunit.client.1.smithi069.stdout:7/874: chown d9/db/d1a/db1/d57/d44/fac 631 1 2024-03-22T07:16:11.221 INFO:tasks.workunit.client.1.smithi069.stdout:0/961: dwrite d3/d7/d17/d1c/f11c [0,4194304] 0 2024-03-22T07:16:11.221 INFO:tasks.workunit.client.1.smithi069.stdout:7/875: rename d9/db/d1a/db1/d57/d44/d53/d74/d6e/f83 to d9/db/d1a/db1/d57/f117 0 2024-03-22T07:16:11.224 INFO:tasks.workunit.client.1.smithi069.stdout:0/962: mknod d3/d7/d15/dce/d127/c13f 0 2024-03-22T07:16:11.224 INFO:tasks.workunit.client.1.smithi069.stdout:0/963: write d3/d7/d15/d29/fee [239052,29809] 0 2024-03-22T07:16:11.224 INFO:tasks.workunit.client.1.smithi069.stdout:9/859: dread d2/dd/f69 [0,4194304] 0 2024-03-22T07:16:11.225 INFO:tasks.workunit.client.1.smithi069.stdout:9/860: fdatasync d2/dd/d3c/d53/f109 0 2024-03-22T07:16:11.226 INFO:tasks.workunit.client.1.smithi069.stdout:7/876: mknod d9/d8e/c118 0 2024-03-22T07:16:11.226 INFO:tasks.workunit.client.1.smithi069.stdout:7/877: chown d9/db/d1a/db1/d57/d75/f81 413490 1 2024-03-22T07:16:11.226 INFO:tasks.workunit.client.1.smithi069.stdout:0/964: mknod d3/d9a/c140 0 2024-03-22T07:16:11.229 INFO:tasks.workunit.client.1.smithi069.stdout:7/878: creat d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/dce/f119 x:0 0 0 2024-03-22T07:16:11.230 INFO:tasks.workunit.client.1.smithi069.stdout:9/861: getdents d2/dd/d3c/d48/d7f/deb 0 2024-03-22T07:16:11.230 INFO:tasks.workunit.client.1.smithi069.stdout:9/862: creat d2/d31/f116 x:0 0 0 2024-03-22T07:16:11.230 INFO:tasks.workunit.client.1.smithi069.stdout:0/965: dread d3/d7/d17/d1c/fb3 [4194304,4194304] 0 2024-03-22T07:16:11.232 INFO:tasks.workunit.client.1.smithi069.stdout:7/879: symlink d9/db/d1a/db1/d57/d75/l11a 0 2024-03-22T07:16:11.232 INFO:tasks.workunit.client.1.smithi069.stdout:7/880: fsync f4 0 2024-03-22T07:16:11.232 INFO:tasks.workunit.client.1.smithi069.stdout:7/881: truncate d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/df9/faf 4722076 0 2024-03-22T07:16:11.241 INFO:tasks.workunit.client.1.smithi069.stdout:7/882: write d9/db/d1a/db1/d57/d44/d53/d74/ff1 [3573652,100452] 0 2024-03-22T07:16:11.241 INFO:tasks.workunit.client.1.smithi069.stdout:7/883: fsync d9/f6f 0 2024-03-22T07:16:11.262 INFO:tasks.workunit.client.1.smithi069.stdout:0/966: dwrite d3/f118 [0,4194304] 0 2024-03-22T07:16:11.262 INFO:tasks.workunit.client.1.smithi069.stdout:0/967: dread - d3/d7/d15/d29/dcf/de9/fff zero size 2024-03-22T07:16:11.262 INFO:tasks.workunit.client.1.smithi069.stdout:9/863: dwrite d2/dd/d3c/d53/fae [0,4194304] 0 2024-03-22T07:16:11.262 INFO:tasks.workunit.client.1.smithi069.stdout:9/864: readlink d2/d21/l6b 0 2024-03-22T07:16:11.262 INFO:tasks.workunit.client.1.smithi069.stdout:7/884: dwrite d9/db/d1a/db1/d57/d44/d53/d74/d6e/fe6 [0,4194304] 0 2024-03-22T07:16:11.270 INFO:tasks.workunit.client.1.smithi069.stdout:0/968: dread d3/d7/d15/d29/d5a/d5d/d9d/fbd [0,4194304] 0 2024-03-22T07:16:11.270 INFO:tasks.workunit.client.1.smithi069.stdout:0/969: chown d3/d7/d17/fde 4923 1 2024-03-22T07:16:11.271 INFO:tasks.workunit.client.1.smithi069.stdout:0/970: unlink d3/d7/d15/dce/ff3 0 2024-03-22T07:16:11.271 INFO:tasks.workunit.client.1.smithi069.stdout:0/971: creat d3/d7/d15/d29/d5a/f141 x:0 0 0 2024-03-22T07:16:11.272 INFO:tasks.workunit.client.1.smithi069.stdout:9/865: write d2/dd/d3c/d2e/f32 [5243322,71385] 0 2024-03-22T07:16:11.277 INFO:tasks.workunit.client.1.smithi069.stdout:7/885: dread d9/db/d1a/db1/d57/d44/d53/d74/f91 [0,4194304] 0 2024-03-22T07:16:11.278 INFO:tasks.workunit.client.1.smithi069.stdout:0/972: write d3/d7/d15/fa3 [1665954,68794] 0 2024-03-22T07:16:11.281 INFO:tasks.workunit.client.1.smithi069.stdout:7/886: dread d9/db/d1a/db1/d57/d44/d53/d74/f115 [0,4194304] 0 2024-03-22T07:16:11.294 INFO:tasks.workunit.client.1.smithi069.stdout:9/866: unlink d2/d71/c8a 0 2024-03-22T07:16:11.297 INFO:tasks.workunit.client.1.smithi069.stdout:9/867: mkdir d2/dd/d3c/d48/d7f/d86/d117 0 2024-03-22T07:16:11.298 INFO:tasks.workunit.client.1.smithi069.stdout:9/868: chown d2/lb 49834171 1 2024-03-22T07:16:11.298 INFO:tasks.workunit.client.1.smithi069.stdout:0/973: truncate d3/d7/d5e/f61 2862530 0 2024-03-22T07:16:11.298 INFO:tasks.workunit.client.1.smithi069.stdout:7/887: rename d9/db/d1a/db1/d57/d44/d53/c55 to d9/d8e/df7/c11b 0 2024-03-22T07:16:11.300 INFO:tasks.workunit.client.1.smithi069.stdout:9/869: symlink d2/dd/d3c/d48/d7f/d86/l118 0 2024-03-22T07:16:11.300 INFO:tasks.workunit.client.1.smithi069.stdout:0/974: symlink d3/d7/d15/d1f/l142 0 2024-03-22T07:16:11.308 INFO:tasks.workunit.client.1.smithi069.stdout:0/975: link d3/d7/d15/d1f/c97 d3/d7/d17/d1c/dd6/c143 0 2024-03-22T07:16:11.309 INFO:tasks.workunit.client.1.smithi069.stdout:0/976: mkdir d3/d7/d15/d144 0 2024-03-22T07:16:11.309 INFO:tasks.workunit.client.1.smithi069.stdout:0/977: dread - d3/fe6 zero size 2024-03-22T07:16:11.310 INFO:tasks.workunit.client.1.smithi069.stdout:0/978: mknod d3/d9a/d133/c145 0 2024-03-22T07:16:11.312 INFO:tasks.workunit.client.1.smithi069.stdout:9/870: dwrite d2/d51/f5e [0,4194304] 0 2024-03-22T07:16:11.330 INFO:tasks.workunit.client.1.smithi069.stdout:9/871: dwrite d2/dd/d6e/f107 [0,4194304] 0 2024-03-22T07:16:11.330 INFO:tasks.workunit.client.1.smithi069.stdout:9/872: read d2/d31/f40 [841524,74495] 0 2024-03-22T07:16:11.331 INFO:tasks.workunit.client.1.smithi069.stdout:9/873: mknod d2/d21/c119 0 2024-03-22T07:16:11.331 INFO:tasks.workunit.client.1.smithi069.stdout:9/874: write d2/dd/d3c/d53/d75/f10b [630270,12314] 0 2024-03-22T07:16:11.343 INFO:tasks.workunit.client.1.smithi069.stdout:9/875: write d2/dd/d3c/d53/fae [535272,43362] 0 2024-03-22T07:16:11.344 INFO:tasks.workunit.client.1.smithi069.stdout:9/876: truncate d2/dd/d3c/d48/f4d 669710 0 2024-03-22T07:16:11.344 INFO:tasks.workunit.client.1.smithi069.stdout:9/877: creat d2/d83/f11a x:0 0 0 2024-03-22T07:16:11.353 INFO:tasks.workunit.client.1.smithi069.stdout:9/878: write d2/f5 [2437532,80090] 0 2024-03-22T07:16:11.354 INFO:tasks.workunit.client.1.smithi069.stdout:9/879: mkdir d2/dd/d3c/d53/d75/d11b 0 2024-03-22T07:16:11.355 INFO:tasks.workunit.client.1.smithi069.stdout:9/880: creat d2/d21/d77/d10c/d113/f11c x:0 0 0 2024-03-22T07:16:11.355 INFO:tasks.workunit.client.1.smithi069.stdout:9/881: fdatasync d2/dd/d3c/d48/fca 0 2024-03-22T07:16:11.355 INFO:tasks.workunit.client.1.smithi069.stdout:9/882: stat d2/db6 0 2024-03-22T07:16:11.371 INFO:tasks.workunit.client.1.smithi069.stdout:9/883: dwrite d2/dd/d3c/d48/f4d [0,4194304] 0 2024-03-22T07:16:11.372 INFO:tasks.workunit.client.1.smithi069.stdout:9/884: symlink d2/dd/d3c/d53/l11d 0 2024-03-22T07:16:11.374 INFO:tasks.workunit.client.1.smithi069.stdout:9/885: rename d2/d31/cec to d2/d21/d77/d10c/c11e 0 2024-03-22T07:16:11.377 INFO:tasks.workunit.client.1.smithi069.stdout:9/886: rename d2/d21/d77/d10c/d113 to d2/d31/dc6/d11f 0 2024-03-22T07:16:11.378 INFO:tasks.workunit.client.1.smithi069.stdout:9/887: mkdir d2/dd/d3c/d48/d7c/dff/d120 0 2024-03-22T07:16:11.378 INFO:tasks.workunit.client.1.smithi069.stdout:9/888: readlink d2/dd/d3c/d48/d7f/da2/lcc 0 2024-03-22T07:16:11.379 INFO:tasks.workunit.client.1.smithi069.stdout:9/889: symlink d2/dd/d3c/d48/d7f/db0/l121 0 2024-03-22T07:16:11.380 INFO:tasks.workunit.client.1.smithi069.stdout:9/890: symlink d2/d51/l122 0 2024-03-22T07:16:11.380 INFO:tasks.workunit.client.1.smithi069.stdout:9/891: creat d2/dd/f123 x:0 0 0 2024-03-22T07:16:11.382 INFO:tasks.workunit.client.1.smithi069.stdout:9/892: getdents d2/d83 0 2024-03-22T07:16:11.384 INFO:tasks.workunit.client.1.smithi069.stdout:9/893: symlink d2/d51/dbd/de4/l124 0 2024-03-22T07:16:11.386 INFO:tasks.workunit.client.1.smithi069.stdout:9/894: creat d2/d51/dbd/de4/f125 x:0 0 0 2024-03-22T07:16:11.554 INFO:tasks.workunit.client.1.smithi069.stdout:5/764: sync 2024-03-22T07:16:11.554 INFO:tasks.workunit.client.1.smithi069.stdout:6/737: sync 2024-03-22T07:16:11.555 INFO:tasks.workunit.client.1.smithi069.stdout:5/765: symlink d1/d7/l108 0 2024-03-22T07:16:11.555 INFO:tasks.workunit.client.1.smithi069.stdout:5/766: chown d1/d6/d26/d3e/d9c/d9d/dc3/cc6 2093 1 2024-03-22T07:16:11.555 INFO:tasks.workunit.client.1.smithi069.stdout:5/767: chown d1/d6/d86/d99/l98 182916567 1 2024-03-22T07:16:11.556 INFO:tasks.workunit.client.1.smithi069.stdout:5/768: rmdir d1/d6/d26/d33 39 2024-03-22T07:16:11.558 INFO:tasks.workunit.client.1.smithi069.stdout:6/738: rename d4/dd/d12/d44/d7f to d4/dd/d12/d13/de9 0 2024-03-22T07:16:11.558 INFO:tasks.workunit.client.1.smithi069.stdout:6/739: stat d4/dd/dbd/fda 0 2024-03-22T07:16:11.563 INFO:tasks.workunit.client.1.smithi069.stdout:5/769: dread d1/d6/d26/d33/f80 [4194304,4194304] 0 2024-03-22T07:16:11.563 INFO:tasks.workunit.client.1.smithi069.stdout:5/770: chown d1/d28/f91 81132 1 2024-03-22T07:16:11.578 INFO:tasks.workunit.client.1.smithi069.stdout:5/771: dwrite d1/d7/d3d/f9a [0,4194304] 0 2024-03-22T07:16:11.587 INFO:tasks.workunit.client.1.smithi069.stdout:5/772: write d1/d6/d26/d33/d5d/fa4 [633320,4497] 0 2024-03-22T07:16:11.587 INFO:tasks.workunit.client.1.smithi069.stdout:5/773: write d1/d6/d86/dab/ff0 [999871,4141] 0 2024-03-22T07:16:11.588 INFO:tasks.workunit.client.1.smithi069.stdout:5/774: rmdir d1/d7 39 2024-03-22T07:16:11.588 INFO:tasks.workunit.client.1.smithi069.stdout:5/775: truncate d1/d10/d44/d50/f96 1278309 0 2024-03-22T07:16:11.589 INFO:tasks.workunit.client.1.smithi069.stdout:5/776: mkdir d1/d10/df1/d109 0 2024-03-22T07:16:11.591 INFO:tasks.workunit.client.1.smithi069.stdout:5/777: symlink d1/d6/d86/dab/l10a 0 2024-03-22T07:16:11.592 INFO:tasks.workunit.client.1.smithi069.stdout:5/778: chown d1/d6/c2f 1 1 2024-03-22T07:16:11.592 INFO:tasks.workunit.client.1.smithi069.stdout:5/779: fdatasync d1/d10/d44/fe2 0 2024-03-22T07:16:11.598 INFO:tasks.workunit.client.1.smithi069.stdout:5/780: write d1/d6/d26/d3e/dbb/fdd [3241506,24772] 0 2024-03-22T07:16:11.619 INFO:tasks.workunit.client.1.smithi069.stdout:5/781: dwrite d1/d6/d86/d99/f52 [0,4194304] 0 2024-03-22T07:16:11.623 INFO:tasks.workunit.client.1.smithi069.stdout:5/782: creat d1/d6/d86/d99/dec/f10b x:0 0 0 2024-03-22T07:16:11.624 INFO:tasks.workunit.client.1.smithi069.stdout:5/783: mknod d1/d6/d86/dab/dcf/dde/c10c 0 2024-03-22T07:16:11.626 INFO:tasks.workunit.client.1.smithi069.stdout:5/784: creat d1/d6/d26/d3a/dbd/f10d x:0 0 0 2024-03-22T07:16:11.637 INFO:tasks.workunit.client.1.smithi069.stdout:2/823: sync 2024-03-22T07:16:11.637 INFO:tasks.workunit.client.1.smithi069.stdout:7/888: sync 2024-03-22T07:16:11.639 INFO:tasks.workunit.client.1.smithi069.stdout:7/889: rename d9/db/d1a/db1/d57/d44/d53/dc7/c92 to d9/dff/c11c 0 2024-03-22T07:16:11.639 INFO:tasks.workunit.client.1.smithi069.stdout:0/979: sync 2024-03-22T07:16:11.639 INFO:tasks.workunit.client.1.smithi069.stdout:9/895: sync 2024-03-22T07:16:11.639 INFO:tasks.workunit.client.1.smithi069.stdout:1/887: sync 2024-03-22T07:16:11.639 INFO:tasks.workunit.client.1.smithi069.stdout:0/980: stat d3/d7/d15/fa3 0 2024-03-22T07:16:11.640 INFO:tasks.workunit.client.1.smithi069.stdout:1/888: dread - d1/d2a/db6/ff6 zero size 2024-03-22T07:16:11.640 INFO:tasks.workunit.client.1.smithi069.stdout:7/890: mknod d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/dd8/c11d 0 2024-03-22T07:16:11.640 INFO:tasks.workunit.client.1.smithi069.stdout:7/891: chown d9/db/d1a/db1/d57/d44/d53/d74/d30/f54 2026 1 2024-03-22T07:16:11.640 INFO:tasks.workunit.client.1.smithi069.stdout:5/785: dwrite d1/d10/d44/fb4 [0,4194304] 0 2024-03-22T07:16:11.641 INFO:tasks.workunit.client.1.smithi069.stdout:0/981: rename d3/d7/d15/dad/ldd to d3/d7/d17/d1c/l146 0 2024-03-22T07:16:11.642 INFO:tasks.workunit.client.1.smithi069.stdout:7/892: truncate d9/db/d1a/db1/d57/d75/d7a/fa2 1286468 0 2024-03-22T07:16:11.642 INFO:tasks.workunit.client.1.smithi069.stdout:7/893: creat d9/db/d1a/db1/d57/d75/f11e x:0 0 0 2024-03-22T07:16:11.644 INFO:tasks.workunit.client.1.smithi069.stdout:5/786: creat d1/d10/df1/f10e x:0 0 0 2024-03-22T07:16:11.649 INFO:tasks.workunit.client.1.smithi069.stdout:2/824: dread d18/d4d/d27/d33/f9c [0,4194304] 0 2024-03-22T07:16:11.649 INFO:tasks.workunit.client.1.smithi069.stdout:0/982: dread d3/d7/d15/d29/d5a/d5d/ffd [0,4194304] 0 2024-03-22T07:16:11.649 INFO:tasks.workunit.client.1.smithi069.stdout:0/983: creat d3/d7/d15/dad/f147 x:0 0 0 2024-03-22T07:16:11.658 INFO:tasks.workunit.client.1.smithi069.stdout:0/984: dread d3/d7/fb [4194304,4194304] 0 2024-03-22T07:16:11.658 INFO:tasks.workunit.client.1.smithi069.stdout:0/985: creat d3/d7/d15/d29/d5a/d5d/db4/f148 x:0 0 0 2024-03-22T07:16:11.683 INFO:tasks.workunit.client.1.smithi069.stdout:1/889: rename d1/d2a/d5e/d93/l10e to d1/d13/d36/de5/d88/dae/de7/l120 0 2024-03-22T07:16:11.686 INFO:tasks.workunit.client.1.smithi069.stdout:5/787: mkdir d1/d6/d26/d33/d5d/d72/d78/d102/dea/d10f 0 2024-03-22T07:16:11.687 INFO:tasks.workunit.client.1.smithi069.stdout:2/825: stat d18/d4d/f24 0 2024-03-22T07:16:11.687 INFO:tasks.workunit.client.1.smithi069.stdout:2/826: chown d18/d4d/d27/d33/f9c 55534 1 2024-03-22T07:16:11.693 INFO:tasks.workunit.client.1.smithi069.stdout:6/740: sync 2024-03-22T07:16:11.696 INFO:tasks.workunit.client.1.smithi069.stdout:7/894: symlink d9/db/d1a/db1/d57/d44/d53/dc7/l11f 0 2024-03-22T07:16:11.696 INFO:tasks.workunit.client.1.smithi069.stdout:7/895: write d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/dce/f119 [877731,49602] 0 2024-03-22T07:16:11.697 INFO:tasks.workunit.client.1.smithi069.stdout:2/827: symlink d18/d4d/d27/d33/d4f/l107 0 2024-03-22T07:16:11.698 INFO:tasks.workunit.client.1.smithi069.stdout:0/986: rmdir d3/d7/d15/d29/d5a/d84/d10f 39 2024-03-22T07:16:11.701 INFO:tasks.workunit.client.1.smithi069.stdout:5/788: dwrite d1/d6/fa8 [0,4194304] 0 2024-03-22T07:16:11.701 INFO:tasks.workunit.client.1.smithi069.stdout:5/789: truncate d1/d6/d26/d7d/d9f/f104 663314 0 2024-03-22T07:16:11.702 INFO:tasks.workunit.client.1.smithi069.stdout:7/896: mkdir d9/db/d1a/db1/d57/d44/d53/d1d/d120 0 2024-03-22T07:16:11.703 INFO:tasks.workunit.client.1.smithi069.stdout:7/897: read - d9/db/d1a/db1/d57/d44/d53/d1d/fdf zero size 2024-03-22T07:16:11.703 INFO:tasks.workunit.client.1.smithi069.stdout:7/898: creat d9/db/d1a/db1/d57/d44/d53/d74/f121 x:0 0 0 2024-03-22T07:16:11.706 INFO:tasks.workunit.client.1.smithi069.stdout:0/987: mkdir d3/d7/d15/d29/d5a/d84/df4/d129/d149 0 2024-03-22T07:16:11.707 INFO:tasks.workunit.client.1.smithi069.stdout:1/890: rename d1/d13/d36/de5/d88/d41 to d1/d13/dc7/d121 0 2024-03-22T07:16:11.707 INFO:tasks.workunit.client.1.smithi069.stdout:6/741: rename d4/dd/d41/d78 to d4/dd/d41/d78/d9a/dea 22 2024-03-22T07:16:11.707 INFO:tasks.workunit.client.1.smithi069.stdout:6/742: fdatasync d4/dd/d12/d44/f4e 0 2024-03-22T07:16:11.708 INFO:tasks.workunit.client.1.smithi069.stdout:1/891: write d1/d13/d36/de5/d88/f38 [3672418,19735] 0 2024-03-22T07:16:11.708 INFO:tasks.workunit.client.1.smithi069.stdout:5/790: mkdir d1/d7/d3d/dfa/d110 0 2024-03-22T07:16:11.710 INFO:tasks.workunit.client.1.smithi069.stdout:2/828: symlink d18/d4d/d27/d33/d60/d63/l108 0 2024-03-22T07:16:11.711 INFO:tasks.workunit.client.1.smithi069.stdout:0/988: mkdir d3/d7/d15/dce/d127/d14a 0 2024-03-22T07:16:11.714 INFO:tasks.workunit.client.1.smithi069.stdout:7/899: rename d9/db/d1a/db1/d57/d44/d53/d2e to d9/db/d1a/db1/d57/d75/de2/d122 0 2024-03-22T07:16:11.714 INFO:tasks.workunit.client.1.smithi069.stdout:7/900: write d9/d8e/df7/d103/f104 [213759,130458] 0 2024-03-22T07:16:11.715 INFO:tasks.workunit.client.1.smithi069.stdout:1/892: mkdir d1/d13/d36/de5/d88/dec/d122 0 2024-03-22T07:16:11.716 INFO:tasks.workunit.client.1.smithi069.stdout:5/791: mknod d1/d6/d26/d33/d5d/d72/da7/c111 0 2024-03-22T07:16:11.717 INFO:tasks.workunit.client.1.smithi069.stdout:2/829: mknod d18/d4d/d27/d33/d60/d63/d80/c109 0 2024-03-22T07:16:11.721 INFO:tasks.workunit.client.1.smithi069.stdout:0/989: creat d3/d7/d15/d29/d5a/d84/d10f/f14b x:0 0 0 2024-03-22T07:16:11.722 INFO:tasks.workunit.client.1.smithi069.stdout:0/990: creat d3/d7/d15/dce/f14c x:0 0 0 2024-03-22T07:16:11.724 INFO:tasks.workunit.client.1.smithi069.stdout:6/743: rename d4/dd/d41/d78/da3/fc2 to d4/dd/d12/d2b/da5/feb 0 2024-03-22T07:16:11.724 INFO:tasks.workunit.client.1.smithi069.stdout:6/744: dread - d4/dd/d12/fd6 zero size 2024-03-22T07:16:11.725 INFO:tasks.workunit.client.1.smithi069.stdout:5/792: link d1/d10/d60/fc1 d1/d6/d26/d33/d5d/d72/d95/f112 0 2024-03-22T07:16:11.725 INFO:tasks.workunit.client.1.smithi069.stdout:5/793: readlink d1/d28/l82 0 2024-03-22T07:16:11.725 INFO:tasks.workunit.client.1.smithi069.stdout:1/893: read d1/d13/d36/d5b/f8e [4061050,103126] 0 2024-03-22T07:16:11.725 INFO:tasks.workunit.client.1.smithi069.stdout:1/894: read d1/d13/d36/de5/d4c/d55/f97 [4081365,112314] 0 2024-03-22T07:16:11.729 INFO:tasks.workunit.client.1.smithi069.stdout:0/991: link d3/d7/d17/d1c/l1d d3/d7/d15/d29/d5a/d5d/d9d/l14d 0 2024-03-22T07:16:11.729 INFO:tasks.workunit.client.1.smithi069.stdout:0/992: chown d3/d7 1904413216 1 2024-03-22T07:16:11.729 INFO:tasks.workunit.client.1.smithi069.stdout:0/993: readlink d3/d7/d15/d29/d5a/d5d/lc8 0 2024-03-22T07:16:11.730 INFO:tasks.workunit.client.1.smithi069.stdout:5/794: mknod d1/d10/df1/c113 0 2024-03-22T07:16:11.730 INFO:tasks.workunit.client.1.smithi069.stdout:0/994: truncate d3/d7/d15/d29/d5a/f6c 1119437 0 2024-03-22T07:16:11.731 INFO:tasks.workunit.client.1.smithi069.stdout:6/745: dread d4/dd/f54 [0,4194304] 0 2024-03-22T07:16:11.735 INFO:tasks.workunit.client.1.smithi069.stdout:2/830: dwrite d18/d4d/d3e/d40/f61 [0,4194304] 0 2024-03-22T07:16:11.736 INFO:tasks.workunit.client.1.smithi069.stdout:5/795: dread d1/d10/f5b [0,4194304] 0 2024-03-22T07:16:11.736 INFO:tasks.workunit.client.1.smithi069.stdout:5/796: chown d1/d7/l108 341 1 2024-03-22T07:16:11.737 INFO:tasks.workunit.client.1.smithi069.stdout:5/797: rmdir d1/d7/d3d/dfa 39 2024-03-22T07:16:11.760 INFO:tasks.workunit.client.1.smithi069.stdout:6/746: dwrite d4/d7/d5b/f70 [0,4194304] 0 2024-03-22T07:16:11.761 INFO:tasks.workunit.client.1.smithi069.stdout:2/831: dwrite d18/d4d/d27/d33/d60/d64/fda [0,4194304] 0 2024-03-22T07:16:11.761 INFO:tasks.workunit.client.1.smithi069.stdout:2/832: creat d18/d4d/d27/d33/d60/d63/d73/dc9/f10a x:0 0 0 2024-03-22T07:16:11.761 INFO:tasks.workunit.client.1.smithi069.stdout:5/798: dwrite d1/d6/d26/d33/d5d/d72/d95/fe4 [0,4194304] 0 2024-03-22T07:16:11.767 INFO:tasks.workunit.client.1.smithi069.stdout:5/799: symlink d1/d6/d26/d33/d5d/d72/d78/d102/dea/dba/l114 0 2024-03-22T07:16:11.768 INFO:tasks.workunit.client.1.smithi069.stdout:5/800: chown d1/d6/d26/d33/d5d/d72/d78/d102/ccc 0 1 2024-03-22T07:16:11.769 INFO:tasks.workunit.client.1.smithi069.stdout:6/747: creat d4/d7/da/d3f/d76/fec x:0 0 0 2024-03-22T07:16:11.771 INFO:tasks.workunit.client.1.smithi069.stdout:5/801: unlink d1/d6/d26/d33/d5d/fc4 0 2024-03-22T07:16:11.771 INFO:tasks.workunit.client.1.smithi069.stdout:6/748: mkdir d4/d7/da/d3f/d76/dab/ded 0 2024-03-22T07:16:11.771 INFO:tasks.workunit.client.1.smithi069.stdout:5/802: write d1/d6/d26/d33/d5d/d72/d95/fe4 [4514955,23158] 0 2024-03-22T07:16:11.779 INFO:tasks.workunit.client.1.smithi069.stdout:6/749: dread d4/fb [8388608,4194304] 0 2024-03-22T07:16:11.779 INFO:tasks.workunit.client.1.smithi069.stdout:6/750: truncate d4/dd/d41/d78/f9c 4593667 0 2024-03-22T07:16:11.779 INFO:tasks.workunit.client.1.smithi069.stdout:6/751: chown d4/dd/d12/d2b/da5/db2 3639 1 2024-03-22T07:16:11.779 INFO:tasks.workunit.client.1.smithi069.stdout:2/833: dwrite d18/d4d/d3e/d40/f5c [4194304,4194304] 0 2024-03-22T07:16:11.780 INFO:tasks.workunit.client.1.smithi069.stdout:2/834: creat d18/d4d/d27/d33/d4f/ddd/f10b x:0 0 0 2024-03-22T07:16:11.780 INFO:tasks.workunit.client.1.smithi069.stdout:2/835: stat d18/d91/f98 0 2024-03-22T07:16:11.780 INFO:tasks.workunit.client.1.smithi069.stdout:6/752: rename d4/dd/d41/d78/f85 to d4/d7/da/d3f/d76/fee 0 2024-03-22T07:16:11.781 INFO:tasks.workunit.client.1.smithi069.stdout:2/836: mknod d18/d4d/d27/d33/d60/c10c 0 2024-03-22T07:16:11.782 INFO:tasks.workunit.client.1.smithi069.stdout:2/837: creat d18/d4d/d27/d33/f10d x:0 0 0 2024-03-22T07:16:11.782 INFO:tasks.workunit.client.1.smithi069.stdout:2/838: write d18/d4d/d27/d33/f6b [514955,43824] 0 2024-03-22T07:16:11.783 INFO:tasks.workunit.client.1.smithi069.stdout:2/839: mknod d18/c10e 0 2024-03-22T07:16:11.786 INFO:tasks.workunit.client.1.smithi069.stdout:2/840: rename d18/d4d/d27/d33/d3d/dc6 to d18/d4d/d27/d33/d60/d64/d89/d10f 0 2024-03-22T07:16:11.790 INFO:tasks.workunit.client.1.smithi069.stdout:6/753: write d4/fb [1440868,86095] 0 2024-03-22T07:16:11.790 INFO:tasks.workunit.client.1.smithi069.stdout:6/754: write d4/dd/d12/d44/d58/db4/fbc [150062,59992] 0 2024-03-22T07:16:11.791 INFO:tasks.workunit.client.1.smithi069.stdout:6/755: rename d4/f6c to d4/d7/da/dc6/fef 0 2024-03-22T07:16:11.798 INFO:tasks.workunit.client.1.smithi069.stdout:5/803: dwrite d1/d10/d60/f83 [0,4194304] 0 2024-03-22T07:16:11.799 INFO:tasks.workunit.client.1.smithi069.stdout:5/804: write d1/d6/d26/d33/f4e [1620357,76863] 0 2024-03-22T07:16:11.799 INFO:tasks.workunit.client.1.smithi069.stdout:5/805: chown d1/d6/c81 28306728 1 2024-03-22T07:16:11.803 INFO:tasks.workunit.client.1.smithi069.stdout:5/806: creat d1/d10/d44/dd8/d101/f115 x:0 0 0 2024-03-22T07:16:11.803 INFO:tasks.workunit.client.1.smithi069.stdout:5/807: write d1/d6/d86/d99/d43/ff6 [3484281,30094] 0 2024-03-22T07:16:11.804 INFO:tasks.workunit.client.1.smithi069.stdout:6/756: dread d4/dd/d12/d13/f23 [0,4194304] 0 2024-03-22T07:16:11.804 INFO:tasks.workunit.client.1.smithi069.stdout:6/757: creat d4/dd/d12/d44/ff0 x:0 0 0 2024-03-22T07:16:11.804 INFO:tasks.workunit.client.1.smithi069.stdout:6/758: chown d4/dd/d12/d2b/d72 4125456 1 2024-03-22T07:16:11.804 INFO:tasks.workunit.client.1.smithi069.stdout:6/759: fdatasync d4/dd/d12/d44/d58/db4/fc4 0 2024-03-22T07:16:11.804 INFO:tasks.workunit.client.1.smithi069.stdout:5/808: symlink d1/d6/d26/d33/d5d/d72/d95/l116 0 2024-03-22T07:16:11.809 INFO:tasks.workunit.client.1.smithi069.stdout:6/760: dread d4/dd/f4a [0,4194304] 0 2024-03-22T07:16:11.810 INFO:tasks.workunit.client.1.smithi069.stdout:6/761: write d4/dd/f42 [139812,106764] 0 2024-03-22T07:16:11.827 INFO:tasks.workunit.client.1.smithi069.stdout:5/809: dwrite d1/d6/d88/fd7 [0,4194304] 0 2024-03-22T07:16:11.827 INFO:tasks.workunit.client.1.smithi069.stdout:6/762: dwrite d4/dd/d12/d13/f59 [0,4194304] 0 2024-03-22T07:16:11.827 INFO:tasks.workunit.client.1.smithi069.stdout:6/763: chown d4/dd/dbd/lc0 1970 1 2024-03-22T07:16:11.827 INFO:tasks.workunit.client.1.smithi069.stdout:6/764: fsync d4/dd/d12/d2b/d72/f95 0 2024-03-22T07:16:11.829 INFO:tasks.workunit.client.1.smithi069.stdout:6/765: dread d4/dd/f54 [0,4194304] 0 2024-03-22T07:16:11.829 INFO:tasks.workunit.client.1.smithi069.stdout:6/766: fdatasync d4/dd/d12/f32 0 2024-03-22T07:16:11.830 INFO:tasks.workunit.client.1.smithi069.stdout:5/810: rename d1/d6/d26/d33/d5d/d72/d95/d103 to d1/d6/d26/d3e/dbb/d117 0 2024-03-22T07:16:11.832 INFO:tasks.workunit.client.1.smithi069.stdout:6/767: rename d4/d7/d5b/f70 to d4/dd/d12/d2b/d72/d73/d9d/dc3/ff1 0 2024-03-22T07:16:11.833 INFO:tasks.workunit.client.1.smithi069.stdout:6/768: readlink d4/dd/d41/d78/da3/lb8 0 2024-03-22T07:16:11.835 INFO:tasks.workunit.client.1.smithi069.stdout:5/811: rename d1/d6/d88/fa1 to d1/d6/d26/d33/d5d/d72/d78/f118 0 2024-03-22T07:16:11.837 INFO:tasks.workunit.client.1.smithi069.stdout:5/812: creat d1/d7/d3d/dfa/f119 x:0 0 0 2024-03-22T07:16:11.837 INFO:tasks.workunit.client.1.smithi069.stdout:5/813: write d1/d6/d26/d7d/fda [81801,69682] 0 2024-03-22T07:16:11.837 INFO:tasks.workunit.client.1.smithi069.stdout:5/814: chown d1/d6/d26/d33/d5d/d72/d78/d102/dea/d10f 2 1 2024-03-22T07:16:11.837 INFO:tasks.workunit.client.1.smithi069.stdout:6/769: dread d4/f6b [0,4194304] 0 2024-03-22T07:16:11.839 INFO:tasks.workunit.client.1.smithi069.stdout:6/770: symlink d4/dd/d12/d2b/d72/lf2 0 2024-03-22T07:16:11.839 INFO:tasks.workunit.client.1.smithi069.stdout:6/771: truncate d4/d7/da/d3f/d76/fee 2772034 0 2024-03-22T07:16:11.866 INFO:tasks.workunit.client.1.smithi069.stdout:5/815: dwrite d1/d6/d86/d99/f52 [0,4194304] 0 2024-03-22T07:16:11.867 INFO:tasks.workunit.client.1.smithi069.stdout:6/772: dwrite d4/dd/d41/d78/d9a/fd8 [0,4194304] 0 2024-03-22T07:16:11.867 INFO:tasks.workunit.client.1.smithi069.stdout:6/773: chown d4/dd/d41/d78/da3/lb8 62377876 1 2024-03-22T07:16:11.867 INFO:tasks.workunit.client.1.smithi069.stdout:6/774: write d4/f46 [5140982,94753] 0 2024-03-22T07:16:11.867 INFO:tasks.workunit.client.1.smithi069.stdout:6/775: readlink d4/d7/da/l38 0 2024-03-22T07:16:11.872 INFO:tasks.workunit.client.1.smithi069.stdout:6/776: fdatasync d4/dd/d12/d44/d58/db4/fbc 0 2024-03-22T07:16:11.887 INFO:tasks.workunit.client.1.smithi069.stdout:6/777: write d4/dd/d12/d44/d58/f75 [291046,119256] 0 2024-03-22T07:16:11.901 INFO:tasks.workunit.client.1.smithi069.stdout:5/816: dwrite d1/d6/d86/d99/f58 [4194304,4194304] 0 2024-03-22T07:16:12.002 INFO:tasks.workunit.client.1.smithi069.stdout:6/778: rename d4/dd/d12/d44/f9b to d4/dd/d12/d13/de9/ff3 0 2024-03-22T07:16:12.027 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:11 smithi063 ceph-mon[29200]: pgmap v8: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 202 MiB/s rd, 227 MiB/s wr, 335 op/s 2024-03-22T07:16:12.028 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:11 smithi063 ceph-mon[29200]: Standby manager daemon smithi063.mnxayh started 2024-03-22T07:16:12.028 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:11 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.63:0/2894250342' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.mnxayh/crt"}]: dispatch 2024-03-22T07:16:12.028 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:11 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.63:0/2894250342' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:16:12.028 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:11 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.63:0/2894250342' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.mnxayh/key"}]: dispatch 2024-03-22T07:16:12.028 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:11 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.63:0/2894250342' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:16:12.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:11 smithi069 ceph-mon[36441]: pgmap v8: 65 pgs: 65 active+clean; 3.3 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 202 MiB/s rd, 227 MiB/s wr, 335 op/s 2024-03-22T07:16:12.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:11 smithi069 ceph-mon[36441]: Standby manager daemon smithi063.mnxayh started 2024-03-22T07:16:12.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:11 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.63:0/2894250342' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.mnxayh/crt"}]: dispatch 2024-03-22T07:16:12.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:11 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.63:0/2894250342' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:16:12.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:11 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.63:0/2894250342' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.mnxayh/key"}]: dispatch 2024-03-22T07:16:12.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:11 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.63:0/2894250342' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:16:12.048 INFO:tasks.workunit.client.1.smithi069.stdout:5/817: mknod d1/d6/d26/d3e/dbb/dbc/c11a 0 2024-03-22T07:16:12.097 INFO:tasks.workunit.client.1.smithi069.stdout:6/779: rmdir d4/dd/d12 39 2024-03-22T07:16:12.097 INFO:tasks.workunit.client.1.smithi069.stdout:6/780: chown d4/d7/l91 398857141 1 2024-03-22T07:16:12.097 INFO:tasks.workunit.client.1.smithi069.stdout:6/781: fsync d4/dd/f25 0 2024-03-22T07:16:12.097 INFO:tasks.workunit.client.1.smithi069.stdout:6/782: write d4/dd/d12/d44/f4e [1977554,74987] 0 2024-03-22T07:16:12.110 INFO:tasks.workunit.client.1.smithi069.stdout:6/783: dread d4/dd/d12/d2b/d72/d73/d9d/dc3/ff1 [0,4194304] 0 2024-03-22T07:16:12.114 INFO:tasks.workunit.client.1.smithi069.stdout:9/896: sync 2024-03-22T07:16:12.114 INFO:tasks.workunit.client.1.smithi069.stdout:9/897: write d2/dd/d3c/d48/fe8 [806662,63140] 0 2024-03-22T07:16:12.114 INFO:tasks.workunit.client.1.smithi069.stdout:2/841: sync 2024-03-22T07:16:12.114 INFO:tasks.workunit.client.1.smithi069.stdout:7/901: sync 2024-03-22T07:16:12.114 INFO:tasks.workunit.client.1.smithi069.stdout:7/902: chown d9/db/d1a/db1/d57/d75/de2/d122/f6a 6778 1 2024-03-22T07:16:12.114 INFO:tasks.workunit.client.1.smithi069.stdout:2/842: creat d18/d4d/d27/d33/d60/d63/d73/f110 x:0 0 0 2024-03-22T07:16:12.115 INFO:tasks.workunit.client.1.smithi069.stdout:2/843: truncate d18/d4d/d3e/f86 4195645 0 2024-03-22T07:16:12.115 INFO:tasks.workunit.client.1.smithi069.stdout:0/995: sync 2024-03-22T07:16:12.115 INFO:tasks.workunit.client.1.smithi069.stdout:1/895: sync 2024-03-22T07:16:12.115 INFO:tasks.workunit.client.1.smithi069.stdout:6/784: write d4/d7/da/d3f/f99 [1025607,117690] 0 2024-03-22T07:16:12.116 INFO:tasks.workunit.client.1.smithi069.stdout:7/903: mkdir d9/d8e/d123 0 2024-03-22T07:16:12.116 INFO:tasks.workunit.client.1.smithi069.stdout:7/904: truncate d9/d8e/f102 709360 0 2024-03-22T07:16:12.117 INFO:tasks.workunit.client.1.smithi069.stdout:2/844: mknod d18/d4d/d27/d33/d60/d63/d93/def/c111 0 2024-03-22T07:16:12.120 INFO:tasks.workunit.client.1.smithi069.stdout:1/896: unlink d1/d13/dc7/d121/d1e/c52 0 2024-03-22T07:16:12.120 INFO:tasks.workunit.client.1.smithi069.stdout:1/897: truncate d1/d2a/db6/df8/d73/df0/dbc/fce 903641 0 2024-03-22T07:16:12.120 INFO:tasks.workunit.client.1.smithi069.stdout:1/898: chown d1/d13/d36/de5/d88/dae/de7/lfb 14215883 1 2024-03-22T07:16:12.122 INFO:tasks.workunit.client.1.smithi069.stdout:6/785: creat d4/dd/d41/d78/ff4 x:0 0 0 2024-03-22T07:16:12.123 INFO:tasks.workunit.client.1.smithi069.stdout:6/786: stat d4/dd/d41/d78/da3/cc5 0 2024-03-22T07:16:12.123 INFO:tasks.workunit.client.1.smithi069.stdout:6/787: chown d4/d7/da/d3f/d76/dab/laf 7 1 2024-03-22T07:16:12.125 INFO:tasks.workunit.client.1.smithi069.stdout:7/905: dread d9/db/d1a/db1/d57/d75/d7a/fa2 [0,4194304] 0 2024-03-22T07:16:12.125 INFO:tasks.workunit.client.1.smithi069.stdout:0/996: rmdir d3/d7/d15/d29/d5a/d5d/d9d/df9 39 2024-03-22T07:16:12.126 INFO:tasks.workunit.client.1.smithi069.stdout:1/899: creat d1/d2a/d5e/f123 x:0 0 0 2024-03-22T07:16:12.129 INFO:tasks.workunit.client.1.smithi069.stdout:0/997: dread d3/d7/d17/f85 [0,4194304] 0 2024-03-22T07:16:12.147 INFO:tasks.workunit.client.1.smithi069.stdout:9/898: dwrite d2/dd/d3c/d48/d7f/fa0 [0,4194304] 0 2024-03-22T07:16:12.147 INFO:tasks.workunit.client.1.smithi069.stdout:9/899: dread - d2/dd/d3c/d2e/d84/f88 zero size 2024-03-22T07:16:12.149 INFO:tasks.workunit.client.1.smithi069.stdout:9/900: creat d2/dd/f126 x:0 0 0 2024-03-22T07:16:12.157 INFO:tasks.workunit.client.1.smithi069.stdout:9/901: dread d2/dd/d3c/d48/d7f/d86/fc0 [0,4194304] 0 2024-03-22T07:16:12.162 INFO:tasks.workunit.client.1.smithi069.stdout:0/998: dwrite d3/d7/d15/d29/d5a/d5d/db4/f148 [0,4194304] 0 2024-03-22T07:16:12.162 INFO:tasks.workunit.client.1.smithi069.stdout:0/999: chown d3/d7/d15/d1f/c66 132861 1 2024-03-22T07:16:12.162 INFO:tasks.workunit.client.1.smithi069.stdout:2/845: dwrite d18/d4d/d27/d56/f62 [4194304,4194304] 0 2024-03-22T07:16:12.163 INFO:tasks.workunit.client.1.smithi069.stdout:2/846: read - d18/d4d/d27/d33/d4f/ddd/fe4 zero size 2024-03-22T07:16:12.163 INFO:tasks.workunit.client.1.smithi069.stdout:2/847: write f1 [541904,119775] 0 2024-03-22T07:16:12.163 INFO:tasks.workunit.client.1.smithi069.stdout:1/900: dwrite d1/d13/dc7/d121/d1e/fd5 [0,4194304] 0 2024-03-22T07:16:12.163 INFO:tasks.workunit.client.1.smithi069.stdout:1/901: write d1/d13/d36/de5/d88/f38 [1011565,101324] 0 2024-03-22T07:16:12.167 INFO:tasks.workunit.client.1.smithi069.stdout:2/848: mknod d18/d4d/d27/d33/d4f/ddd/df4/c112 0 2024-03-22T07:16:12.169 INFO:tasks.workunit.client.1.smithi069.stdout:1/902: unlink d1/f48 0 2024-03-22T07:16:12.171 INFO:tasks.workunit.client.1.smithi069.stdout:2/849: mkdir d18/d4d/d27/d33/d60/d64/d89/d8b/dab/d113 0 2024-03-22T07:16:12.171 INFO:tasks.workunit.client.1.smithi069.stdout:2/850: mkdir d18/d4d/d27/d33/d60/d64/dcd/d114 0 2024-03-22T07:16:12.173 INFO:tasks.workunit.client.1.smithi069.stdout:9/902: dwrite d2/d21/f3a [0,4194304] 0 2024-03-22T07:16:12.173 INFO:tasks.workunit.client.1.smithi069.stdout:9/903: read - d2/d31/dc6/d11f/f11c zero size 2024-03-22T07:16:12.176 INFO:tasks.workunit.client.1.smithi069.stdout:2/851: dread d18/d4d/d27/d77/fc7 [0,4194304] 0 2024-03-22T07:16:12.178 INFO:tasks.workunit.client.1.smithi069.stdout:9/904: rename d2/dd/d6e/f98 to d2/dd/d3c/d48/d7f/d86/f127 0 2024-03-22T07:16:12.178 INFO:tasks.workunit.client.1.smithi069.stdout:2/852: rename d18 to d18/d4d/d27/d33/d60/d64/d89/d8b/dab/d115 22 2024-03-22T07:16:12.182 INFO:tasks.workunit.client.1.smithi069.stdout:2/853: truncate d18/d4d/d3e/d40/d96/fed 1092866 0 2024-03-22T07:16:12.182 INFO:tasks.workunit.client.1.smithi069.stdout:2/854: chown d18/d4d/d27/d33/d60/d63/d73/dc9/le5 4 1 2024-03-22T07:16:12.188 INFO:tasks.workunit.client.1.smithi069.stdout:2/855: write d18/f1e [534591,98214] 0 2024-03-22T07:16:12.189 INFO:tasks.workunit.client.1.smithi069.stdout:9/905: dread d2/d51/f59 [4194304,4194304] 0 2024-03-22T07:16:12.189 INFO:tasks.workunit.client.1.smithi069.stdout:9/906: dread - d2/d83/f108 zero size 2024-03-22T07:16:12.191 INFO:tasks.workunit.client.1.smithi069.stdout:2/856: creat d18/d4d/d27/d33/d60/d64/d89/d8b/f116 x:0 0 0 2024-03-22T07:16:12.194 INFO:tasks.workunit.client.1.smithi069.stdout:2/857: dread d18/d4d/d27/d33/d60/d64/f99 [0,4194304] 0 2024-03-22T07:16:12.194 INFO:tasks.workunit.client.1.smithi069.stdout:2/858: dread - d18/d4d/d27/d33/d60/d63/d93/fcf zero size 2024-03-22T07:16:12.195 INFO:tasks.workunit.client.1.smithi069.stdout:2/859: rmdir d18/d4d/d27/d33/d60/d63 39 2024-03-22T07:16:12.199 INFO:tasks.workunit.client.1.smithi069.stdout:2/860: mkdir d18/d91/d117 0 2024-03-22T07:16:12.201 INFO:tasks.workunit.client.1.smithi069.stdout:2/861: creat d18/d4d/d3e/f118 x:0 0 0 2024-03-22T07:16:12.204 INFO:tasks.workunit.client.1.smithi069.stdout:2/862: unlink d18/d4d/d27/d72/fb7 0 2024-03-22T07:16:12.209 INFO:tasks.workunit.client.1.smithi069.stdout:9/907: dwrite d2/dd/fac [0,4194304] 0 2024-03-22T07:16:12.213 INFO:tasks.workunit.client.1.smithi069.stdout:9/908: link d2/d76/f106 d2/d21/dd7/f128 0 2024-03-22T07:16:12.214 INFO:tasks.workunit.client.1.smithi069.stdout:9/909: symlink d2/d71/d90/dfa/l129 0 2024-03-22T07:16:12.214 INFO:tasks.workunit.client.1.smithi069.stdout:9/910: creat d2/dd/d3c/d48/d7f/da2/f12a x:0 0 0 2024-03-22T07:16:12.215 INFO:tasks.workunit.client.1.smithi069.stdout:9/911: mkdir d2/dd/d3c/d48/d7f/da2/d12b 0 2024-03-22T07:16:12.215 INFO:tasks.workunit.client.1.smithi069.stdout:9/912: readlink d2/d21/l6b 0 2024-03-22T07:16:12.215 INFO:tasks.workunit.client.1.smithi069.stdout:9/913: stat d2/dd/d3c/d53/da6 0 2024-03-22T07:16:12.216 INFO:tasks.workunit.client.1.smithi069.stdout:9/914: creat d2/d21/dd7/f12c x:0 0 0 2024-03-22T07:16:12.216 INFO:tasks.workunit.client.1.smithi069.stdout:9/915: creat d2/d76/f12d x:0 0 0 2024-03-22T07:16:12.217 INFO:tasks.workunit.client.1.smithi069.stdout:2/863: dwrite d18/d4d/d27/d33/f6b [0,4194304] 0 2024-03-22T07:16:12.221 INFO:tasks.workunit.client.1.smithi069.stdout:2/864: mkdir d18/d4d/d27/d33/d60/d63/d73/d119 0 2024-03-22T07:16:12.221 INFO:tasks.workunit.client.1.smithi069.stdout:2/865: fsync d18/d4d/d27/d33/d60/d64/d89/f90 0 2024-03-22T07:16:12.224 INFO:tasks.workunit.client.1.smithi069.stdout:2/866: rmdir d18/d4d/d27/d33/d60/d64/d89/dcb/dd3 0 2024-03-22T07:16:12.224 INFO:tasks.workunit.client.1.smithi069.stdout:2/867: chown d18/d4d/d25 24992865 1 2024-03-22T07:16:12.224 INFO:tasks.workunit.client.1.smithi069.stdout:2/868: dread - d18/d4d/d27/d33/d60/d64/dcd/ff2 zero size 2024-03-22T07:16:12.233 INFO:tasks.workunit.client.1.smithi069.stdout:2/869: write d18/f19 [2594990,128459] 0 2024-03-22T07:16:12.233 INFO:tasks.workunit.client.1.smithi069.stdout:2/870: readlink d18/d4d/d3e/d40/l75 0 2024-03-22T07:16:12.234 INFO:tasks.workunit.client.1.smithi069.stdout:2/871: rmdir d18/d4d 39 2024-03-22T07:16:12.234 INFO:tasks.workunit.client.1.smithi069.stdout:2/872: creat d18/d4d/d27/d33/d3d/f11a x:0 0 0 2024-03-22T07:16:12.234 INFO:tasks.workunit.client.1.smithi069.stdout:2/873: write d18/f3a [2655691,17066] 0 2024-03-22T07:16:12.237 INFO:tasks.workunit.client.1.smithi069.stdout:2/874: rename fa to d18/d4d/d27/d33/d60/d63/d80/d100/f11b 0 2024-03-22T07:16:12.239 INFO:tasks.workunit.client.1.smithi069.stdout:2/875: creat d18/d4d/d27/d33/d60/d64/dcd/d114/f11c x:0 0 0 2024-03-22T07:16:12.239 INFO:tasks.workunit.client.1.smithi069.stdout:2/876: truncate d18/d4d/d27/d56/f94 1806019 0 2024-03-22T07:16:12.240 INFO:tasks.workunit.client.1.smithi069.stdout:2/877: symlink d18/d4d/d27/d33/d60/l11d 0 2024-03-22T07:16:12.240 INFO:tasks.workunit.client.1.smithi069.stdout:2/878: rmdir d18/d4d/d27/d33/d60/d64/d89/d8b/dab 39 2024-03-22T07:16:12.241 INFO:tasks.workunit.client.1.smithi069.stdout:2/879: mknod d18/d4d/d27/d33/d60/d64/dcd/c11e 0 2024-03-22T07:16:12.242 INFO:tasks.workunit.client.1.smithi069.stdout:2/880: symlink d18/d4d/d27/d33/d60/dbd/ddc/l11f 0 2024-03-22T07:16:12.257 INFO:tasks.workunit.client.1.smithi069.stdout:2/881: dwrite d18/d4d/d3e/fc8 [0,4194304] 0 2024-03-22T07:16:12.258 INFO:tasks.workunit.client.1.smithi069.stdout:2/882: unlink d18/f3a 0 2024-03-22T07:16:12.260 INFO:tasks.workunit.client.1.smithi069.stdout:2/883: dread d18/d4d/d3e/d40/d96/fed [0,4194304] 0 2024-03-22T07:16:12.260 INFO:tasks.workunit.client.1.smithi069.stdout:2/884: creat d18/d4d/d3e/f120 x:0 0 0 2024-03-22T07:16:12.261 INFO:tasks.workunit.client.1.smithi069.stdout:2/885: creat d18/d4d/d27/d33/d60/d64/d89/f121 x:0 0 0 2024-03-22T07:16:12.261 INFO:tasks.workunit.client.1.smithi069.stdout:2/886: dread - d18/d4d/d27/d33/fd0 zero size 2024-03-22T07:16:12.261 INFO:tasks.workunit.client.1.smithi069.stdout:2/887: chown d18/d4d/d27/d33/d3d/f54 3694812 1 2024-03-22T07:16:12.275 INFO:tasks.workunit.client.1.smithi069.stdout:2/888: dwrite d18/d4d/d27/d33/d60/fb9 [0,4194304] 0 2024-03-22T07:16:12.275 INFO:tasks.workunit.client.1.smithi069.stdout:2/889: read - d18/d4d/d27/d33/d60/d63/d93/fcf zero size 2024-03-22T07:16:12.275 INFO:tasks.workunit.client.1.smithi069.stdout:2/890: dread - d18/d4d/d27/d33/f82 zero size 2024-03-22T07:16:12.284 INFO:tasks.workunit.client.1.smithi069.stdout:2/891: dread d18/d4d/d3e/fc8 [0,4194304] 0 2024-03-22T07:16:12.284 INFO:tasks.workunit.client.1.smithi069.stdout:2/892: chown d18/d91/d117 4 1 2024-03-22T07:16:12.287 INFO:tasks.workunit.client.1.smithi069.stdout:2/893: truncate f1 3553952 0 2024-03-22T07:16:12.289 INFO:tasks.workunit.client.1.smithi069.stdout:2/894: symlink d18/d4d/d27/d33/d60/d63/d93/def/l122 0 2024-03-22T07:16:12.289 INFO:tasks.workunit.client.1.smithi069.stdout:2/895: chown d18/d4d/d27/d33/d60/f5a 3579630 1 2024-03-22T07:16:12.304 INFO:tasks.workunit.client.1.smithi069.stdout:2/896: dwrite d18/d4d/d27/d33/d60/d64/d89/d10f/fe2 [0,4194304] 0 2024-03-22T07:16:12.307 INFO:tasks.workunit.client.1.smithi069.stdout:2/897: rename d18/d4d/d27/d33/l3b to d18/d4d/d27/d77/l123 0 2024-03-22T07:16:12.308 INFO:tasks.workunit.client.1.smithi069.stdout:2/898: mknod d18/d91/c124 0 2024-03-22T07:16:12.309 INFO:tasks.workunit.client.1.smithi069.stdout:2/899: rmdir d18/d4d/d27/d72 39 2024-03-22T07:16:12.309 INFO:tasks.workunit.client.1.smithi069.stdout:2/900: symlink d18/d4d/d27/d77/l125 0 2024-03-22T07:16:12.343 INFO:tasks.workunit.client.1.smithi069.stdout:2/901: dwrite d18/d4d/d27/d33/d60/d64/dcd/d114/f11c [0,4194304] 0 2024-03-22T07:16:12.345 INFO:tasks.workunit.client.1.smithi069.stdout:2/902: creat d18/d4d/d27/d33/d4f/f126 x:0 0 0 2024-03-22T07:16:12.346 INFO:tasks.workunit.client.1.smithi069.stdout:2/903: getdents d18/d4d/d27/d33/d60/d64/d89/d8b/dab 0 2024-03-22T07:16:12.346 INFO:tasks.workunit.client.1.smithi069.stdout:2/904: write d18/d4d/d27/d33/f82 [678574,3022] 0 2024-03-22T07:16:12.348 INFO:tasks.workunit.client.1.smithi069.stdout:2/905: dread d18/d4d/d27/d33/d60/d64/fc5 [0,4194304] 0 2024-03-22T07:16:12.351 INFO:tasks.workunit.client.1.smithi069.stdout:2/906: dread d18/d4d/f26 [0,4194304] 0 2024-03-22T07:16:12.358 INFO:tasks.workunit.client.1.smithi069.stdout:2/907: read d18/d4d/d3e/f5d [1082021,84466] 0 2024-03-22T07:16:12.358 INFO:tasks.workunit.client.1.smithi069.stdout:2/908: rmdir d18/d4d/d3e 39 2024-03-22T07:16:12.360 INFO:tasks.workunit.client.1.smithi069.stdout:2/909: dread d18/d4d/d27/d33/d60/d64/fac [0,4194304] 0 2024-03-22T07:16:12.362 INFO:tasks.workunit.client.1.smithi069.stdout:2/910: dread d18/d4d/d27/d33/d60/d64/f99 [0,4194304] 0 2024-03-22T07:16:12.365 INFO:tasks.workunit.client.1.smithi069.stdout:2/911: truncate d18/d4d/d27/d56/f94 1718366 0 2024-03-22T07:16:12.367 INFO:tasks.workunit.client.1.smithi069.stdout:2/912: dread d18/f1e [0,4194304] 0 2024-03-22T07:16:12.368 INFO:tasks.workunit.client.1.smithi069.stdout:2/913: truncate d18/d4d/d27/d77/fc7 951653 0 2024-03-22T07:16:12.368 INFO:tasks.workunit.client.1.smithi069.stdout:2/914: creat d18/d4d/d27/d33/d60/d63/f127 x:0 0 0 2024-03-22T07:16:12.384 INFO:tasks.workunit.client.1.smithi069.stdout:2/915: dwrite d18/d4d/d27/d77/fa0 [0,4194304] 0 2024-03-22T07:16:12.384 INFO:tasks.workunit.client.1.smithi069.stdout:2/916: symlink d18/d4d/d27/d33/d60/d64/d89/dcb/l128 0 2024-03-22T07:16:12.385 INFO:tasks.workunit.client.1.smithi069.stdout:2/917: truncate d18/d4d/d27/d33/d60/d63/d93/fcf 689641 0 2024-03-22T07:16:12.385 INFO:tasks.workunit.client.1.smithi069.stdout:2/918: creat d18/d4d/d3e/d40/f129 x:0 0 0 2024-03-22T07:16:12.386 INFO:tasks.workunit.client.1.smithi069.stdout:2/919: link d18/d4d/d27/d33/d60/d63/d93/fcf d18/d4d/d27/d33/d60/d64/d89/d8b/dab/dee/f12a 0 2024-03-22T07:16:12.387 INFO:tasks.workunit.client.1.smithi069.stdout:2/920: getdents d18/d4d/d27/d33/d60/d64/d89/d8b 0 2024-03-22T07:16:12.403 INFO:tasks.workunit.client.1.smithi069.stdout:2/921: dwrite d18/d4d/d27/d33/d4f/ddd/fe4 [0,4194304] 0 2024-03-22T07:16:12.404 INFO:tasks.workunit.client.1.smithi069.stdout:2/922: read d18/d4d/d3e/f86 [2449041,101588] 0 2024-03-22T07:16:12.591 INFO:tasks.workunit.client.1.smithi069.stdout:5/818: sync 2024-03-22T07:16:12.591 INFO:tasks.workunit.client.1.smithi069.stdout:6/788: sync 2024-03-22T07:16:12.592 INFO:tasks.workunit.client.1.smithi069.stdout:5/819: symlink d1/d6/d26/d33/d5d/d72/d78/d102/dea/d10f/l11b 0 2024-03-22T07:16:12.593 INFO:tasks.workunit.client.1.smithi069.stdout:5/820: creat d1/d6/d26/d33/d5d/d72/d78/f11c x:0 0 0 2024-03-22T07:16:12.599 INFO:tasks.workunit.client.1.smithi069.stdout:6/789: dread d4/d7/da/f1b [0,4194304] 0 2024-03-22T07:16:12.600 INFO:tasks.workunit.client.1.smithi069.stdout:6/790: fsync d4/dd/d12/d2b/f36 0 2024-03-22T07:16:12.600 INFO:tasks.workunit.client.1.smithi069.stdout:6/791: fsync d4/dd/d41/d78/f9c 0 2024-03-22T07:16:12.607 INFO:tasks.workunit.client.1.smithi069.stdout:6/792: write d4/fb [11000407,63108] 0 2024-03-22T07:16:12.608 INFO:tasks.workunit.client.1.smithi069.stdout:6/793: read - d4/d7/da/d3f/f6d zero size 2024-03-22T07:16:12.609 INFO:tasks.workunit.client.1.smithi069.stdout:5/821: dwrite d1/d10/d44/fb4 [0,4194304] 0 2024-03-22T07:16:12.610 INFO:tasks.workunit.client.1.smithi069.stdout:5/822: fdatasync d1/d6/d86/dab/fcb 0 2024-03-22T07:16:12.610 INFO:tasks.workunit.client.1.smithi069.stdout:5/823: write d1/d6/d86/dab/dcf/fcd [932666,91545] 0 2024-03-22T07:16:12.610 INFO:tasks.workunit.client.1.smithi069.stdout:6/794: rename d4/d7/da/d3f/c4d to d4/dd/cf5 0 2024-03-22T07:16:12.614 INFO:tasks.workunit.client.1.smithi069.stdout:5/824: rename d1/d6/d26/d33/d5d/d72/d78/d102/lbf to d1/d7/d3d/d100/l11d 0 2024-03-22T07:16:12.631 INFO:tasks.workunit.client.1.smithi069.stdout:6/795: dwrite d4/dd/f42 [0,4194304] 0 2024-03-22T07:16:12.631 INFO:tasks.workunit.client.1.smithi069.stdout:5/825: dwrite d1/d6/d26/d33/d5d/d72/d95/fe4 [0,4194304] 0 2024-03-22T07:16:12.634 INFO:tasks.workunit.client.1.smithi069.stdout:6/796: creat d4/dd/ff6 x:0 0 0 2024-03-22T07:16:12.634 INFO:tasks.workunit.client.1.smithi069.stdout:6/797: fdatasync d4/d7/da/d3f/f6d 0 2024-03-22T07:16:12.634 INFO:tasks.workunit.client.1.smithi069.stdout:5/826: symlink d1/d6/d86/dab/l11e 0 2024-03-22T07:16:12.634 INFO:tasks.workunit.client.1.smithi069.stdout:5/827: truncate d1/d6/d86/dab/fcb 844507 0 2024-03-22T07:16:12.635 INFO:tasks.workunit.client.1.smithi069.stdout:6/798: rename d4/dd/d41/d78/da3/lb8 to d4/d7/da/dc6/dc7/lf7 0 2024-03-22T07:16:12.637 INFO:tasks.workunit.client.1.smithi069.stdout:5/828: symlink d1/d6/d26/d33/d5d/l11f 0 2024-03-22T07:16:12.640 INFO:tasks.workunit.client.1.smithi069.stdout:6/799: rename d4/dd/d12/f1f to d4/dd/d12/d2b/ff8 0 2024-03-22T07:16:12.640 INFO:tasks.workunit.client.1.smithi069.stdout:6/800: chown d4/dd/d12/d2b/f36 249520 1 2024-03-22T07:16:12.640 INFO:tasks.workunit.client.1.smithi069.stdout:5/829: mkdir d1/d6/d26/d3e/dbb/d117/d120 0 2024-03-22T07:16:12.641 INFO:tasks.workunit.client.1.smithi069.stdout:5/830: read d1/d6/d86/f77 [417633,76103] 0 2024-03-22T07:16:12.641 INFO:tasks.workunit.client.1.smithi069.stdout:6/801: rename d4/f6 to d4/dd/ff9 0 2024-03-22T07:16:12.644 INFO:tasks.workunit.client.1.smithi069.stdout:6/802: rename d4/d7/da/f1b to d4/dd/d12/d44/d58/ffa 0 2024-03-22T07:16:12.646 INFO:tasks.workunit.client.1.smithi069.stdout:6/803: fsync d4/fb 0 2024-03-22T07:16:12.648 INFO:tasks.workunit.client.1.smithi069.stdout:6/804: rename d4/d7/da/d3f/d76/fec to d4/dd/d41/d78/ffb 0 2024-03-22T07:16:12.650 INFO:tasks.workunit.client.1.smithi069.stdout:6/805: write d4/d7/da/dc6/fef [497377,127720] 0 2024-03-22T07:16:12.650 INFO:tasks.workunit.client.1.smithi069.stdout:5/831: dread d1/d7/f59 [0,4194304] 0 2024-03-22T07:16:12.651 INFO:tasks.workunit.client.1.smithi069.stdout:5/832: fsync d1/d10/d60/f83 0 2024-03-22T07:16:12.651 INFO:tasks.workunit.client.1.smithi069.stdout:5/833: rmdir d1/d6/d26/d33/d5d/d72/d78/d102/dea/dba 39 2024-03-22T07:16:12.651 INFO:tasks.workunit.client.1.smithi069.stdout:5/834: chown d1/d6/d86/d99/f4b 22878 1 2024-03-22T07:16:12.652 INFO:tasks.workunit.client.1.smithi069.stdout:5/835: mknod d1/d10/d60/c121 0 2024-03-22T07:16:12.654 INFO:tasks.workunit.client.1.smithi069.stdout:5/836: getdents d1/d6/d26/d3a 0 2024-03-22T07:16:12.655 INFO:tasks.workunit.client.1.smithi069.stdout:5/837: unlink d1/d6/d21/f79 0 2024-03-22T07:16:12.655 INFO:tasks.workunit.client.1.smithi069.stdout:5/838: write d1/d6/d26/d33/d5d/d72/d95/faf [521242,10454] 0 2024-03-22T07:16:12.657 INFO:tasks.workunit.client.1.smithi069.stdout:5/839: truncate d1/d10/d60/fac 903202 0 2024-03-22T07:16:12.660 INFO:tasks.workunit.client.1.smithi069.stdout:6/806: rename d4/dd/d12/d2b/ff8 to d4/dd/d12/d13/de9/ffc 0 2024-03-22T07:16:12.660 INFO:tasks.workunit.client.1.smithi069.stdout:6/807: creat d4/d7/d5b/ffd x:0 0 0 2024-03-22T07:16:12.660 INFO:tasks.workunit.client.1.smithi069.stdout:6/808: chown d4/dd/d12/d2b/d72/d73/l7b 3 1 2024-03-22T07:16:12.660 INFO:tasks.workunit.client.1.smithi069.stdout:5/840: chown d1/d6/d26/d33/d5d/d72/c74 0 1 2024-03-22T07:16:12.662 INFO:tasks.workunit.client.1.smithi069.stdout:6/809: mkdir d4/dd/d12/d2b/d72/dfe 0 2024-03-22T07:16:12.662 INFO:tasks.workunit.client.1.smithi069.stdout:6/810: fsync d4/dd/d12/fd6 0 2024-03-22T07:16:12.663 INFO:tasks.workunit.client.1.smithi069.stdout:5/841: dread d1/d28/f7e [0,4194304] 0 2024-03-22T07:16:12.663 INFO:tasks.workunit.client.1.smithi069.stdout:5/842: readlink d1/d6/d86/d99/le7 0 2024-03-22T07:16:12.665 INFO:tasks.workunit.client.1.smithi069.stdout:5/843: link d1/d6/d26/d33/d5d/d72/d78/d102/cb6 d1/d6/d26/d7d/c122 0 2024-03-22T07:16:12.665 INFO:tasks.workunit.client.1.smithi069.stdout:5/844: chown d1/d10/d60/caa 942542101 1 2024-03-22T07:16:12.666 INFO:tasks.workunit.client.1.smithi069.stdout:5/845: readlink d1/d6/d26/l51 0 2024-03-22T07:16:12.669 INFO:tasks.workunit.client.1.smithi069.stdout:5/846: write d1/d6/d86/d99/d43/d8a/fff [1021934,101676] 0 2024-03-22T07:16:12.669 INFO:tasks.workunit.client.1.smithi069.stdout:5/847: fdatasync d1/d6/d26/d33/d5d/d72/d78/f118 0 2024-03-22T07:16:12.670 INFO:tasks.workunit.client.1.smithi069.stdout:5/848: readlink d1/d6/d86/d99/l1b 0 2024-03-22T07:16:12.671 INFO:tasks.workunit.client.1.smithi069.stdout:5/849: rename d1/d6/d26/d33/d5d/d72/da7/c111 to d1/d6/d26/d33/d5d/d72/d78/d102/dea/c123 0 2024-03-22T07:16:12.671 INFO:tasks.workunit.client.1.smithi069.stdout:5/850: dread - d1/d10/d44/dd8/de6/ffe zero size 2024-03-22T07:16:12.671 INFO:tasks.workunit.client.1.smithi069.stdout:5/851: chown d1/d10/df1/d109 198398 1 2024-03-22T07:16:12.676 INFO:tasks.workunit.client.1.smithi069.stdout:6/811: dwrite d4/dd/f8f [4194304,4194304] 0 2024-03-22T07:16:12.677 INFO:tasks.workunit.client.1.smithi069.stdout:6/812: chown d4/dd/d12/d13/f8a 105917 1 2024-03-22T07:16:12.677 INFO:tasks.workunit.client.1.smithi069.stdout:6/813: creat d4/d7/da/d3f/fff x:0 0 0 2024-03-22T07:16:12.679 INFO:tasks.workunit.client.1.smithi069.stdout:9/916: sync 2024-03-22T07:16:12.679 INFO:tasks.workunit.client.1.smithi069.stdout:7/906: sync 2024-03-22T07:16:12.679 INFO:tasks.workunit.client.1.smithi069.stdout:2/923: sync 2024-03-22T07:16:12.680 INFO:tasks.workunit.client.1.smithi069.stdout:1/903: sync 2024-03-22T07:16:12.681 INFO:tasks.workunit.client.1.smithi069.stdout:7/907: stat d9/db/d1a/db1/db3/cf8 0 2024-03-22T07:16:12.683 INFO:tasks.workunit.client.1.smithi069.stdout:2/924: creat d18/d4d/d27/d33/d60/d63/d93/f12b x:0 0 0 2024-03-22T07:16:12.685 INFO:tasks.workunit.client.1.smithi069.stdout:1/904: creat d1/f124 x:0 0 0 2024-03-22T07:16:12.687 INFO:tasks.workunit.client.1.smithi069.stdout:6/814: dread d4/dd/d12/d44/f4e [0,4194304] 0 2024-03-22T07:16:12.688 INFO:tasks.workunit.client.1.smithi069.stdout:7/908: mkdir d9/db/d1a/db1/d57/d44/d53/dc7/db5/d124 0 2024-03-22T07:16:12.688 INFO:tasks.workunit.client.1.smithi069.stdout:7/909: creat d9/db/d48/dab/f125 x:0 0 0 2024-03-22T07:16:12.690 INFO:tasks.workunit.client.1.smithi069.stdout:7/910: read d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/fae [784335,77462] 0 2024-03-22T07:16:12.690 INFO:tasks.workunit.client.1.smithi069.stdout:7/911: write d9/db/d1a/db1/d57/d75/de2/d122/f7c [4205646,69286] 0 2024-03-22T07:16:12.690 INFO:tasks.workunit.client.1.smithi069.stdout:2/925: creat d18/d4d/d27/d33/d60/d64/d89/dcb/f12c x:0 0 0 2024-03-22T07:16:12.694 INFO:tasks.workunit.client.1.smithi069.stdout:2/926: getdents d18/d4d/d27/d72 0 2024-03-22T07:16:12.694 INFO:tasks.workunit.client.1.smithi069.stdout:2/927: stat d18/d4d/d3e/d40/f5c 0 2024-03-22T07:16:12.696 INFO:tasks.workunit.client.1.smithi069.stdout:9/917: dwrite d2/d71/d90/f95 [0,4194304] 0 2024-03-22T07:16:12.697 INFO:tasks.workunit.client.1.smithi069.stdout:6/815: rmdir d4/dd/d41/d78 39 2024-03-22T07:16:12.698 INFO:tasks.workunit.client.1.smithi069.stdout:2/928: rmdir d18/d4d/d27 39 2024-03-22T07:16:12.698 INFO:tasks.workunit.client.1.smithi069.stdout:2/929: read - d18/d4d/d27/d33/d60/d63/d73/dc9/f10a zero size 2024-03-22T07:16:12.699 INFO:tasks.workunit.client.1.smithi069.stdout:9/918: symlink d2/d31/l12e 0 2024-03-22T07:16:12.699 INFO:tasks.workunit.client.1.smithi069.stdout:9/919: chown d2/dd/d3c/d2e/d84/da5/de0/lea 6 1 2024-03-22T07:16:12.700 INFO:tasks.workunit.client.1.smithi069.stdout:2/930: write d18/d4d/d27/d33/d60/d63/faa [1750162,27974] 0 2024-03-22T07:16:12.702 INFO:tasks.workunit.client.1.smithi069.stdout:9/920: mkdir d2/dd/d3c/d48/d7f/d86/d12f 0 2024-03-22T07:16:12.703 INFO:tasks.workunit.client.1.smithi069.stdout:7/912: write d9/db/d1a/db1/d57/f117 [2499710,103990] 0 2024-03-22T07:16:12.704 INFO:tasks.workunit.client.1.smithi069.stdout:2/931: creat d18/d4d/d27/d33/d4f/ddd/df4/f12d x:0 0 0 2024-03-22T07:16:12.707 INFO:tasks.workunit.client.1.smithi069.stdout:2/932: symlink d18/d4d/dec/l12e 0 2024-03-22T07:16:12.707 INFO:tasks.workunit.client.1.smithi069.stdout:6/816: write d4/dd/d12/f32 [3631737,11312] 0 2024-03-22T07:16:12.707 INFO:tasks.workunit.client.1.smithi069.stdout:6/817: chown d4/dd/d12/d2b/da5/db2 18 1 2024-03-22T07:16:12.707 INFO:tasks.workunit.client.1.smithi069.stdout:6/818: write d4/d7/da/dc6/fef [787930,88058] 0 2024-03-22T07:16:12.708 INFO:tasks.workunit.client.1.smithi069.stdout:7/913: mknod d9/db/d1a/db1/d57/d44/d53/c126 0 2024-03-22T07:16:12.708 INFO:tasks.workunit.client.1.smithi069.stdout:2/933: getdents d18/d4d/d27/d56 0 2024-03-22T07:16:12.708 INFO:tasks.workunit.client.1.smithi069.stdout:2/934: readlink d18/d4d/l84 0 2024-03-22T07:16:12.708 INFO:tasks.workunit.client.1.smithi069.stdout:6/819: symlink d4/d7/da6/l100 0 2024-03-22T07:16:12.711 INFO:tasks.workunit.client.1.smithi069.stdout:2/935: rename f6 to d18/d4d/d27/d33/d60/d63/d93/def/f12f 0 2024-03-22T07:16:12.712 INFO:tasks.workunit.client.1.smithi069.stdout:7/914: rmdir d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/dce 39 2024-03-22T07:16:12.714 INFO:tasks.workunit.client.1.smithi069.stdout:5/852: dwrite d1/d6/d86/d99/f4b [0,4194304] 0 2024-03-22T07:16:12.714 INFO:tasks.workunit.client.1.smithi069.stdout:5/853: write d1/d6/d26/d33/d5d/d72/d78/fd3 [2673064,10154] 0 2024-03-22T07:16:12.715 INFO:tasks.workunit.client.1.smithi069.stdout:6/820: rename d4/d7/d5b/cae to d4/dd/d12/d44/d58/db4/c101 0 2024-03-22T07:16:12.715 INFO:tasks.workunit.client.1.smithi069.stdout:1/905: dwrite d1/d13/d36/de5/d88/ffc [0,4194304] 0 2024-03-22T07:16:12.715 INFO:tasks.workunit.client.1.smithi069.stdout:1/906: truncate d1/d13/d36/f81 1065030 0 2024-03-22T07:16:12.715 INFO:tasks.workunit.client.1.smithi069.stdout:1/907: write d1/f124 [447065,87615] 0 2024-03-22T07:16:12.715 INFO:tasks.workunit.client.1.smithi069.stdout:6/821: creat d4/d7/d5b/f102 x:0 0 0 2024-03-22T07:16:12.715 INFO:tasks.workunit.client.1.smithi069.stdout:1/908: creat d1/d2a/d84/dd1/f125 x:0 0 0 2024-03-22T07:16:12.716 INFO:tasks.workunit.client.1.smithi069.stdout:2/936: dread d18/d4d/d25/f47 [0,4194304] 0 2024-03-22T07:16:12.716 INFO:tasks.workunit.client.1.smithi069.stdout:2/937: fsync d18/d4d/dec/f104 0 2024-03-22T07:16:12.716 INFO:tasks.workunit.client.1.smithi069.stdout:2/938: dread - d18/d4d/d27/d33/fd0 zero size 2024-03-22T07:16:12.716 INFO:tasks.workunit.client.1.smithi069.stdout:2/939: chown d18/d4d/d27/d33/d60/l11d 506 1 2024-03-22T07:16:12.719 INFO:tasks.workunit.client.1.smithi069.stdout:7/915: rename d9/d42 to d9/db/d1a/db1/d57/d75/de2/d122/d127 0 2024-03-22T07:16:12.720 INFO:tasks.workunit.client.1.smithi069.stdout:9/921: write d2/d51/f5e [853884,30730] 0 2024-03-22T07:16:12.720 INFO:tasks.workunit.client.1.smithi069.stdout:5/854: dread d1/d6/d21/fb1 [0,4194304] 0 2024-03-22T07:16:12.765 INFO:tasks.workunit.client.1.smithi069.stdout:1/909: dwrite d1/d13/d36/de5/d4c/d55/f109 [0,4194304] 0 2024-03-22T07:16:12.765 INFO:tasks.workunit.client.1.smithi069.stdout:1/910: creat d1/d2a/d5e/f126 x:0 0 0 2024-03-22T07:16:12.766 INFO:tasks.workunit.client.1.smithi069.stdout:1/911: write d1/d13/d36/de5/d4c/d55/f80 [1155031,6384] 0 2024-03-22T07:16:12.766 INFO:tasks.workunit.client.1.smithi069.stdout:1/912: dread - d1/d2a/d84/d9a/fed zero size 2024-03-22T07:16:12.766 INFO:tasks.workunit.client.1.smithi069.stdout:1/913: creat d1/d2a/d84/d9a/f127 x:0 0 0 2024-03-22T07:16:12.766 INFO:tasks.workunit.client.1.smithi069.stdout:9/922: dwrite d2/f9f [0,4194304] 0 2024-03-22T07:16:12.766 INFO:tasks.workunit.client.1.smithi069.stdout:9/923: chown d2/dd/d3c/d48/d7f/da2/lcc 207053790 1 2024-03-22T07:16:12.772 INFO:tasks.workunit.client.1.smithi069.stdout:9/924: rename d2/dd/f126 to d2/d71/d90/f130 0 2024-03-22T07:16:12.772 INFO:tasks.workunit.client.1.smithi069.stdout:9/925: chown d2/d51/ce5 446 1 2024-03-22T07:16:12.776 INFO:tasks.workunit.client.1.smithi069.stdout:1/914: dread d1/d2a/d5e/d93/fa0 [0,4194304] 0 2024-03-22T07:16:12.778 INFO:tasks.workunit.client.1.smithi069.stdout:1/915: creat d1/d2a/f128 x:0 0 0 2024-03-22T07:16:12.778 INFO:tasks.workunit.client.1.smithi069.stdout:1/916: fdatasync d1/d2a/d84/fde 0 2024-03-22T07:16:12.781 INFO:tasks.workunit.client.1.smithi069.stdout:1/917: link d1/d13/dc7/d121/d64/lbf d1/d2a/db6/df8/d73/df0/l129 0 2024-03-22T07:16:12.781 INFO:tasks.workunit.client.1.smithi069.stdout:1/918: chown d1/d13/dc7/d100/f11d 104041 1 2024-03-22T07:16:12.788 INFO:tasks.workunit.client.1.smithi069.stdout:9/926: dwrite d2/db6/fda [4194304,4194304] 0 2024-03-22T07:16:12.790 INFO:tasks.workunit.client.1.smithi069.stdout:9/927: getdents d2/dd/d3c/d48 0 2024-03-22T07:16:12.790 INFO:tasks.workunit.client.1.smithi069.stdout:9/928: fdatasync d2/dd/d3c/d48/fe8 0 2024-03-22T07:16:12.793 INFO:tasks.workunit.client.1.smithi069.stdout:9/929: rename d2/d51/l122 to d2/dd/d3c/d2e/d84/da5/de0/l131 0 2024-03-22T07:16:12.799 INFO:tasks.workunit.client.1.smithi069.stdout:9/930: dread d2/dd/d3c/f3b [0,4194304] 0 2024-03-22T07:16:12.800 INFO:tasks.workunit.client.1.smithi069.stdout:1/919: dwrite d1/d2a/db6/df8/f11e [0,4194304] 0 2024-03-22T07:16:12.803 INFO:tasks.workunit.client.1.smithi069.stdout:1/920: mknod d1/d13/d36/de5/d88/dec/d122/c12a 0 2024-03-22T07:16:12.803 INFO:tasks.workunit.client.1.smithi069.stdout:1/921: dread - d1/d13/d82/dc2/fcb zero size 2024-03-22T07:16:12.805 INFO:tasks.workunit.client.1.smithi069.stdout:1/922: symlink d1/d13/d36/de5/d4c/d55/l12b 0 2024-03-22T07:16:12.808 INFO:tasks.workunit.client.1.smithi069.stdout:1/923: symlink d1/d13/d36/de5/d88/dec/d122/l12c 0 2024-03-22T07:16:12.810 INFO:tasks.workunit.client.1.smithi069.stdout:1/924: rename d1/d2a/l53 to d1/d13/d36/de5/d88/dec/l12d 0 2024-03-22T07:16:12.812 INFO:tasks.workunit.client.1.smithi069.stdout:1/925: creat d1/d2a/d84/dd1/ddb/f12e x:0 0 0 2024-03-22T07:16:12.814 INFO:tasks.workunit.client.1.smithi069.stdout:1/926: mknod d1/d13/d36/d5b/c12f 0 2024-03-22T07:16:12.817 INFO:tasks.workunit.client.1.smithi069.stdout:1/927: link d1/d13/dc7/d121/f19 d1/d2a/db6/df8/d73/df0/f130 0 2024-03-22T07:16:12.818 INFO:tasks.workunit.client.1.smithi069.stdout:1/928: mkdir d1/d13/dc7/d121/d131 0 2024-03-22T07:16:12.819 INFO:tasks.workunit.client.1.smithi069.stdout:1/929: creat d1/d13/dc7/d121/d1e/f132 x:0 0 0 2024-03-22T07:16:12.822 INFO:tasks.workunit.client.1.smithi069.stdout:9/931: dwrite d2/dd/d3c/d2e/d84/f88 [0,4194304] 0 2024-03-22T07:16:12.840 INFO:tasks.workunit.client.1.smithi069.stdout:9/932: dwrite d2/dd/d3c/d48/fe8 [0,4194304] 0 2024-03-22T07:16:12.846 INFO:tasks.workunit.client.1.smithi069.stdout:9/933: symlink d2/d51/dbd/de4/df3/l132 0 2024-03-22T07:16:12.847 INFO:tasks.workunit.client.1.smithi069.stdout:9/934: unlink d2/dd/d6e/c93 0 2024-03-22T07:16:12.847 INFO:tasks.workunit.client.1.smithi069.stdout:9/935: write d2/f9f [5117752,119212] 0 2024-03-22T07:16:12.850 INFO:tasks.workunit.client.1.smithi069.stdout:1/930: dwrite d1/d2a/db6/fd4 [0,4194304] 0 2024-03-22T07:16:12.850 INFO:tasks.workunit.client.1.smithi069.stdout:1/931: chown d1/d13/d36/de5/d4c/d55/f97 138559 1 2024-03-22T07:16:12.850 INFO:tasks.workunit.client.1.smithi069.stdout:1/932: stat d1/d2a/d84/d9a/f127 0 2024-03-22T07:16:12.862 INFO:tasks.workunit.client.1.smithi069.stdout:1/933: write d1/d13/d36/de5/d4c/d55/f56 [1402944,95288] 0 2024-03-22T07:16:12.864 INFO:tasks.workunit.client.1.smithi069.stdout:1/934: rename d1/d13/dc7/d121/d64/f99 to d1/d2a/db6/df8/f133 0 2024-03-22T07:16:12.872 INFO:tasks.workunit.client.1.smithi069.stdout:9/936: dwrite d2/dd/fac [0,4194304] 0 2024-03-22T07:16:12.872 INFO:tasks.workunit.client.1.smithi069.stdout:9/937: truncate d2/dd/d3c/d53/da6/f103 4371144 0 2024-03-22T07:16:12.889 INFO:tasks.workunit.client.1.smithi069.stdout:9/938: dwrite d2/d51/dbd/de4/f125 [0,4194304] 0 2024-03-22T07:16:12.891 INFO:tasks.workunit.client.1.smithi069.stdout:9/939: rename d2/dd/l68 to d2/dd/d3c/d48/d7c/l133 0 2024-03-22T07:16:12.892 INFO:tasks.workunit.client.1.smithi069.stdout:9/940: mknod d2/d51/dbd/c134 0 2024-03-22T07:16:12.893 INFO:tasks.workunit.client.1.smithi069.stdout:9/941: rename d2/dd/d3c/d2e/d84/da5/de0/l131 to d2/dd/d3c/d48/d7f/da2/l135 0 2024-03-22T07:16:12.894 INFO:tasks.workunit.client.1.smithi069.stdout:9/942: chown d2/dd/d3c/d53/c70 8233313 1 2024-03-22T07:16:12.894 INFO:tasks.workunit.client.1.smithi069.stdout:9/943: write d2/d51/fc2 [976713,122277] 0 2024-03-22T07:16:12.907 INFO:tasks.workunit.client.1.smithi069.stdout:9/944: dwrite d2/dd/d3c/d2e/d84/da5/de0/ff0 [0,4194304] 0 2024-03-22T07:16:12.908 INFO:tasks.workunit.client.1.smithi069.stdout:9/945: dread - d2/dd/d3c/fd6 zero size 2024-03-22T07:16:12.909 INFO:tasks.workunit.client.1.smithi069.stdout:9/946: link d2/dd/f26 d2/d83/f136 0 2024-03-22T07:16:12.922 INFO:tasks.workunit.client.1.smithi069.stdout:9/947: dread d2/dd/d3c/d48/d7f/fa0 [0,4194304] 0 2024-03-22T07:16:12.925 INFO:tasks.workunit.client.1.smithi069.stdout:9/948: rename d2/f9f to d2/dd/d3c/d53/d9d/f137 0 2024-03-22T07:16:12.925 INFO:tasks.workunit.client.1.smithi069.stdout:9/949: creat d2/dd/d3c/d2e/d84/f138 x:0 0 0 2024-03-22T07:16:12.925 INFO:tasks.workunit.client.1.smithi069.stdout:9/950: fdatasync d2/dd/d3c/d2e/f6a 0 2024-03-22T07:16:12.925 INFO:tasks.workunit.client.1.smithi069.stdout:9/951: dread - d2/d71/f8d zero size 2024-03-22T07:16:12.925 INFO:tasks.workunit.client.1.smithi069.stdout:9/952: truncate d2/dd/d3c/d2e/d84/fd9 798069 0 2024-03-22T07:16:13.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:12 smithi069 ceph-mon[36441]: mgrmap e24: smithi069.gfziwh(active, since 11s), standbys: smithi063.mnxayh 2024-03-22T07:16:13.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:12 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mgr metadata", "who": "smithi063.mnxayh", "id": "smithi063.mnxayh"}]: dispatch 2024-03-22T07:16:13.033 INFO:tasks.workunit.client.1.smithi069.stdout:6/822: sync 2024-03-22T07:16:13.033 INFO:tasks.workunit.client.1.smithi069.stdout:6/823: write d4/d7/da/d3f/fff [769631,88689] 0 2024-03-22T07:16:13.034 INFO:tasks.workunit.client.1.smithi069.stdout:6/824: rename d4/dd/d12/d44/d58/db4/fdf to d4/dd/d12/d2b/f103 0 2024-03-22T07:16:13.038 INFO:tasks.workunit.client.1.smithi069.stdout:6/825: mknod d4/dd/d41/d78/c104 0 2024-03-22T07:16:13.039 INFO:tasks.workunit.client.1.smithi069.stdout:6/826: symlink d4/dd/d41/d78/da3/l105 0 2024-03-22T07:16:13.039 INFO:tasks.workunit.client.1.smithi069.stdout:6/827: dread - d4/fe3 zero size 2024-03-22T07:16:13.039 INFO:tasks.workunit.client.1.smithi069.stdout:6/828: read - d4/dd/d41/d78/ffb zero size 2024-03-22T07:16:13.042 INFO:tasks.workunit.client.1.smithi069.stdout:6/829: rename d4/dd/d12/d44 to d4/dd/d12/d106 0 2024-03-22T07:16:13.043 INFO:tasks.workunit.client.1.smithi069.stdout:6/830: write d4/dd/d12/d13/de9/fb1 [523571,91410] 0 2024-03-22T07:16:13.043 INFO:tasks.workunit.client.1.smithi069.stdout:6/831: creat d4/dd/dbd/f107 x:0 0 0 2024-03-22T07:16:13.043 INFO:tasks.workunit.client.1.smithi069.stdout:6/832: creat d4/dd/dbd/f108 x:0 0 0 2024-03-22T07:16:13.049 INFO:tasks.workunit.client.1.smithi069.stdout:6/833: dread d4/d7/f3e [0,4194304] 0 2024-03-22T07:16:13.052 INFO:tasks.workunit.client.1.smithi069.stdout:6/834: rename d4/d7/da/dc6 to d4/dd/d109 0 2024-03-22T07:16:13.052 INFO:tasks.workunit.client.1.smithi069.stdout:6/835: chown d4/dd/la4 4428172 1 2024-03-22T07:16:13.053 INFO:tasks.workunit.client.1.smithi069.stdout:6/836: creat d4/dd/d12/d13/de9/f10a x:0 0 0 2024-03-22T07:16:13.054 INFO:tasks.workunit.client.1.smithi069.stdout:6/837: mkdir d4/dd/d12/d106/d58/d10b 0 2024-03-22T07:16:13.054 INFO:tasks.workunit.client.1.smithi069.stdout:6/838: write d4/dd/d12/d13/de9/fba [849603,67371] 0 2024-03-22T07:16:13.055 INFO:tasks.workunit.client.1.smithi069.stdout:6/839: symlink d4/dd/d12/d2b/d72/d73/d74/l10c 0 2024-03-22T07:16:13.056 INFO:tasks.workunit.client.1.smithi069.stdout:6/840: creat d4/dd/d41/d78/d9a/f10d x:0 0 0 2024-03-22T07:16:13.057 INFO:tasks.workunit.client.1.smithi069.stdout:6/841: link d4/dd/ff9 d4/dd/dbd/f10e 0 2024-03-22T07:16:13.073 INFO:tasks.workunit.client.1.smithi069.stdout:1/935: sync 2024-03-22T07:16:13.073 INFO:tasks.workunit.client.1.smithi069.stdout:2/940: sync 2024-03-22T07:16:13.073 INFO:tasks.workunit.client.1.smithi069.stdout:5/855: sync 2024-03-22T07:16:13.073 INFO:tasks.workunit.client.1.smithi069.stdout:9/953: sync 2024-03-22T07:16:13.073 INFO:tasks.workunit.client.1.smithi069.stdout:7/916: sync 2024-03-22T07:16:13.073 INFO:tasks.workunit.client.1.smithi069.stdout:2/941: readlink d18/d4d/d27/d33/d4f/l9e 0 2024-03-22T07:16:13.074 INFO:tasks.workunit.client.1.smithi069.stdout:2/942: creat d18/d4d/d27/d33/d60/d64/d89/d8b/dab/dee/f130 x:0 0 0 2024-03-22T07:16:13.074 INFO:tasks.workunit.client.1.smithi069.stdout:9/954: mkdir d2/d76/d139 0 2024-03-22T07:16:13.075 INFO:tasks.workunit.client.1.smithi069.stdout:5/856: symlink d1/d6/d26/d33/d5d/d72/d95/l124 0 2024-03-22T07:16:13.075 INFO:tasks.workunit.client.1.smithi069.stdout:5/857: fdatasync d1/d6/d86/f73 0 2024-03-22T07:16:13.075 INFO:tasks.workunit.client.1.smithi069.stdout:5/858: creat d1/d6/d86/dab/f125 x:0 0 0 2024-03-22T07:16:13.075 INFO:tasks.workunit.client.1.smithi069.stdout:1/936: dread d1/d13/dc7/d121/d7a/f90 [0,4194304] 0 2024-03-22T07:16:13.076 INFO:tasks.workunit.client.1.smithi069.stdout:7/917: symlink d9/db/d1a/db1/d57/d44/d53/d1d/l128 0 2024-03-22T07:16:13.076 INFO:tasks.workunit.client.1.smithi069.stdout:7/918: chown d9/db/d1a/db1/d57/d44/d53/dc7/fcc 105110307 1 2024-03-22T07:16:13.078 INFO:tasks.workunit.client.1.smithi069.stdout:1/937: read d1/d2a/d84/fdc [468441,45110] 0 2024-03-22T07:16:13.078 INFO:tasks.workunit.client.1.smithi069.stdout:6/842: dwrite d4/d7/da/f7c [0,4194304] 0 2024-03-22T07:16:13.082 INFO:tasks.workunit.client.1.smithi069.stdout:5/859: unlink d1/d6/c81 0 2024-03-22T07:16:13.086 INFO:tasks.workunit.client.1.smithi069.stdout:5/860: creat d1/d6/d26/d33/d5d/d72/d95/dc7/dce/f126 x:0 0 0 2024-03-22T07:16:13.086 INFO:tasks.workunit.client.1.smithi069.stdout:5/861: chown d1/d10/d44/dd8 86 1 2024-03-22T07:16:13.086 INFO:tasks.workunit.client.1.smithi069.stdout:2/943: dread d18/d4d/d25/f4b [0,4194304] 0 2024-03-22T07:16:13.087 INFO:tasks.workunit.client.1.smithi069.stdout:7/919: rename d9/db/d1a/db1/d57/d75/cea to d9/db/d1a/db1/d57/d44/d53/dc7/c129 0 2024-03-22T07:16:13.089 INFO:tasks.workunit.client.1.smithi069.stdout:1/938: rmdir d1/d13/dc7 39 2024-03-22T07:16:13.090 INFO:tasks.workunit.client.1.smithi069.stdout:2/944: rename d18/d4d/d27/d33/d60/d63/d80/lf8 to d18/d4d/d3e/d40/l131 0 2024-03-22T07:16:13.098 INFO:tasks.workunit.client.1.smithi069.stdout:9/955: dwrite d2/d83/f136 [0,4194304] 0 2024-03-22T07:16:13.101 INFO:tasks.workunit.client.1.smithi069.stdout:5/862: write d1/d6/f6f [879120,85117] 0 2024-03-22T07:16:13.102 INFO:tasks.workunit.client.1.smithi069.stdout:5/863: symlink d1/d6/d26/d33/d5d/l127 0 2024-03-22T07:16:13.104 INFO:tasks.workunit.client.1.smithi069.stdout:2/945: read d18/d4d/d27/d33/d60/d64/d89/f90 [340204,53098] 0 2024-03-22T07:16:13.105 INFO:tasks.workunit.client.1.smithi069.stdout:2/946: chown d18/d4d/d27/d33/d60/d64/d89/d8b/dab/fa7 780 1 2024-03-22T07:16:13.109 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:12 smithi063 ceph-mon[29200]: mgrmap e24: smithi069.gfziwh(active, since 11s), standbys: smithi063.mnxayh 2024-03-22T07:16:13.109 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:12 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mgr metadata", "who": "smithi063.mnxayh", "id": "smithi063.mnxayh"}]: dispatch 2024-03-22T07:16:13.111 INFO:tasks.workunit.client.1.smithi069.stdout:9/956: dread d2/d21/f73 [0,4194304] 0 2024-03-22T07:16:13.120 INFO:tasks.workunit.client.1.smithi069.stdout:6/843: dwrite d4/dd/f4a [0,4194304] 0 2024-03-22T07:16:13.121 INFO:tasks.workunit.client.1.smithi069.stdout:6/844: getdents d4/dd/d12/d106/d58/db4/dd2 0 2024-03-22T07:16:13.122 INFO:tasks.workunit.client.1.smithi069.stdout:6/845: symlink d4/dd/d12/d106/d58/db4/l10f 0 2024-03-22T07:16:13.122 INFO:tasks.workunit.client.1.smithi069.stdout:6/846: fdatasync d4/dd/d12/d2b/f3a 0 2024-03-22T07:16:13.122 INFO:tasks.workunit.client.1.smithi069.stdout:6/847: mkdir d4/dd/d12/d2b/d72/d73/d110 0 2024-03-22T07:16:13.130 INFO:tasks.workunit.client.1.smithi069.stdout:7/920: dwrite d9/db/d1a/db1/d57/d75/ff4 [0,4194304] 0 2024-03-22T07:16:13.130 INFO:tasks.workunit.client.1.smithi069.stdout:1/939: dwrite d1/d13/f2d [0,4194304] 0 2024-03-22T07:16:13.132 INFO:tasks.workunit.client.1.smithi069.stdout:7/921: mkdir d9/db/d48/d12a 0 2024-03-22T07:16:13.134 INFO:tasks.workunit.client.1.smithi069.stdout:1/940: truncate d1/d13/f27 815186 0 2024-03-22T07:16:13.134 INFO:tasks.workunit.client.1.smithi069.stdout:1/941: dread - d1/d2a/db6/df8/d73/fbd zero size 2024-03-22T07:16:13.138 INFO:tasks.workunit.client.1.smithi069.stdout:6/848: dread d4/dd/f8f [0,4194304] 0 2024-03-22T07:16:13.138 INFO:tasks.workunit.client.1.smithi069.stdout:5/864: dwrite d1/d6/d26/d33/f80 [0,4194304] 0 2024-03-22T07:16:13.139 INFO:tasks.workunit.client.1.smithi069.stdout:5/865: write d1/d6/d26/d33/d5d/d72/da7/fc5 [511437,24368] 0 2024-03-22T07:16:13.139 INFO:tasks.workunit.client.1.smithi069.stdout:7/922: rename d9/db/f94 to d9/de4/df5/f12b 0 2024-03-22T07:16:13.139 INFO:tasks.workunit.client.1.smithi069.stdout:7/923: truncate d9/db/d48/dab/f10b 756446 0 2024-03-22T07:16:13.139 INFO:tasks.workunit.client.1.smithi069.stdout:9/957: dwrite d2/dd/d3c/f35 [0,4194304] 0 2024-03-22T07:16:13.139 INFO:tasks.workunit.client.1.smithi069.stdout:5/866: symlink d1/d6/d86/d99/l128 0 2024-03-22T07:16:13.139 INFO:tasks.workunit.client.1.smithi069.stdout:5/867: write d1/d6/f2e [725589,41780] 0 2024-03-22T07:16:13.144 INFO:tasks.workunit.client.1.smithi069.stdout:9/958: mkdir d2/dd/d3c/d48/d7f/d86/d13a 0 2024-03-22T07:16:13.146 INFO:tasks.workunit.client.1.smithi069.stdout:1/942: dread d1/d13/d36/f68 [0,4194304] 0 2024-03-22T07:16:13.147 INFO:tasks.workunit.client.1.smithi069.stdout:5/868: unlink d1/d6/d86/dab/l11e 0 2024-03-22T07:16:13.149 INFO:tasks.workunit.client.1.smithi069.stdout:9/959: rename d2/d51/l10f to d2/dd/d3c/d2e/d84/da5/de0/l13b 0 2024-03-22T07:16:13.152 INFO:tasks.workunit.client.1.smithi069.stdout:1/943: link d1/d13/d36/de5/d88/dec/l12d d1/d13/d36/de5/d88/dae/l134 0 2024-03-22T07:16:13.157 INFO:tasks.workunit.client.1.smithi069.stdout:7/924: dwrite d9/db/d1a/db1/d57/d44/d53/d1d/fdf [0,4194304] 0 2024-03-22T07:16:13.159 INFO:tasks.workunit.client.1.smithi069.stdout:7/925: symlink d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/dce/l12c 0 2024-03-22T07:16:13.160 INFO:tasks.workunit.client.1.smithi069.stdout:7/926: rmdir d9/db 39 2024-03-22T07:16:13.169 INFO:tasks.workunit.client.1.smithi069.stdout:6/849: dwrite d4/dd/d12/d13/fb0 [0,4194304] 0 2024-03-22T07:16:13.169 INFO:tasks.workunit.client.1.smithi069.stdout:5/869: dwrite d1/d6/d26/d33/d5d/d72/d78/fb5 [0,4194304] 0 2024-03-22T07:16:13.170 INFO:tasks.workunit.client.1.smithi069.stdout:5/870: write d1/d6/d26/d33/f4e [1049241,73613] 0 2024-03-22T07:16:13.171 INFO:tasks.workunit.client.1.smithi069.stdout:9/960: rename d2/dd/d3c/d2e/d84/da5/de0/l13b to d2/dd/d3c/d53/l13c 0 2024-03-22T07:16:13.179 INFO:tasks.workunit.client.1.smithi069.stdout:1/944: rename d1/d13/d36/de5/d4c/d55/f79 to d1/d13/d36/de5/f135 0 2024-03-22T07:16:13.179 INFO:tasks.workunit.client.1.smithi069.stdout:1/945: chown d1/f124 488 1 2024-03-22T07:16:13.179 INFO:tasks.workunit.client.1.smithi069.stdout:9/961: mkdir d2/d51/dbd/de4/d13d 0 2024-03-22T07:16:13.182 INFO:tasks.workunit.client.1.smithi069.stdout:7/927: dwrite d9/db/d1a/db1/d57/d44/d53/d74/f115 [0,4194304] 0 2024-03-22T07:16:13.182 INFO:tasks.workunit.client.1.smithi069.stdout:7/928: dread - d9/db/d1a/db1/d57/d75/d7a/dcf/f10e zero size 2024-03-22T07:16:13.182 INFO:tasks.workunit.client.1.smithi069.stdout:1/946: rename d1/d2a/db6/df8/lb0 to d1/d13/d36/de5/d88/dae/l136 0 2024-03-22T07:16:13.183 INFO:tasks.workunit.client.1.smithi069.stdout:1/947: chown d1/d13/d36/lab 75 1 2024-03-22T07:16:13.184 INFO:tasks.workunit.client.1.smithi069.stdout:9/962: rename d2/dd/l2a to d2/dd/d3c/d53/d9d/l13e 0 2024-03-22T07:16:13.189 INFO:tasks.workunit.client.1.smithi069.stdout:7/929: dread d9/db/d1a/db1/d57/d44/d53/f6d [0,4194304] 0 2024-03-22T07:16:13.211 INFO:tasks.workunit.client.1.smithi069.stdout:6/850: dwrite d4/fe3 [0,4194304] 0 2024-03-22T07:16:13.211 INFO:tasks.workunit.client.1.smithi069.stdout:5/871: dwrite d1/d10/d44/d50/f96 [0,4194304] 0 2024-03-22T07:16:13.213 INFO:tasks.workunit.client.1.smithi069.stdout:9/963: dwrite d2/dd/d3c/d48/d7c/ffe [0,4194304] 0 2024-03-22T07:16:13.213 INFO:tasks.workunit.client.1.smithi069.stdout:9/964: chown d2/dd/d3c/d48/d7f/d86/f8e 31 1 2024-03-22T07:16:13.214 INFO:tasks.workunit.client.1.smithi069.stdout:7/930: dwrite d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/df9/f65 [0,4194304] 0 2024-03-22T07:16:13.219 INFO:tasks.workunit.client.1.smithi069.stdout:7/931: symlink d9/d8e/df7/l12d 0 2024-03-22T07:16:13.219 INFO:tasks.workunit.client.1.smithi069.stdout:7/932: chown d9/db/d1a/ce1 101662 1 2024-03-22T07:16:13.220 INFO:tasks.workunit.client.1.smithi069.stdout:7/933: mkdir d9/dff/d105/d12e 0 2024-03-22T07:16:13.220 INFO:tasks.workunit.client.1.smithi069.stdout:7/934: dread - d9/db/d1a/db1/d57/d75/dec/fba zero size 2024-03-22T07:16:13.221 INFO:tasks.workunit.client.1.smithi069.stdout:5/872: write d1/d6/d86/d99/f4b [1159877,53094] 0 2024-03-22T07:16:13.221 INFO:tasks.workunit.client.1.smithi069.stdout:5/873: fdatasync d1/d6/f2e 0 2024-03-22T07:16:13.223 INFO:tasks.workunit.client.1.smithi069.stdout:5/874: mkdir d1/d7/d3d/d100/d129 0 2024-03-22T07:16:13.223 INFO:tasks.workunit.client.1.smithi069.stdout:5/875: chown d1/d6/d26/d3a/db2 6543 1 2024-03-22T07:16:13.224 INFO:tasks.workunit.client.1.smithi069.stdout:7/935: dread d9/db/d1a/db1/d57/d44/d53/d1d/faa [0,4194304] 0 2024-03-22T07:16:13.226 INFO:tasks.workunit.client.1.smithi069.stdout:7/936: link d9/db/d1a/db1/d57/d44/d53/ld4 d9/db/d1a/db1/l12f 0 2024-03-22T07:16:13.227 INFO:tasks.workunit.client.1.smithi069.stdout:7/937: rmdir d9/db 39 2024-03-22T07:16:13.228 INFO:tasks.workunit.client.1.smithi069.stdout:7/938: symlink d9/db/d1a/db1/d57/d75/de2/d122/d127/da8/l130 0 2024-03-22T07:16:13.231 INFO:tasks.workunit.client.1.smithi069.stdout:7/939: mkdir d9/db/d1a/db1/d57/d44/d53/d131 0 2024-03-22T07:16:13.231 INFO:tasks.workunit.client.1.smithi069.stdout:9/965: dread d2/dd/d3c/d53/d9d/ff4 [0,4194304] 0 2024-03-22T07:16:13.233 INFO:tasks.workunit.client.1.smithi069.stdout:6/851: dwrite d4/dd/d41/d78/d9a/f10d [0,4194304] 0 2024-03-22T07:16:13.237 INFO:tasks.workunit.client.1.smithi069.stdout:6/852: rename d4/dd/d41 to d4/d7/da/d111 0 2024-03-22T07:16:13.239 INFO:tasks.workunit.client.1.smithi069.stdout:9/966: dread d2/db6/fda [4194304,4194304] 0 2024-03-22T07:16:13.239 INFO:tasks.workunit.client.1.smithi069.stdout:9/967: readlink d2/d31/ld3 0 2024-03-22T07:16:13.240 INFO:tasks.workunit.client.1.smithi069.stdout:9/968: write d2/d71/f8d [337376,8068] 0 2024-03-22T07:16:13.240 INFO:tasks.workunit.client.1.smithi069.stdout:9/969: creat d2/dd/d3c/d48/d7f/d86/d117/f13f x:0 0 0 2024-03-22T07:16:13.240 INFO:tasks.workunit.client.1.smithi069.stdout:9/970: dread - d2/dd/d3c/d48/d7f/d86/d117/f13f zero size 2024-03-22T07:16:13.241 INFO:tasks.workunit.client.1.smithi069.stdout:5/876: dwrite d1/d6/d86/d99/f15 [0,4194304] 0 2024-03-22T07:16:13.242 INFO:tasks.workunit.client.1.smithi069.stdout:5/877: dread - d1/d28/ff2 zero size 2024-03-22T07:16:13.244 INFO:tasks.workunit.client.1.smithi069.stdout:7/940: dread d9/db/d1a/fc3 [0,4194304] 0 2024-03-22T07:16:13.244 INFO:tasks.workunit.client.1.smithi069.stdout:7/941: write d9/db/d1a/db1/d57/d44/d53/d74/f115 [4399467,79927] 0 2024-03-22T07:16:13.244 INFO:tasks.workunit.client.1.smithi069.stdout:7/942: write d9/db/d1a/db1/d57/d44/d53/d74/f121 [185195,104004] 0 2024-03-22T07:16:13.244 INFO:tasks.workunit.client.1.smithi069.stdout:7/943: chown d9/db/d1a/db1/d57/d75/d7a/f80 6 1 2024-03-22T07:16:13.246 INFO:tasks.workunit.client.1.smithi069.stdout:5/878: creat d1/d6/d88/dfb/f12a x:0 0 0 2024-03-22T07:16:13.246 INFO:tasks.workunit.client.1.smithi069.stdout:5/879: chown d1/d10/d60/fac 164 1 2024-03-22T07:16:13.249 INFO:tasks.workunit.client.1.smithi069.stdout:5/880: link d1/d6/d86/dab/f125 d1/d6/d26/d3e/d9c/f12b 0 2024-03-22T07:16:13.251 INFO:tasks.workunit.client.1.smithi069.stdout:7/944: mkdir d9/db/d1a/d132 0 2024-03-22T07:16:13.251 INFO:tasks.workunit.client.1.smithi069.stdout:7/945: write d9/db/d1a/db1/d57/d44/d53/d74/d30/f106 [448506,76236] 0 2024-03-22T07:16:13.251 INFO:tasks.workunit.client.1.smithi069.stdout:7/946: fdatasync d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/df9/f4f 0 2024-03-22T07:16:13.251 INFO:tasks.workunit.client.1.smithi069.stdout:7/947: read d9/db/d1a/db1/d57/d44/d53/d74/f121 [271732,69789] 0 2024-03-22T07:16:13.251 INFO:tasks.workunit.client.1.smithi069.stdout:7/948: truncate d9/db/d1a/fd6 55125 0 2024-03-22T07:16:13.252 INFO:tasks.workunit.client.1.smithi069.stdout:5/881: link d1/d6/d26/d3a/c3c d1/d6/d86/d99/dec/c12c 0 2024-03-22T07:16:13.252 INFO:tasks.workunit.client.1.smithi069.stdout:5/882: fsync d1/d6/d26/d7d/d9f/f104 0 2024-03-22T07:16:13.259 INFO:tasks.workunit.client.1.smithi069.stdout:5/883: dread d1/d6/d88/fd7 [0,4194304] 0 2024-03-22T07:16:13.264 INFO:tasks.workunit.client.1.smithi069.stdout:6/853: dwrite d4/d7/da/d3f/fff [0,4194304] 0 2024-03-22T07:16:13.268 INFO:tasks.workunit.client.1.smithi069.stdout:7/949: dwrite d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/df9/f97 [0,4194304] 0 2024-03-22T07:16:13.272 INFO:tasks.workunit.client.1.smithi069.stdout:7/950: creat d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/f133 x:0 0 0 2024-03-22T07:16:13.272 INFO:tasks.workunit.client.1.smithi069.stdout:7/951: chown d9/db/d1a/db1/d57/d44/d53/dc7/dfe 0 1 2024-03-22T07:16:13.272 INFO:tasks.workunit.client.1.smithi069.stdout:7/952: creat d9/db/d48/dab/f134 x:0 0 0 2024-03-22T07:16:13.272 INFO:tasks.workunit.client.1.smithi069.stdout:7/953: fsync d9/db/d1a/f85 0 2024-03-22T07:16:13.276 INFO:tasks.workunit.client.1.smithi069.stdout:7/954: rename d9/db/d1a/cdc to d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/c135 0 2024-03-22T07:16:13.277 INFO:tasks.workunit.client.1.smithi069.stdout:7/955: symlink d9/db/d1a/db1/d57/d75/d7a/d96/l136 0 2024-03-22T07:16:13.282 INFO:tasks.workunit.client.1.smithi069.stdout:7/956: truncate d9/db/d1a/db1/d57/d44/d53/d74/f98 1380760 0 2024-03-22T07:16:13.282 INFO:tasks.workunit.client.1.smithi069.stdout:7/957: creat d9/db/d1a/db1/d57/d44/d53/f137 x:0 0 0 2024-03-22T07:16:13.282 INFO:tasks.workunit.client.1.smithi069.stdout:7/958: symlink d9/db/d1a/db1/d57/d44/d53/d1d/l138 0 2024-03-22T07:16:13.282 INFO:tasks.workunit.client.1.smithi069.stdout:6/854: dwrite d4/dd/d12/d13/f86 [0,4194304] 0 2024-03-22T07:16:13.282 INFO:tasks.workunit.client.1.smithi069.stdout:6/855: read d4/d7/da/d3f/fc9 [3493275,113807] 0 2024-03-22T07:16:13.282 INFO:tasks.workunit.client.1.smithi069.stdout:6/856: fdatasync d4/dd/f54 0 2024-03-22T07:16:13.282 INFO:tasks.workunit.client.1.smithi069.stdout:6/857: dread - d4/dd/d12/d2b/d72/d73/fce zero size 2024-03-22T07:16:13.282 INFO:tasks.workunit.client.1.smithi069.stdout:6/858: fsync d4/dd/f42 0 2024-03-22T07:16:13.283 INFO:tasks.workunit.client.1.smithi069.stdout:6/859: getdents d4/dd/d12/d106/d58/db4 0 2024-03-22T07:16:13.285 INFO:tasks.workunit.client.1.smithi069.stdout:6/860: getdents d4/d7/da/d111/d78/da3 0 2024-03-22T07:16:13.285 INFO:tasks.workunit.client.1.smithi069.stdout:6/861: write d4/dd/d12/d2b/da5/fcc [866395,81240] 0 2024-03-22T07:16:13.287 INFO:tasks.workunit.client.1.smithi069.stdout:6/862: getdents d4/dd/d12/d13 0 2024-03-22T07:16:13.304 INFO:tasks.workunit.client.1.smithi069.stdout:7/959: dwrite d9/db/d1a/db1/d57/d75/f11e [0,4194304] 0 2024-03-22T07:16:13.305 INFO:tasks.workunit.client.1.smithi069.stdout:6/863: dwrite d4/dd/d12/d106/d58/db4/fc4 [0,4194304] 0 2024-03-22T07:16:13.306 INFO:tasks.workunit.client.1.smithi069.stdout:6/864: chown d4/d7/da/d111/d78/c90 163075733 1 2024-03-22T07:16:13.306 INFO:tasks.workunit.client.1.smithi069.stdout:6/865: stat d4/f47 0 2024-03-22T07:16:13.306 INFO:tasks.workunit.client.1.smithi069.stdout:7/960: unlink d9/db/d1a/db1/d57/d44/d53/dc7/fcc 0 2024-03-22T07:16:13.309 INFO:tasks.workunit.client.1.smithi069.stdout:6/866: mkdir d4/d7/da/d3f/d76/dab/d112 0 2024-03-22T07:16:13.311 INFO:tasks.workunit.client.1.smithi069.stdout:6/867: creat d4/dd/d109/f113 x:0 0 0 2024-03-22T07:16:13.311 INFO:tasks.workunit.client.1.smithi069.stdout:6/868: write d4/d7/da/f94 [43806,85723] 0 2024-03-22T07:16:13.314 INFO:tasks.workunit.client.1.smithi069.stdout:6/869: symlink d4/dd/d12/d106/d58/l114 0 2024-03-22T07:16:13.314 INFO:tasks.workunit.client.1.smithi069.stdout:6/870: chown d4/d7/da/d111/d78/d9a/laa 96368546 1 2024-03-22T07:16:13.314 INFO:tasks.workunit.client.1.smithi069.stdout:6/871: chown d4/dd/d12/d2b/d72 1235 1 2024-03-22T07:16:13.321 INFO:tasks.workunit.client.1.smithi069.stdout:7/961: dwrite d9/db/d1a/db1/d57/d44/d53/d1d/f87 [0,4194304] 0 2024-03-22T07:16:13.321 INFO:tasks.workunit.client.1.smithi069.stdout:7/962: chown d9/fd7 2057537 1 2024-03-22T07:16:13.334 INFO:tasks.workunit.client.1.smithi069.stdout:6/872: dwrite d4/f47 [0,4194304] 0 2024-03-22T07:16:13.334 INFO:tasks.workunit.client.1.smithi069.stdout:6/873: fsync d4/dd/d12/d2b/d72/d73/d9d/fc8 0 2024-03-22T07:16:13.344 INFO:tasks.workunit.client.1.smithi069.stdout:7/963: dwrite d9/db/d1a/db1/d57/d75/f11e [4194304,4194304] 0 2024-03-22T07:16:13.345 INFO:tasks.workunit.client.1.smithi069.stdout:7/964: fsync d9/db/d1a/db1/db3/f109 0 2024-03-22T07:16:13.345 INFO:tasks.workunit.client.1.smithi069.stdout:7/965: fsync d9/db/d1a/db1/d57/d44/d53/d74/d30/fc8 0 2024-03-22T07:16:13.345 INFO:tasks.workunit.client.1.smithi069.stdout:7/966: chown d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/df9/f97 775 1 2024-03-22T07:16:13.349 INFO:tasks.workunit.client.1.smithi069.stdout:6/874: dwrite d4/dd/d12/d2b/d72/f95 [0,4194304] 0 2024-03-22T07:16:13.350 INFO:tasks.workunit.client.1.smithi069.stdout:6/875: link d4/d7/da/d3f/d76/dab/ce0 d4/d7/da/d3f/d76/dab/d112/c115 0 2024-03-22T07:16:13.351 INFO:tasks.workunit.client.1.smithi069.stdout:6/876: creat d4/d7/f116 x:0 0 0 2024-03-22T07:16:13.351 INFO:tasks.workunit.client.1.smithi069.stdout:6/877: stat d4/dd/d12/d2b/d72/d73/d9d 0 2024-03-22T07:16:13.357 INFO:tasks.workunit.client.1.smithi069.stdout:7/967: dwrite d9/db/fe3 [0,4194304] 0 2024-03-22T07:16:13.358 INFO:tasks.workunit.client.1.smithi069.stdout:7/968: rename d9/db/d1a/db1/d57/d44/d53/d74/fed to d9/db/d1a/db1/d57/d75/de2/d122/f139 0 2024-03-22T07:16:13.359 INFO:tasks.workunit.client.1.smithi069.stdout:7/969: chown d9/de4 0 1 2024-03-22T07:16:13.367 INFO:tasks.workunit.client.1.smithi069.stdout:7/970: dread d9/db/d1a/db1/d57/d75/f11e [0,4194304] 0 2024-03-22T07:16:13.369 INFO:tasks.workunit.client.1.smithi069.stdout:7/971: rename d9/db/d1a/fc3 to d9/d8e/d123/f13a 0 2024-03-22T07:16:13.369 INFO:tasks.workunit.client.1.smithi069.stdout:7/972: creat d9/d8e/f13b x:0 0 0 2024-03-22T07:16:13.369 INFO:tasks.workunit.client.1.smithi069.stdout:7/973: creat d9/db/d1a/db1/d57/d44/d53/d1d/f13c x:0 0 0 2024-03-22T07:16:13.373 INFO:tasks.workunit.client.1.smithi069.stdout:7/974: mknod d9/de4/c13d 0 2024-03-22T07:16:13.373 INFO:tasks.workunit.client.1.smithi069.stdout:7/975: write d9/db/d1a/f82 [152216,86463] 0 2024-03-22T07:16:13.373 INFO:tasks.workunit.client.1.smithi069.stdout:7/976: chown d9/db/d1a/db1/d57 6 1 2024-03-22T07:16:13.386 INFO:tasks.workunit.client.1.smithi069.stdout:7/977: dwrite d9/d8e/f102 [0,4194304] 0 2024-03-22T07:16:13.386 INFO:tasks.workunit.client.1.smithi069.stdout:7/978: chown d9/db/d1a/db1/d57/d44/d53/d74/f8d 140 1 2024-03-22T07:16:13.386 INFO:tasks.workunit.client.1.smithi069.stdout:7/979: fsync d9/de4/df5/f12b 0 2024-03-22T07:16:13.388 INFO:tasks.workunit.client.1.smithi069.stdout:7/980: creat d9/db/d1a/db1/d57/d75/de2/d122/d127/da8/f13e x:0 0 0 2024-03-22T07:16:13.388 INFO:tasks.workunit.client.1.smithi069.stdout:7/981: symlink d9/db/d1a/db1/d57/d75/de2/d122/l13f 0 2024-03-22T07:16:13.401 INFO:tasks.workunit.client.1.smithi069.stdout:7/982: write d9/db/d1a/db1/d57/d75/de2/d122/f7c [1588614,129517] 0 2024-03-22T07:16:13.412 INFO:tasks.workunit.client.1.smithi069.stdout:7/983: dwrite d9/db/d1a/db1/d57/d75/de2/d122/d127/f71 [4194304,4194304] 0 2024-03-22T07:16:13.412 INFO:tasks.workunit.client.1.smithi069.stdout:7/984: fdatasync d9/db/d1a/db1/d57/d44/d53/d74/d30/d3d/fae 0 2024-03-22T07:16:13.412 INFO:tasks.workunit.client.1.smithi069.stdout:7/985: chown d9/db/d1a/db1/d57/d44/d53/d74/d30 0 1 2024-03-22T07:16:13.413 INFO:tasks.workunit.client.1.smithi069.stdout:7/986: creat d9/db/d48/f140 x:0 0 0 2024-03-22T07:16:13.428 INFO:tasks.workunit.client.1.smithi069.stdout:7/987: dwrite f4 [0,4194304] 0 2024-03-22T07:16:13.428 INFO:tasks.workunit.client.1.smithi069.stdout:7/988: fdatasync d9/db/d1a/db1/db3/fcd 0 2024-03-22T07:16:13.429 INFO:tasks.workunit.client.1.smithi069.stdout:7/989: mkdir d9/db/d1a/db1/d57/d44/d53/d74/d6e/d141 0 2024-03-22T07:16:13.429 INFO:tasks.workunit.client.1.smithi069.stdout:7/990: dread - d9/db/d48/dab/fd3 zero size 2024-03-22T07:16:13.442 INFO:tasks.workunit.client.1.smithi069.stdout:7/991: dread d9/db/f17 [4194304,4194304] 0 2024-03-22T07:16:13.442 INFO:tasks.workunit.client.1.smithi069.stdout:7/992: creat d9/db/d1a/db1/d57/d75/de2/d122/d127/da8/f142 x:0 0 0 2024-03-22T07:16:13.442 INFO:tasks.workunit.client.1.smithi069.stdout:7/993: chown d9/db/d1a/db1/d57/cbe 0 1 2024-03-22T07:16:13.442 INFO:tasks.workunit.client.1.smithi069.stdout:7/994: chown d9/db/d1a/db1/d57/d44/d53/dc7/l11f 795 1 2024-03-22T07:16:13.442 INFO:tasks.workunit.client.1.smithi069.stdout:7/995: creat d9/db/d48/f143 x:0 0 0 2024-03-22T07:16:13.442 INFO:tasks.workunit.client.1.smithi069.stdout:7/996: chown d9/db/d1a/db1/d57/f117 453888377 1 2024-03-22T07:16:13.442 INFO:tasks.workunit.client.1.smithi069.stdout:7/997: truncate d9/db/d1a/db1/d57/d75/d7a/d96/fb4 46905 0 2024-03-22T07:16:13.449 INFO:tasks.workunit.client.1.smithi069.stdout:7/998: dread d9/db/f17 [0,4194304] 0 2024-03-22T07:16:13.450 INFO:tasks.workunit.client.1.smithi069.stdout:7/999: rename d9/db/d1a/db1/d57/d75/d7a/lda to d9/db/d1a/db1/d57/d44/d53/l144 0 2024-03-22T07:16:13.468 INFO:tasks.workunit.client.1.smithi069.stdout:2/947: sync 2024-03-22T07:16:13.483 INFO:tasks.workunit.client.1.smithi069.stdout:2/948: dwrite d18/d91/f98 [0,4194304] 0 2024-03-22T07:16:13.500 INFO:tasks.workunit.client.1.smithi069.stdout:2/949: dwrite d18/d4d/d27/d56/f62 [0,4194304] 0 2024-03-22T07:16:13.513 INFO:tasks.workunit.client.1.smithi069.stdout:2/950: dwrite d18/d4d/d27/d33/d4f/ddd/f10b [0,4194304] 0 2024-03-22T07:16:13.514 INFO:tasks.workunit.client.1.smithi069.stdout:6/878: sync 2024-03-22T07:16:13.514 INFO:tasks.workunit.client.1.smithi069.stdout:1/948: sync 2024-03-22T07:16:13.514 INFO:tasks.workunit.client.1.smithi069.stdout:9/971: sync 2024-03-22T07:16:13.514 INFO:tasks.workunit.client.1.smithi069.stdout:1/949: fdatasync d1/d13/dc7/d121/fc5 0 2024-03-22T07:16:13.514 INFO:tasks.workunit.client.1.smithi069.stdout:5/884: sync 2024-03-22T07:16:13.514 INFO:tasks.workunit.client.1.smithi069.stdout:6/879: creat d4/dd/dbd/f117 x:0 0 0 2024-03-22T07:16:13.514 INFO:tasks.workunit.client.1.smithi069.stdout:5/885: fdatasync d1/d10/d60/fac 0 2024-03-22T07:16:13.514 INFO:tasks.workunit.client.1.smithi069.stdout:6/880: write d4/d7/da/d111/d78/ff4 [757858,38625] 0 2024-03-22T07:16:13.516 INFO:tasks.workunit.client.1.smithi069.stdout:2/951: truncate d18/d91/f98 3390899 0 2024-03-22T07:16:13.518 INFO:tasks.workunit.client.1.smithi069.stdout:1/950: creat d1/d13/f137 x:0 0 0 2024-03-22T07:16:13.518 INFO:tasks.workunit.client.1.smithi069.stdout:1/951: fdatasync d1/d2a/db6/df8/d73/df0/ffd 0 2024-03-22T07:16:13.518 INFO:tasks.workunit.client.1.smithi069.stdout:1/952: write d1/d2a/d5e/f10f [3309573,11738] 0 2024-03-22T07:16:13.518 INFO:tasks.workunit.client.1.smithi069.stdout:1/953: write d1/d13/dc7/d121/f19 [8244678,33512] 0 2024-03-22T07:16:13.519 INFO:tasks.workunit.client.1.smithi069.stdout:9/972: mkdir d2/dd/d6e/d140 0 2024-03-22T07:16:13.522 INFO:tasks.workunit.client.1.smithi069.stdout:5/886: unlink d1/d6/d86/d99/l1b 0 2024-03-22T07:16:13.523 INFO:tasks.workunit.client.1.smithi069.stdout:6/881: creat d4/dd/d12/d2b/d72/d73/f118 x:0 0 0 2024-03-22T07:16:13.524 INFO:tasks.workunit.client.1.smithi069.stdout:2/952: symlink d18/d4d/d27/d56/l132 0 2024-03-22T07:16:13.527 INFO:tasks.workunit.client.1.smithi069.stdout:1/954: rename d1/d2a/d84/fde to d1/d2a/f138 0 2024-03-22T07:16:13.527 INFO:tasks.workunit.client.1.smithi069.stdout:1/955: truncate d1/d2a/d84/dd1/f125 684914 0 2024-03-22T07:16:13.528 INFO:tasks.workunit.client.1.smithi069.stdout:9/973: mknod d2/d21/d89/c141 0 2024-03-22T07:16:13.529 INFO:tasks.workunit.client.1.smithi069.stdout:9/974: creat d2/dd/d3c/d53/d75/f142 x:0 0 0 2024-03-22T07:16:13.530 INFO:tasks.workunit.client.1.smithi069.stdout:6/882: write d4/d7/da/d3f/d76/fee [2506116,93061] 0 2024-03-22T07:16:13.531 INFO:tasks.workunit.client.1.smithi069.stdout:2/953: mkdir d18/d4d/d27/d33/d60/d64/dcd/d101/d133 0 2024-03-22T07:16:13.533 INFO:tasks.workunit.client.1.smithi069.stdout:5/887: rename d1/d10/df1/d109 to d1/d28/d12d 0 2024-03-22T07:16:13.534 INFO:tasks.workunit.client.1.smithi069.stdout:1/956: mknod d1/d13/d82/c139 0 2024-03-22T07:16:13.536 INFO:tasks.workunit.client.1.smithi069.stdout:6/883: mknod d4/dd/d12/d106/d58/c119 0 2024-03-22T07:16:13.537 INFO:tasks.workunit.client.1.smithi069.stdout:2/954: mkdir d18/d4d/d27/d33/d60/d64/dcd/d101/d134 0 2024-03-22T07:16:13.538 INFO:tasks.workunit.client.1.smithi069.stdout:2/955: read d18/d4d/d27/d33/d60/d63/d80/fc3 [3734,103751] 0 2024-03-22T07:16:13.538 INFO:tasks.workunit.client.1.smithi069.stdout:1/957: mknod d1/d13/dc7/d121/d7a/c13a 0 2024-03-22T07:16:13.538 INFO:tasks.workunit.client.1.smithi069.stdout:1/958: creat d1/d13/f13b x:0 0 0 2024-03-22T07:16:13.539 INFO:tasks.workunit.client.1.smithi069.stdout:1/959: write d1/d2a/d84/dd1/f125 [396781,45899] 0 2024-03-22T07:16:13.539 INFO:tasks.workunit.client.1.smithi069.stdout:6/884: symlink d4/dd/d12/d106/d58/db4/l11a 0 2024-03-22T07:16:13.539 INFO:tasks.workunit.client.1.smithi069.stdout:6/885: write d4/dd/d12/d2b/da5/db2/fbb [818344,91842] 0 2024-03-22T07:16:13.541 INFO:tasks.workunit.client.1.smithi069.stdout:2/956: symlink d18/d4d/d27/d33/d60/d63/d73/dc9/ddb/l135 0 2024-03-22T07:16:13.541 INFO:tasks.workunit.client.1.smithi069.stdout:5/888: dread d1/d7/f22 [0,4194304] 0 2024-03-22T07:16:13.546 INFO:tasks.workunit.client.1.smithi069.stdout:1/960: unlink d1/d13/dc7/d121/d1e/f72 0 2024-03-22T07:16:13.547 INFO:tasks.workunit.client.1.smithi069.stdout:1/961: symlink d1/d13/dc7/dd2/l13c 0 2024-03-22T07:16:13.549 INFO:tasks.workunit.client.1.smithi069.stdout:2/957: symlink d18/d4d/d27/d33/d60/dbd/ddc/df6/l136 0 2024-03-22T07:16:13.550 INFO:tasks.workunit.client.1.smithi069.stdout:5/889: rmdir d1/d6/d88/dfb 39 2024-03-22T07:16:13.550 INFO:tasks.workunit.client.1.smithi069.stdout:9/975: dwrite d2/dd/d3c/d48/d7f/d86/fc0 [0,4194304] 0 2024-03-22T07:16:13.550 INFO:tasks.workunit.client.1.smithi069.stdout:9/976: write d2/dd/d6e/fe1 [106096,25397] 0 2024-03-22T07:16:13.551 INFO:tasks.workunit.client.1.smithi069.stdout:5/890: mknod d1/d6/d26/d7d/c12e 0 2024-03-22T07:16:13.551 INFO:tasks.workunit.client.1.smithi069.stdout:5/891: fsync d1/f9 0 2024-03-22T07:16:13.553 INFO:tasks.workunit.client.1.smithi069.stdout:9/977: symlink d2/d76/db8/l143 0 2024-03-22T07:16:13.553 INFO:tasks.workunit.client.1.smithi069.stdout:9/978: write d2/dd/d3c/d2e/d84/da5/de0/ff0 [4985142,37552] 0 2024-03-22T07:16:13.555 INFO:tasks.workunit.client.1.smithi069.stdout:9/979: getdents d2/dd 0 2024-03-22T07:16:13.556 INFO:tasks.workunit.client.1.smithi069.stdout:5/892: dread d1/d28/f75 [0,4194304] 0 2024-03-22T07:16:13.557 INFO:tasks.workunit.client.1.smithi069.stdout:6/886: dwrite d4/dd/d12/fb3 [0,4194304] 0 2024-03-22T07:16:13.562 INFO:tasks.workunit.client.1.smithi069.stdout:1/962: dread d1/d2a/d84/dd1/fe1 [0,4194304] 0 2024-03-22T07:16:13.565 INFO:tasks.workunit.client.1.smithi069.stdout:6/887: link d4/d7/da/d111/f51 d4/dd/d109/dc7/f11b 0 2024-03-22T07:16:13.565 INFO:tasks.workunit.client.1.smithi069.stdout:5/893: dread d1/d10/d44/fb4 [0,4194304] 0 2024-03-22T07:16:13.566 INFO:tasks.workunit.client.1.smithi069.stdout:9/980: rename d2/db6 to d2/d21/d77/d144 0 2024-03-22T07:16:13.566 INFO:tasks.workunit.client.1.smithi069.stdout:2/958: dwrite d18/d4d/d27/d33/f10d [0,4194304] 0 2024-03-22T07:16:13.568 INFO:tasks.workunit.client.1.smithi069.stdout:1/963: link d1/d2a/db6/df8/f11e d1/d13/d36/de5/d4c/f13d 0 2024-03-22T07:16:13.568 INFO:tasks.workunit.client.1.smithi069.stdout:1/964: creat d1/d2a/d5e/f13e x:0 0 0 2024-03-22T07:16:13.569 INFO:tasks.workunit.client.1.smithi069.stdout:6/888: truncate d4/d7/f3e 1248508 0 2024-03-22T07:16:13.570 INFO:tasks.workunit.client.1.smithi069.stdout:1/965: dread d1/d13/f58 [0,4194304] 0 2024-03-22T07:16:13.570 INFO:tasks.workunit.client.1.smithi069.stdout:5/894: truncate d1/d6/d26/d3e/f8d 1269915 0 2024-03-22T07:16:13.571 INFO:tasks.workunit.client.1.smithi069.stdout:5/895: read - d1/d6/d26/d3e/d9c/f12b zero size 2024-03-22T07:16:13.571 INFO:tasks.workunit.client.1.smithi069.stdout:5/896: stat d1/d10/d44/dd8/d101/f115 0 2024-03-22T07:16:13.571 INFO:tasks.workunit.client.1.smithi069.stdout:5/897: readlink d1/d6/l5e 0 2024-03-22T07:16:13.572 INFO:tasks.workunit.client.1.smithi069.stdout:9/981: mkdir d2/dd/d3c/d48/d7f/da2/d145 0 2024-03-22T07:16:13.579 INFO:tasks.workunit.client.1.smithi069.stdout:6/889: creat d4/dd/d12/d2b/f11c x:0 0 0 2024-03-22T07:16:13.580 INFO:tasks.workunit.client.1.smithi069.stdout:5/898: dread d1/d10/d60/f83 [0,4194304] 0 2024-03-22T07:16:13.580 INFO:tasks.workunit.client.1.smithi069.stdout:1/966: mkdir d1/d2a/d13f 0 2024-03-22T07:16:13.581 INFO:tasks.workunit.client.1.smithi069.stdout:5/899: dread d1/d6/d86/f77 [0,4194304] 0 2024-03-22T07:16:13.582 INFO:tasks.workunit.client.1.smithi069.stdout:9/982: link d2/d21/dd7/ce2 d2/d21/d89/c146 0 2024-03-22T07:16:13.590 INFO:tasks.workunit.client.1.smithi069.stdout:5/900: dread d1/d6/d86/d99/d43/d8a/fd0 [0,4194304] 0 2024-03-22T07:16:13.594 INFO:tasks.workunit.client.1.smithi069.stdout:2/959: rmdir d18/d4d/d27/d33 39 2024-03-22T07:16:13.596 INFO:tasks.workunit.client.1.smithi069.stdout:6/890: unlink d4/dd/d12/d13/f40 0 2024-03-22T07:16:13.597 INFO:tasks.workunit.client.1.smithi069.stdout:1/967: symlink d1/d13/d36/d5b/l140 0 2024-03-22T07:16:13.597 INFO:tasks.workunit.client.1.smithi069.stdout:9/983: dwrite d2/d76/f106 [0,4194304] 0 2024-03-22T07:16:13.597 INFO:tasks.workunit.client.1.smithi069.stdout:9/984: creat d2/d31/f147 x:0 0 0 2024-03-22T07:16:13.597 INFO:tasks.workunit.client.1.smithi069.stdout:9/985: dread - d2/dd/d3c/d2e/d84/f96 zero size 2024-03-22T07:16:13.600 INFO:tasks.workunit.client.1.smithi069.stdout:5/901: rename d1/d6/d86/d99/f52 to d1/d10/d44/f12f 0 2024-03-22T07:16:13.602 INFO:tasks.workunit.client.1.smithi069.stdout:2/960: creat d18/d4d/d27/d33/d4f/ddd/f137 x:0 0 0 2024-03-22T07:16:13.603 INFO:tasks.workunit.client.1.smithi069.stdout:6/891: symlink d4/dd/d109/dc7/l11d 0 2024-03-22T07:16:13.603 INFO:tasks.workunit.client.1.smithi069.stdout:6/892: creat d4/dd/d109/f11e x:0 0 0 2024-03-22T07:16:13.605 INFO:tasks.workunit.client.1.smithi069.stdout:1/968: mkdir d1/d2a/d13f/d141 0 2024-03-22T07:16:13.614 INFO:tasks.workunit.client.1.smithi069.stdout:2/961: dread d18/d4d/d27/d33/d60/d64/d89/d8b/dab/fa7 [0,4194304] 0 2024-03-22T07:16:13.614 INFO:tasks.workunit.client.1.smithi069.stdout:5/902: mkdir d1/d6/d26/d3e/d9c/d9d/dc3/d130 0 2024-03-22T07:16:13.617 INFO:tasks.workunit.client.1.smithi069.stdout:1/969: getdents d1/d13/d82 0 2024-03-22T07:16:13.617 INFO:tasks.workunit.client.1.smithi069.stdout:1/970: write d1/d13/d36/de5/d4c/d55/f62 [1622637,92082] 0 2024-03-22T07:16:13.617 INFO:tasks.workunit.client.1.smithi069.stdout:1/971: dread - d1/d13/d36/de5/d88/dec/f11f zero size 2024-03-22T07:16:13.621 INFO:tasks.workunit.client.1.smithi069.stdout:9/986: dwrite d2/d51/fc2 [0,4194304] 0 2024-03-22T07:16:13.622 INFO:tasks.workunit.client.1.smithi069.stdout:5/903: creat d1/d6/d26/d33/d5d/d72/d78/d102/dea/f131 x:0 0 0 2024-03-22T07:16:13.623 INFO:tasks.workunit.client.1.smithi069.stdout:1/972: symlink d1/d2a/d5e/l142 0 2024-03-22T07:16:13.638 INFO:tasks.workunit.client.1.smithi069.stdout:2/962: dwrite d18/d4d/f22 [4194304,4194304] 0 2024-03-22T07:16:13.638 INFO:tasks.workunit.client.1.smithi069.stdout:2/963: readlink d18/d4d/d3e/d40/l75 0 2024-03-22T07:16:13.639 INFO:tasks.workunit.client.1.smithi069.stdout:2/964: rmdir d18/d4d/dec 39 2024-03-22T07:16:13.640 INFO:tasks.workunit.client.1.smithi069.stdout:2/965: write d18/d4d/d3e/d40/f6f [860309,120400] 0 2024-03-22T07:16:13.645 INFO:tasks.workunit.client.1.smithi069.stdout:5/904: dwrite d1/d6/d86/dab/fcb [0,4194304] 0 2024-03-22T07:16:13.646 INFO:tasks.workunit.client.1.smithi069.stdout:1/973: dwrite d1/d13/dc7/d121/d64/f9d [0,4194304] 0 2024-03-22T07:16:13.647 INFO:tasks.workunit.client.1.smithi069.stdout:5/905: symlink d1/d7/d3d/d100/l132 0 2024-03-22T07:16:13.647 INFO:tasks.workunit.client.1.smithi069.stdout:5/906: creat d1/d10/d44/dd8/de6/f133 x:0 0 0 2024-03-22T07:16:13.647 INFO:tasks.workunit.client.1.smithi069.stdout:5/907: write d1/d6/d86/dab/f125 [19658,97923] 0 2024-03-22T07:16:13.648 INFO:tasks.workunit.client.1.smithi069.stdout:1/974: creat d1/d13/dc7/d121/d131/f143 x:0 0 0 2024-03-22T07:16:13.648 INFO:tasks.workunit.client.1.smithi069.stdout:1/975: readlink d1/d2a/db6/df8/d73/df0/dbc/ldf 0 2024-03-22T07:16:13.650 INFO:tasks.workunit.client.1.smithi069.stdout:5/908: link d1/d6/d86/d99/dec/f10b d1/d6/d26/d3e/dbb/dbc/f134 0 2024-03-22T07:16:13.651 INFO:tasks.workunit.client.1.smithi069.stdout:1/976: link d1/d2a/db6/df8/d73/lb3 d1/d2a/d84/d9a/l144 0 2024-03-22T07:16:13.652 INFO:tasks.workunit.client.1.smithi069.stdout:5/909: mkdir d1/d6/d26/d3a/db2/d135 0 2024-03-22T07:16:13.652 INFO:tasks.workunit.client.1.smithi069.stdout:1/977: rmdir d1/d2a/d13f 39 2024-03-22T07:16:13.653 INFO:tasks.workunit.client.1.smithi069.stdout:2/966: dwrite d18/d4d/d27/d56/f94 [0,4194304] 0 2024-03-22T07:16:13.654 INFO:tasks.workunit.client.1.smithi069.stdout:5/910: rename d1/d7/d3d/dfa/f119 to d1/d6/d26/d33/d5d/d72/d78/f136 0 2024-03-22T07:16:13.659 INFO:tasks.workunit.client.1.smithi069.stdout:2/967: mkdir d18/d4d/d27/d33/d60/d64/dcd/d101/d134/d138 0 2024-03-22T07:16:13.659 INFO:tasks.workunit.client.1.smithi069.stdout:2/968: write d18/d4d/d27/d33/d60/d64/fc5 [1095027,66002] 0 2024-03-22T07:16:13.659 INFO:tasks.workunit.client.1.smithi069.stdout:2/969: creat d18/d4d/d27/d33/d4f/ddd/f139 x:0 0 0 2024-03-22T07:16:13.659 INFO:tasks.workunit.client.1.smithi069.stdout:5/911: mknod d1/d6/d26/d3e/dbb/d117/c137 0 2024-03-22T07:16:13.661 INFO:tasks.workunit.client.1.smithi069.stdout:1/978: rmdir d1/d13/d36 39 2024-03-22T07:16:13.664 INFO:tasks.workunit.client.1.smithi069.stdout:1/979: unlink d1/d13/dc7/d121/d7a/c13a 0 2024-03-22T07:16:13.665 INFO:tasks.workunit.client.1.smithi069.stdout:2/970: dread d18/d4d/d25/f36 [0,4194304] 0 2024-03-22T07:16:13.666 INFO:tasks.workunit.client.1.smithi069.stdout:2/971: mknod d18/d4d/dec/c13a 0 2024-03-22T07:16:13.669 INFO:tasks.workunit.client.1.smithi069.stdout:5/912: dread d1/d6/d26/d33/d5d/fa4 [0,4194304] 0 2024-03-22T07:16:13.670 INFO:tasks.workunit.client.1.smithi069.stdout:5/913: creat d1/d6/d86/d99/d43/d8a/f138 x:0 0 0 2024-03-22T07:16:13.672 INFO:tasks.workunit.client.1.smithi069.stdout:5/914: mknod d1/d6/d86/d99/d43/d8a/c139 0 2024-03-22T07:16:13.672 INFO:tasks.workunit.client.1.smithi069.stdout:5/915: dread - d1/d6/d26/d33/d5d/d72/d78/d102/dea/f131 zero size 2024-03-22T07:16:13.673 INFO:tasks.workunit.client.1.smithi069.stdout:1/980: write d1/d13/d36/fb4 [2212094,50814] 0 2024-03-22T07:16:13.684 INFO:tasks.workunit.client.1.smithi069.stdout:5/916: rename d1/d6/d86/d99/d43/d8a to d1/d6/d26/d7d/d9f/d13a 0 2024-03-22T07:16:13.687 INFO:tasks.workunit.client.1.smithi069.stdout:2/972: dwrite d18/fd6 [0,4194304] 0 2024-03-22T07:16:13.687 INFO:tasks.workunit.client.1.smithi069.stdout:2/973: creat d18/d4d/d3e/d40/f13b x:0 0 0 2024-03-22T07:16:13.690 INFO:tasks.workunit.client.1.smithi069.stdout:2/974: mkdir d18/d4d/d27/d33/d60/d64/dcd/d101/d133/d13c 0 2024-03-22T07:16:13.694 INFO:tasks.workunit.client.1.smithi069.stdout:1/981: dwrite d1/d13/dc7/d121/d64/f85 [0,4194304] 0 2024-03-22T07:16:13.694 INFO:tasks.workunit.client.1.smithi069.stdout:2/975: write d18/d4d/d27/d33/d60/fd7 [3456416,73884] 0 2024-03-22T07:16:13.694 INFO:tasks.workunit.client.1.smithi069.stdout:2/976: chown d18/d4d/d27/d33/d4f/ddd/df4/f12d 0 1 2024-03-22T07:16:13.697 INFO:tasks.workunit.client.1.smithi069.stdout:1/982: rename d1/d2a/d5e/d9b/fb7 to d1/d2a/db6/df8/d73/f145 0 2024-03-22T07:16:13.697 INFO:tasks.workunit.client.1.smithi069.stdout:1/983: chown d1/d2a/d5e 8175310 1 2024-03-22T07:16:13.697 INFO:tasks.workunit.client.1.smithi069.stdout:1/984: readlink d1/d13/dc7/d121/l59 0 2024-03-22T07:16:13.698 INFO:tasks.workunit.client.1.smithi069.stdout:2/977: symlink d18/d4d/d27/d33/d60/d63/d73/l13d 0 2024-03-22T07:16:13.698 INFO:tasks.workunit.client.1.smithi069.stdout:2/978: read - d18/d4d/d27/d33/d3d/ffe zero size 2024-03-22T07:16:13.704 INFO:tasks.workunit.client.1.smithi069.stdout:1/985: symlink d1/d13/d36/de5/d4c/d55/d5f/l146 0 2024-03-22T07:16:13.704 INFO:tasks.workunit.client.1.smithi069.stdout:1/986: write d1/d2a/d5e/f126 [770375,92972] 0 2024-03-22T07:16:13.724 INFO:tasks.workunit.client.1.smithi069.stdout:1/987: dwrite d1/d2a/d5e/d93/fa0 [4194304,4194304] 0 2024-03-22T07:16:13.724 INFO:tasks.workunit.client.1.smithi069.stdout:1/988: readlink d1/d13/dc7/d121/d1e/l33 0 2024-03-22T07:16:13.724 INFO:tasks.workunit.client.1.smithi069.stdout:2/979: dwrite d18/f9b [0,4194304] 0 2024-03-22T07:16:13.724 INFO:tasks.workunit.client.1.smithi069.stdout:1/989: stat d1/d13/d36/de5/d4c/d55/f10b 0 2024-03-22T07:16:13.729 INFO:tasks.workunit.client.1.smithi069.stdout:1/990: creat d1/d2a/db6/df8/f147 x:0 0 0 2024-03-22T07:16:13.729 INFO:tasks.workunit.client.1.smithi069.stdout:1/991: write d1/d2a/db6/ff6 [308986,66169] 0 2024-03-22T07:16:13.731 INFO:tasks.workunit.client.1.smithi069.stdout:1/992: creat d1/d2a/db6/df8/d73/f148 x:0 0 0 2024-03-22T07:16:13.740 INFO:tasks.workunit.client.1.smithi069.stdout:1/993: dread d1/d13/d36/de5/d4c/d55/f109 [0,4194304] 0 2024-03-22T07:16:13.740 INFO:tasks.workunit.client.1.smithi069.stdout:1/994: symlink d1/d13/d36/de5/d88/dae/l149 0 2024-03-22T07:16:13.742 INFO:tasks.workunit.client.1.smithi069.stdout:1/995: rmdir d1/d2a/d5e 39 2024-03-22T07:16:13.743 INFO:tasks.workunit.client.1.smithi069.stdout:1/996: symlink d1/d13/d36/de5/d4c/d55/d5f/l14a 0 2024-03-22T07:16:13.745 INFO:tasks.workunit.client.1.smithi069.stdout:1/997: creat d1/d13/d36/de5/d88/dae/de7/d114/f14b x:0 0 0 2024-03-22T07:16:13.749 INFO:tasks.workunit.client.1.smithi069.stdout:1/998: mknod d1/c14c 0 2024-03-22T07:16:13.749 INFO:tasks.workunit.client.1.smithi069.stdout:1/999: write d1/d13/f137 [516288,33952] 0 2024-03-22T07:16:13.897 INFO:tasks.workunit.client.1.smithi069.stdout:6/893: sync 2024-03-22T07:16:13.897 INFO:tasks.workunit.client.1.smithi069.stdout:2/980: sync 2024-03-22T07:16:13.898 INFO:tasks.workunit.client.1.smithi069.stdout:9/987: sync 2024-03-22T07:16:13.898 INFO:tasks.workunit.client.1.smithi069.stdout:5/917: sync 2024-03-22T07:16:13.898 INFO:tasks.workunit.client.1.smithi069.stdout:2/981: rmdir d18/d4d/d27/d33/d60/d64/dcd/d101/d134 39 2024-03-22T07:16:13.900 INFO:tasks.workunit.client.1.smithi069.stdout:6/894: rename d4/dd/d12/d2b/d72/d73/d110 to d4/d7/da/d111/d11f 0 2024-03-22T07:16:13.902 INFO:tasks.workunit.client.1.smithi069.stdout:5/918: unlink d1/d7/d3d/c53 0 2024-03-22T07:16:13.903 INFO:tasks.workunit.client.1.smithi069.stdout:2/982: mkdir d18/d4d/d27/d33/d60/d64/dcd/d101/d13e 0 2024-03-22T07:16:13.904 INFO:tasks.workunit.client.1.smithi069.stdout:2/983: write d18/d4d/d27/d33/d60/d63/d80/fde [844367,5081] 0 2024-03-22T07:16:13.904 INFO:tasks.workunit.client.1.smithi069.stdout:2/984: dread - d18/d4d/d27/d33/d3d/f11a zero size 2024-03-22T07:16:13.904 INFO:tasks.workunit.client.1.smithi069.stdout:2/985: dread - d18/d4d/d27/d33/f103 zero size 2024-03-22T07:16:13.905 INFO:tasks.workunit.client.1.smithi069.stdout:2/986: creat d18/d4d/d27/d33/d60/d64/dcd/d101/d13e/f13f x:0 0 0 2024-03-22T07:16:13.911 INFO:tasks.workunit.client.1.smithi069.stdout:6/895: dread d4/d7/da/d111/f51 [0,4194304] 0 2024-03-22T07:16:13.913 INFO:tasks.workunit.client.1.smithi069.stdout:6/896: rename d4/d7/da/d111/d78/d9a/fd8 to d4/dd/d12/d2b/da5/f120 0 2024-03-22T07:16:13.913 INFO:tasks.workunit.client.1.smithi069.stdout:6/897: creat d4/dd/d109/f121 x:0 0 0 2024-03-22T07:16:13.927 INFO:tasks.workunit.client.1.smithi069.stdout:9/988: dwrite d2/d76/f111 [0,4194304] 0 2024-03-22T07:16:13.927 INFO:tasks.workunit.client.1.smithi069.stdout:9/989: read - d2/d83/fc5 zero size 2024-03-22T07:16:13.927 INFO:tasks.workunit.client.1.smithi069.stdout:9/990: fdatasync d2/dd/d3c/d53/d75/f10b 0 2024-03-22T07:16:13.927 INFO:tasks.workunit.client.1.smithi069.stdout:9/991: chown d2/fbf 611744144 1 2024-03-22T07:16:13.927 INFO:tasks.workunit.client.1.smithi069.stdout:5/919: dwrite d1/d10/d44/dd8/d101/f115 [0,4194304] 0 2024-03-22T07:16:13.927 INFO:tasks.workunit.client.1.smithi069.stdout:5/920: readlink d1/d6/d26/d3e/d9c/d9d/dc3/l107 0 2024-03-22T07:16:13.927 INFO:tasks.workunit.client.1.smithi069.stdout:2/987: dwrite d18/d4d/d27/d33/d60/d63/d80/d100/f11b [0,4194304] 0 2024-03-22T07:16:13.930 INFO:tasks.workunit.client.1.smithi069.stdout:9/992: creat d2/d21/d89/f148 x:0 0 0 2024-03-22T07:16:13.931 INFO:tasks.workunit.client.1.smithi069.stdout:5/921: getdents d1/d6/d86/dab/dcf/dde 0 2024-03-22T07:16:13.936 INFO:tasks.workunit.client.1.smithi069.stdout:5/922: dread d1/d6/d86/dab/ff0 [0,4194304] 0 2024-03-22T07:16:13.937 INFO:tasks.workunit.client.1.smithi069.stdout:2/988: symlink d18/d4d/d27/d33/d60/d63/d80/l140 0 2024-03-22T07:16:13.937 INFO:tasks.workunit.client.1.smithi069.stdout:2/989: chown d18/d4d/d27/d56/l68 13140 1 2024-03-22T07:16:13.948 INFO:tasks.workunit.client.1.smithi069.stdout:5/923: dread d1/d6/d26/d33/d5d/d72/d95/fe4 [0,4194304] 0 2024-03-22T07:16:13.948 INFO:tasks.workunit.client.1.smithi069.stdout:9/993: dwrite d2/dd/d3c/d53/d9d/f137 [4194304,4194304] 0 2024-03-22T07:16:13.950 INFO:tasks.workunit.client.1.smithi069.stdout:5/924: creat d1/d6/d26/d7d/d9f/d13a/f13b x:0 0 0 2024-03-22T07:16:13.950 INFO:tasks.workunit.client.1.smithi069.stdout:2/990: write d18/d4d/d27/d33/d60/d64/fda [850676,81608] 0 2024-03-22T07:16:13.950 INFO:tasks.workunit.client.1.smithi069.stdout:5/925: symlink d1/d7/d3d/dfa/l13c 0 2024-03-22T07:16:13.953 INFO:tasks.workunit.client.1.smithi069.stdout:2/991: rename d18/d4d/d27/d33/d60/d64/d89/d10f/dd1/cfb to d18/d4d/d27/d33/d60/d63/d80/c141 0 2024-03-22T07:16:13.954 INFO:tasks.workunit.client.1.smithi069.stdout:5/926: getdents d1/d10/d44/dd8/de6 0 2024-03-22T07:16:13.954 INFO:tasks.workunit.client.1.smithi069.stdout:5/927: getdents d1/d10/d44/dd8/de6 0 2024-03-22T07:16:13.954 INFO:tasks.workunit.client.1.smithi069.stdout:5/928: truncate d1/d6/d26/d3e/dbb/fdd 4925783 0 2024-03-22T07:16:13.954 INFO:tasks.workunit.client.1.smithi069.stdout:5/929: creat d1/d10/d60/f13d x:0 0 0 2024-03-22T07:16:13.954 INFO:tasks.workunit.client.1.smithi069.stdout:5/930: stat d1/d6/d26/d3a/db2/ff9 0 2024-03-22T07:16:13.955 INFO:tasks.workunit.client.1.smithi069.stdout:9/994: dread d2/dd/d3c/d48/d7c/f9e [0,4194304] 0 2024-03-22T07:16:13.958 INFO:tasks.workunit.client.1.smithi069.stdout:5/931: rename d1/d6/d86/f77 to d1/d6/d26/d33/d5d/d72/d78/d102/dea/f13e 0 2024-03-22T07:16:13.959 INFO:tasks.workunit.client.1.smithi069.stdout:9/995: creat d2/dd/d3c/d48/d7f/d86/f149 x:0 0 0 2024-03-22T07:16:13.962 INFO:tasks.workunit.client.1.smithi069.stdout:5/932: rename d1/d6/d26/c65 to d1/d10/d60/c13f 0 2024-03-22T07:16:13.974 INFO:tasks.workunit.client.1.smithi069.stdout:5/933: write d1/d6/d86/d99/f90 [125184,85891] 0 2024-03-22T07:16:13.976 INFO:tasks.workunit.client.1.smithi069.stdout:5/934: rename d1/d7/fa5 to d1/d10/d44/dd8/f140 0 2024-03-22T07:16:13.977 INFO:tasks.workunit.client.1.smithi069.stdout:5/935: truncate d1/d6/d26/d3e/d9c/d9d/fa6 1294327 0 2024-03-22T07:16:13.977 INFO:tasks.workunit.client.1.smithi069.stdout:2/992: dwrite d18/d4d/d27/d33/d3d/f46 [0,4194304] 0 2024-03-22T07:16:13.978 INFO:tasks.workunit.client.1.smithi069.stdout:2/993: write d18/d4d/d27/d33/d60/d64/d89/d8b/dab/fa7 [4447128,15969] 0 2024-03-22T07:16:13.978 INFO:tasks.workunit.client.1.smithi069.stdout:9/996: dwrite d2/d31/fc4 [0,4194304] 0 2024-03-22T07:16:13.978 INFO:tasks.workunit.client.1.smithi069.stdout:5/936: getdents d1/d6/d26/d3e 0 2024-03-22T07:16:13.981 INFO:tasks.workunit.client.1.smithi069.stdout:9/997: link d2/dd/d3c/d2e/f4b d2/dd/d3c/d2e/d84/f14a 0 2024-03-22T07:16:13.987 INFO:tasks.workunit.client.1.smithi069.stdout:2/994: dread f1 [0,4194304] 0 2024-03-22T07:16:13.988 INFO:tasks.workunit.client.1.smithi069.stdout:2/995: getdents d18/d4d/d3e 0 2024-03-22T07:16:13.990 INFO:tasks.workunit.client.1.smithi069.stdout:2/996: unlink d18/d4d/d27/d33/d4f/c79 0 2024-03-22T07:16:13.991 INFO:tasks.workunit.client.1.smithi069.stdout:2/997: creat d18/d4d/d25/f142 x:0 0 0 2024-03-22T07:16:13.992 INFO:tasks.workunit.client.1.smithi069.stdout:9/998: dread d2/dd/fac [0,4194304] 0 2024-03-22T07:16:13.992 INFO:tasks.workunit.client.1.smithi069.stdout:9/999: chown d2/dd/f123 251309358 1 2024-03-22T07:16:13.992 INFO:tasks.workunit.client.1.smithi069.stdout:2/998: mknod d18/d4d/d27/d33/d60/d64/dcd/d101/d134/d138/c143 0 2024-03-22T07:16:14.001 INFO:tasks.workunit.client.1.smithi069.stdout:5/937: dwrite d1/d6/d26/d33/d5d/d72/d78/f136 [0,4194304] 0 2024-03-22T07:16:14.001 INFO:tasks.workunit.client.1.smithi069.stdout:5/938: write d1/d6/d86/dab/fcb [4725237,130679] 0 2024-03-22T07:16:14.005 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:13 smithi063 ceph-mon[29200]: pgmap v9: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 298 MiB/s rd, 319 MiB/s wr, 492 op/s 2024-03-22T07:16:14.008 INFO:tasks.workunit.client.1.smithi069.stdout:2/999: dwrite d18/d4d/d27/d33/d60/d63/d73/fb4 [0,4194304] 0 2024-03-22T07:16:14.019 INFO:tasks.workunit.client.1.smithi069.stdout:5/939: dwrite d1/d10/d44/dd8/de6/ffe [0,4194304] 0 2024-03-22T07:16:14.025 INFO:tasks.workunit.client.1.smithi069.stdout:5/940: rename d1/d6/d26/d3e/dbb/dbc/c11a to d1/d6/d26/d3a/c141 0 2024-03-22T07:16:14.026 INFO:tasks.workunit.client.1.smithi069.stdout:5/941: creat d1/d6/d88/dfb/f142 x:0 0 0 2024-03-22T07:16:14.026 INFO:tasks.workunit.client.1.smithi069.stdout:5/942: truncate d1/d6/d26/d3e/f6d 166958 0 2024-03-22T07:16:14.027 INFO:tasks.workunit.client.1.smithi069.stdout:5/943: fsync d1/d10/d44/dd8/d101/f115 0 2024-03-22T07:16:14.027 INFO:tasks.workunit.client.1.smithi069.stdout:5/944: creat d1/d6/d26/d33/d5d/d72/d95/f143 x:0 0 0 2024-03-22T07:16:14.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:13 smithi069 ceph-mon[36441]: pgmap v9: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 298 MiB/s rd, 319 MiB/s wr, 492 op/s 2024-03-22T07:16:14.030 INFO:tasks.workunit.client.1.smithi069.stdout:5/945: unlink d1/d6/d26/d33/d5d/d72/da7/fc5 0 2024-03-22T07:16:14.033 INFO:tasks.workunit.client.1.smithi069.stdout:5/946: rename d1/d6/d21/cd1 to d1/d6/d26/d33/d5d/d72/d78/d102/dea/c144 0 2024-03-22T07:16:14.033 INFO:tasks.workunit.client.1.smithi069.stdout:5/947: fsync d1/d10/d60/f83 0 2024-03-22T07:16:14.034 INFO:tasks.workunit.client.1.smithi069.stdout:5/948: truncate d1/d6/d86/d99/f58 1236850 0 2024-03-22T07:16:14.036 INFO:tasks.workunit.client.1.smithi069.stdout:5/949: creat d1/d6/d26/d33/d5d/d72/f145 x:0 0 0 2024-03-22T07:16:14.040 INFO:tasks.workunit.client.1.smithi069.stdout:5/950: rename d1/d6/d26/d33/d5d/d72/d78/d102/dea/f13e to d1/d6/d26/d3e/dbb/d117/f146 0 2024-03-22T07:16:14.040 INFO:tasks.workunit.client.1.smithi069.stdout:5/951: chown d1/d6/d26/d3a/db2 850789 1 2024-03-22T07:16:14.041 INFO:tasks.workunit.client.1.smithi069.stdout:5/952: mkdir d1/d6/d26/d3e/dbb/d117/d120/d147 0 2024-03-22T07:16:14.043 INFO:tasks.workunit.client.1.smithi069.stdout:5/953: symlink d1/d10/d44/d50/dd5/l148 0 2024-03-22T07:16:14.046 INFO:tasks.workunit.client.1.smithi069.stdout:5/954: mknod d1/d10/d44/c149 0 2024-03-22T07:16:14.048 INFO:tasks.workunit.client.1.smithi069.stdout:5/955: unlink d1/d7/c1d 0 2024-03-22T07:16:14.071 INFO:tasks.workunit.client.1.smithi069.stdout:5/956: dwrite d1/d6/d86/f73 [0,4194304] 0 2024-03-22T07:16:14.074 INFO:tasks.workunit.client.1.smithi069.stdout:5/957: rename d1/d7/d3d/l48 to d1/d6/d26/d3e/dbb/d117/l14a 0 2024-03-22T07:16:14.075 INFO:tasks.workunit.client.1.smithi069.stdout:5/958: creat d1/d6/d86/f14b x:0 0 0 2024-03-22T07:16:14.077 INFO:tasks.workunit.client.1.smithi069.stdout:5/959: truncate d1/d6/d88/f93 841136 0 2024-03-22T07:16:14.077 INFO:tasks.workunit.client.1.smithi069.stdout:5/960: creat d1/d6/d26/d33/d5d/d72/d95/f14c x:0 0 0 2024-03-22T07:16:14.078 INFO:tasks.workunit.client.1.smithi069.stdout:5/961: symlink d1/d6/d26/d3e/dbb/l14d 0 2024-03-22T07:16:14.097 INFO:tasks.workunit.client.1.smithi069.stdout:5/962: dwrite d1/d6/d86/d99/dec/f10b [0,4194304] 0 2024-03-22T07:16:14.097 INFO:tasks.workunit.client.1.smithi069.stdout:5/963: chown d1/d7/c2b 1481 1 2024-03-22T07:16:14.097 INFO:tasks.workunit.client.1.smithi069.stdout:5/964: chown d1/d6/d26/d33/d5d/d72/d78/d102/ccc 0 1 2024-03-22T07:16:14.105 INFO:tasks.workunit.client.1.smithi069.stdout:5/965: write d1/d6/d26/d7d/d9f/d13a/fd0 [1689299,51595] 0 2024-03-22T07:16:14.107 INFO:tasks.workunit.client.1.smithi069.stdout:5/966: symlink d1/d7/d3d/d100/l14e 0 2024-03-22T07:16:14.107 INFO:tasks.workunit.client.1.smithi069.stdout:5/967: readlink d1/d6/d26/d3e/d9c/d9d/dc3/l107 0 2024-03-22T07:16:14.109 INFO:tasks.workunit.client.1.smithi069.stdout:5/968: dread d1/d6/d26/d33/d5d/d72/d95/faf [0,4194304] 0 2024-03-22T07:16:14.110 INFO:tasks.workunit.client.1.smithi069.stdout:5/969: dread d1/d10/d44/dd8/f140 [0,4194304] 0 2024-03-22T07:16:14.114 INFO:tasks.workunit.client.1.smithi069.stdout:5/970: rename d1/d6/d26/d33/d5d/d72/d95/dc7 to d1/d6/d86/dab/dcf/d14f 0 2024-03-22T07:16:14.114 INFO:tasks.workunit.client.1.smithi069.stdout:5/971: stat d1/d6/d26/d33/d5d/d72/d78/f11c 0 2024-03-22T07:16:14.114 INFO:tasks.workunit.client.1.smithi069.stdout:6/898: sync 2024-03-22T07:16:14.115 INFO:tasks.workunit.client.1.smithi069.stdout:6/899: creat d4/dd/d109/f122 x:0 0 0 2024-03-22T07:16:14.117 INFO:tasks.workunit.client.1.smithi069.stdout:6/900: getdents d4/d7/da/d111/d78 0 2024-03-22T07:16:14.142 INFO:tasks.workunit.client.1.smithi069.stdout:5/972: dwrite d1/d6/d88/dfb/f12a [0,4194304] 0 2024-03-22T07:16:14.143 INFO:tasks.workunit.client.1.smithi069.stdout:6/901: dwrite d4/d7/da/d3f/fff [0,4194304] 0 2024-03-22T07:16:14.143 INFO:tasks.workunit.client.1.smithi069.stdout:6/902: dread - d4/d7/f116 zero size 2024-03-22T07:16:14.164 INFO:tasks.workunit.client.1.smithi069.stdout:5/973: dwrite d1/d10/d60/f13d [0,4194304] 0 2024-03-22T07:16:14.164 INFO:tasks.workunit.client.1.smithi069.stdout:5/974: fdatasync d1/d6/d26/d33/d5d/d72/d78/f118 0 2024-03-22T07:16:14.164 INFO:tasks.workunit.client.1.smithi069.stdout:5/975: write d1/d28/fb8 [1593194,57075] 0 2024-03-22T07:16:14.164 INFO:tasks.workunit.client.1.smithi069.stdout:6/903: dwrite d4/d7/f35 [0,4194304] 0 2024-03-22T07:16:14.164 INFO:tasks.workunit.client.1.smithi069.stdout:6/904: dread - d4/dd/dbd/f117 zero size 2024-03-22T07:16:14.165 INFO:tasks.workunit.client.1.smithi069.stdout:5/976: symlink d1/d10/d44/dd8/l150 0 2024-03-22T07:16:14.167 INFO:tasks.workunit.client.1.smithi069.stdout:5/977: unlink d1/d6/d26/l51 0 2024-03-22T07:16:14.167 INFO:tasks.workunit.client.1.smithi069.stdout:5/978: creat d1/d6/d26/d33/d5d/d72/d78/d102/dea/f151 x:0 0 0 2024-03-22T07:16:14.168 INFO:tasks.workunit.client.1.smithi069.stdout:5/979: rmdir d1/d6/d26/d33 39 2024-03-22T07:16:14.172 INFO:tasks.workunit.client.1.smithi069.stdout:6/905: write d4/dd/d12/d13/f8a [686750,5437] 0 2024-03-22T07:16:14.172 INFO:tasks.workunit.client.1.smithi069.stdout:5/980: rename d1/d6/d26/d33/d5d/d72/d95 to d1/d6/d88/dfb/d152 0 2024-03-22T07:16:14.173 INFO:tasks.workunit.client.1.smithi069.stdout:5/981: creat d1/d6/d26/d33/d5d/d72/da7/f153 x:0 0 0 2024-03-22T07:16:14.173 INFO:tasks.workunit.client.1.smithi069.stdout:5/982: chown d1/d6/d26/d33/d5d/d72/d78/d102/ccc 1476948 1 2024-03-22T07:16:14.173 INFO:tasks.workunit.client.1.smithi069.stdout:6/906: readlink d4/d7/da/d3f/l87 0 2024-03-22T07:16:14.179 INFO:tasks.workunit.client.1.smithi069.stdout:6/907: getdents d4/d7/da/d3f 0 2024-03-22T07:16:14.179 INFO:tasks.workunit.client.1.smithi069.stdout:6/908: fsync d4/dd/d12/d2b/d72/d73/f118 0 2024-03-22T07:16:14.179 INFO:tasks.workunit.client.1.smithi069.stdout:6/909: chown d4/d7/da/d111/d78/d9a/cd5 29846620 1 2024-03-22T07:16:14.180 INFO:tasks.workunit.client.1.smithi069.stdout:6/910: fdatasync d4/dd/d12/d2b/f3c 0 2024-03-22T07:16:14.180 INFO:tasks.workunit.client.1.smithi069.stdout:6/911: fsync d4/d7/da/d3f/f6d 0 2024-03-22T07:16:14.180 INFO:tasks.workunit.client.1.smithi069.stdout:6/912: creat d4/dd/d12/d2b/d72/d73/f123 x:0 0 0 2024-03-22T07:16:14.182 INFO:tasks.workunit.client.1.smithi069.stdout:6/913: symlink d4/dd/d12/d2b/d72/l124 0 2024-03-22T07:16:14.184 INFO:tasks.workunit.client.1.smithi069.stdout:6/914: truncate d4/f77 1543800 0 2024-03-22T07:16:14.184 INFO:tasks.workunit.client.1.smithi069.stdout:6/915: fdatasync d4/d7/da/d111/d78/d9a/f10d 0 2024-03-22T07:16:14.186 INFO:tasks.workunit.client.1.smithi069.stdout:6/916: link d4/d7/da/d111/d78/d9a/cd5 d4/dd/d12/d2b/d72/d73/c125 0 2024-03-22T07:16:14.186 INFO:tasks.workunit.client.1.smithi069.stdout:6/917: truncate d4/d7/d5b/ffd 182576 0 2024-03-22T07:16:14.196 INFO:tasks.workunit.client.1.smithi069.stdout:6/918: write d4/dd/d12/d2b/f30 [8219472,62117] 0 2024-03-22T07:16:14.197 INFO:tasks.workunit.client.1.smithi069.stdout:5/983: dwrite d1/d6/d26/d3e/d9c/d9d/fa6 [0,4194304] 0 2024-03-22T07:16:14.199 INFO:tasks.workunit.client.1.smithi069.stdout:6/919: creat d4/d7/da/d111/d78/f126 x:0 0 0 2024-03-22T07:16:14.203 INFO:tasks.workunit.client.1.smithi069.stdout:6/920: dread d4/dd/d12/d106/f4e [0,4194304] 0 2024-03-22T07:16:14.205 INFO:tasks.workunit.client.1.smithi069.stdout:6/921: mknod d4/dd/c127 0 2024-03-22T07:16:14.205 INFO:tasks.workunit.client.1.smithi069.stdout:6/922: chown d4/dd/d12/d106/le8 504515565 1 2024-03-22T07:16:14.207 INFO:tasks.workunit.client.1.smithi069.stdout:6/923: unlink d4/dd/d12/d13/f59 0 2024-03-22T07:16:14.207 INFO:tasks.workunit.client.1.smithi069.stdout:6/924: readlink d4/d7/l67 0 2024-03-22T07:16:14.207 INFO:tasks.workunit.client.1.smithi069.stdout:6/925: read - d4/dd/d12/d13/fe6 zero size 2024-03-22T07:16:14.207 INFO:tasks.workunit.client.1.smithi069.stdout:6/926: mkdir d4/dd/d109/dc7/d128 0 2024-03-22T07:16:14.207 INFO:tasks.workunit.client.1.smithi069.stdout:6/927: write d4/dd/d12/d106/f4e [1306596,11049] 0 2024-03-22T07:16:14.209 INFO:tasks.workunit.client.1.smithi069.stdout:6/928: rmdir d4/dd/d12/d13/de9 39 2024-03-22T07:16:14.210 INFO:tasks.workunit.client.1.smithi069.stdout:6/929: write d4/d7/da/d3f/fff [4554528,15609] 0 2024-03-22T07:16:14.210 INFO:tasks.workunit.client.1.smithi069.stdout:6/930: fsync d4/f77 0 2024-03-22T07:16:14.210 INFO:tasks.workunit.client.1.smithi069.stdout:6/931: truncate d4/dd/d109/f113 486189 0 2024-03-22T07:16:14.212 INFO:tasks.workunit.client.1.smithi069.stdout:6/932: getdents d4/dd/d12/d2b/da5/db2 0 2024-03-22T07:16:14.213 INFO:tasks.workunit.client.1.smithi069.stdout:6/933: creat d4/f129 x:0 0 0 2024-03-22T07:16:14.213 INFO:tasks.workunit.client.1.smithi069.stdout:6/934: mknod d4/d7/d5b/c12a 0 2024-03-22T07:16:14.214 INFO:tasks.workunit.client.1.smithi069.stdout:5/984: dwrite d1/d6/d26/d33/d5d/d72/d78/f11c [0,4194304] 0 2024-03-22T07:16:14.218 INFO:tasks.workunit.client.1.smithi069.stdout:5/985: symlink d1/d6/d26/d3a/dbd/l154 0 2024-03-22T07:16:14.223 INFO:tasks.workunit.client.1.smithi069.stdout:6/935: dread d4/f60 [0,4194304] 0 2024-03-22T07:16:14.224 INFO:tasks.workunit.client.1.smithi069.stdout:6/936: mkdir d4/dd/d12/d12b 0 2024-03-22T07:16:14.225 INFO:tasks.workunit.client.1.smithi069.stdout:5/986: dread d1/d6/d86/f73 [0,4194304] 0 2024-03-22T07:16:14.225 INFO:tasks.workunit.client.1.smithi069.stdout:5/987: chown d1/d10/df1/c113 176367 1 2024-03-22T07:16:14.225 INFO:tasks.workunit.client.1.smithi069.stdout:6/937: rmdir d4/dd/d12/d106/d58/d10b 0 2024-03-22T07:16:14.233 INFO:tasks.workunit.client.1.smithi069.stdout:5/988: dread d1/d6/d26/d3e/dbb/fdd [0,4194304] 0 2024-03-22T07:16:14.233 INFO:tasks.workunit.client.1.smithi069.stdout:5/989: write d1/d6/f6f [3534929,94060] 0 2024-03-22T07:16:14.234 INFO:tasks.workunit.client.1.smithi069.stdout:5/990: rename d1/d6/d26/d3e/d9c to d1/d6/d26/d33/d155 0 2024-03-22T07:16:14.234 INFO:tasks.workunit.client.1.smithi069.stdout:5/991: truncate d1/d6/d86/d99/d43/ff6 2953923 0 2024-03-22T07:16:14.301 INFO:tasks.workunit.client.1.smithi069.stdout:5/992: creat d1/d6/d26/d33/d5d/d72/d78/d102/f156 x:0 0 0 2024-03-22T07:16:14.302 INFO:tasks.workunit.client.1.smithi069.stdout:5/993: link d1/d6/d26/d3e/dbb/dbc/ff4 d1/d10/d44/dd8/f157 0 2024-03-22T07:16:14.302 INFO:tasks.workunit.client.1.smithi069.stdout:5/994: fsync d1/d6/d86/dab/dcf/d14f/dce/f126 0 2024-03-22T07:16:14.310 INFO:tasks.workunit.client.1.smithi069.stdout:5/995: dread d1/d10/d44/d50/f96 [0,4194304] 0 2024-03-22T07:16:14.331 INFO:tasks.workunit.client.1.smithi069.stdout:5/996: dwrite d1/d6/d86/d99/f4b [0,4194304] 0 2024-03-22T07:16:14.331 INFO:tasks.workunit.client.1.smithi069.stdout:5/997: chown d1/d6/d26/d33/d5d/d72/da7 202792 1 2024-03-22T07:16:14.332 INFO:tasks.workunit.client.1.smithi069.stdout:5/998: creat d1/d6/d26/d33/dd6/f158 x:0 0 0 2024-03-22T07:16:14.332 INFO:tasks.workunit.client.1.smithi069.stdout:5/999: rmdir d1/d6/d26/d33/d5d 39 2024-03-22T07:16:14.441 INFO:tasks.workunit.client.1.smithi069.stdout:6/938: sync 2024-03-22T07:16:14.441 INFO:tasks.workunit.client.1.smithi069.stdout:6/939: fdatasync d4/dd/d12/d13/f86 0 2024-03-22T07:16:14.457 INFO:tasks.workunit.client.1.smithi069.stdout:6/940: dwrite d4/f47 [0,4194304] 0 2024-03-22T07:16:14.457 INFO:tasks.workunit.client.1.smithi069.stdout:6/941: fsync d4/d7/d5b/f97 0 2024-03-22T07:16:14.472 INFO:tasks.workunit.client.1.smithi069.stdout:6/942: dwrite d4/dd/d12/d13/de9/ff3 [0,4194304] 0 2024-03-22T07:16:14.595 INFO:tasks.workunit.client.1.smithi069.stdout:6/943: sync 2024-03-22T07:16:14.596 INFO:tasks.workunit.client.1.smithi069.stdout:6/944: creat d4/d7/da/d111/d78/da3/f12c x:0 0 0 2024-03-22T07:16:14.597 INFO:tasks.workunit.client.1.smithi069.stdout:6/945: write d4/dd/d12/d13/de9/fb1 [323740,110672] 0 2024-03-22T07:16:14.598 INFO:tasks.workunit.client.1.smithi069.stdout:6/946: link d4/dd/d12/d13/l88 d4/dd/d12/d2b/da5/l12d 0 2024-03-22T07:16:14.610 INFO:tasks.workunit.client.1.smithi069.stdout:6/947: dread d4/dd/d12/fb5 [0,4194304] 0 2024-03-22T07:16:14.611 INFO:tasks.workunit.client.1.smithi069.stdout:6/948: truncate d4/dd/d12/d106/d58/ffa 2802602 0 2024-03-22T07:16:14.611 INFO:tasks.workunit.client.1.smithi069.stdout:6/949: creat d4/dd/f12e x:0 0 0 2024-03-22T07:16:14.621 INFO:tasks.workunit.client.1.smithi069.stdout:6/950: dread d4/d7/da/d3f/fc9 [0,4194304] 0 2024-03-22T07:16:14.622 INFO:tasks.workunit.client.1.smithi069.stdout:6/951: link d4/d7/da/d111/l69 d4/d7/da/l12f 0 2024-03-22T07:16:14.622 INFO:tasks.workunit.client.1.smithi069.stdout:6/952: write d4/dd/d12/d106/f4e [4934287,52463] 0 2024-03-22T07:16:14.624 INFO:tasks.workunit.client.1.smithi069.stdout:6/953: dread d4/dd/d12/d2b/da5/fcc [0,4194304] 0 2024-03-22T07:16:14.625 INFO:tasks.workunit.client.1.smithi069.stdout:6/954: mknod d4/d7/da/d3f/d76/dab/d112/c130 0 2024-03-22T07:16:14.626 INFO:tasks.workunit.client.1.smithi069.stdout:6/955: creat d4/dd/d12/d12b/f131 x:0 0 0 2024-03-22T07:16:14.627 INFO:tasks.workunit.client.1.smithi069.stdout:6/956: rename d4/f129 to d4/dd/d12/d2b/d72/d73/d74/f132 0 2024-03-22T07:16:14.628 INFO:tasks.workunit.client.1.smithi069.stdout:6/957: mkdir d4/d133 0 2024-03-22T07:16:14.774 INFO:tasks.workunit.client.1.smithi069.stdout:6/958: sync 2024-03-22T07:16:14.775 INFO:tasks.workunit.client.1.smithi069.stdout:6/959: rmdir d4/d7/d5b 39 2024-03-22T07:16:14.776 INFO:tasks.workunit.client.1.smithi069.stdout:6/960: chown d4/d7/d5b/l64 1 1 2024-03-22T07:16:14.776 INFO:tasks.workunit.client.1.smithi069.stdout:6/961: creat d4/d7/da6/f134 x:0 0 0 2024-03-22T07:16:14.777 INFO:tasks.workunit.client.1.smithi069.stdout:6/962: rmdir d4/d7/d5b 39 2024-03-22T07:16:14.783 INFO:tasks.workunit.client.1.smithi069.stdout:6/963: dread d4/dd/d12/d13/f86 [0,4194304] 0 2024-03-22T07:16:14.785 INFO:tasks.workunit.client.1.smithi069.stdout:6/964: unlink d4/dd/d12/d2b/f103 0 2024-03-22T07:16:14.785 INFO:tasks.workunit.client.1.smithi069.stdout:6/965: write d4/dd/d12/d2b/d72/d73/d74/f132 [422749,43246] 0 2024-03-22T07:16:14.799 INFO:tasks.workunit.client.1.smithi069.stdout:6/966: dwrite d4/dd/f25 [4194304,4194304] 0 2024-03-22T07:16:14.808 INFO:tasks.workunit.client.1.smithi069.stdout:6/967: dread d4/dd/d12/d2b/d72/d73/d9d/dc3/ff1 [0,4194304] 0 2024-03-22T07:16:14.824 INFO:tasks.workunit.client.1.smithi069.stdout:6/968: dwrite d4/fe3 [0,4194304] 0 2024-03-22T07:16:14.825 INFO:tasks.workunit.client.1.smithi069.stdout:6/969: write d4/dd/d109/dc7/fe7 [655639,19750] 0 2024-03-22T07:16:14.825 INFO:tasks.workunit.client.1.smithi069.stdout:6/970: mknod d4/d7/da/d3f/d76/dab/c135 0 2024-03-22T07:16:14.826 INFO:tasks.workunit.client.1.smithi069.stdout:6/971: mkdir d4/dd/d12/d2b/d136 0 2024-03-22T07:16:14.826 INFO:tasks.workunit.client.1.smithi069.stdout:6/972: dread - d4/dd/d109/f11e zero size 2024-03-22T07:16:14.826 INFO:tasks.workunit.client.1.smithi069.stdout:6/973: chown d4/dd/d12/d106/l4b 3548339 1 2024-03-22T07:16:14.826 INFO:tasks.workunit.client.1.smithi069.stdout:6/974: write d4/dd/d12/d13/f5e [2768830,75063] 0 2024-03-22T07:16:14.838 INFO:tasks.workunit.client.1.smithi069.stdout:6/975: dwrite d4/dd/d12/d106/ff0 [0,4194304] 0 2024-03-22T07:16:14.840 INFO:tasks.workunit.client.1.smithi069.stdout:6/976: rename d4/d7/da/d3f/fff to d4/d7/da/d111/d11f/f137 0 2024-03-22T07:16:14.841 INFO:tasks.workunit.client.1.smithi069.stdout:6/977: mkdir d4/d7/da/d3f/d76/d138 0 2024-03-22T07:16:14.843 INFO:tasks.workunit.client.1.smithi069.stdout:6/978: dread d4/d7/f6e [0,4194304] 0 2024-03-22T07:16:14.843 INFO:tasks.workunit.client.1.smithi069.stdout:6/979: getdents d4/d7/da 0 2024-03-22T07:16:14.844 INFO:tasks.workunit.client.1.smithi069.stdout:6/980: write d4/d7/f116 [637252,17223] 0 2024-03-22T07:16:14.852 INFO:tasks.workunit.client.1.smithi069.stdout:6/981: dread d4/f6b [4194304,4194304] 0 2024-03-22T07:16:14.859 INFO:tasks.workunit.client.1.smithi069.stdout:6/982: dread d4/d7/da/d111/d11f/f137 [0,4194304] 0 2024-03-22T07:16:14.860 INFO:tasks.workunit.client.1.smithi069.stdout:6/983: unlink d4/d7/f3e 0 2024-03-22T07:16:14.860 INFO:tasks.workunit.client.1.smithi069.stdout:6/984: mknod d4/d7/da/d3f/d76/dab/c139 0 2024-03-22T07:16:14.862 INFO:tasks.workunit.client.1.smithi069.stdout:6/985: creat d4/dd/d12/d13/f13a x:0 0 0 2024-03-22T07:16:14.862 INFO:tasks.workunit.client.1.smithi069.stdout:6/986: mknod d4/d7/d5b/c13b 0 2024-03-22T07:16:14.863 INFO:tasks.workunit.client.1.smithi069.stdout:6/987: creat d4/dd/d12/d2b/da5/db2/f13c x:0 0 0 2024-03-22T07:16:14.863 INFO:tasks.workunit.client.1.smithi069.stdout:6/988: stat d4/d7/da/d3f/d76/dab/ded 0 2024-03-22T07:16:14.864 INFO:tasks.workunit.client.1.smithi069.stdout:6/989: symlink d4/d133/l13d 0 2024-03-22T07:16:14.864 INFO:tasks.workunit.client.1.smithi069.stdout:6/990: chown d4/dd/lca 1723 1 2024-03-22T07:16:14.865 INFO:tasks.workunit.client.1.smithi069.stdout:6/991: read d4/dd/d12/d2b/da5/db2/fbb [684569,78332] 0 2024-03-22T07:16:14.874 INFO:tasks.workunit.client.1.smithi069.stdout:6/992: dread d4/fb [0,4194304] 0 2024-03-22T07:16:14.875 INFO:tasks.workunit.client.1.smithi069.stdout:6/993: mkdir d4/d7/da/d111/d78/d9a/d13e 0 2024-03-22T07:16:14.875 INFO:tasks.workunit.client.1.smithi069.stdout:6/994: mknod d4/d7/da/d111/d78/d9a/c13f 0 2024-03-22T07:16:14.875 INFO:tasks.workunit.client.1.smithi069.stdout:6/995: creat d4/dd/d12/d2b/f140 x:0 0 0 2024-03-22T07:16:14.877 INFO:tasks.workunit.client.1.smithi069.stdout:6/996: symlink d4/dd/d12/d2b/l141 0 2024-03-22T07:16:14.877 INFO:tasks.workunit.client.1.smithi069.stdout:6/997: chown d4/dd/d12/d106/c6a 6120124 1 2024-03-22T07:16:14.877 INFO:tasks.workunit.client.1.smithi069.stdout:6/998: dread - d4/dd/d12/d2b/f3c zero size 2024-03-22T07:16:14.877 INFO:tasks.workunit.client.1.smithi069.stdout:6/999: read d4/dd/d12/d13/de9/fb1 [238837,37517] 0 2024-03-22T07:16:14.879 INFO:tasks.workunit.client.1.smithi069.stderr:+ rm -rf -- ./tmp.XQHUhCuUpu 2024-03-22T07:16:15.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:15 smithi069 ceph-mon[36441]: pgmap v10: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 273 MiB/s rd, 292 MiB/s wr, 450 op/s 2024-03-22T07:16:15.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:15 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:15.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:15 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:15.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:15 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:16:15.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:15 smithi063 ceph-mon[29200]: pgmap v10: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 273 MiB/s rd, 292 MiB/s wr, 450 op/s 2024-03-22T07:16:15.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:15 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:15.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:15 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:15.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:15 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:16:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:17 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:17 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:17 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:16:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:17 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:16:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:17 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:16:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:17 smithi063 ceph-mon[29200]: Adjusting osd_memory_target on smithi063 to 1557M 2024-03-22T07:16:17.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:17 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:17.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:17 smithi063 ceph-mon[29200]: pgmap v11: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 278 MiB/s rd, 295 MiB/s wr, 461 op/s 2024-03-22T07:16:17.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:17 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:17.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:17 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:17.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:17 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:17.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:17 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:17.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:17 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:16:17.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:17 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:16:17.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:17 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:16:17.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:17 smithi069 ceph-mon[36441]: Adjusting osd_memory_target on smithi063 to 1557M 2024-03-22T07:16:17.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:17 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:17.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:17 smithi069 ceph-mon[36441]: pgmap v11: 65 pgs: 65 active+clean; 3.8 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 278 MiB/s rd, 295 MiB/s wr, 461 op/s 2024-03-22T07:16:17.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:17 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:17.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:17 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:18.834 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:18 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:18.834 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:18 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:19.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:18 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:19.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:18 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:19.901 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:19 smithi069 ceph-mon[36441]: pgmap v12: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 280 MiB/s rd, 300 MiB/s wr, 568 op/s 2024-03-22T07:16:20.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:19 smithi063 ceph-mon[29200]: pgmap v12: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 280 MiB/s rd, 300 MiB/s wr, 568 op/s 2024-03-22T07:16:20.433 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-22T07:16:20.433 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-22T07:16:22.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:21 smithi063 ceph-mon[29200]: pgmap v13: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 152 MiB/s rd, 161 MiB/s wr, 341 op/s 2024-03-22T07:16:22.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:21 smithi069 ceph-mon[36441]: pgmap v13: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 152 MiB/s rd, 161 MiB/s wr, 341 op/s 2024-03-22T07:16:23.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:23 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:23.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:23 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:23.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:23 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:16:23.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:23 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:16:23.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:23 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:16:23.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:23 smithi069 ceph-mon[36441]: Adjusting osd_memory_target on smithi069 to 2581M 2024-03-22T07:16:23.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:23 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:23.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:23 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:16:23.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:23 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:16:23.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:23 smithi069 ceph-mon[36441]: Updating smithi063:/etc/ceph/ceph.conf 2024-03-22T07:16:23.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:23 smithi069 ceph-mon[36441]: Updating smithi069:/etc/ceph/ceph.conf 2024-03-22T07:16:23.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:23 smithi069 ceph-mon[36441]: pgmap v14: 65 pgs: 65 active+clean; 1.2 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 153 MiB/s rd, 162 MiB/s wr, 486 op/s 2024-03-22T07:16:23.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:23 smithi069 ceph-mon[36441]: Updating smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:16:23.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:23 smithi069 ceph-mon[36441]: Updating smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:16:23.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:23 smithi069 ceph-mon[36441]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:16:23.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:23 smithi069 ceph-mon[36441]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:16:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:23 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:23 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:23 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:16:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:23 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:16:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:23 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-22T07:16:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:23 smithi063 ceph-mon[29200]: Adjusting osd_memory_target on smithi069 to 2581M 2024-03-22T07:16:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:23 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:23 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:16:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:23 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:16:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:23 smithi063 ceph-mon[29200]: Updating smithi063:/etc/ceph/ceph.conf 2024-03-22T07:16:23.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:23 smithi063 ceph-mon[29200]: Updating smithi069:/etc/ceph/ceph.conf 2024-03-22T07:16:23.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:23 smithi063 ceph-mon[29200]: pgmap v14: 65 pgs: 65 active+clean; 1.2 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 153 MiB/s rd, 162 MiB/s wr, 486 op/s 2024-03-22T07:16:23.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:23 smithi063 ceph-mon[29200]: Updating smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:16:23.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:23 smithi063 ceph-mon[29200]: Updating smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:16:23.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:23 smithi063 ceph-mon[29200]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:16:23.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:23 smithi063 ceph-mon[29200]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:16:24.529 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-22T07:16:24.530 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-22T07:16:24.537 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:24 smithi063 ceph-mon[29200]: Updating smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:16:24.537 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:24 smithi063 ceph-mon[29200]: Updating smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:16:24.537 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:24 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:24.537 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:24 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:24.537 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:24 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:24.537 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:24 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:24.537 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:24 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:24.537 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:24 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:24.537 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:24 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:24.538 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:24 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:24.538 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:24 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:24.538 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:24 smithi063 ceph-mon[29200]: Reconfiguring node-exporter.smithi063 (dependencies changed)... 2024-03-22T07:16:24.538 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:24 smithi063 ceph-mon[29200]: Reconfiguring daemon node-exporter.smithi063 on smithi063 2024-03-22T07:16:24.596 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:24 smithi069 ceph-mon[36441]: Updating smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:16:24.596 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:24 smithi069 ceph-mon[36441]: Updating smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:16:24.596 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:24 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:24.596 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:24 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:24.596 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:24 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:24.596 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:24 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:24.596 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:24 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:24.596 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:24 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:24.596 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:24 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:24.597 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:24 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:24.597 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:24 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:24.597 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:24 smithi069 ceph-mon[36441]: Reconfiguring node-exporter.smithi063 (dependencies changed)... 2024-03-22T07:16:24.597 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:24 smithi069 ceph-mon[36441]: Reconfiguring daemon node-exporter.smithi063 on smithi063 2024-03-22T07:16:25.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:25 smithi063 ceph-mon[29200]: pgmap v15: 65 pgs: 65 active+clean; 1.2 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 31 MiB/s rd, 41 MiB/s wr, 286 op/s 2024-03-22T07:16:25.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:25 smithi069 ceph-mon[36441]: pgmap v15: 65 pgs: 65 active+clean; 1.2 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 31 MiB/s rd, 41 MiB/s wr, 286 op/s 2024-03-22T07:16:27.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:27 smithi063 ceph-mon[29200]: pgmap v16: 65 pgs: 65 active+clean; 900 MiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 32 MiB/s rd, 41 MiB/s wr, 358 op/s 2024-03-22T07:16:27.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:27 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:27.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:27 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:27.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:27 smithi063 ceph-mon[29200]: Reconfiguring alertmanager.smithi063 (dependencies changed)... 2024-03-22T07:16:27.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:27 smithi063 ceph-mon[29200]: Reconfiguring daemon alertmanager.smithi063 on smithi063 2024-03-22T07:16:28.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:27 smithi069 ceph-mon[36441]: pgmap v16: 65 pgs: 65 active+clean; 900 MiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 32 MiB/s rd, 41 MiB/s wr, 358 op/s 2024-03-22T07:16:28.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:27 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:28.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:27 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:28.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:27 smithi069 ceph-mon[36441]: Reconfiguring alertmanager.smithi063 (dependencies changed)... 2024-03-22T07:16:28.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:27 smithi069 ceph-mon[36441]: Reconfiguring daemon alertmanager.smithi063 on smithi063 2024-03-22T07:16:29.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:29 smithi063 ceph-mon[29200]: pgmap v17: 65 pgs: 65 active+clean; 292 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 28 MiB/s rd, 38 MiB/s wr, 469 op/s 2024-03-22T07:16:30.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:29 smithi069 ceph-mon[36441]: pgmap v17: 65 pgs: 65 active+clean; 292 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 28 MiB/s rd, 38 MiB/s wr, 469 op/s 2024-03-22T07:16:30.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.062+0000 7f86884ba700 1 -- 172.21.15.63:0/624861596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8680066af0 msgr2=0x7f8680101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:30.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.062+0000 7f86884ba700 1 --2- 172.21.15.63:0/624861596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8680066af0 0x7f8680101e50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f867c004940 tx=0x7f867c009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:30.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.062+0000 7f86884ba700 1 -- 172.21.15.63:0/624861596 shutdown_connections 2024-03-22T07:16:30.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.062+0000 7f86884ba700 1 --2- 172.21.15.63:0/624861596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8680102390 0x7f8680104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:30.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.062+0000 7f86884ba700 1 --2- 172.21.15.63:0/624861596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8680066af0 0x7f8680101e50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:30.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.062+0000 7f86884ba700 1 -- 172.21.15.63:0/624861596 >> 172.21.15.63:0/624861596 conn(0x7f86800f7480 msgr2=0x7f86800f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:30.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.062+0000 7f86884ba700 1 -- 172.21.15.63:0/624861596 shutdown_connections 2024-03-22T07:16:30.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.062+0000 7f86884ba700 1 -- 172.21.15.63:0/624861596 wait complete. 2024-03-22T07:16:30.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.063+0000 7f86884ba700 1 Processor -- start 2024-03-22T07:16:30.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.063+0000 7f86884ba700 1 -- start start 2024-03-22T07:16:30.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.063+0000 7f86884ba700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8680066af0 0x7f868010fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:30.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.063+0000 7f86884ba700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8680102390 0x7f8680110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:30.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.063+0000 7f86884ba700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8680110770 con 0x7f8680102390 2024-03-22T07:16:30.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.063+0000 7f86884ba700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8680110fc0 con 0x7f8680066af0 2024-03-22T07:16:30.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.063+0000 7f8685a55700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8680102390 0x7f8680110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:30.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.063+0000 7f8685a55700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8680102390 0x7f8680110170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56384/0 (socket says 172.21.15.63:56384) 2024-03-22T07:16:30.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.063+0000 7f8685a55700 1 -- 172.21.15.63:0/2804756935 learned_addr learned my addr 172.21.15.63:0/2804756935 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:16:30.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.063+0000 7f8686256700 1 --2- 172.21.15.63:0/2804756935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8680066af0 0x7f868010fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:30.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.063+0000 7f8685a55700 1 -- 172.21.15.63:0/2804756935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8680066af0 msgr2=0x7f868010fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:30.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.063+0000 7f8685a55700 1 --2- 172.21.15.63:0/2804756935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8680066af0 0x7f868010fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:30.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.063+0000 7f8685a55700 1 -- 172.21.15.63:0/2804756935 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c005040 con 0x7f8680102390 2024-03-22T07:16:30.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.064+0000 7f8685a55700 1 --2- 172.21.15.63:0/2804756935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8680102390 0x7f8680110170 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8670002a80 tx=0x7f86700046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:30.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.064+0000 7f8676ffd700 1 -- 172.21.15.63:0/2804756935 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8670007cc0 con 0x7f8680102390 2024-03-22T07:16:30.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.064+0000 7f8676ffd700 1 -- 172.21.15.63:0/2804756935 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f867000f040 con 0x7f8680102390 2024-03-22T07:16:30.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.064+0000 7f86884ba700 1 -- 172.21.15.63:0/2804756935 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86801112a0 con 0x7f8680102390 2024-03-22T07:16:30.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.064+0000 7f86884ba700 1 -- 172.21.15.63:0/2804756935 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86801117e0 con 0x7f8680102390 2024-03-22T07:16:30.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.064+0000 7f8676ffd700 1 -- 172.21.15.63:0/2804756935 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f867000dac0 con 0x7f8680102390 2024-03-22T07:16:30.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.065+0000 7f86884ba700 1 -- 172.21.15.63:0/2804756935 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86640052e0 con 0x7f8680102390 2024-03-22T07:16:30.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.067+0000 7f8676ffd700 1 -- 172.21.15.63:0/2804756935 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) v1 ==== 88853+0+0 (secure 0 0 0) 0x7f867000dc20 con 0x7f8680102390 2024-03-22T07:16:30.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.067+0000 7f8676ffd700 1 --2- 172.21.15.63:0/2804756935 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f866c06aeb0 0x7f866c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:30.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.067+0000 7f8676ffd700 1 -- 172.21.15.63:0/2804756935 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f8670089c70 con 0x7f8680102390 2024-03-22T07:16:30.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.067+0000 7f8686256700 1 --2- 172.21.15.63:0/2804756935 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f866c06aeb0 0x7f866c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:30.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.068+0000 7f8686256700 1 --2- 172.21.15.63:0/2804756935 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f866c06aeb0 0x7f866c06d360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f867c00b3b0 tx=0x7f867c009e30 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:30.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.075+0000 7f8676ffd700 1 -- 172.21.15.63:0/2804756935 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f8670057ab0 con 0x7f8680102390 2024-03-22T07:16:30.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.428+0000 7f86884ba700 1 -- 172.21.15.63:0/2804756935 --> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8664000bf0 con 0x7f866c06aeb0 2024-03-22T07:16:30.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.430+0000 7f8676ffd700 1 -- 172.21.15.63:0/2804756935 <== mgr.24385 v2:172.21.15.69:6828/1862357042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f8664000bf0 con 0x7f866c06aeb0 2024-03-22T07:16:30.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.435+0000 7f86884ba700 1 -- 172.21.15.63:0/2804756935 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f866c06aeb0 msgr2=0x7f866c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:30.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.435+0000 7f86884ba700 1 --2- 172.21.15.63:0/2804756935 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f866c06aeb0 0x7f866c06d360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f867c00b3b0 tx=0x7f867c009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:30.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.435+0000 7f86884ba700 1 -- 172.21.15.63:0/2804756935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8680102390 msgr2=0x7f8680110170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:30.435 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.435+0000 7f86884ba700 1 --2- 172.21.15.63:0/2804756935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8680102390 0x7f8680110170 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8670002a80 tx=0x7f86700046b0 comp rx=0 tx=0).stop 2024-03-22T07:16:30.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.436+0000 7f86884ba700 1 -- 172.21.15.63:0/2804756935 shutdown_connections 2024-03-22T07:16:30.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.436+0000 7f86884ba700 1 --2- 172.21.15.63:0/2804756935 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f866c06aeb0 0x7f866c06d360 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:30.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.436+0000 7f86884ba700 1 --2- 172.21.15.63:0/2804756935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8680066af0 0x7f868010fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:30.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.436+0000 7f86884ba700 1 --2- 172.21.15.63:0/2804756935 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8680102390 0x7f8680110170 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:30.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.436+0000 7f86884ba700 1 -- 172.21.15.63:0/2804756935 >> 172.21.15.63:0/2804756935 conn(0x7f86800f7480 msgr2=0x7f86800f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:30.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.438+0000 7f86884ba700 1 -- 172.21.15.63:0/2804756935 shutdown_connections 2024-03-22T07:16:30.438 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.438+0000 7f86884ba700 1 -- 172.21.15.63:0/2804756935 wait complete. 2024-03-22T07:16:30.458 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:16:30.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.615+0000 7f21d9197700 1 -- 172.21.15.63:0/3712908325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21d40fb440 msgr2=0x7f21d40fb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:30.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.615+0000 7f21d9197700 1 --2- 172.21.15.63:0/3712908325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21d40fb440 0x7f21d40fb850 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f21c4004940 tx=0x7f21c4009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:30.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.615+0000 7f21d9197700 1 -- 172.21.15.63:0/3712908325 shutdown_connections 2024-03-22T07:16:30.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.615+0000 7f21d9197700 1 --2- 172.21.15.63:0/3712908325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21d40fc6f0 0x7f21d40fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:30.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.615+0000 7f21d9197700 1 --2- 172.21.15.63:0/3712908325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21d40fb440 0x7f21d40fb850 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:30.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.615+0000 7f21d9197700 1 -- 172.21.15.63:0/3712908325 >> 172.21.15.63:0/3712908325 conn(0x7f21d40f6ba0 msgr2=0x7f21d40f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:30.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.616+0000 7f21d9197700 1 -- 172.21.15.63:0/3712908325 shutdown_connections 2024-03-22T07:16:30.616 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.616+0000 7f21d9197700 1 -- 172.21.15.63:0/3712908325 wait complete. 2024-03-22T07:16:30.618 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.616+0000 7f21d9197700 1 Processor -- start 2024-03-22T07:16:30.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.616+0000 7f21d9197700 1 -- start start 2024-03-22T07:16:30.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.617+0000 7f21d9197700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21d40fb440 0x7f21d410d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:30.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.617+0000 7f21d9197700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21d40fc6f0 0x7f21d410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:30.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.617+0000 7f21d9197700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d410e4e0 con 0x7f21d40fb440 2024-03-22T07:16:30.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.617+0000 7f21d9197700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d410ed30 con 0x7f21d40fc6f0 2024-03-22T07:16:30.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.617+0000 7f21d2d9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21d40fb440 0x7f21d410d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:30.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.617+0000 7f21d259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21d40fc6f0 0x7f21d410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:30.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.617+0000 7f21d259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21d40fc6f0 0x7f21d410dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.63:35706/0 (socket says 172.21.15.63:35706) 2024-03-22T07:16:30.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.617+0000 7f21d259c700 1 -- 172.21.15.63:0/3892865278 learned_addr learned my addr 172.21.15.63:0/3892865278 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:16:30.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.618+0000 7f21d259c700 1 -- 172.21.15.63:0/3892865278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21d40fb440 msgr2=0x7f21d410d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:30.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.618+0000 7f21d259c700 1 --2- 172.21.15.63:0/3892865278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21d40fb440 0x7f21d410d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:30.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.618+0000 7f21d259c700 1 -- 172.21.15.63:0/3892865278 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c4005040 con 0x7f21d40fc6f0 2024-03-22T07:16:30.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.618+0000 7f21d259c700 1 --2- 172.21.15.63:0/3892865278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21d40fc6f0 0x7f21d410dee0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f21c800eb40 tx=0x7f21c800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:30.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.618+0000 7f21bb7fe700 1 -- 172.21.15.63:0/3892865278 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21c8007cc0 con 0x7f21d40fc6f0 2024-03-22T07:16:30.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.618+0000 7f21bb7fe700 1 -- 172.21.15.63:0/3892865278 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21c800f040 con 0x7f21d40fc6f0 2024-03-22T07:16:30.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.618+0000 7f21bb7fe700 1 -- 172.21.15.63:0/3892865278 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21c800ca20 con 0x7f21d40fc6f0 2024-03-22T07:16:30.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.618+0000 7f21d9197700 1 -- 172.21.15.63:0/3892865278 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d410efb0 con 0x7f21d40fc6f0 2024-03-22T07:16:30.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.618+0000 7f21d9197700 1 -- 172.21.15.63:0/3892865278 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d410f4f0 con 0x7f21d40fc6f0 2024-03-22T07:16:30.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.619+0000 7f21d9197700 1 -- 172.21.15.63:0/3892865278 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d405f5f0 con 0x7f21d40fc6f0 2024-03-22T07:16:30.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.620+0000 7f21bb7fe700 1 -- 172.21.15.63:0/3892865278 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) v1 ==== 88853+0+0 (secure 0 0 0) 0x7f21c800cb80 con 0x7f21d40fc6f0 2024-03-22T07:16:30.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.620+0000 7f21bb7fe700 1 --2- 172.21.15.63:0/3892865278 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f21bc06b180 0x7f21bc06d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:30.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.620+0000 7f21bb7fe700 1 -- 172.21.15.63:0/3892865278 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f21c80896a0 con 0x7f21d40fc6f0 2024-03-22T07:16:30.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.620+0000 7f21d2d9d700 1 --2- 172.21.15.63:0/3892865278 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f21bc06b180 0x7f21bc06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:30.621 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.621+0000 7f21d2d9d700 1 --2- 172.21.15.63:0/3892865278 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f21bc06b180 0x7f21bc06d630 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f21c400b0d0 tx=0x7f21c4009e30 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:30.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.625+0000 7f21bb7fe700 1 -- 172.21.15.63:0/3892865278 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f21c804f620 con 0x7f21d40fc6f0 2024-03-22T07:16:30.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:30 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:16:30.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.890+0000 7f21d9197700 1 -- 172.21.15.63:0/3892865278 --> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f21d410f760 con 0x7f21bc06b180 2024-03-22T07:16:30.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.891+0000 7f21bb7fe700 1 -- 172.21.15.63:0/3892865278 <== mgr.24385 v2:172.21.15.69:6828/1862357042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f21d410f760 con 0x7f21bc06b180 2024-03-22T07:16:30.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.894+0000 7f21d9197700 1 -- 172.21.15.63:0/3892865278 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f21bc06b180 msgr2=0x7f21bc06d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:30.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.894+0000 7f21d9197700 1 --2- 172.21.15.63:0/3892865278 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f21bc06b180 0x7f21bc06d630 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f21c400b0d0 tx=0x7f21c4009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:30.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.894+0000 7f21d9197700 1 -- 172.21.15.63:0/3892865278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21d40fc6f0 msgr2=0x7f21d410dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:30.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.894+0000 7f21d9197700 1 --2- 172.21.15.63:0/3892865278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21d40fc6f0 0x7f21d410dee0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f21c800eb40 tx=0x7f21c800b040 comp rx=0 tx=0).stop 2024-03-22T07:16:30.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.894+0000 7f21d9197700 1 -- 172.21.15.63:0/3892865278 shutdown_connections 2024-03-22T07:16:30.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.894+0000 7f21d9197700 1 --2- 172.21.15.63:0/3892865278 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f21bc06b180 0x7f21bc06d630 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:30.894 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.894+0000 7f21d9197700 1 --2- 172.21.15.63:0/3892865278 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f21d40fb440 0x7f21d410d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:30.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.894+0000 7f21d9197700 1 --2- 172.21.15.63:0/3892865278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21d40fc6f0 0x7f21d410dee0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:30.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.894+0000 7f21d9197700 1 -- 172.21.15.63:0/3892865278 >> 172.21.15.63:0/3892865278 conn(0x7f21d40f6ba0 msgr2=0x7f21d40f81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:30.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.894+0000 7f21d9197700 1 -- 172.21.15.63:0/3892865278 shutdown_connections 2024-03-22T07:16:30.895 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:30.894+0000 7f21d9197700 1 -- 172.21.15.63:0/3892865278 wait complete. 2024-03-22T07:16:31.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.005+0000 7f049e00b700 1 -- 172.21.15.63:0/2950091751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04980fcc00 msgr2=0x7f04980fd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:31.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.005+0000 7f049e00b700 1 --2- 172.21.15.63:0/2950091751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04980fcc00 0x7f04980fd070 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f048c004940 tx=0x7f048c009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:31.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.005+0000 7f049e00b700 1 -- 172.21.15.63:0/2950091751 shutdown_connections 2024-03-22T07:16:31.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.005+0000 7f049e00b700 1 --2- 172.21.15.63:0/2950091751 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04980fcc00 0x7f04980fd070 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:31.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.005+0000 7f049e00b700 1 --2- 172.21.15.63:0/2950091751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04980fc2b0 0x7f04980fc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:31.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.005+0000 7f049e00b700 1 -- 172.21.15.63:0/2950091751 >> 172.21.15.63:0/2950091751 conn(0x7f04980f6ba0 msgr2=0x7f04980f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:31.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.006+0000 7f049e00b700 1 -- 172.21.15.63:0/2950091751 shutdown_connections 2024-03-22T07:16:31.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.006+0000 7f049e00b700 1 -- 172.21.15.63:0/2950091751 wait complete. 2024-03-22T07:16:31.006 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.006+0000 7f049e00b700 1 Processor -- start 2024-03-22T07:16:31.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.006+0000 7f049e00b700 1 -- start start 2024-03-22T07:16:31.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.006+0000 7f049e00b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04980fc2b0 0x7f049810d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:31.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.006+0000 7f049e00b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04980fcc00 0x7f049810dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:31.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.006+0000 7f049e00b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049810e4e0 con 0x7f04980fcc00 2024-03-22T07:16:31.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.006+0000 7f049e00b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049810ed30 con 0x7f04980fc2b0 2024-03-22T07:16:31.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.007+0000 7f0496ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04980fcc00 0x7f049810dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:31.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.007+0000 7f04977fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04980fc2b0 0x7f049810d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:31.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.007+0000 7f0496ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04980fcc00 0x7f049810dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56412/0 (socket says 172.21.15.63:56412) 2024-03-22T07:16:31.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.007+0000 7f04977fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04980fc2b0 0x7f049810d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.63:35716/0 (socket says 172.21.15.63:35716) 2024-03-22T07:16:31.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.007+0000 7f0496ffd700 1 -- 172.21.15.63:0/3667056385 learned_addr learned my addr 172.21.15.63:0/3667056385 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:16:31.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.007+0000 7f04977fe700 1 -- 172.21.15.63:0/3667056385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04980fcc00 msgr2=0x7f049810dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:31.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.007+0000 7f04977fe700 1 --2- 172.21.15.63:0/3667056385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04980fcc00 0x7f049810dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:31.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.007+0000 7f04977fe700 1 -- 172.21.15.63:0/3667056385 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c005040 con 0x7f04980fc2b0 2024-03-22T07:16:31.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.007+0000 7f04977fe700 1 --2- 172.21.15.63:0/3667056385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04980fc2b0 0x7f049810d9a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0498065360 tx=0x7f048800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:31.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.007+0000 7f047ffff700 1 -- 172.21.15.63:0/3667056385 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0488004e90 con 0x7f04980fc2b0 2024-03-22T07:16:31.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.007+0000 7f047ffff700 1 -- 172.21.15.63:0/3667056385 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f048800f040 con 0x7f04980fc2b0 2024-03-22T07:16:31.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.007+0000 7f049e00b700 1 -- 172.21.15.63:0/3667056385 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049810efb0 con 0x7f04980fc2b0 2024-03-22T07:16:31.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.007+0000 7f047ffff700 1 -- 172.21.15.63:0/3667056385 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f048800c980 con 0x7f04980fc2b0 2024-03-22T07:16:31.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.007+0000 7f049e00b700 1 -- 172.21.15.63:0/3667056385 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049810f4f0 con 0x7f04980fc2b0 2024-03-22T07:16:31.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.008+0000 7f049e00b700 1 -- 172.21.15.63:0/3667056385 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0498101c00 con 0x7f04980fc2b0 2024-03-22T07:16:31.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.009+0000 7f047ffff700 1 -- 172.21.15.63:0/3667056385 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) v1 ==== 88853+0+0 (secure 0 0 0) 0x7f0488007cc0 con 0x7f04980fc2b0 2024-03-22T07:16:31.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.009+0000 7f047ffff700 1 --2- 172.21.15.63:0/3667056385 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f048006b0b0 0x7f048006d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:31.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.009+0000 7f047ffff700 1 -- 172.21.15.63:0/3667056385 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f04880893b0 con 0x7f04980fc2b0 2024-03-22T07:16:31.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.009+0000 7f0496ffd700 1 --2- 172.21.15.63:0/3667056385 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f048006b0b0 0x7f048006d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:31.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.010+0000 7f0496ffd700 1 --2- 172.21.15.63:0/3667056385 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f048006b0b0 0x7f048006d560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f048c013740 tx=0x7f048c009e30 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:31.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.014+0000 7f047ffff700 1 -- 172.21.15.63:0/3667056385 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f048804f620 con 0x7f04980fc2b0 2024-03-22T07:16:31.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:30 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:16:31.286 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.287+0000 7f049e00b700 1 -- 172.21.15.63:0/3667056385 --> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0498104910 con 0x7f048006b0b0 2024-03-22T07:16:31.297 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.297+0000 7f047ffff700 1 -- 172.21.15.63:0/3667056385 <== mgr.24385 v2:172.21.15.69:6828/1862357042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f0498104910 con 0x7f048006b0b0 2024-03-22T07:16:31.297 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:16:31.297 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (7m) 16s ago 8m 24.2M - 0.25.0 c8568f914cd2 2f8544438b7c 2024-03-22T07:16:31.297 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (8m) 16s ago 8m 6660k - 17.2.7-726-g9c4e0c34 837124da804f d6c6bbeb6904 2024-03-22T07:16:31.297 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (7m) 14s ago 7m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 30992f8aa88c 2024-03-22T07:16:31.297 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (6m) 16s ago 8m 85.3M - 9.4.7 954c08fa6188 f0b1d2a54439 2024-03-22T07:16:31.297 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (2m) 16s ago 2m 90.3M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:16:31.297 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (2m) 16s ago 2m 152M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:16:31.297 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (2m) 14s ago 2m 78.8M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:16:31.298 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (2m) 14s ago 2m 135M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:16:31.298 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:9283 running (9m) 16s ago 9m 393M - 17.2.7-726-g9c4e0c34 837124da804f 422b5e836153 2024-03-22T07:16:31.298 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283 running (66s) 14s ago 7m 491M - 19.0.0-1667-gdb0330b1 4f325f8440e9 5ad05e49e512 2024-03-22T07:16:31.298 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (9m) 16s ago 9m 51.4M 2048M 17.2.7-726-g9c4e0c34 837124da804f f92a5bb6caee 2024-03-22T07:16:31.298 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (7m) 14s ago 7m 38.1M 2048M 17.2.7-726-g9c4e0c34 837124da804f 95f6103f7603 2024-03-22T07:16:31.298 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 starting - - - - 2024-03-22T07:16:31.298 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (7m) 14s ago 7m 16.4M - 1.5.0 0da6a335fe13 49f4ca105fb9 2024-03-22T07:16:31.298 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (6m) 16s ago 6m 251M 1557M 17.2.7-726-g9c4e0c34 837124da804f 580f22188a10 2024-03-22T07:16:31.298 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (6m) 16s ago 6m 294M 1557M 17.2.7-726-g9c4e0c34 837124da804f fe1ace35c88a 2024-03-22T07:16:31.298 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (5m) 16s ago 5m 223M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4cd995265dcf 2024-03-22T07:16:31.298 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (5m) 14s ago 5m 298M 2581M 17.2.7-726-g9c4e0c34 837124da804f 45c9a759b76d 2024-03-22T07:16:31.298 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (4m) 14s ago 4m 265M 2581M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:16:31.298 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (4m) 14s ago 4m 299M 2581M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:16:31.298 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (6m) 16s ago 7m 56.4M - 2.43.0 a07b618ecd1d 827638a5870a 2024-03-22T07:16:31.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.299+0000 7f049e00b700 1 -- 172.21.15.63:0/3667056385 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f048006b0b0 msgr2=0x7f048006d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:31.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.299+0000 7f049e00b700 1 --2- 172.21.15.63:0/3667056385 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f048006b0b0 0x7f048006d560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f048c013740 tx=0x7f048c009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:31.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.299+0000 7f049e00b700 1 -- 172.21.15.63:0/3667056385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04980fc2b0 msgr2=0x7f049810d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:31.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.299+0000 7f049e00b700 1 --2- 172.21.15.63:0/3667056385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04980fc2b0 0x7f049810d9a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0498065360 tx=0x7f048800b040 comp rx=0 tx=0).stop 2024-03-22T07:16:31.299 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.299+0000 7f049e00b700 1 -- 172.21.15.63:0/3667056385 shutdown_connections 2024-03-22T07:16:31.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.299+0000 7f049e00b700 1 --2- 172.21.15.63:0/3667056385 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f048006b0b0 0x7f048006d560 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:31.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.299+0000 7f049e00b700 1 --2- 172.21.15.63:0/3667056385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04980fc2b0 0x7f049810d9a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:31.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.299+0000 7f049e00b700 1 --2- 172.21.15.63:0/3667056385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f04980fcc00 0x7f049810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:31.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.299+0000 7f049e00b700 1 -- 172.21.15.63:0/3667056385 >> 172.21.15.63:0/3667056385 conn(0x7f04980f6ba0 msgr2=0x7f04980f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:31.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.299+0000 7f049e00b700 1 -- 172.21.15.63:0/3667056385 shutdown_connections 2024-03-22T07:16:31.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.299+0000 7f049e00b700 1 -- 172.21.15.63:0/3667056385 wait complete. 2024-03-22T07:16:31.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.404+0000 7f00d7fe3700 1 -- 172.21.15.63:0/3845788653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d00fff00 msgr2=0x7f00d0102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:31.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.404+0000 7f00d7fe3700 1 --2- 172.21.15.63:0/3845788653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d00fff00 0x7f00d0102320 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f00c0004940 tx=0x7f00c0009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:31.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.405+0000 7f00d7fe3700 1 -- 172.21.15.63:0/3845788653 shutdown_connections 2024-03-22T07:16:31.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.405+0000 7f00d7fe3700 1 --2- 172.21.15.63:0/3845788653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d00fff00 0x7f00d0102320 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:31.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.405+0000 7f00d7fe3700 1 --2- 172.21.15.63:0/3845788653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00d00662e0 0x7f00d00ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:31.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.405+0000 7f00d7fe3700 1 -- 172.21.15.63:0/3845788653 >> 172.21.15.63:0/3845788653 conn(0x7f00d00f7430 msgr2=0x7f00d00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:31.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.406+0000 7f00d7fe3700 1 -- 172.21.15.63:0/3845788653 shutdown_connections 2024-03-22T07:16:31.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.406+0000 7f00d7fe3700 1 -- 172.21.15.63:0/3845788653 wait complete. 2024-03-22T07:16:31.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.407+0000 7f00d7fe3700 1 Processor -- start 2024-03-22T07:16:31.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.407+0000 7f00d7fe3700 1 -- start start 2024-03-22T07:16:31.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.407+0000 7f00d7fe3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d00662e0 0x7f00d00fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:31.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.407+0000 7f00d7fe3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00d00fff00 0x7f00d00fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:31.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.407+0000 7f00d7fe3700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d01116b0 con 0x7f00d00662e0 2024-03-22T07:16:31.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.407+0000 7f00d7fe3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d01117f0 con 0x7f00d00fff00 2024-03-22T07:16:31.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.407+0000 7f00d5d7f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d00662e0 0x7f00d00fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:31.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.407+0000 7f00d5d7f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d00662e0 0x7f00d00fd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56434/0 (socket says 172.21.15.63:56434) 2024-03-22T07:16:31.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.407+0000 7f00d5d7f700 1 -- 172.21.15.63:0/1513803701 learned_addr learned my addr 172.21.15.63:0/1513803701 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:16:31.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.408+0000 7f00d5d7f700 1 -- 172.21.15.63:0/1513803701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00d00fff00 msgr2=0x7f00d00fb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:31.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.408+0000 7f00d557e700 1 --2- 172.21.15.63:0/1513803701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00d00fff00 0x7f00d00fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:31.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.408+0000 7f00d5d7f700 1 --2- 172.21.15.63:0/1513803701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00d00fff00 0x7f00d00fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:31.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.408+0000 7f00d5d7f700 1 -- 172.21.15.63:0/1513803701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c0005040 con 0x7f00d00662e0 2024-03-22T07:16:31.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.408+0000 7f00d5d7f700 1 --2- 172.21.15.63:0/1513803701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d00662e0 0x7f00d00fd3a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00eb40 tx=0x7f00cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:31.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.409+0000 7f00c67fc700 1 -- 172.21.15.63:0/1513803701 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00cc007cc0 con 0x7f00d00662e0 2024-03-22T07:16:31.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.409+0000 7f00c67fc700 1 -- 172.21.15.63:0/1513803701 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00cc00f040 con 0x7f00d00662e0 2024-03-22T07:16:31.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.409+0000 7f00c67fc700 1 -- 172.21.15.63:0/1513803701 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00cc0095e0 con 0x7f00d00662e0 2024-03-22T07:16:31.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.410+0000 7f00d7fe3700 1 -- 172.21.15.63:0/1513803701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d00fbe60 con 0x7f00d00662e0 2024-03-22T07:16:31.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.410+0000 7f00d7fe3700 1 -- 172.21.15.63:0/1513803701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d00fc370 con 0x7f00d00662e0 2024-03-22T07:16:31.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.412+0000 7f00d7fe3700 1 -- 172.21.15.63:0/1513803701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d005f5f0 con 0x7f00d00662e0 2024-03-22T07:16:31.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.413+0000 7f00c67fc700 1 -- 172.21.15.63:0/1513803701 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) v1 ==== 88853+0+0 (secure 0 0 0) 0x7f00cc009740 con 0x7f00d00662e0 2024-03-22T07:16:31.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.414+0000 7f00c67fc700 1 --2- 172.21.15.63:0/1513803701 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f00bc06b250 0x7f00bc06d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:31.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.414+0000 7f00c67fc700 1 -- 172.21.15.63:0/1513803701 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f00cc089300 con 0x7f00d00662e0 2024-03-22T07:16:31.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.415+0000 7f00d557e700 1 --2- 172.21.15.63:0/1513803701 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f00bc06b250 0x7f00bc06d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:31.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.416+0000 7f00d557e700 1 --2- 172.21.15.63:0/1513803701 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f00bc06b250 0x7f00bc06d700 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f00c00043d0 tx=0x7f00c0009e30 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:31.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.423+0000 7f00c67fc700 1 -- 172.21.15.63:0/1513803701 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f00cc053550 con 0x7f00d00662e0 2024-03-22T07:16:31.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.777+0000 7f00d7fe3700 1 -- 172.21.15.63:0/1513803701 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f00d00fc6f0 con 0x7f00d00662e0 2024-03-22T07:16:31.777 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.778+0000 7f00c67fc700 1 -- 172.21.15.63:0/1513803701 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f00cc056c10 con 0x7f00d00662e0 2024-03-22T07:16:31.778 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:16:31.778 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:16:31.778 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:16:31.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:16:31.778 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:16:31.778 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T07:16:31.778 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-22T07:16:31.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:16:31.778 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:16:31.778 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:16:31.778 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:16:31.778 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:16:31.779 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:16:31.779 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:16:31.779 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:16:31.779 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 13, 2024-03-22T07:16:31.779 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-22T07:16:31.779 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:16:31.779 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:16:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.781+0000 7f00d7fe3700 1 -- 172.21.15.63:0/1513803701 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f00bc06b250 msgr2=0x7f00bc06d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.781+0000 7f00d7fe3700 1 --2- 172.21.15.63:0/1513803701 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f00bc06b250 0x7f00bc06d700 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f00c00043d0 tx=0x7f00c0009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.781+0000 7f00d7fe3700 1 -- 172.21.15.63:0/1513803701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d00662e0 msgr2=0x7f00d00fd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.781+0000 7f00d7fe3700 1 --2- 172.21.15.63:0/1513803701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d00662e0 0x7f00d00fd3a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00eb40 tx=0x7f00cc00b040 comp rx=0 tx=0).stop 2024-03-22T07:16:31.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.781+0000 7f00d7fe3700 1 -- 172.21.15.63:0/1513803701 shutdown_connections 2024-03-22T07:16:31.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.781+0000 7f00d7fe3700 1 --2- 172.21.15.63:0/1513803701 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f00bc06b250 0x7f00bc06d700 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:31.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.781+0000 7f00d7fe3700 1 --2- 172.21.15.63:0/1513803701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00d00662e0 0x7f00d00fd3a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:31.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.781+0000 7f00d7fe3700 1 --2- 172.21.15.63:0/1513803701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00d00fff00 0x7f00d00fb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:31.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.781+0000 7f00d7fe3700 1 -- 172.21.15.63:0/1513803701 >> 172.21.15.63:0/1513803701 conn(0x7f00d00f7430 msgr2=0x7f00d00f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:31.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.781+0000 7f00d7fe3700 1 -- 172.21.15.63:0/1513803701 shutdown_connections 2024-03-22T07:16:31.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.781+0000 7f00d7fe3700 1 -- 172.21.15.63:0/1513803701 wait complete. 2024-03-22T07:16:31.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:31 smithi063 ceph-mon[29200]: pgmap v18: 65 pgs: 65 active+clean; 292 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 339 op/s 2024-03-22T07:16:31.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:31 smithi063 ceph-mon[29200]: from='client.14630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:31.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.904+0000 7f93438c5700 1 -- 172.21.15.63:0/2352710261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f933c0fce40 msgr2=0x7f933c0fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:31.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.904+0000 7f93438c5700 1 --2- 172.21.15.63:0/2352710261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f933c0fce40 0x7f933c0fd250 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9338004940 tx=0x7f9338009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:31.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.904+0000 7f93438c5700 1 -- 172.21.15.63:0/2352710261 shutdown_connections 2024-03-22T07:16:31.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.904+0000 7f93438c5700 1 --2- 172.21.15.63:0/2352710261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f933c0fe040 0x7f933c0fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:31.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.904+0000 7f93438c5700 1 --2- 172.21.15.63:0/2352710261 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f933c0fce40 0x7f933c0fd250 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:31.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.904+0000 7f93438c5700 1 -- 172.21.15.63:0/2352710261 >> 172.21.15.63:0/2352710261 conn(0x7f933c0f8550 msgr2=0x7f933c0fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:31.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.904+0000 7f93438c5700 1 -- 172.21.15.63:0/2352710261 shutdown_connections 2024-03-22T07:16:31.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.904+0000 7f93438c5700 1 -- 172.21.15.63:0/2352710261 wait complete. 2024-03-22T07:16:31.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.905+0000 7f93438c5700 1 Processor -- start 2024-03-22T07:16:31.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.905+0000 7f93438c5700 1 -- start start 2024-03-22T07:16:31.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.905+0000 7f93438c5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f933c0fce40 0x7f933c10da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:31.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.905+0000 7f93438c5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f933c0fe040 0x7f933c10df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:31.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.905+0000 7f93438c5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c10e580 con 0x7f933c0fce40 2024-03-22T07:16:31.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.905+0000 7f93438c5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c10edd0 con 0x7f933c0fe040 2024-03-22T07:16:31.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.905+0000 7f9341661700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f933c0fce40 0x7f933c10da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:31.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.905+0000 7f9341661700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f933c0fce40 0x7f933c10da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:56450/0 (socket says 172.21.15.63:56450) 2024-03-22T07:16:31.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.905+0000 7f9341661700 1 -- 172.21.15.63:0/463837584 learned_addr learned my addr 172.21.15.63:0/463837584 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:16:31.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.905+0000 7f9341661700 1 -- 172.21.15.63:0/463837584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f933c0fe040 msgr2=0x7f933c10df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:31.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.905+0000 7f9340e60700 1 --2- 172.21.15.63:0/463837584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f933c0fe040 0x7f933c10df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:31.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.905+0000 7f9341661700 1 --2- 172.21.15.63:0/463837584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f933c0fe040 0x7f933c10df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:31.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.905+0000 7f9341661700 1 -- 172.21.15.63:0/463837584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9338005040 con 0x7f933c0fce40 2024-03-22T07:16:31.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.905+0000 7f9340e60700 1 --2- 172.21.15.63:0/463837584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f933c0fe040 0x7f933c10df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:16:31.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.905+0000 7f9341661700 1 --2- 172.21.15.63:0/463837584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f933c0fce40 0x7f933c10da40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f933800b3b0 tx=0x7f9338009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:31.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.906+0000 7f9331ffb700 1 -- 172.21.15.63:0/463837584 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f933800e040 con 0x7f933c0fce40 2024-03-22T07:16:31.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.906+0000 7f9331ffb700 1 -- 172.21.15.63:0/463837584 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f933800bcc0 con 0x7f933c0fce40 2024-03-22T07:16:31.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.906+0000 7f93438c5700 1 -- 172.21.15.63:0/463837584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c10f050 con 0x7f933c0fce40 2024-03-22T07:16:31.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.906+0000 7f9331ffb700 1 -- 172.21.15.63:0/463837584 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f933801c630 con 0x7f933c0fce40 2024-03-22T07:16:31.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.906+0000 7f93438c5700 1 -- 172.21.15.63:0/463837584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933c10f590 con 0x7f933c0fce40 2024-03-22T07:16:31.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.907+0000 7f93438c5700 1 -- 172.21.15.63:0/463837584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933c05f5f0 con 0x7f933c0fce40 2024-03-22T07:16:31.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.907+0000 7f9331ffb700 1 -- 172.21.15.63:0/463837584 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) v1 ==== 88853+0+0 (secure 0 0 0) 0x7f93380139f0 con 0x7f933c0fce40 2024-03-22T07:16:31.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.908+0000 7f9331ffb700 1 --2- 172.21.15.63:0/463837584 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f932806b0b0 0x7f932806d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:16:31.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.908+0000 7f9331ffb700 1 -- 172.21.15.63:0/463837584 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f933808a810 con 0x7f933c0fce40 2024-03-22T07:16:31.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.908+0000 7f9340e60700 1 --2- 172.21.15.63:0/463837584 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f932806b0b0 0x7f932806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:16:31.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.908+0000 7f9340e60700 1 --2- 172.21.15.63:0/463837584 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f932806b0b0 0x7f932806d560 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f932c009fd0 tx=0x7f932c00b040 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:16:31.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:31.912+0000 7f9331ffb700 1 -- 172.21.15.63:0/463837584 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f93380586b0 con 0x7f933c0fce40 2024-03-22T07:16:32.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:31 smithi069 ceph-mon[36441]: pgmap v18: 65 pgs: 65 active+clean; 292 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 339 op/s 2024-03-22T07:16:32.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:31 smithi069 ceph-mon[36441]: from='client.14630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:32.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:32.163+0000 7f93438c5700 1 -- 172.21.15.63:0/463837584 --> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f933c10f8e0 con 0x7f932806b0b0 2024-03-22T07:16:32.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:32.164+0000 7f9331ffb700 1 -- 172.21.15.63:0/463837584 <== mgr.24385 v2:172.21.15.69:6828/1862357042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f933c10f8e0 con 0x7f932806b0b0 2024-03-22T07:16:32.164 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:16:32.164 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:16:32.164 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:16:32.164 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-22T07:16:32.164 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-03-22T07:16:32.164 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "1/2 daemons upgraded", 2024-03-22T07:16:32.164 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "", 2024-03-22T07:16:32.164 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:16:32.164 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:16:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:32.166+0000 7f93438c5700 1 -- 172.21.15.63:0/463837584 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f932806b0b0 msgr2=0x7f932806d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:32.166+0000 7f93438c5700 1 --2- 172.21.15.63:0/463837584 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f932806b0b0 0x7f932806d560 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f932c009fd0 tx=0x7f932c00b040 comp rx=0 tx=0).stop 2024-03-22T07:16:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:32.166+0000 7f93438c5700 1 -- 172.21.15.63:0/463837584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f933c0fce40 msgr2=0x7f933c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:16:32.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:32.166+0000 7f93438c5700 1 --2- 172.21.15.63:0/463837584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f933c0fce40 0x7f933c10da40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f933800b3b0 tx=0x7f9338009e30 comp rx=0 tx=0).stop 2024-03-22T07:16:32.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:32.166+0000 7f93438c5700 1 -- 172.21.15.63:0/463837584 shutdown_connections 2024-03-22T07:16:32.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:32.166+0000 7f93438c5700 1 --2- 172.21.15.63:0/463837584 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f932806b0b0 0x7f932806d560 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:32.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:32.166+0000 7f93438c5700 1 --2- 172.21.15.63:0/463837584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f933c0fce40 0x7f933c10da40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:32.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:32.166+0000 7f93438c5700 1 --2- 172.21.15.63:0/463837584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f933c0fe040 0x7f933c10df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:16:32.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:32.166+0000 7f93438c5700 1 -- 172.21.15.63:0/463837584 >> 172.21.15.63:0/463837584 conn(0x7f933c0f8550 msgr2=0x7f933c0f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:16:32.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:32.166+0000 7f93438c5700 1 -- 172.21.15.63:0/463837584 shutdown_connections 2024-03-22T07:16:32.171 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:16:32.166+0000 7f93438c5700 1 -- 172.21.15.63:0/463837584 wait complete. 2024-03-22T07:16:32.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:32 smithi063 ceph-mon[29200]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:32.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:32 smithi063 ceph-mon[29200]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:32.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:32 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/1513803701' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:16:32.926 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:32 smithi069 ceph-mon[36441]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:32.926 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:32 smithi069 ceph-mon[36441]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:32.926 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:32 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/1513803701' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:16:33.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:33 smithi063 ceph-mon[29200]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:33.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:33 smithi063 ceph-mon[29200]: pgmap v19: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 498 op/s 2024-03-22T07:16:34.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:33 smithi069 ceph-mon[36441]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:16:34.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:33 smithi069 ceph-mon[36441]: pgmap v19: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 498 op/s 2024-03-22T07:16:35.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:35 smithi063 ceph-mon[29200]: pgmap v20: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 353 op/s 2024-03-22T07:16:36.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:35 smithi069 ceph-mon[36441]: pgmap v20: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 353 op/s 2024-03-22T07:16:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:37 smithi063 ceph-mon[29200]: pgmap v21: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 389 op/s 2024-03-22T07:16:38.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:37 smithi069 ceph-mon[36441]: pgmap v21: 65 pgs: 65 active+clean; 284 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 389 op/s 2024-03-22T07:16:39.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:39 smithi063 ceph-mon[29200]: pgmap v22: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 406 op/s 2024-03-22T07:16:40.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:39 smithi069 ceph-mon[36441]: pgmap v22: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 406 op/s 2024-03-22T07:16:41.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:41 smithi063 ceph-mon[29200]: pgmap v23: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 285 op/s 2024-03-22T07:16:42.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:41 smithi069 ceph-mon[36441]: pgmap v23: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 285 op/s 2024-03-22T07:16:43.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:43 smithi063 ceph-mon[29200]: pgmap v24: 65 pgs: 65 active+clean; 291 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 438 op/s 2024-03-22T07:16:44.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:43 smithi069 ceph-mon[36441]: pgmap v24: 65 pgs: 65 active+clean; 291 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 438 op/s 2024-03-22T07:16:45.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:45 smithi063 ceph-mon[29200]: pgmap v25: 65 pgs: 65 active+clean; 291 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 278 op/s 2024-03-22T07:16:45.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:45 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:16:46.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:45 smithi069 ceph-mon[36441]: pgmap v25: 65 pgs: 65 active+clean; 291 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 278 op/s 2024-03-22T07:16:46.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:45 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:16:47.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:46 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:47.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:46 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:47.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:46 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:47.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:46 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:16:48.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:47 smithi069 ceph-mon[36441]: Reconfiguring grafana.smithi063 (dependencies changed)... 2024-03-22T07:16:48.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:47 smithi069 ceph-mon[36441]: Reconfiguring daemon grafana.smithi063 on smithi063 2024-03-22T07:16:48.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:47 smithi069 ceph-mon[36441]: pgmap v26: 65 pgs: 65 active+clean; 288 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 335 op/s 2024-03-22T07:16:48.122 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:47 smithi063 ceph-mon[29200]: Reconfiguring grafana.smithi063 (dependencies changed)... 2024-03-22T07:16:48.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:47 smithi063 ceph-mon[29200]: Reconfiguring daemon grafana.smithi063 on smithi063 2024-03-22T07:16:48.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:47 smithi063 ceph-mon[29200]: pgmap v26: 65 pgs: 65 active+clean; 288 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 335 op/s 2024-03-22T07:16:50.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:49 smithi069 ceph-mon[36441]: pgmap v27: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 407 op/s 2024-03-22T07:16:50.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:49 smithi063 ceph-mon[29200]: pgmap v27: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 407 op/s 2024-03-22T07:16:52.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:51 smithi069 ceph-mon[36441]: pgmap v28: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 318 op/s 2024-03-22T07:16:52.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:51 smithi063 ceph-mon[29200]: pgmap v28: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 318 op/s 2024-03-22T07:16:54.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:53 smithi069 ceph-mon[36441]: pgmap v29: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 457 op/s 2024-03-22T07:16:54.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:53 smithi069 ceph-mon[36441]: fsmap cephfs:2 {0=cephfs.smithi063.xeelpk=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2 up:standby-replay 2024-03-22T07:16:54.122 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:53 smithi063 ceph-mon[29200]: pgmap v29: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 457 op/s 2024-03-22T07:16:54.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:53 smithi063 ceph-mon[29200]: fsmap cephfs:2 {0=cephfs.smithi063.xeelpk=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2 up:standby-replay 2024-03-22T07:16:56.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:55 smithi069 ceph-mon[36441]: pgmap v30: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 303 op/s 2024-03-22T07:16:56.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:55 smithi063 ceph-mon[29200]: pgmap v30: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 303 op/s 2024-03-22T07:16:58.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:57 smithi069 ceph-mon[36441]: pgmap v31: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 355 op/s 2024-03-22T07:16:58.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:57 smithi063 ceph-mon[29200]: pgmap v31: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 355 op/s 2024-03-22T07:16:59.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:16:59 smithi063 ceph-mon[29200]: pgmap v32: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 404 op/s 2024-03-22T07:17:00.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:16:59 smithi069 ceph-mon[36441]: pgmap v32: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 404 op/s 2024-03-22T07:17:01.056 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:00 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:17:01.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:00 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:17:02.018 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:01 smithi063 ceph-mon[29200]: pgmap v33: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 295 op/s 2024-03-22T07:17:02.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:01 smithi069 ceph-mon[36441]: pgmap v33: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 295 op/s 2024-03-22T07:17:02.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.288+0000 7ff165caf700 1 -- 172.21.15.63:0/4129099073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1600fcf80 msgr2=0x7ff1600fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:02.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.288+0000 7ff165caf700 1 --2- 172.21.15.63:0/4129099073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1600fcf80 0x7ff1600fd3f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff154004940 tx=0x7ff154009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:02.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.288+0000 7ff165caf700 1 -- 172.21.15.63:0/4129099073 shutdown_connections 2024-03-22T07:17:02.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.288+0000 7ff165caf700 1 --2- 172.21.15.63:0/4129099073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1600fcf80 0x7ff1600fd3f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:02.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.288+0000 7ff165caf700 1 --2- 172.21.15.63:0/4129099073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1600fbcd0 0x7ff1600fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:02.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.288+0000 7ff165caf700 1 -- 172.21.15.63:0/4129099073 >> 172.21.15.63:0/4129099073 conn(0x7ff1600f7430 msgr2=0x7ff1600f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:02.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.288+0000 7ff165caf700 1 -- 172.21.15.63:0/4129099073 shutdown_connections 2024-03-22T07:17:02.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.288+0000 7ff165caf700 1 -- 172.21.15.63:0/4129099073 wait complete. 2024-03-22T07:17:02.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.289+0000 7ff165caf700 1 Processor -- start 2024-03-22T07:17:02.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.289+0000 7ff165caf700 1 -- start start 2024-03-22T07:17:02.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.289+0000 7ff165caf700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1600fbcd0 0x7ff160103be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:02.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.289+0000 7ff165caf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1600fcf80 0x7ff1601022e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:02.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.289+0000 7ff165caf700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1601041e0 con 0x7ff1600fbcd0 2024-03-22T07:17:02.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.289+0000 7ff165caf700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff160102820 con 0x7ff1600fcf80 2024-03-22T07:17:02.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.289+0000 7ff15f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1600fbcd0 0x7ff160103be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:02.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.289+0000 7ff15f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1600fbcd0 0x7ff160103be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33108/0 (socket says 172.21.15.63:33108) 2024-03-22T07:17:02.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.289+0000 7ff15f7fe700 1 -- 172.21.15.63:0/2430629984 learned_addr learned my addr 172.21.15.63:0/2430629984 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:17:02.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.289+0000 7ff15effd700 1 --2- 172.21.15.63:0/2430629984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1600fcf80 0x7ff1601022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:02.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.290+0000 7ff15f7fe700 1 -- 172.21.15.63:0/2430629984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1600fcf80 msgr2=0x7ff1601022e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:02.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.290+0000 7ff15f7fe700 1 --2- 172.21.15.63:0/2430629984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1600fcf80 0x7ff1601022e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:02.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.290+0000 7ff15f7fe700 1 -- 172.21.15.63:0/2430629984 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff150005040 con 0x7ff1600fbcd0 2024-03-22T07:17:02.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.290+0000 7ff15effd700 1 --2- 172.21.15.63:0/2430629984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1600fcf80 0x7ff1601022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:17:02.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.290+0000 7ff15f7fe700 1 --2- 172.21.15.63:0/2430629984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1600fbcd0 0x7ff160103be0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff1500096d0 tx=0x7ff150009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:02.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.290+0000 7ff147fff700 1 -- 172.21.15.63:0/2430629984 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff15000e840 con 0x7ff1600fbcd0 2024-03-22T07:17:02.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.290+0000 7ff165caf700 1 -- 172.21.15.63:0/2430629984 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff154005040 con 0x7ff1600fbcd0 2024-03-22T07:17:02.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.290+0000 7ff147fff700 1 -- 172.21.15.63:0/2430629984 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff15000f040 con 0x7ff1600fbcd0 2024-03-22T07:17:02.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.290+0000 7ff147fff700 1 -- 172.21.15.63:0/2430629984 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff150013dc0 con 0x7ff1600fbcd0 2024-03-22T07:17:02.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.290+0000 7ff165caf700 1 -- 172.21.15.63:0/2430629984 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff160102e50 con 0x7ff1600fbcd0 2024-03-22T07:17:02.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.292+0000 7ff165caf700 1 -- 172.21.15.63:0/2430629984 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff16004c710 con 0x7ff1600fbcd0 2024-03-22T07:17:02.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.292+0000 7ff147fff700 1 -- 172.21.15.63:0/2430629984 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) v1 ==== 88853+0+0 (secure 0 0 0) 0x7ff15001c410 con 0x7ff1600fbcd0 2024-03-22T07:17:02.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.293+0000 7ff147fff700 1 --2- 172.21.15.63:0/2430629984 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7ff14806b0c0 0x7ff14806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:02.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.293+0000 7ff147fff700 1 -- 172.21.15.63:0/2430629984 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7ff150089870 con 0x7ff1600fbcd0 2024-03-22T07:17:02.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.293+0000 7ff15effd700 1 --2- 172.21.15.63:0/2430629984 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7ff14806b0c0 0x7ff14806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:02.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.293+0000 7ff15effd700 1 --2- 172.21.15.63:0/2430629984 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7ff14806b0c0 0x7ff14806d570 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff15400b0d0 tx=0x7ff154009e30 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:02.298 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.298+0000 7ff147fff700 1 -- 172.21.15.63:0/2430629984 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7ff150057740 con 0x7ff1600fbcd0 2024-03-22T07:17:02.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.583+0000 7ff165caf700 1 -- 172.21.15.63:0/2430629984 --> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1601011c0 con 0x7ff14806b0c0 2024-03-22T07:17:02.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.584+0000 7ff147fff700 1 -- 172.21.15.63:0/2430629984 <== mgr.24385 v2:172.21.15.69:6828/1862357042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7ff1601011c0 con 0x7ff14806b0c0 2024-03-22T07:17:02.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.586+0000 7ff165caf700 1 -- 172.21.15.63:0/2430629984 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7ff14806b0c0 msgr2=0x7ff14806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:02.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.586+0000 7ff165caf700 1 --2- 172.21.15.63:0/2430629984 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7ff14806b0c0 0x7ff14806d570 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff15400b0d0 tx=0x7ff154009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:02.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.586+0000 7ff165caf700 1 -- 172.21.15.63:0/2430629984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1600fbcd0 msgr2=0x7ff160103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:02.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.586+0000 7ff165caf700 1 --2- 172.21.15.63:0/2430629984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1600fbcd0 0x7ff160103be0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff1500096d0 tx=0x7ff150009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:02.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.587+0000 7ff165caf700 1 -- 172.21.15.63:0/2430629984 shutdown_connections 2024-03-22T07:17:02.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.587+0000 7ff165caf700 1 --2- 172.21.15.63:0/2430629984 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7ff14806b0c0 0x7ff14806d570 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:02.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.587+0000 7ff165caf700 1 --2- 172.21.15.63:0/2430629984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff1600fbcd0 0x7ff160103be0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:02.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.587+0000 7ff165caf700 1 --2- 172.21.15.63:0/2430629984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff1600fcf80 0x7ff1601022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:02.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.587+0000 7ff165caf700 1 -- 172.21.15.63:0/2430629984 >> 172.21.15.63:0/2430629984 conn(0x7ff1600f7430 msgr2=0x7ff1600f8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:02.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.587+0000 7ff165caf700 1 -- 172.21.15.63:0/2430629984 shutdown_connections 2024-03-22T07:17:02.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.587+0000 7ff165caf700 1 -- 172.21.15.63:0/2430629984 wait complete. 2024-03-22T07:17:02.602 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:17:02.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.741+0000 7f97d061a700 1 -- 172.21.15.63:0/3526085143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c80fcf80 msgr2=0x7f97c80fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:02.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.741+0000 7f97d061a700 1 --2- 172.21.15.63:0/3526085143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c80fcf80 0x7f97c80fd3f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f97b8004940 tx=0x7f97b8009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:02.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.741+0000 7f97d061a700 1 -- 172.21.15.63:0/3526085143 shutdown_connections 2024-03-22T07:17:02.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.741+0000 7f97d061a700 1 --2- 172.21.15.63:0/3526085143 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c80fcf80 0x7f97c80fd3f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:02.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.741+0000 7f97d061a700 1 --2- 172.21.15.63:0/3526085143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c80fbcd0 0x7f97c80fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:02.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.741+0000 7f97d061a700 1 -- 172.21.15.63:0/3526085143 >> 172.21.15.63:0/3526085143 conn(0x7f97c80f7430 msgr2=0x7f97c80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:02.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.742+0000 7f97d061a700 1 -- 172.21.15.63:0/3526085143 shutdown_connections 2024-03-22T07:17:02.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.742+0000 7f97d061a700 1 -- 172.21.15.63:0/3526085143 wait complete. 2024-03-22T07:17:02.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.743+0000 7f97d061a700 1 Processor -- start 2024-03-22T07:17:02.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.743+0000 7f97d061a700 1 -- start start 2024-03-22T07:17:02.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.743+0000 7f97d061a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c80fbcd0 0x7f97c80681a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:02.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.743+0000 7f97d061a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c80fcf80 0x7f97c80686e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:02.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.743+0000 7f97d061a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8068d30 con 0x7f97c80fcf80 2024-03-22T07:17:02.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.743+0000 7f97d061a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8069580 con 0x7f97c80fbcd0 2024-03-22T07:17:02.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.743+0000 7f97cdbb5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c80fcf80 0x7f97c80686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:02.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.743+0000 7f97cdbb5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c80fcf80 0x7f97c80686e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33128/0 (socket says 172.21.15.63:33128) 2024-03-22T07:17:02.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.743+0000 7f97cdbb5700 1 -- 172.21.15.63:0/1017799149 learned_addr learned my addr 172.21.15.63:0/1017799149 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:17:02.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.744+0000 7f97ce3b6700 1 --2- 172.21.15.63:0/1017799149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c80fbcd0 0x7f97c80681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:02.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.744+0000 7f97cdbb5700 1 -- 172.21.15.63:0/1017799149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c80fbcd0 msgr2=0x7f97c80681a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:02.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.744+0000 7f97cdbb5700 1 --2- 172.21.15.63:0/1017799149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c80fbcd0 0x7f97c80681a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:02.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.744+0000 7f97cdbb5700 1 -- 172.21.15.63:0/1017799149 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b8005040 con 0x7f97c80fcf80 2024-03-22T07:17:02.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.744+0000 7f97cdbb5700 1 --2- 172.21.15.63:0/1017799149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c80fcf80 0x7f97c80686e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f97b80137d0 tx=0x7f97b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:02.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.745+0000 7f97beffd700 1 -- 172.21.15.63:0/1017799149 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97b800e070 con 0x7f97c80fcf80 2024-03-22T07:17:02.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.745+0000 7f97beffd700 1 -- 172.21.15.63:0/1017799149 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f97b801fd40 con 0x7f97c80fcf80 2024-03-22T07:17:02.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.745+0000 7f97beffd700 1 -- 172.21.15.63:0/1017799149 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97b800b740 con 0x7f97c80fcf80 2024-03-22T07:17:02.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.745+0000 7f97d061a700 1 -- 172.21.15.63:0/1017799149 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c80697a0 con 0x7f97c80fcf80 2024-03-22T07:17:02.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.745+0000 7f97d061a700 1 -- 172.21.15.63:0/1017799149 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c8069c40 con 0x7f97c80fcf80 2024-03-22T07:17:02.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.746+0000 7f97d061a700 1 -- 172.21.15.63:0/1017799149 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac0052e0 con 0x7f97c80fcf80 2024-03-22T07:17:02.748 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.747+0000 7f97beffd700 1 -- 172.21.15.63:0/1017799149 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) v1 ==== 88853+0+0 (secure 0 0 0) 0x7f97b8013870 con 0x7f97c80fcf80 2024-03-22T07:17:02.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.748+0000 7f97beffd700 1 --2- 172.21.15.63:0/1017799149 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f97b406b0b0 0x7f97b406d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:02.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.748+0000 7f97beffd700 1 -- 172.21.15.63:0/1017799149 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f97b8054b90 con 0x7f97c80fcf80 2024-03-22T07:17:02.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.748+0000 7f97ce3b6700 1 --2- 172.21.15.63:0/1017799149 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f97b406b0b0 0x7f97b406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:02.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.749+0000 7f97ce3b6700 1 --2- 172.21.15.63:0/1017799149 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f97b406b0b0 0x7f97b406d560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f97c8067f30 tx=0x7f97c400b040 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:02.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:02.755+0000 7f97beffd700 1 -- 172.21.15.63:0/1017799149 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f97b80547c0 con 0x7f97c80fcf80 2024-03-22T07:17:03.068 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.067+0000 7f97d061a700 1 -- 172.21.15.63:0/1017799149 --> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f97ac000bf0 con 0x7f97b406b0b0 2024-03-22T07:17:03.069 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.068+0000 7f97beffd700 1 -- 172.21.15.63:0/1017799149 <== mgr.24385 v2:172.21.15.69:6828/1862357042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f97ac000bf0 con 0x7f97b406b0b0 2024-03-22T07:17:03.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.071+0000 7f97d061a700 1 -- 172.21.15.63:0/1017799149 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f97b406b0b0 msgr2=0x7f97b406d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:03.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.071+0000 7f97d061a700 1 --2- 172.21.15.63:0/1017799149 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f97b406b0b0 0x7f97b406d560 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f97c8067f30 tx=0x7f97c400b040 comp rx=0 tx=0).stop 2024-03-22T07:17:03.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.071+0000 7f97d061a700 1 -- 172.21.15.63:0/1017799149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c80fcf80 msgr2=0x7f97c80686e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:03.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.071+0000 7f97d061a700 1 --2- 172.21.15.63:0/1017799149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c80fcf80 0x7f97c80686e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f97b80137d0 tx=0x7f97b8009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:03.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.071+0000 7f97d061a700 1 -- 172.21.15.63:0/1017799149 shutdown_connections 2024-03-22T07:17:03.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.071+0000 7f97d061a700 1 --2- 172.21.15.63:0/1017799149 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f97b406b0b0 0x7f97b406d560 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:03.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.071+0000 7f97d061a700 1 --2- 172.21.15.63:0/1017799149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c80fbcd0 0x7f97c80681a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:03.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.071+0000 7f97d061a700 1 --2- 172.21.15.63:0/1017799149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c80fcf80 0x7f97c80686e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:03.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.071+0000 7f97d061a700 1 -- 172.21.15.63:0/1017799149 >> 172.21.15.63:0/1017799149 conn(0x7f97c80f7430 msgr2=0x7f97c80f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:03.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.071+0000 7f97d061a700 1 -- 172.21.15.63:0/1017799149 shutdown_connections 2024-03-22T07:17:03.073 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.071+0000 7f97d061a700 1 -- 172.21.15.63:0/1017799149 wait complete. 2024-03-22T07:17:03.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.236+0000 7f1bbf3a0700 1 -- 172.21.15.63:0/1276234421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb80fdcf0 msgr2=0x7f1bb8100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:03.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.236+0000 7f1bbf3a0700 1 --2- 172.21.15.63:0/1276234421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb80fdcf0 0x7f1bb8100170 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8004940 tx=0x7f1ba8009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:03.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.236+0000 7f1bbf3a0700 1 -- 172.21.15.63:0/1276234421 shutdown_connections 2024-03-22T07:17:03.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.236+0000 7f1bbf3a0700 1 --2- 172.21.15.63:0/1276234421 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb80fdcf0 0x7f1bb8100170 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:03.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.236+0000 7f1bbf3a0700 1 --2- 172.21.15.63:0/1276234421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1bb80662e0 0x7f1bb80fd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:03.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.236+0000 7f1bbf3a0700 1 -- 172.21.15.63:0/1276234421 >> 172.21.15.63:0/1276234421 conn(0x7f1bb80f7430 msgr2=0x7f1bb80f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:03.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.237+0000 7f1bbf3a0700 1 -- 172.21.15.63:0/1276234421 shutdown_connections 2024-03-22T07:17:03.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.237+0000 7f1bbf3a0700 1 -- 172.21.15.63:0/1276234421 wait complete. 2024-03-22T07:17:03.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.237+0000 7f1bbf3a0700 1 Processor -- start 2024-03-22T07:17:03.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.238+0000 7f1bbf3a0700 1 -- start start 2024-03-22T07:17:03.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.238+0000 7f1bbf3a0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1bb80662e0 0x7f1bb81085e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:03.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.238+0000 7f1bbf3a0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb80fdcf0 0x7f1bb8108b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:03.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.238+0000 7f1bbf3a0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb810a1d0 con 0x7f1bb80fdcf0 2024-03-22T07:17:03.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.238+0000 7f1bbf3a0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb810a310 con 0x7f1bb80662e0 2024-03-22T07:17:03.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.238+0000 7f1bbd13c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1bb80662e0 0x7f1bb81085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:03.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.238+0000 7f1bbd13c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1bb80662e0 0x7f1bb81085e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.63:42626/0 (socket says 172.21.15.63:42626) 2024-03-22T07:17:03.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.238+0000 7f1bbd13c700 1 -- 172.21.15.63:0/1163972329 learned_addr learned my addr 172.21.15.63:0/1163972329 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:17:03.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.239+0000 7f1bbd13c700 1 -- 172.21.15.63:0/1163972329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb80fdcf0 msgr2=0x7f1bb8108b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:03.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.239+0000 7f1bbd13c700 1 --2- 172.21.15.63:0/1163972329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb80fdcf0 0x7f1bb8108b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:03.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.239+0000 7f1bbd13c700 1 -- 172.21.15.63:0/1163972329 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb4005040 con 0x7f1bb80662e0 2024-03-22T07:17:03.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.239+0000 7f1bbd13c700 1 --2- 172.21.15.63:0/1163972329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1bb80662e0 0x7f1bb81085e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009700 tx=0x7f1bb4009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:03.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.239+0000 7f1badffb700 1 -- 172.21.15.63:0/1163972329 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bb400e7e0 con 0x7f1bb80662e0 2024-03-22T07:17:03.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.239+0000 7f1badffb700 1 -- 172.21.15.63:0/1163972329 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bb400f040 con 0x7f1bb80662e0 2024-03-22T07:17:03.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.240+0000 7f1badffb700 1 -- 172.21.15.63:0/1163972329 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bb4013e90 con 0x7f1bb80662e0 2024-03-22T07:17:03.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.240+0000 7f1bbf3a0700 1 -- 172.21.15.63:0/1163972329 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba8005040 con 0x7f1bb80662e0 2024-03-22T07:17:03.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.240+0000 7f1bbf3a0700 1 -- 172.21.15.63:0/1163972329 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb8109300 con 0x7f1bb80662e0 2024-03-22T07:17:03.242 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.241+0000 7f1bbf3a0700 1 -- 172.21.15.63:0/1163972329 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb805f5f0 con 0x7f1bb80662e0 2024-03-22T07:17:03.243 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.242+0000 7f1badffb700 1 -- 172.21.15.63:0/1163972329 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) v1 ==== 88853+0+0 (secure 0 0 0) 0x7f1bb400e940 con 0x7f1bb80662e0 2024-03-22T07:17:03.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.243+0000 7f1badffb700 1 --2- 172.21.15.63:0/1163972329 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f1ba406b0b0 0x7f1ba406d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:03.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.243+0000 7f1badffb700 1 -- 172.21.15.63:0/1163972329 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f1bb4089960 con 0x7f1bb80662e0 2024-03-22T07:17:03.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.243+0000 7f1bbc93b700 1 --2- 172.21.15.63:0/1163972329 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f1ba406b0b0 0x7f1ba406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:03.246 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.246+0000 7f1bbc93b700 1 --2- 172.21.15.63:0/1163972329 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f1ba406b0b0 0x7f1ba406d560 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800b0d0 tx=0x7f1ba8009e30 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:03.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.250+0000 7f1badffb700 1 -- 172.21.15.63:0/1163972329 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f1bb4053b30 con 0x7f1bb80662e0 2024-03-22T07:17:03.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.573+0000 7f1bbf3a0700 1 -- 172.21.15.63:0/1163972329 --> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1bb80f8f70 con 0x7f1ba406b0b0 2024-03-22T07:17:03.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.583+0000 7f1badffb700 1 -- 172.21.15.63:0/1163972329 <== mgr.24385 v2:172.21.15.69:6828/1862357042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f1bb80f8f70 con 0x7f1ba406b0b0 2024-03-22T07:17:03.584 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:17:03.584 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 starting - - - - 2024-03-22T07:17:03.584 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (9m) 49s ago 9m 6660k - 17.2.7-726-g9c4e0c34 837124da804f d6c6bbeb6904 2024-03-22T07:17:03.584 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (8m) 47s ago 8m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 30992f8aa88c 2024-03-22T07:17:03.584 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (7m) 49s ago 8m 85.3M - 9.4.7 954c08fa6188 f0b1d2a54439 2024-03-22T07:17:03.584 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (3m) 49s ago 3m 90.3M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:17:03.584 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (3m) 49s ago 3m 152M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:17:03.584 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (3m) 47s ago 3m 78.8M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:17:03.584 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (3m) 47s ago 3m 135M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:17:03.585 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:9283 running (10m) 49s ago 10m 393M - 17.2.7-726-g9c4e0c34 837124da804f 422b5e836153 2024-03-22T07:17:03.585 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283 running (98s) 47s ago 7m 491M - 19.0.0-1667-gdb0330b1 4f325f8440e9 5ad05e49e512 2024-03-22T07:17:03.585 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (10m) 49s ago 10m 51.4M 2048M 17.2.7-726-g9c4e0c34 837124da804f f92a5bb6caee 2024-03-22T07:17:03.585 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (7m) 47s ago 7m 38.1M 2048M 17.2.7-726-g9c4e0c34 837124da804f 95f6103f7603 2024-03-22T07:17:03.585 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 starting - - - - 2024-03-22T07:17:03.585 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (8m) 47s ago 8m 16.4M - 1.5.0 0da6a335fe13 49f4ca105fb9 2024-03-22T07:17:03.585 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (7m) 49s ago 7m 251M 1557M 17.2.7-726-g9c4e0c34 837124da804f 580f22188a10 2024-03-22T07:17:03.585 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (6m) 49s ago 6m 294M 1557M 17.2.7-726-g9c4e0c34 837124da804f fe1ace35c88a 2024-03-22T07:17:03.585 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (6m) 49s ago 6m 223M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4cd995265dcf 2024-03-22T07:17:03.585 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (5m) 47s ago 5m 298M 2581M 17.2.7-726-g9c4e0c34 837124da804f 45c9a759b76d 2024-03-22T07:17:03.585 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (5m) 47s ago 5m 265M 2581M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:17:03.585 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (5m) 47s ago 5m 299M 2581M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:17:03.585 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (7m) 49s ago 8m 56.4M - 2.43.0 a07b618ecd1d 827638a5870a 2024-03-22T07:17:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.586+0000 7f1bbf3a0700 1 -- 172.21.15.63:0/1163972329 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f1ba406b0b0 msgr2=0x7f1ba406d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.586+0000 7f1bbf3a0700 1 --2- 172.21.15.63:0/1163972329 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f1ba406b0b0 0x7f1ba406d560 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800b0d0 tx=0x7f1ba8009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.586+0000 7f1bbf3a0700 1 -- 172.21.15.63:0/1163972329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1bb80662e0 msgr2=0x7f1bb81085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:03.587 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.586+0000 7f1bbf3a0700 1 --2- 172.21.15.63:0/1163972329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1bb80662e0 0x7f1bb81085e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009700 tx=0x7f1bb4009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.587+0000 7f1bbf3a0700 1 -- 172.21.15.63:0/1163972329 shutdown_connections 2024-03-22T07:17:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.587+0000 7f1bbf3a0700 1 --2- 172.21.15.63:0/1163972329 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f1ba406b0b0 0x7f1ba406d560 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.587+0000 7f1bbf3a0700 1 --2- 172.21.15.63:0/1163972329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1bb80662e0 0x7f1bb81085e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.587+0000 7f1bbf3a0700 1 --2- 172.21.15.63:0/1163972329 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1bb80fdcf0 0x7f1bb8108b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.587+0000 7f1bbf3a0700 1 -- 172.21.15.63:0/1163972329 >> 172.21.15.63:0/1163972329 conn(0x7f1bb80f7430 msgr2=0x7f1bb80f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:03.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.587+0000 7f1bbf3a0700 1 -- 172.21.15.63:0/1163972329 shutdown_connections 2024-03-22T07:17:03.589 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.587+0000 7f1bbf3a0700 1 -- 172.21.15.63:0/1163972329 wait complete. 2024-03-22T07:17:03.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.753+0000 7f40524e9700 1 -- 172.21.15.63:0/2556140962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f404c0fcef0 msgr2=0x7f404c0fd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:03.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.753+0000 7f40524e9700 1 --2- 172.21.15.63:0/2556140962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f404c0fcef0 0x7f404c0fd300 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4034004940 tx=0x7f4034009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:03.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.753+0000 7f40524e9700 1 -- 172.21.15.63:0/2556140962 shutdown_connections 2024-03-22T07:17:03.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.753+0000 7f40524e9700 1 --2- 172.21.15.63:0/2556140962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c0fe060 0x7f404c0fe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:03.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.753+0000 7f40524e9700 1 --2- 172.21.15.63:0/2556140962 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f404c0fcef0 0x7f404c0fd300 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:03.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.753+0000 7f40524e9700 1 -- 172.21.15.63:0/2556140962 >> 172.21.15.63:0/2556140962 conn(0x7f404c0f85b0 msgr2=0x7f404c0fa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:03.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.753+0000 7f40524e9700 1 -- 172.21.15.63:0/2556140962 shutdown_connections 2024-03-22T07:17:03.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.753+0000 7f40524e9700 1 -- 172.21.15.63:0/2556140962 wait complete. 2024-03-22T07:17:03.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.754+0000 7f40524e9700 1 Processor -- start 2024-03-22T07:17:03.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.754+0000 7f40524e9700 1 -- start start 2024-03-22T07:17:03.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.755+0000 7f40524e9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f404c0fcef0 0x7f404c10da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:03.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.755+0000 7f40524e9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c0fe060 0x7f404c10dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:03.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.755+0000 7f40524e9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c10e660 con 0x7f404c0fcef0 2024-03-22T07:17:03.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.755+0000 7f40524e9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c10ee50 con 0x7f404c0fe060 2024-03-22T07:17:03.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.755+0000 7f404bfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f404c0fcef0 0x7f404c10da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:03.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.755+0000 7f404bfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f404c0fcef0 0x7f404c10da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33146/0 (socket says 172.21.15.63:33146) 2024-03-22T07:17:03.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.755+0000 7f404bfff700 1 -- 172.21.15.63:0/4034466647 learned_addr learned my addr 172.21.15.63:0/4034466647 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:17:03.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.755+0000 7f40437fe700 1 --2- 172.21.15.63:0/4034466647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c0fe060 0x7f404c10dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:03.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.755+0000 7f404bfff700 1 -- 172.21.15.63:0/4034466647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c0fe060 msgr2=0x7f404c10dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:03.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.755+0000 7f404bfff700 1 --2- 172.21.15.63:0/4034466647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c0fe060 0x7f404c10dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:03.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.755+0000 7f404bfff700 1 -- 172.21.15.63:0/4034466647 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4034005040 con 0x7f404c0fcef0 2024-03-22T07:17:03.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.755+0000 7f40437fe700 1 --2- 172.21.15.63:0/4034466647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c0fe060 0x7f404c10dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:17:03.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.756+0000 7f404bfff700 1 --2- 172.21.15.63:0/4034466647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f404c0fcef0 0x7f404c10da90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f403400b3b0 tx=0x7f4034009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:03.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.756+0000 7f40497fa700 1 -- 172.21.15.63:0/4034466647 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f403400e030 con 0x7f404c0fcef0 2024-03-22T07:17:03.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.756+0000 7f40497fa700 1 -- 172.21.15.63:0/4034466647 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4034013420 con 0x7f404c0fcef0 2024-03-22T07:17:03.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.756+0000 7f40524e9700 1 -- 172.21.15.63:0/4034466647 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f404c10f0d0 con 0x7f404c0fcef0 2024-03-22T07:17:03.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.756+0000 7f40497fa700 1 -- 172.21.15.63:0/4034466647 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f403400bee0 con 0x7f404c0fcef0 2024-03-22T07:17:03.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.756+0000 7f40524e9700 1 -- 172.21.15.63:0/4034466647 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404c10f5d0 con 0x7f404c0fcef0 2024-03-22T07:17:03.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.758+0000 7f40524e9700 1 -- 172.21.15.63:0/4034466647 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f404c05f5f0 con 0x7f404c0fcef0 2024-03-22T07:17:03.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.758+0000 7f40497fa700 1 -- 172.21.15.63:0/4034466647 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 24) v1 ==== 88853+0+0 (secure 0 0 0) 0x7f403401c6d0 con 0x7f404c0fcef0 2024-03-22T07:17:03.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.758+0000 7f40497fa700 1 --2- 172.21.15.63:0/4034466647 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f403806b100 0x7f403806d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:03.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.759+0000 7f40497fa700 1 -- 172.21.15.63:0/4034466647 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f403408b400 con 0x7f404c0fcef0 2024-03-22T07:17:03.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.759+0000 7f40437fe700 1 --2- 172.21.15.63:0/4034466647 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f403806b100 0x7f403806d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:03.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.759+0000 7f40437fe700 1 --2- 172.21.15.63:0/4034466647 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f403806b100 0x7f403806d5b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f403c006fd0 tx=0x7f403c000f90 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:03.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:03.765+0000 7f40497fa700 1 -- 172.21.15.63:0/4034466647 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f4034059240 con 0x7f404c0fcef0 2024-03-22T07:17:03.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:03 smithi063 ceph-mon[29200]: pgmap v34: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 537 op/s 2024-03-22T07:17:03.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:03 smithi063 ceph-mon[29200]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:04.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.182+0000 7f40524e9700 1 -- 172.21.15.63:0/4034466647 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f404c105980 con 0x7f404c0fcef0 2024-03-22T07:17:04.184 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:17:04.184 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:17:04.185 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:17:04.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:17:04.185 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:17:04.185 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T07:17:04.185 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-22T07:17:04.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:17:04.185 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:17:04.185 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:17:04.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:17:04.185 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:17:04.185 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:17:04.185 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:17:04.185 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:17:04.185 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 13, 2024-03-22T07:17:04.186 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-22T07:17:04.186 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:17:04.186 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:17:04.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.183+0000 7f40497fa700 1 -- 172.21.15.63:0/4034466647 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f4034020360 con 0x7f404c0fcef0 2024-03-22T07:17:04.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.188+0000 7f40524e9700 1 -- 172.21.15.63:0/4034466647 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f403806b100 msgr2=0x7f403806d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:04.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.188+0000 7f40524e9700 1 --2- 172.21.15.63:0/4034466647 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f403806b100 0x7f403806d5b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f403c006fd0 tx=0x7f403c000f90 comp rx=0 tx=0).stop 2024-03-22T07:17:04.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.188+0000 7f40524e9700 1 -- 172.21.15.63:0/4034466647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f404c0fcef0 msgr2=0x7f404c10da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:04.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.188+0000 7f40524e9700 1 --2- 172.21.15.63:0/4034466647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f404c0fcef0 0x7f404c10da90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f403400b3b0 tx=0x7f4034009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:04.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.188+0000 7f40524e9700 1 -- 172.21.15.63:0/4034466647 shutdown_connections 2024-03-22T07:17:04.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.188+0000 7f40524e9700 1 --2- 172.21.15.63:0/4034466647 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f403806b100 0x7f403806d5b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:04.189 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.188+0000 7f40524e9700 1 --2- 172.21.15.63:0/4034466647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f404c0fcef0 0x7f404c10da90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:04.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.188+0000 7f40524e9700 1 --2- 172.21.15.63:0/4034466647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f404c0fe060 0x7f404c10dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:04.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.188+0000 7f40524e9700 1 -- 172.21.15.63:0/4034466647 >> 172.21.15.63:0/4034466647 conn(0x7f404c0f85b0 msgr2=0x7f404c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:04.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.188+0000 7f40524e9700 1 -- 172.21.15.63:0/4034466647 shutdown_connections 2024-03-22T07:17:04.190 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.188+0000 7f40524e9700 1 -- 172.21.15.63:0/4034466647 wait complete. 2024-03-22T07:17:04.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:03 smithi069 ceph-mon[36441]: pgmap v34: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 537 op/s 2024-03-22T07:17:04.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:03 smithi069 ceph-mon[36441]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:04.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.343+0000 7fafbb9c9700 1 -- 172.21.15.63:0/1393676174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb40fce40 msgr2=0x7fafb40fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:04.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.343+0000 7fafbb9c9700 1 --2- 172.21.15.63:0/1393676174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb40fce40 0x7fafb40fd250 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fafb0004a90 tx=0x7fafb0009dd0 comp rx=0 tx=0).stop 2024-03-22T07:17:04.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.343+0000 7fafbb9c9700 1 -- 172.21.15.63:0/1393676174 shutdown_connections 2024-03-22T07:17:04.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.343+0000 7fafbb9c9700 1 --2- 172.21.15.63:0/1393676174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb40fe040 0x7fafb40fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:04.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.343+0000 7fafbb9c9700 1 --2- 172.21.15.63:0/1393676174 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb40fce40 0x7fafb40fd250 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:04.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.343+0000 7fafbb9c9700 1 -- 172.21.15.63:0/1393676174 >> 172.21.15.63:0/1393676174 conn(0x7fafb40f8570 msgr2=0x7fafb40fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:04.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.344+0000 7fafbb9c9700 1 -- 172.21.15.63:0/1393676174 shutdown_connections 2024-03-22T07:17:04.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.344+0000 7fafbb9c9700 1 -- 172.21.15.63:0/1393676174 wait complete. 2024-03-22T07:17:04.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.344+0000 7fafbb9c9700 1 Processor -- start 2024-03-22T07:17:04.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.344+0000 7fafbb9c9700 1 -- start start 2024-03-22T07:17:04.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.345+0000 7fafbb9c9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb40fce40 0x7fafb410d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:04.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.345+0000 7fafbb9c9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb40fe040 0x7fafb410df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:04.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.345+0000 7fafbb9c9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb410e530 con 0x7fafb40fce40 2024-03-22T07:17:04.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.345+0000 7fafbb9c9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb410ed80 con 0x7fafb40fe040 2024-03-22T07:17:04.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.345+0000 7fafb9765700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb40fce40 0x7fafb410d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:04.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.345+0000 7fafb8f64700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb40fe040 0x7fafb410df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:04.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.345+0000 7fafb8f64700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb40fe040 0x7fafb410df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.63:42644/0 (socket says 172.21.15.63:42644) 2024-03-22T07:17:04.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.345+0000 7fafb8f64700 1 -- 172.21.15.63:0/2055526816 learned_addr learned my addr 172.21.15.63:0/2055526816 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:17:04.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.345+0000 7fafb8f64700 1 -- 172.21.15.63:0/2055526816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb40fce40 msgr2=0x7fafb410d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:04.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.345+0000 7fafb8f64700 1 --2- 172.21.15.63:0/2055526816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb40fce40 0x7fafb410d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:04.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.345+0000 7fafb8f64700 1 -- 172.21.15.63:0/2055526816 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb0004650 con 0x7fafb40fe040 2024-03-22T07:17:04.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.345+0000 7fafb8f64700 1 --2- 172.21.15.63:0/2055526816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb40fe040 0x7fafb410df30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fafa400eb70 tx=0x7fafa400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:04.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.346+0000 7fafa9ffb700 1 -- 172.21.15.63:0/2055526816 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafa4007cc0 con 0x7fafb40fe040 2024-03-22T07:17:04.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.346+0000 7fafa9ffb700 1 -- 172.21.15.63:0/2055526816 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fafa400f040 con 0x7fafb40fe040 2024-03-22T07:17:04.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.346+0000 7fafbb9c9700 1 -- 172.21.15.63:0/2055526816 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb410f000 con 0x7fafb40fe040 2024-03-22T07:17:04.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.346+0000 7fafa9ffb700 1 -- 172.21.15.63:0/2055526816 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafa400ca90 con 0x7fafb40fe040 2024-03-22T07:17:04.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.346+0000 7fafbb9c9700 1 -- 172.21.15.63:0/2055526816 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb410f540 con 0x7fafb40fe040 2024-03-22T07:17:04.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.347+0000 7fafbb9c9700 1 -- 172.21.15.63:0/2055526816 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb405f5f0 con 0x7fafb40fe040 2024-03-22T07:17:04.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.348+0000 7fafa9ffb700 1 -- 172.21.15.63:0/2055526816 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) v1 ==== 88853+0+0 (secure 0 0 0) 0x7fafa400cbf0 con 0x7fafb40fe040 2024-03-22T07:17:04.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.348+0000 7fafa9ffb700 1 --2- 172.21.15.63:0/2055526816 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7fafa006b0b0 0x7fafa006d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:04.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.348+0000 7fafa9ffb700 1 -- 172.21.15.63:0/2055526816 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fafa4089610 con 0x7fafb40fe040 2024-03-22T07:17:04.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.348+0000 7fafb9765700 1 --2- 172.21.15.63:0/2055526816 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7fafa006b0b0 0x7fafa006d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:04.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.349+0000 7fafb9765700 1 --2- 172.21.15.63:0/2055526816 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7fafa006b0b0 0x7fafa006d560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fafb000b8b0 tx=0x7fafb0016040 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:04.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.354+0000 7fafa9ffb700 1 -- 172.21.15.63:0/2055526816 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fafa4057390 con 0x7fafb40fe040 2024-03-22T07:17:04.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.734+0000 7fafbb9c9700 1 -- 172.21.15.63:0/2055526816 --> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fafb410f820 con 0x7fafa006b0b0 2024-03-22T07:17:04.736 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.735+0000 7fafa9ffb700 1 -- 172.21.15.63:0/2055526816 <== mgr.24385 v2:172.21.15.69:6828/1862357042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fafb410f820 con 0x7fafa006b0b0 2024-03-22T07:17:04.736 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:17:04.736 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:17:04.736 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:17:04.736 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-22T07:17:04.737 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-03-22T07:17:04.737 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "1/2 daemons upgraded", 2024-03-22T07:17:04.737 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "", 2024-03-22T07:17:04.737 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:17:04.737 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:17:04.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.739+0000 7fafbb9c9700 1 -- 172.21.15.63:0/2055526816 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7fafa006b0b0 msgr2=0x7fafa006d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:04.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.739+0000 7fafbb9c9700 1 --2- 172.21.15.63:0/2055526816 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7fafa006b0b0 0x7fafa006d560 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fafb000b8b0 tx=0x7fafb0016040 comp rx=0 tx=0).stop 2024-03-22T07:17:04.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.739+0000 7fafbb9c9700 1 -- 172.21.15.63:0/2055526816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb40fe040 msgr2=0x7fafb410df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:04.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.739+0000 7fafbb9c9700 1 --2- 172.21.15.63:0/2055526816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb40fe040 0x7fafb410df30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fafa400eb70 tx=0x7fafa400b040 comp rx=0 tx=0).stop 2024-03-22T07:17:04.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.740+0000 7fafbb9c9700 1 -- 172.21.15.63:0/2055526816 shutdown_connections 2024-03-22T07:17:04.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.740+0000 7fafbb9c9700 1 --2- 172.21.15.63:0/2055526816 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7fafa006b0b0 0x7fafa006d560 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fafb000b8b0 tx=0x7fafb0016040 comp rx=0 tx=0).stop 2024-03-22T07:17:04.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.740+0000 7fafbb9c9700 1 --2- 172.21.15.63:0/2055526816 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fafb40fce40 0x7fafb410d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:04.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.740+0000 7fafbb9c9700 1 --2- 172.21.15.63:0/2055526816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fafb40fe040 0x7fafb410df30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:04.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.740+0000 7fafbb9c9700 1 -- 172.21.15.63:0/2055526816 >> 172.21.15.63:0/2055526816 conn(0x7fafb40f8570 msgr2=0x7fafb40f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:04.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.740+0000 7fafbb9c9700 1 -- 172.21.15.63:0/2055526816 shutdown_connections 2024-03-22T07:17:04.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:04.741+0000 7fafbb9c9700 1 -- 172.21.15.63:0/2055526816 wait complete. 2024-03-22T07:17:05.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:04 smithi063 ceph-mon[29200]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:05.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:04 smithi063 ceph-mon[29200]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:05.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:04 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/4034466647' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:05.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:04 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:05.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:04 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:05.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:04 smithi069 ceph-mon[36441]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:05.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:04 smithi069 ceph-mon[36441]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:05.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:04 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/4034466647' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:05.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:04 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:05.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:04 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:05.869 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:05 smithi063 ceph-mon[29200]: Reconfiguring prometheus.smithi063 (dependencies changed)... 2024-03-22T07:17:05.869 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:05 smithi063 ceph-mon[29200]: pgmap v35: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 399 op/s 2024-03-22T07:17:05.869 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:05 smithi063 ceph-mon[29200]: Reconfiguring daemon prometheus.smithi063 on smithi063 2024-03-22T07:17:05.869 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:05 smithi063 ceph-mon[29200]: from='client.24461 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:06.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:05 smithi069 ceph-mon[36441]: Reconfiguring prometheus.smithi063 (dependencies changed)... 2024-03-22T07:17:06.308 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:05 smithi069 ceph-mon[36441]: pgmap v35: 65 pgs: 65 active+clean; 304 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 399 op/s 2024-03-22T07:17:06.309 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:05 smithi069 ceph-mon[36441]: Reconfiguring daemon prometheus.smithi063 on smithi063 2024-03-22T07:17:06.309 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:05 smithi069 ceph-mon[36441]: from='client.24461 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:08.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:07 smithi063 ceph-mon[29200]: pgmap v36: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 508 op/s 2024-03-22T07:17:08.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:07 smithi069 ceph-mon[36441]: pgmap v36: 65 pgs: 65 active+clean; 310 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 508 op/s 2024-03-22T07:17:10.119 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:09 smithi063 ceph-mon[29200]: pgmap v37: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 607 op/s 2024-03-22T07:17:10.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:09 smithi069 ceph-mon[36441]: pgmap v37: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 607 op/s 2024-03-22T07:17:11.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:10 smithi063 ceph-mon[29200]: pgmap v38: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 501 op/s 2024-03-22T07:17:11.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:10 smithi069 ceph-mon[36441]: pgmap v38: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 501 op/s 2024-03-22T07:17:12.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:11 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:12.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:11 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:12.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:11 smithi063 ceph-mon[29200]: Reconfiguring node-exporter.smithi069 (dependencies changed)... 2024-03-22T07:17:12.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:11 smithi063 ceph-mon[29200]: Reconfiguring daemon node-exporter.smithi069 on smithi069 2024-03-22T07:17:12.208 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:11 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:12.209 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:11 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:12.209 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:11 smithi069 ceph-mon[36441]: Reconfiguring node-exporter.smithi069 (dependencies changed)... 2024-03-22T07:17:12.209 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:11 smithi069 ceph-mon[36441]: Reconfiguring daemon node-exporter.smithi069 on smithi069 2024-03-22T07:17:13.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:12 smithi063 ceph-mon[29200]: pgmap v39: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 698 op/s 2024-03-22T07:17:13.134 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:12 smithi069 ceph-mon[36441]: pgmap v39: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 698 op/s 2024-03-22T07:17:15.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:15 smithi069 ceph-mon[36441]: pgmap v40: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 456 op/s 2024-03-22T07:17:15.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:15 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:15.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:15 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:15.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:15 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:17:15.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:15 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:17:15.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:15 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:17:15.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:15 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:15.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:15 smithi063 ceph-mon[29200]: pgmap v40: 65 pgs: 65 active+clean; 306 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 456 op/s 2024-03-22T07:17:15.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:15 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:15.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:15 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:15.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:15 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:17:15.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:15 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:17:15.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:15 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:17:15.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:15 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:16.267 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-03-22T07:17:16.268 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-22T07:17:16.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:16 smithi063 ceph-mon[29200]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:17:16.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:16 smithi063 ceph-mon[29200]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:17:16.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:16 smithi063 ceph-mon[29200]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:17:16.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:16 smithi063 ceph-mon[29200]: Upgrade: Need to upgrade myself (mgr.smithi069.gfziwh) 2024-03-22T07:17:16.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:16 smithi063 ceph-mon[29200]: Upgrade: Need to upgrade myself (mgr.smithi069.gfziwh) 2024-03-22T07:17:16.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:16 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:17:16.709 DEBUG:teuthology.parallel:result is None 2024-03-22T07:17:16.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:16 smithi069 ceph-mon[36441]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:17:16.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:16 smithi069 ceph-mon[36441]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:17:16.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:16 smithi069 ceph-mon[36441]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:17:16.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:16 smithi069 ceph-mon[36441]: Upgrade: Need to upgrade myself (mgr.smithi069.gfziwh) 2024-03-22T07:17:16.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:16 smithi069 ceph-mon[36441]: Upgrade: Need to upgrade myself (mgr.smithi069.gfziwh) 2024-03-22T07:17:16.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:16 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:17:17.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:17 smithi069 ceph-mon[36441]: pgmap v41: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 529 op/s 2024-03-22T07:17:17.839 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:17 smithi063 ceph-mon[29200]: pgmap v41: 65 pgs: 65 active+clean; 302 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 529 op/s 2024-03-22T07:17:19.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:19 smithi069 ceph-mon[36441]: pgmap v42: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 620 op/s 2024-03-22T07:17:19.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:19 smithi063 ceph-mon[29200]: pgmap v42: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 620 op/s 2024-03-22T07:17:21.753 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:21 smithi069 ceph-mon[36441]: pgmap v43: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 469 op/s 2024-03-22T07:17:21.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:21 smithi063 ceph-mon[29200]: pgmap v43: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 469 op/s 2024-03-22T07:17:23.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:23 smithi069 ceph-mon[36441]: pgmap v44: 65 pgs: 65 active+clean; 303 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 630 op/s 2024-03-22T07:17:23.872 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:23 smithi063 ceph-mon[29200]: pgmap v44: 65 pgs: 65 active+clean; 303 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 630 op/s 2024-03-22T07:17:25.526 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-03-22T07:17:25.526 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-03-22T07:17:25.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:25 smithi069 ceph-mon[36441]: pgmap v45: 65 pgs: 65 active+clean; 303 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 433 op/s 2024-03-22T07:17:25.815 DEBUG:teuthology.parallel:result is None 2024-03-22T07:17:25.816 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-22T07:17:25.856 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:25 smithi063 ceph-mon[29200]: pgmap v45: 65 pgs: 65 active+clean; 303 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 433 op/s 2024-03-22T07:17:25.868 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-22T07:17:25.869 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-03-22T07:17:25.942 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-03-22T07:17:25.942 DEBUG:teuthology.parallel:result is None 2024-03-22T07:17:27.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:27 smithi069 ceph-mon[36441]: pgmap v46: 65 pgs: 65 active+clean; 301 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 481 op/s 2024-03-22T07:17:27.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:27 smithi063 ceph-mon[29200]: pgmap v46: 65 pgs: 65 active+clean; 301 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 481 op/s 2024-03-22T07:17:29.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:29 smithi063 ceph-mon[29200]: pgmap v47: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 497 op/s 2024-03-22T07:17:29.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:29 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:29.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:29 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi063.mnxayh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:17:29.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:29 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:17:29.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:29 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:17:29.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:29 smithi069 ceph-mon[36441]: pgmap v47: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 497 op/s 2024-03-22T07:17:29.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:29 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:29.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:29 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi063.mnxayh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:17:29.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:29 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:17:29.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:29 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:17:30.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:30 smithi069 ceph-mon[36441]: Upgrade: Updating mgr.smithi063.mnxayh 2024-03-22T07:17:30.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:30 smithi069 ceph-mon[36441]: Deploying daemon mgr.smithi063.mnxayh on smithi063 2024-03-22T07:17:30.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:30 smithi063 ceph-mon[29200]: Upgrade: Updating mgr.smithi063.mnxayh 2024-03-22T07:17:30.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:30 smithi063 ceph-mon[29200]: Deploying daemon mgr.smithi063.mnxayh on smithi063 2024-03-22T07:17:31.563 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:31 smithi063 ceph-mon[29200]: pgmap v48: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 297 op/s 2024-03-22T07:17:31.563 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:31 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:17:31.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:31 smithi069 ceph-mon[36441]: pgmap v48: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 297 op/s 2024-03-22T07:17:31.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:31 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:17:33.689 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:33 smithi063 ceph-mon[29200]: pgmap v49: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 301 op/s 2024-03-22T07:17:33.689 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:33 smithi063 ceph-mon[29200]: mgrmap e25: smithi069.gfziwh(active, since 92s), standbys: smithi063.mnxayh 2024-03-22T07:17:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:33 smithi069 ceph-mon[36441]: pgmap v49: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 301 op/s 2024-03-22T07:17:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:33 smithi069 ceph-mon[36441]: mgrmap e25: smithi069.gfziwh(active, since 92s), standbys: smithi063.mnxayh 2024-03-22T07:17:34.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.876+0000 7fce15c46700 1 -- 172.21.15.63:0/272445305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce100fed50 msgr2=0x7fce100fae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:34.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.876+0000 7fce15c46700 1 --2- 172.21.15.63:0/272445305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce100fed50 0x7fce100fae90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fce04004940 tx=0x7fce04009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.876+0000 7fce15c46700 1 -- 172.21.15.63:0/272445305 shutdown_connections 2024-03-22T07:17:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.876+0000 7fce15c46700 1 --2- 172.21.15.63:0/272445305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce100fed50 0x7fce100fae90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.876+0000 7fce15c46700 1 --2- 172.21.15.63:0/272445305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce100fe400 0x7fce100fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.876+0000 7fce15c46700 1 -- 172.21.15.63:0/272445305 >> 172.21.15.63:0/272445305 conn(0x7fce100f6ba0 msgr2=0x7fce100f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.876+0000 7fce15c46700 1 -- 172.21.15.63:0/272445305 shutdown_connections 2024-03-22T07:17:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.876+0000 7fce15c46700 1 -- 172.21.15.63:0/272445305 wait complete. 2024-03-22T07:17:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.877+0000 7fce15c46700 1 Processor -- start 2024-03-22T07:17:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.877+0000 7fce15c46700 1 -- start start 2024-03-22T07:17:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.877+0000 7fce15c46700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce100fe400 0x7fce1010d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.877+0000 7fce15c46700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce100fed50 0x7fce1010de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.877+0000 7fce15c46700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1010e490 con 0x7fce100fed50 2024-03-22T07:17:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.877+0000 7fce15c46700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1010ece0 con 0x7fce100fe400 2024-03-22T07:17:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.877+0000 7fce0effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce100fed50 0x7fce1010de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:34.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.877+0000 7fce0effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce100fed50 0x7fce1010de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49508/0 (socket says 172.21.15.63:49508) 2024-03-22T07:17:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.877+0000 7fce0effd700 1 -- 172.21.15.63:0/2048352482 learned_addr learned my addr 172.21.15.63:0/2048352482 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:17:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.877+0000 7fce0f7fe700 1 --2- 172.21.15.63:0/2048352482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce100fe400 0x7fce1010d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.878+0000 7fce0effd700 1 -- 172.21.15.63:0/2048352482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce100fe400 msgr2=0x7fce1010d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.878+0000 7fce0effd700 1 --2- 172.21.15.63:0/2048352482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce100fe400 0x7fce1010d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.878+0000 7fce0effd700 1 -- 172.21.15.63:0/2048352482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce04005040 con 0x7fce100fed50 2024-03-22T07:17:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.878+0000 7fce0f7fe700 1 --2- 172.21.15.63:0/2048352482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce100fe400 0x7fce1010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:17:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.878+0000 7fce0effd700 1 --2- 172.21.15.63:0/2048352482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce100fed50 0x7fce1010de90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fce04013870 tx=0x7fce04009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.878+0000 7fcdf7fff700 1 -- 172.21.15.63:0/2048352482 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce0400e070 con 0x7fce100fed50 2024-03-22T07:17:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.878+0000 7fce15c46700 1 -- 172.21.15.63:0/2048352482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1010ef60 con 0x7fce100fed50 2024-03-22T07:17:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.878+0000 7fce15c46700 1 -- 172.21.15.63:0/2048352482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1010f440 con 0x7fce100fed50 2024-03-22T07:17:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.878+0000 7fcdf7fff700 1 -- 172.21.15.63:0/2048352482 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce04013de0 con 0x7fce100fed50 2024-03-22T07:17:34.880 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.878+0000 7fcdf7fff700 1 -- 172.21.15.63:0/2048352482 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce0401cea0 con 0x7fce100fed50 2024-03-22T07:17:34.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.880+0000 7fce15c46700 1 -- 172.21.15.63:0/2048352482 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1005f5f0 con 0x7fce100fed50 2024-03-22T07:17:34.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.880+0000 7fcdf7fff700 1 -- 172.21.15.63:0/2048352482 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 88751+0+0 (secure 0 0 0) 0x7fce040264a0 con 0x7fce100fed50 2024-03-22T07:17:34.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.880+0000 7fcdf7fff700 1 --2- 172.21.15.63:0/2048352482 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7fcdf806afa0 0x7fcdf806d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:34.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.881+0000 7fcdf7fff700 1 -- 172.21.15.63:0/2048352482 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fce0408aeb0 con 0x7fce100fed50 2024-03-22T07:17:34.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.881+0000 7fce0f7fe700 1 --2- 172.21.15.63:0/2048352482 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7fcdf806afa0 0x7fcdf806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:34.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.882+0000 7fce0f7fe700 1 --2- 172.21.15.63:0/2048352482 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7fcdf806afa0 0x7fcdf806d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fce00009b80 tx=0x7fce0000b040 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:34.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:34.885+0000 7fcdf7fff700 1 -- 172.21.15.63:0/2048352482 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fce04059f40 con 0x7fce100fed50 2024-03-22T07:17:35.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:34 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:35.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:34 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:35.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:34 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:35.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:34 smithi069 ceph-mon[36441]: pgmap v50: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 530 KiB/s rd, 457 KiB/s wr, 140 op/s 2024-03-22T07:17:35.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:34 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:35.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:34 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:35.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.120+0000 7fce15c46700 1 -- 172.21.15.63:0/2048352482 --> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fce1010f720 con 0x7fcdf806afa0 2024-03-22T07:17:35.120 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:34 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:35.120 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:34 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:35.120 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:34 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:35.120 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:34 smithi063 ceph-mon[29200]: pgmap v50: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 530 KiB/s rd, 457 KiB/s wr, 140 op/s 2024-03-22T07:17:35.120 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:34 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:35.121 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:34 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:35.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.121+0000 7fcdf7fff700 1 -- 172.21.15.63:0/2048352482 <== mgr.24385 v2:172.21.15.69:6828/1862357042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fce1010f720 con 0x7fcdf806afa0 2024-03-22T07:17:35.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.123+0000 7fce15c46700 1 -- 172.21.15.63:0/2048352482 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7fcdf806afa0 msgr2=0x7fcdf806d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:35.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.123+0000 7fce15c46700 1 --2- 172.21.15.63:0/2048352482 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7fcdf806afa0 0x7fcdf806d450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fce00009b80 tx=0x7fce0000b040 comp rx=0 tx=0).stop 2024-03-22T07:17:35.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.123+0000 7fce15c46700 1 -- 172.21.15.63:0/2048352482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce100fed50 msgr2=0x7fce1010de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:35.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.123+0000 7fce15c46700 1 --2- 172.21.15.63:0/2048352482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce100fed50 0x7fce1010de90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fce04013870 tx=0x7fce04009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:35.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.123+0000 7fce15c46700 1 -- 172.21.15.63:0/2048352482 shutdown_connections 2024-03-22T07:17:35.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.123+0000 7fce15c46700 1 --2- 172.21.15.63:0/2048352482 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7fcdf806afa0 0x7fcdf806d450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.123+0000 7fce15c46700 1 --2- 172.21.15.63:0/2048352482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce100fe400 0x7fce1010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.123+0000 7fce15c46700 1 --2- 172.21.15.63:0/2048352482 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce100fed50 0x7fce1010de90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.123+0000 7fce15c46700 1 -- 172.21.15.63:0/2048352482 >> 172.21.15.63:0/2048352482 conn(0x7fce100f6ba0 msgr2=0x7fce100f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:35.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.123+0000 7fce15c46700 1 -- 172.21.15.63:0/2048352482 shutdown_connections 2024-03-22T07:17:35.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.123+0000 7fce15c46700 1 -- 172.21.15.63:0/2048352482 wait complete. 2024-03-22T07:17:35.134 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:17:35.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.230+0000 7f98ca898700 1 -- 172.21.15.63:0/1409911057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98c40fcf80 msgr2=0x7f98c40fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:35.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.230+0000 7f98ca898700 1 --2- 172.21.15.63:0/1409911057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98c40fcf80 0x7f98c40fd3f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f98b8004940 tx=0x7f98b8009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:35.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.230+0000 7f98ca898700 1 -- 172.21.15.63:0/1409911057 shutdown_connections 2024-03-22T07:17:35.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.230+0000 7f98ca898700 1 --2- 172.21.15.63:0/1409911057 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98c40fcf80 0x7f98c40fd3f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.230+0000 7f98ca898700 1 --2- 172.21.15.63:0/1409911057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98c40fbcd0 0x7f98c40fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.230+0000 7f98ca898700 1 -- 172.21.15.63:0/1409911057 >> 172.21.15.63:0/1409911057 conn(0x7f98c40f7430 msgr2=0x7f98c40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:35.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.230+0000 7f98ca898700 1 -- 172.21.15.63:0/1409911057 shutdown_connections 2024-03-22T07:17:35.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.230+0000 7f98ca898700 1 -- 172.21.15.63:0/1409911057 wait complete. 2024-03-22T07:17:35.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.231+0000 7f98ca898700 1 Processor -- start 2024-03-22T07:17:35.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.231+0000 7f98ca898700 1 -- start start 2024-03-22T07:17:35.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.231+0000 7f98ca898700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98c40fbcd0 0x7f98c410da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:35.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.231+0000 7f98ca898700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98c40fcf80 0x7f98c410df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.231+0000 7f98ca898700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c410e540 con 0x7f98c40fbcd0 2024-03-22T07:17:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.231+0000 7f98ca898700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c410e6b0 con 0x7f98c40fcf80 2024-03-22T07:17:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.231+0000 7f98c3fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98c40fbcd0 0x7f98c410da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.231+0000 7f98c3fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98c40fbcd0 0x7f98c410da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49518/0 (socket says 172.21.15.63:49518) 2024-03-22T07:17:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.231+0000 7f98c3fff700 1 -- 172.21.15.63:0/1959257457 learned_addr learned my addr 172.21.15.63:0/1959257457 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:17:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.231+0000 7f98c37fe700 1 --2- 172.21.15.63:0/1959257457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98c40fcf80 0x7f98c410df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.232+0000 7f98c3fff700 1 -- 172.21.15.63:0/1959257457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98c40fcf80 msgr2=0x7f98c410df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.232+0000 7f98c3fff700 1 --2- 172.21.15.63:0/1959257457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98c40fcf80 0x7f98c410df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.232+0000 7f98c3fff700 1 -- 172.21.15.63:0/1959257457 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b8005040 con 0x7f98c40fbcd0 2024-03-22T07:17:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.232+0000 7f98c37fe700 1 --2- 172.21.15.63:0/1959257457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98c40fcf80 0x7f98c410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:17:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.232+0000 7f98c3fff700 1 --2- 172.21.15.63:0/1959257457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98c40fbcd0 0x7f98c410da00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f98b4002a50 tx=0x7f98b40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.232+0000 7f98c0ff9700 1 -- 172.21.15.63:0/1959257457 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98b4007cc0 con 0x7f98c40fbcd0 2024-03-22T07:17:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.232+0000 7f98c0ff9700 1 -- 172.21.15.63:0/1959257457 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98b400f040 con 0x7f98c40fbcd0 2024-03-22T07:17:35.233 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.232+0000 7f98ca898700 1 -- 172.21.15.63:0/1959257457 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c4113070 con 0x7f98c40fbcd0 2024-03-22T07:17:35.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.232+0000 7f98c0ff9700 1 -- 172.21.15.63:0/1959257457 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98b4009640 con 0x7f98c40fbcd0 2024-03-22T07:17:35.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.232+0000 7f98ca898700 1 -- 172.21.15.63:0/1959257457 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c4113630 con 0x7f98c40fbcd0 2024-03-22T07:17:35.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.233+0000 7f98ca898700 1 -- 172.21.15.63:0/1959257457 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c4101b10 con 0x7f98c40fbcd0 2024-03-22T07:17:35.234 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.234+0000 7f98c0ff9700 1 -- 172.21.15.63:0/1959257457 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 88751+0+0 (secure 0 0 0) 0x7f98b40097c0 con 0x7f98c40fbcd0 2024-03-22T07:17:35.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.234+0000 7f98c0ff9700 1 --2- 172.21.15.63:0/1959257457 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f98ac06af90 0x7f98ac06d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:35.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.234+0000 7f98c0ff9700 1 -- 172.21.15.63:0/1959257457 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f98b4089cc0 con 0x7f98c40fbcd0 2024-03-22T07:17:35.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.234+0000 7f98c37fe700 1 --2- 172.21.15.63:0/1959257457 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f98ac06af90 0x7f98ac06d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:35.235 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.235+0000 7f98c37fe700 1 --2- 172.21.15.63:0/1959257457 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f98ac06af90 0x7f98ac06d440 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f98b800b970 tx=0x7f98b8009e30 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:35.238 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.238+0000 7f98c0ff9700 1 -- 172.21.15.63:0/1959257457 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f98b4057c60 con 0x7f98c40fbcd0 2024-03-22T07:17:35.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.468+0000 7f98ca898700 1 -- 172.21.15.63:0/1959257457 --> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f98c402b3b0 con 0x7f98ac06af90 2024-03-22T07:17:35.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.469+0000 7f98c0ff9700 1 -- 172.21.15.63:0/1959257457 <== mgr.24385 v2:172.21.15.69:6828/1862357042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f98c402b3b0 con 0x7f98ac06af90 2024-03-22T07:17:35.471 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.471+0000 7f98ca898700 1 -- 172.21.15.63:0/1959257457 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f98ac06af90 msgr2=0x7f98ac06d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:35.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.471+0000 7f98ca898700 1 --2- 172.21.15.63:0/1959257457 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f98ac06af90 0x7f98ac06d440 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f98b800b970 tx=0x7f98b8009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:35.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.471+0000 7f98ca898700 1 -- 172.21.15.63:0/1959257457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98c40fbcd0 msgr2=0x7f98c410da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:35.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.471+0000 7f98ca898700 1 --2- 172.21.15.63:0/1959257457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98c40fbcd0 0x7f98c410da00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f98b4002a50 tx=0x7f98b40046b0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.472+0000 7f98ca898700 1 -- 172.21.15.63:0/1959257457 shutdown_connections 2024-03-22T07:17:35.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.472+0000 7f98ca898700 1 --2- 172.21.15.63:0/1959257457 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f98ac06af90 0x7f98ac06d440 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.472+0000 7f98ca898700 1 --2- 172.21.15.63:0/1959257457 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f98c40fbcd0 0x7f98c410da00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.472+0000 7f98ca898700 1 --2- 172.21.15.63:0/1959257457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98c40fcf80 0x7f98c410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.472+0000 7f98ca898700 1 -- 172.21.15.63:0/1959257457 >> 172.21.15.63:0/1959257457 conn(0x7f98c40f7430 msgr2=0x7f98c4066800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:35.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.472+0000 7f98ca898700 1 -- 172.21.15.63:0/1959257457 shutdown_connections 2024-03-22T07:17:35.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.472+0000 7f98ca898700 1 -- 172.21.15.63:0/1959257457 wait complete. 2024-03-22T07:17:35.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.577+0000 7f61842f0700 1 -- 172.21.15.63:0/129960129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f617c0fbcd0 msgr2=0x7f617c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:35.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.577+0000 7f61842f0700 1 --2- 172.21.15.63:0/129960129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f617c0fbcd0 0x7f617c0fc0e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6178004940 tx=0x7f6178009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:35.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.577+0000 7f61842f0700 1 -- 172.21.15.63:0/129960129 shutdown_connections 2024-03-22T07:17:35.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.577+0000 7f61842f0700 1 --2- 172.21.15.63:0/129960129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f617c0fcf80 0x7f617c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.577+0000 7f61842f0700 1 --2- 172.21.15.63:0/129960129 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f617c0fbcd0 0x7f617c0fc0e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.577+0000 7f61842f0700 1 -- 172.21.15.63:0/129960129 >> 172.21.15.63:0/129960129 conn(0x7f617c0f7430 msgr2=0x7f617c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:35.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.577+0000 7f61842f0700 1 -- 172.21.15.63:0/129960129 shutdown_connections 2024-03-22T07:17:35.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.577+0000 7f61842f0700 1 -- 172.21.15.63:0/129960129 wait complete. 2024-03-22T07:17:35.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.578+0000 7f61842f0700 1 Processor -- start 2024-03-22T07:17:35.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.578+0000 7f61842f0700 1 -- start start 2024-03-22T07:17:35.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.578+0000 7f61842f0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f617c0fbcd0 0x7f617c110670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:35.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.578+0000 7f61842f0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f617c0fcf80 0x7f617c112bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:35.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.578+0000 7f61842f0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617c066370 con 0x7f617c0fbcd0 2024-03-22T07:17:35.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.578+0000 7f61842f0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617c0664b0 con 0x7f617c0fcf80 2024-03-22T07:17:35.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.578+0000 7f618208c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f617c0fbcd0 0x7f617c110670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:35.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.578+0000 7f618208c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f617c0fbcd0 0x7f617c110670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49536/0 (socket says 172.21.15.63:49536) 2024-03-22T07:17:35.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.578+0000 7f618208c700 1 -- 172.21.15.63:0/3509723664 learned_addr learned my addr 172.21.15.63:0/3509723664 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:17:35.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.578+0000 7f618188b700 1 --2- 172.21.15.63:0/3509723664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f617c0fcf80 0x7f617c112bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:35.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.579+0000 7f618208c700 1 -- 172.21.15.63:0/3509723664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f617c0fcf80 msgr2=0x7f617c112bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:35.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.579+0000 7f618208c700 1 --2- 172.21.15.63:0/3509723664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f617c0fcf80 0x7f617c112bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.579+0000 7f618208c700 1 -- 172.21.15.63:0/3509723664 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c005040 con 0x7f617c0fbcd0 2024-03-22T07:17:35.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.579+0000 7f618188b700 1 --2- 172.21.15.63:0/3509723664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f617c0fcf80 0x7f617c112bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:17:35.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.579+0000 7f618208c700 1 --2- 172.21.15.63:0/3509723664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f617c0fbcd0 0x7f617c110670 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f617800b0d0 tx=0x7f6178009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:35.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.579+0000 7f6172ffd700 1 -- 172.21.15.63:0/3509723664 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f617800e030 con 0x7f617c0fbcd0 2024-03-22T07:17:35.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.579+0000 7f6172ffd700 1 -- 172.21.15.63:0/3509723664 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f617801c400 con 0x7f617c0fbcd0 2024-03-22T07:17:35.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.579+0000 7f61842f0700 1 -- 172.21.15.63:0/3509723664 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6178005040 con 0x7f617c0fbcd0 2024-03-22T07:17:35.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.579+0000 7f6172ffd700 1 -- 172.21.15.63:0/3509723664 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f617800b630 con 0x7f617c0fbcd0 2024-03-22T07:17:35.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.579+0000 7f61842f0700 1 -- 172.21.15.63:0/3509723664 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617c113560 con 0x7f617c0fbcd0 2024-03-22T07:17:35.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.580+0000 7f61842f0700 1 -- 172.21.15.63:0/3509723664 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f617c05f5f0 con 0x7f617c0fbcd0 2024-03-22T07:17:35.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.581+0000 7f6172ffd700 1 -- 172.21.15.63:0/3509723664 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 88751+0+0 (secure 0 0 0) 0x7f617801c570 con 0x7f617c0fbcd0 2024-03-22T07:17:35.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.581+0000 7f6172ffd700 1 --2- 172.21.15.63:0/3509723664 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f616806af90 0x7f616806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:35.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.581+0000 7f6172ffd700 1 -- 172.21.15.63:0/3509723664 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f617808b140 con 0x7f617c0fbcd0 2024-03-22T07:17:35.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.582+0000 7f618188b700 1 --2- 172.21.15.63:0/3509723664 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f616806af90 0x7f616806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:35.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.582+0000 7f618188b700 1 --2- 172.21.15.63:0/3509723664 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f616806af90 0x7f616806d440 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f616c009fd0 tx=0x7f616c00b040 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:35.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.586+0000 7f6172ffd700 1 -- 172.21.15.63:0/3509723664 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f61780556d0 con 0x7f617c0fbcd0 2024-03-22T07:17:35.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.834+0000 7f61842f0700 1 -- 172.21.15.63:0/3509723664 --> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f617c1012f0 con 0x7f616806af90 2024-03-22T07:17:35.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.844+0000 7f6172ffd700 1 -- 172.21.15.63:0/3509723664 <== mgr.24385 v2:172.21.15.69:6828/1862357042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f617c1012f0 con 0x7f616806af90 2024-03-22T07:17:35.844 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:17:35.844 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 starting - - - - 2024-03-22T07:17:35.844 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (9m) 81s ago 9m 6660k - 17.2.7-726-g9c4e0c34 837124da804f d6c6bbeb6904 2024-03-22T07:17:35.844 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (8m) 79s ago 8m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 30992f8aa88c 2024-03-22T07:17:35.844 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 starting - - - - 2024-03-22T07:17:35.844 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (4m) 81s ago 4m 90.3M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:17:35.844 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (3m) 81s ago 3m 152M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:17:35.844 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (3m) 79s ago 3m 78.8M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:17:35.844 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (3m) 79s ago 3m 135M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:17:35.845 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:8443,9283,8765 starting - - - - 2024-03-22T07:17:35.845 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283 running (2m) 79s ago 8m 491M - 19.0.0-1667-gdb0330b1 4f325f8440e9 5ad05e49e512 2024-03-22T07:17:35.845 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (10m) 81s ago 10m 51.4M 2048M 17.2.7-726-g9c4e0c34 837124da804f f92a5bb6caee 2024-03-22T07:17:35.845 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (8m) 79s ago 8m 38.1M 2048M 17.2.7-726-g9c4e0c34 837124da804f 95f6103f7603 2024-03-22T07:17:35.845 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 starting - - - - 2024-03-22T07:17:35.845 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 starting - - - - 2024-03-22T07:17:35.845 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (7m) 81s ago 7m 251M 1557M 17.2.7-726-g9c4e0c34 837124da804f 580f22188a10 2024-03-22T07:17:35.845 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (7m) 81s ago 7m 294M 1557M 17.2.7-726-g9c4e0c34 837124da804f fe1ace35c88a 2024-03-22T07:17:35.845 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (6m) 81s ago 6m 223M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4cd995265dcf 2024-03-22T07:17:35.845 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (6m) 79s ago 6m 298M 2581M 17.2.7-726-g9c4e0c34 837124da804f 45c9a759b76d 2024-03-22T07:17:35.845 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (5m) 79s ago 5m 265M 2581M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:17:35.845 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (5m) 79s ago 5m 299M 2581M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:17:35.845 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 starting - - - - 2024-03-22T07:17:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.846+0000 7f61842f0700 1 -- 172.21.15.63:0/3509723664 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f616806af90 msgr2=0x7f616806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.847+0000 7f61842f0700 1 --2- 172.21.15.63:0/3509723664 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f616806af90 0x7f616806d440 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f616c009fd0 tx=0x7f616c00b040 comp rx=0 tx=0).stop 2024-03-22T07:17:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.847+0000 7f61842f0700 1 -- 172.21.15.63:0/3509723664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f617c0fbcd0 msgr2=0x7f617c110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.847+0000 7f61842f0700 1 --2- 172.21.15.63:0/3509723664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f617c0fbcd0 0x7f617c110670 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f617800b0d0 tx=0x7f6178009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.847+0000 7f61842f0700 1 -- 172.21.15.63:0/3509723664 shutdown_connections 2024-03-22T07:17:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.847+0000 7f61842f0700 1 --2- 172.21.15.63:0/3509723664 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f616806af90 0x7f616806d440 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.847+0000 7f61842f0700 1 --2- 172.21.15.63:0/3509723664 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f617c0fbcd0 0x7f617c110670 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.847+0000 7f61842f0700 1 --2- 172.21.15.63:0/3509723664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f617c0fcf80 0x7f617c112bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.847+0000 7f61842f0700 1 -- 172.21.15.63:0/3509723664 >> 172.21.15.63:0/3509723664 conn(0x7f617c0f7430 msgr2=0x7f617c0f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.847+0000 7f61842f0700 1 -- 172.21.15.63:0/3509723664 shutdown_connections 2024-03-22T07:17:35.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.847+0000 7f61842f0700 1 -- 172.21.15.63:0/3509723664 wait complete. 2024-03-22T07:17:35.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.953+0000 7f09a5e54700 1 -- 172.21.15.63:0/2288960726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a00662e0 msgr2=0x7f09a00ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:35.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.953+0000 7f09a5e54700 1 --2- 172.21.15.63:0/2288960726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a00662e0 0x7f09a00ff9c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0990004940 tx=0x7f0990009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:35.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.953+0000 7f09a5e54700 1 -- 172.21.15.63:0/2288960726 shutdown_connections 2024-03-22T07:17:35.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.953+0000 7f09a5e54700 1 --2- 172.21.15.63:0/2288960726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09a00fff00 0x7f09a0102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.953+0000 7f09a5e54700 1 --2- 172.21.15.63:0/2288960726 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a00662e0 0x7f09a00ff9c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.953+0000 7f09a5e54700 1 -- 172.21.15.63:0/2288960726 >> 172.21.15.63:0/2288960726 conn(0x7f09a00f7430 msgr2=0x7f09a00f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:35.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.954+0000 7f09a5e54700 1 -- 172.21.15.63:0/2288960726 shutdown_connections 2024-03-22T07:17:35.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.954+0000 7f09a5e54700 1 -- 172.21.15.63:0/2288960726 wait complete. 2024-03-22T07:17:35.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.954+0000 7f09a5e54700 1 Processor -- start 2024-03-22T07:17:35.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.954+0000 7f09a5e54700 1 -- start start 2024-03-22T07:17:35.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.955+0000 7f09a5e54700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09a00662e0 0x7f09a010da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:35.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.955+0000 7f09a5e54700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a00fff00 0x7f09a010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:35.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.955+0000 7f09a5e54700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a010e570 con 0x7f09a00fff00 2024-03-22T07:17:35.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.955+0000 7f09a5e54700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a010edc0 con 0x7f09a00662e0 2024-03-22T07:17:35.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.955+0000 7f099effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a00fff00 0x7f09a010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:35.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.955+0000 7f099effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a00fff00 0x7f09a010df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49556/0 (socket says 172.21.15.63:49556) 2024-03-22T07:17:35.956 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.955+0000 7f099effd700 1 -- 172.21.15.63:0/3279051500 learned_addr learned my addr 172.21.15.63:0/3279051500 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:17:35.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.955+0000 7f099f7fe700 1 --2- 172.21.15.63:0/3279051500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09a00662e0 0x7f09a010da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:35.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.955+0000 7f099effd700 1 -- 172.21.15.63:0/3279051500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09a00662e0 msgr2=0x7f09a010da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:35.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.955+0000 7f099effd700 1 --2- 172.21.15.63:0/3279051500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09a00662e0 0x7f09a010da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:35.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.955+0000 7f099effd700 1 -- 172.21.15.63:0/3279051500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0990005040 con 0x7f09a00fff00 2024-03-22T07:17:35.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.955+0000 7f099effd700 1 --2- 172.21.15.63:0/3279051500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a00fff00 0x7f09a010df70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0994002a50 tx=0x7f09940046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:35.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.956+0000 7f0987fff700 1 -- 172.21.15.63:0/3279051500 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0994007cc0 con 0x7f09a00fff00 2024-03-22T07:17:35.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.956+0000 7f0987fff700 1 -- 172.21.15.63:0/3279051500 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f099400f040 con 0x7f09a00fff00 2024-03-22T07:17:35.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.956+0000 7f09a5e54700 1 -- 172.21.15.63:0/3279051500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a010f0a0 con 0x7f09a00fff00 2024-03-22T07:17:35.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.956+0000 7f0987fff700 1 -- 172.21.15.63:0/3279051500 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f099400da30 con 0x7f09a00fff00 2024-03-22T07:17:35.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.956+0000 7f09a5e54700 1 -- 172.21.15.63:0/3279051500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a010f5e0 con 0x7f09a00fff00 2024-03-22T07:17:35.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.957+0000 7f0987fff700 1 -- 172.21.15.63:0/3279051500 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 88751+0+0 (secure 0 0 0) 0x7f099400dc70 con 0x7f09a00fff00 2024-03-22T07:17:35.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.957+0000 7f09a5e54700 1 -- 172.21.15.63:0/3279051500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a005f5f0 con 0x7f09a00fff00 2024-03-22T07:17:35.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.957+0000 7f0987fff700 1 --2- 172.21.15.63:0/3279051500 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f098806af90 0x7f098806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:35.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.958+0000 7f0987fff700 1 -- 172.21.15.63:0/3279051500 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f099408d170 con 0x7f09a00fff00 2024-03-22T07:17:35.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.958+0000 7f099f7fe700 1 --2- 172.21.15.63:0/3279051500 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f098806af90 0x7f098806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:35.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.958+0000 7f099f7fe700 1 --2- 172.21.15.63:0/3279051500 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f098806af90 0x7f098806d440 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f099000b970 tx=0x7f0990009e30 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:35.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:35.962+0000 7f0987fff700 1 -- 172.21.15.63:0/3279051500 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f099408e060 con 0x7f09a00fff00 2024-03-22T07:17:36.102 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:35 smithi063 ceph-mon[29200]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:36.102 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:35 smithi063 ceph-mon[29200]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:36.190 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:35 smithi069 ceph-mon[36441]: from='client.14672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:36.190 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:35 smithi069 ceph-mon[36441]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:36.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.258+0000 7f09a5e54700 1 -- 172.21.15.63:0/3279051500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f09a01058c0 con 0x7f09a00fff00 2024-03-22T07:17:36.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.258+0000 7f0987fff700 1 -- 172.21.15.63:0/3279051500 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f0994056d90 con 0x7f09a00fff00 2024-03-22T07:17:36.258 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:17:36.259 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:17:36.259 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:17:36.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:17:36.259 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:17:36.259 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T07:17:36.259 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-22T07:17:36.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:17:36.259 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:17:36.259 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:17:36.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:17:36.259 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:17:36.259 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:17:36.259 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:17:36.260 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:17:36.260 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 13, 2024-03-22T07:17:36.260 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-22T07:17:36.260 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:17:36.260 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:17:36.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.261+0000 7f09a5e54700 1 -- 172.21.15.63:0/3279051500 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f098806af90 msgr2=0x7f098806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:36.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.261+0000 7f09a5e54700 1 --2- 172.21.15.63:0/3279051500 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f098806af90 0x7f098806d440 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f099000b970 tx=0x7f0990009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:36.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.261+0000 7f09a5e54700 1 -- 172.21.15.63:0/3279051500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a00fff00 msgr2=0x7f09a010df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:36.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.261+0000 7f09a5e54700 1 --2- 172.21.15.63:0/3279051500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a00fff00 0x7f09a010df70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0994002a50 tx=0x7f09940046b0 comp rx=0 tx=0).stop 2024-03-22T07:17:36.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.261+0000 7f09a5e54700 1 -- 172.21.15.63:0/3279051500 shutdown_connections 2024-03-22T07:17:36.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.261+0000 7f09a5e54700 1 --2- 172.21.15.63:0/3279051500 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f098806af90 0x7f098806d440 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:36.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.261+0000 7f09a5e54700 1 --2- 172.21.15.63:0/3279051500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09a00662e0 0x7f09a010da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:36.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.261+0000 7f09a5e54700 1 --2- 172.21.15.63:0/3279051500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f09a00fff00 0x7f09a010df70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:36.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.261+0000 7f09a5e54700 1 -- 172.21.15.63:0/3279051500 >> 172.21.15.63:0/3279051500 conn(0x7f09a00f7430 msgr2=0x7f09a00f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:36.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.261+0000 7f09a5e54700 1 -- 172.21.15.63:0/3279051500 shutdown_connections 2024-03-22T07:17:36.262 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.261+0000 7f09a5e54700 1 -- 172.21.15.63:0/3279051500 wait complete. 2024-03-22T07:17:36.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.368+0000 7f302c953700 1 -- 172.21.15.63:0/3159582553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30280fff40 msgr2=0x7f3028102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:36.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.368+0000 7f302c953700 1 --2- 172.21.15.63:0/3159582553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30280fff40 0x7f3028102320 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3010004940 tx=0x7f3010009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:36.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.368+0000 7f302c953700 1 -- 172.21.15.63:0/3159582553 shutdown_connections 2024-03-22T07:17:36.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.368+0000 7f302c953700 1 --2- 172.21.15.63:0/3159582553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30280fff40 0x7f3028102320 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:36.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.368+0000 7f302c953700 1 --2- 172.21.15.63:0/3159582553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3028066b30 0x7f30280ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:36.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.368+0000 7f302c953700 1 -- 172.21.15.63:0/3159582553 >> 172.21.15.63:0/3159582553 conn(0x7f30280f7410 msgr2=0x7f30280f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:36.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.368+0000 7f302c953700 1 -- 172.21.15.63:0/3159582553 shutdown_connections 2024-03-22T07:17:36.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.368+0000 7f302c953700 1 -- 172.21.15.63:0/3159582553 wait complete. 2024-03-22T07:17:36.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.369+0000 7f302c953700 1 Processor -- start 2024-03-22T07:17:36.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.369+0000 7f302c953700 1 -- start start 2024-03-22T07:17:36.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.369+0000 7f302c953700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3028066b30 0x7f302810da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:36.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.370+0000 7f302c953700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30280fff40 0x7f302810df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:36.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.370+0000 7f302c953700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302810e580 con 0x7f30280fff40 2024-03-22T07:17:36.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.370+0000 7f302c953700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302810ed80 con 0x7f3028066b30 2024-03-22T07:17:36.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.370+0000 7f301dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30280fff40 0x7f302810df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:36.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.370+0000 7f301dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30280fff40 0x7f302810df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:49562/0 (socket says 172.21.15.63:49562) 2024-03-22T07:17:36.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.370+0000 7f302659c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3028066b30 0x7f302810da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:36.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.370+0000 7f301dd9b700 1 -- 172.21.15.63:0/3066045553 learned_addr learned my addr 172.21.15.63:0/3066045553 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:17:36.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.370+0000 7f301dd9b700 1 -- 172.21.15.63:0/3066045553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3028066b30 msgr2=0x7f302810da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:36.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.370+0000 7f301dd9b700 1 --2- 172.21.15.63:0/3066045553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3028066b30 0x7f302810da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:36.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.370+0000 7f301dd9b700 1 -- 172.21.15.63:0/3066045553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3010005040 con 0x7f30280fff40 2024-03-22T07:17:36.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.370+0000 7f301dd9b700 1 --2- 172.21.15.63:0/3066045553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30280fff40 0x7f302810df80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f301000b3b0 tx=0x7f3010009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:36.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.370+0000 7f301f7fe700 1 -- 172.21.15.63:0/3066045553 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f301000e040 con 0x7f30280fff40 2024-03-22T07:17:36.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.370+0000 7f301f7fe700 1 -- 172.21.15.63:0/3066045553 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f301000bbf0 con 0x7f30280fff40 2024-03-22T07:17:36.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.370+0000 7f302c953700 1 -- 172.21.15.63:0/3066045553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302810f000 con 0x7f30280fff40 2024-03-22T07:17:36.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.370+0000 7f301f7fe700 1 -- 172.21.15.63:0/3066045553 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f301001c630 con 0x7f30280fff40 2024-03-22T07:17:36.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.371+0000 7f302c953700 1 -- 172.21.15.63:0/3066045553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302810f540 con 0x7f30280fff40 2024-03-22T07:17:36.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.372+0000 7f302c953700 1 -- 172.21.15.63:0/3066045553 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f302805f5f0 con 0x7f30280fff40 2024-03-22T07:17:36.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.372+0000 7f301f7fe700 1 -- 172.21.15.63:0/3066045553 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 25) v1 ==== 88751+0+0 (secure 0 0 0) 0x7f3010026440 con 0x7f30280fff40 2024-03-22T07:17:36.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.372+0000 7f301f7fe700 1 --2- 172.21.15.63:0/3066045553 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f301406af90 0x7f301406d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:17:36.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.372+0000 7f301f7fe700 1 -- 172.21.15.63:0/3066045553 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f301008a450 con 0x7f30280fff40 2024-03-22T07:17:36.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.373+0000 7f302659c700 1 --2- 172.21.15.63:0/3066045553 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f301406af90 0x7f301406d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:17:36.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.373+0000 7f302659c700 1 --2- 172.21.15.63:0/3066045553 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f301406af90 0x7f301406d440 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3028066870 tx=0x7f3018009040 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:17:36.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.377+0000 7f301f7fe700 1 -- 172.21.15.63:0/3066045553 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f3010058510 con 0x7f30280fff40 2024-03-22T07:17:36.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.624+0000 7f302c953700 1 -- 172.21.15.63:0/3066045553 --> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f302810f820 con 0x7f301406af90 2024-03-22T07:17:36.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.625+0000 7f301f7fe700 1 -- 172.21.15.63:0/3066045553 <== mgr.24385 v2:172.21.15.69:6828/1862357042 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f302810f820 con 0x7f301406af90 2024-03-22T07:17:36.626 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:17:36.626 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:17:36.626 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:17:36.626 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-22T07:17:36.627 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-03-22T07:17:36.627 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "", 2024-03-22T07:17:36.627 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mgr daemons", 2024-03-22T07:17:36.627 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:17:36.627 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:17:36.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.629+0000 7f302c953700 1 -- 172.21.15.63:0/3066045553 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f301406af90 msgr2=0x7f301406d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:36.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.629+0000 7f302c953700 1 --2- 172.21.15.63:0/3066045553 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f301406af90 0x7f301406d440 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3028066870 tx=0x7f3018009040 comp rx=0 tx=0).stop 2024-03-22T07:17:36.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.629+0000 7f302c953700 1 -- 172.21.15.63:0/3066045553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30280fff40 msgr2=0x7f302810df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:17:36.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.629+0000 7f302c953700 1 --2- 172.21.15.63:0/3066045553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30280fff40 0x7f302810df80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f301000b3b0 tx=0x7f3010009e30 comp rx=0 tx=0).stop 2024-03-22T07:17:36.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.629+0000 7f302c953700 1 -- 172.21.15.63:0/3066045553 shutdown_connections 2024-03-22T07:17:36.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.629+0000 7f302c953700 1 --2- 172.21.15.63:0/3066045553 >> [v2:172.21.15.69:6828/1862357042,v1:172.21.15.69:6829/1862357042] conn(0x7f301406af90 0x7f301406d440 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:36.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.629+0000 7f302c953700 1 --2- 172.21.15.63:0/3066045553 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3028066b30 0x7f302810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:36.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.629+0000 7f302c953700 1 --2- 172.21.15.63:0/3066045553 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30280fff40 0x7f302810df80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:17:36.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.629+0000 7f302c953700 1 -- 172.21.15.63:0/3066045553 >> 172.21.15.63:0/3066045553 conn(0x7f30280f7410 msgr2=0x7f30280f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:17:36.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.630+0000 7f302c953700 1 -- 172.21.15.63:0/3066045553 shutdown_connections 2024-03-22T07:17:36.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:17:36.630+0000 7f302c953700 1 -- 172.21.15.63:0/3066045553 wait complete. 2024-03-22T07:17:36.948 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:36 smithi063 ceph-mon[29200]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:37.006 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:36 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3279051500' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:37.006 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:36 smithi063 ceph-mon[29200]: pgmap v51: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 531 KiB/s rd, 457 KiB/s wr, 142 op/s 2024-03-22T07:17:37.006 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:36 smithi063 ceph-mon[29200]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:37.006 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:36 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:37.006 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:36 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:37.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:36 smithi069 ceph-mon[36441]: from='client.14680 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:37.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:36 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/3279051500' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:17:37.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:36 smithi069 ceph-mon[36441]: pgmap v51: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 531 KiB/s rd, 457 KiB/s wr, 142 op/s 2024-03-22T07:17:37.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:36 smithi069 ceph-mon[36441]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:17:37.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:36 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:37.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:36 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:39.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:39 smithi063 ceph-mon[29200]: pgmap v52: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 342 KiB/s rd, 261 KiB/s wr, 96 op/s 2024-03-22T07:17:39.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:39 smithi069 ceph-mon[36441]: pgmap v52: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 342 KiB/s rd, 261 KiB/s wr, 96 op/s 2024-03-22T07:17:41.010 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:40 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:41.010 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:40 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:41.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:40 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:41.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:40 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:41.847 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:41 smithi063 ceph-mon[29200]: pgmap v53: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 4.5 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-03-22T07:17:41.847 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:41 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:41.847 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:41 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:41.847 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:41 smithi063 ceph-mon[29200]: Standby manager daemon smithi063.mnxayh restarted 2024-03-22T07:17:41.847 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:41 smithi063 ceph-mon[29200]: Standby manager daemon smithi063.mnxayh started 2024-03-22T07:17:41.847 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:41 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.mnxayh/crt"}]: dispatch 2024-03-22T07:17:41.848 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:41 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:17:41.848 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:41 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.mnxayh/key"}]: dispatch 2024-03-22T07:17:41.848 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:41 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:17:42.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:41 smithi069 ceph-mon[36441]: pgmap v53: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 4.5 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-03-22T07:17:42.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:41 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:42.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:41 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:42.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:41 smithi069 ceph-mon[36441]: Standby manager daemon smithi063.mnxayh restarted 2024-03-22T07:17:42.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:41 smithi069 ceph-mon[36441]: Standby manager daemon smithi063.mnxayh started 2024-03-22T07:17:42.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:41 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.mnxayh/crt"}]: dispatch 2024-03-22T07:17:42.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:41 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:17:42.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:41 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi063.mnxayh/key"}]: dispatch 2024-03-22T07:17:42.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:41 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:17:43.200 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:42 smithi063 ceph-mon[29200]: mgrmap e26: smithi069.gfziwh(active, since 101s), standbys: smithi063.mnxayh 2024-03-22T07:17:43.200 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:42 smithi063 ceph-mon[29200]: pgmap v54: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-03-22T07:17:43.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:42 smithi069 ceph-mon[36441]: mgrmap e26: smithi069.gfziwh(active, since 101s), standbys: smithi063.mnxayh 2024-03-22T07:17:43.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:42 smithi069 ceph-mon[36441]: pgmap v54: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 5.3 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-03-22T07:17:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: pgmap v55: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 426 B/s wr, 4 op/s 2024-03-22T07:17:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:17:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:17:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mgr fail", "who": "smithi069.gfziwh"}]: dispatch 2024-03-22T07:17:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: osdmap e49: 6 total, 6 up, 6 in 2024-03-22T07:17:45.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd='[{"prefix": "mgr fail", "who": "smithi069.gfziwh"}]': finished 2024-03-22T07:17:45.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: mgrmap e27: smithi063.mnxayh(active, starting, since 0.00640267s) 2024-03-22T07:17:45.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:17:45.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:17:45.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.xeelpk"}]: dispatch 2024-03-22T07:17:45.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.oytapz"}]: dispatch 2024-03-22T07:17:45.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.wwtsjn"}]: dispatch 2024-03-22T07:17:45.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.cijxah"}]: dispatch 2024-03-22T07:17:45.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr metadata", "who": "smithi063.mnxayh", "id": "smithi063.mnxayh"}]: dispatch 2024-03-22T07:17:45.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:17:45.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:17:45.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:17:45.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:17:45.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:17:45.532 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:17:45.532 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:17:45.532 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:17:45.532 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:17:45.532 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: Manager daemon smithi063.mnxayh is now available 2024-03-22T07:17:45.532 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:45.532 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:45.532 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:17:45.532 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.mnxayh/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:17:45.532 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.mnxayh/trash_purge_schedule"}]: dispatch 2024-03-22T07:17:45.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: pgmap v55: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 426 B/s wr, 4 op/s 2024-03-22T07:17:45.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:45.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:45.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:17:45.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:17:45.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' 2024-03-22T07:17:45.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:45.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "mgr fail", "who": "smithi069.gfziwh"}]: dispatch 2024-03-22T07:17:45.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: osdmap e49: 6 total, 6 up, 6 in 2024-03-22T07:17:45.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.24385 172.21.15.69:0/4231838745' entity='mgr.smithi069.gfziwh' cmd='[{"prefix": "mgr fail", "who": "smithi069.gfziwh"}]': finished 2024-03-22T07:17:45.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: mgrmap e27: smithi063.mnxayh(active, starting, since 0.00640267s) 2024-03-22T07:17:45.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:17:45.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:17:45.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.xeelpk"}]: dispatch 2024-03-22T07:17:45.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.oytapz"}]: dispatch 2024-03-22T07:17:45.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.wwtsjn"}]: dispatch 2024-03-22T07:17:45.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.cijxah"}]: dispatch 2024-03-22T07:17:45.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr metadata", "who": "smithi063.mnxayh", "id": "smithi063.mnxayh"}]: dispatch 2024-03-22T07:17:45.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:17:45.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:17:45.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:17:45.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:17:45.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:17:45.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:17:45.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:17:45.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:17:45.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:17:45.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: Manager daemon smithi063.mnxayh is now available 2024-03-22T07:17:45.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:45.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:17:45.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:17:45.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.mnxayh/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:17:45.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.mnxayh/trash_purge_schedule"}]: dispatch 2024-03-22T07:17:47.063 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:47 smithi069 ceph-mon[36441]: mgrmap e28: smithi063.mnxayh(active, since 1.00839s) 2024-03-22T07:17:47.063 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:47 smithi069 ceph-mon[36441]: [22/Mar/2024:07:17:46] ENGINE Bus STARTING 2024-03-22T07:17:47.063 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:47 smithi069 ceph-mon[36441]: [22/Mar/2024:07:17:46] ENGINE Serving on https://172.21.15.63:7150 2024-03-22T07:17:47.063 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:47 smithi069 ceph-mon[36441]: [22/Mar/2024:07:17:46] ENGINE Serving on http://172.21.15.63:8765 2024-03-22T07:17:47.063 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:47 smithi069 ceph-mon[36441]: [22/Mar/2024:07:17:46] ENGINE Bus STARTED 2024-03-22T07:17:47.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:47 smithi063 ceph-mon[29200]: mgrmap e28: smithi063.mnxayh(active, since 1.00839s) 2024-03-22T07:17:47.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:47 smithi063 ceph-mon[29200]: [22/Mar/2024:07:17:46] ENGINE Bus STARTING 2024-03-22T07:17:47.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:47 smithi063 ceph-mon[29200]: [22/Mar/2024:07:17:46] ENGINE Serving on https://172.21.15.63:7150 2024-03-22T07:17:47.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:47 smithi063 ceph-mon[29200]: [22/Mar/2024:07:17:46] ENGINE Serving on http://172.21.15.63:8765 2024-03-22T07:17:47.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:47 smithi063 ceph-mon[29200]: [22/Mar/2024:07:17:46] ENGINE Bus STARTED 2024-03-22T07:17:48.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:48 smithi069 ceph-mon[36441]: pgmap v4: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:48.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:48 smithi063 ceph-mon[29200]: pgmap v4: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:49.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:49 smithi069 ceph-mon[36441]: mgrmap e29: smithi063.mnxayh(active, since 3s) 2024-03-22T07:17:49.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:49 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:49.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:49 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:49.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:49 smithi063 ceph-mon[29200]: mgrmap e29: smithi063.mnxayh(active, since 3s) 2024-03-22T07:17:49.364 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:49 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:49.365 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:49 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:51.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:50 smithi069 ceph-mon[36441]: pgmap v5: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:51.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:50 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:51.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:50 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:51.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:50 smithi069 ceph-mon[36441]: mgrmap e30: smithi063.mnxayh(active, since 5s) 2024-03-22T07:17:51.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:50 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:51.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:50 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:51.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:50 smithi063 ceph-mon[29200]: pgmap v5: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:17:51.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:50 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:51.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:50 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:51.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:50 smithi063 ceph-mon[29200]: mgrmap e30: smithi063.mnxayh(active, since 5s) 2024-03-22T07:17:51.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:50 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:51.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:50 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:53.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:52 smithi069 ceph-mon[36441]: pgmap v6: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-03-22T07:17:53.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:53.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:53.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:52 smithi069 ceph-mon[36441]: Standby manager daemon smithi069.gfziwh started 2024-03-22T07:17:53.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:52 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/4030167829' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/crt"}]: dispatch 2024-03-22T07:17:53.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:52 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/4030167829' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:17:53.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:52 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/4030167829' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/key"}]: dispatch 2024-03-22T07:17:53.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:52 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/4030167829' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:17:53.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:52 smithi063 ceph-mon[29200]: pgmap v6: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-03-22T07:17:53.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:53.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:53.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:52 smithi063 ceph-mon[29200]: Standby manager daemon smithi069.gfziwh started 2024-03-22T07:17:53.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:52 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.69:0/4030167829' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/crt"}]: dispatch 2024-03-22T07:17:53.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:52 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.69:0/4030167829' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:17:53.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:52 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.69:0/4030167829' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/key"}]: dispatch 2024-03-22T07:17:53.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:52 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.69:0/4030167829' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:17:54.018 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:53 smithi063 ceph-mon[29200]: mgrmap e31: smithi063.mnxayh(active, since 7s), standbys: smithi069.gfziwh 2024-03-22T07:17:54.018 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:53 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr metadata", "who": "smithi069.gfziwh", "id": "smithi069.gfziwh"}]: dispatch 2024-03-22T07:17:54.018 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:53 smithi063 ceph-mon[29200]: pgmap v7: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-03-22T07:17:54.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:53 smithi069 ceph-mon[36441]: mgrmap e31: smithi063.mnxayh(active, since 7s), standbys: smithi069.gfziwh 2024-03-22T07:17:54.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:53 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr metadata", "who": "smithi069.gfziwh", "id": "smithi069.gfziwh"}]: dispatch 2024-03-22T07:17:54.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:53 smithi069 ceph-mon[36441]: pgmap v7: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-03-22T07:17:56.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:56 smithi069 ceph-mon[36441]: pgmap v8: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-03-22T07:17:56.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:56 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:56.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:56 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:56.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:56 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:17:56.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:56 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:17:56.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:56 smithi069 ceph-mon[36441]: Updating smithi063:/etc/ceph/ceph.conf 2024-03-22T07:17:56.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:56 smithi069 ceph-mon[36441]: Updating smithi069:/etc/ceph/ceph.conf 2024-03-22T07:17:56.877 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:56 smithi063 ceph-mon[29200]: pgmap v8: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-03-22T07:17:56.878 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:56 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:56.878 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:56 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:56.878 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:56 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:17:56.878 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:56 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:17:56.878 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:56 smithi063 ceph-mon[29200]: Updating smithi063:/etc/ceph/ceph.conf 2024-03-22T07:17:56.878 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:56 smithi063 ceph-mon[29200]: Updating smithi069:/etc/ceph/ceph.conf 2024-03-22T07:17:57.774 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:57 smithi063 ceph-mon[29200]: Updating smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:17:57.774 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:57 smithi063 ceph-mon[29200]: Updating smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:17:57.774 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:57 smithi063 ceph-mon[29200]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:17:57.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:57 smithi063 ceph-mon[29200]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:17:57.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:57 smithi063 ceph-mon[29200]: Updating smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:17:57.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:57 smithi063 ceph-mon[29200]: Updating smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:17:57.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:57 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:57.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:57 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:57.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:57 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:57.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:57 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:57.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:57 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:57.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:57 smithi063 ceph-mon[29200]: Reconfiguring prometheus.smithi063 (dependencies changed)... 2024-03-22T07:17:57.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:57 smithi069 ceph-mon[36441]: Updating smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:17:57.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:57 smithi069 ceph-mon[36441]: Updating smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:17:57.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:57 smithi069 ceph-mon[36441]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:17:57.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:57 smithi069 ceph-mon[36441]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:17:57.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:57 smithi069 ceph-mon[36441]: Updating smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:17:57.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:57 smithi069 ceph-mon[36441]: Updating smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:17:57.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:57 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:57.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:57 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:57.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:57 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:57.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:57 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:57.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:57 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:17:57.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:57 smithi069 ceph-mon[36441]: Reconfiguring prometheus.smithi063 (dependencies changed)... 2024-03-22T07:17:58.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:58 smithi069 ceph-mon[36441]: Reconfiguring daemon prometheus.smithi063 on smithi063 2024-03-22T07:17:58.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:17:58 smithi069 ceph-mon[36441]: pgmap v9: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-22T07:17:58.800 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:58 smithi063 ceph-mon[29200]: Reconfiguring daemon prometheus.smithi063 on smithi063 2024-03-22T07:17:58.800 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:17:58 smithi063 ceph-mon[29200]: pgmap v9: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-22T07:18:00.858 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:00 smithi069 ceph-mon[36441]: pgmap v10: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-22T07:18:00.858 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:00 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:18:00.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:00 smithi063 ceph-mon[29200]: pgmap v10: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-22T07:18:00.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:00 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:18:02.804 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:02 smithi069 ceph-mon[36441]: pgmap v11: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-22T07:18:02.804 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:02 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:02.804 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:02 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:02.804 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:02 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:18:02.804 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:02 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:02.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:02 smithi063 ceph-mon[29200]: pgmap v11: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-22T07:18:02.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:02 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:02.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:02 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:02.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:02 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:18:02.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:02 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:03.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:03 smithi063 ceph-mon[29200]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:18:04.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:03 smithi069 ceph-mon[36441]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:18:04.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:04 smithi063 ceph-mon[29200]: pgmap v12: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-22T07:18:04.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:04 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi069.gfziwh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:18:04.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:04 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:18:04.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:04 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:18:04.968 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:04 smithi069 ceph-mon[36441]: pgmap v12: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-22T07:18:04.968 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:04 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi069.gfziwh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:18:04.968 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:04 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:18:04.968 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:04 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:18:05.853 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:05 smithi069 ceph-mon[36441]: Upgrade: Updating mgr.smithi069.gfziwh 2024-03-22T07:18:05.853 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:05 smithi069 ceph-mon[36441]: Deploying daemon mgr.smithi069.gfziwh on smithi069 2024-03-22T07:18:05.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:05 smithi063 ceph-mon[29200]: Upgrade: Updating mgr.smithi069.gfziwh 2024-03-22T07:18:05.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:05 smithi063 ceph-mon[29200]: Deploying daemon mgr.smithi069.gfziwh on smithi069 2024-03-22T07:18:06.750 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.748+0000 7f383101f700 1 -- 172.21.15.63:0/1451404071 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3828013b70 con 0x7f382c0fdfe0 2024-03-22T07:18:06.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.749+0000 7f3834285700 1 -- 172.21.15.63:0/1451404071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f382c0fdfe0 msgr2=0x7f382c0fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:06.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.749+0000 7f3834285700 1 --2- 172.21.15.63:0/1451404071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f382c0fdfe0 0x7f382c0fe450 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3828004940 tx=0x7f3828009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:06.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.749+0000 7f3834285700 1 -- 172.21.15.63:0/1451404071 shutdown_connections 2024-03-22T07:18:06.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.749+0000 7f3834285700 1 --2- 172.21.15.63:0/1451404071 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f382c0fdfe0 0x7f382c0fe450 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:06.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.749+0000 7f3834285700 1 --2- 172.21.15.63:0/1451404071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f382c0fce70 0x7f382c0fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:06.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.749+0000 7f3834285700 1 -- 172.21.15.63:0/1451404071 >> 172.21.15.63:0/1451404071 conn(0x7f382c0f8550 msgr2=0x7f382c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:06.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.749+0000 7f3834285700 1 -- 172.21.15.63:0/1451404071 shutdown_connections 2024-03-22T07:18:06.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.749+0000 7f3834285700 1 -- 172.21.15.63:0/1451404071 wait complete. 2024-03-22T07:18:06.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.749+0000 7f3834285700 1 Processor -- start 2024-03-22T07:18:06.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.749+0000 7f3834285700 1 -- start start 2024-03-22T07:18:06.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.749+0000 7f3834285700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f382c0fce70 0x7f382c10d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:06.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.749+0000 7f3834285700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f382c0fdfe0 0x7f382c10ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:06.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.749+0000 7f3834285700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c10e3f0 con 0x7f382c0fce70 2024-03-22T07:18:06.751 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.749+0000 7f3834285700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c10e560 con 0x7f382c0fdfe0 2024-03-22T07:18:06.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.750+0000 7f3832021700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f382c0fce70 0x7f382c10d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:06.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.750+0000 7f3832021700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f382c0fce70 0x7f382c10d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51930/0 (socket says 172.21.15.63:51930) 2024-03-22T07:18:06.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.750+0000 7f3832021700 1 -- 172.21.15.63:0/2511494906 learned_addr learned my addr 172.21.15.63:0/2511494906 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:18:06.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.750+0000 7f3831820700 1 --2- 172.21.15.63:0/2511494906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f382c0fdfe0 0x7f382c10ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:06.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.750+0000 7f3832021700 1 -- 172.21.15.63:0/2511494906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f382c0fdfe0 msgr2=0x7f382c10ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:06.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.750+0000 7f3832021700 1 --2- 172.21.15.63:0/2511494906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f382c0fdfe0 0x7f382c10ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:06.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.750+0000 7f3832021700 1 -- 172.21.15.63:0/2511494906 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3828005040 con 0x7f382c0fce70 2024-03-22T07:18:06.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.750+0000 7f3831820700 1 --2- 172.21.15.63:0/2511494906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f382c0fdfe0 0x7f382c10ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:18:06.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.750+0000 7f3832021700 1 --2- 172.21.15.63:0/2511494906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f382c0fce70 0x7f382c10d8b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f381c00eb70 tx=0x7f381c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:06.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.750+0000 7f3822ffd700 1 -- 172.21.15.63:0/2511494906 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f381c007cc0 con 0x7f382c0fce70 2024-03-22T07:18:06.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.750+0000 7f3822ffd700 1 -- 172.21.15.63:0/2511494906 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f381c00f040 con 0x7f382c0fce70 2024-03-22T07:18:06.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.750+0000 7f3834285700 1 -- 172.21.15.63:0/2511494906 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f382c112f60 con 0x7f382c0fce70 2024-03-22T07:18:06.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.750+0000 7f3822ffd700 1 -- 172.21.15.63:0/2511494906 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f381c00ca90 con 0x7f382c0fce70 2024-03-22T07:18:06.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.750+0000 7f3834285700 1 -- 172.21.15.63:0/2511494906 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f382c113410 con 0x7f382c0fce70 2024-03-22T07:18:06.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.751+0000 7f3834285700 1 -- 172.21.15.63:0/2511494906 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f382c04c710 con 0x7f382c0fce70 2024-03-22T07:18:06.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.752+0000 7f3822ffd700 1 -- 172.21.15.63:0/2511494906 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 31) v1 ==== 90983+0+0 (secure 0 0 0) 0x7f381c00cbf0 con 0x7f382c0fce70 2024-03-22T07:18:06.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.752+0000 7f3822ffd700 1 --2- 172.21.15.63:0/2511494906 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f381806cdb0 0x7f381806f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:06.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.753+0000 7f3822ffd700 1 -- 172.21.15.63:0/2511494906 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f381c08bda0 con 0x7f382c0fce70 2024-03-22T07:18:06.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.753+0000 7f3831820700 1 --2- 172.21.15.63:0/2511494906 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f381806cdb0 0x7f381806f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:06.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.753+0000 7f3831820700 1 --2- 172.21.15.63:0/2511494906 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f381806cdb0 0x7f381806f260 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3828009b30 tx=0x7f3828009e30 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:06.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.757+0000 7f3822ffd700 1 -- 172.21.15.63:0/2511494906 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f381c059990 con 0x7f382c0fce70 2024-03-22T07:18:06.778 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:06 smithi069 ceph-mon[36441]: pgmap v13: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:06.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:06 smithi063 ceph-mon[29200]: pgmap v13: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:06.979 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.979+0000 7f3834285700 1 -- 172.21.15.63:0/2511494906 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f382c113630 con 0x7f381806cdb0 2024-03-22T07:18:06.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.981+0000 7f3822ffd700 1 -- 172.21.15.63:0/2511494906 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f382c113630 con 0x7f381806cdb0 2024-03-22T07:18:06.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.983+0000 7f3834285700 1 -- 172.21.15.63:0/2511494906 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f381806cdb0 msgr2=0x7f381806f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:06.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.983+0000 7f3834285700 1 --2- 172.21.15.63:0/2511494906 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f381806cdb0 0x7f381806f260 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3828009b30 tx=0x7f3828009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:06.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.983+0000 7f3834285700 1 -- 172.21.15.63:0/2511494906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f382c0fce70 msgr2=0x7f382c10d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:06.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.983+0000 7f3834285700 1 --2- 172.21.15.63:0/2511494906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f382c0fce70 0x7f382c10d8b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f381c00eb70 tx=0x7f381c00b040 comp rx=0 tx=0).stop 2024-03-22T07:18:06.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.983+0000 7f3834285700 1 -- 172.21.15.63:0/2511494906 shutdown_connections 2024-03-22T07:18:06.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.983+0000 7f3834285700 1 --2- 172.21.15.63:0/2511494906 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f381806cdb0 0x7f381806f260 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:06.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.983+0000 7f3834285700 1 --2- 172.21.15.63:0/2511494906 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f382c0fce70 0x7f382c10d8b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:06.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.983+0000 7f3834285700 1 --2- 172.21.15.63:0/2511494906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f382c0fdfe0 0x7f382c10ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:06.983 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.983+0000 7f3834285700 1 -- 172.21.15.63:0/2511494906 >> 172.21.15.63:0/2511494906 conn(0x7f382c0f8550 msgr2=0x7f382c0f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:06.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.983+0000 7f3834285700 1 -- 172.21.15.63:0/2511494906 shutdown_connections 2024-03-22T07:18:06.984 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:06.983+0000 7f3834285700 1 -- 172.21.15.63:0/2511494906 wait complete. 2024-03-22T07:18:06.993 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:18:07.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.086+0000 7f4956fbc700 1 -- 172.21.15.63:0/2119762070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49500fdfe0 msgr2=0x7f49500fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:07.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.086+0000 7f4956fbc700 1 --2- 172.21.15.63:0/2119762070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49500fdfe0 0x7f49500fe450 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f493c004940 tx=0x7f493c009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:07.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.086+0000 7f4956fbc700 1 -- 172.21.15.63:0/2119762070 shutdown_connections 2024-03-22T07:18:07.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.086+0000 7f4956fbc700 1 --2- 172.21.15.63:0/2119762070 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49500fdfe0 0x7f49500fe450 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:07.088 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.087+0000 7f4956fbc700 1 --2- 172.21.15.63:0/2119762070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49500fce70 0x7f49500fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:07.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.087+0000 7f4956fbc700 1 -- 172.21.15.63:0/2119762070 >> 172.21.15.63:0/2119762070 conn(0x7f49500f8550 msgr2=0x7f49500fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:07.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.087+0000 7f4956fbc700 1 -- 172.21.15.63:0/2119762070 shutdown_connections 2024-03-22T07:18:07.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.087+0000 7f4956fbc700 1 -- 172.21.15.63:0/2119762070 wait complete. 2024-03-22T07:18:07.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.087+0000 7f4956fbc700 1 Processor -- start 2024-03-22T07:18:07.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.087+0000 7f4956fbc700 1 -- start start 2024-03-22T07:18:07.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.087+0000 7f4956fbc700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49500fce70 0x7f495010d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:07.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.087+0000 7f4956fbc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49500fdfe0 0x7f495010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:07.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.087+0000 7f4956fbc700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495010e460 con 0x7f49500fdfe0 2024-03-22T07:18:07.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.087+0000 7f4956fbc700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495010e5d0 con 0x7f49500fce70 2024-03-22T07:18:07.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.087+0000 7f494ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49500fdfe0 0x7f495010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:07.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.088+0000 7f494ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49500fdfe0 0x7f495010de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51962/0 (socket says 172.21.15.63:51962) 2024-03-22T07:18:07.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.088+0000 7f494ffff700 1 -- 172.21.15.63:0/1315000049 learned_addr learned my addr 172.21.15.63:0/1315000049 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:18:07.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.088+0000 7f4954d58700 1 --2- 172.21.15.63:0/1315000049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49500fce70 0x7f495010d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:07.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.088+0000 7f494ffff700 1 -- 172.21.15.63:0/1315000049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49500fce70 msgr2=0x7f495010d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:07.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.088+0000 7f494ffff700 1 --2- 172.21.15.63:0/1315000049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49500fce70 0x7f495010d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:07.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.088+0000 7f494ffff700 1 -- 172.21.15.63:0/1315000049 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c005040 con 0x7f49500fdfe0 2024-03-22T07:18:07.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.088+0000 7f494ffff700 1 --2- 172.21.15.63:0/1315000049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49500fdfe0 0x7f495010de60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f493c009150 tx=0x7f493c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:07.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.088+0000 7f494d7fa700 1 -- 172.21.15.63:0/1315000049 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f493c00e040 con 0x7f49500fdfe0 2024-03-22T07:18:07.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.088+0000 7f494d7fa700 1 -- 172.21.15.63:0/1315000049 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f493c004290 con 0x7f49500fdfe0 2024-03-22T07:18:07.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.088+0000 7f4956fbc700 1 -- 172.21.15.63:0/1315000049 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4950112fd0 con 0x7f49500fdfe0 2024-03-22T07:18:07.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.088+0000 7f494d7fa700 1 -- 172.21.15.63:0/1315000049 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f493c01c630 con 0x7f49500fdfe0 2024-03-22T07:18:07.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.088+0000 7f4956fbc700 1 -- 172.21.15.63:0/1315000049 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4950113560 con 0x7f49500fdfe0 2024-03-22T07:18:07.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.089+0000 7f4956fbc700 1 -- 172.21.15.63:0/1315000049 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f495004c710 con 0x7f49500fdfe0 2024-03-22T07:18:07.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.090+0000 7f494d7fa700 1 -- 172.21.15.63:0/1315000049 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 31) v1 ==== 90983+0+0 (secure 0 0 0) 0x7f493c00bbf0 con 0x7f49500fdfe0 2024-03-22T07:18:07.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.090+0000 7f494d7fa700 1 --2- 172.21.15.63:0/1315000049 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f494006cdb0 0x7f494006f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:07.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.090+0000 7f494d7fa700 1 -- 172.21.15.63:0/1315000049 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f493c08cda0 con 0x7f49500fdfe0 2024-03-22T07:18:07.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.090+0000 7f4954d58700 1 --2- 172.21.15.63:0/1315000049 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f494006cdb0 0x7f494006f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:07.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.091+0000 7f4954d58700 1 --2- 172.21.15.63:0/1315000049 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f494006cdb0 0x7f494006f260 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4944004f40 tx=0x7f494400b040 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:07.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.095+0000 7f494d7fa700 1 -- 172.21.15.63:0/1315000049 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f493c05aa10 con 0x7f49500fdfe0 2024-03-22T07:18:07.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.314+0000 7f4956fbc700 1 -- 172.21.15.63:0/1315000049 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f49501023a0 con 0x7f494006cdb0 2024-03-22T07:18:07.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.315+0000 7f494d7fa700 1 -- 172.21.15.63:0/1315000049 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f49501023a0 con 0x7f494006cdb0 2024-03-22T07:18:07.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.317+0000 7f4956fbc700 1 -- 172.21.15.63:0/1315000049 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f494006cdb0 msgr2=0x7f494006f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:07.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.317+0000 7f4956fbc700 1 --2- 172.21.15.63:0/1315000049 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f494006cdb0 0x7f494006f260 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4944004f40 tx=0x7f494400b040 comp rx=0 tx=0).stop 2024-03-22T07:18:07.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.317+0000 7f4956fbc700 1 -- 172.21.15.63:0/1315000049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49500fdfe0 msgr2=0x7f495010de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:07.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.317+0000 7f4956fbc700 1 --2- 172.21.15.63:0/1315000049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49500fdfe0 0x7f495010de60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f493c009150 tx=0x7f493c009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:07.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.318+0000 7f4956fbc700 1 -- 172.21.15.63:0/1315000049 shutdown_connections 2024-03-22T07:18:07.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.318+0000 7f4956fbc700 1 --2- 172.21.15.63:0/1315000049 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f494006cdb0 0x7f494006f260 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:07.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.318+0000 7f4956fbc700 1 --2- 172.21.15.63:0/1315000049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49500fce70 0x7f495010d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:07.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.318+0000 7f4956fbc700 1 --2- 172.21.15.63:0/1315000049 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f49500fdfe0 0x7f495010de60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:07.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.318+0000 7f4956fbc700 1 -- 172.21.15.63:0/1315000049 >> 172.21.15.63:0/1315000049 conn(0x7f49500f8550 msgr2=0x7f49500f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:07.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.318+0000 7f4956fbc700 1 -- 172.21.15.63:0/1315000049 shutdown_connections 2024-03-22T07:18:07.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.318+0000 7f4956fbc700 1 -- 172.21.15.63:0/1315000049 wait complete. 2024-03-22T07:18:07.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.420+0000 7fbe833ba700 1 -- 172.21.15.63:0/862027454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe7c0fe440 msgr2=0x7fbe7c0fe850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:07.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.420+0000 7fbe833ba700 1 --2- 172.21.15.63:0/862027454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe7c0fe440 0x7fbe7c0fe850 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbe78004940 tx=0x7fbe78009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:07.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.420+0000 7fbe833ba700 1 -- 172.21.15.63:0/862027454 shutdown_connections 2024-03-22T07:18:07.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.420+0000 7fbe833ba700 1 --2- 172.21.15.63:0/862027454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe7c0fed90 0x7fbe7c0faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:07.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.420+0000 7fbe833ba700 1 --2- 172.21.15.63:0/862027454 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe7c0fe440 0x7fbe7c0fe850 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:07.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.420+0000 7fbe833ba700 1 -- 172.21.15.63:0/862027454 >> 172.21.15.63:0/862027454 conn(0x7fbe7c0f6be0 msgr2=0x7fbe7c0f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:07.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.420+0000 7fbe833ba700 1 -- 172.21.15.63:0/862027454 shutdown_connections 2024-03-22T07:18:07.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.420+0000 7fbe833ba700 1 -- 172.21.15.63:0/862027454 wait complete. 2024-03-22T07:18:07.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.420+0000 7fbe833ba700 1 Processor -- start 2024-03-22T07:18:07.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.421+0000 7fbe833ba700 1 -- start start 2024-03-22T07:18:07.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.421+0000 7fbe833ba700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe7c0fe440 0x7fbe7c0fda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:07.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.421+0000 7fbe833ba700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe7c0fed90 0x7fbe7c0fc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:07.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.421+0000 7fbe833ba700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c0fe050 con 0x7fbe7c0fed90 2024-03-22T07:18:07.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.421+0000 7fbe833ba700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c0fc640 con 0x7fbe7c0fe440 2024-03-22T07:18:07.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.421+0000 7fbe80955700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe7c0fed90 0x7fbe7c0fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:07.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.421+0000 7fbe80955700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe7c0fed90 0x7fbe7c0fc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51976/0 (socket says 172.21.15.63:51976) 2024-03-22T07:18:07.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.421+0000 7fbe80955700 1 -- 172.21.15.63:0/1833560667 learned_addr learned my addr 172.21.15.63:0/1833560667 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:18:07.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.421+0000 7fbe81156700 1 --2- 172.21.15.63:0/1833560667 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe7c0fe440 0x7fbe7c0fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:07.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.421+0000 7fbe80955700 1 -- 172.21.15.63:0/1833560667 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe7c0fe440 msgr2=0x7fbe7c0fda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:07.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.421+0000 7fbe80955700 1 --2- 172.21.15.63:0/1833560667 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe7c0fe440 0x7fbe7c0fda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:07.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.421+0000 7fbe80955700 1 -- 172.21.15.63:0/1833560667 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe78005040 con 0x7fbe7c0fed90 2024-03-22T07:18:07.423 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.421+0000 7fbe80955700 1 --2- 172.21.15.63:0/1833560667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe7c0fed90 0x7fbe7c0fc100 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c002a50 tx=0x7fbe6c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:07.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.421+0000 7fbe71ffb700 1 -- 172.21.15.63:0/1833560667 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe6c007cc0 con 0x7fbe7c0fed90 2024-03-22T07:18:07.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.422+0000 7fbe71ffb700 1 -- 172.21.15.63:0/1833560667 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe6c00f040 con 0x7fbe7c0fed90 2024-03-22T07:18:07.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.422+0000 7fbe833ba700 1 -- 172.21.15.63:0/1833560667 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7c0fc8f0 con 0x7fbe7c0fed90 2024-03-22T07:18:07.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.422+0000 7fbe71ffb700 1 -- 172.21.15.63:0/1833560667 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe6c00da30 con 0x7fbe7c0fed90 2024-03-22T07:18:07.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.422+0000 7fbe833ba700 1 -- 172.21.15.63:0/1833560667 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7c0fceb0 con 0x7fbe7c0fed90 2024-03-22T07:18:07.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.423+0000 7fbe833ba700 1 -- 172.21.15.63:0/1833560667 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe7c04c710 con 0x7fbe7c0fed90 2024-03-22T07:18:07.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.423+0000 7fbe71ffb700 1 -- 172.21.15.63:0/1833560667 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 31) v1 ==== 90983+0+0 (secure 0 0 0) 0x7fbe6c00db90 con 0x7fbe7c0fed90 2024-03-22T07:18:07.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.424+0000 7fbe71ffb700 1 --2- 172.21.15.63:0/1833560667 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fbe6806cdb0 0x7fbe6806f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:07.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.424+0000 7fbe71ffb700 1 -- 172.21.15.63:0/1833560667 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fbe6c08beb0 con 0x7fbe7c0fed90 2024-03-22T07:18:07.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.424+0000 7fbe81156700 1 --2- 172.21.15.63:0/1833560667 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fbe6806cdb0 0x7fbe6806f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:07.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.424+0000 7fbe81156700 1 --2- 172.21.15.63:0/1833560667 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fbe6806cdb0 0x7fbe6806f260 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800b3b0 tx=0x7fbe78009e30 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:07.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.428+0000 7fbe71ffb700 1 -- 172.21.15.63:0/1833560667 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fbe6c059b20 con 0x7fbe7c0fed90 2024-03-22T07:18:07.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.642+0000 7fbe833ba700 1 -- 172.21.15.63:0/1833560667 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe7c102560 con 0x7fbe6806cdb0 2024-03-22T07:18:07.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.652+0000 7fbe71ffb700 1 -- 172.21.15.63:0/1833560667 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fbe7c102560 con 0x7fbe6806cdb0 2024-03-22T07:18:07.654 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:18:07.654 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (88s) 17s ago 9m 26.5M - 0.25.0 c8568f914cd2 6c224caf539d 2024-03-22T07:18:07.654 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (10m) 17s ago 10m 6660k - 17.2.7-726-g9c4e0c34 837124da804f d6c6bbeb6904 2024-03-22T07:18:07.654 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (9m) 18s ago 9m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 30992f8aa88c 2024-03-22T07:18:07.654 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (64s) 17s ago 9m 80.0M - 9.4.7 954c08fa6188 ad0acea62f7f 2024-03-22T07:18:07.654 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (4m) 17s ago 4m 73.8M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:18:07.654 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (4m) 17s ago 4m 124M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:18:07.654 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (4m) 18s ago 4m 62.9M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:18:07.654 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (4m) 18s ago 4m 130M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:18:07.654 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:8443,9283,8765 running (34s) 17s ago 11m 472M - 19.0.0-1667-gdb0330b1 4f325f8440e9 e49468f39991 2024-03-22T07:18:07.654 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283,8765 starting - - - - 2024-03-22T07:18:07.654 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (11m) 17s ago 11m 57.9M 2048M 17.2.7-726-g9c4e0c34 837124da804f f92a5bb6caee 2024-03-22T07:18:07.654 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (8m) 18s ago 8m 38.7M 2048M 17.2.7-726-g9c4e0c34 837124da804f 95f6103f7603 2024-03-22T07:18:07.655 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (101s) 17s ago 10m 17.0M - 1.5.0 0da6a335fe13 0b1b5be4d602 2024-03-22T07:18:07.655 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (52s) 18s ago 9m 8665k - 1.5.0 0da6a335fe13 a3c073d4754d 2024-03-22T07:18:07.655 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (8m) 17s ago 8m 214M 1557M 17.2.7-726-g9c4e0c34 837124da804f 580f22188a10 2024-03-22T07:18:07.655 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (7m) 17s ago 7m 241M 1557M 17.2.7-726-g9c4e0c34 837124da804f fe1ace35c88a 2024-03-22T07:18:07.655 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (7m) 17s ago 7m 197M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4cd995265dcf 2024-03-22T07:18:07.655 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (6m) 18s ago 6m 279M 2581M 17.2.7-726-g9c4e0c34 837124da804f 45c9a759b76d 2024-03-22T07:18:07.655 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (6m) 18s ago 6m 238M 2581M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:18:07.655 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (6m) 18s ago 6m 238M 2581M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:18:07.655 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 starting - - - - 2024-03-22T07:18:07.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.654+0000 7fbe833ba700 1 -- 172.21.15.63:0/1833560667 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fbe6806cdb0 msgr2=0x7fbe6806f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:07.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.654+0000 7fbe833ba700 1 --2- 172.21.15.63:0/1833560667 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fbe6806cdb0 0x7fbe6806f260 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800b3b0 tx=0x7fbe78009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:07.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.654+0000 7fbe833ba700 1 -- 172.21.15.63:0/1833560667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe7c0fed90 msgr2=0x7fbe7c0fc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:07.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.654+0000 7fbe833ba700 1 --2- 172.21.15.63:0/1833560667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe7c0fed90 0x7fbe7c0fc100 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c002a50 tx=0x7fbe6c0046b0 comp rx=0 tx=0).stop 2024-03-22T07:18:07.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.654+0000 7fbe833ba700 1 -- 172.21.15.63:0/1833560667 shutdown_connections 2024-03-22T07:18:07.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.654+0000 7fbe833ba700 1 --2- 172.21.15.63:0/1833560667 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fbe6806cdb0 0x7fbe6806f260 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:07.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.654+0000 7fbe833ba700 1 --2- 172.21.15.63:0/1833560667 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe7c0fe440 0x7fbe7c0fda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:07.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.654+0000 7fbe833ba700 1 --2- 172.21.15.63:0/1833560667 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe7c0fed90 0x7fbe7c0fc100 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:07.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.654+0000 7fbe833ba700 1 -- 172.21.15.63:0/1833560667 >> 172.21.15.63:0/1833560667 conn(0x7fbe7c0f6be0 msgr2=0x7fbe7c0f82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:07.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.654+0000 7fbe833ba700 1 -- 172.21.15.63:0/1833560667 shutdown_connections 2024-03-22T07:18:07.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.654+0000 7fbe833ba700 1 -- 172.21.15.63:0/1833560667 wait complete. 2024-03-22T07:18:07.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.757+0000 7f2c4ecd0700 1 -- 172.21.15.63:0/3790782004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c48066b70 msgr2=0x7f2c480ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:07.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.757+0000 7f2c4ecd0700 1 --2- 172.21.15.63:0/3790782004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c48066b70 0x7f2c480ff9d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c004940 tx=0x7f2c3c009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:07.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.758+0000 7f2c4ecd0700 1 -- 172.21.15.63:0/3790782004 shutdown_connections 2024-03-22T07:18:07.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.758+0000 7f2c4ecd0700 1 --2- 172.21.15.63:0/3790782004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c480fff10 0x7f2c48102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:07.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.758+0000 7f2c4ecd0700 1 --2- 172.21.15.63:0/3790782004 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c48066b70 0x7f2c480ff9d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:07.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.758+0000 7f2c4ecd0700 1 -- 172.21.15.63:0/3790782004 >> 172.21.15.63:0/3790782004 conn(0x7f2c480f7440 msgr2=0x7f2c480f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:07.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.758+0000 7f2c4ecd0700 1 -- 172.21.15.63:0/3790782004 shutdown_connections 2024-03-22T07:18:07.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.758+0000 7f2c4ecd0700 1 -- 172.21.15.63:0/3790782004 wait complete. 2024-03-22T07:18:07.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.758+0000 7f2c4ecd0700 1 Processor -- start 2024-03-22T07:18:07.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.758+0000 7f2c4ecd0700 1 -- start start 2024-03-22T07:18:07.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.759+0000 7f2c4ecd0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c48066b70 0x7f2c4810da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:07.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.759+0000 7f2c4ecd0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c480fff10 0x7f2c4810dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:07.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.759+0000 7f2c4ecd0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4810e5b0 con 0x7f2c480fff10 2024-03-22T07:18:07.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.759+0000 7f2c4ecd0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4810ed60 con 0x7f2c48066b70 2024-03-22T07:18:07.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.759+0000 7f2c47fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c480fff10 0x7f2c4810dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:07.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.759+0000 7f2c47fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c480fff10 0x7f2c4810dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51992/0 (socket says 172.21.15.63:51992) 2024-03-22T07:18:07.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.759+0000 7f2c47fff700 1 -- 172.21.15.63:0/2559575439 learned_addr learned my addr 172.21.15.63:0/2559575439 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:18:07.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.759+0000 7f2c4ca6c700 1 --2- 172.21.15.63:0/2559575439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c48066b70 0x7f2c4810da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:07.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.759+0000 7f2c47fff700 1 -- 172.21.15.63:0/2559575439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c48066b70 msgr2=0x7f2c4810da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:07.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.759+0000 7f2c47fff700 1 --2- 172.21.15.63:0/2559575439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c48066b70 0x7f2c4810da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:07.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.759+0000 7f2c47fff700 1 -- 172.21.15.63:0/2559575439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c005040 con 0x7f2c480fff10 2024-03-22T07:18:07.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.759+0000 7f2c47fff700 1 --2- 172.21.15.63:0/2559575439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c480fff10 0x7f2c4810dfb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400eaf0 tx=0x7f2c3400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:07.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.760+0000 7f2c457fa700 1 -- 172.21.15.63:0/2559575439 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c34007cc0 con 0x7f2c480fff10 2024-03-22T07:18:07.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.760+0000 7f2c4ecd0700 1 -- 172.21.15.63:0/2559575439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4810efe0 con 0x7f2c480fff10 2024-03-22T07:18:07.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.760+0000 7f2c4ecd0700 1 -- 172.21.15.63:0/2559575439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4810f520 con 0x7f2c480fff10 2024-03-22T07:18:07.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.760+0000 7f2c457fa700 1 -- 172.21.15.63:0/2559575439 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c3400f040 con 0x7f2c480fff10 2024-03-22T07:18:07.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.760+0000 7f2c457fa700 1 -- 172.21.15.63:0/2559575439 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c3400cb30 con 0x7f2c480fff10 2024-03-22T07:18:07.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.761+0000 7f2c4ecd0700 1 -- 172.21.15.63:0/2559575439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2c0052e0 con 0x7f2c480fff10 2024-03-22T07:18:07.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.761+0000 7f2c457fa700 1 -- 172.21.15.63:0/2559575439 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 31) v1 ==== 90983+0+0 (secure 0 0 0) 0x7f2c3400cc90 con 0x7f2c480fff10 2024-03-22T07:18:07.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.761+0000 7f2c457fa700 1 --2- 172.21.15.63:0/2559575439 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f2c3806cda0 0x7f2c3806f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:07.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.762+0000 7f2c457fa700 1 -- 172.21.15.63:0/2559575439 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f2c3408bcb0 con 0x7f2c480fff10 2024-03-22T07:18:07.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.762+0000 7f2c4ca6c700 1 --2- 172.21.15.63:0/2559575439 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f2c3806cda0 0x7f2c3806f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:07.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.762+0000 7f2c4ca6c700 1 --2- 172.21.15.63:0/2559575439 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f2c3806cda0 0x7f2c3806f250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c004890 tx=0x7f2c3c009e30 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:07.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:07.766+0000 7f2c457fa700 1 -- 172.21.15.63:0/2559575439 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f2c340599d0 con 0x7f2c480fff10 2024-03-22T07:18:08.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.060+0000 7f2c4ecd0700 1 -- 172.21.15.63:0/2559575439 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2c2c005860 con 0x7f2c480fff10 2024-03-22T07:18:08.060 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.060+0000 7f2c457fa700 1 -- 172.21.15.63:0/2559575439 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f2c34059560 con 0x7f2c480fff10 2024-03-22T07:18:08.061 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:18:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:18:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:18:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:18:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:18:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:18:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:18:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:18:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:18:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:18:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:18:08.061 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:18:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:18:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:18:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 12, 2024-03-22T07:18:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:18:08.062 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:18:08.062 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:18:08.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.063+0000 7f2c4ecd0700 1 -- 172.21.15.63:0/2559575439 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f2c3806cda0 msgr2=0x7f2c3806f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:08.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.063+0000 7f2c4ecd0700 1 --2- 172.21.15.63:0/2559575439 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f2c3806cda0 0x7f2c3806f250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c004890 tx=0x7f2c3c009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:08.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.063+0000 7f2c4ecd0700 1 -- 172.21.15.63:0/2559575439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c480fff10 msgr2=0x7f2c4810dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:08.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.063+0000 7f2c4ecd0700 1 --2- 172.21.15.63:0/2559575439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c480fff10 0x7f2c4810dfb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400eaf0 tx=0x7f2c3400b040 comp rx=0 tx=0).stop 2024-03-22T07:18:08.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.063+0000 7f2c4ecd0700 1 -- 172.21.15.63:0/2559575439 shutdown_connections 2024-03-22T07:18:08.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.063+0000 7f2c4ecd0700 1 --2- 172.21.15.63:0/2559575439 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f2c3806cda0 0x7f2c3806f250 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:08.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.063+0000 7f2c4ecd0700 1 --2- 172.21.15.63:0/2559575439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c48066b70 0x7f2c4810da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:08.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.063+0000 7f2c4ecd0700 1 --2- 172.21.15.63:0/2559575439 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2c480fff10 0x7f2c4810dfb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:08.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.063+0000 7f2c4ecd0700 1 -- 172.21.15.63:0/2559575439 >> 172.21.15.63:0/2559575439 conn(0x7f2c480f7440 msgr2=0x7f2c480f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:08.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.063+0000 7f2c4ecd0700 1 -- 172.21.15.63:0/2559575439 shutdown_connections 2024-03-22T07:18:08.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.063+0000 7f2c4ecd0700 1 -- 172.21.15.63:0/2559575439 wait complete. 2024-03-22T07:18:08.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.171+0000 7f765c26c700 1 -- 172.21.15.63:0/2487296409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76540fce40 msgr2=0x7f76540fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:08.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.171+0000 7f765c26c700 1 --2- 172.21.15.63:0/2487296409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76540fce40 0x7f76540fd250 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f764c009ce0 tx=0x7f764c004d10 comp rx=0 tx=0).stop 2024-03-22T07:18:08.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.171+0000 7f765c26c700 1 -- 172.21.15.63:0/2487296409 shutdown_connections 2024-03-22T07:18:08.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.171+0000 7f765c26c700 1 --2- 172.21.15.63:0/2487296409 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76540fe040 0x7f76540fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:08.175 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.171+0000 7f765c26c700 1 --2- 172.21.15.63:0/2487296409 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76540fce40 0x7f76540fd250 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:08.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.171+0000 7f765c26c700 1 -- 172.21.15.63:0/2487296409 >> 172.21.15.63:0/2487296409 conn(0x7f76540f8550 msgr2=0x7f76540fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:08.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.171+0000 7f765c26c700 1 -- 172.21.15.63:0/2487296409 shutdown_connections 2024-03-22T07:18:08.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.171+0000 7f765c26c700 1 -- 172.21.15.63:0/2487296409 wait complete. 2024-03-22T07:18:08.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.171+0000 7f765c26c700 1 Processor -- start 2024-03-22T07:18:08.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.171+0000 7f765c26c700 1 -- start start 2024-03-22T07:18:08.176 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.171+0000 7f765c26c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76540fce40 0x7f765410db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:08.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.172+0000 7f765c26c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76540fe040 0x7f765410e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:08.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.172+0000 7f765c26c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765410e670 con 0x7f76540fe040 2024-03-22T07:18:08.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.172+0000 7f765c26c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765410eec0 con 0x7f76540fce40 2024-03-22T07:18:08.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.172+0000 7f7659807700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76540fe040 0x7f765410e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:08.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.172+0000 7f765a008700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76540fce40 0x7f765410db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:08.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.172+0000 7f7659807700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76540fe040 0x7f765410e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:52002/0 (socket says 172.21.15.63:52002) 2024-03-22T07:18:08.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.172+0000 7f7659807700 1 -- 172.21.15.63:0/2588438500 learned_addr learned my addr 172.21.15.63:0/2588438500 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:18:08.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.172+0000 7f7659807700 1 -- 172.21.15.63:0/2588438500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76540fce40 msgr2=0x7f765410db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:08.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.172+0000 7f7659807700 1 --2- 172.21.15.63:0/2588438500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76540fce40 0x7f765410db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:08.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.172+0000 7f7659807700 1 -- 172.21.15.63:0/2588438500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c009940 con 0x7f76540fe040 2024-03-22T07:18:08.177 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.172+0000 7f765a008700 1 --2- 172.21.15.63:0/2588438500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76540fce40 0x7f765410db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:18:08.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.172+0000 7f7659807700 1 --2- 172.21.15.63:0/2588438500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76540fe040 0x7f765410e070 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7648002a80 tx=0x7f76480046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:08.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.172+0000 7f7652ffd700 1 -- 172.21.15.63:0/2588438500 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7648007cc0 con 0x7f76540fe040 2024-03-22T07:18:08.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.172+0000 7f7652ffd700 1 -- 172.21.15.63:0/2588438500 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f764800f040 con 0x7f76540fe040 2024-03-22T07:18:08.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.172+0000 7f765c26c700 1 -- 172.21.15.63:0/2588438500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765410f1a0 con 0x7f76540fe040 2024-03-22T07:18:08.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.172+0000 7f7652ffd700 1 -- 172.21.15.63:0/2588438500 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f764800dac0 con 0x7f76540fe040 2024-03-22T07:18:08.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.172+0000 7f765c26c700 1 -- 172.21.15.63:0/2588438500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765410f6e0 con 0x7f76540fe040 2024-03-22T07:18:08.178 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.173+0000 7f765c26c700 1 -- 172.21.15.63:0/2588438500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765405f5f0 con 0x7f76540fe040 2024-03-22T07:18:08.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.174+0000 7f7652ffd700 1 -- 172.21.15.63:0/2588438500 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 31) v1 ==== 90983+0+0 (secure 0 0 0) 0x7f764800dc20 con 0x7f76540fe040 2024-03-22T07:18:08.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.174+0000 7f7652ffd700 1 --2- 172.21.15.63:0/2588438500 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f763c06cda0 0x7f763c06f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:08.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.174+0000 7f7652ffd700 1 -- 172.21.15.63:0/2588438500 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f764808bde0 con 0x7f76540fe040 2024-03-22T07:18:08.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.174+0000 7f765a008700 1 --2- 172.21.15.63:0/2588438500 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f763c06cda0 0x7f763c06f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:08.179 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.175+0000 7f765a008700 1 --2- 172.21.15.63:0/2588438500 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f763c06cda0 0x7f763c06f250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f764c009470 tx=0x7f764c004d10 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:08.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.179+0000 7f7652ffd700 1 -- 172.21.15.63:0/2588438500 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f7648059a50 con 0x7f76540fe040 2024-03-22T07:18:08.403 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.403+0000 7f765c26c700 1 -- 172.21.15.63:0/2588438500 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f765410fa60 con 0x7f763c06cda0 2024-03-22T07:18:08.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.405+0000 7f7652ffd700 1 -- 172.21.15.63:0/2588438500 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f765410fa60 con 0x7f763c06cda0 2024-03-22T07:18:08.405 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:18:08.405 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:18:08.405 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:18:08.405 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-22T07:18:08.405 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-03-22T07:18:08.405 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "", 2024-03-22T07:18:08.405 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mgr daemons", 2024-03-22T07:18:08.405 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:18:08.405 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:18:08.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.407+0000 7f765c26c700 1 -- 172.21.15.63:0/2588438500 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f763c06cda0 msgr2=0x7f763c06f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:08.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.407+0000 7f765c26c700 1 --2- 172.21.15.63:0/2588438500 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f763c06cda0 0x7f763c06f250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f764c009470 tx=0x7f764c004d10 comp rx=0 tx=0).stop 2024-03-22T07:18:08.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.407+0000 7f765c26c700 1 -- 172.21.15.63:0/2588438500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76540fe040 msgr2=0x7f765410e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:08.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.407+0000 7f765c26c700 1 --2- 172.21.15.63:0/2588438500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76540fe040 0x7f765410e070 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7648002a80 tx=0x7f76480046b0 comp rx=0 tx=0).stop 2024-03-22T07:18:08.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.407+0000 7f765c26c700 1 -- 172.21.15.63:0/2588438500 shutdown_connections 2024-03-22T07:18:08.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.407+0000 7f765c26c700 1 --2- 172.21.15.63:0/2588438500 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f763c06cda0 0x7f763c06f250 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:08.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.407+0000 7f765c26c700 1 --2- 172.21.15.63:0/2588438500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76540fce40 0x7f765410db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:08.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.407+0000 7f765c26c700 1 --2- 172.21.15.63:0/2588438500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76540fe040 0x7f765410e070 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:08.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.407+0000 7f765c26c700 1 -- 172.21.15.63:0/2588438500 >> 172.21.15.63:0/2588438500 conn(0x7f76540f8550 msgr2=0x7f76540f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:08.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.407+0000 7f765c26c700 1 -- 172.21.15.63:0/2588438500 shutdown_connections 2024-03-22T07:18:08.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:08.407+0000 7f765c26c700 1 -- 172.21.15.63:0/2588438500 wait complete. 2024-03-22T07:18:08.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:08 smithi069 ceph-mon[36441]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:08.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:08 smithi069 ceph-mon[36441]: pgmap v14: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:08.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:08 smithi069 ceph-mon[36441]: from='client.14722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:08.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:08 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:08.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:08 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:08.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:08 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:08.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:08 smithi069 ceph-mon[36441]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:08.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:08 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/2559575439' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:08.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:08 smithi063 ceph-mon[29200]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:08.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:08 smithi063 ceph-mon[29200]: pgmap v14: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:08.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:08 smithi063 ceph-mon[29200]: from='client.14722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:08.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:08 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:08.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:08 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:08.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:08 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:08.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:08 smithi063 ceph-mon[29200]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:08.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:08 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2559575439' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:09.709 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:09 smithi069 ceph-mon[36441]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:09.835 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:09 smithi063 ceph-mon[29200]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:10.773 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:10 smithi063 ceph-mon[29200]: pgmap v15: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:10.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:10 smithi069 ceph-mon[36441]: pgmap v15: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:11.724 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:11 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:11.725 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:11 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:11.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:11 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:11.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:11 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:12.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:12 smithi069 ceph-mon[36441]: pgmap v16: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:12.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:12 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:12.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:12 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:13.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:12 smithi063 ceph-mon[29200]: pgmap v16: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:13.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:12 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:13.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:12 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:14.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:13 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:14.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:13 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:14.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:13 smithi069 ceph-mon[36441]: pgmap v17: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:14.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:13 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:14.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:13 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:14.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:13 smithi063 ceph-mon[29200]: pgmap v17: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:14.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:14 smithi069 ceph-mon[36441]: Standby manager daemon smithi069.gfziwh restarted 2024-03-22T07:18:14.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:14 smithi069 ceph-mon[36441]: Standby manager daemon smithi069.gfziwh started 2024-03-22T07:18:14.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:14 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/706073157' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/crt"}]: dispatch 2024-03-22T07:18:14.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:14 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/706073157' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:18:14.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:14 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/706073157' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/key"}]: dispatch 2024-03-22T07:18:14.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:14 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/706073157' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:18:15.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:14 smithi063 ceph-mon[29200]: Standby manager daemon smithi069.gfziwh restarted 2024-03-22T07:18:15.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:14 smithi063 ceph-mon[29200]: Standby manager daemon smithi069.gfziwh started 2024-03-22T07:18:15.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:14 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.69:0/706073157' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/crt"}]: dispatch 2024-03-22T07:18:15.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:14 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.69:0/706073157' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:18:15.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:14 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.69:0/706073157' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/key"}]: dispatch 2024-03-22T07:18:15.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:14 smithi063 ceph-mon[29200]: from='mgr.? 172.21.15.69:0/706073157' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:18:15.991 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: mgrmap e32: smithi063.mnxayh(active, since 29s), standbys: smithi069.gfziwh 2024-03-22T07:18:15.991 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: pgmap v18: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:15.991 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:18:15.991 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:15.991 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:15.991 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:18:15.991 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:18:15.991 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:15.991 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:15.991 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:15.991 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: Upgrade: Setting container_image for all mgr 2024-03-22T07:18:15.991 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:15.992 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi063.mnxayh"}]: dispatch 2024-03-22T07:18:15.992 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi063.mnxayh"}]': finished 2024-03-22T07:18:15.992 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi069.gfziwh"}]: dispatch 2024-03-22T07:18:15.992 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi069.gfziwh"}]': finished 2024-03-22T07:18:15.992 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:15.992 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: Upgrade: Setting container_image for all rgw 2024-03-22T07:18:15.992 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:15.992 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:15.992 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: Upgrade: Setting container_image for all rbd-mirror 2024-03-22T07:18:15.992 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:15.992 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:15.992 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-22T07:18:15.992 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:15.992 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:15.992 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: Upgrade: Setting container_image for all ceph-exporter 2024-03-22T07:18:15.993 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:15.993 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:15.993 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: Upgrade: Setting container_image for all iscsi 2024-03-22T07:18:15.993 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:15.993 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:15.993 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: Upgrade: Setting container_image for all nfs 2024-03-22T07:18:15.993 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:15.993 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:15.993 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: Upgrade: Setting container_image for all nvmeof 2024-03-22T07:18:15.993 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:16.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: mgrmap e32: smithi063.mnxayh(active, since 29s), standbys: smithi069.gfziwh 2024-03-22T07:18:16.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: pgmap v18: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:16.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:18:16.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:16.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:16.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:18:16.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:18:16.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:16.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:16.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:16.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: Upgrade: Setting container_image for all mgr 2024-03-22T07:18:16.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:16.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi063.mnxayh"}]: dispatch 2024-03-22T07:18:16.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi063.mnxayh"}]': finished 2024-03-22T07:18:16.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi069.gfziwh"}]: dispatch 2024-03-22T07:18:16.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi069.gfziwh"}]': finished 2024-03-22T07:18:16.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:16.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: Upgrade: Setting container_image for all rgw 2024-03-22T07:18:16.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:16.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:16.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: Upgrade: Setting container_image for all rbd-mirror 2024-03-22T07:18:16.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:16.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:16.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-22T07:18:16.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:16.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:16.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: Upgrade: Setting container_image for all ceph-exporter 2024-03-22T07:18:16.032 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:16.032 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:16.032 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: Upgrade: Setting container_image for all iscsi 2024-03-22T07:18:16.032 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:16.032 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:16.032 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: Upgrade: Setting container_image for all nfs 2024-03-22T07:18:16.032 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:16.032 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:16.032 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: Upgrade: Setting container_image for all nvmeof 2024-03-22T07:18:16.032 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:18.310 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:18 smithi063 ceph-mon[29200]: pgmap v19: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:18.310 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:18 smithi063 ceph-mon[29200]: Upgrade: Updating node-exporter.smithi063 (1/2) 2024-03-22T07:18:18.310 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:18 smithi063 ceph-mon[29200]: Deploying daemon node-exporter.smithi063 on smithi063 2024-03-22T07:18:18.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:18 smithi069 ceph-mon[36441]: pgmap v19: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:18.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:18 smithi069 ceph-mon[36441]: Upgrade: Updating node-exporter.smithi063 (1/2) 2024-03-22T07:18:18.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:18 smithi069 ceph-mon[36441]: Deploying daemon node-exporter.smithi063 on smithi063 2024-03-22T07:18:20.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:20 smithi069 ceph-mon[36441]: pgmap v20: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:20.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:20 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:20.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:20 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:20.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:20 smithi063 ceph-mon[29200]: pgmap v20: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:20.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:20 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:20.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:20 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:22.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:22 smithi069 ceph-mon[36441]: pgmap v21: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:22.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:22 smithi069 ceph-mon[36441]: Upgrade: Updating node-exporter.smithi069 (2/2) 2024-03-22T07:18:22.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:22 smithi069 ceph-mon[36441]: Deploying daemon node-exporter.smithi069 on smithi069 2024-03-22T07:18:22.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:22 smithi063 ceph-mon[29200]: pgmap v21: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:22.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:22 smithi063 ceph-mon[29200]: Upgrade: Updating node-exporter.smithi069 (2/2) 2024-03-22T07:18:22.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:22 smithi063 ceph-mon[29200]: Deploying daemon node-exporter.smithi069 on smithi069 2024-03-22T07:18:24.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:24 smithi069 ceph-mon[36441]: pgmap v22: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:24.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:24 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:24.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:24 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:24.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:24 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:24.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:24 smithi063 ceph-mon[29200]: pgmap v22: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:24.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:24 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:24.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:24 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:24.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:24 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:26 smithi063 ceph-mon[29200]: pgmap v23: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:26.399 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:26 smithi069 ceph-mon[36441]: pgmap v23: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:28.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:28.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:28.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:27 smithi069 ceph-mon[36441]: pgmap v24: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:28.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:28.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:28.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:27 smithi063 ceph-mon[29200]: pgmap v24: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:29.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:29 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:29.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:29 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:29.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:29 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:29.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:29 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:30.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:30 smithi069 ceph-mon[36441]: pgmap v25: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:30.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:30 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:30.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:30 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:30.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:30 smithi063 ceph-mon[29200]: pgmap v25: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:30.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:30 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:30.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:30 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:31.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:31 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:18:31.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:31 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:31.106 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:31 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:31.505 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:31 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:18:31.506 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:31 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:31.506 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:31 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:32.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:32 smithi063 ceph-mon[29200]: pgmap v26: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:32.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:32 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:32.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:32 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:33.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:32 smithi069 ceph-mon[36441]: pgmap v26: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:33.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:32 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:33.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:32 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:34.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:34 smithi063 ceph-mon[29200]: pgmap v27: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:34.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:34 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:34.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:34 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:34.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:34 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:18:34.730 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:34 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:18:34.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:34 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:34.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:34 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:34.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:34 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:34.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:34 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:34.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:34 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:34.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:34 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:34.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:34 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:34.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:34 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:34.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:34 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:34.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:34 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:34.731 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:34 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:35.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:34 smithi069 ceph-mon[36441]: pgmap v27: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:35.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:34 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:35.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:34 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:35.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:34 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:18:35.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:34 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:18:35.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:34 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:35.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:34 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:35.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:34 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:35.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:34 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:35.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:34 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:35.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:34 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:35.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:34 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:35.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:34 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:35.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:34 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:35.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:34 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:35.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:34 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:36.838 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:36 smithi063 ceph-mon[29200]: pgmap v28: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:36.839 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:36 smithi063 ceph-mon[29200]: Upgrade: Updating prometheus.smithi063 2024-03-22T07:18:37.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:36 smithi069 ceph-mon[36441]: pgmap v28: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:37.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:36 smithi069 ceph-mon[36441]: Upgrade: Updating prometheus.smithi063 2024-03-22T07:18:37.836 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:37 smithi063 ceph-mon[29200]: Deploying daemon prometheus.smithi063 on smithi063 2024-03-22T07:18:38.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:37 smithi069 ceph-mon[36441]: Deploying daemon prometheus.smithi063 on smithi063 2024-03-22T07:18:38.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.521+0000 7f57cf0ff700 1 -- 172.21.15.63:0/3713884598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c80fbe00 msgr2=0x7f57c80fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:38.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.521+0000 7f57cf0ff700 1 --2- 172.21.15.63:0/3713884598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c80fbe00 0x7f57c80fc210 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f57bc004940 tx=0x7f57bc009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:38.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.522+0000 7f57cf0ff700 1 -- 172.21.15.63:0/3713884598 shutdown_connections 2024-03-22T07:18:38.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.522+0000 7f57cf0ff700 1 --2- 172.21.15.63:0/3713884598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57c80fcf70 0x7f57c80fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:38.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.522+0000 7f57cf0ff700 1 --2- 172.21.15.63:0/3713884598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c80fbe00 0x7f57c80fc210 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:38.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.522+0000 7f57cf0ff700 1 -- 172.21.15.63:0/3713884598 >> 172.21.15.63:0/3713884598 conn(0x7f57c80f74e0 msgr2=0x7f57c80f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:38.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.522+0000 7f57cf0ff700 1 -- 172.21.15.63:0/3713884598 shutdown_connections 2024-03-22T07:18:38.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.522+0000 7f57cf0ff700 1 -- 172.21.15.63:0/3713884598 wait complete. 2024-03-22T07:18:38.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.522+0000 7f57cf0ff700 1 Processor -- start 2024-03-22T07:18:38.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.522+0000 7f57cf0ff700 1 -- start start 2024-03-22T07:18:38.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.522+0000 7f57cf0ff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57c80fbe00 0x7f57c810da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:38.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.523+0000 7f57cf0ff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c80fcf70 0x7f57c810dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:38.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.523+0000 7f57cf0ff700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c810e630 con 0x7f57c80fcf70 2024-03-22T07:18:38.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.523+0000 7f57cf0ff700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c810edd0 con 0x7f57c80fbe00 2024-03-22T07:18:38.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.523+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c80fcf70 0x7f57c810dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:38.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.523+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c80fcf70 0x7f57c810dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60224/0 (socket says 172.21.15.63:60224) 2024-03-22T07:18:38.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.523+0000 7f57c7fff700 1 -- 172.21.15.63:0/2624480314 learned_addr learned my addr 172.21.15.63:0/2624480314 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:18:38.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.523+0000 7f57cce9b700 1 --2- 172.21.15.63:0/2624480314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57c80fbe00 0x7f57c810da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:38.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.523+0000 7f57c7fff700 1 -- 172.21.15.63:0/2624480314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57c80fbe00 msgr2=0x7f57c810da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:38.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.523+0000 7f57c7fff700 1 --2- 172.21.15.63:0/2624480314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57c80fbe00 0x7f57c810da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:38.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.523+0000 7f57c7fff700 1 -- 172.21.15.63:0/2624480314 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57bc005040 con 0x7f57c80fcf70 2024-03-22T07:18:38.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.523+0000 7f57c7fff700 1 --2- 172.21.15.63:0/2624480314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c80fcf70 0x7f57c810dfa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f57b400b790 tx=0x7f57b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:38.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.523+0000 7f57c57fa700 1 -- 172.21.15.63:0/2624480314 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57b400f770 con 0x7f57c80fcf70 2024-03-22T07:18:38.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.523+0000 7f57c57fa700 1 -- 172.21.15.63:0/2624480314 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57b400fda0 con 0x7f57c80fcf70 2024-03-22T07:18:38.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.523+0000 7f57cf0ff700 1 -- 172.21.15.63:0/2624480314 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c810f0b0 con 0x7f57c80fcf70 2024-03-22T07:18:38.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.523+0000 7f57c57fa700 1 -- 172.21.15.63:0/2624480314 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57b400d310 con 0x7f57c80fcf70 2024-03-22T07:18:38.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.523+0000 7f57cf0ff700 1 -- 172.21.15.63:0/2624480314 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c810f670 con 0x7f57c80fcf70 2024-03-22T07:18:38.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.524+0000 7f57cf0ff700 1 -- 172.21.15.63:0/2624480314 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c8101b00 con 0x7f57c80fcf70 2024-03-22T07:18:38.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.525+0000 7f57c57fa700 1 -- 172.21.15.63:0/2624480314 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7f57b4010420 con 0x7f57c80fcf70 2024-03-22T07:18:38.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.525+0000 7f57c57fa700 1 --2- 172.21.15.63:0/2624480314 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f57b806cda0 0x7f57b806f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:38.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.526+0000 7f57c57fa700 1 -- 172.21.15.63:0/2624480314 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f57b408c080 con 0x7f57c80fcf70 2024-03-22T07:18:38.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.526+0000 7f57cce9b700 1 --2- 172.21.15.63:0/2624480314 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f57b806cda0 0x7f57b806f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:38.528 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.526+0000 7f57cce9b700 1 --2- 172.21.15.63:0/2624480314 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f57b806cda0 0x7f57b806f250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00b3b0 tx=0x7f57bc009e30 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:38.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.530+0000 7f57c57fa700 1 -- 172.21.15.63:0/2624480314 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f57b4059ea0 con 0x7f57c80fcf70 2024-03-22T07:18:38.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.752+0000 7f57cf0ff700 1 -- 172.21.15.63:0/2624480314 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f57c81049e0 con 0x7f57b806cda0 2024-03-22T07:18:38.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.753+0000 7f57c57fa700 1 -- 172.21.15.63:0/2624480314 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f57c81049e0 con 0x7f57b806cda0 2024-03-22T07:18:38.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.755+0000 7f57cf0ff700 1 -- 172.21.15.63:0/2624480314 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f57b806cda0 msgr2=0x7f57b806f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:38.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.755+0000 7f57cf0ff700 1 --2- 172.21.15.63:0/2624480314 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f57b806cda0 0x7f57b806f250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00b3b0 tx=0x7f57bc009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:38.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.755+0000 7f57cf0ff700 1 -- 172.21.15.63:0/2624480314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c80fcf70 msgr2=0x7f57c810dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:38.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.755+0000 7f57cf0ff700 1 --2- 172.21.15.63:0/2624480314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c80fcf70 0x7f57c810dfa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f57b400b790 tx=0x7f57b4009250 comp rx=0 tx=0).stop 2024-03-22T07:18:38.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.755+0000 7f57cf0ff700 1 -- 172.21.15.63:0/2624480314 shutdown_connections 2024-03-22T07:18:38.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.755+0000 7f57cf0ff700 1 --2- 172.21.15.63:0/2624480314 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f57b806cda0 0x7f57b806f250 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:38.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.755+0000 7f57cf0ff700 1 --2- 172.21.15.63:0/2624480314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57c80fbe00 0x7f57c810da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:38.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.755+0000 7f57cf0ff700 1 --2- 172.21.15.63:0/2624480314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f57c80fcf70 0x7f57c810dfa0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:38.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.756+0000 7f57cf0ff700 1 -- 172.21.15.63:0/2624480314 >> 172.21.15.63:0/2624480314 conn(0x7f57c80f74e0 msgr2=0x7f57c8066390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:38.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.756+0000 7f57cf0ff700 1 -- 172.21.15.63:0/2624480314 shutdown_connections 2024-03-22T07:18:38.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.756+0000 7f57cf0ff700 1 -- 172.21.15.63:0/2624480314 wait complete. 2024-03-22T07:18:38.768 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:18:38.851 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:38 smithi063 ceph-mon[29200]: pgmap v29: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:38.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.860+0000 7f17dac8c700 1 -- 172.21.15.63:0/832678548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17d40fcf80 msgr2=0x7f17d40fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:38.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.860+0000 7f17dac8c700 1 --2- 172.21.15.63:0/832678548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17d40fcf80 0x7f17d40fd3f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f17c0004940 tx=0x7f17c0009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:38.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.860+0000 7f17dac8c700 1 -- 172.21.15.63:0/832678548 shutdown_connections 2024-03-22T07:18:38.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.860+0000 7f17dac8c700 1 --2- 172.21.15.63:0/832678548 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17d40fcf80 0x7f17d40fd3f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:38.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.860+0000 7f17dac8c700 1 --2- 172.21.15.63:0/832678548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17d40fbcd0 0x7f17d40fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:38.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.860+0000 7f17dac8c700 1 -- 172.21.15.63:0/832678548 >> 172.21.15.63:0/832678548 conn(0x7f17d40f7430 msgr2=0x7f17d40f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:38.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.860+0000 7f17dac8c700 1 -- 172.21.15.63:0/832678548 shutdown_connections 2024-03-22T07:18:38.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.860+0000 7f17dac8c700 1 -- 172.21.15.63:0/832678548 wait complete. 2024-03-22T07:18:38.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.861+0000 7f17dac8c700 1 Processor -- start 2024-03-22T07:18:38.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.861+0000 7f17dac8c700 1 -- start start 2024-03-22T07:18:38.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.861+0000 7f17dac8c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17d40fbcd0 0x7f17d410da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:38.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.861+0000 7f17dac8c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17d40fcf80 0x7f17d410df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:38.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.861+0000 7f17dac8c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d410e550 con 0x7f17d40fcf80 2024-03-22T07:18:38.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.861+0000 7f17dac8c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d410eda0 con 0x7f17d40fbcd0 2024-03-22T07:18:38.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.861+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17d40fcf80 0x7f17d410df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:38.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.861+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17d40fcf80 0x7f17d410df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60242/0 (socket says 172.21.15.63:60242) 2024-03-22T07:18:38.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.861+0000 7f17d3fff700 1 -- 172.21.15.63:0/2126823183 learned_addr learned my addr 172.21.15.63:0/2126823183 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:18:38.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.861+0000 7f17d8a28700 1 --2- 172.21.15.63:0/2126823183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17d40fbcd0 0x7f17d410da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:38.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.861+0000 7f17d3fff700 1 -- 172.21.15.63:0/2126823183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17d40fbcd0 msgr2=0x7f17d410da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:38.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.861+0000 7f17d3fff700 1 --2- 172.21.15.63:0/2126823183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17d40fbcd0 0x7f17d410da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:38.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.861+0000 7f17d3fff700 1 -- 172.21.15.63:0/2126823183 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c0005040 con 0x7f17d40fcf80 2024-03-22T07:18:38.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.862+0000 7f17d3fff700 1 --2- 172.21.15.63:0/2126823183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17d40fcf80 0x7f17d410df50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f17c000b970 tx=0x7f17c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:38.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.862+0000 7f17d17fa700 1 -- 172.21.15.63:0/2126823183 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17c000e040 con 0x7f17d40fcf80 2024-03-22T07:18:38.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.862+0000 7f17d17fa700 1 -- 172.21.15.63:0/2126823183 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17c000bcc0 con 0x7f17d40fcf80 2024-03-22T07:18:38.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.862+0000 7f17dac8c700 1 -- 172.21.15.63:0/2126823183 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d410f020 con 0x7f17d40fcf80 2024-03-22T07:18:38.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.862+0000 7f17d17fa700 1 -- 172.21.15.63:0/2126823183 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17c0013660 con 0x7f17d40fcf80 2024-03-22T07:18:38.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.862+0000 7f17dac8c700 1 -- 172.21.15.63:0/2126823183 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d410f560 con 0x7f17d40fcf80 2024-03-22T07:18:38.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.863+0000 7f17dac8c700 1 -- 172.21.15.63:0/2126823183 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d4101b10 con 0x7f17d40fcf80 2024-03-22T07:18:38.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.863+0000 7f17d17fa700 1 -- 172.21.15.63:0/2126823183 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7f17c001ca00 con 0x7f17d40fcf80 2024-03-22T07:18:38.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.864+0000 7f17d17fa700 1 --2- 172.21.15.63:0/2126823183 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f17c406cda0 0x7f17c406f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:38.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.864+0000 7f17d17fa700 1 -- 172.21.15.63:0/2126823183 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f17c008ceb0 con 0x7f17d40fcf80 2024-03-22T07:18:38.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.864+0000 7f17d8a28700 1 --2- 172.21.15.63:0/2126823183 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f17c406cda0 0x7f17c406f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:38.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.864+0000 7f17d8a28700 1 --2- 172.21.15.63:0/2126823183 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f17c406cda0 0x7f17c406f250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f17c80041d0 tx=0x7f17c8009040 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:38.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:38.868+0000 7f17d17fa700 1 -- 172.21.15.63:0/2126823183 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f17c005abd0 con 0x7f17d40fcf80 2024-03-22T07:18:39.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:38 smithi069 ceph-mon[36441]: pgmap v29: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:39.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.094+0000 7f17dac8c700 1 -- 172.21.15.63:0/2126823183 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f17d4104930 con 0x7f17c406cda0 2024-03-22T07:18:39.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.095+0000 7f17d17fa700 1 -- 172.21.15.63:0/2126823183 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f17d4104930 con 0x7f17c406cda0 2024-03-22T07:18:39.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.097+0000 7f17dac8c700 1 -- 172.21.15.63:0/2126823183 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f17c406cda0 msgr2=0x7f17c406f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:39.098 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.097+0000 7f17dac8c700 1 --2- 172.21.15.63:0/2126823183 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f17c406cda0 0x7f17c406f250 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f17c80041d0 tx=0x7f17c8009040 comp rx=0 tx=0).stop 2024-03-22T07:18:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.097+0000 7f17dac8c700 1 -- 172.21.15.63:0/2126823183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17d40fcf80 msgr2=0x7f17d410df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.097+0000 7f17dac8c700 1 --2- 172.21.15.63:0/2126823183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17d40fcf80 0x7f17d410df50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f17c000b970 tx=0x7f17c0009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.097+0000 7f17dac8c700 1 -- 172.21.15.63:0/2126823183 shutdown_connections 2024-03-22T07:18:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.097+0000 7f17dac8c700 1 --2- 172.21.15.63:0/2126823183 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f17c406cda0 0x7f17c406f250 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.097+0000 7f17dac8c700 1 --2- 172.21.15.63:0/2126823183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17d40fbcd0 0x7f17d410da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.097+0000 7f17dac8c700 1 --2- 172.21.15.63:0/2126823183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f17d40fcf80 0x7f17d410df50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.098+0000 7f17dac8c700 1 -- 172.21.15.63:0/2126823183 >> 172.21.15.63:0/2126823183 conn(0x7f17d40f7430 msgr2=0x7f17d4065b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.098+0000 7f17dac8c700 1 -- 172.21.15.63:0/2126823183 shutdown_connections 2024-03-22T07:18:39.099 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.098+0000 7f17dac8c700 1 -- 172.21.15.63:0/2126823183 wait complete. 2024-03-22T07:18:39.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.208+0000 7f5470b08700 1 -- 172.21.15.63:0/4182891524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546c0fbce0 msgr2=0x7f546c0fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:39.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.208+0000 7f5470b08700 1 --2- 172.21.15.63:0/4182891524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546c0fbce0 0x7f546c0fc0f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5460004940 tx=0x7f5460009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:39.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.208+0000 7f5470b08700 1 -- 172.21.15.63:0/4182891524 shutdown_connections 2024-03-22T07:18:39.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.208+0000 7f5470b08700 1 --2- 172.21.15.63:0/4182891524 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f546c0fcf90 0x7f546c0fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.208+0000 7f5470b08700 1 --2- 172.21.15.63:0/4182891524 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546c0fbce0 0x7f546c0fc0f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.208+0000 7f5470b08700 1 -- 172.21.15.63:0/4182891524 >> 172.21.15.63:0/4182891524 conn(0x7f546c0f7440 msgr2=0x7f546c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:39.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.208+0000 7f5470b08700 1 -- 172.21.15.63:0/4182891524 shutdown_connections 2024-03-22T07:18:39.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.208+0000 7f5470b08700 1 -- 172.21.15.63:0/4182891524 wait complete. 2024-03-22T07:18:39.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.209+0000 7f5470b08700 1 Processor -- start 2024-03-22T07:18:39.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.209+0000 7f5470b08700 1 -- start start 2024-03-22T07:18:39.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.209+0000 7f5470b08700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f546c0fbce0 0x7f546c10da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:39.211 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.209+0000 7f5470b08700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546c0fcf90 0x7f546c10dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:39.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.209+0000 7f5470b08700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c10e5c0 con 0x7f546c0fcf90 2024-03-22T07:18:39.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.209+0000 7f5470b08700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546c10edc0 con 0x7f546c0fbce0 2024-03-22T07:18:39.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.209+0000 7f5469d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546c0fcf90 0x7f546c10dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:39.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.209+0000 7f5469d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546c0fcf90 0x7f546c10dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60262/0 (socket says 172.21.15.63:60262) 2024-03-22T07:18:39.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.209+0000 7f5469d9b700 1 -- 172.21.15.63:0/2930915177 learned_addr learned my addr 172.21.15.63:0/2930915177 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:18:39.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.209+0000 7f546a59c700 1 --2- 172.21.15.63:0/2930915177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f546c0fbce0 0x7f546c10da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:39.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.209+0000 7f5469d9b700 1 -- 172.21.15.63:0/2930915177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f546c0fbce0 msgr2=0x7f546c10da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:39.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.209+0000 7f5469d9b700 1 --2- 172.21.15.63:0/2930915177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f546c0fbce0 0x7f546c10da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.209+0000 7f5469d9b700 1 -- 172.21.15.63:0/2930915177 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5460005040 con 0x7f546c0fcf90 2024-03-22T07:18:39.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.209+0000 7f5469d9b700 1 --2- 172.21.15.63:0/2930915177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546c0fcf90 0x7f546c10dfc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f545c002a50 tx=0x7f545c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:39.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.210+0000 7f545affd700 1 -- 172.21.15.63:0/2930915177 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f545c007cc0 con 0x7f546c0fcf90 2024-03-22T07:18:39.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.210+0000 7f545affd700 1 -- 172.21.15.63:0/2930915177 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f545c00f040 con 0x7f546c0fcf90 2024-03-22T07:18:39.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.210+0000 7f5470b08700 1 -- 172.21.15.63:0/2930915177 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f546c10f0a0 con 0x7f546c0fcf90 2024-03-22T07:18:39.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.210+0000 7f545affd700 1 -- 172.21.15.63:0/2930915177 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f545c00da30 con 0x7f546c0fcf90 2024-03-22T07:18:39.212 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.210+0000 7f5470b08700 1 -- 172.21.15.63:0/2930915177 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f546c10f5e0 con 0x7f546c0fcf90 2024-03-22T07:18:39.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.211+0000 7f5470b08700 1 -- 172.21.15.63:0/2930915177 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f546c101b20 con 0x7f546c0fcf90 2024-03-22T07:18:39.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.211+0000 7f545affd700 1 -- 172.21.15.63:0/2930915177 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7f545c00dbb0 con 0x7f546c0fcf90 2024-03-22T07:18:39.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.211+0000 7f545affd700 1 --2- 172.21.15.63:0/2930915177 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f545406cda0 0x7f545406f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:39.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.212+0000 7f545affd700 1 -- 172.21.15.63:0/2930915177 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f545c08bfe0 con 0x7f546c0fcf90 2024-03-22T07:18:39.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.212+0000 7f546a59c700 1 --2- 172.21.15.63:0/2930915177 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f545406cda0 0x7f545406f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:39.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.212+0000 7f546a59c700 1 --2- 172.21.15.63:0/2930915177 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f545406cda0 0x7f545406f250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f546000b970 tx=0x7f5460009e30 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:39.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.216+0000 7f545affd700 1 -- 172.21.15.63:0/2930915177 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f545c059bb0 con 0x7f546c0fcf90 2024-03-22T07:18:39.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.448+0000 7f5470b08700 1 -- 172.21.15.63:0/2930915177 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f546c104940 con 0x7f545406cda0 2024-03-22T07:18:39.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.459+0000 7f545affd700 1 -- 172.21.15.63:0/2930915177 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f546c104940 con 0x7f545406cda0 2024-03-22T07:18:39.460 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:18:39.460 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (119s) 10s ago 10m 26.5M - 0.25.0 c8568f914cd2 6c224caf539d 2024-03-22T07:18:39.460 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (10m) 10s ago 10m 6660k - 17.2.7-726-g9c4e0c34 837124da804f d6c6bbeb6904 2024-03-22T07:18:39.460 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (9m) 12s ago 9m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 30992f8aa88c 2024-03-22T07:18:39.460 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (95s) 10s ago 10m 80.2M - 9.4.7 954c08fa6188 ad0acea62f7f 2024-03-22T07:18:39.460 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (5m) 10s ago 5m 73.8M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:18:39.460 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (4m) 10s ago 4m 124M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:18:39.461 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (4m) 12s ago 4m 62.7M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:18:39.461 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (5m) 12s ago 5m 129M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:18:39.461 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:8443,9283,8765 running (66s) 10s ago 11m 495M - 19.0.0-1667-gdb0330b1 4f325f8440e9 e49468f39991 2024-03-22T07:18:39.461 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283,8765 running (32s) 12s ago 9m 450M - 19.0.0-1667-gdb0330b1 4f325f8440e9 13c3f034ebb2 2024-03-22T07:18:39.461 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (11m) 10s ago 12m 58.0M 2048M 17.2.7-726-g9c4e0c34 837124da804f f92a5bb6caee 2024-03-22T07:18:39.461 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (9m) 12s ago 9m 41.6M 2048M 17.2.7-726-g9c4e0c34 837124da804f 95f6103f7603 2024-03-22T07:18:39.461 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (20s) 10s ago 10m 8040k - 1.5.0 0da6a335fe13 0c825ee2a196 2024-03-22T07:18:39.461 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (15s) 12s ago 9m 5435k - 1.5.0 0da6a335fe13 12edb2f35498 2024-03-22T07:18:39.461 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (8m) 10s ago 8m 215M 1557M 17.2.7-726-g9c4e0c34 837124da804f 580f22188a10 2024-03-22T07:18:39.461 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (8m) 10s ago 8m 241M 1557M 17.2.7-726-g9c4e0c34 837124da804f fe1ace35c88a 2024-03-22T07:18:39.461 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (7m) 10s ago 7m 197M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4cd995265dcf 2024-03-22T07:18:39.461 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (7m) 12s ago 7m 279M 2581M 17.2.7-726-g9c4e0c34 837124da804f 45c9a759b76d 2024-03-22T07:18:39.461 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (7m) 12s ago 7m 239M 2581M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:18:39.461 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (6m) 12s ago 6m 238M 2581M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:18:39.461 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (37s) 10s ago 10m 46.0M - 2.43.0 a07b618ecd1d 715abaca086f 2024-03-22T07:18:39.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.461+0000 7f5470b08700 1 -- 172.21.15.63:0/2930915177 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f545406cda0 msgr2=0x7f545406f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:39.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.461+0000 7f5470b08700 1 --2- 172.21.15.63:0/2930915177 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f545406cda0 0x7f545406f250 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f546000b970 tx=0x7f5460009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:39.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.461+0000 7f5470b08700 1 -- 172.21.15.63:0/2930915177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546c0fcf90 msgr2=0x7f546c10dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:39.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.461+0000 7f5470b08700 1 --2- 172.21.15.63:0/2930915177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546c0fcf90 0x7f546c10dfc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f545c002a50 tx=0x7f545c0046b0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.461+0000 7f5470b08700 1 -- 172.21.15.63:0/2930915177 shutdown_connections 2024-03-22T07:18:39.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.461+0000 7f5470b08700 1 --2- 172.21.15.63:0/2930915177 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f545406cda0 0x7f545406f250 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.461+0000 7f5470b08700 1 --2- 172.21.15.63:0/2930915177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f546c0fbce0 0x7f546c10da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.461+0000 7f5470b08700 1 --2- 172.21.15.63:0/2930915177 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f546c0fcf90 0x7f546c10dfc0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.461+0000 7f5470b08700 1 -- 172.21.15.63:0/2930915177 >> 172.21.15.63:0/2930915177 conn(0x7f546c0f7440 msgr2=0x7f546c0664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:39.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.461+0000 7f5470b08700 1 -- 172.21.15.63:0/2930915177 shutdown_connections 2024-03-22T07:18:39.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.461+0000 7f5470b08700 1 -- 172.21.15.63:0/2930915177 wait complete. 2024-03-22T07:18:39.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.562+0000 7ff6af24b700 1 -- 172.21.15.63:0/440986752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6a80fce40 msgr2=0x7ff6a80fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:39.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.562+0000 7ff6af24b700 1 --2- 172.21.15.63:0/440986752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6a80fce40 0x7ff6a80fd250 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff69c004940 tx=0x7ff69c009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:39.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.562+0000 7ff6af24b700 1 -- 172.21.15.63:0/440986752 shutdown_connections 2024-03-22T07:18:39.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.562+0000 7ff6af24b700 1 --2- 172.21.15.63:0/440986752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a80fe040 0x7ff6a80fe4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.562+0000 7ff6af24b700 1 --2- 172.21.15.63:0/440986752 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6a80fce40 0x7ff6a80fd250 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.564 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.562+0000 7ff6af24b700 1 -- 172.21.15.63:0/440986752 >> 172.21.15.63:0/440986752 conn(0x7ff6a80f8570 msgr2=0x7ff6a80fa980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:39.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.562+0000 7ff6af24b700 1 -- 172.21.15.63:0/440986752 shutdown_connections 2024-03-22T07:18:39.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.562+0000 7ff6af24b700 1 -- 172.21.15.63:0/440986752 wait complete. 2024-03-22T07:18:39.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.563+0000 7ff6af24b700 1 Processor -- start 2024-03-22T07:18:39.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.563+0000 7ff6af24b700 1 -- start start 2024-03-22T07:18:39.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.563+0000 7ff6af24b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6a80fce40 0x7ff6a810d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:39.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.563+0000 7ff6af24b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a80fe040 0x7ff6a810dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:39.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.563+0000 7ff6af24b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a810e4e0 con 0x7ff6a80fce40 2024-03-22T07:18:39.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.563+0000 7ff6af24b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a810ed30 con 0x7ff6a80fe040 2024-03-22T07:18:39.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.563+0000 7ff6acfe7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6a80fce40 0x7ff6a810d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:39.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.563+0000 7ff6acfe7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6a80fce40 0x7ff6a810d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60288/0 (socket says 172.21.15.63:60288) 2024-03-22T07:18:39.565 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.563+0000 7ff6acfe7700 1 -- 172.21.15.63:0/2702993474 learned_addr learned my addr 172.21.15.63:0/2702993474 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:18:39.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.564+0000 7ff6acfe7700 1 -- 172.21.15.63:0/2702993474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a80fe040 msgr2=0x7ff6a810dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:39.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.564+0000 7ff6a7fff700 1 --2- 172.21.15.63:0/2702993474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a80fe040 0x7ff6a810dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:39.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.564+0000 7ff6acfe7700 1 --2- 172.21.15.63:0/2702993474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a80fe040 0x7ff6a810dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.564+0000 7ff6acfe7700 1 -- 172.21.15.63:0/2702993474 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c005040 con 0x7ff6a80fce40 2024-03-22T07:18:39.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.564+0000 7ff6acfe7700 1 --2- 172.21.15.63:0/2702993474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6a80fce40 0x7ff6a810d9a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00b970 tx=0x7ff69c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:39.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.564+0000 7ff6a57fa700 1 -- 172.21.15.63:0/2702993474 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff69c00e040 con 0x7ff6a80fce40 2024-03-22T07:18:39.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.564+0000 7ff6a57fa700 1 -- 172.21.15.63:0/2702993474 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff69c004290 con 0x7ff6a80fce40 2024-03-22T07:18:39.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.564+0000 7ff6af24b700 1 -- 172.21.15.63:0/2702993474 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a810efb0 con 0x7ff6a80fce40 2024-03-22T07:18:39.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.564+0000 7ff6a57fa700 1 -- 172.21.15.63:0/2702993474 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff69c01c630 con 0x7ff6a80fce40 2024-03-22T07:18:39.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.564+0000 7ff6af24b700 1 -- 172.21.15.63:0/2702993474 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a810f4a0 con 0x7ff6a80fce40 2024-03-22T07:18:39.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.565+0000 7ff6af24b700 1 -- 172.21.15.63:0/2702993474 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a805f5f0 con 0x7ff6a80fce40 2024-03-22T07:18:39.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.565+0000 7ff6a57fa700 1 -- 172.21.15.63:0/2702993474 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7ff69c00bbf0 con 0x7ff6a80fce40 2024-03-22T07:18:39.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.566+0000 7ff6a57fa700 1 --2- 172.21.15.63:0/2702993474 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff69806cda0 0x7ff69806f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:39.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.566+0000 7ff6a57fa700 1 -- 172.21.15.63:0/2702993474 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff69c08d5a0 con 0x7ff6a80fce40 2024-03-22T07:18:39.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.566+0000 7ff6a7fff700 1 --2- 172.21.15.63:0/2702993474 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff69806cda0 0x7ff69806f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:39.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.566+0000 7ff6a7fff700 1 --2- 172.21.15.63:0/2702993474 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff69806cda0 0x7ff69806f250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff694009b80 tx=0x7ff69400b040 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:39.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.570+0000 7ff6a57fa700 1 -- 172.21.15.63:0/2702993474 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7ff69c05b210 con 0x7ff6a80fce40 2024-03-22T07:18:39.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.860+0000 7ff6af24b700 1 -- 172.21.15.63:0/2702993474 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff6a81058d0 con 0x7ff6a80fce40 2024-03-22T07:18:39.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.860+0000 7ff6a57fa700 1 -- 172.21.15.63:0/2702993474 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7ff69c013e60 con 0x7ff6a80fce40 2024-03-22T07:18:39.861 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:18:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:18:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:18:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:18:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:18:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:18:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:18:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:18:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:18:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:18:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:18:39.862 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:18:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:18:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:18:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 12, 2024-03-22T07:18:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:18:39.863 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:18:39.863 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:18:39.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.862+0000 7ff6af24b700 1 -- 172.21.15.63:0/2702993474 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff69806cda0 msgr2=0x7ff69806f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:39.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.862+0000 7ff6af24b700 1 --2- 172.21.15.63:0/2702993474 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff69806cda0 0x7ff69806f250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff694009b80 tx=0x7ff69400b040 comp rx=0 tx=0).stop 2024-03-22T07:18:39.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.862+0000 7ff6af24b700 1 -- 172.21.15.63:0/2702993474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6a80fce40 msgr2=0x7ff6a810d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:39.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.862+0000 7ff6af24b700 1 --2- 172.21.15.63:0/2702993474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6a80fce40 0x7ff6a810d9a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00b970 tx=0x7ff69c009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:39.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.863+0000 7ff6af24b700 1 -- 172.21.15.63:0/2702993474 shutdown_connections 2024-03-22T07:18:39.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.863+0000 7ff6af24b700 1 --2- 172.21.15.63:0/2702993474 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff69806cda0 0x7ff69806f250 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.863+0000 7ff6af24b700 1 --2- 172.21.15.63:0/2702993474 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6a80fce40 0x7ff6a810d9a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.863+0000 7ff6af24b700 1 --2- 172.21.15.63:0/2702993474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6a80fe040 0x7ff6a810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.863+0000 7ff6af24b700 1 -- 172.21.15.63:0/2702993474 >> 172.21.15.63:0/2702993474 conn(0x7ff6a80f8570 msgr2=0x7ff6a80f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:39.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.863+0000 7ff6af24b700 1 -- 172.21.15.63:0/2702993474 shutdown_connections 2024-03-22T07:18:39.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.863+0000 7ff6af24b700 1 -- 172.21.15.63:0/2702993474 wait complete. 2024-03-22T07:18:39.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.966+0000 7ff850095700 1 -- 172.21.15.63:0/248619274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8480fce80 msgr2=0x7ff8480fd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:39.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.966+0000 7ff850095700 1 --2- 172.21.15.63:0/248619274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8480fce80 0x7ff8480fd290 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff83c004940 tx=0x7ff83c009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.966+0000 7ff850095700 1 -- 172.21.15.63:0/248619274 shutdown_connections 2024-03-22T07:18:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.966+0000 7ff850095700 1 --2- 172.21.15.63:0/248619274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8480fdff0 0x7ff8480fe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.966+0000 7ff850095700 1 --2- 172.21.15.63:0/248619274 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8480fce80 0x7ff8480fd290 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.966+0000 7ff850095700 1 -- 172.21.15.63:0/248619274 >> 172.21.15.63:0/248619274 conn(0x7ff8480f8560 msgr2=0x7ff8480fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.966+0000 7ff850095700 1 -- 172.21.15.63:0/248619274 shutdown_connections 2024-03-22T07:18:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.966+0000 7ff850095700 1 -- 172.21.15.63:0/248619274 wait complete. 2024-03-22T07:18:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.966+0000 7ff850095700 1 Processor -- start 2024-03-22T07:18:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.966+0000 7ff850095700 1 -- start start 2024-03-22T07:18:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.967+0000 7ff850095700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8480fce80 0x7ff848071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.967+0000 7ff850095700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8480fdff0 0x7ff848070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.967+0000 7ff850095700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8480708c0 con 0x7ff8480fdff0 2024-03-22T07:18:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.967+0000 7ff850095700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff848070a30 con 0x7ff8480fce80 2024-03-22T07:18:39.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.967+0000 7ff84d630700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8480fdff0 0x7ff848070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:39.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.967+0000 7ff84d630700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8480fdff0 0x7ff848070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:60306/0 (socket says 172.21.15.63:60306) 2024-03-22T07:18:39.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.967+0000 7ff84d630700 1 -- 172.21.15.63:0/4114491790 learned_addr learned my addr 172.21.15.63:0/4114491790 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:18:39.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.967+0000 7ff84de31700 1 --2- 172.21.15.63:0/4114491790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8480fce80 0x7ff848071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:39.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.967+0000 7ff84d630700 1 -- 172.21.15.63:0/4114491790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8480fce80 msgr2=0x7ff848071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:39.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.967+0000 7ff84d630700 1 --2- 172.21.15.63:0/4114491790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8480fce80 0x7ff848071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:39.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.967+0000 7ff84d630700 1 -- 172.21.15.63:0/4114491790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83c005040 con 0x7ff8480fdff0 2024-03-22T07:18:39.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.967+0000 7ff84d630700 1 --2- 172.21.15.63:0/4114491790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8480fdff0 0x7ff848070310 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff844002a50 tx=0x7ff8440046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:39.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.967+0000 7ff83a7fc700 1 -- 172.21.15.63:0/4114491790 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff844007cc0 con 0x7ff8480fdff0 2024-03-22T07:18:39.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.967+0000 7ff83a7fc700 1 -- 172.21.15.63:0/4114491790 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff84400f040 con 0x7ff8480fdff0 2024-03-22T07:18:39.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.967+0000 7ff850095700 1 -- 172.21.15.63:0/4114491790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff848070d10 con 0x7ff8480fdff0 2024-03-22T07:18:39.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.967+0000 7ff83a7fc700 1 -- 172.21.15.63:0/4114491790 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff84400da70 con 0x7ff8480fdff0 2024-03-22T07:18:39.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.967+0000 7ff850095700 1 -- 172.21.15.63:0/4114491790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff848115a70 con 0x7ff8480fdff0 2024-03-22T07:18:39.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.968+0000 7ff850095700 1 -- 172.21.15.63:0/4114491790 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff84805f5f0 con 0x7ff8480fdff0 2024-03-22T07:18:39.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.969+0000 7ff83a7fc700 1 -- 172.21.15.63:0/4114491790 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7ff84400dc10 con 0x7ff8480fdff0 2024-03-22T07:18:39.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.969+0000 7ff83a7fc700 1 --2- 172.21.15.63:0/4114491790 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff83406cda0 0x7ff83406f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:18:39.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.969+0000 7ff83a7fc700 1 -- 172.21.15.63:0/4114491790 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff84408b890 con 0x7ff8480fdff0 2024-03-22T07:18:39.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.969+0000 7ff84de31700 1 --2- 172.21.15.63:0/4114491790 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff83406cda0 0x7ff83406f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:18:39.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.970+0000 7ff84de31700 1 --2- 172.21.15.63:0/4114491790 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff83406cda0 0x7ff83406f250 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00b970 tx=0x7ff83c009e30 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:18:39.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:39.974+0000 7ff83a7fc700 1 -- 172.21.15.63:0/4114491790 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7ff844055950 con 0x7ff8480fdff0 2024-03-22T07:18:40.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:40.197+0000 7ff850095700 1 -- 172.21.15.63:0/4114491790 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff848102310 con 0x7ff83406cda0 2024-03-22T07:18:40.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:40.198+0000 7ff83a7fc700 1 -- 172.21.15.63:0/4114491790 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7ff848102310 con 0x7ff83406cda0 2024-03-22T07:18:40.199 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:18:40.199 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:18:40.199 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:18:40.199 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-22T07:18:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-03-22T07:18:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-03-22T07:18:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-03-22T07:18:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "2/2 daemons upgraded", 2024-03-22T07:18:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading prometheus daemons", 2024-03-22T07:18:40.200 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:18:40.200 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:18:40.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:40.200+0000 7ff850095700 1 -- 172.21.15.63:0/4114491790 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff83406cda0 msgr2=0x7ff83406f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:40.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:40.200+0000 7ff850095700 1 --2- 172.21.15.63:0/4114491790 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff83406cda0 0x7ff83406f250 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00b970 tx=0x7ff83c009e30 comp rx=0 tx=0).stop 2024-03-22T07:18:40.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:40.200+0000 7ff850095700 1 -- 172.21.15.63:0/4114491790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8480fdff0 msgr2=0x7ff848070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:18:40.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:40.200+0000 7ff850095700 1 --2- 172.21.15.63:0/4114491790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8480fdff0 0x7ff848070310 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff844002a50 tx=0x7ff8440046b0 comp rx=0 tx=0).stop 2024-03-22T07:18:40.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:40.200+0000 7ff850095700 1 -- 172.21.15.63:0/4114491790 shutdown_connections 2024-03-22T07:18:40.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:40.200+0000 7ff850095700 1 --2- 172.21.15.63:0/4114491790 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff83406cda0 0x7ff83406f250 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:40.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:40.200+0000 7ff850095700 1 --2- 172.21.15.63:0/4114491790 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8480fce80 0x7ff848071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:40.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:40.200+0000 7ff850095700 1 --2- 172.21.15.63:0/4114491790 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff8480fdff0 0x7ff848070310 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:18:40.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:40.200+0000 7ff850095700 1 -- 172.21.15.63:0/4114491790 >> 172.21.15.63:0/4114491790 conn(0x7ff8480f8560 msgr2=0x7ff8480f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:18:40.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:40.200+0000 7ff850095700 1 -- 172.21.15.63:0/4114491790 shutdown_connections 2024-03-22T07:18:40.202 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:18:40.200+0000 7ff850095700 1 -- 172.21.15.63:0/4114491790 wait complete. 2024-03-22T07:18:40.683 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:40 smithi063 ceph-mon[29200]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:40.683 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:40 smithi063 ceph-mon[29200]: pgmap v30: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:40.683 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:40 smithi063 ceph-mon[29200]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:40.683 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:40 smithi063 ceph-mon[29200]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:40.683 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:40 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2702993474' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:41.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:40 smithi069 ceph-mon[36441]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:41.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:40 smithi069 ceph-mon[36441]: pgmap v30: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:41.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:40 smithi069 ceph-mon[36441]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:41.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:40 smithi069 ceph-mon[36441]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:41.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:40 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/2702993474' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:41.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:41 smithi063 ceph-mon[29200]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:42.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:41 smithi069 ceph-mon[36441]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:18:42.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:42 smithi063 ceph-mon[29200]: pgmap v31: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:42.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:42 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:42.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:42 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:42.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:42 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:43.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:42 smithi069 ceph-mon[36441]: pgmap v31: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:43.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:42 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:43.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:42 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:43.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:42 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:44.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:43 smithi063 ceph-mon[29200]: pgmap v32: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:44.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:43 smithi069 ceph-mon[36441]: pgmap v32: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:46.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:46 smithi063 ceph-mon[29200]: pgmap v33: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:46.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:46 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:18:46.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:46 smithi069 ceph-mon[36441]: pgmap v33: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:46.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:46 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:18:48.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:47 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:48.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:47 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:48.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:47 smithi069 ceph-mon[36441]: pgmap v34: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:48.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:47 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:48.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:47 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:48.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:47 smithi063 ceph-mon[29200]: pgmap v34: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:49.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:48 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:49.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:48 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:49.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:48 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:49.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:48 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:50.061 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:49 smithi063 ceph-mon[29200]: pgmap v35: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:50.180 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:49 smithi069 ceph-mon[36441]: pgmap v35: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:52.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:52 smithi063 ceph-mon[29200]: pgmap v36: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:52.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:52.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:52.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:18:52.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:18:52.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:52.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:18:52.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:52.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:52 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:52 smithi069 ceph-mon[36441]: pgmap v36: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:18:52.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:52.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:52.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:18:52.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:18:52.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:18:52.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:18:52.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:18:52.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:52.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:52 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:18:53.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:53 smithi063 ceph-mon[29200]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:18:53.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:53 smithi069 ceph-mon[36441]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-22T07:18:54.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:54 smithi063 ceph-mon[29200]: pgmap v37: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:54.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:54 smithi063 ceph-mon[29200]: Upgrade: Updating alertmanager.smithi063 2024-03-22T07:18:54.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:54 smithi063 ceph-mon[29200]: Deploying daemon alertmanager.smithi063 on smithi063 2024-03-22T07:18:54.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:54 smithi069 ceph-mon[36441]: pgmap v37: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:54.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:54 smithi069 ceph-mon[36441]: Upgrade: Updating alertmanager.smithi063 2024-03-22T07:18:54.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:54 smithi069 ceph-mon[36441]: Deploying daemon alertmanager.smithi063 on smithi063 2024-03-22T07:18:56.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:56 smithi063 ceph-mon[29200]: pgmap v38: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:56.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:56 smithi069 ceph-mon[36441]: pgmap v38: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:18:58.245 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:18:58 smithi063 ceph-mon[29200]: pgmap v39: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-22T07:18:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:18:58 smithi069 ceph-mon[36441]: pgmap v39: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-22T07:19:00.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:00 smithi063 ceph-mon[29200]: pgmap v40: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-22T07:19:00.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:00 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:00.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:00 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:00.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:00 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:19:00.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:00 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:00.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:00 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:00.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:00 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:19:00.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:00 smithi069 ceph-mon[36441]: pgmap v40: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-22T07:19:00.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:00 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:00.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:00 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:00.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:00 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:19:00.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:00 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:00.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:00 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:00.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:00 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:19:02.483 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:02 smithi063 ceph-mon[29200]: pgmap v41: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-22T07:19:02.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:02 smithi069 ceph-mon[36441]: pgmap v41: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-22T07:19:04.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:04 smithi069 ceph-mon[36441]: pgmap v42: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-22T07:19:04.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:04 smithi063 ceph-mon[29200]: pgmap v42: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-22T07:19:05.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:05 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:05.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:05 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:05.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:05 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:05.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:05 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:06.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:06 smithi063 ceph-mon[29200]: pgmap v43: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-22T07:19:06.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:06 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:06.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:06 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:07.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:06 smithi069 ceph-mon[36441]: pgmap v43: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-22T07:19:07.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:06 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:07.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:06 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:08.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:08 smithi063 ceph-mon[29200]: pgmap v44: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-22T07:19:09.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:08 smithi069 ceph-mon[36441]: pgmap v44: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-22T07:19:10.311 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.310+0000 7fe4351aa700 1 -- 172.21.15.63:0/894037544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4300fdff0 msgr2=0x7fe4300fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:10.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.310+0000 7fe4351aa700 1 --2- 172.21.15.63:0/894037544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4300fdff0 0x7fe4300fe460 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe424004940 tx=0x7fe424009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:10.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.310+0000 7fe4351aa700 1 -- 172.21.15.63:0/894037544 shutdown_connections 2024-03-22T07:19:10.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.310+0000 7fe4351aa700 1 --2- 172.21.15.63:0/894037544 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4300fdff0 0x7fe4300fe460 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:10.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.310+0000 7fe4351aa700 1 --2- 172.21.15.63:0/894037544 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4300fce80 0x7fe4300fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:10.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.310+0000 7fe4351aa700 1 -- 172.21.15.63:0/894037544 >> 172.21.15.63:0/894037544 conn(0x7fe4300f8580 msgr2=0x7fe4300fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:10.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.310+0000 7fe4351aa700 1 -- 172.21.15.63:0/894037544 shutdown_connections 2024-03-22T07:19:10.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.310+0000 7fe4351aa700 1 -- 172.21.15.63:0/894037544 wait complete. 2024-03-22T07:19:10.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.310+0000 7fe4351aa700 1 Processor -- start 2024-03-22T07:19:10.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.310+0000 7fe4351aa700 1 -- start start 2024-03-22T07:19:10.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.311+0000 7fe4351aa700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4300fce80 0x7fe430071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:10.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.311+0000 7fe4351aa700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4300fdff0 0x7fe430070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:10.312 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.311+0000 7fe4351aa700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4300708c0 con 0x7fe4300fdff0 2024-03-22T07:19:10.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.311+0000 7fe4351aa700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe430070a30 con 0x7fe4300fce80 2024-03-22T07:19:10.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.311+0000 7fe42e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4300fdff0 0x7fe430070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:10.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.311+0000 7fe42e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4300fdff0 0x7fe430070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58060/0 (socket says 172.21.15.63:58060) 2024-03-22T07:19:10.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.311+0000 7fe42e59c700 1 -- 172.21.15.63:0/3721526626 learned_addr learned my addr 172.21.15.63:0/3721526626 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:19:10.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.311+0000 7fe42ed9d700 1 --2- 172.21.15.63:0/3721526626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4300fce80 0x7fe430071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:10.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.311+0000 7fe42e59c700 1 -- 172.21.15.63:0/3721526626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4300fce80 msgr2=0x7fe430071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:10.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.311+0000 7fe42e59c700 1 --2- 172.21.15.63:0/3721526626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4300fce80 0x7fe430071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:10.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.311+0000 7fe42e59c700 1 -- 172.21.15.63:0/3721526626 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe424005040 con 0x7fe4300fdff0 2024-03-22T07:19:10.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.311+0000 7fe42e59c700 1 --2- 172.21.15.63:0/3721526626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4300fdff0 0x7fe430070310 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe42400b3b0 tx=0x7fe424009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:10.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.311+0000 7fe4177fe700 1 -- 172.21.15.63:0/3721526626 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe42400e040 con 0x7fe4300fdff0 2024-03-22T07:19:10.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.311+0000 7fe4351aa700 1 -- 172.21.15.63:0/3721526626 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe430070cb0 con 0x7fe4300fdff0 2024-03-22T07:19:10.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.311+0000 7fe4177fe700 1 -- 172.21.15.63:0/3721526626 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe42400bbf0 con 0x7fe4300fdff0 2024-03-22T07:19:10.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.311+0000 7fe4177fe700 1 -- 172.21.15.63:0/3721526626 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe42401c630 con 0x7fe4300fdff0 2024-03-22T07:19:10.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.311+0000 7fe4351aa700 1 -- 172.21.15.63:0/3721526626 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe430115a40 con 0x7fe4300fdff0 2024-03-22T07:19:10.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.312+0000 7fe4351aa700 1 -- 172.21.15.63:0/3721526626 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe43005f5f0 con 0x7fe4300fdff0 2024-03-22T07:19:10.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.313+0000 7fe4177fe700 1 -- 172.21.15.63:0/3721526626 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7fe42400bd60 con 0x7fe4300fdff0 2024-03-22T07:19:10.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.313+0000 7fe4177fe700 1 --2- 172.21.15.63:0/3721526626 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fe41806cda0 0x7fe41806f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:10.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.313+0000 7fe4177fe700 1 -- 172.21.15.63:0/3721526626 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fe42408cdf0 con 0x7fe4300fdff0 2024-03-22T07:19:10.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.313+0000 7fe42ed9d700 1 --2- 172.21.15.63:0/3721526626 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fe41806cda0 0x7fe41806f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:10.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.314+0000 7fe42ed9d700 1 --2- 172.21.15.63:0/3721526626 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fe41806cda0 0x7fe41806f250 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe430071bd0 tx=0x7fe42000b040 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:10.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.318+0000 7fe4177fe700 1 -- 172.21.15.63:0/3721526626 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fe424056f30 con 0x7fe4300fdff0 2024-03-22T07:19:10.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.544+0000 7fe4351aa700 1 -- 172.21.15.63:0/3721526626 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe430102400 con 0x7fe41806cda0 2024-03-22T07:19:10.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.545+0000 7fe4177fe700 1 -- 172.21.15.63:0/3721526626 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fe430102400 con 0x7fe41806cda0 2024-03-22T07:19:10.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.548+0000 7fe4351aa700 1 -- 172.21.15.63:0/3721526626 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fe41806cda0 msgr2=0x7fe41806f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:10.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.548+0000 7fe4351aa700 1 --2- 172.21.15.63:0/3721526626 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fe41806cda0 0x7fe41806f250 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe430071bd0 tx=0x7fe42000b040 comp rx=0 tx=0).stop 2024-03-22T07:19:10.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.548+0000 7fe4351aa700 1 -- 172.21.15.63:0/3721526626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4300fdff0 msgr2=0x7fe430070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:10.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.548+0000 7fe4351aa700 1 --2- 172.21.15.63:0/3721526626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4300fdff0 0x7fe430070310 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe42400b3b0 tx=0x7fe424009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:10.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.548+0000 7fe4351aa700 1 -- 172.21.15.63:0/3721526626 shutdown_connections 2024-03-22T07:19:10.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.548+0000 7fe4351aa700 1 --2- 172.21.15.63:0/3721526626 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fe41806cda0 0x7fe41806f250 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:10.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.548+0000 7fe4351aa700 1 --2- 172.21.15.63:0/3721526626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4300fce80 0x7fe430071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:10.548 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.548+0000 7fe4351aa700 1 --2- 172.21.15.63:0/3721526626 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4300fdff0 0x7fe430070310 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:10.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.548+0000 7fe4351aa700 1 -- 172.21.15.63:0/3721526626 >> 172.21.15.63:0/3721526626 conn(0x7fe4300f8580 msgr2=0x7fe4300f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:10.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.548+0000 7fe4351aa700 1 -- 172.21.15.63:0/3721526626 shutdown_connections 2024-03-22T07:19:10.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.548+0000 7fe4351aa700 1 -- 172.21.15.63:0/3721526626 wait complete. 2024-03-22T07:19:10.558 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:19:10.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.651+0000 7fa5732ce700 1 -- 172.21.15.63:0/941388836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56c0fbcd0 msgr2=0x7fa56c0fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:10.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.651+0000 7fa5732ce700 1 --2- 172.21.15.63:0/941388836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56c0fbcd0 0x7fa56c0fc0e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa568004940 tx=0x7fa568009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:10.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.651+0000 7fa5732ce700 1 -- 172.21.15.63:0/941388836 shutdown_connections 2024-03-22T07:19:10.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.651+0000 7fa5732ce700 1 --2- 172.21.15.63:0/941388836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa56c0fcf80 0x7fa56c0fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:10.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.651+0000 7fa5732ce700 1 --2- 172.21.15.63:0/941388836 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56c0fbcd0 0x7fa56c0fc0e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:10.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.651+0000 7fa5732ce700 1 -- 172.21.15.63:0/941388836 >> 172.21.15.63:0/941388836 conn(0x7fa56c0f7430 msgr2=0x7fa56c0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:10.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.652+0000 7fa5732ce700 1 -- 172.21.15.63:0/941388836 shutdown_connections 2024-03-22T07:19:10.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.652+0000 7fa5732ce700 1 -- 172.21.15.63:0/941388836 wait complete. 2024-03-22T07:19:10.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.652+0000 7fa5732ce700 1 Processor -- start 2024-03-22T07:19:10.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.652+0000 7fa5732ce700 1 -- start start 2024-03-22T07:19:10.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.652+0000 7fa5732ce700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56c0fbcd0 0x7fa56c10c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:10.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.652+0000 7fa5732ce700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa56c0fcf80 0x7fa56c10ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:10.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.652+0000 7fa5732ce700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c10d440 con 0x7fa56c0fbcd0 2024-03-22T07:19:10.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.652+0000 7fa5732ce700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c10dc90 con 0x7fa56c0fcf80 2024-03-22T07:19:10.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.652+0000 7fa57106a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56c0fbcd0 0x7fa56c10c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:10.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.652+0000 7fa57106a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56c0fbcd0 0x7fa56c10c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58074/0 (socket says 172.21.15.63:58074) 2024-03-22T07:19:10.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.652+0000 7fa57106a700 1 -- 172.21.15.63:0/3108160547 learned_addr learned my addr 172.21.15.63:0/3108160547 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:19:10.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.653+0000 7fa570869700 1 --2- 172.21.15.63:0/3108160547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa56c0fcf80 0x7fa56c10ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:10.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.653+0000 7fa57106a700 1 -- 172.21.15.63:0/3108160547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa56c0fcf80 msgr2=0x7fa56c10ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:10.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.653+0000 7fa57106a700 1 --2- 172.21.15.63:0/3108160547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa56c0fcf80 0x7fa56c10ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:10.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.653+0000 7fa57106a700 1 -- 172.21.15.63:0/3108160547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa568005040 con 0x7fa56c0fbcd0 2024-03-22T07:19:10.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.653+0000 7fa570869700 1 --2- 172.21.15.63:0/3108160547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa56c0fcf80 0x7fa56c10ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:19:10.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.653+0000 7fa57106a700 1 --2- 172.21.15.63:0/3108160547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56c0fbcd0 0x7fa56c10c900 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa568009150 tx=0x7fa568009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:10.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.653+0000 7fa561ffb700 1 -- 172.21.15.63:0/3108160547 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa56800e040 con 0x7fa56c0fbcd0 2024-03-22T07:19:10.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.653+0000 7fa561ffb700 1 -- 172.21.15.63:0/3108160547 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa56800bcc0 con 0x7fa56c0fbcd0 2024-03-22T07:19:10.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.653+0000 7fa5732ce700 1 -- 172.21.15.63:0/3108160547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56c10df10 con 0x7fa56c0fbcd0 2024-03-22T07:19:10.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.653+0000 7fa561ffb700 1 -- 172.21.15.63:0/3108160547 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa56801c630 con 0x7fa56c0fbcd0 2024-03-22T07:19:10.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.653+0000 7fa5732ce700 1 -- 172.21.15.63:0/3108160547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56c10e450 con 0x7fa56c0fbcd0 2024-03-22T07:19:10.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.654+0000 7fa5732ce700 1 -- 172.21.15.63:0/3108160547 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56c05f5f0 con 0x7fa56c0fbcd0 2024-03-22T07:19:10.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.655+0000 7fa561ffb700 1 -- 172.21.15.63:0/3108160547 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7fa568013420 con 0x7fa56c0fbcd0 2024-03-22T07:19:10.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.655+0000 7fa561ffb700 1 --2- 172.21.15.63:0/3108160547 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa55806cda0 0x7fa55806f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:10.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.655+0000 7fa561ffb700 1 -- 172.21.15.63:0/3108160547 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa56808ced0 con 0x7fa56c0fbcd0 2024-03-22T07:19:10.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.655+0000 7fa570869700 1 --2- 172.21.15.63:0/3108160547 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa55806cda0 0x7fa55806f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:10.656 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.656+0000 7fa570869700 1 --2- 172.21.15.63:0/3108160547 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa55806cda0 0x7fa55806f250 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0041d0 tx=0x7fa55c009040 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:10.659 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.660+0000 7fa561ffb700 1 -- 172.21.15.63:0/3108160547 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fa56805ab40 con 0x7fa56c0fbcd0 2024-03-22T07:19:10.660 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: pgmap v45: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:10.660 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:10.660 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:10.660 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:19:10.660 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:19:10.660 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:10.660 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:19:10.660 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:19:10.660 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:19:10.660 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.661 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.661 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.661 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.661 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.661 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.661 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.661 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.661 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.661 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.661 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:10 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: pgmap v45: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:10.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:10.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:10.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:19:10.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:19:10.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:10.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:19:10.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-22T07:19:10.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:19:10.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:10 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:10.888 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.888+0000 7fa5732ce700 1 -- 172.21.15.63:0/3108160547 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa56c10e760 con 0x7fa55806cda0 2024-03-22T07:19:10.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.889+0000 7fa561ffb700 1 -- 172.21.15.63:0/3108160547 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fa56c10e760 con 0x7fa55806cda0 2024-03-22T07:19:10.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.891+0000 7fa5732ce700 1 -- 172.21.15.63:0/3108160547 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa55806cda0 msgr2=0x7fa55806f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:10.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.891+0000 7fa5732ce700 1 --2- 172.21.15.63:0/3108160547 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa55806cda0 0x7fa55806f250 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0041d0 tx=0x7fa55c009040 comp rx=0 tx=0).stop 2024-03-22T07:19:10.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.891+0000 7fa5732ce700 1 -- 172.21.15.63:0/3108160547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56c0fbcd0 msgr2=0x7fa56c10c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:10.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.891+0000 7fa5732ce700 1 --2- 172.21.15.63:0/3108160547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56c0fbcd0 0x7fa56c10c900 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa568009150 tx=0x7fa568009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:10.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.891+0000 7fa5732ce700 1 -- 172.21.15.63:0/3108160547 shutdown_connections 2024-03-22T07:19:10.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.891+0000 7fa5732ce700 1 --2- 172.21.15.63:0/3108160547 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa55806cda0 0x7fa55806f250 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:10.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.891+0000 7fa5732ce700 1 --2- 172.21.15.63:0/3108160547 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa56c0fbcd0 0x7fa56c10c900 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:10.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.891+0000 7fa5732ce700 1 --2- 172.21.15.63:0/3108160547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa56c0fcf80 0x7fa56c10ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:10.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.891+0000 7fa5732ce700 1 -- 172.21.15.63:0/3108160547 >> 172.21.15.63:0/3108160547 conn(0x7fa56c0f7430 msgr2=0x7fa56c0f8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:10.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.892+0000 7fa5732ce700 1 -- 172.21.15.63:0/3108160547 shutdown_connections 2024-03-22T07:19:10.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.892+0000 7fa5732ce700 1 -- 172.21.15.63:0/3108160547 wait complete. 2024-03-22T07:19:10.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.994+0000 7f67d4b1b700 1 -- 172.21.15.63:0/407416908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d00fed50 msgr2=0x7f67d00fae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:10.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.994+0000 7f67d4b1b700 1 --2- 172.21.15.63:0/407416908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d00fed50 0x7f67d00fae90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f67b8004940 tx=0x7f67b8009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:10.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.994+0000 7f67d4b1b700 1 -- 172.21.15.63:0/407416908 shutdown_connections 2024-03-22T07:19:10.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.994+0000 7f67d4b1b700 1 --2- 172.21.15.63:0/407416908 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d00fed50 0x7f67d00fae90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:10.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.994+0000 7f67d4b1b700 1 --2- 172.21.15.63:0/407416908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d00fe400 0x7f67d00fe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:10.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.994+0000 7f67d4b1b700 1 -- 172.21.15.63:0/407416908 >> 172.21.15.63:0/407416908 conn(0x7f67d00f6ba0 msgr2=0x7f67d00f8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:10.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.994+0000 7f67d4b1b700 1 -- 172.21.15.63:0/407416908 shutdown_connections 2024-03-22T07:19:10.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.994+0000 7f67d4b1b700 1 -- 172.21.15.63:0/407416908 wait complete. 2024-03-22T07:19:10.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.995+0000 7f67d4b1b700 1 Processor -- start 2024-03-22T07:19:10.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.995+0000 7f67d4b1b700 1 -- start start 2024-03-22T07:19:10.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.995+0000 7f67d4b1b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d00fe400 0x7f67d00fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:10.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.995+0000 7f67d4b1b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d00fed50 0x7f67d00fc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:10.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.995+0000 7f67d4b1b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d00fc670 con 0x7f67d00fe400 2024-03-22T07:19:10.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.995+0000 7f67d4b1b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d00fc7e0 con 0x7f67d00fed50 2024-03-22T07:19:10.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.995+0000 7f67ce59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d00fe400 0x7f67d00fdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:10.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.995+0000 7f67ce59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d00fe400 0x7f67d00fdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58102/0 (socket says 172.21.15.63:58102) 2024-03-22T07:19:10.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.995+0000 7f67ce59c700 1 -- 172.21.15.63:0/1762648661 learned_addr learned my addr 172.21.15.63:0/1762648661 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:19:10.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.995+0000 7f67cdd9b700 1 --2- 172.21.15.63:0/1762648661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d00fed50 0x7f67d00fc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:10.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.995+0000 7f67ce59c700 1 -- 172.21.15.63:0/1762648661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d00fed50 msgr2=0x7f67d00fc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:10.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.995+0000 7f67ce59c700 1 --2- 172.21.15.63:0/1762648661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d00fed50 0x7f67d00fc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:10.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.995+0000 7f67ce59c700 1 -- 172.21.15.63:0/1762648661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b8005040 con 0x7f67d00fe400 2024-03-22T07:19:10.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.995+0000 7f67cdd9b700 1 --2- 172.21.15.63:0/1762648661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d00fed50 0x7f67d00fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:19:10.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.996+0000 7f67ce59c700 1 --2- 172.21.15.63:0/1762648661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d00fe400 0x7f67d00fdbf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f67c000cb10 tx=0x7f67c0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:10.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.996+0000 7f67c6ffd700 1 -- 172.21.15.63:0/1762648661 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67c0004200 con 0x7f67d00fe400 2024-03-22T07:19:10.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.996+0000 7f67c6ffd700 1 -- 172.21.15.63:0/1762648661 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67c000acc0 con 0x7f67d00fe400 2024-03-22T07:19:10.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.996+0000 7f67d4b1b700 1 -- 172.21.15.63:0/1762648661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d00fcac0 con 0x7f67d00fe400 2024-03-22T07:19:10.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.996+0000 7f67c6ffd700 1 -- 172.21.15.63:0/1762648661 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67c0003d90 con 0x7f67d00fe400 2024-03-22T07:19:10.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.996+0000 7f67d4b1b700 1 -- 172.21.15.63:0/1762648661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d01159f0 con 0x7f67d00fe400 2024-03-22T07:19:10.998 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.997+0000 7f67d4b1b700 1 -- 172.21.15.63:0/1762648661 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d005f5f0 con 0x7f67d00fe400 2024-03-22T07:19:11.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.997+0000 7f67c6ffd700 1 -- 172.21.15.63:0/1762648661 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7f67c0003430 con 0x7f67d00fe400 2024-03-22T07:19:11.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.998+0000 7f67c6ffd700 1 --2- 172.21.15.63:0/1762648661 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f67bc06cda0 0x7f67bc06f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:11.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.998+0000 7f67c6ffd700 1 -- 172.21.15.63:0/1762648661 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f67c008c6a0 con 0x7f67d00fe400 2024-03-22T07:19:11.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.998+0000 7f67cdd9b700 1 --2- 172.21.15.63:0/1762648661 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f67bc06cda0 0x7f67bc06f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:11.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:10.998+0000 7f67cdd9b700 1 --2- 172.21.15.63:0/1762648661 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f67bc06cda0 0x7f67bc06f250 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f67b800b970 tx=0x7f67b8009e30 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:11.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.002+0000 7f67c6ffd700 1 -- 172.21.15.63:0/1762648661 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f67c0056870 con 0x7f67d00fe400 2024-03-22T07:19:11.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.218+0000 7f67d4b1b700 1 -- 172.21.15.63:0/1762648661 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f67d0102520 con 0x7f67bc06cda0 2024-03-22T07:19:11.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.228+0000 7f67c6ffd700 1 -- 172.21.15.63:0/1762648661 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f67d0102520 con 0x7f67bc06cda0 2024-03-22T07:19:11.228 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:19:11.228 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (12s) 6s ago 10m 17.7M - 0.25.0 c8568f914cd2 06ef21da90e5 2024-03-22T07:19:11.228 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (11m) 6s ago 11m 6660k - 17.2.7-726-g9c4e0c34 837124da804f d6c6bbeb6904 2024-03-22T07:19:11.228 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (10m) 44s ago 10m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 30992f8aa88c 2024-03-22T07:19:11.228 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (2m) 6s ago 10m 77.6M - 9.4.7 954c08fa6188 ad0acea62f7f 2024-03-22T07:19:11.228 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (5m) 6s ago 5m 73.8M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:19:11.229 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (5m) 6s ago 5m 123M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:19:11.229 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (5m) 44s ago 5m 62.7M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:19:11.229 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (5m) 44s ago 5m 129M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:19:11.229 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:8443,9283,8765 running (97s) 6s ago 12m 499M - 19.0.0-1667-gdb0330b1 4f325f8440e9 e49468f39991 2024-03-22T07:19:11.229 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283,8765 running (64s) 44s ago 10m 450M - 19.0.0-1667-gdb0330b1 4f325f8440e9 13c3f034ebb2 2024-03-22T07:19:11.229 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (12m) 6s ago 12m 53.7M 2048M 17.2.7-726-g9c4e0c34 837124da804f f92a5bb6caee 2024-03-22T07:19:11.229 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (10m) 44s ago 10m 41.6M 2048M 17.2.7-726-g9c4e0c34 837124da804f 95f6103f7603 2024-03-22T07:19:11.229 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (51s) 6s ago 11m 8066k - 1.5.0 0da6a335fe13 0c825ee2a196 2024-03-22T07:19:11.229 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (47s) 44s ago 10m 5435k - 1.5.0 0da6a335fe13 12edb2f35498 2024-03-22T07:19:11.229 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (9m) 6s ago 9m 216M 1557M 17.2.7-726-g9c4e0c34 837124da804f 580f22188a10 2024-03-22T07:19:11.229 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (8m) 6s ago 8m 241M 1557M 17.2.7-726-g9c4e0c34 837124da804f fe1ace35c88a 2024-03-22T07:19:11.229 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (8m) 6s ago 8m 198M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4cd995265dcf 2024-03-22T07:19:11.230 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (7m) 44s ago 7m 279M 2581M 17.2.7-726-g9c4e0c34 837124da804f 45c9a759b76d 2024-03-22T07:19:11.230 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (7m) 44s ago 7m 239M 2581M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:19:11.230 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (7m) 44s ago 7m 238M 2581M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:19:11.230 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (30s) 6s ago 10m 48.8M - 2.43.0 a07b618ecd1d 1d89b928f6e7 2024-03-22T07:19:11.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.230+0000 7f67d4b1b700 1 -- 172.21.15.63:0/1762648661 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f67bc06cda0 msgr2=0x7f67bc06f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:11.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.230+0000 7f67d4b1b700 1 --2- 172.21.15.63:0/1762648661 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f67bc06cda0 0x7f67bc06f250 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f67b800b970 tx=0x7f67b8009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:11.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.230+0000 7f67d4b1b700 1 -- 172.21.15.63:0/1762648661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d00fe400 msgr2=0x7f67d00fdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:11.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.230+0000 7f67d4b1b700 1 --2- 172.21.15.63:0/1762648661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d00fe400 0x7f67d00fdbf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f67c000cb10 tx=0x7f67c0007f90 comp rx=0 tx=0).stop 2024-03-22T07:19:11.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.230+0000 7f67d4b1b700 1 -- 172.21.15.63:0/1762648661 shutdown_connections 2024-03-22T07:19:11.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.230+0000 7f67d4b1b700 1 --2- 172.21.15.63:0/1762648661 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f67bc06cda0 0x7f67bc06f250 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:11.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.230+0000 7f67d4b1b700 1 --2- 172.21.15.63:0/1762648661 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d00fe400 0x7f67d00fdbf0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:11.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.230+0000 7f67d4b1b700 1 --2- 172.21.15.63:0/1762648661 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d00fed50 0x7f67d00fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:11.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.230+0000 7f67d4b1b700 1 -- 172.21.15.63:0/1762648661 >> 172.21.15.63:0/1762648661 conn(0x7f67d00f6ba0 msgr2=0x7f67d00f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:11.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.230+0000 7f67d4b1b700 1 -- 172.21.15.63:0/1762648661 shutdown_connections 2024-03-22T07:19:11.231 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.230+0000 7f67d4b1b700 1 -- 172.21.15.63:0/1762648661 wait complete. 2024-03-22T07:19:11.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.338+0000 7fdfb1722700 1 -- 172.21.15.63:0/784130770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfac0fff00 msgr2=0x7fdfac102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:11.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.338+0000 7fdfb1722700 1 --2- 172.21.15.63:0/784130770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfac0fff00 0x7fdfac102320 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c004940 tx=0x7fdf9c009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:11.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.339+0000 7fdfb1722700 1 -- 172.21.15.63:0/784130770 shutdown_connections 2024-03-22T07:19:11.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.339+0000 7fdfb1722700 1 --2- 172.21.15.63:0/784130770 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfac0fff00 0x7fdfac102320 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:11.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.339+0000 7fdfb1722700 1 --2- 172.21.15.63:0/784130770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfac0662e0 0x7fdfac0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:11.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.339+0000 7fdfb1722700 1 -- 172.21.15.63:0/784130770 >> 172.21.15.63:0/784130770 conn(0x7fdfac0f7430 msgr2=0x7fdfac0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:11.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.339+0000 7fdfb1722700 1 -- 172.21.15.63:0/784130770 shutdown_connections 2024-03-22T07:19:11.339 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.339+0000 7fdfb1722700 1 -- 172.21.15.63:0/784130770 wait complete. 2024-03-22T07:19:11.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.339+0000 7fdfb1722700 1 Processor -- start 2024-03-22T07:19:11.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.340+0000 7fdfb1722700 1 -- start start 2024-03-22T07:19:11.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.340+0000 7fdfb1722700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfac0662e0 0x7fdfac10fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:11.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.340+0000 7fdfb1722700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfac0fff00 0x7fdfac110250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:11.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.340+0000 7fdfb1722700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac110850 con 0x7fdfac0662e0 2024-03-22T07:19:11.341 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.340+0000 7fdfb1722700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac110f90 con 0x7fdfac0fff00 2024-03-22T07:19:11.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.340+0000 7fdfaaffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfac0662e0 0x7fdfac10fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:11.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.340+0000 7fdfaaffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfac0662e0 0x7fdfac10fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58130/0 (socket says 172.21.15.63:58130) 2024-03-22T07:19:11.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.340+0000 7fdfaaffd700 1 -- 172.21.15.63:0/3811374733 learned_addr learned my addr 172.21.15.63:0/3811374733 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:19:11.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.340+0000 7fdfaa7fc700 1 --2- 172.21.15.63:0/3811374733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfac0fff00 0x7fdfac110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:11.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.340+0000 7fdfaaffd700 1 -- 172.21.15.63:0/3811374733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfac0fff00 msgr2=0x7fdfac110250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:11.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.340+0000 7fdfaaffd700 1 --2- 172.21.15.63:0/3811374733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfac0fff00 0x7fdfac110250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:11.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.340+0000 7fdfaaffd700 1 -- 172.21.15.63:0/3811374733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c005040 con 0x7fdfac0662e0 2024-03-22T07:19:11.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.341+0000 7fdfaaffd700 1 --2- 172.21.15.63:0/3811374733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfac0662e0 0x7fdfac10fd10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000c890 tx=0x7fdfa00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:11.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.341+0000 7fdf937fe700 1 -- 172.21.15.63:0/3811374733 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfa000a700 con 0x7fdfac0662e0 2024-03-22T07:19:11.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.341+0000 7fdf937fe700 1 -- 172.21.15.63:0/3811374733 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfa000ad30 con 0x7fdfac0662e0 2024-03-22T07:19:11.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.341+0000 7fdfb1722700 1 -- 172.21.15.63:0/3811374733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfac111270 con 0x7fdfac0662e0 2024-03-22T07:19:11.342 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.341+0000 7fdf937fe700 1 -- 172.21.15.63:0/3811374733 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfa00113a0 con 0x7fdfac0662e0 2024-03-22T07:19:11.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.341+0000 7fdfb1722700 1 -- 172.21.15.63:0/3811374733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfac111830 con 0x7fdfac0662e0 2024-03-22T07:19:11.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.342+0000 7fdfb1722700 1 -- 172.21.15.63:0/3811374733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfac0f8fb0 con 0x7fdfac0662e0 2024-03-22T07:19:11.343 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.343+0000 7fdf937fe700 1 -- 172.21.15.63:0/3811374733 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7fdfa0012400 con 0x7fdfac0662e0 2024-03-22T07:19:11.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.344+0000 7fdf937fe700 1 --2- 172.21.15.63:0/3811374733 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fdf9406cda0 0x7fdf9406f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:11.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.344+0000 7fdf937fe700 1 -- 172.21.15.63:0/3811374733 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fdfa008c9e0 con 0x7fdfac0662e0 2024-03-22T07:19:11.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.344+0000 7fdfaa7fc700 1 --2- 172.21.15.63:0/3811374733 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fdf9406cda0 0x7fdf9406f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:11.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.345+0000 7fdfaa7fc700 1 --2- 172.21.15.63:0/3811374733 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fdf9406cda0 0x7fdf9406f250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00b970 tx=0x7fdf9c009e30 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:11.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.348+0000 7fdf937fe700 1 -- 172.21.15.63:0/3811374733 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fdfa005a800 con 0x7fdfac0662e0 2024-03-22T07:19:11.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:11 smithi063 ceph-mon[29200]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:19:11.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.639+0000 7fdfb1722700 1 -- 172.21.15.63:0/3811374733 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdfac05f5f0 con 0x7fdfac0662e0 2024-03-22T07:19:11.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.640+0000 7fdf937fe700 1 -- 172.21.15.63:0/3811374733 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fdfa005a390 con 0x7fdfac0662e0 2024-03-22T07:19:11.640 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:19:11.640 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:19:11.640 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:19:11.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:19:11.640 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:19:11.640 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:19:11.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:19:11.640 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:19:11.640 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:19:11.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:19:11.640 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:19:11.640 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:19:11.640 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:19:11.641 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:19:11.641 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 12, 2024-03-22T07:19:11.641 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:19:11.641 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:19:11.641 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:19:11.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.642+0000 7fdfb1722700 1 -- 172.21.15.63:0/3811374733 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fdf9406cda0 msgr2=0x7fdf9406f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:11.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.642+0000 7fdfb1722700 1 --2- 172.21.15.63:0/3811374733 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fdf9406cda0 0x7fdf9406f250 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00b970 tx=0x7fdf9c009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:11.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.642+0000 7fdfb1722700 1 -- 172.21.15.63:0/3811374733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfac0662e0 msgr2=0x7fdfac10fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:11.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.642+0000 7fdfb1722700 1 --2- 172.21.15.63:0/3811374733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfac0662e0 0x7fdfac10fd10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000c890 tx=0x7fdfa00044c0 comp rx=0 tx=0).stop 2024-03-22T07:19:11.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.642+0000 7fdfb1722700 1 -- 172.21.15.63:0/3811374733 shutdown_connections 2024-03-22T07:19:11.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.642+0000 7fdfb1722700 1 --2- 172.21.15.63:0/3811374733 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fdf9406cda0 0x7fdf9406f250 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:11.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.642+0000 7fdfb1722700 1 --2- 172.21.15.63:0/3811374733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdfac0662e0 0x7fdfac10fd10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:11.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.642+0000 7fdfb1722700 1 --2- 172.21.15.63:0/3811374733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfac0fff00 0x7fdfac110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:11.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.642+0000 7fdfb1722700 1 -- 172.21.15.63:0/3811374733 >> 172.21.15.63:0/3811374733 conn(0x7fdfac0f7430 msgr2=0x7fdfac0fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:11.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.642+0000 7fdfb1722700 1 -- 172.21.15.63:0/3811374733 shutdown_connections 2024-03-22T07:19:11.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.642+0000 7fdfb1722700 1 -- 172.21.15.63:0/3811374733 wait complete. 2024-03-22T07:19:11.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.752+0000 7fb5366b8700 1 -- 172.21.15.63:0/1965437029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5300fff60 msgr2=0x7fb530102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:11.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.752+0000 7fb5366b8700 1 --2- 172.21.15.63:0/1965437029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5300fff60 0x7fb530102340 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb520004940 tx=0x7fb520009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:11.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.752+0000 7fb5366b8700 1 -- 172.21.15.63:0/1965437029 shutdown_connections 2024-03-22T07:19:11.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.752+0000 7fb5366b8700 1 --2- 172.21.15.63:0/1965437029 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb5300fff60 0x7fb530102340 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:11.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.752+0000 7fb5366b8700 1 --2- 172.21.15.63:0/1965437029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb530066280 0x7fb5300ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:11.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.752+0000 7fb5366b8700 1 -- 172.21.15.63:0/1965437029 >> 172.21.15.63:0/1965437029 conn(0x7fb5300f7430 msgr2=0x7fb5300f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:11.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.752+0000 7fb5366b8700 1 -- 172.21.15.63:0/1965437029 shutdown_connections 2024-03-22T07:19:11.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.752+0000 7fb5366b8700 1 -- 172.21.15.63:0/1965437029 wait complete. 2024-03-22T07:19:11.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.753+0000 7fb5366b8700 1 Processor -- start 2024-03-22T07:19:11.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.753+0000 7fb5366b8700 1 -- start start 2024-03-22T07:19:11.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.753+0000 7fb5366b8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb530066280 0x7fb53010daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:11.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.753+0000 7fb5366b8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5300fff60 0x7fb53010e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:11.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.753+0000 7fb5366b8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53010e6c0 con 0x7fb530066280 2024-03-22T07:19:11.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.753+0000 7fb5366b8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53010ee10 con 0x7fb5300fff60 2024-03-22T07:19:11.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.753+0000 7fb52ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb530066280 0x7fb53010daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:11.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.753+0000 7fb52ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb530066280 0x7fb53010daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58146/0 (socket says 172.21.15.63:58146) 2024-03-22T07:19:11.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.753+0000 7fb52ffff700 1 -- 172.21.15.63:0/2234205960 learned_addr learned my addr 172.21.15.63:0/2234205960 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:19:11.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.754+0000 7fb52f7fe700 1 --2- 172.21.15.63:0/2234205960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5300fff60 0x7fb53010e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:11.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.754+0000 7fb52ffff700 1 -- 172.21.15.63:0/2234205960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5300fff60 msgr2=0x7fb53010e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:11.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.754+0000 7fb52ffff700 1 --2- 172.21.15.63:0/2234205960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5300fff60 0x7fb53010e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:11.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.754+0000 7fb52ffff700 1 -- 172.21.15.63:0/2234205960 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb520005040 con 0x7fb530066280 2024-03-22T07:19:11.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.754+0000 7fb52ffff700 1 --2- 172.21.15.63:0/2234205960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb530066280 0x7fb53010daf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb51800b790 tx=0x7fb518009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:11.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.754+0000 7fb52cff9700 1 -- 172.21.15.63:0/2234205960 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb51800f6f0 con 0x7fb530066280 2024-03-22T07:19:11.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.754+0000 7fb52cff9700 1 -- 172.21.15.63:0/2234205960 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb51800fd20 con 0x7fb530066280 2024-03-22T07:19:11.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.754+0000 7fb5366b8700 1 -- 172.21.15.63:0/2234205960 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53010f0f0 con 0x7fb530066280 2024-03-22T07:19:11.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.754+0000 7fb52cff9700 1 -- 172.21.15.63:0/2234205960 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5180103a0 con 0x7fb530066280 2024-03-22T07:19:11.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.754+0000 7fb5366b8700 1 -- 172.21.15.63:0/2234205960 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53010f6b0 con 0x7fb530066280 2024-03-22T07:19:11.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.755+0000 7fb5366b8700 1 -- 172.21.15.63:0/2234205960 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53005f5f0 con 0x7fb530066280 2024-03-22T07:19:11.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.756+0000 7fb52cff9700 1 -- 172.21.15.63:0/2234205960 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 32) v1 ==== 90983+0+0 (secure 0 0 0) 0x7fb518011400 con 0x7fb530066280 2024-03-22T07:19:11.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.756+0000 7fb52cff9700 1 --2- 172.21.15.63:0/2234205960 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fb51c06cda0 0x7fb51c06f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:11.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.756+0000 7fb52cff9700 1 -- 172.21.15.63:0/2234205960 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb51808bea0 con 0x7fb530066280 2024-03-22T07:19:11.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.756+0000 7fb52f7fe700 1 --2- 172.21.15.63:0/2234205960 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fb51c06cda0 0x7fb51c06f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:11.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.757+0000 7fb52f7fe700 1 --2- 172.21.15.63:0/2234205960 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fb51c06cda0 0x7fb51c06f250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb52000b3b0 tx=0x7fb520009e30 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:11.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.761+0000 7fb52cff9700 1 -- 172.21.15.63:0/2234205960 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fb518059c20 con 0x7fb530066280 2024-03-22T07:19:11.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:11 smithi069 ceph-mon[36441]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:19:11.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.992+0000 7fb5366b8700 1 -- 172.21.15.63:0/2234205960 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb53010f9f0 con 0x7fb51c06cda0 2024-03-22T07:19:11.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.993+0000 7fb52cff9700 1 -- 172.21.15.63:0/2234205960 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fb53010f9f0 con 0x7fb51c06cda0 2024-03-22T07:19:11.993 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:19:11.993 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:19:11.993 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:19:11.993 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-22T07:19:11.993 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-03-22T07:19:11.993 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-03-22T07:19:11.994 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-03-22T07:19:11.994 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "2/2 daemons upgraded", 2024-03-22T07:19:11.994 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading grafana daemons", 2024-03-22T07:19:11.994 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:19:11.994 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:19:11.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.995+0000 7fb5366b8700 1 -- 172.21.15.63:0/2234205960 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fb51c06cda0 msgr2=0x7fb51c06f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:11.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.995+0000 7fb5366b8700 1 --2- 172.21.15.63:0/2234205960 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fb51c06cda0 0x7fb51c06f250 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb52000b3b0 tx=0x7fb520009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:11.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.995+0000 7fb5366b8700 1 -- 172.21.15.63:0/2234205960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb530066280 msgr2=0x7fb53010daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:11.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.995+0000 7fb5366b8700 1 --2- 172.21.15.63:0/2234205960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb530066280 0x7fb53010daf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb51800b790 tx=0x7fb518009220 comp rx=0 tx=0).stop 2024-03-22T07:19:11.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.995+0000 7fb5366b8700 1 -- 172.21.15.63:0/2234205960 shutdown_connections 2024-03-22T07:19:11.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.995+0000 7fb5366b8700 1 --2- 172.21.15.63:0/2234205960 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fb51c06cda0 0x7fb51c06f250 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:11.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.995+0000 7fb5366b8700 1 --2- 172.21.15.63:0/2234205960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb530066280 0x7fb53010daf0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:11.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.995+0000 7fb5366b8700 1 --2- 172.21.15.63:0/2234205960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb5300fff60 0x7fb53010e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:11.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.995+0000 7fb5366b8700 1 -- 172.21.15.63:0/2234205960 >> 172.21.15.63:0/2234205960 conn(0x7fb5300f7430 msgr2=0x7fb5300f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:11.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.996+0000 7fb5366b8700 1 -- 172.21.15.63:0/2234205960 shutdown_connections 2024-03-22T07:19:11.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:11.996+0000 7fb5366b8700 1 -- 172.21.15.63:0/2234205960 wait complete. 2024-03-22T07:19:12.774 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:12 smithi063 ceph-mon[29200]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:19:12.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:12 smithi063 ceph-mon[29200]: pgmap v46: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:19:12.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:12 smithi063 ceph-mon[29200]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:19:12.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:12 smithi063 ceph-mon[29200]: Upgrade: Updating grafana.smithi063 2024-03-22T07:19:12.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:12 smithi063 ceph-mon[29200]: Deploying daemon grafana.smithi063 on smithi063 2024-03-22T07:19:12.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:12 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3811374733' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:12.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:12 smithi069 ceph-mon[36441]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:19:12.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:12 smithi069 ceph-mon[36441]: pgmap v46: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:19:12.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:12 smithi069 ceph-mon[36441]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:19:12.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:12 smithi069 ceph-mon[36441]: Upgrade: Updating grafana.smithi063 2024-03-22T07:19:12.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:12 smithi069 ceph-mon[36441]: Deploying daemon grafana.smithi063 on smithi063 2024-03-22T07:19:12.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:12 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/3811374733' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:13 smithi063 ceph-mon[29200]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:19:13.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:13 smithi069 ceph-mon[36441]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:19:14.760 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:14 smithi063 ceph-mon[29200]: pgmap v47: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:14.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:14 smithi069 ceph-mon[36441]: pgmap v47: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:15.724 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:15 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:19:15.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:15 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:19:16.665 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:16 smithi063 ceph-mon[29200]: pgmap v48: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:16.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:16 smithi069 ceph-mon[36441]: pgmap v48: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:17.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:17 smithi063 ceph-mon[29200]: mgrmap e33: smithi063.mnxayh(active, since 91s), standbys: smithi069.gfziwh 2024-03-22T07:19:17.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:17 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:17.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:17 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:17.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:17 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:19:17.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:17 smithi069 ceph-mon[36441]: mgrmap e33: smithi063.mnxayh(active, since 91s), standbys: smithi069.gfziwh 2024-03-22T07:19:17.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:17 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:17.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:17 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:17.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:17 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:19:18.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:18 smithi063 ceph-mon[29200]: pgmap v49: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-22T07:19:19.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:18 smithi069 ceph-mon[36441]: pgmap v49: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-22T07:19:20.831 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:20 smithi069 ceph-mon[36441]: pgmap v50: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-22T07:19:20.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:20 smithi063 ceph-mon[29200]: pgmap v50: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-22T07:19:22.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:22 smithi063 ceph-mon[29200]: pgmap v51: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-22T07:19:22.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:22 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:22.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:22 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:23.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:22 smithi069 ceph-mon[36441]: pgmap v51: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-22T07:19:23.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:22 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:23.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:22 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:24.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:23 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:24.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:23 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:24.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:23 smithi069 ceph-mon[36441]: pgmap v52: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-22T07:19:24.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:23 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:24.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:23 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:24.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:23 smithi063 ceph-mon[29200]: pgmap v52: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-22T07:19:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:26 smithi063 ceph-mon[29200]: pgmap v53: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-22T07:19:26.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:26 smithi069 ceph-mon[36441]: pgmap v53: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-22T07:19:27.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:27.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:27.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:19:27.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:19:27.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:27.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:19:27.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:19:27.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:19:27.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: Upgrade: Finalizing container_image settings 2024-03-22T07:19:27.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:27.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-22T07:19:27.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-22T07:19:27.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:19:27.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-22T07:19:27.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-22T07:19:27.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-22T07:19:27.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-22T07:19:27.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-22T07:19:27.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-22T07:19:27.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-22T07:19:27.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-22T07:19:27.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:19:27.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-22T07:19:27.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-22T07:19:27.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-22T07:19:27.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-22T07:19:27.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-22T07:19:27.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-22T07:19:27.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-22T07:19:27.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-22T07:19:27.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:19:27.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:19:27.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:19:27.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:19:27.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:19:27.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:19:27.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: Upgrade: Complete! 2024-03-22T07:19:27.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-22T07:19:27.784 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-22T07:19:27.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:19:27.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:19:27.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:19:27.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:27.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:19:27.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:19:27.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:19:27.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:27 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:27.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:27.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:27.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:19:27.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:19:27.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:27.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:19:27.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-22T07:19:27.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:19:27.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:27.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: Upgrade: Finalizing container_image settings 2024-03-22T07:19:27.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:27.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-22T07:19:27.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-22T07:19:27.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:19:27.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-22T07:19:27.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-22T07:19:27.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-22T07:19:27.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-22T07:19:27.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-22T07:19:27.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-22T07:19:27.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-22T07:19:27.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-22T07:19:27.876 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:19:27.877 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-22T07:19:27.877 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-22T07:19:27.877 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-22T07:19:27.877 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-22T07:19:27.877 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-22T07:19:27.877 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-22T07:19:27.877 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-22T07:19:27.877 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-22T07:19:27.877 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:19:27.877 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:19:27.877 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:19:27.877 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:19:27.877 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:19:27.877 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:19:27.877 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: Upgrade: Complete! 2024-03-22T07:19:27.878 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-22T07:19:27.878 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-22T07:19:27.878 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:19:27.878 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:19:27.878 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:19:27.878 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:27.878 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:19:27.878 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:19:27.878 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:19:27.878 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:27 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:28.660 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:28 smithi063 ceph-mon[29200]: pgmap v54: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-22T07:19:28.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:28 smithi069 ceph-mon[36441]: pgmap v54: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-22T07:19:30.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:30 smithi069 ceph-mon[36441]: pgmap v55: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:30.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:30 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:19:30.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:30 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:30.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:30 smithi063 ceph-mon[29200]: pgmap v55: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:30.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:30 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:19:30.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:30 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:19:32.774 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:32 smithi063 ceph-mon[29200]: pgmap v56: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-22T07:19:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:32 smithi069 ceph-mon[36441]: pgmap v56: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 5 op/s 2024-03-22T07:19:34.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:34 smithi069 ceph-mon[36441]: pgmap v57: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:34.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:34 smithi063 ceph-mon[29200]: pgmap v57: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:36.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:36 smithi069 ceph-mon[36441]: pgmap v58: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:36.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:36 smithi063 ceph-mon[29200]: pgmap v58: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:38.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:38 smithi069 ceph-mon[36441]: pgmap v59: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:19:38.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:38 smithi063 ceph-mon[29200]: pgmap v59: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:19:40.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:40 smithi069 ceph-mon[36441]: pgmap v60: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:40.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:40 smithi063 ceph-mon[29200]: pgmap v60: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:42.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.114+0000 7f521a923700 1 -- 172.21.15.63:0/2426844191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52140fff00 msgr2=0x7f5214102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:42.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.114+0000 7f521a923700 1 --2- 172.21.15.63:0/2426844191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52140fff00 0x7f5214102320 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5208004940 tx=0x7f5208009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:42.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.114+0000 7f521a923700 1 -- 172.21.15.63:0/2426844191 shutdown_connections 2024-03-22T07:19:42.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.114+0000 7f521a923700 1 --2- 172.21.15.63:0/2426844191 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52140fff00 0x7f5214102320 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:42.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.114+0000 7f521a923700 1 --2- 172.21.15.63:0/2426844191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5214066f50 0x7f52140ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:42.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.114+0000 7f521a923700 1 -- 172.21.15.63:0/2426844191 >> 172.21.15.63:0/2426844191 conn(0x7f52140f7450 msgr2=0x7f52140f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:42.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.114+0000 7f521a923700 1 -- 172.21.15.63:0/2426844191 shutdown_connections 2024-03-22T07:19:42.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.114+0000 7f521a923700 1 -- 172.21.15.63:0/2426844191 wait complete. 2024-03-22T07:19:42.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.114+0000 7f521a923700 1 Processor -- start 2024-03-22T07:19:42.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.115+0000 7f521a923700 1 -- start start 2024-03-22T07:19:42.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.115+0000 7f521a923700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5214066f50 0x7f52140fd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:42.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.115+0000 7f521a923700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52140fff00 0x7f52140fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:42.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.115+0000 7f521a923700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52140fd810 con 0x7f52140fff00 2024-03-22T07:19:42.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.115+0000 7f521a923700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52140fbe00 con 0x7f5214066f50 2024-03-22T07:19:42.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.115+0000 7f52137fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52140fff00 0x7f52140fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:42.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.115+0000 7f52137fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52140fff00 0x7f52140fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:59854/0 (socket says 172.21.15.63:59854) 2024-03-22T07:19:42.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.115+0000 7f52137fe700 1 -- 172.21.15.63:0/1172614584 learned_addr learned my addr 172.21.15.63:0/1172614584 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:19:42.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.115+0000 7f5213fff700 1 --2- 172.21.15.63:0/1172614584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5214066f50 0x7f52140fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:42.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.115+0000 7f52137fe700 1 -- 172.21.15.63:0/1172614584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5214066f50 msgr2=0x7f52140fd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:42.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.115+0000 7f52137fe700 1 --2- 172.21.15.63:0/1172614584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5214066f50 0x7f52140fd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:42.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.115+0000 7f52137fe700 1 -- 172.21.15.63:0/1172614584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5208005040 con 0x7f52140fff00 2024-03-22T07:19:42.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.115+0000 7f5213fff700 1 --2- 172.21.15.63:0/1172614584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5214066f50 0x7f52140fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:19:42.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.116+0000 7f52137fe700 1 --2- 172.21.15.63:0/1172614584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52140fff00 0x7f52140fb8c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f520800b3b0 tx=0x7f5208009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:42.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.116+0000 7f5210ff9700 1 -- 172.21.15.63:0/1172614584 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f520800e040 con 0x7f52140fff00 2024-03-22T07:19:42.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.116+0000 7f5210ff9700 1 -- 172.21.15.63:0/1172614584 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f520800bcc0 con 0x7f52140fff00 2024-03-22T07:19:42.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.116+0000 7f521a923700 1 -- 172.21.15.63:0/1172614584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52140fc050 con 0x7f52140fff00 2024-03-22T07:19:42.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.116+0000 7f5210ff9700 1 -- 172.21.15.63:0/1172614584 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f520801c630 con 0x7f52140fff00 2024-03-22T07:19:42.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.116+0000 7f521a923700 1 -- 172.21.15.63:0/1172614584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52140fc610 con 0x7f52140fff00 2024-03-22T07:19:42.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.117+0000 7f521a923700 1 -- 172.21.15.63:0/1172614584 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521404c710 con 0x7f52140fff00 2024-03-22T07:19:42.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.117+0000 7f5210ff9700 1 -- 172.21.15.63:0/1172614584 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f5208013980 con 0x7f52140fff00 2024-03-22T07:19:42.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.118+0000 7f5210ff9700 1 --2- 172.21.15.63:0/1172614584 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f51fc06cc80 0x7f51fc06f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:42.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.118+0000 7f5210ff9700 1 -- 172.21.15.63:0/1172614584 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f520808cbd0 con 0x7f52140fff00 2024-03-22T07:19:42.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.118+0000 7f5213fff700 1 --2- 172.21.15.63:0/1172614584 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f51fc06cc80 0x7f51fc06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:42.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.119+0000 7f5213fff700 1 --2- 172.21.15.63:0/1172614584 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f51fc06cc80 0x7f51fc06f130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5204007fd0 tx=0x7f5204009040 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:42.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.123+0000 7f5210ff9700 1 -- 172.21.15.63:0/1172614584 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f520805aa60 con 0x7f52140fff00 2024-03-22T07:19:42.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.344+0000 7f521a923700 1 -- 172.21.15.63:0/1172614584 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f52140f8f40 con 0x7f51fc06cc80 2024-03-22T07:19:42.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.345+0000 7f5210ff9700 1 -- 172.21.15.63:0/1172614584 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f52140f8f40 con 0x7f51fc06cc80 2024-03-22T07:19:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.347+0000 7f521a923700 1 -- 172.21.15.63:0/1172614584 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f51fc06cc80 msgr2=0x7f51fc06f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.347+0000 7f521a923700 1 --2- 172.21.15.63:0/1172614584 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f51fc06cc80 0x7f51fc06f130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5204007fd0 tx=0x7f5204009040 comp rx=0 tx=0).stop 2024-03-22T07:19:42.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.347+0000 7f521a923700 1 -- 172.21.15.63:0/1172614584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52140fff00 msgr2=0x7f52140fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:42.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.347+0000 7f521a923700 1 --2- 172.21.15.63:0/1172614584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52140fff00 0x7f52140fb8c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f520800b3b0 tx=0x7f5208009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:42.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.347+0000 7f521a923700 1 -- 172.21.15.63:0/1172614584 shutdown_connections 2024-03-22T07:19:42.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.347+0000 7f521a923700 1 --2- 172.21.15.63:0/1172614584 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f51fc06cc80 0x7f51fc06f130 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:42.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.347+0000 7f521a923700 1 --2- 172.21.15.63:0/1172614584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5214066f50 0x7f52140fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:42.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.347+0000 7f521a923700 1 --2- 172.21.15.63:0/1172614584 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52140fff00 0x7f52140fb8c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:42.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.347+0000 7f521a923700 1 -- 172.21.15.63:0/1172614584 >> 172.21.15.63:0/1172614584 conn(0x7f52140f7450 msgr2=0x7f52140f7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:42.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.347+0000 7f521a923700 1 -- 172.21.15.63:0/1172614584 shutdown_connections 2024-03-22T07:19:42.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:42.347+0000 7f521a923700 1 -- 172.21.15.63:0/1172614584 wait complete. 2024-03-22T07:19:42.774 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:42 smithi063 ceph-mon[29200]: pgmap v61: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:19:42.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:42 smithi069 ceph-mon[36441]: pgmap v61: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:19:43.009 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-03-22T07:19:43.274 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:19:43.711 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:43 smithi063 ceph-mon[29200]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:19:43.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:43 smithi069 ceph-mon[36441]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:19:44.777 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:44 smithi063 ceph-mon[29200]: pgmap v62: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:44.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:44 smithi069 ceph-mon[36441]: pgmap v62: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:44.989 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.988+0000 7fa31c34a700 1 -- 172.21.15.63:0/97292073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3141026c0 msgr2=0x7fa3140fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:44.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.988+0000 7fa31c34a700 1 --2- 172.21.15.63:0/97292073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3141026c0 0x7fa3140fbd30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa304004940 tx=0x7fa304009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:44.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.988+0000 7fa31c34a700 1 -- 172.21.15.63:0/97292073 shutdown_connections 2024-03-22T07:19:44.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.988+0000 7fa31c34a700 1 --2- 172.21.15.63:0/97292073 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3141026c0 0x7fa3140fbd30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:44.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.988+0000 7fa31c34a700 1 --2- 172.21.15.63:0/97292073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa314101d20 0x7fa3141020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:44.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.988+0000 7fa31c34a700 1 -- 172.21.15.63:0/97292073 >> 172.21.15.63:0/97292073 conn(0x7fa3140f7870 msgr2=0x7fa3140f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:44.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.988+0000 7fa31c34a700 1 -- 172.21.15.63:0/97292073 shutdown_connections 2024-03-22T07:19:44.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.988+0000 7fa31c34a700 1 -- 172.21.15.63:0/97292073 wait complete. 2024-03-22T07:19:44.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.988+0000 7fa31c34a700 1 Processor -- start 2024-03-22T07:19:44.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.988+0000 7fa31c34a700 1 -- start start 2024-03-22T07:19:44.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.989+0000 7fa31c34a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa314101d20 0x7fa31410eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:44.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.989+0000 7fa31c34a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3141026c0 0x7fa31410f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:44.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.989+0000 7fa31c34a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31410f5c0 con 0x7fa3141026c0 2024-03-22T07:19:44.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.989+0000 7fa31c34a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31410f700 con 0x7fa314101d20 2024-03-22T07:19:44.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.989+0000 7fa3198e5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3141026c0 0x7fa31410f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:44.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.989+0000 7fa3198e5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3141026c0 0x7fa31410f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41236/0 (socket says 172.21.15.63:41236) 2024-03-22T07:19:44.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.989+0000 7fa3198e5700 1 -- 172.21.15.63:0/1310253805 learned_addr learned my addr 172.21.15.63:0/1310253805 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:19:44.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.989+0000 7fa31a0e6700 1 --2- 172.21.15.63:0/1310253805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa314101d20 0x7fa31410eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:44.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.989+0000 7fa3198e5700 1 -- 172.21.15.63:0/1310253805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa314101d20 msgr2=0x7fa31410eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:44.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.989+0000 7fa3198e5700 1 --2- 172.21.15.63:0/1310253805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa314101d20 0x7fa31410eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:44.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.989+0000 7fa3198e5700 1 -- 172.21.15.63:0/1310253805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa304005040 con 0x7fa3141026c0 2024-03-22T07:19:44.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.989+0000 7fa3198e5700 1 --2- 172.21.15.63:0/1310253805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3141026c0 0x7fa31410f080 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa304009100 tx=0x7fa304009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:44.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.989+0000 7fa30affd700 1 -- 172.21.15.63:0/1310253805 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa30400e040 con 0x7fa3141026c0 2024-03-22T07:19:44.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.989+0000 7fa30affd700 1 -- 172.21.15.63:0/1310253805 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa304009af0 con 0x7fa3141026c0 2024-03-22T07:19:44.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.989+0000 7fa30affd700 1 -- 172.21.15.63:0/1310253805 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa30401c3f0 con 0x7fa3141026c0 2024-03-22T07:19:44.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.989+0000 7fa31c34a700 1 -- 172.21.15.63:0/1310253805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31410f980 con 0x7fa3141026c0 2024-03-22T07:19:44.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.990+0000 7fa31c34a700 1 -- 172.21.15.63:0/1310253805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa314117f00 con 0x7fa3141026c0 2024-03-22T07:19:44.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.991+0000 7fa31c34a700 1 -- 172.21.15.63:0/1310253805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3140fd470 con 0x7fa3141026c0 2024-03-22T07:19:44.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.991+0000 7fa30affd700 1 -- 172.21.15.63:0/1310253805 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7fa30400baf0 con 0x7fa3141026c0 2024-03-22T07:19:44.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.991+0000 7fa30affd700 1 --2- 172.21.15.63:0/1310253805 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa30006cc80 0x7fa30006f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:44.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.992+0000 7fa30affd700 1 -- 172.21.15.63:0/1310253805 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa30408d100 con 0x7fa3141026c0 2024-03-22T07:19:44.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.992+0000 7fa31a0e6700 1 --2- 172.21.15.63:0/1310253805 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa30006cc80 0x7fa30006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:44.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.992+0000 7fa31a0e6700 1 --2- 172.21.15.63:0/1310253805 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa30006cc80 0x7fa30006f130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa310004eb0 tx=0x7fa31000b040 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:44.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:44.996+0000 7fa30affd700 1 -- 172.21.15.63:0/1310253805 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fa304057500 con 0x7fa3141026c0 2024-03-22T07:19:45.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:45.287+0000 7fa31c34a700 1 -- 172.21.15.63:0/1310253805 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa314108e00 con 0x7fa3141026c0 2024-03-22T07:19:45.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:45.288+0000 7fa30affd700 1 -- 172.21.15.63:0/1310253805 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fa30405abc0 con 0x7fa3141026c0 2024-03-22T07:19:45.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:45.290+0000 7fa31c34a700 1 -- 172.21.15.63:0/1310253805 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa30006cc80 msgr2=0x7fa30006f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:45.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:45.290+0000 7fa31c34a700 1 --2- 172.21.15.63:0/1310253805 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa30006cc80 0x7fa30006f130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa310004eb0 tx=0x7fa31000b040 comp rx=0 tx=0).stop 2024-03-22T07:19:45.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:45.290+0000 7fa31c34a700 1 -- 172.21.15.63:0/1310253805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3141026c0 msgr2=0x7fa31410f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:45.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:45.290+0000 7fa31c34a700 1 --2- 172.21.15.63:0/1310253805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3141026c0 0x7fa31410f080 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa304009100 tx=0x7fa304009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:45.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:45.290+0000 7fa31c34a700 1 -- 172.21.15.63:0/1310253805 shutdown_connections 2024-03-22T07:19:45.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:45.290+0000 7fa31c34a700 1 --2- 172.21.15.63:0/1310253805 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa30006cc80 0x7fa30006f130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:45.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:45.290+0000 7fa31c34a700 1 --2- 172.21.15.63:0/1310253805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa314101d20 0x7fa31410eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:45.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:45.290+0000 7fa31c34a700 1 --2- 172.21.15.63:0/1310253805 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa3141026c0 0x7fa31410f080 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:45.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:45.290+0000 7fa31c34a700 1 -- 172.21.15.63:0/1310253805 >> 172.21.15.63:0/1310253805 conn(0x7fa3140f7870 msgr2=0x7fa3140f8920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:45.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:45.290+0000 7fa31c34a700 1 -- 172.21.15.63:0/1310253805 shutdown_connections 2024-03-22T07:19:45.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:45.290+0000 7fa31c34a700 1 -- 172.21.15.63:0/1310253805 wait complete. 2024-03-22T07:19:45.300 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:19:45.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:45 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:19:45.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:45 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/1310253805' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:45.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:45 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:19:45.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:45 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/1310253805' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:45.948 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-03-22T07:19:46.208 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:19:46.603 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:46 smithi063 ceph-mon[29200]: pgmap v63: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:46.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:46 smithi069 ceph-mon[36441]: pgmap v63: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:47.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.820+0000 7f19e7104700 1 -- 172.21.15.63:0/3917174104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19e0101d70 msgr2=0x7f19e0102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:47.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.820+0000 7f19e7104700 1 --2- 172.21.15.63:0/3917174104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19e0101d70 0x7f19e0102140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009ce0 tx=0x7f19d8004d10 comp rx=0 tx=0).stop 2024-03-22T07:19:47.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.820+0000 7f19e7104700 1 -- 172.21.15.63:0/3917174104 shutdown_connections 2024-03-22T07:19:47.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.820+0000 7f19e7104700 1 --2- 172.21.15.63:0/3917174104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19e0102710 0x7f19e00fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:47.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.820+0000 7f19e7104700 1 --2- 172.21.15.63:0/3917174104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19e0101d70 0x7f19e0102140 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:47.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.820+0000 7f19e7104700 1 -- 172.21.15.63:0/3917174104 >> 172.21.15.63:0/3917174104 conn(0x7f19e0072bc0 msgr2=0x7f19e0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:47.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.820+0000 7f19e7104700 1 -- 172.21.15.63:0/3917174104 shutdown_connections 2024-03-22T07:19:47.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.820+0000 7f19e7104700 1 -- 172.21.15.63:0/3917174104 wait complete. 2024-03-22T07:19:47.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.821+0000 7f19e7104700 1 Processor -- start 2024-03-22T07:19:47.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.821+0000 7f19e7104700 1 -- start start 2024-03-22T07:19:47.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.821+0000 7f19e7104700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19e0101d70 0x7f19e010eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:47.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.821+0000 7f19e7104700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19e0102710 0x7f19e010f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:47.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.821+0000 7f19e7104700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e010f920 con 0x7f19e0101d70 2024-03-22T07:19:47.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.821+0000 7f19e7104700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e010fa60 con 0x7f19e0102710 2024-03-22T07:19:47.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.821+0000 7f19e4ea0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19e0101d70 0x7f19e010eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:47.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.822+0000 7f19e4ea0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19e0101d70 0x7f19e010eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41252/0 (socket says 172.21.15.63:41252) 2024-03-22T07:19:47.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.822+0000 7f19e4ea0700 1 -- 172.21.15.63:0/4036618001 learned_addr learned my addr 172.21.15.63:0/4036618001 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:19:47.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.822+0000 7f19dffff700 1 --2- 172.21.15.63:0/4036618001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19e0102710 0x7f19e010f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:47.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.822+0000 7f19e4ea0700 1 -- 172.21.15.63:0/4036618001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19e0102710 msgr2=0x7f19e010f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:47.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.822+0000 7f19e4ea0700 1 --2- 172.21.15.63:0/4036618001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19e0102710 0x7f19e010f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:47.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.822+0000 7f19e4ea0700 1 -- 172.21.15.63:0/4036618001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d8009940 con 0x7f19e0101d70 2024-03-22T07:19:47.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.822+0000 7f19e4ea0700 1 --2- 172.21.15.63:0/4036618001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19e0101d70 0x7f19e010eea0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f19d8005010 tx=0x7f19d8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:47.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.822+0000 7f19dd7fa700 1 -- 172.21.15.63:0/4036618001 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d800e040 con 0x7f19e0101d70 2024-03-22T07:19:47.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.822+0000 7f19e7104700 1 -- 172.21.15.63:0/4036618001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e0117d50 con 0x7f19e0101d70 2024-03-22T07:19:47.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.822+0000 7f19dd7fa700 1 -- 172.21.15.63:0/4036618001 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19d8007d10 con 0x7f19e0101d70 2024-03-22T07:19:47.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.822+0000 7f19dd7fa700 1 -- 172.21.15.63:0/4036618001 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d8012650 con 0x7f19e0101d70 2024-03-22T07:19:47.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.822+0000 7f19e7104700 1 -- 172.21.15.63:0/4036618001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e0118280 con 0x7f19e0101d70 2024-03-22T07:19:47.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.823+0000 7f19e7104700 1 -- 172.21.15.63:0/4036618001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e00ff080 con 0x7f19e0101d70 2024-03-22T07:19:47.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.824+0000 7f19dd7fa700 1 -- 172.21.15.63:0/4036618001 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f19d801ca10 con 0x7f19e0101d70 2024-03-22T07:19:47.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.824+0000 7f19dd7fa700 1 --2- 172.21.15.63:0/4036618001 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f19c806cc90 0x7f19c806f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:47.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.824+0000 7f19dd7fa700 1 -- 172.21.15.63:0/4036618001 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f19d808db60 con 0x7f19e0101d70 2024-03-22T07:19:47.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.824+0000 7f19dffff700 1 --2- 172.21.15.63:0/4036618001 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f19c806cc90 0x7f19c806f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:47.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.825+0000 7f19dffff700 1 --2- 172.21.15.63:0/4036618001 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f19c806cc90 0x7f19c806f140 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f19e0066020 tx=0x7f19d400b040 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:47.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:47.829+0000 7f19dd7fa700 1 -- 172.21.15.63:0/4036618001 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f19d8057e40 con 0x7f19e0101d70 2024-03-22T07:19:48.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:48.117+0000 7f19e7104700 1 -- 172.21.15.63:0/4036618001 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f19e004bed0 con 0x7f19e0101d70 2024-03-22T07:19:48.117 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:48.117+0000 7f19dd7fa700 1 -- 172.21.15.63:0/4036618001 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f19d8004e10 con 0x7f19e0101d70 2024-03-22T07:19:48.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:48.119+0000 7f19e7104700 1 -- 172.21.15.63:0/4036618001 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f19c806cc90 msgr2=0x7f19c806f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:48.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:48.119+0000 7f19e7104700 1 --2- 172.21.15.63:0/4036618001 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f19c806cc90 0x7f19c806f140 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f19e0066020 tx=0x7f19d400b040 comp rx=0 tx=0).stop 2024-03-22T07:19:48.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:48.119+0000 7f19e7104700 1 -- 172.21.15.63:0/4036618001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19e0101d70 msgr2=0x7f19e010eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:48.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:48.119+0000 7f19e7104700 1 --2- 172.21.15.63:0/4036618001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19e0101d70 0x7f19e010eea0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f19d8005010 tx=0x7f19d8004d10 comp rx=0 tx=0).stop 2024-03-22T07:19:48.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:48.119+0000 7f19e7104700 1 -- 172.21.15.63:0/4036618001 shutdown_connections 2024-03-22T07:19:48.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:48.119+0000 7f19e7104700 1 --2- 172.21.15.63:0/4036618001 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f19c806cc90 0x7f19c806f140 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:48.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:48.119+0000 7f19e7104700 1 --2- 172.21.15.63:0/4036618001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19e0101d70 0x7f19e010eea0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:48.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:48.119+0000 7f19e7104700 1 --2- 172.21.15.63:0/4036618001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19e0102710 0x7f19e010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:48.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:48.119+0000 7f19e7104700 1 -- 172.21.15.63:0/4036618001 >> 172.21.15.63:0/4036618001 conn(0x7f19e0072bc0 msgr2=0x7f19e00fb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:48.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:48.119+0000 7f19e7104700 1 -- 172.21.15.63:0/4036618001 shutdown_connections 2024-03-22T07:19:48.120 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:48.119+0000 7f19e7104700 1 -- 172.21.15.63:0/4036618001 wait complete. 2024-03-22T07:19:48.130 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)" 2024-03-22T07:19:48.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:48 smithi069 ceph-mon[36441]: pgmap v64: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:19:48.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:48 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/4036618001' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:48.781 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:48 smithi063 ceph-mon[29200]: pgmap v64: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:19:48.782 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:48 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/4036618001' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:48.809 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-03-22T07:19:49.072 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:19:50.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.675+0000 7fb5a3b57700 1 -- 172.21.15.63:0/923389660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb59c101d70 msgr2=0x7fb59c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:50.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.675+0000 7fb5a3b57700 1 --2- 172.21.15.63:0/923389660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb59c101d70 0x7fb59c102140 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb58c004940 tx=0x7fb58c009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:50.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.675+0000 7fb5a3b57700 1 -- 172.21.15.63:0/923389660 shutdown_connections 2024-03-22T07:19:50.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.675+0000 7fb5a3b57700 1 --2- 172.21.15.63:0/923389660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb59c102710 0x7fb59c0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:50.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.675+0000 7fb5a3b57700 1 --2- 172.21.15.63:0/923389660 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb59c101d70 0x7fb59c102140 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:50.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.675+0000 7fb5a3b57700 1 -- 172.21.15.63:0/923389660 >> 172.21.15.63:0/923389660 conn(0x7fb59c072bc0 msgr2=0x7fb59c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:50.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.675+0000 7fb5a3b57700 1 -- 172.21.15.63:0/923389660 shutdown_connections 2024-03-22T07:19:50.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.675+0000 7fb5a3b57700 1 -- 172.21.15.63:0/923389660 wait complete. 2024-03-22T07:19:50.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.676+0000 7fb5a3b57700 1 Processor -- start 2024-03-22T07:19:50.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.676+0000 7fb5a3b57700 1 -- start start 2024-03-22T07:19:50.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.676+0000 7fb5a3b57700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb59c101d70 0x7fb59c102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:50.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.676+0000 7fb5a3b57700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb59c102710 0x7fb59c103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:50.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.676+0000 7fb5a3b57700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c103650 con 0x7fb59c101d70 2024-03-22T07:19:50.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.676+0000 7fb5a3b57700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c1037c0 con 0x7fb59c102710 2024-03-22T07:19:50.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.676+0000 7fb5a18f3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb59c101d70 0x7fb59c102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:50.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.676+0000 7fb5a18f3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb59c101d70 0x7fb59c102bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41278/0 (socket says 172.21.15.63:41278) 2024-03-22T07:19:50.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.676+0000 7fb5a18f3700 1 -- 172.21.15.63:0/2147073414 learned_addr learned my addr 172.21.15.63:0/2147073414 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:19:50.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.676+0000 7fb5a10f2700 1 --2- 172.21.15.63:0/2147073414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb59c102710 0x7fb59c103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:50.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.676+0000 7fb5a18f3700 1 -- 172.21.15.63:0/2147073414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb59c102710 msgr2=0x7fb59c103110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:50.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.676+0000 7fb5a18f3700 1 --2- 172.21.15.63:0/2147073414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb59c102710 0x7fb59c103110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:50.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.676+0000 7fb5a18f3700 1 -- 172.21.15.63:0/2147073414 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c005040 con 0x7fb59c101d70 2024-03-22T07:19:50.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.677+0000 7fb5a18f3700 1 --2- 172.21.15.63:0/2147073414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb59c101d70 0x7fb59c102bd0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00b8b0 tx=0x7fb58c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:50.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.677+0000 7fb5927fc700 1 -- 172.21.15.63:0/2147073414 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb58c00e040 con 0x7fb59c101d70 2024-03-22T07:19:50.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.677+0000 7fb5a3b57700 1 -- 172.21.15.63:0/2147073414 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59c103a40 con 0x7fb59c101d70 2024-03-22T07:19:50.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.677+0000 7fb5927fc700 1 -- 172.21.15.63:0/2147073414 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb58c004290 con 0x7fb59c101d70 2024-03-22T07:19:50.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.677+0000 7fb5927fc700 1 -- 172.21.15.63:0/2147073414 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb58c01c3f0 con 0x7fb59c101d70 2024-03-22T07:19:50.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.677+0000 7fb5a3b57700 1 -- 172.21.15.63:0/2147073414 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59c11a210 con 0x7fb59c101d70 2024-03-22T07:19:50.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.678+0000 7fb5a3b57700 1 -- 172.21.15.63:0/2147073414 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59c0fd4c0 con 0x7fb59c101d70 2024-03-22T07:19:50.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.678+0000 7fb5927fc700 1 -- 172.21.15.63:0/2147073414 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7fb58c013420 con 0x7fb59c101d70 2024-03-22T07:19:50.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.679+0000 7fb5927fc700 1 --2- 172.21.15.63:0/2147073414 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fb58806cc80 0x7fb58806f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:50.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.679+0000 7fb5927fc700 1 -- 172.21.15.63:0/2147073414 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb58c08cff0 con 0x7fb59c101d70 2024-03-22T07:19:50.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.679+0000 7fb5a10f2700 1 --2- 172.21.15.63:0/2147073414 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fb58806cc80 0x7fb58806f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:50.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.679+0000 7fb5a10f2700 1 --2- 172.21.15.63:0/2147073414 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fb58806cc80 0x7fb58806f130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0fa670 tx=0x7fb598009040 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:50.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.683+0000 7fb5927fc700 1 -- 172.21.15.63:0/2147073414 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fb58c057350 con 0x7fb59c101d70 2024-03-22T07:19:50.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:50 smithi069 ceph-mon[36441]: pgmap v65: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:50.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:50 smithi063 ceph-mon[29200]: pgmap v65: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.972+0000 7fb5a3b57700 1 -- 172.21.15.63:0/2147073414 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb59c103a40 con 0x7fb59c101d70 2024-03-22T07:19:50.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.972+0000 7fb5927fc700 1 -- 172.21.15.63:0/2147073414 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fb59c103a40 con 0x7fb59c101d70 2024-03-22T07:19:50.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.974+0000 7fb5a3b57700 1 -- 172.21.15.63:0/2147073414 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fb58806cc80 msgr2=0x7fb58806f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:50.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.974+0000 7fb5a3b57700 1 --2- 172.21.15.63:0/2147073414 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fb58806cc80 0x7fb58806f130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0fa670 tx=0x7fb598009040 comp rx=0 tx=0).stop 2024-03-22T07:19:50.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.974+0000 7fb5a3b57700 1 -- 172.21.15.63:0/2147073414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb59c101d70 msgr2=0x7fb59c102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.974+0000 7fb5a3b57700 1 --2- 172.21.15.63:0/2147073414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb59c101d70 0x7fb59c102bd0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb58c00b8b0 tx=0x7fb58c009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.974+0000 7fb5a3b57700 1 -- 172.21.15.63:0/2147073414 shutdown_connections 2024-03-22T07:19:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.974+0000 7fb5a3b57700 1 --2- 172.21.15.63:0/2147073414 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fb58806cc80 0x7fb58806f130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.974+0000 7fb5a3b57700 1 --2- 172.21.15.63:0/2147073414 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb59c101d70 0x7fb59c102bd0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.974+0000 7fb5a3b57700 1 --2- 172.21.15.63:0/2147073414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb59c102710 0x7fb59c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.974+0000 7fb5a3b57700 1 -- 172.21.15.63:0/2147073414 >> 172.21.15.63:0/2147073414 conn(0x7fb59c072bc0 msgr2=0x7fb59c065940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.974+0000 7fb5a3b57700 1 -- 172.21.15.63:0/2147073414 shutdown_connections 2024-03-22T07:19:50.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:50.974+0000 7fb5a3b57700 1 -- 172.21.15.63:0/2147073414 wait complete. 2024-03-22T07:19:50.984 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:19:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:51 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2147073414' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:51.741 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-03-22T07:19:51.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:51 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/2147073414' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:19:52.004 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:19:52.774 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:52 smithi063 ceph-mon[29200]: pgmap v66: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:19:52.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:52 smithi069 ceph-mon[36441]: pgmap v66: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:19:53.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.676+0000 7f2e37658700 1 -- 172.21.15.63:0/3958945951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e30101d20 msgr2=0x7f2e301020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:53.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.676+0000 7f2e37658700 1 --2- 172.21.15.63:0/3958945951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e30101d20 0x7f2e301020f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c004940 tx=0x7f2e2c009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:53.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.676+0000 7f2e37658700 1 -- 172.21.15.63:0/3958945951 shutdown_connections 2024-03-22T07:19:53.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.676+0000 7f2e37658700 1 --2- 172.21.15.63:0/3958945951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e301026c0 0x7f2e300fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:53.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.676+0000 7f2e37658700 1 --2- 172.21.15.63:0/3958945951 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e30101d20 0x7f2e301020f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:53.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.676+0000 7f2e37658700 1 -- 172.21.15.63:0/3958945951 >> 172.21.15.63:0/3958945951 conn(0x7f2e300f7870 msgr2=0x7f2e300f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:53.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.676+0000 7f2e37658700 1 -- 172.21.15.63:0/3958945951 shutdown_connections 2024-03-22T07:19:53.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.676+0000 7f2e37658700 1 -- 172.21.15.63:0/3958945951 wait complete. 2024-03-22T07:19:53.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.676+0000 7f2e37658700 1 Processor -- start 2024-03-22T07:19:53.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.676+0000 7f2e37658700 1 -- start start 2024-03-22T07:19:53.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.676+0000 7f2e37658700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e30101d20 0x7f2e301139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:53.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.676+0000 7f2e37658700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e301026c0 0x7f2e30113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:53.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.677+0000 7f2e37658700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e301145c0 con 0x7f2e30101d20 2024-03-22T07:19:53.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.677+0000 7f2e37658700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3010db30 con 0x7f2e301026c0 2024-03-22T07:19:53.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.677+0000 7f2e353f4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e30101d20 0x7f2e301139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:53.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.677+0000 7f2e353f4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e30101d20 0x7f2e301139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41300/0 (socket says 172.21.15.63:41300) 2024-03-22T07:19:53.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.677+0000 7f2e353f4700 1 -- 172.21.15.63:0/270529118 learned_addr learned my addr 172.21.15.63:0/270529118 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:19:53.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.677+0000 7f2e34bf3700 1 --2- 172.21.15.63:0/270529118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e301026c0 0x7f2e30113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:53.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.677+0000 7f2e353f4700 1 -- 172.21.15.63:0/270529118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e301026c0 msgr2=0x7f2e30113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:53.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.677+0000 7f2e353f4700 1 --2- 172.21.15.63:0/270529118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e301026c0 0x7f2e30113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:53.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.677+0000 7f2e353f4700 1 -- 172.21.15.63:0/270529118 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c005040 con 0x7f2e30101d20 2024-03-22T07:19:53.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.677+0000 7f2e353f4700 1 --2- 172.21.15.63:0/270529118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e30101d20 0x7f2e301139a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00b3b0 tx=0x7f2e2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:53.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.677+0000 7f2e25ffb700 1 -- 172.21.15.63:0/270529118 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e2c00e040 con 0x7f2e30101d20 2024-03-22T07:19:53.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.677+0000 7f2e25ffb700 1 -- 172.21.15.63:0/270529118 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e2c00bbc0 con 0x7f2e30101d20 2024-03-22T07:19:53.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.677+0000 7f2e37658700 1 -- 172.21.15.63:0/270529118 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3010ddb0 con 0x7f2e30101d20 2024-03-22T07:19:53.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.677+0000 7f2e25ffb700 1 -- 172.21.15.63:0/270529118 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e2c01c3f0 con 0x7f2e30101d20 2024-03-22T07:19:53.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.678+0000 7f2e37658700 1 -- 172.21.15.63:0/270529118 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3010e370 con 0x7f2e30101d20 2024-03-22T07:19:53.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.678+0000 7f2e37658700 1 -- 172.21.15.63:0/270529118 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e300fd470 con 0x7f2e30101d20 2024-03-22T07:19:53.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.679+0000 7f2e25ffb700 1 -- 172.21.15.63:0/270529118 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f2e2c00bd30 con 0x7f2e30101d20 2024-03-22T07:19:53.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.679+0000 7f2e25ffb700 1 --2- 172.21.15.63:0/270529118 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f2e1c06cc80 0x7f2e1c06f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:53.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.679+0000 7f2e25ffb700 1 -- 172.21.15.63:0/270529118 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f2e2c08d290 con 0x7f2e30101d20 2024-03-22T07:19:53.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.679+0000 7f2e34bf3700 1 --2- 172.21.15.63:0/270529118 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f2e1c06cc80 0x7f2e1c06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:53.680 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.680+0000 7f2e34bf3700 1 --2- 172.21.15.63:0/270529118 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f2e1c06cc80 0x7f2e1c06f130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2e20006fd0 tx=0x7f2e20008040 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:53.684 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.684+0000 7f2e25ffb700 1 -- 172.21.15.63:0/270529118 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f2e2c05b0a0 con 0x7f2e30101d20 2024-03-22T07:19:53.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:53.905+0000 7f2e37658700 1 -- 172.21.15.63:0/270529118 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e30108e90 con 0x7f2e1c06cc80 2024-03-22T07:19:54.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:54 smithi069 ceph-mon[36441]: pgmap v67: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:54.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:54 smithi063 ceph-mon[29200]: pgmap v67: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:55.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:55 smithi069 ceph-mon[36441]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:19:55.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:55 smithi063 ceph-mon[29200]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:19:56.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:56.075+0000 7f2e25ffb700 1 -- 172.21.15.63:0/270529118 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5092 (secure 0 0 0) 0x7f2e30108e90 con 0x7f2e1c06cc80 2024-03-22T07:19:56.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:56.078+0000 7f2e37658700 1 -- 172.21.15.63:0/270529118 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f2e1c06cc80 msgr2=0x7f2e1c06f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:56.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:56.078+0000 7f2e37658700 1 --2- 172.21.15.63:0/270529118 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f2e1c06cc80 0x7f2e1c06f130 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2e20006fd0 tx=0x7f2e20008040 comp rx=0 tx=0).stop 2024-03-22T07:19:56.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:56.078+0000 7f2e37658700 1 -- 172.21.15.63:0/270529118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e30101d20 msgr2=0x7f2e301139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:56.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:56.078+0000 7f2e37658700 1 --2- 172.21.15.63:0/270529118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e30101d20 0x7f2e301139a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c00b3b0 tx=0x7f2e2c009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:56.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:56.078+0000 7f2e37658700 1 -- 172.21.15.63:0/270529118 shutdown_connections 2024-03-22T07:19:56.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:56.078+0000 7f2e37658700 1 --2- 172.21.15.63:0/270529118 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f2e1c06cc80 0x7f2e1c06f130 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:56.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:56.078+0000 7f2e37658700 1 --2- 172.21.15.63:0/270529118 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e30101d20 0x7f2e301139a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:56.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:56.078+0000 7f2e37658700 1 --2- 172.21.15.63:0/270529118 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e301026c0 0x7f2e30113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:56.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:56.078+0000 7f2e37658700 1 -- 172.21.15.63:0/270529118 >> 172.21.15.63:0/270529118 conn(0x7f2e300f7870 msgr2=0x7f2e300f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:56.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:56.078+0000 7f2e37658700 1 -- 172.21.15.63:0/270529118 shutdown_connections 2024-03-22T07:19:56.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:56.078+0000 7f2e37658700 1 -- 172.21.15.63:0/270529118 wait complete. 2024-03-22T07:19:56.088 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:19:56.684 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:56 smithi063 ceph-mon[29200]: pgmap v68: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:56.711 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch ps' 2024-03-22T07:19:56.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:56 smithi069 ceph-mon[36441]: pgmap v68: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:19:56.978 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:19:58.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.622+0000 7f35fda33700 1 -- 172.21.15.63:0/1217982960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35f80fe830 msgr2=0x7f35f8106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:58.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.622+0000 7f35fda33700 1 --2- 172.21.15.63:0/1217982960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35f80fe830 0x7f35f8106f00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f35ec004940 tx=0x7f35ec009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:58.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.622+0000 7f35fda33700 1 -- 172.21.15.63:0/1217982960 shutdown_connections 2024-03-22T07:19:58.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.622+0000 7f35fda33700 1 --2- 172.21.15.63:0/1217982960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35f80fe830 0x7f35f8106f00 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:58.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.622+0000 7f35fda33700 1 --2- 172.21.15.63:0/1217982960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35f80fde90 0x7f35f80fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:58.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.622+0000 7f35fda33700 1 -- 172.21.15.63:0/1217982960 >> 172.21.15.63:0/1217982960 conn(0x7f35f8072bc0 msgr2=0x7f35f8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:58.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.622+0000 7f35fda33700 1 -- 172.21.15.63:0/1217982960 shutdown_connections 2024-03-22T07:19:58.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.623+0000 7f35fda33700 1 -- 172.21.15.63:0/1217982960 wait complete. 2024-03-22T07:19:58.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.623+0000 7f35fda33700 1 Processor -- start 2024-03-22T07:19:58.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.623+0000 7f35fda33700 1 -- start start 2024-03-22T07:19:58.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.624+0000 7f35fda33700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35f80fde90 0x7f35f8115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:58.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.624+0000 7f35fda33700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35f80fe830 0x7f35f81162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:58.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.624+0000 7f35fda33700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f8116930 con 0x7f35f80fe830 2024-03-22T07:19:58.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.624+0000 7f35fda33700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f810fd00 con 0x7f35f80fde90 2024-03-22T07:19:58.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.624+0000 7f35f67fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35f80fe830 0x7f35f81162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:58.625 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.624+0000 7f35f67fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35f80fe830 0x7f35f81162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37390/0 (socket says 172.21.15.63:37390) 2024-03-22T07:19:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.624+0000 7f35f67fc700 1 -- 172.21.15.63:0/1607954450 learned_addr learned my addr 172.21.15.63:0/1607954450 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:19:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.624+0000 7f35f6ffd700 1 --2- 172.21.15.63:0/1607954450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35f80fde90 0x7f35f8115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.624+0000 7f35f67fc700 1 -- 172.21.15.63:0/1607954450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35f80fde90 msgr2=0x7f35f8115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.624+0000 7f35f67fc700 1 --2- 172.21.15.63:0/1607954450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35f80fde90 0x7f35f8115da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.624+0000 7f35f67fc700 1 -- 172.21.15.63:0/1607954450 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ec005040 con 0x7f35f80fe830 2024-03-22T07:19:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.624+0000 7f35f6ffd700 1 --2- 172.21.15.63:0/1607954450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35f80fde90 0x7f35f8115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:19:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.624+0000 7f35f67fc700 1 --2- 172.21.15.63:0/1607954450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35f80fe830 0x7f35f81162e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009360 tx=0x7f35ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.625+0000 7f35df7fe700 1 -- 172.21.15.63:0/1607954450 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35ec00e040 con 0x7f35f80fe830 2024-03-22T07:19:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.625+0000 7f35fda33700 1 -- 172.21.15.63:0/1607954450 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f810ff80 con 0x7f35f80fe830 2024-03-22T07:19:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.625+0000 7f35fda33700 1 -- 172.21.15.63:0/1607954450 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f81104c0 con 0x7f35f80fe830 2024-03-22T07:19:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.625+0000 7f35df7fe700 1 -- 172.21.15.63:0/1607954450 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35ec013420 con 0x7f35f80fe830 2024-03-22T07:19:58.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.625+0000 7f35df7fe700 1 -- 172.21.15.63:0/1607954450 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35ec01c630 con 0x7f35f80fe830 2024-03-22T07:19:58.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.626+0000 7f35fda33700 1 -- 172.21.15.63:0/1607954450 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e40052b0 con 0x7f35f80fe830 2024-03-22T07:19:58.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.626+0000 7f35df7fe700 1 -- 172.21.15.63:0/1607954450 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f35ec00ba90 con 0x7f35f80fe830 2024-03-22T07:19:58.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.627+0000 7f35df7fe700 1 --2- 172.21.15.63:0/1607954450 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f35e006cc80 0x7f35e006f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:19:58.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.627+0000 7f35df7fe700 1 -- 172.21.15.63:0/1607954450 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f35ec08cfa0 con 0x7f35f80fe830 2024-03-22T07:19:58.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.627+0000 7f35f6ffd700 1 --2- 172.21.15.63:0/1607954450 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f35e006cc80 0x7f35e006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:19:58.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.627+0000 7f35f6ffd700 1 --2- 172.21.15.63:0/1607954450 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f35e006cc80 0x7f35e006f130 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009fd0 tx=0x7f35e8009ee0 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:19:58.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.631+0000 7f35df7fe700 1 -- 172.21.15.63:0/1607954450 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f35ec05aed0 con 0x7f35f80fe830 2024-03-22T07:19:58.660 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:19:58 smithi063 ceph-mon[29200]: pgmap v69: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:19:58.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:19:58 smithi069 ceph-mon[36441]: pgmap v69: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:19:58.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.844+0000 7f35fda33700 1 -- 172.21.15.63:0/1607954450 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f35e4000bc0 con 0x7f35e006cc80 2024-03-22T07:19:58.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.855+0000 7f35df7fe700 1 -- 172.21.15.63:0/1607954450 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f35e4000bc0 con 0x7f35e006cc80 2024-03-22T07:19:58.855 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:19:58.855 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (60s) 37s ago 11m 17.7M - 0.25.0 c8568f914cd2 06ef21da90e5 2024-03-22T07:19:58.855 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (12m) 37s ago 12m 6660k - 17.2.7-726-g9c4e0c34 837124da804f d6c6bbeb6904 2024-03-22T07:19:58.855 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (11m) 91s ago 11m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 30992f8aa88c 2024-03-22T07:19:58.855 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (42s) 37s ago 11m 57.4M - 9.4.12 a0ea5e3c46d9 9015fae0d5e9 2024-03-22T07:19:58.855 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (6m) 37s ago 6m 73.8M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:19:58.855 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (6m) 37s ago 6m 124M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:19:58.855 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (6m) 91s ago 6m 62.7M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:19:58.855 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (6m) 91s ago 6m 129M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:19:58.855 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:8443,9283,8765 running (2m) 37s ago 13m 500M - 19.0.0-1667-gdb0330b1 4f325f8440e9 e49468f39991 2024-03-22T07:19:58.855 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283,8765 running (111s) 91s ago 10m 450M - 19.0.0-1667-gdb0330b1 4f325f8440e9 13c3f034ebb2 2024-03-22T07:19:58.856 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (13m) 37s ago 13m 54.8M 2048M 17.2.7-726-g9c4e0c34 837124da804f f92a5bb6caee 2024-03-22T07:19:58.856 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (10m) 91s ago 10m 41.6M 2048M 17.2.7-726-g9c4e0c34 837124da804f 95f6103f7603 2024-03-22T07:19:58.856 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (99s) 37s ago 11m 14.5M - 1.5.0 0da6a335fe13 0c825ee2a196 2024-03-22T07:19:58.856 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (95s) 91s ago 10m 5435k - 1.5.0 0da6a335fe13 12edb2f35498 2024-03-22T07:19:58.856 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (10m) 37s ago 10m 216M 1557M 17.2.7-726-g9c4e0c34 837124da804f 580f22188a10 2024-03-22T07:19:58.856 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (9m) 37s ago 9m 242M 1557M 17.2.7-726-g9c4e0c34 837124da804f fe1ace35c88a 2024-03-22T07:19:58.856 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (9m) 37s ago 9m 198M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4cd995265dcf 2024-03-22T07:19:58.856 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (8m) 91s ago 8m 279M 2581M 17.2.7-726-g9c4e0c34 837124da804f 45c9a759b76d 2024-03-22T07:19:58.856 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (8m) 91s ago 8m 239M 2581M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:19:58.856 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (7m) 91s ago 7m 238M 2581M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:19:58.856 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (77s) 37s ago 11m 49.0M - 2.43.0 a07b618ecd1d 1d89b928f6e7 2024-03-22T07:19:58.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.857+0000 7f35fda33700 1 -- 172.21.15.63:0/1607954450 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f35e006cc80 msgr2=0x7f35e006f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:58.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.857+0000 7f35fda33700 1 --2- 172.21.15.63:0/1607954450 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f35e006cc80 0x7f35e006f130 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009fd0 tx=0x7f35e8009ee0 comp rx=0 tx=0).stop 2024-03-22T07:19:58.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.857+0000 7f35fda33700 1 -- 172.21.15.63:0/1607954450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35f80fe830 msgr2=0x7f35f81162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:19:58.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.857+0000 7f35fda33700 1 --2- 172.21.15.63:0/1607954450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35f80fe830 0x7f35f81162e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009360 tx=0x7f35ec009e30 comp rx=0 tx=0).stop 2024-03-22T07:19:58.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.857+0000 7f35fda33700 1 -- 172.21.15.63:0/1607954450 shutdown_connections 2024-03-22T07:19:58.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.857+0000 7f35fda33700 1 --2- 172.21.15.63:0/1607954450 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f35e006cc80 0x7f35e006f130 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:58.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.857+0000 7f35fda33700 1 --2- 172.21.15.63:0/1607954450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35f80fde90 0x7f35f8115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:58.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.857+0000 7f35fda33700 1 --2- 172.21.15.63:0/1607954450 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f35f80fe830 0x7f35f81162e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:19:58.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.857+0000 7f35fda33700 1 -- 172.21.15.63:0/1607954450 >> 172.21.15.63:0/1607954450 conn(0x7f35f8072bc0 msgr2=0x7f35f80fb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:19:58.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.857+0000 7f35fda33700 1 -- 172.21.15.63:0/1607954450 shutdown_connections 2024-03-22T07:19:58.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:19:58.857+0000 7f35fda33700 1 -- 172.21.15.63:0/1607954450 wait complete. 2024-03-22T07:19:59.526 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-22T07:19:59.527 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-03-22T07:19:59.527 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-03-22T07:19:59.811 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:20:00.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:00 smithi069 ceph-mon[36441]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:00.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:00 smithi069 ceph-mon[36441]: pgmap v70: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:00.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:00 smithi069 ceph-mon[36441]: overall HEALTH_OK 2024-03-22T07:20:00.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:00 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:20:00.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:00 smithi063 ceph-mon[29200]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:00.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:00 smithi063 ceph-mon[29200]: pgmap v70: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:00.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:00 smithi063 ceph-mon[29200]: overall HEALTH_OK 2024-03-22T07:20:00.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:00 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:20:01.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.500+0000 7f8dae5be700 1 -- 172.21.15.63:0/1431292090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8da8102710 msgr2=0x7f8da80fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:01.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.500+0000 7f8dae5be700 1 --2- 172.21.15.63:0/1431292090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8da8102710 0x7f8da80fbd80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c004940 tx=0x7f8d9c009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:01.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.500+0000 7f8dae5be700 1 -- 172.21.15.63:0/1431292090 shutdown_connections 2024-03-22T07:20:01.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.500+0000 7f8dae5be700 1 --2- 172.21.15.63:0/1431292090 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8da8102710 0x7f8da80fbd80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:01.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.500+0000 7f8dae5be700 1 --2- 172.21.15.63:0/1431292090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8da8101d70 0x7f8da8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:01.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.500+0000 7f8dae5be700 1 -- 172.21.15.63:0/1431292090 >> 172.21.15.63:0/1431292090 conn(0x7f8da8072bc0 msgr2=0x7f8da8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:01.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.500+0000 7f8dae5be700 1 -- 172.21.15.63:0/1431292090 shutdown_connections 2024-03-22T07:20:01.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.500+0000 7f8dae5be700 1 -- 172.21.15.63:0/1431292090 wait complete. 2024-03-22T07:20:01.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.500+0000 7f8dae5be700 1 Processor -- start 2024-03-22T07:20:01.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.501+0000 7f8dae5be700 1 -- start start 2024-03-22T07:20:01.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.501+0000 7f8dae5be700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8da8101d70 0x7f8da8113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:01.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.501+0000 7f8dae5be700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8da8102710 0x7f8da81140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:01.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.501+0000 7f8dae5be700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da8114790 con 0x7f8da8101d70 2024-03-22T07:20:01.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.501+0000 7f8dae5be700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da810daf0 con 0x7f8da8102710 2024-03-22T07:20:01.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.501+0000 7f8da7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8da8101d70 0x7f8da8113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:01.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.501+0000 7f8da7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8da8101d70 0x7f8da8113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37404/0 (socket says 172.21.15.63:37404) 2024-03-22T07:20:01.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.501+0000 7f8da7fff700 1 -- 172.21.15.63:0/1042759363 learned_addr learned my addr 172.21.15.63:0/1042759363 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:01.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.501+0000 7f8da77fe700 1 --2- 172.21.15.63:0/1042759363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8da8102710 0x7f8da81140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:01.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.501+0000 7f8da7fff700 1 -- 172.21.15.63:0/1042759363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8da8102710 msgr2=0x7f8da81140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:01.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.501+0000 7f8da7fff700 1 --2- 172.21.15.63:0/1042759363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8da8102710 0x7f8da81140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:01.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.501+0000 7f8da7fff700 1 -- 172.21.15.63:0/1042759363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c005040 con 0x7f8da8101d70 2024-03-22T07:20:01.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.501+0000 7f8da7fff700 1 --2- 172.21.15.63:0/1042759363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8da8101d70 0x7f8da8113b70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800b790 tx=0x7f8d98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:01.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.502+0000 7f8da4ff9700 1 -- 172.21.15.63:0/1042759363 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d9800f6f0 con 0x7f8da8101d70 2024-03-22T07:20:01.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.502+0000 7f8dae5be700 1 -- 172.21.15.63:0/1042759363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da810ddd0 con 0x7f8da8101d70 2024-03-22T07:20:01.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.502+0000 7f8da4ff9700 1 -- 172.21.15.63:0/1042759363 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d9800fd20 con 0x7f8da8101d70 2024-03-22T07:20:01.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.502+0000 7f8da4ff9700 1 -- 172.21.15.63:0/1042759363 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d980103a0 con 0x7f8da8101d70 2024-03-22T07:20:01.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.502+0000 7f8dae5be700 1 -- 172.21.15.63:0/1042759363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da810e360 con 0x7f8da8101d70 2024-03-22T07:20:01.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.503+0000 7f8dae5be700 1 -- 172.21.15.63:0/1042759363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da80fd4c0 con 0x7f8da8101d70 2024-03-22T07:20:01.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.503+0000 7f8da4ff9700 1 -- 172.21.15.63:0/1042759363 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f8d98010500 con 0x7f8da8101d70 2024-03-22T07:20:01.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.504+0000 7f8da4ff9700 1 --2- 172.21.15.63:0/1042759363 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f8d9006cc80 0x7f8d9006f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:01.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.504+0000 7f8da4ff9700 1 -- 172.21.15.63:0/1042759363 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f8d9808bd90 con 0x7f8da8101d70 2024-03-22T07:20:01.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.504+0000 7f8da77fe700 1 --2- 172.21.15.63:0/1042759363 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f8d9006cc80 0x7f8d9006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:01.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.504+0000 7f8da77fe700 1 --2- 172.21.15.63:0/1042759363 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f8d9006cc80 0x7f8d9006f130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c004890 tx=0x7f8d9c009e30 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:01.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.509+0000 7f8da4ff9700 1 -- 172.21.15.63:0/1042759363 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f8d98059c20 con 0x7f8da8101d70 2024-03-22T07:20:01.714 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.715+0000 7f8dae5be700 1 -- 172.21.15.63:0/1042759363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f8da8108f30 con 0x7f8da8101d70 2024-03-22T07:20:01.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.719+0000 7f8da4ff9700 1 -- 172.21.15.63:0/1042759363 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v46)=0 v46) v1 ==== 125+0+0 (secure 0 0 0) 0x7f8d980597b0 con 0x7f8da8101d70 2024-03-22T07:20:01.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.721+0000 7f8dae5be700 1 -- 172.21.15.63:0/1042759363 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f8d9006cc80 msgr2=0x7f8d9006f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:01.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.721+0000 7f8dae5be700 1 --2- 172.21.15.63:0/1042759363 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f8d9006cc80 0x7f8d9006f130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c004890 tx=0x7f8d9c009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:01.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.721+0000 7f8dae5be700 1 -- 172.21.15.63:0/1042759363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8da8101d70 msgr2=0x7f8da8113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:01.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.721+0000 7f8dae5be700 1 --2- 172.21.15.63:0/1042759363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8da8101d70 0x7f8da8113b70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800b790 tx=0x7f8d98009250 comp rx=0 tx=0).stop 2024-03-22T07:20:01.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.721+0000 7f8dae5be700 1 -- 172.21.15.63:0/1042759363 shutdown_connections 2024-03-22T07:20:01.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.721+0000 7f8dae5be700 1 --2- 172.21.15.63:0/1042759363 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f8d9006cc80 0x7f8d9006f130 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:01.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.721+0000 7f8dae5be700 1 --2- 172.21.15.63:0/1042759363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8da8101d70 0x7f8da8113b70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:01.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.721+0000 7f8dae5be700 1 --2- 172.21.15.63:0/1042759363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8da8102710 0x7f8da81140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:01.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.721+0000 7f8dae5be700 1 -- 172.21.15.63:0/1042759363 >> 172.21.15.63:0/1042759363 conn(0x7f8da8072bc0 msgr2=0x7f8da8065c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:01.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.721+0000 7f8dae5be700 1 -- 172.21.15.63:0/1042759363 shutdown_connections 2024-03-22T07:20:01.722 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:01.721+0000 7f8dae5be700 1 -- 172.21.15.63:0/1042759363 wait complete. 2024-03-22T07:20:02.347 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-22T07:20:02.348 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-03-22T07:20:02.348 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-22T07:20:02.618 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:20:02.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:02 smithi063 ceph-mon[29200]: pgmap v71: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:20:02.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:02 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/1042759363' entity='client.admin' 2024-03-22T07:20:02.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:02 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:20:02.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:02 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:20:02.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:02 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:20:02.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:02 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:03.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:02 smithi069 ceph-mon[36441]: pgmap v71: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:20:03.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:02 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/1042759363' entity='client.admin' 2024-03-22T07:20:03.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:02 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:20:03.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:02 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:20:03.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:02 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:20:03.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:02 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:04.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.255+0000 7f1cd8773700 1 -- 172.21.15.63:0/1556412257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cd0102e00 msgr2=0x7f1cd01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:04.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.255+0000 7f1cd8773700 1 --2- 172.21.15.63:0/1556412257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cd0102e00 0x7f1cd01031d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4004940 tx=0x7f1cc4009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:04.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.255+0000 7f1cd8773700 1 -- 172.21.15.63:0/1556412257 shutdown_connections 2024-03-22T07:20:04.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.255+0000 7f1cd8773700 1 --2- 172.21.15.63:0/1556412257 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cd01037a0 0x7f1cd00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:04.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.255+0000 7f1cd8773700 1 --2- 172.21.15.63:0/1556412257 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cd0102e00 0x7f1cd01031d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:04.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.255+0000 7f1cd8773700 1 -- 172.21.15.63:0/1556412257 >> 172.21.15.63:0/1556412257 conn(0x7f1cd00f89b0 msgr2=0x7f1cd00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:04.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.255+0000 7f1cd8773700 1 -- 172.21.15.63:0/1556412257 shutdown_connections 2024-03-22T07:20:04.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.255+0000 7f1cd8773700 1 -- 172.21.15.63:0/1556412257 wait complete. 2024-03-22T07:20:04.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.256+0000 7f1cd8773700 1 Processor -- start 2024-03-22T07:20:04.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.256+0000 7f1cd8773700 1 -- start start 2024-03-22T07:20:04.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.256+0000 7f1cd8773700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cd0102e00 0x7f1cd00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:04.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.256+0000 7f1cd8773700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cd01037a0 0x7f1cd0070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:04.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.256+0000 7f1cd8773700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd0072210 con 0x7f1cd0102e00 2024-03-22T07:20:04.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.256+0000 7f1cd8773700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd0070d40 con 0x7f1cd01037a0 2024-03-22T07:20:04.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.256+0000 7f1cd650f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cd0102e00 0x7f1cd00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:04.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.256+0000 7f1cd650f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cd0102e00 0x7f1cd00702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37430/0 (socket says 172.21.15.63:37430) 2024-03-22T07:20:04.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.256+0000 7f1cd650f700 1 -- 172.21.15.63:0/3547574051 learned_addr learned my addr 172.21.15.63:0/3547574051 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:04.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.256+0000 7f1cd650f700 1 -- 172.21.15.63:0/3547574051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cd01037a0 msgr2=0x7f1cd0070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:04.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.256+0000 7f1cd5d0e700 1 --2- 172.21.15.63:0/3547574051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cd01037a0 0x7f1cd0070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:04.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.256+0000 7f1cd650f700 1 --2- 172.21.15.63:0/3547574051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cd01037a0 0x7f1cd0070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:04.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.256+0000 7f1cd650f700 1 -- 172.21.15.63:0/3547574051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc4005040 con 0x7f1cd0102e00 2024-03-22T07:20:04.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.257+0000 7f1cd650f700 1 --2- 172.21.15.63:0/3547574051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cd0102e00 0x7f1cd00702c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400b8b0 tx=0x7f1cc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:04.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.257+0000 7f1cc2ffd700 1 -- 172.21.15.63:0/3547574051 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cc400e040 con 0x7f1cd0102e00 2024-03-22T07:20:04.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.257+0000 7f1cd8773700 1 -- 172.21.15.63:0/3547574051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd0070fc0 con 0x7f1cd0102e00 2024-03-22T07:20:04.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.257+0000 7f1cc2ffd700 1 -- 172.21.15.63:0/3547574051 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1cc4004290 con 0x7f1cd0102e00 2024-03-22T07:20:04.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.257+0000 7f1cc2ffd700 1 -- 172.21.15.63:0/3547574051 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cc401c3f0 con 0x7f1cd0102e00 2024-03-22T07:20:04.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.257+0000 7f1cd8773700 1 -- 172.21.15.63:0/3547574051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd011a0d0 con 0x7f1cd0102e00 2024-03-22T07:20:04.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.258+0000 7f1cd8773700 1 -- 172.21.15.63:0/3547574051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd0100110 con 0x7f1cd0102e00 2024-03-22T07:20:04.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.258+0000 7f1cc2ffd700 1 -- 172.21.15.63:0/3547574051 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f1cc40137b0 con 0x7f1cd0102e00 2024-03-22T07:20:04.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.259+0000 7f1cc2ffd700 1 --2- 172.21.15.63:0/3547574051 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f1cbc06cc80 0x7f1cbc06f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:04.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.259+0000 7f1cc2ffd700 1 -- 172.21.15.63:0/3547574051 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f1cc408d2c0 con 0x7f1cd0102e00 2024-03-22T07:20:04.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.259+0000 7f1cd5d0e700 1 --2- 172.21.15.63:0/3547574051 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f1cbc06cc80 0x7f1cbc06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:04.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.259+0000 7f1cd5d0e700 1 --2- 172.21.15.63:0/3547574051 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f1cbc06cc80 0x7f1cbc06f130 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00719f0 tx=0x7f1ccc00b040 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:04.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.263+0000 7f1cc2ffd700 1 -- 172.21.15.63:0/3547574051 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f1cc4057620 con 0x7f1cd0102e00 2024-03-22T07:20:04.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.463+0000 7f1cd8773700 1 -- 172.21.15.63:0/3547574051 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f1cd00fe570 con 0x7f1cd0102e00 2024-03-22T07:20:04.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.463+0000 7f1cc2ffd700 1 -- 172.21.15.63:0/3547574051 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v46)=0 v46) v1 ==== 155+0+0 (secure 0 0 0) 0x7f1cc405ace0 con 0x7f1cd0102e00 2024-03-22T07:20:04.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.465+0000 7f1cd8773700 1 -- 172.21.15.63:0/3547574051 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f1cbc06cc80 msgr2=0x7f1cbc06f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:04.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.465+0000 7f1cd8773700 1 --2- 172.21.15.63:0/3547574051 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f1cbc06cc80 0x7f1cbc06f130 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00719f0 tx=0x7f1ccc00b040 comp rx=0 tx=0).stop 2024-03-22T07:20:04.465 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.465+0000 7f1cd8773700 1 -- 172.21.15.63:0/3547574051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cd0102e00 msgr2=0x7f1cd00702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:04.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.465+0000 7f1cd8773700 1 --2- 172.21.15.63:0/3547574051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cd0102e00 0x7f1cd00702c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400b8b0 tx=0x7f1cc4009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:04.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.465+0000 7f1cd8773700 1 -- 172.21.15.63:0/3547574051 shutdown_connections 2024-03-22T07:20:04.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.465+0000 7f1cd8773700 1 --2- 172.21.15.63:0/3547574051 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f1cbc06cc80 0x7f1cbc06f130 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:04.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.465+0000 7f1cd8773700 1 --2- 172.21.15.63:0/3547574051 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1cd0102e00 0x7f1cd00702c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:04.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.465+0000 7f1cd8773700 1 --2- 172.21.15.63:0/3547574051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cd01037a0 0x7f1cd0070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:04.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.465+0000 7f1cd8773700 1 -- 172.21.15.63:0/3547574051 >> 172.21.15.63:0/3547574051 conn(0x7f1cd00f89b0 msgr2=0x7f1cd0107350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:04.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.466+0000 7f1cd8773700 1 -- 172.21.15.63:0/3547574051 shutdown_connections 2024-03-22T07:20:04.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:04.466+0000 7f1cd8773700 1 -- 172.21.15.63:0/3547574051 wait complete. 2024-03-22T07:20:04.822 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:04 smithi063 ceph-mon[29200]: pgmap v72: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:05.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:04 smithi069 ceph-mon[36441]: pgmap v72: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:05.155 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-22T07:20:05.422 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:20:06.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:06 smithi063 ceph-mon[29200]: pgmap v73: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:07.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:06 smithi069 ceph-mon[36441]: pgmap v73: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:07.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.109+0000 7ff74f662700 1 -- 172.21.15.63:0/3219144086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff748102f70 msgr2=0x7ff748103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:07.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.109+0000 7ff74f662700 1 --2- 172.21.15.63:0/3219144086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff748102f70 0x7ff748103340 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff744004940 tx=0x7ff744009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:07.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.110+0000 7ff74f662700 1 -- 172.21.15.63:0/3219144086 shutdown_connections 2024-03-22T07:20:07.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.110+0000 7ff74f662700 1 --2- 172.21.15.63:0/3219144086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7480fb3b0 0x7ff7480fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:07.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.110+0000 7ff74f662700 1 --2- 172.21.15.63:0/3219144086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff748102f70 0x7ff748103340 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:07.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.110+0000 7ff74f662700 1 -- 172.21.15.63:0/3219144086 >> 172.21.15.63:0/3219144086 conn(0x7ff748072310 msgr2=0x7ff748072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:07.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.110+0000 7ff74f662700 1 -- 172.21.15.63:0/3219144086 shutdown_connections 2024-03-22T07:20:07.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.110+0000 7ff74f662700 1 -- 172.21.15.63:0/3219144086 wait complete. 2024-03-22T07:20:07.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.110+0000 7ff74f662700 1 Processor -- start 2024-03-22T07:20:07.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.110+0000 7ff74f662700 1 -- start start 2024-03-22T07:20:07.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.110+0000 7ff74f662700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7480fb3b0 0x7ff748113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:07.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.110+0000 7ff74f662700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff748102f70 0x7ff748113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:07.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.110+0000 7ff74f662700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7481144b0 con 0x7ff748102f70 2024-03-22T07:20:07.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.110+0000 7ff74f662700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74810dab0 con 0x7ff7480fb3b0 2024-03-22T07:20:07.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.111+0000 7ff74cbfd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff748102f70 0x7ff748113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:07.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.111+0000 7ff74cbfd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff748102f70 0x7ff748113e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45978/0 (socket says 172.21.15.63:45978) 2024-03-22T07:20:07.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.111+0000 7ff74cbfd700 1 -- 172.21.15.63:0/1681598053 learned_addr learned my addr 172.21.15.63:0/1681598053 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:07.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.111+0000 7ff74d3fe700 1 --2- 172.21.15.63:0/1681598053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7480fb3b0 0x7ff748113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:07.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.111+0000 7ff74cbfd700 1 -- 172.21.15.63:0/1681598053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7480fb3b0 msgr2=0x7ff748113920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:07.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.111+0000 7ff74cbfd700 1 --2- 172.21.15.63:0/1681598053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7480fb3b0 0x7ff748113920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:07.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.111+0000 7ff74cbfd700 1 -- 172.21.15.63:0/1681598053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff744005040 con 0x7ff748102f70 2024-03-22T07:20:07.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.111+0000 7ff74cbfd700 1 --2- 172.21.15.63:0/1681598053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff748102f70 0x7ff748113e60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff738002a50 tx=0x7ff7380046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:07.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.111+0000 7ff73dffb700 1 -- 172.21.15.63:0/1681598053 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff738007cc0 con 0x7ff748102f70 2024-03-22T07:20:07.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.111+0000 7ff73dffb700 1 -- 172.21.15.63:0/1681598053 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff73800f040 con 0x7ff748102f70 2024-03-22T07:20:07.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.111+0000 7ff74f662700 1 -- 172.21.15.63:0/1681598053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74810dd90 con 0x7ff748102f70 2024-03-22T07:20:07.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.111+0000 7ff73dffb700 1 -- 172.21.15.63:0/1681598053 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff73800da30 con 0x7ff748102f70 2024-03-22T07:20:07.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.112+0000 7ff74f662700 1 -- 172.21.15.63:0/1681598053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74810e350 con 0x7ff748102f70 2024-03-22T07:20:07.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.112+0000 7ff74f662700 1 -- 172.21.15.63:0/1681598053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7480fe6d0 con 0x7ff748102f70 2024-03-22T07:20:07.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.113+0000 7ff73dffb700 1 -- 172.21.15.63:0/1681598053 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7ff73800db90 con 0x7ff748102f70 2024-03-22T07:20:07.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.113+0000 7ff73dffb700 1 --2- 172.21.15.63:0/1681598053 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff73406cc80 0x7ff73406f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:07.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.113+0000 7ff73dffb700 1 -- 172.21.15.63:0/1681598053 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff73808bd40 con 0x7ff748102f70 2024-03-22T07:20:07.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.113+0000 7ff74d3fe700 1 --2- 172.21.15.63:0/1681598053 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff73406cc80 0x7ff73406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:07.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.114+0000 7ff74d3fe700 1 --2- 172.21.15.63:0/1681598053 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff73406cc80 0x7ff73406f130 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff744004f90 tx=0x7ff744009e30 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:07.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.118+0000 7ff73dffb700 1 -- 172.21.15.63:0/1681598053 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7ff738059b50 con 0x7ff748102f70 2024-03-22T07:20:07.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.318+0000 7ff74f662700 1 -- 172.21.15.63:0/1681598053 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7ff74804c710 con 0x7ff748102f70 2024-03-22T07:20:07.318 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.318+0000 7ff73dffb700 1 -- 172.21.15.63:0/1681598053 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v46)=0 v46) v1 ==== 163+0+0 (secure 0 0 0) 0x7ff7380596e0 con 0x7ff748102f70 2024-03-22T07:20:07.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.320+0000 7ff74f662700 1 -- 172.21.15.63:0/1681598053 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff73406cc80 msgr2=0x7ff73406f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:07.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.320+0000 7ff74f662700 1 --2- 172.21.15.63:0/1681598053 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff73406cc80 0x7ff73406f130 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff744004f90 tx=0x7ff744009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:07.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.320+0000 7ff74f662700 1 -- 172.21.15.63:0/1681598053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff748102f70 msgr2=0x7ff748113e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:07.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.320+0000 7ff74f662700 1 --2- 172.21.15.63:0/1681598053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff748102f70 0x7ff748113e60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff738002a50 tx=0x7ff7380046b0 comp rx=0 tx=0).stop 2024-03-22T07:20:07.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.320+0000 7ff74f662700 1 -- 172.21.15.63:0/1681598053 shutdown_connections 2024-03-22T07:20:07.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.320+0000 7ff74f662700 1 --2- 172.21.15.63:0/1681598053 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff73406cc80 0x7ff73406f130 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:07.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.320+0000 7ff74f662700 1 --2- 172.21.15.63:0/1681598053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7480fb3b0 0x7ff748113920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:07.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.320+0000 7ff74f662700 1 --2- 172.21.15.63:0/1681598053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff748102f70 0x7ff748113e60 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:07.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.320+0000 7ff74f662700 1 -- 172.21.15.63:0/1681598053 >> 172.21.15.63:0/1681598053 conn(0x7ff748072310 msgr2=0x7ff7480f9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:07.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.320+0000 7ff74f662700 1 -- 172.21.15.63:0/1681598053 shutdown_connections 2024-03-22T07:20:07.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:07.320+0000 7ff74f662700 1 -- 172.21.15.63:0/1681598053 wait complete. 2024-03-22T07:20:08.035 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-22T07:20:08.298 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:20:08.755 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:08 smithi063 ceph-mon[29200]: pgmap v74: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:20:09.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:08 smithi069 ceph-mon[36441]: pgmap v74: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:20:09.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.990+0000 7f416f831700 1 -- 172.21.15.63:0/1531881653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4168101d70 msgr2=0x7f4168102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:09.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.990+0000 7f416f831700 1 --2- 172.21.15.63:0/1531881653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4168101d70 0x7f4168102140 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4164004940 tx=0x7f4164009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:09.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.990+0000 7f416f831700 1 -- 172.21.15.63:0/1531881653 shutdown_connections 2024-03-22T07:20:09.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.990+0000 7f416f831700 1 --2- 172.21.15.63:0/1531881653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4168102710 0x7f41680fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:09.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.990+0000 7f416f831700 1 --2- 172.21.15.63:0/1531881653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4168101d70 0x7f4168102140 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:09.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.990+0000 7f416f831700 1 -- 172.21.15.63:0/1531881653 >> 172.21.15.63:0/1531881653 conn(0x7f4168072bc0 msgr2=0x7f4168072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:09.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.990+0000 7f416f831700 1 -- 172.21.15.63:0/1531881653 shutdown_connections 2024-03-22T07:20:09.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.990+0000 7f416f831700 1 -- 172.21.15.63:0/1531881653 wait complete. 2024-03-22T07:20:09.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.991+0000 7f416f831700 1 Processor -- start 2024-03-22T07:20:09.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.991+0000 7f416f831700 1 -- start start 2024-03-22T07:20:09.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.991+0000 7f416f831700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4168101d70 0x7f416810c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:09.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.991+0000 7f416f831700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4168102710 0x7f416810ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:09.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.991+0000 7f416f831700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416810d370 con 0x7f4168101d70 2024-03-22T07:20:09.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.991+0000 7f416f831700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416810d4e0 con 0x7f4168102710 2024-03-22T07:20:09.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.991+0000 7f416d5cd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4168101d70 0x7f416810c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.991+0000 7f416d5cd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4168101d70 0x7f416810c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45990/0 (socket says 172.21.15.63:45990) 2024-03-22T07:20:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.991+0000 7f416d5cd700 1 -- 172.21.15.63:0/1249463037 learned_addr learned my addr 172.21.15.63:0/1249463037 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.991+0000 7f416d5cd700 1 -- 172.21.15.63:0/1249463037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4168102710 msgr2=0x7f416810ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.992+0000 7f416cdcc700 1 --2- 172.21.15.63:0/1249463037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4168102710 0x7f416810ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.992+0000 7f416d5cd700 1 --2- 172.21.15.63:0/1249463037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4168102710 0x7f416810ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.992+0000 7f416d5cd700 1 -- 172.21.15.63:0/1249463037 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4164005040 con 0x7f4168101d70 2024-03-22T07:20:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.992+0000 7f416cdcc700 1 --2- 172.21.15.63:0/1249463037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4168102710 0x7f416810ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:20:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.992+0000 7f416d5cd700 1 --2- 172.21.15.63:0/1249463037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4168101d70 0x7f416810c8f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f416400b8b0 tx=0x7f4164009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.992+0000 7f415dffb700 1 -- 172.21.15.63:0/1249463037 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f416400e040 con 0x7f4168101d70 2024-03-22T07:20:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.992+0000 7f415dffb700 1 -- 172.21.15.63:0/1249463037 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4164004290 con 0x7f4168101d70 2024-03-22T07:20:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.992+0000 7f416f831700 1 -- 172.21.15.63:0/1249463037 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416810d760 con 0x7f4168101d70 2024-03-22T07:20:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.992+0000 7f415dffb700 1 -- 172.21.15.63:0/1249463037 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f416401c3f0 con 0x7f4168101d70 2024-03-22T07:20:09.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.992+0000 7f416f831700 1 -- 172.21.15.63:0/1249463037 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416811a2e0 con 0x7f4168101d70 2024-03-22T07:20:09.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.993+0000 7f416f831700 1 -- 172.21.15.63:0/1249463037 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41680ff080 con 0x7f4168101d70 2024-03-22T07:20:09.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.994+0000 7f415dffb700 1 -- 172.21.15.63:0/1249463037 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f41640137b0 con 0x7f4168101d70 2024-03-22T07:20:09.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.994+0000 7f415dffb700 1 --2- 172.21.15.63:0/1249463037 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f415406cc80 0x7f415406f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:09.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.994+0000 7f415dffb700 1 -- 172.21.15.63:0/1249463037 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f416408cfe0 con 0x7f4168101d70 2024-03-22T07:20:09.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.994+0000 7f416cdcc700 1 --2- 172.21.15.63:0/1249463037 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f415406cc80 0x7f415406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:09.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.994+0000 7f416cdcc700 1 --2- 172.21.15.63:0/1249463037 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f415406cc80 0x7f415406f130 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4158007fd0 tx=0x7f4158009040 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:10.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:09.999+0000 7f415dffb700 1 -- 172.21.15.63:0/1249463037 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f416405af10 con 0x7f4168101d70 2024-03-22T07:20:10.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:10.200+0000 7f416f831700 1 -- 172.21.15.63:0/1249463037 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f416810d700 con 0x7f4168101d70 2024-03-22T07:20:10.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:10.200+0000 7f415dffb700 1 -- 172.21.15.63:0/1249463037 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v46)=0 v46) v1 ==== 135+0+0 (secure 0 0 0) 0x7f416405aaa0 con 0x7f4168101d70 2024-03-22T07:20:10.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:10.202+0000 7f416f831700 1 -- 172.21.15.63:0/1249463037 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f415406cc80 msgr2=0x7f415406f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:10.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:10.202+0000 7f416f831700 1 --2- 172.21.15.63:0/1249463037 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f415406cc80 0x7f415406f130 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4158007fd0 tx=0x7f4158009040 comp rx=0 tx=0).stop 2024-03-22T07:20:10.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:10.202+0000 7f416f831700 1 -- 172.21.15.63:0/1249463037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4168101d70 msgr2=0x7f416810c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:10.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:10.202+0000 7f416f831700 1 --2- 172.21.15.63:0/1249463037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4168101d70 0x7f416810c8f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f416400b8b0 tx=0x7f4164009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:10.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:10.202+0000 7f416f831700 1 -- 172.21.15.63:0/1249463037 shutdown_connections 2024-03-22T07:20:10.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:10.202+0000 7f416f831700 1 --2- 172.21.15.63:0/1249463037 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f415406cc80 0x7f415406f130 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:10.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:10.202+0000 7f416f831700 1 --2- 172.21.15.63:0/1249463037 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4168101d70 0x7f416810c8f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:10.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:10.202+0000 7f416f831700 1 --2- 172.21.15.63:0/1249463037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4168102710 0x7f416810ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:10.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:10.202+0000 7f416f831700 1 -- 172.21.15.63:0/1249463037 >> 172.21.15.63:0/1249463037 conn(0x7f4168072bc0 msgr2=0x7f41680fafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:10.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:10.202+0000 7f416f831700 1 -- 172.21.15.63:0/1249463037 shutdown_connections 2024-03-22T07:20:10.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:10.202+0000 7f416f831700 1 -- 172.21.15.63:0/1249463037 wait complete. 2024-03-22T07:20:10.851 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:10 smithi063 ceph-mon[29200]: pgmap v75: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:10.879 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-03-22T07:20:11.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:10 smithi069 ceph-mon[36441]: pgmap v75: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:11.145 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:20:12.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.751+0000 7fd467e5a700 1 -- 172.21.15.63:0/3760665022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4600fb4a0 msgr2=0x7fd4600fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.751+0000 7fd467e5a700 1 --2- 172.21.15.63:0/3760665022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4600fb4a0 0x7fd4600fb910 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd450004940 tx=0x7fd450009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.751+0000 7fd467e5a700 1 -- 172.21.15.63:0/3760665022 shutdown_connections 2024-03-22T07:20:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.751+0000 7fd467e5a700 1 --2- 172.21.15.63:0/3760665022 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd4600fb4a0 0x7fd4600fb910 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.751+0000 7fd467e5a700 1 --2- 172.21.15.63:0/3760665022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd460102e70 0x7fd460103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.751+0000 7fd467e5a700 1 -- 172.21.15.63:0/3760665022 >> 172.21.15.63:0/3760665022 conn(0x7fd4600714c0 msgr2=0x7fd4600718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.751+0000 7fd467e5a700 1 -- 172.21.15.63:0/3760665022 shutdown_connections 2024-03-22T07:20:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.751+0000 7fd467e5a700 1 -- 172.21.15.63:0/3760665022 wait complete. 2024-03-22T07:20:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.751+0000 7fd467e5a700 1 Processor -- start 2024-03-22T07:20:12.754 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.751+0000 7fd467e5a700 1 -- start start 2024-03-22T07:20:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.751+0000 7fd467e5a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4600fb4a0 0x7fd4600fc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.751+0000 7fd467e5a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd460102e70 0x7fd4600fcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.751+0000 7fd467e5a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4600fd150 con 0x7fd460102e70 2024-03-22T07:20:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.751+0000 7fd467e5a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4600fd290 con 0x7fd4600fb4a0 2024-03-22T07:20:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.752+0000 7fd4653f5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd460102e70 0x7fd4600fcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.752+0000 7fd4653f5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd460102e70 0x7fd4600fcbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46010/0 (socket says 172.21.15.63:46010) 2024-03-22T07:20:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.752+0000 7fd4653f5700 1 -- 172.21.15.63:0/3421973178 learned_addr learned my addr 172.21.15.63:0/3421973178 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.752+0000 7fd465bf6700 1 --2- 172.21.15.63:0/3421973178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4600fb4a0 0x7fd4600fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.752+0000 7fd4653f5700 1 -- 172.21.15.63:0/3421973178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4600fb4a0 msgr2=0x7fd4600fc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.752+0000 7fd4653f5700 1 --2- 172.21.15.63:0/3421973178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4600fb4a0 0x7fd4600fc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.752+0000 7fd4653f5700 1 -- 172.21.15.63:0/3421973178 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd450005040 con 0x7fd460102e70 2024-03-22T07:20:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.752+0000 7fd4653f5700 1 --2- 172.21.15.63:0/3421973178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd460102e70 0x7fd4600fcbc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd45000b3b0 tx=0x7fd450009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.752+0000 7fd4567fc700 1 -- 172.21.15.63:0/3421973178 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd45000e040 con 0x7fd460102e70 2024-03-22T07:20:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.752+0000 7fd4567fc700 1 -- 172.21.15.63:0/3421973178 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd450013420 con 0x7fd460102e70 2024-03-22T07:20:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.752+0000 7fd467e5a700 1 -- 172.21.15.63:0/3421973178 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4600fd4e0 con 0x7fd460102e70 2024-03-22T07:20:12.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.752+0000 7fd4567fc700 1 -- 172.21.15.63:0/3421973178 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd45001c630 con 0x7fd460102e70 2024-03-22T07:20:12.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.752+0000 7fd467e5a700 1 -- 172.21.15.63:0/3421973178 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46011a2a0 con 0x7fd460102e70 2024-03-22T07:20:12.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.753+0000 7fd467e5a700 1 -- 172.21.15.63:0/3421973178 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd460100170 con 0x7fd460102e70 2024-03-22T07:20:12.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.754+0000 7fd4567fc700 1 -- 172.21.15.63:0/3421973178 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7fd450013590 con 0x7fd460102e70 2024-03-22T07:20:12.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.754+0000 7fd4567fc700 1 --2- 172.21.15.63:0/3421973178 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fd44c06cc80 0x7fd44c06f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:12.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.754+0000 7fd4567fc700 1 -- 172.21.15.63:0/3421973178 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fd45008d1c0 con 0x7fd460102e70 2024-03-22T07:20:12.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.754+0000 7fd465bf6700 1 --2- 172.21.15.63:0/3421973178 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fd44c06cc80 0x7fd44c06f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:12.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.755+0000 7fd465bf6700 1 --2- 172.21.15.63:0/3421973178 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fd44c06cc80 0x7fd44c06f130 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0041d0 tx=0x7fd45c0043c0 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:12.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.759+0000 7fd4567fc700 1 -- 172.21.15.63:0/3421973178 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fd450057520 con 0x7fd460102e70 2024-03-22T07:20:12.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:12 smithi063 ceph-mon[29200]: pgmap v76: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:20:12.980 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.979+0000 7fd467e5a700 1 -- 172.21.15.63:0/3421973178 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "target": ["mon-mgr", ""]}) v1 -- 0x7fd460109f60 con 0x7fd44c06cc80 2024-03-22T07:20:12.985 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.984+0000 7fd4567fc700 1 -- 172.21.15.63:0/3421973178 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fd460109f60 con 0x7fd44c06cc80 2024-03-22T07:20:12.985 INFO:teuthology.orchestra.run.smithi063.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:20:12.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.986+0000 7fd467e5a700 1 -- 172.21.15.63:0/3421973178 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fd44c06cc80 msgr2=0x7fd44c06f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:12.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.986+0000 7fd467e5a700 1 --2- 172.21.15.63:0/3421973178 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fd44c06cc80 0x7fd44c06f130 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0041d0 tx=0x7fd45c0043c0 comp rx=0 tx=0).stop 2024-03-22T07:20:12.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.986+0000 7fd467e5a700 1 -- 172.21.15.63:0/3421973178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd460102e70 msgr2=0x7fd4600fcbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:12.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.986+0000 7fd467e5a700 1 --2- 172.21.15.63:0/3421973178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd460102e70 0x7fd4600fcbc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd45000b3b0 tx=0x7fd450009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:12.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.986+0000 7fd467e5a700 1 -- 172.21.15.63:0/3421973178 shutdown_connections 2024-03-22T07:20:12.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.986+0000 7fd467e5a700 1 --2- 172.21.15.63:0/3421973178 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fd44c06cc80 0x7fd44c06f130 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:12.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.986+0000 7fd467e5a700 1 --2- 172.21.15.63:0/3421973178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4600fb4a0 0x7fd4600fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:12.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.986+0000 7fd467e5a700 1 --2- 172.21.15.63:0/3421973178 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd460102e70 0x7fd4600fcbc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:12.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.986+0000 7fd467e5a700 1 -- 172.21.15.63:0/3421973178 >> 172.21.15.63:0/3421973178 conn(0x7fd4600714c0 msgr2=0x7fd4600fa890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:12.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.987+0000 7fd467e5a700 1 -- 172.21.15.63:0/3421973178 shutdown_connections 2024-03-22T07:20:12.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:12.987+0000 7fd467e5a700 1 -- 172.21.15.63:0/3421973178 wait complete. 2024-03-22T07:20:13.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:12 smithi069 ceph-mon[36441]: pgmap v76: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:20:13.762 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-22T07:20:13.762 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-03-22T07:20:13.762 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-03-22T07:20:13.985 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:13 smithi063 ceph-mon[29200]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:13.985 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:13 smithi063 ceph-mon[29200]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:20:13.985 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:13 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:13.985 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:13 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:20:13.985 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:13 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:20:13.985 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:13 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:20:13.986 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:13 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:13.986 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:13 smithi063 ceph-mon[29200]: pgmap v77: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:13.986 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:13 smithi063 ceph-mon[29200]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:20:14.048 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:20:14.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:13 smithi069 ceph-mon[36441]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:14.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:13 smithi069 ceph-mon[36441]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:20:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:13 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:13 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:20:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:13 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:20:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:13 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:20:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:13 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:13 smithi069 ceph-mon[36441]: pgmap v77: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:14.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:13 smithi069 ceph-mon[36441]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:db0330b1e4e2470d52b750e251e55a522b4f7d69 2024-03-22T07:20:16.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:16 smithi063 ceph-mon[29200]: pgmap v78: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:16.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:16 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:20:16.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:16 smithi069 ceph-mon[36441]: pgmap v78: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:16.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:16 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:20:16.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.803+0000 7f82801b4700 1 -- 172.21.15.63:0/3179560324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8278102e10 msgr2=0x7f82781031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:16.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.803+0000 7f82801b4700 1 --2- 172.21.15.63:0/3179560324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8278102e10 0x7f82781031e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8268004940 tx=0x7f8268009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:16.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.803+0000 7f82801b4700 1 -- 172.21.15.63:0/3179560324 shutdown_connections 2024-03-22T07:20:16.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.803+0000 7f82801b4700 1 --2- 172.21.15.63:0/3179560324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82781037b0 0x7f82780fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:16.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.803+0000 7f82801b4700 1 --2- 172.21.15.63:0/3179560324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8278102e10 0x7f82781031e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:16.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.803+0000 7f82801b4700 1 -- 172.21.15.63:0/3179560324 >> 172.21.15.63:0/3179560324 conn(0x7f82780f8980 msgr2=0x7f82780fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:16.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.803+0000 7f82801b4700 1 -- 172.21.15.63:0/3179560324 shutdown_connections 2024-03-22T07:20:16.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.803+0000 7f82801b4700 1 -- 172.21.15.63:0/3179560324 wait complete. 2024-03-22T07:20:16.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.804+0000 7f82801b4700 1 Processor -- start 2024-03-22T07:20:16.806 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.804+0000 7f82801b4700 1 -- start start 2024-03-22T07:20:16.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.804+0000 7f82801b4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8278102e10 0x7f8278115dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:16.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.804+0000 7f82801b4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82781037b0 0x7f8278116310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:16.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.804+0000 7f82801b4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8278116960 con 0x7f8278102e10 2024-03-22T07:20:16.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.804+0000 7f82801b4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827810fd30 con 0x7f82781037b0 2024-03-22T07:20:16.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.804+0000 7f827df50700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8278102e10 0x7f8278115dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:16.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.804+0000 7f827df50700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8278102e10 0x7f8278115dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45534/0 (socket says 172.21.15.63:45534) 2024-03-22T07:20:16.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.804+0000 7f827df50700 1 -- 172.21.15.63:0/305763183 learned_addr learned my addr 172.21.15.63:0/305763183 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:16.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.804+0000 7f827d74f700 1 --2- 172.21.15.63:0/305763183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82781037b0 0x7f8278116310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:16.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.804+0000 7f827df50700 1 -- 172.21.15.63:0/305763183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82781037b0 msgr2=0x7f8278116310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:16.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.804+0000 7f827df50700 1 --2- 172.21.15.63:0/305763183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82781037b0 0x7f8278116310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:16.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.804+0000 7f827df50700 1 -- 172.21.15.63:0/305763183 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8268005040 con 0x7f8278102e10 2024-03-22T07:20:16.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.804+0000 7f827df50700 1 --2- 172.21.15.63:0/305763183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8278102e10 0x7f8278115dd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f826800b3b0 tx=0x7f8268009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:16.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.804+0000 7f826e7fc700 1 -- 172.21.15.63:0/305763183 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f826800e040 con 0x7f8278102e10 2024-03-22T07:20:16.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.804+0000 7f82801b4700 1 -- 172.21.15.63:0/305763183 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827810ffb0 con 0x7f8278102e10 2024-03-22T07:20:16.807 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.805+0000 7f826e7fc700 1 -- 172.21.15.63:0/305763183 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f826800bbc0 con 0x7f8278102e10 2024-03-22T07:20:16.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.805+0000 7f826e7fc700 1 -- 172.21.15.63:0/305763183 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f826801c3f0 con 0x7f8278102e10 2024-03-22T07:20:16.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.805+0000 7f82801b4700 1 -- 172.21.15.63:0/305763183 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8278110570 con 0x7f8278102e10 2024-03-22T07:20:16.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.806+0000 7f82801b4700 1 -- 172.21.15.63:0/305763183 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8278107310 con 0x7f8278102e10 2024-03-22T07:20:16.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.806+0000 7f826e7fc700 1 -- 172.21.15.63:0/305763183 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f826800bd30 con 0x7f8278102e10 2024-03-22T07:20:16.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.806+0000 7f826e7fc700 1 --2- 172.21.15.63:0/305763183 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f826406cc80 0x7f826406f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:16.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.807+0000 7f826e7fc700 1 -- 172.21.15.63:0/305763183 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f826808d290 con 0x7f8278102e10 2024-03-22T07:20:16.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.807+0000 7f827d74f700 1 --2- 172.21.15.63:0/305763183 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f826406cc80 0x7f826406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:16.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.807+0000 7f827d74f700 1 --2- 172.21.15.63:0/305763183 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f826406cc80 0x7f826406f130 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8274009fd0 tx=0x7f8274009f20 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:16.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:16.811+0000 7f826e7fc700 1 -- 172.21.15.63:0/305763183 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f826805b0a0 con 0x7f8278102e10 2024-03-22T07:20:17.042 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.041+0000 7f82801b4700 1 -- 172.21.15.63:0/305763183 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8278109f30 con 0x7f826406cc80 2024-03-22T07:20:17.043 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.042+0000 7f826e7fc700 1 -- 172.21.15.63:0/305763183 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f8278109f30 con 0x7f826406cc80 2024-03-22T07:20:17.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.044+0000 7f82801b4700 1 -- 172.21.15.63:0/305763183 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f826406cc80 msgr2=0x7f826406f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:17.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.044+0000 7f82801b4700 1 --2- 172.21.15.63:0/305763183 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f826406cc80 0x7f826406f130 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8274009fd0 tx=0x7f8274009f20 comp rx=0 tx=0).stop 2024-03-22T07:20:17.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.044+0000 7f82801b4700 1 -- 172.21.15.63:0/305763183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8278102e10 msgr2=0x7f8278115dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:17.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.044+0000 7f82801b4700 1 --2- 172.21.15.63:0/305763183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8278102e10 0x7f8278115dd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f826800b3b0 tx=0x7f8268009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:17.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.044+0000 7f82801b4700 1 -- 172.21.15.63:0/305763183 shutdown_connections 2024-03-22T07:20:17.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.044+0000 7f82801b4700 1 --2- 172.21.15.63:0/305763183 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f826406cc80 0x7f826406f130 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:17.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.044+0000 7f82801b4700 1 --2- 172.21.15.63:0/305763183 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8278102e10 0x7f8278115dd0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:17.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.044+0000 7f82801b4700 1 --2- 172.21.15.63:0/305763183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82781037b0 0x7f8278116310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:17.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.044+0000 7f82801b4700 1 -- 172.21.15.63:0/305763183 >> 172.21.15.63:0/305763183 conn(0x7f82780f8980 msgr2=0x7f8278070bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:17.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.044+0000 7f82801b4700 1 -- 172.21.15.63:0/305763183 shutdown_connections 2024-03-22T07:20:17.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.044+0000 7f82801b4700 1 -- 172.21.15.63:0/305763183 wait complete. 2024-03-22T07:20:17.055 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:20:17.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.152+0000 7fa334994700 1 -- 172.21.15.63:0/2237404569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa330102e10 msgr2=0x7fa3301031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:17.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.152+0000 7fa334994700 1 --2- 172.21.15.63:0/2237404569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa330102e10 0x7fa3301031e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa320004940 tx=0x7fa320009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:17.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.152+0000 7fa334994700 1 -- 172.21.15.63:0/2237404569 shutdown_connections 2024-03-22T07:20:17.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.152+0000 7fa334994700 1 --2- 172.21.15.63:0/2237404569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3301037b0 0x7fa3300fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:17.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.152+0000 7fa334994700 1 --2- 172.21.15.63:0/2237404569 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa330102e10 0x7fa3301031e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:17.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.152+0000 7fa334994700 1 -- 172.21.15.63:0/2237404569 >> 172.21.15.63:0/2237404569 conn(0x7fa3300f8980 msgr2=0x7fa3300fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:17.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.152+0000 7fa334994700 1 -- 172.21.15.63:0/2237404569 shutdown_connections 2024-03-22T07:20:17.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.152+0000 7fa334994700 1 -- 172.21.15.63:0/2237404569 wait complete. 2024-03-22T07:20:17.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.152+0000 7fa334994700 1 Processor -- start 2024-03-22T07:20:17.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.152+0000 7fa334994700 1 -- start start 2024-03-22T07:20:17.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.153+0000 7fa334994700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa330102e10 0x7fa3300702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:17.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.153+0000 7fa334994700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3301037b0 0x7fa330070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:17.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.153+0000 7fa334994700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa330070e20 con 0x7fa330102e10 2024-03-22T07:20:17.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.153+0000 7fa334994700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa330070f60 con 0x7fa3301037b0 2024-03-22T07:20:17.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.153+0000 7fa32e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa330102e10 0x7fa3300702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:17.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.153+0000 7fa32e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa330102e10 0x7fa3300702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45556/0 (socket says 172.21.15.63:45556) 2024-03-22T07:20:17.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.153+0000 7fa32e59c700 1 -- 172.21.15.63:0/4140833608 learned_addr learned my addr 172.21.15.63:0/4140833608 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:17.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.153+0000 7fa32dd9b700 1 --2- 172.21.15.63:0/4140833608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3301037b0 0x7fa330070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:17.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.153+0000 7fa32e59c700 1 -- 172.21.15.63:0/4140833608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3301037b0 msgr2=0x7fa330070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:17.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.153+0000 7fa32e59c700 1 --2- 172.21.15.63:0/4140833608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3301037b0 0x7fa330070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:17.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.153+0000 7fa32e59c700 1 -- 172.21.15.63:0/4140833608 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa320005040 con 0x7fa330102e10 2024-03-22T07:20:17.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.153+0000 7fa32e59c700 1 --2- 172.21.15.63:0/4140833608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa330102e10 0x7fa3300702c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa320009150 tx=0x7fa320009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:17.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.153+0000 7fa31effd700 1 -- 172.21.15.63:0/4140833608 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa32000e040 con 0x7fa330102e10 2024-03-22T07:20:17.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.153+0000 7fa31effd700 1 -- 172.21.15.63:0/4140833608 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa320013420 con 0x7fa330102e10 2024-03-22T07:20:17.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.153+0000 7fa334994700 1 -- 172.21.15.63:0/4140833608 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa330119eb0 con 0x7fa330102e10 2024-03-22T07:20:17.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.154+0000 7fa31effd700 1 -- 172.21.15.63:0/4140833608 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa32001c630 con 0x7fa330102e10 2024-03-22T07:20:17.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.154+0000 7fa334994700 1 -- 172.21.15.63:0/4140833608 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33011a390 con 0x7fa330102e10 2024-03-22T07:20:17.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.155+0000 7fa334994700 1 -- 172.21.15.63:0/4140833608 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3300fe560 con 0x7fa330102e10 2024-03-22T07:20:17.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.155+0000 7fa31effd700 1 -- 172.21.15.63:0/4140833608 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7fa320013590 con 0x7fa330102e10 2024-03-22T07:20:17.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.155+0000 7fa31effd700 1 --2- 172.21.15.63:0/4140833608 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa31806cc90 0x7fa31806f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:17.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.156+0000 7fa31effd700 1 -- 172.21.15.63:0/4140833608 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa32008d210 con 0x7fa330102e10 2024-03-22T07:20:17.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.156+0000 7fa32dd9b700 1 --2- 172.21.15.63:0/4140833608 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa31806cc90 0x7fa31806f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:17.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.156+0000 7fa32dd9b700 1 --2- 172.21.15.63:0/4140833608 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa31806cc90 0x7fa31806f140 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa324006fd0 tx=0x7fa324008040 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:17.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.160+0000 7fa31effd700 1 -- 172.21.15.63:0/4140833608 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fa320057570 con 0x7fa330102e10 2024-03-22T07:20:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:17 smithi063 ceph-mon[29200]: fsmap cephfs:2 {0=cephfs.smithi063.xeelpk=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2 up:standby-replay 2024-03-22T07:20:17.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.394+0000 7fa334994700 1 -- 172.21.15.63:0/4140833608 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa33004bf70 con 0x7fa31806cc90 2024-03-22T07:20:17.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.395+0000 7fa31effd700 1 -- 172.21.15.63:0/4140833608 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fa32400d630 con 0x7fa31806cc90 2024-03-22T07:20:17.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.397+0000 7fa334994700 1 -- 172.21.15.63:0/4140833608 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa31806cc90 msgr2=0x7fa31806f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:17.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.397+0000 7fa334994700 1 --2- 172.21.15.63:0/4140833608 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa31806cc90 0x7fa31806f140 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa324006fd0 tx=0x7fa324008040 comp rx=0 tx=0).stop 2024-03-22T07:20:17.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.397+0000 7fa334994700 1 -- 172.21.15.63:0/4140833608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa330102e10 msgr2=0x7fa3300702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:17.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.397+0000 7fa334994700 1 --2- 172.21.15.63:0/4140833608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa330102e10 0x7fa3300702c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa320009150 tx=0x7fa320009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:17.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.397+0000 7fa334994700 1 -- 172.21.15.63:0/4140833608 shutdown_connections 2024-03-22T07:20:17.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.397+0000 7fa334994700 1 --2- 172.21.15.63:0/4140833608 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa31806cc90 0x7fa31806f140 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:17.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.397+0000 7fa334994700 1 --2- 172.21.15.63:0/4140833608 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa330102e10 0x7fa3300702c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:17.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.397+0000 7fa334994700 1 --2- 172.21.15.63:0/4140833608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3301037b0 0x7fa330070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:17.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.397+0000 7fa334994700 1 -- 172.21.15.63:0/4140833608 >> 172.21.15.63:0/4140833608 conn(0x7fa3300f8980 msgr2=0x7fa3300f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:17.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.398+0000 7fa334994700 1 -- 172.21.15.63:0/4140833608 shutdown_connections 2024-03-22T07:20:17.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.398+0000 7fa334994700 1 -- 172.21.15.63:0/4140833608 wait complete. 2024-03-22T07:20:17.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.502+0000 7f71a4a42700 1 -- 172.21.15.63:0/192194911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a01026c0 msgr2=0x7f71a00fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:17.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.502+0000 7f71a4a42700 1 --2- 172.21.15.63:0/192194911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a01026c0 0x7f71a00fbd30 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7194004940 tx=0x7f7194009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:17.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.502+0000 7f71a4a42700 1 -- 172.21.15.63:0/192194911 shutdown_connections 2024-03-22T07:20:17.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.502+0000 7f71a4a42700 1 --2- 172.21.15.63:0/192194911 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a01026c0 0x7f71a00fbd30 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:17.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.502+0000 7f71a4a42700 1 --2- 172.21.15.63:0/192194911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71a0101d20 0x7f71a01020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:17.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.502+0000 7f71a4a42700 1 -- 172.21.15.63:0/192194911 >> 172.21.15.63:0/192194911 conn(0x7f71a00f78b0 msgr2=0x7f71a00f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:17.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.502+0000 7f71a4a42700 1 -- 172.21.15.63:0/192194911 shutdown_connections 2024-03-22T07:20:17.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.503+0000 7f71a4a42700 1 -- 172.21.15.63:0/192194911 wait complete. 2024-03-22T07:20:17.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.503+0000 7f71a4a42700 1 Processor -- start 2024-03-22T07:20:17.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.503+0000 7f71a4a42700 1 -- start start 2024-03-22T07:20:17.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.503+0000 7f71a4a42700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a0101d20 0x7f71a0102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:17.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.503+0000 7f71a4a42700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71a01026c0 0x7f71a01030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:17.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.503+0000 7f71a4a42700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a0103600 con 0x7f71a0101d20 2024-03-22T07:20:17.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.503+0000 7f71a4a42700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a0103770 con 0x7f71a01026c0 2024-03-22T07:20:17.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.503+0000 7f719e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a0101d20 0x7f71a0102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:17.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.503+0000 7f719e59c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a0101d20 0x7f71a0102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45574/0 (socket says 172.21.15.63:45574) 2024-03-22T07:20:17.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.503+0000 7f719e59c700 1 -- 172.21.15.63:0/1921558528 learned_addr learned my addr 172.21.15.63:0/1921558528 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:17.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.504+0000 7f719dd9b700 1 --2- 172.21.15.63:0/1921558528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71a01026c0 0x7f71a01030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:17.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.504+0000 7f719e59c700 1 -- 172.21.15.63:0/1921558528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71a01026c0 msgr2=0x7f71a01030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:17.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.504+0000 7f719e59c700 1 --2- 172.21.15.63:0/1921558528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71a01026c0 0x7f71a01030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:17.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.504+0000 7f719e59c700 1 -- 172.21.15.63:0/1921558528 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7194005040 con 0x7f71a0101d20 2024-03-22T07:20:17.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.504+0000 7f719dd9b700 1 --2- 172.21.15.63:0/1921558528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71a01026c0 0x7f71a01030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:20:17.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.504+0000 7f719e59c700 1 --2- 172.21.15.63:0/1921558528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a0101d20 0x7f71a0102b80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f719000b790 tx=0x7f7190009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:17.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.504+0000 7f718effd700 1 -- 172.21.15.63:0/1921558528 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f719000f770 con 0x7f71a0101d20 2024-03-22T07:20:17.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.504+0000 7f71a4a42700 1 -- 172.21.15.63:0/1921558528 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a0119ea0 con 0x7f71a0101d20 2024-03-22T07:20:17.508 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.504+0000 7f718effd700 1 -- 172.21.15.63:0/1921558528 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f719000fda0 con 0x7f71a0101d20 2024-03-22T07:20:17.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.505+0000 7f718effd700 1 -- 172.21.15.63:0/1921558528 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f719000d310 con 0x7f71a0101d20 2024-03-22T07:20:17.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.505+0000 7f71a4a42700 1 -- 172.21.15.63:0/1921558528 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a011a1f0 con 0x7f71a0101d20 2024-03-22T07:20:17.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.506+0000 7f718effd700 1 -- 172.21.15.63:0/1921558528 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7f719000f8d0 con 0x7f71a0101d20 2024-03-22T07:20:17.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.506+0000 7f71a4a42700 1 -- 172.21.15.63:0/1921558528 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a00fd470 con 0x7f71a0101d20 2024-03-22T07:20:17.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.506+0000 7f718effd700 1 --2- 172.21.15.63:0/1921558528 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f718806ca70 0x7f718806ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:17.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.507+0000 7f718effd700 1 -- 172.21.15.63:0/1921558528 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f719008b830 con 0x7f71a0101d20 2024-03-22T07:20:17.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.507+0000 7f719dd9b700 1 --2- 172.21.15.63:0/1921558528 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f718806ca70 0x7f718806ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:17.509 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.507+0000 7f719dd9b700 1 --2- 172.21.15.63:0/1921558528 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f718806ca70 0x7f718806ef20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7194000c00 tx=0x7f7194009e30 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:17.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.511+0000 7f718effd700 1 -- 172.21.15.63:0/1921558528 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7f7190090020 con 0x7f71a0101d20 2024-03-22T07:20:17.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:17 smithi069 ceph-mon[36441]: fsmap cephfs:2 {0=cephfs.smithi063.xeelpk=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2 up:standby-replay 2024-03-22T07:20:17.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.727+0000 7f71a4a42700 1 -- 172.21.15.63:0/1921558528 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f71a0104060 con 0x7f718806ca70 2024-03-22T07:20:17.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.737+0000 7f718effd700 1 -- 172.21.15.63:0/1921558528 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f71a0104060 con 0x7f718806ca70 2024-03-22T07:20:17.738 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:20:17.738 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (79s) 55s ago 12m 17.7M - 0.25.0 c8568f914cd2 06ef21da90e5 2024-03-22T07:20:17.739 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (12m) 55s ago 12m 6660k - 17.2.7-726-g9c4e0c34 837124da804f d6c6bbeb6904 2024-03-22T07:20:17.739 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (11m) 110s ago 11m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 30992f8aa88c 2024-03-22T07:20:17.739 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (61s) 55s ago 11m 57.4M - 9.4.12 a0ea5e3c46d9 9015fae0d5e9 2024-03-22T07:20:17.739 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (6m) 55s ago 6m 73.8M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:20:17.739 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (6m) 55s ago 6m 124M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:20:17.739 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (6m) 110s ago 6m 62.7M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:20:17.739 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (6m) 110s ago 6m 129M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:20:17.739 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:8443,9283,8765 running (2m) 55s ago 13m 500M - 19.0.0-1667-gdb0330b1 4f325f8440e9 e49468f39991 2024-03-22T07:20:17.739 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283,8765 running (2m) 110s ago 11m 450M - 19.0.0-1667-gdb0330b1 4f325f8440e9 13c3f034ebb2 2024-03-22T07:20:17.739 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (13m) 55s ago 13m 54.8M 2048M 17.2.7-726-g9c4e0c34 837124da804f f92a5bb6caee 2024-03-22T07:20:17.739 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (11m) 110s ago 11m 41.6M 2048M 17.2.7-726-g9c4e0c34 837124da804f 95f6103f7603 2024-03-22T07:20:17.739 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (118s) 55s ago 12m 14.5M - 1.5.0 0da6a335fe13 0c825ee2a196 2024-03-22T07:20:17.739 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (114s) 110s ago 11m 5435k - 1.5.0 0da6a335fe13 12edb2f35498 2024-03-22T07:20:17.739 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (10m) 55s ago 10m 216M 1557M 17.2.7-726-g9c4e0c34 837124da804f 580f22188a10 2024-03-22T07:20:17.740 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (9m) 55s ago 9m 242M 1557M 17.2.7-726-g9c4e0c34 837124da804f fe1ace35c88a 2024-03-22T07:20:17.740 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (9m) 55s ago 9m 198M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4cd995265dcf 2024-03-22T07:20:17.740 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (9m) 110s ago 9m 279M 2581M 17.2.7-726-g9c4e0c34 837124da804f 45c9a759b76d 2024-03-22T07:20:17.740 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (8m) 110s ago 8m 239M 2581M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:20:17.740 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (8m) 110s ago 8m 238M 2581M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:20:17.740 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (96s) 55s ago 11m 49.0M - 2.43.0 a07b618ecd1d 1d89b928f6e7 2024-03-22T07:20:17.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.739+0000 7f71a4a42700 1 -- 172.21.15.63:0/1921558528 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f718806ca70 msgr2=0x7f718806ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:17.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.739+0000 7f71a4a42700 1 --2- 172.21.15.63:0/1921558528 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f718806ca70 0x7f718806ef20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7194000c00 tx=0x7f7194009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:17.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.739+0000 7f71a4a42700 1 -- 172.21.15.63:0/1921558528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a0101d20 msgr2=0x7f71a0102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:17.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.739+0000 7f71a4a42700 1 --2- 172.21.15.63:0/1921558528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a0101d20 0x7f71a0102b80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f719000b790 tx=0x7f7190009250 comp rx=0 tx=0).stop 2024-03-22T07:20:17.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.739+0000 7f71a4a42700 1 -- 172.21.15.63:0/1921558528 shutdown_connections 2024-03-22T07:20:17.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.739+0000 7f71a4a42700 1 --2- 172.21.15.63:0/1921558528 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7f718806ca70 0x7f718806ef20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:17.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.739+0000 7f71a4a42700 1 --2- 172.21.15.63:0/1921558528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f71a0101d20 0x7f71a0102b80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:17.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.739+0000 7f71a4a42700 1 --2- 172.21.15.63:0/1921558528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f71a01026c0 0x7f71a01030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:17.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.739+0000 7f71a4a42700 1 -- 172.21.15.63:0/1921558528 >> 172.21.15.63:0/1921558528 conn(0x7f71a00f78b0 msgr2=0x7f71a0066290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:17.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.739+0000 7f71a4a42700 1 -- 172.21.15.63:0/1921558528 shutdown_connections 2024-03-22T07:20:17.741 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.739+0000 7f71a4a42700 1 -- 172.21.15.63:0/1921558528 wait complete. 2024-03-22T07:20:17.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.864+0000 7fb9c38eb700 1 -- 172.21.15.63:0/4148553701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9bc0fde10 msgr2=0x7fb9bc0fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:17.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.864+0000 7fb9c38eb700 1 --2- 172.21.15.63:0/4148553701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9bc0fde10 0x7fb9bc0fe1e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009ce0 tx=0x7fb9ac004d10 comp rx=0 tx=0).stop 2024-03-22T07:20:17.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.864+0000 7fb9c38eb700 1 -- 172.21.15.63:0/4148553701 shutdown_connections 2024-03-22T07:20:17.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.864+0000 7fb9c38eb700 1 --2- 172.21.15.63:0/4148553701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9bc0fe7b0 0x7fb9bc106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:17.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.864+0000 7fb9c38eb700 1 --2- 172.21.15.63:0/4148553701 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9bc0fde10 0x7fb9bc0fe1e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:17.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.864+0000 7fb9c38eb700 1 -- 172.21.15.63:0/4148553701 >> 172.21.15.63:0/4148553701 conn(0x7fb9bc0f7870 msgr2=0x7fb9bc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:17.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.864+0000 7fb9c38eb700 1 -- 172.21.15.63:0/4148553701 shutdown_connections 2024-03-22T07:20:17.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.864+0000 7fb9c38eb700 1 -- 172.21.15.63:0/4148553701 wait complete. 2024-03-22T07:20:17.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.864+0000 7fb9c38eb700 1 Processor -- start 2024-03-22T07:20:17.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.864+0000 7fb9c38eb700 1 -- start start 2024-03-22T07:20:17.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.865+0000 7fb9c38eb700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9bc0fde10 0x7fb9bc0fbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:17.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.865+0000 7fb9c38eb700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9bc0fe7b0 0x7fb9bc0fc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:17.867 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.865+0000 7fb9c38eb700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc0fdc20 con 0x7fb9bc0fde10 2024-03-22T07:20:17.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.865+0000 7fb9c38eb700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc0fc750 con 0x7fb9bc0fe7b0 2024-03-22T07:20:17.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.865+0000 7fb9c1687700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9bc0fde10 0x7fb9bc0fbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:17.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.865+0000 7fb9c1687700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9bc0fde10 0x7fb9bc0fbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45594/0 (socket says 172.21.15.63:45594) 2024-03-22T07:20:17.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.865+0000 7fb9c1687700 1 -- 172.21.15.63:0/3957082145 learned_addr learned my addr 172.21.15.63:0/3957082145 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:17.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.865+0000 7fb9c0e86700 1 --2- 172.21.15.63:0/3957082145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9bc0fe7b0 0x7fb9bc0fc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:17.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.865+0000 7fb9c1687700 1 -- 172.21.15.63:0/3957082145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9bc0fe7b0 msgr2=0x7fb9bc0fc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:17.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.865+0000 7fb9c1687700 1 --2- 172.21.15.63:0/3957082145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9bc0fe7b0 0x7fb9bc0fc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:17.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.865+0000 7fb9c1687700 1 -- 172.21.15.63:0/3957082145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac009940 con 0x7fb9bc0fde10 2024-03-22T07:20:17.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.865+0000 7fb9c1687700 1 --2- 172.21.15.63:0/3957082145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9bc0fde10 0x7fb9bc0fbcd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac007c00 tx=0x7fb9ac004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:17.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.865+0000 7fb9b9ffb700 1 -- 172.21.15.63:0/3957082145 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9ac00e050 con 0x7fb9bc0fde10 2024-03-22T07:20:17.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.865+0000 7fb9c38eb700 1 -- 172.21.15.63:0/3957082145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9bc0fc9d0 con 0x7fb9bc0fde10 2024-03-22T07:20:17.868 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.865+0000 7fb9b9ffb700 1 -- 172.21.15.63:0/3957082145 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9ac01d910 con 0x7fb9bc0fde10 2024-03-22T07:20:17.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.865+0000 7fb9b9ffb700 1 -- 172.21.15.63:0/3957082145 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9ac0128c0 con 0x7fb9bc0fde10 2024-03-22T07:20:17.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.866+0000 7fb9c38eb700 1 -- 172.21.15.63:0/3957082145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9bc11a130 con 0x7fb9bc0fde10 2024-03-22T07:20:17.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.866+0000 7fb9c38eb700 1 -- 172.21.15.63:0/3957082145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9bc066af0 con 0x7fb9bc0fde10 2024-03-22T07:20:17.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.867+0000 7fb9b9ffb700 1 -- 172.21.15.63:0/3957082145 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7fb9ac019040 con 0x7fb9bc0fde10 2024-03-22T07:20:17.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.867+0000 7fb9b9ffb700 1 --2- 172.21.15.63:0/3957082145 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fb9a406cc80 0x7fb9a406f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:17.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.868+0000 7fb9b9ffb700 1 -- 172.21.15.63:0/3957082145 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb9ac08db00 con 0x7fb9bc0fde10 2024-03-22T07:20:17.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.868+0000 7fb9c0e86700 1 --2- 172.21.15.63:0/3957082145 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fb9a406cc80 0x7fb9a406f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:17.870 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.868+0000 7fb9c0e86700 1 --2- 172.21.15.63:0/3957082145 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fb9a406cc80 0x7fb9a406f130 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0fd400 tx=0x7fb9b000b040 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:17.873 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:17.872+0000 7fb9b9ffb700 1 -- 172.21.15.63:0/3957082145 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fb9ac057de0 con 0x7fb9bc0fde10 2024-03-22T07:20:18.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.165+0000 7fb9c38eb700 1 -- 172.21.15.63:0/3957082145 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb9bc11a320 con 0x7fb9bc0fde10 2024-03-22T07:20:18.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.165+0000 7fb9b9ffb700 1 -- 172.21.15.63:0/3957082145 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fb9ac05b4a0 con 0x7fb9bc0fde10 2024-03-22T07:20:18.167 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:20:18.167 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:20:18.167 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2 2024-03-22T07:20:18.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:20:18.167 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:20:18.167 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:20:18.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:20:18.167 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:20:18.167 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:20:18.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:20:18.167 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:20:18.167 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:20:18.167 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:20:18.168 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:20:18.168 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 12, 2024-03-22T07:20:18.168 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:20:18.168 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:20:18.168 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:20:18.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.168+0000 7fb9c38eb700 1 -- 172.21.15.63:0/3957082145 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fb9a406cc80 msgr2=0x7fb9a406f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:18.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.168+0000 7fb9c38eb700 1 --2- 172.21.15.63:0/3957082145 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fb9a406cc80 0x7fb9a406f130 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc0fd400 tx=0x7fb9b000b040 comp rx=0 tx=0).stop 2024-03-22T07:20:18.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.168+0000 7fb9c38eb700 1 -- 172.21.15.63:0/3957082145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9bc0fde10 msgr2=0x7fb9bc0fbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:18.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.168+0000 7fb9c38eb700 1 --2- 172.21.15.63:0/3957082145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9bc0fde10 0x7fb9bc0fbcd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac007c00 tx=0x7fb9ac004d10 comp rx=0 tx=0).stop 2024-03-22T07:20:18.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.168+0000 7fb9c38eb700 1 -- 172.21.15.63:0/3957082145 shutdown_connections 2024-03-22T07:20:18.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.168+0000 7fb9c38eb700 1 --2- 172.21.15.63:0/3957082145 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fb9a406cc80 0x7fb9a406f130 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.168+0000 7fb9c38eb700 1 --2- 172.21.15.63:0/3957082145 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9bc0fde10 0x7fb9bc0fbcd0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.168+0000 7fb9c38eb700 1 --2- 172.21.15.63:0/3957082145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9bc0fe7b0 0x7fb9bc0fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.168+0000 7fb9c38eb700 1 -- 172.21.15.63:0/3957082145 >> 172.21.15.63:0/3957082145 conn(0x7fb9bc0f7870 msgr2=0x7fb9bc0f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:18.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.168+0000 7fb9c38eb700 1 -- 172.21.15.63:0/3957082145 shutdown_connections 2024-03-22T07:20:18.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.168+0000 7fb9c38eb700 1 -- 172.21.15.63:0/3957082145 wait complete. 2024-03-22T07:20:18.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.272+0000 7ff3adf45700 1 -- 172.21.15.63:0/3235128104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a81037a0 msgr2=0x7ff3a80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:18.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.272+0000 7ff3adf45700 1 --2- 172.21.15.63:0/3235128104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a81037a0 0x7ff3a80fce10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff39c004940 tx=0x7ff39c009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:18.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.272+0000 7ff3adf45700 1 -- 172.21.15.63:0/3235128104 shutdown_connections 2024-03-22T07:20:18.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.272+0000 7ff3adf45700 1 --2- 172.21.15.63:0/3235128104 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a81037a0 0x7ff3a80fce10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.272+0000 7ff3adf45700 1 --2- 172.21.15.63:0/3235128104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3a8102e00 0x7ff3a81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.272+0000 7ff3adf45700 1 -- 172.21.15.63:0/3235128104 >> 172.21.15.63:0/3235128104 conn(0x7ff3a80f89b0 msgr2=0x7ff3a80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:18.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.272+0000 7ff3adf45700 1 -- 172.21.15.63:0/3235128104 shutdown_connections 2024-03-22T07:20:18.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.272+0000 7ff3adf45700 1 -- 172.21.15.63:0/3235128104 wait complete. 2024-03-22T07:20:18.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.273+0000 7ff3adf45700 1 Processor -- start 2024-03-22T07:20:18.276 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.273+0000 7ff3adf45700 1 -- start start 2024-03-22T07:20:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.273+0000 7ff3adf45700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a8102e00 0x7ff3a8113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.273+0000 7ff3adf45700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3a81037a0 0x7ff3a81140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.273+0000 7ff3adf45700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a8114710 con 0x7ff3a8102e00 2024-03-22T07:20:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.273+0000 7ff3adf45700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a810db70 con 0x7ff3a81037a0 2024-03-22T07:20:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.274+0000 7ff3a77fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a8102e00 0x7ff3a8113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.274+0000 7ff3a77fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a8102e00 0x7ff3a8113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45612/0 (socket says 172.21.15.63:45612) 2024-03-22T07:20:18.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.274+0000 7ff3a77fe700 1 -- 172.21.15.63:0/66673205 learned_addr learned my addr 172.21.15.63:0/66673205 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:18.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.274+0000 7ff3a6ffd700 1 --2- 172.21.15.63:0/66673205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3a81037a0 0x7ff3a81140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:18.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.274+0000 7ff3a77fe700 1 -- 172.21.15.63:0/66673205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3a81037a0 msgr2=0x7ff3a81140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:18.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.274+0000 7ff3a77fe700 1 --2- 172.21.15.63:0/66673205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3a81037a0 0x7ff3a81140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.274+0000 7ff3a77fe700 1 -- 172.21.15.63:0/66673205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39c005040 con 0x7ff3a8102e00 2024-03-22T07:20:18.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.274+0000 7ff3a6ffd700 1 --2- 172.21.15.63:0/66673205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3a81037a0 0x7ff3a81140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:20:18.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.274+0000 7ff3a77fe700 1 --2- 172.21.15.63:0/66673205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a8102e00 0x7ff3a8113b80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff39800c8c0 tx=0x7ff3980044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:18.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.274+0000 7ff38ffff700 1 -- 172.21.15.63:0/66673205 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff39800a980 con 0x7ff3a8102e00 2024-03-22T07:20:18.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.274+0000 7ff3adf45700 1 -- 172.21.15.63:0/66673205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a810de50 con 0x7ff3a8102e00 2024-03-22T07:20:18.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.274+0000 7ff38ffff700 1 -- 172.21.15.63:0/66673205 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff39800ce60 con 0x7ff3a8102e00 2024-03-22T07:20:18.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.274+0000 7ff38ffff700 1 -- 172.21.15.63:0/66673205 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3980115e0 con 0x7ff3a8102e00 2024-03-22T07:20:18.279 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.274+0000 7ff3adf45700 1 -- 172.21.15.63:0/66673205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a810e410 con 0x7ff3a8102e00 2024-03-22T07:20:18.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.276+0000 7ff3adf45700 1 -- 172.21.15.63:0/66673205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a81052d0 con 0x7ff3a8102e00 2024-03-22T07:20:18.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.276+0000 7ff38ffff700 1 -- 172.21.15.63:0/66673205 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7ff398011740 con 0x7ff3a8102e00 2024-03-22T07:20:18.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.276+0000 7ff38ffff700 1 --2- 172.21.15.63:0/66673205 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff39006cc80 0x7ff39006f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:18.280 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.276+0000 7ff38ffff700 1 -- 172.21.15.63:0/66673205 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff39808bfa0 con 0x7ff3a8102e00 2024-03-22T07:20:18.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.277+0000 7ff3a6ffd700 1 --2- 172.21.15.63:0/66673205 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff39006cc80 0x7ff39006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:18.281 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.277+0000 7ff3a6ffd700 1 --2- 172.21.15.63:0/66673205 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff39006cc80 0x7ff39006f130 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00b3b0 tx=0x7ff39c009e30 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:18.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.281+0000 7ff38ffff700 1 -- 172.21.15.63:0/66673205 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7ff398056410 con 0x7ff3a8102e00 2024-03-22T07:20:18.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:18 smithi063 ceph-mon[29200]: pgmap v79: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:20:18.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:18 smithi063 ceph-mon[29200]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:18.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:18 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:18.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:18 smithi063 ceph-mon[29200]: Upgrade: Target is version 19.0.0-1667-gdb0330b1 (squid) 2024-03-22T07:20:18.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:18 smithi063 ceph-mon[29200]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10'] 2024-03-22T07:20:18.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:18 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:20:18.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:18 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:20:18.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:18 smithi063 ceph-mon[29200]: Upgrade: Setting container_image for all mgr 2024-03-22T07:20:18.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:18 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:18.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:18 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-22T07:20:18.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:18 smithi063 ceph-mon[29200]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:18.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:18 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/3957082145' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:20:18.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.539+0000 7ff3adf45700 1 -- 172.21.15.63:0/66673205 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff3a8109ee0 con 0x7ff3a8102e00 2024-03-22T07:20:18.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.540+0000 7ff38ffff700 1 -- 172.21.15.63:0/66673205 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1847 (secure 0 0 0) 0x7ff398060020 con 0x7ff3a8102e00 2024-03-22T07:20:18.541 INFO:teuthology.orchestra.run.smithi063.stdout:e23 2024-03-22T07:20:18.541 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:20:18.541 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:20:18.541 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-03-22T07:20:18.541 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:20:18.541 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:20:18.541 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-03-22T07:20:18.541 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 23 2024-03-22T07:20:18.541 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-22T07:20:18.541 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-03-22T07:13:29.602905+0000 2024-03-22T07:20:18.542 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-03-22T07:20:16.089229+0000 2024-03-22T07:20:18.542 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-03-22T07:20:18.542 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-03-22T07:20:18.542 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-03-22T07:20:18.542 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-03-22T07:20:18.542 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-03-22T07:20:18.542 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-03-22T07:20:18.542 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-03-22T07:20:18.542 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-03-22T07:20:18.542 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:20:18.542 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 2 2024-03-22T07:20:18.542 INFO:teuthology.orchestra.run.smithi063.stdout:in 0,1 2024-03-22T07:20:18.542 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=14480,1=24285} 2024-03-22T07:20:18.542 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-03-22T07:20:18.543 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-03-22T07:20:18.543 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-03-22T07:20:18.543 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-03-22T07:20:18.543 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-03-22T07:20:18.543 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-03-22T07:20:18.543 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-03-22T07:20:18.543 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-03-22T07:20:18.543 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.xeelpk{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:20:18.543 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.oytapz{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3211595802,v1:172.21.15.63:6827/3211595802] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:20:18.543 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.wwtsjn{1:24285} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.69:6824/3798247699,v1:172.21.15.69:6825/3798247699] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:20:18.543 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.cijxah{1:24301} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.69:6826/3380719311,v1:172.21.15.69:6827/3380719311] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:20:18.543 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:20:18.543 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:20:18.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.542+0000 7ff3adf45700 1 -- 172.21.15.63:0/66673205 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff39006cc80 msgr2=0x7ff39006f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:18.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.542+0000 7ff3adf45700 1 --2- 172.21.15.63:0/66673205 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff39006cc80 0x7ff39006f130 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff39c00b3b0 tx=0x7ff39c009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:18.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.542+0000 7ff3adf45700 1 -- 172.21.15.63:0/66673205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a8102e00 msgr2=0x7ff3a8113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:18.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.542+0000 7ff3adf45700 1 --2- 172.21.15.63:0/66673205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a8102e00 0x7ff3a8113b80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff39800c8c0 tx=0x7ff3980044c0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.542+0000 7ff3adf45700 1 -- 172.21.15.63:0/66673205 shutdown_connections 2024-03-22T07:20:18.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.542+0000 7ff3adf45700 1 --2- 172.21.15.63:0/66673205 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7ff39006cc80 0x7ff39006f130 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.542+0000 7ff3adf45700 1 --2- 172.21.15.63:0/66673205 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3a8102e00 0x7ff3a8113b80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.542+0000 7ff3adf45700 1 --2- 172.21.15.63:0/66673205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3a81037a0 0x7ff3a81140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.542+0000 7ff3adf45700 1 -- 172.21.15.63:0/66673205 >> 172.21.15.63:0/66673205 conn(0x7ff3a80f89b0 msgr2=0x7ff3a8102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:18.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.543+0000 7ff3adf45700 1 -- 172.21.15.63:0/66673205 shutdown_connections 2024-03-22T07:20:18.545 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.543+0000 7ff3adf45700 1 -- 172.21.15.63:0/66673205 wait complete. 2024-03-22T07:20:18.545 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 23 2024-03-22T07:20:18.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.645+0000 7eff2bae5700 1 -- 172.21.15.63:0/3020855562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff240fb540 msgr2=0x7eff240fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:18.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.645+0000 7eff2bae5700 1 --2- 172.21.15.63:0/3020855562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff240fb540 0x7eff240fb990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7eff14004940 tx=0x7eff14009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:18.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.645+0000 7eff2bae5700 1 -- 172.21.15.63:0/3020855562 shutdown_connections 2024-03-22T07:20:18.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.645+0000 7eff2bae5700 1 --2- 172.21.15.63:0/3020855562 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff240fb540 0x7eff240fb990 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.645+0000 7eff2bae5700 1 --2- 172.21.15.63:0/3020855562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff24102fd0 0x7eff241033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.645+0000 7eff2bae5700 1 -- 172.21.15.63:0/3020855562 >> 172.21.15.63:0/3020855562 conn(0x7eff24071530 msgr2=0x7eff24071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:18.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.646+0000 7eff2bae5700 1 -- 172.21.15.63:0/3020855562 shutdown_connections 2024-03-22T07:20:18.647 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.646+0000 7eff2bae5700 1 -- 172.21.15.63:0/3020855562 wait complete. 2024-03-22T07:20:18.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.646+0000 7eff2bae5700 1 Processor -- start 2024-03-22T07:20:18.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.646+0000 7eff2bae5700 1 -- start start 2024-03-22T07:20:18.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.646+0000 7eff2bae5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff240fb540 0x7eff241138f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:18.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.646+0000 7eff2bae5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff24102fd0 0x7eff24113e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:18.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.646+0000 7eff2bae5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff24114480 con 0x7eff24102fd0 2024-03-22T07:20:18.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.646+0000 7eff2bae5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2410dac0 con 0x7eff240fb540 2024-03-22T07:20:18.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.647+0000 7eff29080700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff24102fd0 0x7eff24113e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:18.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.647+0000 7eff29080700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff24102fd0 0x7eff24113e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45640/0 (socket says 172.21.15.63:45640) 2024-03-22T07:20:18.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.647+0000 7eff29080700 1 -- 172.21.15.63:0/4070744964 learned_addr learned my addr 172.21.15.63:0/4070744964 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:18.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.647+0000 7eff29881700 1 --2- 172.21.15.63:0/4070744964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff240fb540 0x7eff241138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:18.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.647+0000 7eff29080700 1 -- 172.21.15.63:0/4070744964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff240fb540 msgr2=0x7eff241138f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:18.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.647+0000 7eff29080700 1 --2- 172.21.15.63:0/4070744964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff240fb540 0x7eff241138f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.647+0000 7eff29080700 1 -- 172.21.15.63:0/4070744964 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff14005040 con 0x7eff24102fd0 2024-03-22T07:20:18.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.647+0000 7eff29881700 1 --2- 172.21.15.63:0/4070744964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff240fb540 0x7eff241138f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:20:18.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.647+0000 7eff29080700 1 --2- 172.21.15.63:0/4070744964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff24102fd0 0x7eff24113e30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7eff1400b3b0 tx=0x7eff14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:18.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.647+0000 7eff1a7fc700 1 -- 172.21.15.63:0/4070744964 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff1400e040 con 0x7eff24102fd0 2024-03-22T07:20:18.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.647+0000 7eff1a7fc700 1 -- 172.21.15.63:0/4070744964 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff1400bbc0 con 0x7eff24102fd0 2024-03-22T07:20:18.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.647+0000 7eff2bae5700 1 -- 172.21.15.63:0/4070744964 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2410dd40 con 0x7eff24102fd0 2024-03-22T07:20:18.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.647+0000 7eff1a7fc700 1 -- 172.21.15.63:0/4070744964 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff1401c630 con 0x7eff24102fd0 2024-03-22T07:20:18.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.647+0000 7eff2bae5700 1 -- 172.21.15.63:0/4070744964 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2410e300 con 0x7eff24102fd0 2024-03-22T07:20:18.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.648+0000 7eff2bae5700 1 -- 172.21.15.63:0/4070744964 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff241053b0 con 0x7eff24102fd0 2024-03-22T07:20:18.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.649+0000 7eff1a7fc700 1 -- 172.21.15.63:0/4070744964 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7eff1400bd30 con 0x7eff24102fd0 2024-03-22T07:20:18.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.649+0000 7eff1a7fc700 1 --2- 172.21.15.63:0/4070744964 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7eff1006cc80 0x7eff1006f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:18.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.649+0000 7eff1a7fc700 1 -- 172.21.15.63:0/4070744964 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7eff1408d130 con 0x7eff24102fd0 2024-03-22T07:20:18.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.649+0000 7eff29881700 1 --2- 172.21.15.63:0/4070744964 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7eff1006cc80 0x7eff1006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:18.651 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.650+0000 7eff29881700 1 --2- 172.21.15.63:0/4070744964 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7eff1006cc80 0x7eff1006f130 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7eff20009fd0 tx=0x7eff2000b040 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:18.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.654+0000 7eff1a7fc700 1 -- 172.21.15.63:0/4070744964 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7eff1405b060 con 0x7eff24102fd0 2024-03-22T07:20:18.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:18 smithi069 ceph-mon[36441]: pgmap v79: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:20:18.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:18 smithi069 ceph-mon[36441]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:18.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:18 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:18.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:18 smithi069 ceph-mon[36441]: Upgrade: Target is version 19.0.0-1667-gdb0330b1 (squid) 2024-03-22T07:20:18.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:18 smithi069 ceph-mon[36441]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10'] 2024-03-22T07:20:18.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:18 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:20:18.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:18 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:20:18.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:18 smithi069 ceph-mon[36441]: Upgrade: Setting container_image for all mgr 2024-03-22T07:20:18.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:18 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:18.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:18 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-22T07:20:18.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:18 smithi069 ceph-mon[36441]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:18.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:18 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/3957082145' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:20:18.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.881+0000 7eff2bae5700 1 -- 172.21.15.63:0/4070744964 --> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7eff2410efa0 con 0x7eff1006cc80 2024-03-22T07:20:18.883 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.882+0000 7eff1a7fc700 1 -- 172.21.15.63:0/4070744964 <== mgr.14668 v2:172.21.15.63:6800/2390873586 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7eff2410efa0 con 0x7eff1006cc80 2024-03-22T07:20:18.883 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:20:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:20:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:20:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:20:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-03-22T07:20:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-03-22T07:20:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-03-22T07:20:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "2/21 daemons upgraded", 2024-03-22T07:20:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mon daemons", 2024-03-22T07:20:18.883 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:20:18.884 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:20:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.884+0000 7eff2bae5700 1 -- 172.21.15.63:0/4070744964 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7eff1006cc80 msgr2=0x7eff1006f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.884+0000 7eff2bae5700 1 --2- 172.21.15.63:0/4070744964 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7eff1006cc80 0x7eff1006f130 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7eff20009fd0 tx=0x7eff2000b040 comp rx=0 tx=0).stop 2024-03-22T07:20:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.884+0000 7eff2bae5700 1 -- 172.21.15.63:0/4070744964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff24102fd0 msgr2=0x7eff24113e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:18.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.884+0000 7eff2bae5700 1 --2- 172.21.15.63:0/4070744964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff24102fd0 0x7eff24113e30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7eff1400b3b0 tx=0x7eff14009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:18.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.884+0000 7eff2bae5700 1 -- 172.21.15.63:0/4070744964 shutdown_connections 2024-03-22T07:20:18.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.884+0000 7eff2bae5700 1 --2- 172.21.15.63:0/4070744964 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7eff1006cc80 0x7eff1006f130 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.884+0000 7eff2bae5700 1 --2- 172.21.15.63:0/4070744964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff240fb540 0x7eff241138f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.884+0000 7eff2bae5700 1 --2- 172.21.15.63:0/4070744964 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff24102fd0 0x7eff24113e30 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.884+0000 7eff2bae5700 1 -- 172.21.15.63:0/4070744964 >> 172.21.15.63:0/4070744964 conn(0x7eff24071530 msgr2=0x7eff240fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:18.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.884+0000 7eff2bae5700 1 -- 172.21.15.63:0/4070744964 shutdown_connections 2024-03-22T07:20:18.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.884+0000 7eff2bae5700 1 -- 172.21.15.63:0/4070744964 wait complete. 2024-03-22T07:20:18.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.990+0000 7fa41e41b700 1 -- 172.21.15.63:0/4092032149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa418101d00 msgr2=0x7fa4181020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.990+0000 7fa41e41b700 1 --2- 172.21.15.63:0/4092032149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa418101d00 0x7fa4181020d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa408004940 tx=0x7fa408009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.990+0000 7fa41e41b700 1 -- 172.21.15.63:0/4092032149 shutdown_connections 2024-03-22T07:20:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.990+0000 7fa41e41b700 1 --2- 172.21.15.63:0/4092032149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4181026a0 0x7fa4180fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.990+0000 7fa41e41b700 1 --2- 172.21.15.63:0/4092032149 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa418101d00 0x7fa4181020d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.990+0000 7fa41e41b700 1 -- 172.21.15.63:0/4092032149 >> 172.21.15.63:0/4092032149 conn(0x7fa4180f78d0 msgr2=0x7fa4180f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.990+0000 7fa41e41b700 1 -- 172.21.15.63:0/4092032149 shutdown_connections 2024-03-22T07:20:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.990+0000 7fa41e41b700 1 -- 172.21.15.63:0/4092032149 wait complete. 2024-03-22T07:20:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.991+0000 7fa41e41b700 1 Processor -- start 2024-03-22T07:20:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.991+0000 7fa41e41b700 1 -- start start 2024-03-22T07:20:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.991+0000 7fa41e41b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa418101d00 0x7fa41810ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.991+0000 7fa41e41b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4181026a0 0x7fa41810f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.991+0000 7fa41e41b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41810f660 con 0x7fa418101d00 2024-03-22T07:20:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.991+0000 7fa41e41b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41810f7a0 con 0x7fa4181026a0 2024-03-22T07:20:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.992+0000 7fa417fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa418101d00 0x7fa41810ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:18.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.992+0000 7fa417fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa418101d00 0x7fa41810ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:45660/0 (socket says 172.21.15.63:45660) 2024-03-22T07:20:18.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.992+0000 7fa417fff700 1 -- 172.21.15.63:0/2214710988 learned_addr learned my addr 172.21.15.63:0/2214710988 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:18.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.992+0000 7fa4177fe700 1 --2- 172.21.15.63:0/2214710988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4181026a0 0x7fa41810f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:18.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.992+0000 7fa417fff700 1 -- 172.21.15.63:0/2214710988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4181026a0 msgr2=0x7fa41810f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:18.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.992+0000 7fa417fff700 1 --2- 172.21.15.63:0/2214710988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4181026a0 0x7fa41810f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:18.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.992+0000 7fa417fff700 1 -- 172.21.15.63:0/2214710988 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa408005040 con 0x7fa418101d00 2024-03-22T07:20:18.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.992+0000 7fa417fff700 1 --2- 172.21.15.63:0/2214710988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa418101d00 0x7fa41810ead0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa408009b30 tx=0x7fa408009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:18.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.992+0000 7fa414ff9700 1 -- 172.21.15.63:0/2214710988 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa408021070 con 0x7fa418101d00 2024-03-22T07:20:18.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.992+0000 7fa414ff9700 1 -- 172.21.15.63:0/2214710988 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa408013c00 con 0x7fa418101d00 2024-03-22T07:20:18.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.992+0000 7fa41e41b700 1 -- 172.21.15.63:0/2214710988 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa418117b40 con 0x7fa418101d00 2024-03-22T07:20:18.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.992+0000 7fa414ff9700 1 -- 172.21.15.63:0/2214710988 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa40800e070 con 0x7fa418101d00 2024-03-22T07:20:18.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.992+0000 7fa41e41b700 1 -- 172.21.15.63:0/2214710988 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa418117fa0 con 0x7fa418101d00 2024-03-22T07:20:18.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.994+0000 7fa41e41b700 1 -- 172.21.15.63:0/2214710988 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4180fd450 con 0x7fa418101d00 2024-03-22T07:20:18.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.994+0000 7fa414ff9700 1 -- 172.21.15.63:0/2214710988 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 33) v1 ==== 90881+0+0 (secure 0 0 0) 0x7fa40801cad0 con 0x7fa418101d00 2024-03-22T07:20:18.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.995+0000 7fa414ff9700 1 --2- 172.21.15.63:0/2214710988 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa40006cc80 0x7fa40006f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:18.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.995+0000 7fa414ff9700 1 -- 172.21.15.63:0/2214710988 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa40808d3e0 con 0x7fa418101d00 2024-03-22T07:20:18.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.995+0000 7fa4177fe700 1 --2- 172.21.15.63:0/2214710988 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa40006cc80 0x7fa40006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:18.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:18.996+0000 7fa4177fe700 1 --2- 172.21.15.63:0/2214710988 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa40006cc80 0x7fa40006f130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa418066890 tx=0x7fa40c00b040 comp rx=0 tx=0).ready entity=mgr.14668 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:19.001 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:19.000+0000 7fa414ff9700 1 -- 172.21.15.63:0/2214710988 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181772 (secure 0 0 0) 0x7fa408057740 con 0x7fa418101d00 2024-03-22T07:20:19.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:19.290+0000 7fa41e41b700 1 -- 172.21.15.63:0/2214710988 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa41804bed0 con 0x7fa418101d00 2024-03-22T07:20:19.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:19.290+0000 7fa414ff9700 1 -- 172.21.15.63:0/2214710988 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa408004490 con 0x7fa418101d00 2024-03-22T07:20:19.292 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-03-22T07:20:19.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:19.293+0000 7fa41e41b700 1 -- 172.21.15.63:0/2214710988 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa40006cc80 msgr2=0x7fa40006f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:19.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:19.293+0000 7fa41e41b700 1 --2- 172.21.15.63:0/2214710988 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa40006cc80 0x7fa40006f130 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa418066890 tx=0x7fa40c00b040 comp rx=0 tx=0).stop 2024-03-22T07:20:19.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:19.293+0000 7fa41e41b700 1 -- 172.21.15.63:0/2214710988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa418101d00 msgr2=0x7fa41810ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:19.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:19.293+0000 7fa41e41b700 1 --2- 172.21.15.63:0/2214710988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa418101d00 0x7fa41810ead0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa408009b30 tx=0x7fa408009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:19.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:19.293+0000 7fa41e41b700 1 -- 172.21.15.63:0/2214710988 shutdown_connections 2024-03-22T07:20:19.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:19.293+0000 7fa41e41b700 1 --2- 172.21.15.63:0/2214710988 >> [v2:172.21.15.63:6800/2390873586,v1:172.21.15.63:6801/2390873586] conn(0x7fa40006cc80 0x7fa40006f130 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:19.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:19.293+0000 7fa41e41b700 1 --2- 172.21.15.63:0/2214710988 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa418101d00 0x7fa41810ead0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:19.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:19.293+0000 7fa41e41b700 1 --2- 172.21.15.63:0/2214710988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa4181026a0 0x7fa41810f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:19.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:19.293+0000 7fa41e41b700 1 -- 172.21.15.63:0/2214710988 >> 172.21.15.63:0/2214710988 conn(0x7fa4180f78d0 msgr2=0x7fa4180f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:19.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:19.293+0000 7fa41e41b700 1 -- 172.21.15.63:0/2214710988 shutdown_connections 2024-03-22T07:20:19.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:19.293+0000 7fa41e41b700 1 -- 172.21.15.63:0/2214710988 wait complete. 2024-03-22T07:20:19.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:19 smithi063 ceph-mon[29200]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:19.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:19 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/66673205' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:20:19.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:19 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:19.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:19 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:20:19.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:19 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:20:19.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:19 smithi063 ceph-mon[29200]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:20:19.607 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:19 smithi063 ceph-mon[29200]: from='client.? 172.21.15.63:0/2214710988' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:20:19.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:19 smithi069 ceph-mon[36441]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:19.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:19 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/66673205' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:20:19.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:19 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:19.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:19 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:20:19.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:19 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:20:19.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:19 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:20:19.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:19 smithi069 ceph-mon[36441]: from='client.? 172.21.15.63:0/2214710988' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:20:20.556 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:20 smithi063 ceph-mon[29200]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:20.556 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:20 smithi063 ceph-mon[29200]: Upgrade: Updating mon.smithi063 2024-03-22T07:20:20.556 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:20 smithi063 ceph-mon[29200]: Deploying daemon mon.smithi063 on smithi063 2024-03-22T07:20:20.556 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:20 smithi063 ceph-mon[29200]: pgmap v80: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:20.681 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:20 smithi069 ceph-mon[36441]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:20.681 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:20 smithi069 ceph-mon[36441]: Upgrade: Updating mon.smithi063 2024-03-22T07:20:20.681 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:20 smithi069 ceph-mon[36441]: Deploying daemon mon.smithi063 on smithi063 2024-03-22T07:20:20.681 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:20 smithi069 ceph-mon[36441]: pgmap v80: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:21.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:20 smithi063 systemd[1]: Stopping Ceph mon.smithi063 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:20:21.454 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:21 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063[29196]: 2024-03-22T07:20:21.134+0000 7fa8afce6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi063 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:20:21.454 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:21 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063[29196]: 2024-03-22T07:20:21.135+0000 7fa8afce6700 -1 mon.smithi063@0(leader) e2 *** Got Signal Terminated *** 2024-03-22T07:20:21.454 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:21 smithi063 podman[112608]: 2024-03-22 07:20:21.17708549 +0000 UTC m=+0.183401739 container died f92a5bb6caeef0bdbdc34d9d2b881f196c82b11c21e6f81e8f225dea9312f40b (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=quincy-9c4e0c3, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, CEPH_POINT_RELEASE=) 2024-03-22T07:20:21.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:21 smithi063 podman[112608]: 2024-03-22 07:20:21.553057824 +0000 UTC m=+0.559374076 container cleanup f92a5bb6caeef0bdbdc34d9d2b881f196c82b11c21e6f81e8f225dea9312f40b (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=quincy-9c4e0c3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-22T07:20:21.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:21 smithi063 bash[112608]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063 2024-03-22T07:20:22.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:21 smithi063 podman[112633]: 2024-03-22 07:20:21.812847717 +0000 UTC m=+0.634257074 container remove f92a5bb6caeef0bdbdc34d9d2b881f196c82b11c21e6f81e8f225dea9312f40b (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063, RELEASE=quincy-9c4e0c3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-22T07:20:22.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:21 smithi063 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mon.smithi063.service: Deactivated successfully. 2024-03-22T07:20:22.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:21 smithi063 systemd[1]: Stopped Ceph mon.smithi063 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:20:22.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:21 smithi063 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mon.smithi063.service: Consumed 12.694s CPU time. 2024-03-22T07:20:22.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 systemd[1]: Starting Ceph mon.smithi063 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:20:22.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 podman[112765]: 2024-03-22 07:20:22.419046815 +0000 UTC m=+0.019237786 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:20:22.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 podman[112765]: 2024-03-22 07:20:22.530509648 +0000 UTC m=+0.130700600 container create ab96538a07c1161a342a4f70fabad331e02fec061f547ca5d6f74b69114212b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=) 2024-03-22T07:20:23.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 podman[112765]: 2024-03-22 07:20:22.76657498 +0000 UTC m=+0.366765944 container init ab96538a07c1161a342a4f70fabad331e02fec061f547ca5d6f74b69114212b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-db0330b) 2024-03-22T07:20:23.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 podman[112765]: 2024-03-22 07:20:22.769633943 +0000 UTC m=+0.369824895 container start ab96538a07c1161a342a4f70fabad331e02fec061f547ca5d6f74b69114212b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-22T07:20:23.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: set uid:gid to 167:167 (ceph:ceph) 2024-03-22T07:20:23.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev), process ceph-mon, pid 2 2024-03-22T07:20:23.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: pidfile_write: ignore empty --pid-file 2024-03-22T07:20:23.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: load: jerasure load: lrc 2024-03-22T07:20:23.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: RocksDB version: 7.9.2 2024-03-22T07:20:23.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Git sha 0 2024-03-22T07:20:23.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Compile date 2024-03-16 10:36:11 2024-03-22T07:20:23.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: DB SUMMARY 2024-03-22T07:20:23.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: DB Session ID: JUQX1RYOF44NQPNRBWU0 2024-03-22T07:20:23.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: CURRENT file: CURRENT 2024-03-22T07:20:23.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: IDENTITY file: IDENTITY 2024-03-22T07:20:23.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1172 Bytes 2024-03-22T07:20:23.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi063/store.db dir, Total Num: 1, files: 000033.sst 2024-03-22T07:20:23.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi063/store.db: 000031.log size: 1640982 ; 2024-03-22T07:20:23.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.error_if_exists: 0 2024-03-22T07:20:23.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.create_if_missing: 0 2024-03-22T07:20:23.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.paranoid_checks: 1 2024-03-22T07:20:23.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-22T07:20:23.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-22T07:20:23.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-22T07:20:23.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.env: 0x56387a7c0ae0 2024-03-22T07:20:23.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.fs: PosixFileSystem 2024-03-22T07:20:23.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.info_log: 0x56387c067760 2024-03-22T07:20:23.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_file_opening_threads: 16 2024-03-22T07:20:23.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.statistics: (nil) 2024-03-22T07:20:23.125 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.use_fsync: 0 2024-03-22T07:20:23.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_log_file_size: 0 2024-03-22T07:20:23.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-22T07:20:23.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-22T07:20:23.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.keep_log_file_num: 1000 2024-03-22T07:20:23.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.recycle_log_file_num: 0 2024-03-22T07:20:23.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.allow_fallocate: 1 2024-03-22T07:20:23.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.allow_mmap_reads: 0 2024-03-22T07:20:23.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.allow_mmap_writes: 0 2024-03-22T07:20:23.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.use_direct_reads: 0 2024-03-22T07:20:23.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-22T07:20:23.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.create_missing_column_families: 0 2024-03-22T07:20:23.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.db_log_dir: 2024-03-22T07:20:23.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.wal_dir: 2024-03-22T07:20:23.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-22T07:20:23.126 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-22T07:20:23.127 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-22T07:20:23.127 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-22T07:20:23.127 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-22T07:20:23.127 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-22T07:20:23.127 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.advise_random_on_open: 1 2024-03-22T07:20:23.127 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.db_write_buffer_size: 0 2024-03-22T07:20:23.127 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.write_buffer_manager: 0x56387c03b400 2024-03-22T07:20:23.127 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-22T07:20:23.127 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-22T07:20:23.127 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-22T07:20:23.127 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.rate_limiter: (nil) 2024-03-22T07:20:23.127 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-22T07:20:23.127 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.wal_recovery_mode: 2 2024-03-22T07:20:23.127 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.enable_thread_tracking: 0 2024-03-22T07:20:23.127 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.enable_pipelined_write: 0 2024-03-22T07:20:23.127 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.unordered_write: 0 2024-03-22T07:20:23.127 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-22T07:20:23.128 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-22T07:20:23.128 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-22T07:20:23.128 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-22T07:20:23.128 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.row_cache: None 2024-03-22T07:20:23.128 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.wal_filter: None 2024-03-22T07:20:23.129 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-22T07:20:23.129 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.allow_ingest_behind: 0 2024-03-22T07:20:23.129 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.two_write_queues: 0 2024-03-22T07:20:23.129 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.manual_wal_flush: 0 2024-03-22T07:20:23.129 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.wal_compression: 0 2024-03-22T07:20:23.129 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.atomic_flush: 0 2024-03-22T07:20:23.130 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-22T07:20:23.130 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-22T07:20:23.130 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-22T07:20:23.130 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.log_readahead_size: 0 2024-03-22T07:20:23.130 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-22T07:20:23.130 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.best_efforts_recovery: 0 2024-03-22T07:20:23.130 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-22T07:20:23.130 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-22T07:20:23.130 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.allow_data_in_errors: 0 2024-03-22T07:20:23.130 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.db_host_id: __hostname__ 2024-03-22T07:20:23.130 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-22T07:20:23.130 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_background_jobs: 2 2024-03-22T07:20:23.130 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_background_compactions: -1 2024-03-22T07:20:23.130 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_subcompactions: 1 2024-03-22T07:20:23.130 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-22T07:20:23.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-22T07:20:23.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-22T07:20:23.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_total_wal_size: 0 2024-03-22T07:20:23.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-22T07:20:23.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-22T07:20:23.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-22T07:20:23.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-22T07:20:23.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_open_files: -1 2024-03-22T07:20:23.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.bytes_per_sync: 0 2024-03-22T07:20:23.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-22T07:20:23.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-22T07:20:23.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compaction_readahead_size: 0 2024-03-22T07:20:23.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_background_flushes: -1 2024-03-22T07:20:23.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Compression algorithms supported: 2024-03-22T07:20:23.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-22T07:20:23.131 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: kZSTD supported: 0 2024-03-22T07:20:23.132 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: kXpressCompression supported: 0 2024-03-22T07:20:23.132 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: kLZ4HCCompression supported: 1 2024-03-22T07:20:23.132 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: kZlibCompression supported: 1 2024-03-22T07:20:23.132 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: kSnappyCompression supported: 1 2024-03-22T07:20:23.132 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: kLZ4Compression supported: 1 2024-03-22T07:20:23.132 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: kBZip2Compression supported: 0 2024-03-22T07:20:23.132 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-22T07:20:23.132 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-22T07:20:23.132 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi063/store.db/MANIFEST-000015 2024-03-22T07:20:23.132 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-22T07:20:23.132 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-22T07:20:23.132 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.merge_operator: 2024-03-22T07:20:23.132 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compaction_filter: None 2024-03-22T07:20:23.132 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compaction_filter_factory: None 2024-03-22T07:20:23.132 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.sst_partitioner_factory: None 2024-03-22T07:20:23.133 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-22T07:20:23.133 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-22T07:20:23.133 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56387c067880) 2024-03-22T07:20:23.133 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: cache_index_and_filter_blocks: 1 2024-03-22T07:20:23.133 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-22T07:20:23.133 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-22T07:20:23.133 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: pin_top_level_index_and_filter: 1 2024-03-22T07:20:23.133 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: index_type: 0 2024-03-22T07:20:23.133 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: data_block_index_type: 0 2024-03-22T07:20:23.133 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: index_shortening: 1 2024-03-22T07:20:23.133 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-22T07:20:23.133 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: checksum: 4 2024-03-22T07:20:23.133 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: no_block_cache: 0 2024-03-22T07:20:23.133 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache: 0x56387c06b090 2024-03-22T07:20:23.133 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache_name: BinnedLRUCache 2024-03-22T07:20:23.133 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache_options: 2024-03-22T07:20:23.133 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: capacity : 536870912 2024-03-22T07:20:23.134 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: num_shard_bits : 4 2024-03-22T07:20:23.134 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: strict_capacity_limit : 0 2024-03-22T07:20:23.134 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: high_pri_pool_ratio: 0.000 2024-03-22T07:20:23.134 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_cache_compressed: (nil) 2024-03-22T07:20:23.134 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: persistent_cache: (nil) 2024-03-22T07:20:23.134 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_size: 4096 2024-03-22T07:20:23.134 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_size_deviation: 10 2024-03-22T07:20:23.134 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_restart_interval: 16 2024-03-22T07:20:23.134 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: index_block_restart_interval: 1 2024-03-22T07:20:23.134 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: metadata_block_size: 4096 2024-03-22T07:20:23.134 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: partition_filters: 0 2024-03-22T07:20:23.134 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: use_delta_encoding: 1 2024-03-22T07:20:23.134 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: filter_policy: bloomfilter 2024-03-22T07:20:23.134 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: whole_key_filtering: 1 2024-03-22T07:20:23.134 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: verify_compression: 0 2024-03-22T07:20:23.135 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: read_amp_bytes_per_bit: 0 2024-03-22T07:20:23.135 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: format_version: 5 2024-03-22T07:20:23.135 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: enable_index_compression: 1 2024-03-22T07:20:23.135 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: block_align: 0 2024-03-22T07:20:23.135 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: max_auto_readahead_size: 262144 2024-03-22T07:20:23.135 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: prepopulate_block_cache: 0 2024-03-22T07:20:23.135 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: initial_auto_readahead_size: 8192 2024-03-22T07:20:23.135 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: num_file_reads_for_auto_readahead: 2 2024-03-22T07:20:23.135 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.write_buffer_size: 33554432 2024-03-22T07:20:23.135 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_write_buffer_number: 2 2024-03-22T07:20:23.135 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compression: NoCompression 2024-03-22T07:20:23.135 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.bottommost_compression: Disabled 2024-03-22T07:20:23.135 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.prefix_extractor: nullptr 2024-03-22T07:20:23.135 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-22T07:20:23.135 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.num_levels: 7 2024-03-22T07:20:23.136 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-22T07:20:23.136 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-22T07:20:23.136 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-22T07:20:23.136 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-22T07:20:23.136 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-22T07:20:23.136 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-22T07:20:23.136 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-22T07:20:23.136 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-22T07:20:23.136 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-22T07:20:23.136 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-22T07:20:23.136 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-22T07:20:23.136 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-22T07:20:23.136 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-22T07:20:23.136 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compression_opts.level: 32767 2024-03-22T07:20:23.137 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compression_opts.strategy: 0 2024-03-22T07:20:23.137 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-22T07:20:23.137 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-22T07:20:23.137 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-22T07:20:23.137 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-22T07:20:23.137 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compression_opts.enabled: false 2024-03-22T07:20:23.137 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-22T07:20:23.137 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-22T07:20:23.137 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-22T07:20:23.137 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-22T07:20:23.137 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.target_file_size_base: 67108864 2024-03-22T07:20:23.137 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-22T07:20:23.137 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-22T07:20:23.137 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-22T07:20:23.137 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-22T07:20:23.137 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-22T07:20:23.138 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-22T07:20:23.138 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-22T07:20:23.138 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-22T07:20:23.138 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-22T07:20:23.138 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-22T07:20:23.138 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-22T07:20:23.138 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-22T07:20:23.138 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-22T07:20:23.138 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-22T07:20:23.138 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.arena_block_size: 1048576 2024-03-22T07:20:23.138 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-22T07:20:23.138 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-22T07:20:23.138 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.disable_auto_compactions: 0 2024-03-22T07:20:23.138 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-22T07:20:23.138 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-22T07:20:23.138 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-22T07:20:23.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-22T07:20:23.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-22T07:20:23.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-22T07:20:23.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-22T07:20:23.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-22T07:20:23.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-22T07:20:23.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-22T07:20:23.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.table_properties_collectors: 2024-03-22T07:20:23.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.inplace_update_support: 0 2024-03-22T07:20:23.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-22T07:20:23.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-22T07:20:23.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-22T07:20:23.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-22T07:20:23.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.bloom_locality: 0 2024-03-22T07:20:23.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.max_successive_merges: 0 2024-03-22T07:20:23.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-22T07:20:23.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.paranoid_file_checks: 0 2024-03-22T07:20:23.139 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.force_consistency_checks: 1 2024-03-22T07:20:23.140 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.report_bg_io_stats: 0 2024-03-22T07:20:23.140 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.ttl: 2592000 2024-03-22T07:20:23.140 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-22T07:20:23.140 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-22T07:20:23.140 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-22T07:20:23.140 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.enable_blob_files: false 2024-03-22T07:20:23.140 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.min_blob_size: 0 2024-03-22T07:20:23.140 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.blob_file_size: 268435456 2024-03-22T07:20:23.140 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-22T07:20:23.140 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-22T07:20:23.140 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-22T07:20:23.140 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-22T07:20:23.140 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-22T07:20:23.140 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.blob_file_starting_level: 0 2024-03-22T07:20:23.140 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-22T07:20:23.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 33.sst 2024-03-22T07:20:23.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-22T07:20:23.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi063/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 35, last_sequence is 12004, log_number is 31,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-22T07:20:23.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 31 2024-03-22T07:20:23.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 02b32545-ecc8-447d-b56f-1f610bd5dcf1 2024-03-22T07:20:23.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711092022809859, "job": 1, "event": "recovery_started", "wal_files": [31]} 2024-03-22T07:20:23.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2024-03-22T07:20:23.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711092022815481, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 1476266, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12005, "largest_seqno": 13357, "table_properties": {"data_size": 1469931, "index_size": 3720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1605, "raw_key_size": 14759, "raw_average_key_size": 23, "raw_value_size": 1456904, "raw_average_value_size": 2361, "num_data_blocks": 169, "num_entries": 617, "num_filter_entries": 617, "num_deletions": 7, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711092022, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "02b32545-ecc8-447d-b56f-1f610bd5dcf1", "db_session_id": "JUQX1RYOF44NQPNRBWU0", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2024-03-22T07:20:23.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711092022815577, "job": 1, "event": "recovery_finished"} 2024-03-22T07:20:23.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: [db/version_set.cc:5047] Creating manifest 38 2024-03-22T07:20:23.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-22T07:20:23.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi063/store.db/000031.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-22T07:20:23.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56387c15a000 2024-03-22T07:20:23.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: DB pointer 0x56387c146000 2024-03-22T07:20:23.141 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-22T07:20:23.142 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-22T07:20:23.142 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** DB Stats ** 2024-03-22T07:20:23.142 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T07:20:23.142 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-22T07:20:23.142 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T07:20:23.142 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T07:20:23.142 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-22T07:20:23.142 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T07:20:23.142 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T07:20:23.142 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-03-22T07:20:23.142 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** Compaction Stats [default] ** 2024-03-22T07:20:23.142 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T07:20:23.142 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-22T07:20:23.142 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: L0 1/0 1.41 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 483.1 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-22T07:20:23.142 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: L6 1/0 6.97 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-22T07:20:23.143 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Sum 2/0 8.38 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 483.1 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-22T07:20:23.143 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 483.1 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-22T07:20:23.143 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-03-22T07:20:23.143 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** Compaction Stats [default] ** 2024-03-22T07:20:23.143 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T07:20:23.143 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T07:20:23.143 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 483.1 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-03-22T07:20:23.143 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-03-22T07:20:23.143 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-22T07:20:23.143 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-03-22T07:20:23.143 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T07:20:23.143 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Flush(GB): cumulative 0.001, interval 0.001 2024-03-22T07:20:23.143 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-22T07:20:23.143 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-22T07:20:23.143 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-22T07:20:23.143 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-22T07:20:23.143 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Cumulative compaction: 0.00 GB write, 67.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T07:20:23.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Interval compaction: 0.00 GB write, 67.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T07:20:23.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-22T07:20:23.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Block cache BinnedLRUCache@0x56387c06b090#2 capacity: 512.00 MB usage: 5.86 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-22T07:20:23.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.78 KB,0.000339746%) IndexBlock(1,4.08 KB,0.000777841%) Misc(1,0.00 KB,0%) 2024-03-22T07:20:23.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-03-22T07:20:23.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-22T07:20:23.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: starting mon.smithi063 rank 0 at public addrs [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] at bind addrs [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi063 fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:20:23.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: mon.smithi063@-1(???) e2 preinit fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:20:23.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: mon.smithi063@-1(???).mds e23 new map 2024-03-22T07:20:23.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: mon.smithi063@-1(???).mds e23 print_map 2024-03-22T07:20:23.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: e23 2024-03-22T07:20:23.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:20:23.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:20:23.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: legacy client fscid: 1 2024-03-22T07:20:23.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-03-22T07:20:23.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: Filesystem 'cephfs' (1) 2024-03-22T07:20:23.144 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: fs_name cephfs 2024-03-22T07:20:23.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: epoch 23 2024-03-22T07:20:23.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-22T07:20:23.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: created 2024-03-22T07:13:29.602905+0000 2024-03-22T07:20:23.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: modified 2024-03-22T07:20:16.089229+0000 2024-03-22T07:20:23.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: tableserver 0 2024-03-22T07:20:23.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: root 0 2024-03-22T07:20:23.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: session_timeout 60 2024-03-22T07:20:23.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: session_autoclose 300 2024-03-22T07:20:23.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: max_file_size 1099511627776 2024-03-22T07:20:23.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: max_xattr_size 65536 2024-03-22T07:20:23.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: required_client_features {} 2024-03-22T07:20:23.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: last_failure 0 2024-03-22T07:20:23.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: last_failure_osd_epoch 47 2024-03-22T07:20:23.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:20:23.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: max_mds 2 2024-03-22T07:20:23.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: in 0,1 2024-03-22T07:20:23.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: up {0=14480,1=24285} 2024-03-22T07:20:23.145 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: failed 2024-03-22T07:20:23.146 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: damaged 2024-03-22T07:20:23.146 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: stopped 2024-03-22T07:20:23.146 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: data_pools [3] 2024-03-22T07:20:23.146 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: metadata_pool 2 2024-03-22T07:20:23.146 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: inline_data disabled 2024-03-22T07:20:23.146 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: balancer 2024-03-22T07:20:23.146 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: bal_rank_mask -1 2024-03-22T07:20:23.146 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: standby_count_wanted 1 2024-03-22T07:20:23.146 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: [mds.cephfs.smithi063.xeelpk{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:20:23.146 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: [mds.cephfs.smithi063.oytapz{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3211595802,v1:172.21.15.63:6827/3211595802] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:20:23.146 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: [mds.cephfs.smithi069.wwtsjn{1:24285} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.69:6824/3798247699,v1:172.21.15.69:6825/3798247699] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:20:23.146 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: [mds.cephfs.smithi069.cijxah{1:24301} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.69:6826/3380719311,v1:172.21.15.69:6827/3380719311] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:20:23.146 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-03-22T07:20:23.146 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout: 2024-03-22T07:20:23.146 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: mon.smithi063@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-22T07:20:23.147 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: mon.smithi063@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T07:20:23.147 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: mon.smithi063@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T07:20:23.147 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: mon.smithi063@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T07:20:23.147 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 ceph-mon[112784]: mon.smithi063@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-03-22T07:20:23.147 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 bash[112765]: ab96538a07c1161a342a4f70fabad331e02fec061f547ca5d6f74b69114212b7 2024-03-22T07:20:23.147 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:22 smithi063 systemd[1]: Started Ceph mon.smithi063 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:20:24.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:24 smithi063 ceph-mon[112784]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:20:24.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:24 smithi063 ceph-mon[112784]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:20:24.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:24 smithi063 ceph-mon[112784]: pgmap v82: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:24.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:24 smithi063 ceph-mon[112784]: mon.smithi063 calling monitor election 2024-03-22T07:20:24.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:24 smithi063 ceph-mon[112784]: mon.smithi063 is new leader, mons smithi063,smithi069 in quorum (ranks 0,1) 2024-03-22T07:20:24.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:24 smithi063 ceph-mon[112784]: monmap e2: 2 mons at {smithi063=[v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0],smithi069=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T07:20:24.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:24 smithi063 ceph-mon[112784]: fsmap cephfs:2 {0=cephfs.smithi063.xeelpk=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2 up:standby-replay 2024-03-22T07:20:24.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:24 smithi063 ceph-mon[112784]: osdmap e49: 6 total, 6 up, 6 in 2024-03-22T07:20:24.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:24 smithi063 ceph-mon[112784]: mgrmap e33: smithi063.mnxayh(active, since 2m), standbys: smithi069.gfziwh 2024-03-22T07:20:24.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:24 smithi063 ceph-mon[112784]: overall HEALTH_OK 2024-03-22T07:20:24.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:24 smithi063 ceph-mon[112784]: from='mgr.14668 ' entity='' 2024-03-22T07:20:24.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:24 smithi063 ceph-mon[112784]: mgrmap e34: smithi063.mnxayh(active, since 2m), standbys: smithi069.gfziwh 2024-03-22T07:20:24.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:24 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:20:24.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:24 smithi069 ceph-mon[36441]: from='mgr.14668 172.21.15.63:0/1822005048' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:20:24.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:24 smithi069 ceph-mon[36441]: pgmap v82: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:24.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:24 smithi069 ceph-mon[36441]: mon.smithi063 calling monitor election 2024-03-22T07:20:24.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:24 smithi069 ceph-mon[36441]: mon.smithi063 is new leader, mons smithi063,smithi069 in quorum (ranks 0,1) 2024-03-22T07:20:24.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:24 smithi069 ceph-mon[36441]: monmap e2: 2 mons at {smithi063=[v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0],smithi069=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T07:20:24.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:24 smithi069 ceph-mon[36441]: fsmap cephfs:2 {0=cephfs.smithi063.xeelpk=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2 up:standby-replay 2024-03-22T07:20:24.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:24 smithi069 ceph-mon[36441]: osdmap e49: 6 total, 6 up, 6 in 2024-03-22T07:20:24.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:24 smithi069 ceph-mon[36441]: mgrmap e33: smithi063.mnxayh(active, since 2m), standbys: smithi069.gfziwh 2024-03-22T07:20:24.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:24 smithi069 ceph-mon[36441]: overall HEALTH_OK 2024-03-22T07:20:24.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:24 smithi069 ceph-mon[36441]: from='mgr.14668 ' entity='' 2024-03-22T07:20:24.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:24 smithi069 ceph-mon[36441]: mgrmap e34: smithi063.mnxayh(active, since 2m), standbys: smithi069.gfziwh 2024-03-22T07:20:25.561 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:25 smithi069 ceph-mon[36441]: fsmap cephfs:2 {0=cephfs.smithi063.xeelpk=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2 up:standby-replay 2024-03-22T07:20:25.614 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:25 smithi063 ceph-mon[112784]: fsmap cephfs:2 {0=cephfs.smithi063.xeelpk=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2 up:standby-replay 2024-03-22T07:20:31.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:31 smithi063 ceph-mon[112784]: Standby manager daemon smithi069.gfziwh restarted 2024-03-22T07:20:31.372 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:31 smithi063 ceph-mon[112784]: Standby manager daemon smithi069.gfziwh started 2024-03-22T07:20:31.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:31 smithi063 ceph-mon[112784]: from='mgr.? 172.21.15.69:0/3761096281' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/crt"}]: dispatch 2024-03-22T07:20:31.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:31 smithi063 ceph-mon[112784]: from='mgr.? 172.21.15.69:0/3761096281' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:20:31.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:31 smithi063 ceph-mon[112784]: from='mgr.? 172.21.15.69:0/3761096281' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/key"}]: dispatch 2024-03-22T07:20:31.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:31 smithi063 ceph-mon[112784]: from='mgr.? 172.21.15.69:0/3761096281' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:20:31.388 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:31 smithi069 ceph-mon[36441]: Standby manager daemon smithi069.gfziwh restarted 2024-03-22T07:20:31.388 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:31 smithi069 ceph-mon[36441]: Standby manager daemon smithi069.gfziwh started 2024-03-22T07:20:31.388 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:31 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/3761096281' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/crt"}]: dispatch 2024-03-22T07:20:31.388 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:31 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/3761096281' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-22T07:20:31.388 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:31 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/3761096281' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi069.gfziwh/key"}]: dispatch 2024-03-22T07:20:31.388 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:31 smithi069 ceph-mon[36441]: from='mgr.? 172.21.15.69:0/3761096281' entity='mgr.smithi069.gfziwh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-22T07:20:31.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:31 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi069[36427]: 2024-03-22T07:20:31.387+0000 7f08e43b4700 -1 mon.smithi069@1(peon).osd e50 update_from_paxos full map CRC mismatch, resetting to canonical 2024-03-22T07:20:32.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: mgrmap e35: smithi063.mnxayh(active, since 2m), standbys: smithi069.gfziwh 2024-03-22T07:20:32.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: Active manager daemon smithi063.mnxayh restarted 2024-03-22T07:20:32.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: Activating manager daemon smithi063.mnxayh 2024-03-22T07:20:32.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: osdmap e50: 6 total, 6 up, 6 in 2024-03-22T07:20:32.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: mgrmap e36: smithi063.mnxayh(active, starting, since 0.00549883s), standbys: smithi069.gfziwh 2024-03-22T07:20:32.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: failed to encode map e50 with expected crc 2024-03-22T07:20:32.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:20:32.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:20:32.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.xeelpk"}]: dispatch 2024-03-22T07:20:32.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.oytapz"}]: dispatch 2024-03-22T07:20:32.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.wwtsjn"}]: dispatch 2024-03-22T07:20:32.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.cijxah"}]: dispatch 2024-03-22T07:20:32.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr metadata", "who": "smithi063.mnxayh", "id": "smithi063.mnxayh"}]: dispatch 2024-03-22T07:20:32.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr metadata", "who": "smithi069.gfziwh", "id": "smithi069.gfziwh"}]: dispatch 2024-03-22T07:20:32.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:20:32.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:20:32.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:20:32.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:20:32.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:20:32.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:20:32.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:20:32.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:20:32.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:20:32.532 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: Manager daemon smithi063.mnxayh is now available 2024-03-22T07:20:32.532 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:20:32.532 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:20:32.532 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:20:32.532 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.mnxayh/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:20:32.532 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.mnxayh/trash_purge_schedule"}]: dispatch 2024-03-22T07:20:32.532 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: failed to encode map e50 with expected crc 2024-03-22T07:20:32.532 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:32 smithi069 ceph-mon[36441]: failed to encode map e50 with expected crc 2024-03-22T07:20:32.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: mgrmap e35: smithi063.mnxayh(active, since 2m), standbys: smithi069.gfziwh 2024-03-22T07:20:32.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: Active manager daemon smithi063.mnxayh restarted 2024-03-22T07:20:32.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: Activating manager daemon smithi063.mnxayh 2024-03-22T07:20:32.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: osdmap e50: 6 total, 6 up, 6 in 2024-03-22T07:20:32.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: mgrmap e36: smithi063.mnxayh(active, starting, since 0.00549883s), standbys: smithi069.gfziwh 2024-03-22T07:20:32.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: failed to encode map e50 with expected crc 2024-03-22T07:20:32.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:20:32.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:20:32.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.xeelpk"}]: dispatch 2024-03-22T07:20:32.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.oytapz"}]: dispatch 2024-03-22T07:20:32.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.wwtsjn"}]: dispatch 2024-03-22T07:20:32.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.cijxah"}]: dispatch 2024-03-22T07:20:32.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr metadata", "who": "smithi063.mnxayh", "id": "smithi063.mnxayh"}]: dispatch 2024-03-22T07:20:32.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr metadata", "who": "smithi069.gfziwh", "id": "smithi069.gfziwh"}]: dispatch 2024-03-22T07:20:32.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:20:32.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:20:32.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:20:32.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:20:32.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:20:32.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:20:32.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-22T07:20:32.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-22T07:20:32.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-22T07:20:32.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: Manager daemon smithi063.mnxayh is now available 2024-03-22T07:20:32.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:20:32.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:20:32.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:20:32.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.mnxayh/mirror_snapshot_schedule"}]: dispatch 2024-03-22T07:20:32.550 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi063.mnxayh/trash_purge_schedule"}]: dispatch 2024-03-22T07:20:32.550 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: failed to encode map e50 with expected crc 2024-03-22T07:20:32.550 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:32 smithi063 ceph-mon[112784]: failed to encode map e50 with expected crc 2024-03-22T07:20:33.449 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:33 smithi063 ceph-mon[112784]: [22/Mar/2024:07:20:31] ENGINE Bus STARTING 2024-03-22T07:20:33.449 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:33 smithi063 ceph-mon[112784]: [22/Mar/2024:07:20:31] ENGINE Serving on http://172.21.15.63:8765 2024-03-22T07:20:33.449 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:33 smithi063 ceph-mon[112784]: [22/Mar/2024:07:20:32] ENGINE Serving on https://172.21.15.63:7150 2024-03-22T07:20:33.449 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:33 smithi063 ceph-mon[112784]: [22/Mar/2024:07:20:32] ENGINE Bus STARTED 2024-03-22T07:20:33.449 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:33 smithi063 ceph-mon[112784]: mgrmap e37: smithi063.mnxayh(active, since 1.01006s), standbys: smithi069.gfziwh 2024-03-22T07:20:33.496 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:33 smithi069 ceph-mon[36441]: [22/Mar/2024:07:20:31] ENGINE Bus STARTING 2024-03-22T07:20:33.496 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:33 smithi069 ceph-mon[36441]: [22/Mar/2024:07:20:31] ENGINE Serving on http://172.21.15.63:8765 2024-03-22T07:20:33.497 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:33 smithi069 ceph-mon[36441]: [22/Mar/2024:07:20:32] ENGINE Serving on https://172.21.15.63:7150 2024-03-22T07:20:33.497 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:33 smithi069 ceph-mon[36441]: [22/Mar/2024:07:20:32] ENGINE Bus STARTED 2024-03-22T07:20:33.497 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:33 smithi069 ceph-mon[36441]: mgrmap e37: smithi063.mnxayh(active, since 1.01006s), standbys: smithi069.gfziwh 2024-03-22T07:20:34.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:34 smithi069 ceph-mon[36441]: pgmap v3: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:20:34.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:34 smithi063 ceph-mon[112784]: pgmap v3: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:20:35.427 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:35 smithi063 ceph-mon[112784]: pgmap v4: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:20:35.427 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:35 smithi063 ceph-mon[112784]: mgrmap e38: smithi063.mnxayh(active, since 3s), standbys: smithi069.gfziwh 2024-03-22T07:20:35.427 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:35 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:35.428 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:35 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:35.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:35 smithi069 ceph-mon[36441]: pgmap v4: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:20:35.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:35 smithi069 ceph-mon[36441]: mgrmap e38: smithi063.mnxayh(active, since 3s), standbys: smithi069.gfziwh 2024-03-22T07:20:35.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:35 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:35.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:35 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:37.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:36 smithi063 ceph-mon[112784]: pgmap v5: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:20:37.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:36 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:37.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:36 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:37.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:36 smithi063 ceph-mon[112784]: mgrmap e39: smithi063.mnxayh(active, since 4s), standbys: smithi069.gfziwh 2024-03-22T07:20:37.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:36 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:37.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:36 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:37.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:36 smithi069 ceph-mon[36441]: pgmap v5: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:20:37.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:36 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:37.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:36 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:37.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:36 smithi069 ceph-mon[36441]: mgrmap e39: smithi063.mnxayh(active, since 4s), standbys: smithi069.gfziwh 2024-03-22T07:20:37.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:36 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:37.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:36 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:39.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:38 smithi063 ceph-mon[112784]: pgmap v6: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:20:39.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:38 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:39.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:38 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:39.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:38 smithi069 ceph-mon[36441]: pgmap v6: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-03-22T07:20:39.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:38 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:39.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:38 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:41.031 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:40 smithi063 ceph-mon[112784]: pgmap v7: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-03-22T07:20:41.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:40 smithi069 ceph-mon[36441]: pgmap v7: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-03-22T07:20:42.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:42 smithi063 ceph-mon[112784]: pgmap v8: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-03-22T07:20:42.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:42 smithi063 ceph-mon[112784]: Detected new or changed devices on smithi063 2024-03-22T07:20:42.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:42 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:42.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:42 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:42.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:42 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:20:42.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:42 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:20:42.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:42 smithi063 ceph-mon[112784]: Updating smithi063:/etc/ceph/ceph.conf 2024-03-22T07:20:42.776 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:42 smithi063 ceph-mon[112784]: Updating smithi069:/etc/ceph/ceph.conf 2024-03-22T07:20:42.776 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:42 smithi063 ceph-mon[112784]: Updating smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:20:42.776 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:42 smithi063 ceph-mon[112784]: Updating smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:20:42.776 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:42 smithi063 ceph-mon[112784]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:20:42.776 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:42 smithi063 ceph-mon[112784]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:20:42.776 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:42 smithi063 ceph-mon[112784]: Updating smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:20:42.776 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:42 smithi063 ceph-mon[112784]: Updating smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:20:42.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:42 smithi069 ceph-mon[36441]: pgmap v8: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-03-22T07:20:42.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:42 smithi069 ceph-mon[36441]: Detected new or changed devices on smithi063 2024-03-22T07:20:42.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:42 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:42.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:42 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:42.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:42 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:20:42.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:42 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:20:42.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:42 smithi069 ceph-mon[36441]: Updating smithi063:/etc/ceph/ceph.conf 2024-03-22T07:20:42.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:42 smithi069 ceph-mon[36441]: Updating smithi069:/etc/ceph/ceph.conf 2024-03-22T07:20:42.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:42 smithi069 ceph-mon[36441]: Updating smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:20:42.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:42 smithi069 ceph-mon[36441]: Updating smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-22T07:20:42.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:42 smithi069 ceph-mon[36441]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:20:42.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:42 smithi069 ceph-mon[36441]: Updating smithi063:/etc/ceph/ceph.client.admin.keyring 2024-03-22T07:20:42.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:42 smithi069 ceph-mon[36441]: Updating smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:20:42.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:42 smithi069 ceph-mon[36441]: Updating smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-22T07:20:43.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:43 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:43.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:43 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:43.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:43 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:43.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:43 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:43.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:43 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:43.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:43 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:20:43.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:43 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:20:43.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:43 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-22T07:20:43.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:43.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:43.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:43.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:43.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:43.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:20:43.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:20:43.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-22T07:20:45.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:45 smithi069 ceph-mon[36441]: pgmap v9: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-22T07:20:45.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:45 smithi069 ceph-mon[36441]: Upgrade: Updating mon.smithi069 2024-03-22T07:20:45.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:45 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:45.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:45 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:20:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:45 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:20:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:45 smithi069 ceph-mon[36441]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:20:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:45 smithi069 ceph-mon[36441]: Deploying daemon mon.smithi069 on smithi069 2024-03-22T07:20:45.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:45 smithi063 ceph-mon[112784]: pgmap v9: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-22T07:20:45.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:45 smithi063 ceph-mon[112784]: Upgrade: Updating mon.smithi069 2024-03-22T07:20:45.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:45.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:20:45.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:20:45.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:20:45.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:45 smithi063 ceph-mon[112784]: Deploying daemon mon.smithi069 on smithi069 2024-03-22T07:20:46.398 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:46 smithi069 systemd[1]: Stopping Ceph mon.smithi069 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:20:46.739 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:46 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi069[36427]: 2024-03-22T07:20:46.397+0000 7f08e9bbf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi069 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:20:46.740 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:46 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi069[36427]: 2024-03-22T07:20:46.397+0000 7f08e9bbf700 -1 mon.smithi069@1(peon) e2 *** Got Signal Terminated *** 2024-03-22T07:20:46.740 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:46 smithi069 podman[83466]: 2024-03-22 07:20:46.448602834 +0000 UTC m=+0.196552939 container died 95f6103f760352f41574150c918cb0bb72c0c36d938ccef5e7764ceecdb35a07 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi069, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=quincy-9c4e0c3, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-22T07:20:47.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:46 smithi069 podman[83466]: 2024-03-22 07:20:46.852078528 +0000 UTC m=+0.600028633 container cleanup 95f6103f760352f41574150c918cb0bb72c0c36d938ccef5e7764ceecdb35a07 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi069, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3) 2024-03-22T07:20:47.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:46 smithi069 bash[83466]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi069 2024-03-22T07:20:47.514 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:47 smithi069 podman[83480]: 2024-03-22 07:20:47.182432164 +0000 UTC m=+0.729511112 container remove 95f6103f760352f41574150c918cb0bb72c0c36d938ccef5e7764ceecdb35a07 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi069, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9c4e0c3, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-22T07:20:47.514 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:47 smithi069 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mon.smithi069.service: Deactivated successfully. 2024-03-22T07:20:47.514 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:47 smithi069 systemd[1]: Stopped Ceph mon.smithi069 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:20:47.514 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:47 smithi069 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mon.smithi069.service: Consumed 6.664s CPU time. 2024-03-22T07:20:47.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:47 smithi069 systemd[1]: Starting Ceph mon.smithi069 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:20:47.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:47 smithi069 podman[83611]: 2024-03-22 07:20:47.630799263 +0000 UTC m=+0.018907240 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:20:47.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:47 smithi069 podman[83611]: 2024-03-22 07:20:47.741901846 +0000 UTC m=+0.130009861 container create 01a26e1fb1a1c6343ba1d9a1c8b0901dc592c34bceb7a7506cfbcfaab108b006 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi069, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-22T07:20:48.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:47 smithi069 podman[83611]: 2024-03-22 07:20:47.977651058 +0000 UTC m=+0.365759034 container init 01a26e1fb1a1c6343ba1d9a1c8b0901dc592c34bceb7a7506cfbcfaab108b006 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi069, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.build-date=20240307) 2024-03-22T07:20:48.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:47 smithi069 podman[83611]: 2024-03-22 07:20:47.980621411 +0000 UTC m=+0.368729384 container start 01a26e1fb1a1c6343ba1d9a1c8b0901dc592c34bceb7a7506cfbcfaab108b006 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi069, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-22T07:20:48.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: set uid:gid to 167:167 (ceph:ceph) 2024-03-22T07:20:48.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev), process ceph-mon, pid 2 2024-03-22T07:20:48.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: pidfile_write: ignore empty --pid-file 2024-03-22T07:20:48.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: load: jerasure load: lrc 2024-03-22T07:20:48.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: RocksDB version: 7.9.2 2024-03-22T07:20:48.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Git sha 0 2024-03-22T07:20:48.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Compile date 2024-03-16 10:36:11 2024-03-22T07:20:48.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: DB SUMMARY 2024-03-22T07:20:48.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: DB Session ID: BUOWOGUXAY74HVA6QL85 2024-03-22T07:20:48.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: CURRENT file: CURRENT 2024-03-22T07:20:48.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: IDENTITY file: IDENTITY 2024-03-22T07:20:48.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1097 Bytes 2024-03-22T07:20:48.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi069/store.db dir, Total Num: 1, files: 000027.sst 2024-03-22T07:20:48.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi069/store.db: 000025.log size: 5309205 ; 2024-03-22T07:20:48.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.error_if_exists: 0 2024-03-22T07:20:48.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.create_if_missing: 0 2024-03-22T07:20:48.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.paranoid_checks: 1 2024-03-22T07:20:48.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-22T07:20:48.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-22T07:20:48.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-22T07:20:48.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.env: 0x56022c6f5ae0 2024-03-22T07:20:48.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.fs: PosixFileSystem 2024-03-22T07:20:48.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.info_log: 0x56022e631760 2024-03-22T07:20:48.282 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_file_opening_threads: 16 2024-03-22T07:20:48.282 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.statistics: (nil) 2024-03-22T07:20:48.282 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.use_fsync: 0 2024-03-22T07:20:48.283 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_log_file_size: 0 2024-03-22T07:20:48.283 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-22T07:20:48.283 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-22T07:20:48.283 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.keep_log_file_num: 1000 2024-03-22T07:20:48.283 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.recycle_log_file_num: 0 2024-03-22T07:20:48.283 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.allow_fallocate: 1 2024-03-22T07:20:48.283 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.allow_mmap_reads: 0 2024-03-22T07:20:48.283 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.allow_mmap_writes: 0 2024-03-22T07:20:48.283 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.use_direct_reads: 0 2024-03-22T07:20:48.283 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-22T07:20:48.283 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.create_missing_column_families: 0 2024-03-22T07:20:48.283 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.db_log_dir: 2024-03-22T07:20:48.283 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.wal_dir: 2024-03-22T07:20:48.283 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-22T07:20:48.284 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-22T07:20:48.284 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-22T07:20:48.284 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-22T07:20:48.284 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-22T07:20:48.284 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-22T07:20:48.284 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.advise_random_on_open: 1 2024-03-22T07:20:48.284 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.db_write_buffer_size: 0 2024-03-22T07:20:48.284 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.write_buffer_manager: 0x56022e605400 2024-03-22T07:20:48.284 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-22T07:20:48.284 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-22T07:20:48.284 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-22T07:20:48.284 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.rate_limiter: (nil) 2024-03-22T07:20:48.284 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-22T07:20:48.284 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.wal_recovery_mode: 2 2024-03-22T07:20:48.284 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.enable_thread_tracking: 0 2024-03-22T07:20:48.285 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.enable_pipelined_write: 0 2024-03-22T07:20:48.285 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.unordered_write: 0 2024-03-22T07:20:48.285 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-22T07:20:48.285 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-22T07:20:48.285 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-22T07:20:48.285 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-22T07:20:48.285 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.row_cache: None 2024-03-22T07:20:48.285 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.wal_filter: None 2024-03-22T07:20:48.285 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-22T07:20:48.285 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.allow_ingest_behind: 0 2024-03-22T07:20:48.285 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.two_write_queues: 0 2024-03-22T07:20:48.285 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.manual_wal_flush: 0 2024-03-22T07:20:48.285 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.wal_compression: 0 2024-03-22T07:20:48.286 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.atomic_flush: 0 2024-03-22T07:20:48.286 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-22T07:20:48.286 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-22T07:20:48.286 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-22T07:20:48.286 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.log_readahead_size: 0 2024-03-22T07:20:48.286 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-22T07:20:48.286 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.best_efforts_recovery: 0 2024-03-22T07:20:48.286 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-22T07:20:48.286 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-22T07:20:48.286 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.allow_data_in_errors: 0 2024-03-22T07:20:48.286 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.db_host_id: __hostname__ 2024-03-22T07:20:48.286 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-22T07:20:48.286 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_background_jobs: 2 2024-03-22T07:20:48.286 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_background_compactions: -1 2024-03-22T07:20:48.286 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_subcompactions: 1 2024-03-22T07:20:48.287 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-22T07:20:48.287 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-22T07:20:48.287 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-22T07:20:48.287 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_total_wal_size: 0 2024-03-22T07:20:48.287 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-22T07:20:48.287 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-22T07:20:48.287 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-22T07:20:48.287 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-22T07:20:48.287 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_open_files: -1 2024-03-22T07:20:48.287 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.bytes_per_sync: 0 2024-03-22T07:20:48.287 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-22T07:20:48.287 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-22T07:20:48.287 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compaction_readahead_size: 0 2024-03-22T07:20:48.287 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_background_flushes: -1 2024-03-22T07:20:48.287 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Compression algorithms supported: 2024-03-22T07:20:48.287 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-22T07:20:48.287 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: kZSTD supported: 0 2024-03-22T07:20:48.288 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: kXpressCompression supported: 0 2024-03-22T07:20:48.288 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: kLZ4HCCompression supported: 1 2024-03-22T07:20:48.288 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: kZlibCompression supported: 1 2024-03-22T07:20:48.288 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: kSnappyCompression supported: 1 2024-03-22T07:20:48.288 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: kLZ4Compression supported: 1 2024-03-22T07:20:48.288 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: kBZip2Compression supported: 0 2024-03-22T07:20:48.288 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-22T07:20:48.288 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-22T07:20:48.288 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi069/store.db/MANIFEST-000009 2024-03-22T07:20:48.288 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-22T07:20:48.288 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-22T07:20:48.288 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.merge_operator: 2024-03-22T07:20:48.288 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compaction_filter: None 2024-03-22T07:20:48.288 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compaction_filter_factory: None 2024-03-22T07:20:48.288 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.sst_partitioner_factory: None 2024-03-22T07:20:48.289 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-22T07:20:48.289 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-22T07:20:48.289 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56022e631880) 2024-03-22T07:20:48.289 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: cache_index_and_filter_blocks: 1 2024-03-22T07:20:48.289 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-22T07:20:48.289 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-22T07:20:48.289 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: pin_top_level_index_and_filter: 1 2024-03-22T07:20:48.289 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: index_type: 0 2024-03-22T07:20:48.289 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: data_block_index_type: 0 2024-03-22T07:20:48.289 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: index_shortening: 1 2024-03-22T07:20:48.289 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-22T07:20:48.289 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: checksum: 4 2024-03-22T07:20:48.289 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: no_block_cache: 0 2024-03-22T07:20:48.289 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: block_cache: 0x56022e635090 2024-03-22T07:20:48.289 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: block_cache_name: BinnedLRUCache 2024-03-22T07:20:48.290 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: block_cache_options: 2024-03-22T07:20:48.290 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: capacity : 536870912 2024-03-22T07:20:48.290 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: num_shard_bits : 4 2024-03-22T07:20:48.290 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: strict_capacity_limit : 0 2024-03-22T07:20:48.290 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: high_pri_pool_ratio: 0.000 2024-03-22T07:20:48.290 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: block_cache_compressed: (nil) 2024-03-22T07:20:48.290 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: persistent_cache: (nil) 2024-03-22T07:20:48.290 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: block_size: 4096 2024-03-22T07:20:48.290 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: block_size_deviation: 10 2024-03-22T07:20:48.290 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: block_restart_interval: 16 2024-03-22T07:20:48.290 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: index_block_restart_interval: 1 2024-03-22T07:20:48.290 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: metadata_block_size: 4096 2024-03-22T07:20:48.290 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: partition_filters: 0 2024-03-22T07:20:48.290 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: use_delta_encoding: 1 2024-03-22T07:20:48.290 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: filter_policy: bloomfilter 2024-03-22T07:20:48.290 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: whole_key_filtering: 1 2024-03-22T07:20:48.291 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: verify_compression: 0 2024-03-22T07:20:48.291 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: read_amp_bytes_per_bit: 0 2024-03-22T07:20:48.291 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: format_version: 5 2024-03-22T07:20:48.291 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: enable_index_compression: 1 2024-03-22T07:20:48.291 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: block_align: 0 2024-03-22T07:20:48.291 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: max_auto_readahead_size: 262144 2024-03-22T07:20:48.291 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: prepopulate_block_cache: 0 2024-03-22T07:20:48.291 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: initial_auto_readahead_size: 8192 2024-03-22T07:20:48.291 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: num_file_reads_for_auto_readahead: 2 2024-03-22T07:20:48.291 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.write_buffer_size: 33554432 2024-03-22T07:20:48.291 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_write_buffer_number: 2 2024-03-22T07:20:48.291 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compression: NoCompression 2024-03-22T07:20:48.291 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.bottommost_compression: Disabled 2024-03-22T07:20:48.291 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.prefix_extractor: nullptr 2024-03-22T07:20:48.292 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-22T07:20:48.292 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.num_levels: 7 2024-03-22T07:20:48.292 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-22T07:20:48.292 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-22T07:20:48.292 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-22T07:20:48.292 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-22T07:20:48.292 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-22T07:20:48.292 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-22T07:20:48.292 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-22T07:20:48.292 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-22T07:20:48.292 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-22T07:20:48.292 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-22T07:20:48.292 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-22T07:20:48.292 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-22T07:20:48.292 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-22T07:20:48.292 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compression_opts.level: 32767 2024-03-22T07:20:48.292 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compression_opts.strategy: 0 2024-03-22T07:20:48.293 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-22T07:20:48.293 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-22T07:20:48.293 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-22T07:20:48.293 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-22T07:20:48.293 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compression_opts.enabled: false 2024-03-22T07:20:48.293 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-22T07:20:48.293 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-22T07:20:48.293 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-22T07:20:48.293 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-22T07:20:48.293 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.target_file_size_base: 67108864 2024-03-22T07:20:48.293 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-22T07:20:48.293 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-22T07:20:48.293 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-22T07:20:48.293 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-22T07:20:48.293 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-22T07:20:48.293 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-22T07:20:48.294 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-22T07:20:48.295 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-22T07:20:48.295 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-22T07:20:48.295 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-22T07:20:48.295 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-22T07:20:48.295 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-22T07:20:48.295 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-22T07:20:48.295 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-22T07:20:48.295 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.arena_block_size: 1048576 2024-03-22T07:20:48.295 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-22T07:20:48.295 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-22T07:20:48.295 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.disable_auto_compactions: 0 2024-03-22T07:20:48.295 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-22T07:20:48.295 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-22T07:20:48.295 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-22T07:20:48.295 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-22T07:20:48.296 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-22T07:20:48.296 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-22T07:20:48.296 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-22T07:20:48.296 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-22T07:20:48.296 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-22T07:20:48.296 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-22T07:20:48.296 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.table_properties_collectors: 2024-03-22T07:20:48.296 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.inplace_update_support: 0 2024-03-22T07:20:48.296 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-22T07:20:48.296 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-22T07:20:48.296 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-22T07:20:48.296 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-22T07:20:48.296 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.bloom_locality: 0 2024-03-22T07:20:48.296 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.max_successive_merges: 0 2024-03-22T07:20:48.296 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-22T07:20:48.296 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.paranoid_file_checks: 0 2024-03-22T07:20:48.296 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.force_consistency_checks: 1 2024-03-22T07:20:48.297 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.report_bg_io_stats: 0 2024-03-22T07:20:48.297 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.ttl: 2592000 2024-03-22T07:20:48.297 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-22T07:20:48.297 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-22T07:20:48.297 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-22T07:20:48.297 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.enable_blob_files: false 2024-03-22T07:20:48.297 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.min_blob_size: 0 2024-03-22T07:20:48.297 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.blob_file_size: 268435456 2024-03-22T07:20:48.297 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-22T07:20:48.297 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-22T07:20:48.297 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-22T07:20:48.297 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-22T07:20:48.297 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-22T07:20:48.297 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.blob_file_starting_level: 0 2024-03-22T07:20:48.297 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-22T07:20:48.297 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-03-22T07:20:48.297 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-22T07:20:48.297 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi069/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12282, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-22T07:20:48.298 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-03-22T07:20:48.298 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6504fbbe-97cc-47e0-a0dd-5d7a6c3f3001 2024-03-22T07:20:48.298 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711092048010300, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-03-22T07:20:48.298 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-03-22T07:20:48.298 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711092048024232, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 3275068, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12287, "largest_seqno": 14284, "table_properties": {"data_size": 3266720, "index_size": 5216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 2117, "raw_key_size": 19640, "raw_average_key_size": 23, "raw_value_size": 3249216, "raw_average_value_size": 3933, "num_data_blocks": 236, "num_entries": 826, "num_filter_entries": 826, "num_deletions": 7, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711092048, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6504fbbe-97cc-47e0-a0dd-5d7a6c3f3001", "db_session_id": "BUOWOGUXAY74HVA6QL85", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-03-22T07:20:48.298 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711092048024342, "job": 1, "event": "recovery_finished"} 2024-03-22T07:20:48.298 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-03-22T07:20:48.298 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-22T07:20:48.298 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi069/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-22T07:20:48.298 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56022e724000 2024-03-22T07:20:48.298 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: DB pointer 0x56022e710000 2024-03-22T07:20:48.298 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-22T07:20:48.298 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-22T07:20:48.298 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: ** DB Stats ** 2024-03-22T07:20:48.298 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T07:20:48.298 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-22T07:20:48.299 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T07:20:48.299 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T07:20:48.299 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-22T07:20:48.299 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-22T07:20:48.299 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-22T07:20:48.299 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: 2024-03-22T07:20:48.299 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: ** Compaction Stats [default] ** 2024-03-22T07:20:48.299 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T07:20:48.299 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-22T07:20:48.299 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: L0 1/0 3.12 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 666.8 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-22T07:20:48.299 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: L6 1/0 6.97 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-22T07:20:48.299 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Sum 2/0 10.09 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 666.8 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-22T07:20:48.299 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 666.8 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-22T07:20:48.299 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: 2024-03-22T07:20:48.299 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: ** Compaction Stats [default] ** 2024-03-22T07:20:48.299 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-22T07:20:48.299 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-22T07:20:48.299 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 666.8 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-22T07:20:48.300 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: 2024-03-22T07:20:48.300 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-22T07:20:48.300 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: 2024-03-22T07:20:48.300 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-22T07:20:48.300 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-03-22T07:20:48.300 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-22T07:20:48.300 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-22T07:20:48.300 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-22T07:20:48.300 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-22T07:20:48.300 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Cumulative compaction: 0.00 GB write, 168.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T07:20:48.300 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Interval compaction: 0.00 GB write, 168.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-22T07:20:48.300 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-22T07:20:48.300 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Block cache BinnedLRUCache@0x56022e635090#2 capacity: 512.00 MB usage: 8.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-22T07:20:48.300 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.28 KB,0.000435114%) IndexBlock(1,6.08 KB,0.00115931%) Misc(1,0.00 KB,0%) 2024-03-22T07:20:48.300 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: 2024-03-22T07:20:48.300 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-22T07:20:48.300 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: starting mon.smithi069 rank 1 at public addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] at bind addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi069 fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:20:48.301 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: mon.smithi069@-1(???) e2 preinit fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 2024-03-22T07:20:48.301 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: mon.smithi069@-1(???).mds e24 new map 2024-03-22T07:20:48.301 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: mon.smithi069@-1(???).mds e24 print_map 2024-03-22T07:20:48.301 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: e24 2024-03-22T07:20:48.301 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:20:48.301 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:20:48.301 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: legacy client fscid: 1 2024-03-22T07:20:48.301 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: 2024-03-22T07:20:48.301 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: Filesystem 'cephfs' (1) 2024-03-22T07:20:48.301 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: fs_name cephfs 2024-03-22T07:20:48.301 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: epoch 24 2024-03-22T07:20:48.301 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-22T07:20:48.301 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: created 2024-03-22T07:13:29.602905+0000 2024-03-22T07:20:48.301 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: modified 2024-03-22T07:20:24.253589+0000 2024-03-22T07:20:48.301 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: tableserver 0 2024-03-22T07:20:48.301 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: root 0 2024-03-22T07:20:48.301 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: session_timeout 60 2024-03-22T07:20:48.301 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: session_autoclose 300 2024-03-22T07:20:48.302 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: max_file_size 1099511627776 2024-03-22T07:20:48.302 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: max_xattr_size 65536 2024-03-22T07:20:48.302 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: required_client_features {} 2024-03-22T07:20:48.302 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: last_failure 0 2024-03-22T07:20:48.302 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: last_failure_osd_epoch 47 2024-03-22T07:20:48.302 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:20:48.302 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: max_mds 2 2024-03-22T07:20:48.302 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: in 0,1 2024-03-22T07:20:48.302 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: up {0=14480,1=24285} 2024-03-22T07:20:48.302 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: failed 2024-03-22T07:20:48.302 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: damaged 2024-03-22T07:20:48.302 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: stopped 2024-03-22T07:20:48.302 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: data_pools [3] 2024-03-22T07:20:48.302 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: metadata_pool 2 2024-03-22T07:20:48.302 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: inline_data disabled 2024-03-22T07:20:48.303 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: balancer 2024-03-22T07:20:48.303 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: bal_rank_mask -1 2024-03-22T07:20:48.303 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: standby_count_wanted 1 2024-03-22T07:20:48.303 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: [mds.cephfs.smithi063.xeelpk{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:20:48.303 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: [mds.cephfs.smithi063.oytapz{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3211595802,v1:172.21.15.63:6827/3211595802] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:20:48.303 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: [mds.cephfs.smithi069.wwtsjn{1:24285} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.69:6824/3798247699,v1:172.21.15.69:6825/3798247699] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:20:48.303 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: [mds.cephfs.smithi069.cijxah{1:24301} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.69:6826/3380719311,v1:172.21.15.69:6827/3380719311] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:20:48.303 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: 2024-03-22T07:20:48.303 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout: 2024-03-22T07:20:48.303 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: mon.smithi069@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-22T07:20:48.303 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: mon.smithi069@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T07:20:48.303 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: mon.smithi069@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T07:20:48.303 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: mon.smithi069@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-22T07:20:48.303 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: mon.smithi069@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-03-22T07:20:48.303 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 bash[83611]: 01a26e1fb1a1c6343ba1d9a1c8b0901dc592c34bceb7a7506cfbcfaab108b006 2024-03-22T07:20:48.303 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 systemd[1]: Started Ceph mon.smithi069 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:20:49.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: pgmap v10: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-22T07:20:49.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:48 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:20:49.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:48 smithi063 ceph-mon[112784]: pgmap v10: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-22T07:20:49.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:48 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:20:49.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.413+0000 7fc307a59700 1 -- 172.21.15.63:0/3935896528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc300102e00 msgr2=0x7fc3001031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.413+0000 7fc307a59700 1 --2- 172.21.15.63:0/3935896528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc300102e00 0x7fc3001031d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009ce0 tx=0x7fc2f8004d10 comp rx=0 tx=0).stop 2024-03-22T07:20:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.413+0000 7fc307a59700 1 -- 172.21.15.63:0/3935896528 shutdown_connections 2024-03-22T07:20:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.413+0000 7fc307a59700 1 --2- 172.21.15.63:0/3935896528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc3001037a0 0x7fc3000fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.413+0000 7fc307a59700 1 --2- 172.21.15.63:0/3935896528 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc300102e00 0x7fc3001031d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.413+0000 7fc307a59700 1 -- 172.21.15.63:0/3935896528 >> 172.21.15.63:0/3935896528 conn(0x7fc3000f89b0 msgr2=0x7fc3000fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.413+0000 7fc307a59700 1 -- 172.21.15.63:0/3935896528 shutdown_connections 2024-03-22T07:20:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.413+0000 7fc307a59700 1 -- 172.21.15.63:0/3935896528 wait complete. 2024-03-22T07:20:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.414+0000 7fc307a59700 1 Processor -- start 2024-03-22T07:20:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.414+0000 7fc307a59700 1 -- start start 2024-03-22T07:20:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.414+0000 7fc307a59700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc300102e00 0x7fc300113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.414+0000 7fc307a59700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3001037a0 0x7fc300114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.414+0000 7fc307a59700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc300114660 con 0x7fc3001037a0 2024-03-22T07:20:49.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.414+0000 7fc307a59700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30010dc40 con 0x7fc300102e00 2024-03-22T07:20:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.414+0000 7fc304ff4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3001037a0 0x7fc300114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.414+0000 7fc304ff4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3001037a0 0x7fc300114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41282/0 (socket says 172.21.15.63:41282) 2024-03-22T07:20:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.414+0000 7fc304ff4700 1 -- 172.21.15.63:0/4029786069 learned_addr learned my addr 172.21.15.63:0/4029786069 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.414+0000 7fc3057f5700 1 --2- 172.21.15.63:0/4029786069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc300102e00 0x7fc300113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.414+0000 7fc3057f5700 1 -- 172.21.15.63:0/4029786069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc300102e00 msgr2=0x7fc300113ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-22T07:20:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.414+0000 7fc3057f5700 1 -- 172.21.15.63:0/4029786069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc300102e00 msgr2=0x7fc300113ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T07:20:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.414+0000 7fc3057f5700 1 --2- 172.21.15.63:0/4029786069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc300102e00 0x7fc300113ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-22T07:20:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.414+0000 7fc3057f5700 1 --2- 172.21.15.63:0/4029786069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc300102e00 0x7fc300113ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:20:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.414+0000 7fc304ff4700 1 -- 172.21.15.63:0/4029786069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc300102e00 msgr2=0x7fc300113ad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:20:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.414+0000 7fc304ff4700 1 --2- 172.21.15.63:0/4029786069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc300102e00 0x7fc300113ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.414+0000 7fc304ff4700 1 -- 172.21.15.63:0/4029786069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f8009940 con 0x7fc3001037a0 2024-03-22T07:20:49.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.415+0000 7fc304ff4700 1 --2- 172.21.15.63:0/4029786069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3001037a0 0x7fc300114010 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4002a80 tx=0x7fc2f4004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.415+0000 7fc2fdffb700 1 -- 172.21.15.63:0/4029786069 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2f4007cc0 con 0x7fc3001037a0 2024-03-22T07:20:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.415+0000 7fc2fdffb700 1 -- 172.21.15.63:0/4029786069 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2f400f040 con 0x7fc3001037a0 2024-03-22T07:20:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.415+0000 7fc307a59700 1 -- 172.21.15.63:0/4029786069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30010df20 con 0x7fc3001037a0 2024-03-22T07:20:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.415+0000 7fc2fdffb700 1 -- 172.21.15.63:0/4029786069 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2f400db30 con 0x7fc3001037a0 2024-03-22T07:20:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.415+0000 7fc307a59700 1 -- 172.21.15.63:0/4029786069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30010e4e0 con 0x7fc3001037a0 2024-03-22T07:20:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.416+0000 7fc307a59700 1 -- 172.21.15.63:0/4029786069 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc300100110 con 0x7fc3001037a0 2024-03-22T07:20:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.416+0000 7fc2fdffb700 1 -- 172.21.15.63:0/4029786069 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7fc2f401c400 con 0x7fc3001037a0 2024-03-22T07:20:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.417+0000 7fc2fdffb700 1 --2- 172.21.15.63:0/4029786069 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc2e806d4b0 0x7fc2e806f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.417+0000 7fc2fdffb700 1 -- 172.21.15.63:0/4029786069 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc2f408c210 con 0x7fc3001037a0 2024-03-22T07:20:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.417+0000 7fc3057f5700 1 --2- 172.21.15.63:0/4029786069 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc2e806d4b0 0x7fc2e806f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:49.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.418+0000 7fc3057f5700 1 --2- 172.21.15.63:0/4029786069 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc2e806d4b0 0x7fc2e806f960 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800b740 tx=0x7fc2f8004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:49.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.422+0000 7fc2fdffb700 1 -- 172.21.15.63:0/4029786069 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc2f4059600 con 0x7fc3001037a0 2024-03-22T07:20:49.649 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.649+0000 7fc307a59700 1 -- 172.21.15.63:0/4029786069 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc30010f150 con 0x7fc2e806d4b0 2024-03-22T07:20:49.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.650+0000 7fc2fdffb700 1 -- 172.21.15.63:0/4029786069 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fc30010f150 con 0x7fc2e806d4b0 2024-03-22T07:20:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.652+0000 7fc307a59700 1 -- 172.21.15.63:0/4029786069 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc2e806d4b0 msgr2=0x7fc2e806f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.652+0000 7fc307a59700 1 --2- 172.21.15.63:0/4029786069 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc2e806d4b0 0x7fc2e806f960 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800b740 tx=0x7fc2f8004d10 comp rx=0 tx=0).stop 2024-03-22T07:20:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.652+0000 7fc307a59700 1 -- 172.21.15.63:0/4029786069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3001037a0 msgr2=0x7fc300114010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.652+0000 7fc307a59700 1 --2- 172.21.15.63:0/4029786069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3001037a0 0x7fc300114010 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4002a80 tx=0x7fc2f4004630 comp rx=0 tx=0).stop 2024-03-22T07:20:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.652+0000 7fc307a59700 1 -- 172.21.15.63:0/4029786069 shutdown_connections 2024-03-22T07:20:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.652+0000 7fc307a59700 1 --2- 172.21.15.63:0/4029786069 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc2e806d4b0 0x7fc2e806f960 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.652+0000 7fc307a59700 1 --2- 172.21.15.63:0/4029786069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc300102e00 0x7fc300113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:49.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.652+0000 7fc307a59700 1 --2- 172.21.15.63:0/4029786069 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc3001037a0 0x7fc300114010 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:49.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.652+0000 7fc307a59700 1 -- 172.21.15.63:0/4029786069 >> 172.21.15.63:0/4029786069 conn(0x7fc3000f89b0 msgr2=0x7fc3001073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:49.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.652+0000 7fc307a59700 1 -- 172.21.15.63:0/4029786069 shutdown_connections 2024-03-22T07:20:49.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.652+0000 7fc307a59700 1 -- 172.21.15.63:0/4029786069 wait complete. 2024-03-22T07:20:49.662 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:20:49.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.757+0000 7f7e52608700 1 -- 172.21.15.63:0/1565684742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e4c102e20 msgr2=0x7f7e4c1031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:49.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.757+0000 7f7e52608700 1 --2- 172.21.15.63:0/1565684742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e4c102e20 0x7f7e4c1031f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c004940 tx=0x7f7e3c009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:49.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.757+0000 7f7e52608700 1 -- 172.21.15.63:0/1565684742 shutdown_connections 2024-03-22T07:20:49.757 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.757+0000 7f7e52608700 1 --2- 172.21.15.63:0/1565684742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e4c1037c0 0x7f7e4c0fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:49.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.757+0000 7f7e52608700 1 --2- 172.21.15.63:0/1565684742 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e4c102e20 0x7f7e4c1031f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:49.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.757+0000 7f7e52608700 1 -- 172.21.15.63:0/1565684742 >> 172.21.15.63:0/1565684742 conn(0x7f7e4c0f89d0 msgr2=0x7f7e4c0fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:49.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.757+0000 7f7e52608700 1 -- 172.21.15.63:0/1565684742 shutdown_connections 2024-03-22T07:20:49.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.757+0000 7f7e52608700 1 -- 172.21.15.63:0/1565684742 wait complete. 2024-03-22T07:20:49.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.757+0000 7f7e52608700 1 Processor -- start 2024-03-22T07:20:49.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.757+0000 7f7e52608700 1 -- start start 2024-03-22T07:20:49.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.758+0000 7f7e52608700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e4c102e20 0x7f7e4c113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.758+0000 7f7e52608700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e4c1037c0 0x7f7e4c1140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.758+0000 7f7e52608700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c114720 con 0x7f7e4c1037c0 2024-03-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.758+0000 7f7e52608700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c10db10 con 0x7f7e4c102e20 2024-03-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.758+0000 7f7e4b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e4c1037c0 0x7f7e4c1140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.758+0000 7f7e4bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e4c102e20 0x7f7e4c113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.758+0000 7f7e4b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e4c1037c0 0x7f7e4c1140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41302/0 (socket says 172.21.15.63:41302) 2024-03-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.758+0000 7f7e4bfff700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e4c102e20 0x7f7e4c113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.63:44588/0 (socket says 172.21.15.63:44588) 2024-03-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.758+0000 7f7e4b7fe700 1 -- 172.21.15.63:0/813261199 learned_addr learned my addr 172.21.15.63:0/813261199 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.758+0000 7f7e4bfff700 1 -- 172.21.15.63:0/813261199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e4c102e20 msgr2=0x7f7e4c113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.758+0000 7f7e4bfff700 1 -- 172.21.15.63:0/813261199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e4c102e20 msgr2=0x7f7e4c113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.758+0000 7f7e4bfff700 1 --2- 172.21.15.63:0/813261199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e4c102e20 0x7f7e4c113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.758+0000 7f7e4bfff700 1 --2- 172.21.15.63:0/813261199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e4c102e20 0x7f7e4c113b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:20:49.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.758+0000 7f7e4b7fe700 1 -- 172.21.15.63:0/813261199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e4c102e20 msgr2=0x7f7e4c113b90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.758+0000 7f7e4b7fe700 1 --2- 172.21.15.63:0/813261199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e4c102e20 0x7f7e4c113b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.758+0000 7f7e4b7fe700 1 -- 172.21.15.63:0/813261199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c005040 con 0x7f7e4c1037c0 2024-03-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.758+0000 7f7e4b7fe700 1 --2- 172.21.15.63:0/813261199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e4c1037c0 0x7f7e4c1140d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000c800 tx=0x7f7e40004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.758+0000 7f7e48ff9700 1 -- 172.21.15.63:0/813261199 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e4000a760 con 0x7f7e4c1037c0 2024-03-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.759+0000 7f7e48ff9700 1 -- 172.21.15.63:0/813261199 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e4000ad90 con 0x7f7e4c1037c0 2024-03-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.759+0000 7f7e48ff9700 1 -- 172.21.15.63:0/813261199 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e400113a0 con 0x7f7e4c1037c0 2024-03-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.759+0000 7f7e52608700 1 -- 172.21.15.63:0/813261199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e4c10ddf0 con 0x7f7e4c1037c0 2024-03-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.759+0000 7f7e52608700 1 -- 172.21.15.63:0/813261199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e4c10e3b0 con 0x7f7e4c1037c0 2024-03-22T07:20:49.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.760+0000 7f7e52608700 1 -- 172.21.15.63:0/813261199 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e4c100130 con 0x7f7e4c1037c0 2024-03-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.760+0000 7f7e48ff9700 1 -- 172.21.15.63:0/813261199 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f7e40012400 con 0x7f7e4c1037c0 2024-03-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.760+0000 7f7e48ff9700 1 --2- 172.21.15.63:0/813261199 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7e3406d4a0 0x7f7e3406f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.761+0000 7f7e48ff9700 1 -- 172.21.15.63:0/813261199 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7e4008c910 con 0x7f7e4c1037c0 2024-03-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.761+0000 7f7e4bfff700 1 --2- 172.21.15.63:0/813261199 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7e3406d4a0 0x7f7e3406f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:49.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.761+0000 7f7e4bfff700 1 --2- 172.21.15.63:0/813261199 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7e3406d4a0 0x7f7e3406f950 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00b3b0 tx=0x7f7e3c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:49.765 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.765+0000 7f7e48ff9700 1 -- 172.21.15.63:0/813261199 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7e40059f30 con 0x7f7e4c1037c0 2024-03-22T07:20:49.987 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.986+0000 7f7e52608700 1 -- 172.21.15.63:0/813261199 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e4c10f250 con 0x7f7e3406d4a0 2024-03-22T07:20:49.987 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:49 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:20:49.987 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:49 smithi063 ceph-mon[112784]: mon.smithi069 calling monitor election 2024-03-22T07:20:49.987 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:49 smithi063 ceph-mon[112784]: mon.smithi063 calling monitor election 2024-03-22T07:20:49.987 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:49 smithi063 ceph-mon[112784]: mon.smithi063 is new leader, mons smithi063,smithi069 in quorum (ranks 0,1) 2024-03-22T07:20:49.987 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:49 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:20:49.987 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:49 smithi063 ceph-mon[112784]: monmap e3: 2 mons at {smithi063=[v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0],smithi069=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T07:20:49.987 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:49 smithi063 ceph-mon[112784]: fsmap cephfs:2 {0=cephfs.smithi063.xeelpk=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2 up:standby-replay 2024-03-22T07:20:49.987 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:49 smithi063 ceph-mon[112784]: osdmap e50: 6 total, 6 up, 6 in 2024-03-22T07:20:49.988 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:49 smithi063 ceph-mon[112784]: mgrmap e39: smithi063.mnxayh(active, since 17s), standbys: smithi069.gfziwh 2024-03-22T07:20:49.988 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:49 smithi063 ceph-mon[112784]: overall HEALTH_OK 2024-03-22T07:20:49.988 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:49 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:49.988 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:49 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:49.988 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:49 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:20:49.988 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.988+0000 7f7e48ff9700 1 -- 172.21.15.63:0/813261199 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f7e4c10f250 con 0x7f7e3406d4a0 2024-03-22T07:20:49.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.990+0000 7f7e52608700 1 -- 172.21.15.63:0/813261199 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7e3406d4a0 msgr2=0x7f7e3406f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:49.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.990+0000 7f7e52608700 1 --2- 172.21.15.63:0/813261199 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7e3406d4a0 0x7f7e3406f950 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00b3b0 tx=0x7f7e3c009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:49.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.990+0000 7f7e52608700 1 -- 172.21.15.63:0/813261199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e4c1037c0 msgr2=0x7f7e4c1140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:49.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.990+0000 7f7e52608700 1 --2- 172.21.15.63:0/813261199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e4c1037c0 0x7f7e4c1140d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000c800 tx=0x7f7e40004470 comp rx=0 tx=0).stop 2024-03-22T07:20:49.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.990+0000 7f7e52608700 1 -- 172.21.15.63:0/813261199 shutdown_connections 2024-03-22T07:20:49.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.990+0000 7f7e52608700 1 --2- 172.21.15.63:0/813261199 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7e3406d4a0 0x7f7e3406f950 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:49.990 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.990+0000 7f7e52608700 1 --2- 172.21.15.63:0/813261199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e4c102e20 0x7f7e4c113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:49.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.990+0000 7f7e52608700 1 --2- 172.21.15.63:0/813261199 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7e4c1037c0 0x7f7e4c1140d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:49.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.990+0000 7f7e52608700 1 -- 172.21.15.63:0/813261199 >> 172.21.15.63:0/813261199 conn(0x7f7e4c0f89d0 msgr2=0x7f7e4c107410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:49.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.990+0000 7f7e52608700 1 -- 172.21.15.63:0/813261199 shutdown_connections 2024-03-22T07:20:49.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:49.990+0000 7f7e52608700 1 -- 172.21.15.63:0/813261199 wait complete. 2024-03-22T07:20:50.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:49 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi063"}]: dispatch 2024-03-22T07:20:50.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:49 smithi069 ceph-mon[83631]: mon.smithi069 calling monitor election 2024-03-22T07:20:50.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:49 smithi069 ceph-mon[83631]: mon.smithi063 calling monitor election 2024-03-22T07:20:50.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:49 smithi069 ceph-mon[83631]: mon.smithi063 is new leader, mons smithi063,smithi069 in quorum (ranks 0,1) 2024-03-22T07:20:50.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:49 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mon metadata", "id": "smithi069"}]: dispatch 2024-03-22T07:20:50.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:49 smithi069 ceph-mon[83631]: monmap e3: 2 mons at {smithi063=[v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0],smithi069=[v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-22T07:20:50.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:49 smithi069 ceph-mon[83631]: fsmap cephfs:2 {0=cephfs.smithi063.xeelpk=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2 up:standby-replay 2024-03-22T07:20:50.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:49 smithi069 ceph-mon[83631]: osdmap e50: 6 total, 6 up, 6 in 2024-03-22T07:20:50.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:49 smithi069 ceph-mon[83631]: mgrmap e39: smithi063.mnxayh(active, since 17s), standbys: smithi069.gfziwh 2024-03-22T07:20:50.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:49 smithi069 ceph-mon[83631]: overall HEALTH_OK 2024-03-22T07:20:50.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:49 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:50.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:49 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:50.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:49 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:20:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.092+0000 7fe826cb9700 1 -- 172.21.15.63:0/2078992374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8201037b0 msgr2=0x7fe8200fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.092+0000 7fe826cb9700 1 --2- 172.21.15.63:0/2078992374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8201037b0 0x7fe8200fce20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe810004940 tx=0x7fe810009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.092+0000 7fe826cb9700 1 -- 172.21.15.63:0/2078992374 shutdown_connections 2024-03-22T07:20:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.092+0000 7fe826cb9700 1 --2- 172.21.15.63:0/2078992374 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe8201037b0 0x7fe8200fce20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:50.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.092+0000 7fe826cb9700 1 --2- 172.21.15.63:0/2078992374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe820102e10 0x7fe8201031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:50.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.092+0000 7fe826cb9700 1 -- 172.21.15.63:0/2078992374 >> 172.21.15.63:0/2078992374 conn(0x7fe8200f8980 msgr2=0x7fe8200fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:50.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.092+0000 7fe826cb9700 1 -- 172.21.15.63:0/2078992374 shutdown_connections 2024-03-22T07:20:50.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.092+0000 7fe826cb9700 1 -- 172.21.15.63:0/2078992374 wait complete. 2024-03-22T07:20:50.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.093+0000 7fe826cb9700 1 Processor -- start 2024-03-22T07:20:50.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.093+0000 7fe826cb9700 1 -- start start 2024-03-22T07:20:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.093+0000 7fe826cb9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe820102e10 0x7fe820115d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.093+0000 7fe826cb9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8201037b0 0x7fe8201162d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.093+0000 7fe826cb9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe820116920 con 0x7fe820102e10 2024-03-22T07:20:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.093+0000 7fe826cb9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82010fcf0 con 0x7fe8201037b0 2024-03-22T07:20:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.093+0000 7fe824a55700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe820102e10 0x7fe820115d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.093+0000 7fe824a55700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe820102e10 0x7fe820115d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41314/0 (socket says 172.21.15.63:41314) 2024-03-22T07:20:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.093+0000 7fe824a55700 1 -- 172.21.15.63:0/2886534500 learned_addr learned my addr 172.21.15.63:0/2886534500 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.093+0000 7fe81ffff700 1 --2- 172.21.15.63:0/2886534500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8201037b0 0x7fe8201162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.093+0000 7fe81ffff700 1 -- 172.21.15.63:0/2886534500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8201037b0 msgr2=0x7fe8201162d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-22T07:20:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.093+0000 7fe81ffff700 1 -- 172.21.15.63:0/2886534500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8201037b0 msgr2=0x7fe8201162d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T07:20:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.093+0000 7fe81ffff700 1 --2- 172.21.15.63:0/2886534500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8201037b0 0x7fe8201162d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-22T07:20:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.093+0000 7fe81ffff700 1 --2- 172.21.15.63:0/2886534500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8201037b0 0x7fe8201162d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:20:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.094+0000 7fe824a55700 1 -- 172.21.15.63:0/2886534500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8201037b0 msgr2=0x7fe8201162d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:20:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.094+0000 7fe824a55700 1 --2- 172.21.15.63:0/2886534500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8201037b0 0x7fe8201162d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.094+0000 7fe824a55700 1 -- 172.21.15.63:0/2886534500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe810005040 con 0x7fe820102e10 2024-03-22T07:20:50.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.094+0000 7fe824a55700 1 --2- 172.21.15.63:0/2886534500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe820102e10 0x7fe820115d90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe80800c8c0 tx=0x7fe8080044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:50.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.094+0000 7fe81d7fa700 1 -- 172.21.15.63:0/2886534500 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe80800a980 con 0x7fe820102e10 2024-03-22T07:20:50.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.094+0000 7fe81d7fa700 1 -- 172.21.15.63:0/2886534500 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe80800ce60 con 0x7fe820102e10 2024-03-22T07:20:50.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.094+0000 7fe826cb9700 1 -- 172.21.15.63:0/2886534500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe82010ffd0 con 0x7fe820102e10 2024-03-22T07:20:50.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.094+0000 7fe81d7fa700 1 -- 172.21.15.63:0/2886534500 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8080115e0 con 0x7fe820102e10 2024-03-22T07:20:50.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.094+0000 7fe826cb9700 1 -- 172.21.15.63:0/2886534500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe820110590 con 0x7fe820102e10 2024-03-22T07:20:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.095+0000 7fe826cb9700 1 -- 172.21.15.63:0/2886534500 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe820102130 con 0x7fe820102e10 2024-03-22T07:20:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.095+0000 7fe81d7fa700 1 -- 172.21.15.63:0/2886534500 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7fe808012400 con 0x7fe820102e10 2024-03-22T07:20:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.096+0000 7fe81d7fa700 1 --2- 172.21.15.63:0/2886534500 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe80c06d4a0 0x7fe80c06f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.096+0000 7fe81d7fa700 1 -- 172.21.15.63:0/2886534500 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe80808cb10 con 0x7fe820102e10 2024-03-22T07:20:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.096+0000 7fe81ffff700 1 --2- 172.21.15.63:0/2886534500 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe80c06d4a0 0x7fe80c06f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:50.097 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.096+0000 7fe81ffff700 1 --2- 172.21.15.63:0/2886534500 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe80c06d4a0 0x7fe80c06f950 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe81000b3b0 tx=0x7fe810009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:50.100 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.101+0000 7fe81d7fa700 1 -- 172.21.15.63:0/2886534500 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe80805a090 con 0x7fe820102e10 2024-03-22T07:20:50.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.314+0000 7fe826cb9700 1 -- 172.21.15.63:0/2886534500 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8201113e0 con 0x7fe80c06d4a0 2024-03-22T07:20:50.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.325+0000 7fe81d7fa700 1 -- 172.21.15.63:0/2886534500 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fe8201113e0 con 0x7fe80c06d4a0 2024-03-22T07:20:50.325 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:20:50.325 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (111s) 13s ago 12m 17.8M - 0.25.0 c8568f914cd2 06ef21da90e5 2024-03-22T07:20:50.325 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (12m) 13s ago 12m 6660k - 17.2.7-726-g9c4e0c34 837124da804f d6c6bbeb6904 2024-03-22T07:20:50.326 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (11m) 15s ago 11m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 30992f8aa88c 2024-03-22T07:20:50.326 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (94s) 13s ago 12m 82.9M - 9.4.12 a0ea5e3c46d9 9015fae0d5e9 2024-03-22T07:20:50.326 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (7m) 13s ago 7m 74.1M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:20:50.326 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (7m) 13s ago 7m 124M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:20:50.326 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (7m) 15s ago 7m 62.8M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:20:50.326 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (7m) 15s ago 7m 129M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:20:50.326 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:8443,9283,8765 running (3m) 13s ago 14m 475M - 19.0.0-1667-gdb0330b1 4f325f8440e9 e49468f39991 2024-03-22T07:20:50.326 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283,8765 running (2m) 15s ago 11m 446M - 19.0.0-1667-gdb0330b1 4f325f8440e9 13c3f034ebb2 2024-03-22T07:20:50.326 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (27s) 13s ago 14m 30.8M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ab96538a07c1 2024-03-22T07:20:50.326 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 starting - - - 2048M 2024-03-22T07:20:50.326 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (2m) 13s ago 12m 16.2M - 1.5.0 0da6a335fe13 0c825ee2a196 2024-03-22T07:20:50.326 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (2m) 15s ago 11m 16.0M - 1.5.0 0da6a335fe13 12edb2f35498 2024-03-22T07:20:50.326 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (10m) 13s ago 10m 218M 1557M 17.2.7-726-g9c4e0c34 837124da804f 580f22188a10 2024-03-22T07:20:50.326 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (10m) 13s ago 10m 243M 1557M 17.2.7-726-g9c4e0c34 837124da804f fe1ace35c88a 2024-03-22T07:20:50.327 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (10m) 13s ago 10m 199M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4cd995265dcf 2024-03-22T07:20:50.327 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (9m) 15s ago 9m 280M 2581M 17.2.7-726-g9c4e0c34 837124da804f 45c9a759b76d 2024-03-22T07:20:50.327 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (9m) 15s ago 9m 240M 2581M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:20:50.327 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (8m) 15s ago 8m 239M 2581M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:20:50.327 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (2m) 13s ago 12m 56.8M - 2.43.0 a07b618ecd1d 1d89b928f6e7 2024-03-22T07:20:50.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.327+0000 7fe826cb9700 1 -- 172.21.15.63:0/2886534500 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe80c06d4a0 msgr2=0x7fe80c06f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:50.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.327+0000 7fe826cb9700 1 --2- 172.21.15.63:0/2886534500 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe80c06d4a0 0x7fe80c06f950 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe81000b3b0 tx=0x7fe810009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:50.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.327+0000 7fe826cb9700 1 -- 172.21.15.63:0/2886534500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe820102e10 msgr2=0x7fe820115d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:50.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.327+0000 7fe826cb9700 1 --2- 172.21.15.63:0/2886534500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe820102e10 0x7fe820115d90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe80800c8c0 tx=0x7fe8080044c0 comp rx=0 tx=0).stop 2024-03-22T07:20:50.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.327+0000 7fe826cb9700 1 -- 172.21.15.63:0/2886534500 shutdown_connections 2024-03-22T07:20:50.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.327+0000 7fe826cb9700 1 --2- 172.21.15.63:0/2886534500 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe80c06d4a0 0x7fe80c06f950 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:50.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.327+0000 7fe826cb9700 1 --2- 172.21.15.63:0/2886534500 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe820102e10 0x7fe820115d90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:50.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.327+0000 7fe826cb9700 1 --2- 172.21.15.63:0/2886534500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8201037b0 0x7fe8201162d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:50.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.327+0000 7fe826cb9700 1 -- 172.21.15.63:0/2886534500 >> 172.21.15.63:0/2886534500 conn(0x7fe8200f8980 msgr2=0x7fe8200fa610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:50.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.327+0000 7fe826cb9700 1 -- 172.21.15.63:0/2886534500 shutdown_connections 2024-03-22T07:20:50.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.327+0000 7fe826cb9700 1 -- 172.21.15.63:0/2886534500 wait complete. 2024-03-22T07:20:50.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.429+0000 7f95332ba700 1 -- 172.21.15.63:0/2996826415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f952c1037a0 msgr2=0x7f952c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:50.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.429+0000 7f95332ba700 1 --2- 172.21.15.63:0/2996826415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f952c1037a0 0x7f952c0fce10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f951c004940 tx=0x7f951c009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:50.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.429+0000 7f95332ba700 1 -- 172.21.15.63:0/2996826415 shutdown_connections 2024-03-22T07:20:50.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.429+0000 7f95332ba700 1 --2- 172.21.15.63:0/2996826415 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f952c1037a0 0x7f952c0fce10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:50.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.429+0000 7f95332ba700 1 --2- 172.21.15.63:0/2996826415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f952c102e00 0x7f952c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:50.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.429+0000 7f95332ba700 1 -- 172.21.15.63:0/2996826415 >> 172.21.15.63:0/2996826415 conn(0x7f952c0f89b0 msgr2=0x7f952c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:50.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.429+0000 7f95332ba700 1 -- 172.21.15.63:0/2996826415 shutdown_connections 2024-03-22T07:20:50.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.429+0000 7f95332ba700 1 -- 172.21.15.63:0/2996826415 wait complete. 2024-03-22T07:20:50.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.429+0000 7f95332ba700 1 Processor -- start 2024-03-22T07:20:50.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.429+0000 7f95332ba700 1 -- start start 2024-03-22T07:20:50.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f95332ba700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f952c102e00 0x7f952c1139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:50.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f95332ba700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f952c1037a0 0x7f952c113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:50.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f95332ba700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c114530 con 0x7f952c1037a0 2024-03-22T07:20:50.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f95332ba700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c10db50 con 0x7f952c102e00 2024-03-22T07:20:50.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f9530855700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f952c1037a0 0x7f952c113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:50.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f9530855700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f952c1037a0 0x7f952c113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41332/0 (socket says 172.21.15.63:41332) 2024-03-22T07:20:50.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f9530855700 1 -- 172.21.15.63:0/3343638086 learned_addr learned my addr 172.21.15.63:0/3343638086 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:50.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f9531056700 1 --2- 172.21.15.63:0/3343638086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f952c102e00 0x7f952c1139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:50.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f9531056700 1 -- 172.21.15.63:0/3343638086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f952c102e00 msgr2=0x7f952c1139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-22T07:20:50.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f9531056700 1 -- 172.21.15.63:0/3343638086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f952c102e00 msgr2=0x7f952c1139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T07:20:50.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f9531056700 1 --2- 172.21.15.63:0/3343638086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f952c102e00 0x7f952c1139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-22T07:20:50.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f9531056700 1 --2- 172.21.15.63:0/3343638086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f952c102e00 0x7f952c1139a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:20:50.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f9530855700 1 -- 172.21.15.63:0/3343638086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f952c102e00 msgr2=0x7f952c1139a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:20:50.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f9530855700 1 --2- 172.21.15.63:0/3343638086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f952c102e00 0x7f952c1139a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:50.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f9530855700 1 -- 172.21.15.63:0/3343638086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c005040 con 0x7f952c1037a0 2024-03-22T07:20:50.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f9530855700 1 --2- 172.21.15.63:0/3343638086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f952c1037a0 0x7f952c113ee0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f951c009100 tx=0x7f951c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:50.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f9521ffb700 1 -- 172.21.15.63:0/3343638086 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f951c00e040 con 0x7f952c1037a0 2024-03-22T07:20:50.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f95332ba700 1 -- 172.21.15.63:0/3343638086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952c10ddd0 con 0x7f952c1037a0 2024-03-22T07:20:50.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f9521ffb700 1 -- 172.21.15.63:0/3343638086 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f951c004290 con 0x7f952c1037a0 2024-03-22T07:20:50.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.430+0000 7f9521ffb700 1 -- 172.21.15.63:0/3343638086 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f951c01c3f0 con 0x7f952c1037a0 2024-03-22T07:20:50.433 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.431+0000 7f95332ba700 1 -- 172.21.15.63:0/3343638086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952c10e390 con 0x7f952c1037a0 2024-03-22T07:20:50.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.431+0000 7f95332ba700 1 -- 172.21.15.63:0/3343638086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952c1052d0 con 0x7f952c1037a0 2024-03-22T07:20:50.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.432+0000 7f9521ffb700 1 -- 172.21.15.63:0/3343638086 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f951c013790 con 0x7f952c1037a0 2024-03-22T07:20:50.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.432+0000 7f9521ffb700 1 --2- 172.21.15.63:0/3343638086 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f951806d4b0 0x7f951806f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:50.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.433+0000 7f9531056700 1 --2- 172.21.15.63:0/3343638086 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f951806d4b0 0x7f951806f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:50.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.433+0000 7f9521ffb700 1 -- 172.21.15.63:0/3343638086 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f951c08ddf0 con 0x7f952c1037a0 2024-03-22T07:20:50.434 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.433+0000 7f9531056700 1 --2- 172.21.15.63:0/3343638086 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f951806d4b0 0x7f951806f960 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9528009ce0 tx=0x7f9528009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:50.437 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.437+0000 7f9521ffb700 1 -- 172.21.15.63:0/3343638086 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f951c057730 con 0x7f952c1037a0 2024-03-22T07:20:50.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.724+0000 7f95332ba700 1 -- 172.21.15.63:0/3343638086 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f952c10eff0 con 0x7f952c1037a0 2024-03-22T07:20:50.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.724+0000 7f9521ffb700 1 -- 172.21.15.63:0/3343638086 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f951c013420 con 0x7f952c1037a0 2024-03-22T07:20:50.724 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:20:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:20:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:20:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:20:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:20:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:20:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:20:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:20:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:20:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:20:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:20:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:20:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:20:50.725 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:20:50.726 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 10, 2024-03-22T07:20:50.726 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-22T07:20:50.726 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:20:50.726 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:20:50.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.726+0000 7f95332ba700 1 -- 172.21.15.63:0/3343638086 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f951806d4b0 msgr2=0x7f951806f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.726+0000 7f95332ba700 1 --2- 172.21.15.63:0/3343638086 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f951806d4b0 0x7f951806f960 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9528009ce0 tx=0x7f9528009f70 comp rx=0 tx=0).stop 2024-03-22T07:20:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.726+0000 7f95332ba700 1 -- 172.21.15.63:0/3343638086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f952c1037a0 msgr2=0x7f952c113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.726+0000 7f95332ba700 1 --2- 172.21.15.63:0/3343638086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f952c1037a0 0x7f952c113ee0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f951c009100 tx=0x7f951c009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.726+0000 7f95332ba700 1 -- 172.21.15.63:0/3343638086 shutdown_connections 2024-03-22T07:20:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.726+0000 7f95332ba700 1 --2- 172.21.15.63:0/3343638086 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f951806d4b0 0x7f951806f960 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.726+0000 7f95332ba700 1 --2- 172.21.15.63:0/3343638086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f952c102e00 0x7f952c1139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.726+0000 7f95332ba700 1 --2- 172.21.15.63:0/3343638086 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f952c1037a0 0x7f952c113ee0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.726+0000 7f95332ba700 1 -- 172.21.15.63:0/3343638086 >> 172.21.15.63:0/3343638086 conn(0x7f952c0f89b0 msgr2=0x7f952c1021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.726+0000 7f95332ba700 1 -- 172.21.15.63:0/3343638086 shutdown_connections 2024-03-22T07:20:50.727 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.727+0000 7f95332ba700 1 -- 172.21.15.63:0/3343638086 wait complete. 2024-03-22T07:20:50.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.830+0000 7f624321b700 1 -- 172.21.15.63:0/3472072738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c101d20 msgr2=0x7f623c1020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:50.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.830+0000 7f624321b700 1 --2- 172.21.15.63:0/3472072738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c101d20 0x7f623c1020f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6230004940 tx=0x7f6230009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:50.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.831+0000 7f624321b700 1 -- 172.21.15.63:0/3472072738 shutdown_connections 2024-03-22T07:20:50.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.831+0000 7f624321b700 1 --2- 172.21.15.63:0/3472072738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f623c1026c0 0x7f623c0fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:50.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.831+0000 7f624321b700 1 --2- 172.21.15.63:0/3472072738 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c101d20 0x7f623c1020f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:50.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.831+0000 7f624321b700 1 -- 172.21.15.63:0/3472072738 >> 172.21.15.63:0/3472072738 conn(0x7f623c0f7870 msgr2=0x7f623c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:50.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.831+0000 7f624321b700 1 -- 172.21.15.63:0/3472072738 shutdown_connections 2024-03-22T07:20:50.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.831+0000 7f624321b700 1 -- 172.21.15.63:0/3472072738 wait complete. 2024-03-22T07:20:50.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.831+0000 7f624321b700 1 Processor -- start 2024-03-22T07:20:50.832 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.831+0000 7f624321b700 1 -- start start 2024-03-22T07:20:50.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.831+0000 7f624321b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c101d20 0x7f623c115be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.831+0000 7f624321b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f623c1026c0 0x7f623c116120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.831+0000 7f624321b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c116770 con 0x7f623c101d20 2024-03-22T07:20:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.831+0000 7f624321b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c10fd50 con 0x7f623c1026c0 2024-03-22T07:20:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.832+0000 7f6240fb7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c101d20 0x7f623c115be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.832+0000 7f6240fb7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c101d20 0x7f623c115be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41354/0 (socket says 172.21.15.63:41354) 2024-03-22T07:20:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.832+0000 7f6240fb7700 1 -- 172.21.15.63:0/3466608170 learned_addr learned my addr 172.21.15.63:0/3466608170 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.832+0000 7f623bfff700 1 --2- 172.21.15.63:0/3466608170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f623c1026c0 0x7f623c116120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.832+0000 7f623bfff700 1 -- 172.21.15.63:0/3466608170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f623c1026c0 msgr2=0x7f623c116120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-22T07:20:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.832+0000 7f623bfff700 1 -- 172.21.15.63:0/3466608170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f623c1026c0 msgr2=0x7f623c116120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T07:20:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.832+0000 7f623bfff700 1 --2- 172.21.15.63:0/3466608170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f623c1026c0 0x7f623c116120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-22T07:20:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.832+0000 7f6240fb7700 1 -- 172.21.15.63:0/3466608170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f623c1026c0 msgr2=0x7f623c116120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:50.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.832+0000 7f623bfff700 1 --2- 172.21.15.63:0/3466608170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f623c1026c0 0x7f623c116120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:20:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.832+0000 7f6240fb7700 1 --2- 172.21.15.63:0/3466608170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f623c1026c0 0x7f623c116120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.832+0000 7f6240fb7700 1 -- 172.21.15.63:0/3466608170 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6230005040 con 0x7f623c101d20 2024-03-22T07:20:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.832+0000 7f6240fb7700 1 --2- 172.21.15.63:0/3466608170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c101d20 0x7f623c115be0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f623000b3b0 tx=0x7f6230009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.832+0000 7f62397fa700 1 -- 172.21.15.63:0/3466608170 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f623000e040 con 0x7f623c101d20 2024-03-22T07:20:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.832+0000 7f624321b700 1 -- 172.21.15.63:0/3466608170 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623c10ffd0 con 0x7f623c101d20 2024-03-22T07:20:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.832+0000 7f62397fa700 1 -- 172.21.15.63:0/3466608170 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f623000bbc0 con 0x7f623c101d20 2024-03-22T07:20:50.835 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.832+0000 7f62397fa700 1 -- 172.21.15.63:0/3466608170 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f623001c3f0 con 0x7f623c101d20 2024-03-22T07:20:50.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.832+0000 7f624321b700 1 -- 172.21.15.63:0/3466608170 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623c110590 con 0x7f623c101d20 2024-03-22T07:20:50.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.833+0000 7f624321b700 1 -- 172.21.15.63:0/3466608170 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623c0fd470 con 0x7f623c101d20 2024-03-22T07:20:50.836 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.834+0000 7f62397fa700 1 -- 172.21.15.63:0/3466608170 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f623000bd30 con 0x7f623c101d20 2024-03-22T07:20:50.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.834+0000 7f62397fa700 1 --2- 172.21.15.63:0/3466608170 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f622c06d4b0 0x7f622c06f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:50.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.835+0000 7f623bfff700 1 --2- 172.21.15.63:0/3466608170 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f622c06d4b0 0x7f622c06f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:50.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.835+0000 7f62397fa700 1 -- 172.21.15.63:0/3466608170 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f623008de30 con 0x7f623c101d20 2024-03-22T07:20:50.837 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.835+0000 7f623bfff700 1 --2- 172.21.15.63:0/3466608170 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f622c06d4b0 0x7f622c06f960 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f623c111480 tx=0x7f6228009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:50.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:50.839+0000 7f62397fa700 1 -- 172.21.15.63:0/3466608170 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f623005b220 con 0x7f623c101d20 2024-03-22T07:20:51.019 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:50 smithi069 ceph-mon[83631]: pgmap v12: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-22T07:20:51.019 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:50 smithi069 ceph-mon[83631]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:51.019 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:50 smithi069 ceph-mon[83631]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:51.020 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:50 smithi069 ceph-mon[83631]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:51.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.089+0000 7f624321b700 1 -- 172.21.15.63:0/3466608170 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f623c108e40 con 0x7f623c101d20 2024-03-22T07:20:51.089 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:50 smithi063 ceph-mon[112784]: pgmap v12: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-22T07:20:51.089 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:50 smithi063 ceph-mon[112784]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:51.089 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:50 smithi063 ceph-mon[112784]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:51.089 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:50 smithi063 ceph-mon[112784]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:51.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.089+0000 7f62397fa700 1 -- 172.21.15.63:0/3466608170 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1868 (secure 0 0 0) 0x7f623005adb0 con 0x7f623c101d20 2024-03-22T07:20:51.090 INFO:teuthology.orchestra.run.smithi063.stdout:e24 2024-03-22T07:20:51.091 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:20:51.091 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:20:51.091 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-03-22T07:20:51.091 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:20:51.091 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:20:51.091 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-03-22T07:20:51.091 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 24 2024-03-22T07:20:51.091 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-22T07:20:51.091 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-03-22T07:13:29.602905+0000 2024-03-22T07:20:51.091 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-03-22T07:20:24.253589+0000 2024-03-22T07:20:51.091 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-03-22T07:20:51.091 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-03-22T07:20:51.092 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-03-22T07:20:51.092 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-03-22T07:20:51.092 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-03-22T07:20:51.092 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-03-22T07:20:51.092 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-03-22T07:20:51.092 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-03-22T07:20:51.092 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-03-22T07:20:51.092 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:20:51.092 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 2 2024-03-22T07:20:51.092 INFO:teuthology.orchestra.run.smithi063.stdout:in 0,1 2024-03-22T07:20:51.092 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=14480,1=24285} 2024-03-22T07:20:51.092 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-03-22T07:20:51.092 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-03-22T07:20:51.092 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-03-22T07:20:51.092 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-03-22T07:20:51.093 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-03-22T07:20:51.093 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-03-22T07:20:51.093 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-03-22T07:20:51.093 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-03-22T07:20:51.093 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-03-22T07:20:51.093 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.xeelpk{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:20:51.093 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.oytapz{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3211595802,v1:172.21.15.63:6827/3211595802] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:20:51.093 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.wwtsjn{1:24285} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.69:6824/3798247699,v1:172.21.15.69:6825/3798247699] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:20:51.093 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.cijxah{1:24301} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.69:6826/3380719311,v1:172.21.15.69:6827/3380719311] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:20:51.093 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:20:51.093 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:20:51.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.091+0000 7f624321b700 1 -- 172.21.15.63:0/3466608170 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f622c06d4b0 msgr2=0x7f622c06f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:51.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.091+0000 7f624321b700 1 --2- 172.21.15.63:0/3466608170 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f622c06d4b0 0x7f622c06f960 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f623c111480 tx=0x7f6228009f70 comp rx=0 tx=0).stop 2024-03-22T07:20:51.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.091+0000 7f624321b700 1 -- 172.21.15.63:0/3466608170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c101d20 msgr2=0x7f623c115be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:51.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.091+0000 7f624321b700 1 --2- 172.21.15.63:0/3466608170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c101d20 0x7f623c115be0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f623000b3b0 tx=0x7f6230009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:51.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.091+0000 7f624321b700 1 -- 172.21.15.63:0/3466608170 shutdown_connections 2024-03-22T07:20:51.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.091+0000 7f624321b700 1 --2- 172.21.15.63:0/3466608170 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f622c06d4b0 0x7f622c06f960 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:51.094 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.091+0000 7f624321b700 1 --2- 172.21.15.63:0/3466608170 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f623c101d20 0x7f623c115be0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:51.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.091+0000 7f624321b700 1 --2- 172.21.15.63:0/3466608170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f623c1026c0 0x7f623c116120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:51.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.091+0000 7f624321b700 1 -- 172.21.15.63:0/3466608170 >> 172.21.15.63:0/3466608170 conn(0x7f623c0f7870 msgr2=0x7f623c0f8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:51.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.092+0000 7f624321b700 1 -- 172.21.15.63:0/3466608170 shutdown_connections 2024-03-22T07:20:51.095 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.092+0000 7f624321b700 1 -- 172.21.15.63:0/3466608170 wait complete. 2024-03-22T07:20:51.095 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 24 2024-03-22T07:20:51.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.195+0000 7fd397a0a700 1 -- 172.21.15.63:0/1435017162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd390102e00 msgr2=0x7fd3901031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:51.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.195+0000 7fd397a0a700 1 --2- 172.21.15.63:0/1435017162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd390102e00 0x7fd3901031d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd380004940 tx=0x7fd380009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:51.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.195+0000 7fd397a0a700 1 -- 172.21.15.63:0/1435017162 shutdown_connections 2024-03-22T07:20:51.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.195+0000 7fd397a0a700 1 --2- 172.21.15.63:0/1435017162 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3901037a0 0x7fd3900fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:51.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.195+0000 7fd397a0a700 1 --2- 172.21.15.63:0/1435017162 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd390102e00 0x7fd3901031d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:51.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.195+0000 7fd397a0a700 1 -- 172.21.15.63:0/1435017162 >> 172.21.15.63:0/1435017162 conn(0x7fd3900f89b0 msgr2=0x7fd3900fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:51.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.195+0000 7fd397a0a700 1 -- 172.21.15.63:0/1435017162 shutdown_connections 2024-03-22T07:20:51.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.195+0000 7fd397a0a700 1 -- 172.21.15.63:0/1435017162 wait complete. 2024-03-22T07:20:51.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.196+0000 7fd397a0a700 1 Processor -- start 2024-03-22T07:20:51.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.196+0000 7fd397a0a700 1 -- start start 2024-03-22T07:20:51.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.196+0000 7fd397a0a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd390102e00 0x7fd390113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.196+0000 7fd397a0a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3901037a0 0x7fd390113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.196+0000 7fd397a0a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd390114510 con 0x7fd3901037a0 2024-03-22T07:20:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.196+0000 7fd397a0a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39010db10 con 0x7fd390102e00 2024-03-22T07:20:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.196+0000 7fd394fa5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3901037a0 0x7fd390113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.196+0000 7fd394fa5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3901037a0 0x7fd390113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41386/0 (socket says 172.21.15.63:41386) 2024-03-22T07:20:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.196+0000 7fd394fa5700 1 -- 172.21.15.63:0/3091280351 learned_addr learned my addr 172.21.15.63:0/3091280351 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.196+0000 7fd3957a6700 1 --2- 172.21.15.63:0/3091280351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd390102e00 0x7fd390113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.196+0000 7fd3957a6700 1 -- 172.21.15.63:0/3091280351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd390102e00 msgr2=0x7fd390113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-22T07:20:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.196+0000 7fd3957a6700 1 -- 172.21.15.63:0/3091280351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd390102e00 msgr2=0x7fd390113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T07:20:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.196+0000 7fd3957a6700 1 --2- 172.21.15.63:0/3091280351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd390102e00 0x7fd390113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-22T07:20:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.196+0000 7fd3957a6700 1 --2- 172.21.15.63:0/3091280351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd390102e00 0x7fd390113980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:20:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.196+0000 7fd394fa5700 1 -- 172.21.15.63:0/3091280351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd390102e00 msgr2=0x7fd390113980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:20:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.196+0000 7fd394fa5700 1 --2- 172.21.15.63:0/3091280351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd390102e00 0x7fd390113980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:51.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.196+0000 7fd394fa5700 1 -- 172.21.15.63:0/3091280351 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd380005040 con 0x7fd3901037a0 2024-03-22T07:20:51.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.197+0000 7fd394fa5700 1 --2- 172.21.15.63:0/3091280351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3901037a0 0x7fd390113ec0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd38c002a80 tx=0x7fd38c004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:51.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.197+0000 7fd385ffb700 1 -- 172.21.15.63:0/3091280351 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd38c007cc0 con 0x7fd3901037a0 2024-03-22T07:20:51.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.197+0000 7fd385ffb700 1 -- 172.21.15.63:0/3091280351 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd38c00f040 con 0x7fd3901037a0 2024-03-22T07:20:51.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.197+0000 7fd397a0a700 1 -- 172.21.15.63:0/3091280351 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39010ddf0 con 0x7fd3901037a0 2024-03-22T07:20:51.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.197+0000 7fd385ffb700 1 -- 172.21.15.63:0/3091280351 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd38c00db30 con 0x7fd3901037a0 2024-03-22T07:20:51.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.197+0000 7fd397a0a700 1 -- 172.21.15.63:0/3091280351 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39010e3b0 con 0x7fd3901037a0 2024-03-22T07:20:51.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.198+0000 7fd397a0a700 1 -- 172.21.15.63:0/3091280351 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd390100110 con 0x7fd3901037a0 2024-03-22T07:20:51.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.199+0000 7fd385ffb700 1 -- 172.21.15.63:0/3091280351 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7fd38c00dcd0 con 0x7fd3901037a0 2024-03-22T07:20:51.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.199+0000 7fd385ffb700 1 --2- 172.21.15.63:0/3091280351 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd37c06d4b0 0x7fd37c06f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:51.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.199+0000 7fd385ffb700 1 -- 172.21.15.63:0/3091280351 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd38c08c230 con 0x7fd3901037a0 2024-03-22T07:20:51.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.199+0000 7fd3957a6700 1 --2- 172.21.15.63:0/3091280351 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd37c06d4b0 0x7fd37c06f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:51.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.199+0000 7fd3957a6700 1 --2- 172.21.15.63:0/3091280351 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd37c06d4b0 0x7fd37c06f960 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd380004f90 tx=0x7fd380009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:51.203 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.204+0000 7fd385ffb700 1 -- 172.21.15.63:0/3091280351 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd38c059620 con 0x7fd3901037a0 2024-03-22T07:20:51.426 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.426+0000 7fd397a0a700 1 -- 172.21.15.63:0/3091280351 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd39010f040 con 0x7fd37c06d4b0 2024-03-22T07:20:51.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.428+0000 7fd385ffb700 1 -- 172.21.15.63:0/3091280351 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fd39010f040 con 0x7fd37c06d4b0 2024-03-22T07:20:51.428 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:20:51.428 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:20:51.428 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:20:51.428 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:20:51.428 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-03-22T07:20:51.428 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr" 2024-03-22T07:20:51.428 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-03-22T07:20:51.428 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "3/21 daemons upgraded", 2024-03-22T07:20:51.428 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mon daemons", 2024-03-22T07:20:51.428 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:20:51.428 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:20:51.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.430+0000 7fd397a0a700 1 -- 172.21.15.63:0/3091280351 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd37c06d4b0 msgr2=0x7fd37c06f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:51.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.430+0000 7fd397a0a700 1 --2- 172.21.15.63:0/3091280351 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd37c06d4b0 0x7fd37c06f960 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd380004f90 tx=0x7fd380009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:51.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.430+0000 7fd397a0a700 1 -- 172.21.15.63:0/3091280351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3901037a0 msgr2=0x7fd390113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:51.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.430+0000 7fd397a0a700 1 --2- 172.21.15.63:0/3091280351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3901037a0 0x7fd390113ec0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd38c002a80 tx=0x7fd38c004630 comp rx=0 tx=0).stop 2024-03-22T07:20:51.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.430+0000 7fd397a0a700 1 -- 172.21.15.63:0/3091280351 shutdown_connections 2024-03-22T07:20:51.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.430+0000 7fd397a0a700 1 --2- 172.21.15.63:0/3091280351 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd37c06d4b0 0x7fd37c06f960 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:51.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.430+0000 7fd397a0a700 1 --2- 172.21.15.63:0/3091280351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd390102e00 0x7fd390113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:51.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.430+0000 7fd397a0a700 1 --2- 172.21.15.63:0/3091280351 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd3901037a0 0x7fd390113ec0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:51.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.430+0000 7fd397a0a700 1 -- 172.21.15.63:0/3091280351 >> 172.21.15.63:0/3091280351 conn(0x7fd3900f89b0 msgr2=0x7fd3901073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:51.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.430+0000 7fd397a0a700 1 -- 172.21.15.63:0/3091280351 shutdown_connections 2024-03-22T07:20:51.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.430+0000 7fd397a0a700 1 -- 172.21.15.63:0/3091280351 wait complete. 2024-03-22T07:20:51.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.533+0000 7f20da5c4700 1 -- 172.21.15.63:0/3896300853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20d4102e10 msgr2=0x7f20d41031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:51.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.533+0000 7f20da5c4700 1 --2- 172.21.15.63:0/3896300853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20d4102e10 0x7f20d41031e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f20c4004940 tx=0x7f20c4009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:51.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.533+0000 7f20da5c4700 1 -- 172.21.15.63:0/3896300853 shutdown_connections 2024-03-22T07:20:51.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.533+0000 7f20da5c4700 1 --2- 172.21.15.63:0/3896300853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20d41037b0 0x7f20d40fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:51.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.533+0000 7f20da5c4700 1 --2- 172.21.15.63:0/3896300853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20d4102e10 0x7f20d41031e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:51.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.533+0000 7f20da5c4700 1 -- 172.21.15.63:0/3896300853 >> 172.21.15.63:0/3896300853 conn(0x7f20d40f8980 msgr2=0x7f20d40fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:51.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.533+0000 7f20da5c4700 1 -- 172.21.15.63:0/3896300853 shutdown_connections 2024-03-22T07:20:51.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.533+0000 7f20da5c4700 1 -- 172.21.15.63:0/3896300853 wait complete. 2024-03-22T07:20:51.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.534+0000 7f20da5c4700 1 Processor -- start 2024-03-22T07:20:51.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.534+0000 7f20da5c4700 1 -- start start 2024-03-22T07:20:51.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.534+0000 7f20da5c4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20d4102e10 0x7f20d4113c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:51.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.534+0000 7f20da5c4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20d41037b0 0x7f20d4114140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:51.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.534+0000 7f20da5c4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d4114790 con 0x7f20d41037b0 2024-03-22T07:20:51.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.534+0000 7f20da5c4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d410db80 con 0x7f20d4102e10 2024-03-22T07:20:51.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.534+0000 7f20d37fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20d41037b0 0x7f20d4114140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:51.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.534+0000 7f20d37fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20d41037b0 0x7f20d4114140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41400/0 (socket says 172.21.15.63:41400) 2024-03-22T07:20:51.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.534+0000 7f20d37fe700 1 -- 172.21.15.63:0/2433167327 learned_addr learned my addr 172.21.15.63:0/2433167327 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:20:51.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.534+0000 7f20d3fff700 1 --2- 172.21.15.63:0/2433167327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20d4102e10 0x7f20d4113c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:51.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.534+0000 7f20d3fff700 1 -- 172.21.15.63:0/2433167327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20d4102e10 msgr2=0x7f20d4113c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-22T07:20:51.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.534+0000 7f20d3fff700 1 -- 172.21.15.63:0/2433167327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20d4102e10 msgr2=0x7f20d4113c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-22T07:20:51.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.534+0000 7f20d3fff700 1 --2- 172.21.15.63:0/2433167327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20d4102e10 0x7f20d4113c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-03-22T07:20:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.534+0000 7f20d3fff700 1 --2- 172.21.15.63:0/2433167327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20d4102e10 0x7f20d4113c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-22T07:20:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.534+0000 7f20d37fe700 1 -- 172.21.15.63:0/2433167327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20d4102e10 msgr2=0x7f20d4113c00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-22T07:20:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.534+0000 7f20d37fe700 1 --2- 172.21.15.63:0/2433167327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20d4102e10 0x7f20d4113c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.534+0000 7f20d37fe700 1 -- 172.21.15.63:0/2433167327 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c4005040 con 0x7f20d41037b0 2024-03-22T07:20:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.535+0000 7f20d37fe700 1 --2- 172.21.15.63:0/2433167327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20d41037b0 0x7f20d4114140 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f20bc002a80 tx=0x7f20bc004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.535+0000 7f20d0ff9700 1 -- 172.21.15.63:0/2433167327 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20bc007cc0 con 0x7f20d41037b0 2024-03-22T07:20:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.535+0000 7f20da5c4700 1 -- 172.21.15.63:0/2433167327 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d410de60 con 0x7f20d41037b0 2024-03-22T07:20:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.535+0000 7f20d0ff9700 1 -- 172.21.15.63:0/2433167327 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20bc00f040 con 0x7f20d41037b0 2024-03-22T07:20:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.535+0000 7f20d0ff9700 1 -- 172.21.15.63:0/2433167327 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20bc00db30 con 0x7f20d41037b0 2024-03-22T07:20:51.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.535+0000 7f20da5c4700 1 -- 172.21.15.63:0/2433167327 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d410e420 con 0x7f20d41037b0 2024-03-22T07:20:51.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.536+0000 7f20da5c4700 1 -- 172.21.15.63:0/2433167327 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d4100120 con 0x7f20d41037b0 2024-03-22T07:20:51.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.537+0000 7f20d0ff9700 1 -- 172.21.15.63:0/2433167327 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f20bc01c400 con 0x7f20d41037b0 2024-03-22T07:20:51.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.537+0000 7f20d0ff9700 1 --2- 172.21.15.63:0/2433167327 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f20c006d4a0 0x7f20c006f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:20:51.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.537+0000 7f20d0ff9700 1 -- 172.21.15.63:0/2433167327 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f20bc08c140 con 0x7f20d41037b0 2024-03-22T07:20:51.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.537+0000 7f20d3fff700 1 --2- 172.21.15.63:0/2433167327 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f20c006d4a0 0x7f20c006f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:20:51.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.538+0000 7f20d3fff700 1 --2- 172.21.15.63:0/2433167327 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f20c006d4a0 0x7f20c006f950 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f20c4004f90 tx=0x7f20c4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:20:51.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.542+0000 7f20d0ff9700 1 -- 172.21.15.63:0/2433167327 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f20bc0595b0 con 0x7f20d41037b0 2024-03-22T07:20:51.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.831+0000 7f20da5c4700 1 -- 172.21.15.63:0/2433167327 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f20d410f1e0 con 0x7f20d41037b0 2024-03-22T07:20:51.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.831+0000 7f20d0ff9700 1 -- 172.21.15.63:0/2433167327 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f20bc059140 con 0x7f20d41037b0 2024-03-22T07:20:51.831 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-03-22T07:20:51.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.833+0000 7f20da5c4700 1 -- 172.21.15.63:0/2433167327 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f20c006d4a0 msgr2=0x7f20c006f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:51.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.833+0000 7f20da5c4700 1 --2- 172.21.15.63:0/2433167327 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f20c006d4a0 0x7f20c006f950 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f20c4004f90 tx=0x7f20c4009e30 comp rx=0 tx=0).stop 2024-03-22T07:20:51.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.833+0000 7f20da5c4700 1 -- 172.21.15.63:0/2433167327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20d41037b0 msgr2=0x7f20d4114140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:20:51.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.833+0000 7f20da5c4700 1 --2- 172.21.15.63:0/2433167327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20d41037b0 0x7f20d4114140 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f20bc002a80 tx=0x7f20bc004630 comp rx=0 tx=0).stop 2024-03-22T07:20:51.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.833+0000 7f20da5c4700 1 -- 172.21.15.63:0/2433167327 shutdown_connections 2024-03-22T07:20:51.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.833+0000 7f20da5c4700 1 --2- 172.21.15.63:0/2433167327 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f20c006d4a0 0x7f20c006f950 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:51.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.833+0000 7f20da5c4700 1 --2- 172.21.15.63:0/2433167327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20d4102e10 0x7f20d4113c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:51.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.833+0000 7f20da5c4700 1 --2- 172.21.15.63:0/2433167327 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20d41037b0 0x7f20d4114140 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:20:51.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.833+0000 7f20da5c4700 1 -- 172.21.15.63:0/2433167327 >> 172.21.15.63:0/2433167327 conn(0x7f20d40f8980 msgr2=0x7f20d4107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:20:51.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.833+0000 7f20da5c4700 1 -- 172.21.15.63:0/2433167327 shutdown_connections 2024-03-22T07:20:51.834 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:20:51.833+0000 7f20da5c4700 1 -- 172.21.15.63:0/2433167327 wait complete. 2024-03-22T07:20:52.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:51 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3343638086' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:20:52.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:51 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3466608170' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:20:52.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:51 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:52.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:51 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:52.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:51 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3343638086' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:20:52.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:51 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3466608170' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:20:52.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:51 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:52.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:51 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:53.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:52 smithi069 ceph-mon[83631]: pgmap v13: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:53.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:52 smithi069 ceph-mon[83631]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:53.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:52 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2433167327' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:20:53.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:52 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:53.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:52 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:53.034 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:52 smithi063 ceph-mon[112784]: pgmap v13: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:53.035 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:52 smithi063 ceph-mon[112784]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:20:53.035 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:52 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2433167327' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:20:53.035 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:52 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:53.035 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:52 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:55.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:55 smithi063 ceph-mon[112784]: pgmap v14: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:20:55.463 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:55 smithi069 ceph-mon[83631]: pgmap v14: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:20:57.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:57 smithi069 ceph-mon[83631]: pgmap v15: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:57.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:57 smithi069 ceph-mon[83631]: Detected new or changed devices on smithi069 2024-03-22T07:20:57.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:57 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:57.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:57 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:57.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:57 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:20:57.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:57 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:20:57.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:57 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:57.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:57 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:20:57.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:57 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:20:57.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:57 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:20:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:57 smithi063 ceph-mon[112784]: pgmap v15: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:20:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:57 smithi063 ceph-mon[112784]: Detected new or changed devices on smithi069 2024-03-22T07:20:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:57 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:57 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:57 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:20:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:57 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:20:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:57 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:57 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:20:57.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:57 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:20:57.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:57 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:20:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:58 smithi069 ceph-mon[83631]: Reconfiguring mon.smithi063 (monmap changed)... 2024-03-22T07:20:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:58 smithi069 ceph-mon[83631]: Reconfiguring daemon mon.smithi063 on smithi063 2024-03-22T07:20:58.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:58 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:58.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:58 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:58.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:58 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi063.mnxayh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:20:58.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:58 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:20:58.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:58 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:20:58.620 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:58 smithi063 ceph-mon[112784]: Reconfiguring mon.smithi063 (monmap changed)... 2024-03-22T07:20:58.621 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:58 smithi063 ceph-mon[112784]: Reconfiguring daemon mon.smithi063 on smithi063 2024-03-22T07:20:58.621 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:58 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:58.621 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:58 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:20:58.621 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:58 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi063.mnxayh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:20:58.621 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:58 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:20:58.621 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:58 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:20:59.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:59 smithi069 ceph-mon[83631]: pgmap v16: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:20:59.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:59 smithi069 ceph-mon[83631]: Reconfiguring mgr.smithi063.mnxayh (monmap changed)... 2024-03-22T07:20:59.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:20:59 smithi069 ceph-mon[83631]: Reconfiguring daemon mgr.smithi063.mnxayh on smithi063 2024-03-22T07:20:59.600 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:59 smithi063 ceph-mon[112784]: pgmap v16: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:20:59.600 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:59 smithi063 ceph-mon[112784]: Reconfiguring mgr.smithi063.mnxayh (monmap changed)... 2024-03-22T07:20:59.600 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:20:59 smithi063 ceph-mon[112784]: Reconfiguring daemon mgr.smithi063.mnxayh on smithi063 2024-03-22T07:21:00.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:00 smithi063 ceph-mon[112784]: pgmap v17: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:21:00.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:00 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:00.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:00 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:00.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:00 smithi063 ceph-mon[112784]: Reconfiguring crash.smithi063 (monmap changed)... 2024-03-22T07:21:00.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:00 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:21:00.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:00 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:00.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:00 smithi063 ceph-mon[112784]: Reconfiguring daemon crash.smithi063 on smithi063 2024-03-22T07:21:01.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:00 smithi069 ceph-mon[83631]: pgmap v17: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:21:01.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:00 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:01.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:00 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:01.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:00 smithi069 ceph-mon[83631]: Reconfiguring crash.smithi063 (monmap changed)... 2024-03-22T07:21:01.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:00 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:21:01.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:00 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:01.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:00 smithi069 ceph-mon[83631]: Reconfiguring daemon crash.smithi063 on smithi063 2024-03-22T07:21:02.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:02.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:02.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:02 smithi063 ceph-mon[112784]: Reconfiguring osd.0 (monmap changed)... 2024-03-22T07:21:02.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T07:21:02.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:02.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:02 smithi063 ceph-mon[112784]: Reconfiguring daemon osd.0 on smithi063 2024-03-22T07:21:02.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:02 smithi063 ceph-mon[112784]: pgmap v18: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:02.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:21:02.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:02.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:02.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:02 smithi069 ceph-mon[83631]: Reconfiguring osd.0 (monmap changed)... 2024-03-22T07:21:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T07:21:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:02 smithi069 ceph-mon[83631]: Reconfiguring daemon osd.0 on smithi063 2024-03-22T07:21:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:02 smithi069 ceph-mon[83631]: pgmap v18: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:21:04.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:04 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:04.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:04 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:04.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:04 smithi063 ceph-mon[112784]: Reconfiguring osd.1 (monmap changed)... 2024-03-22T07:21:04.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:04 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T07:21:04.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:04 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:04.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:04 smithi063 ceph-mon[112784]: Reconfiguring daemon osd.1 on smithi063 2024-03-22T07:21:04.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:04 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:04.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:04 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:04.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:04 smithi069 ceph-mon[83631]: Reconfiguring osd.1 (monmap changed)... 2024-03-22T07:21:04.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:04 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T07:21:04.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:04 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:04.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:04 smithi069 ceph-mon[83631]: Reconfiguring daemon osd.1 on smithi063 2024-03-22T07:21:05.399 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:05 smithi063 ceph-mon[112784]: pgmap v19: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:05.399 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:05 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:05.399 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:05 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:05.399 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:05 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T07:21:05.399 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:05 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:05.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:05 smithi069 ceph-mon[83631]: pgmap v19: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:05.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:05 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:05.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:05 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:05.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:05 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T07:21:05.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:05 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:06.448 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:06 smithi063 ceph-mon[112784]: Reconfiguring osd.2 (monmap changed)... 2024-03-22T07:21:06.448 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:06 smithi063 ceph-mon[112784]: Reconfiguring daemon osd.2 on smithi063 2024-03-22T07:21:06.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:06 smithi069 ceph-mon[83631]: Reconfiguring osd.2 (monmap changed)... 2024-03-22T07:21:06.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:06 smithi069 ceph-mon[83631]: Reconfiguring daemon osd.2 on smithi063 2024-03-22T07:21:07.154 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:07 smithi069 ceph-mon[83631]: pgmap v20: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:07.435 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:07 smithi063 ceph-mon[112784]: pgmap v20: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:07.435 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:07 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:07.435 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:07 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:07.435 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:07 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.oytapz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:21:07.435 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:07 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:07.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:07 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:07.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:07 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:07.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:07 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.oytapz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:21:07.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:07 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:08.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:08 smithi069 ceph-mon[83631]: Reconfiguring mds.cephfs.smithi063.oytapz (monmap changed)... 2024-03-22T07:21:08.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:08 smithi069 ceph-mon[83631]: Reconfiguring daemon mds.cephfs.smithi063.oytapz on smithi063 2024-03-22T07:21:08.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:08 smithi063 ceph-mon[112784]: Reconfiguring mds.cephfs.smithi063.oytapz (monmap changed)... 2024-03-22T07:21:08.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:08 smithi063 ceph-mon[112784]: Reconfiguring daemon mds.cephfs.smithi063.oytapz on smithi063 2024-03-22T07:21:09.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:09 smithi069 ceph-mon[83631]: pgmap v21: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:09.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:09 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:09.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:09 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:09.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:09 smithi069 ceph-mon[83631]: Reconfiguring mds.cephfs.smithi063.xeelpk (monmap changed)... 2024-03-22T07:21:09.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:09 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.xeelpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:21:09.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:09 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:09.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:09 smithi069 ceph-mon[83631]: Reconfiguring daemon mds.cephfs.smithi063.xeelpk on smithi063 2024-03-22T07:21:09.600 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:09 smithi063 ceph-mon[112784]: pgmap v21: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:09.600 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:09 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:09.600 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:09 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:09.600 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:09 smithi063 ceph-mon[112784]: Reconfiguring mds.cephfs.smithi063.xeelpk (monmap changed)... 2024-03-22T07:21:09.600 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:09 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.xeelpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:21:09.601 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:09 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:09.601 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:09 smithi063 ceph-mon[112784]: Reconfiguring daemon mds.cephfs.smithi063.xeelpk on smithi063 2024-03-22T07:21:11.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:10 smithi069 ceph-mon[83631]: pgmap v22: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:21:11.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:11.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:11.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:10 smithi069 ceph-mon[83631]: Reconfiguring crash.smithi069 (monmap changed)... 2024-03-22T07:21:11.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi069", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:21:11.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:11.281 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:10 smithi069 ceph-mon[83631]: Reconfiguring daemon crash.smithi069 on smithi069 2024-03-22T07:21:11.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:10 smithi063 ceph-mon[112784]: pgmap v22: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:21:11.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:11.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:11.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:10 smithi063 ceph-mon[112784]: Reconfiguring crash.smithi069 (monmap changed)... 2024-03-22T07:21:11.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi069", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:21:11.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:11.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:10 smithi063 ceph-mon[112784]: Reconfiguring daemon crash.smithi069 on smithi069 2024-03-22T07:21:12.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:12 smithi069 ceph-mon[83631]: pgmap v23: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:12 smithi069 ceph-mon[83631]: Reconfiguring mgr.smithi069.gfziwh (monmap changed)... 2024-03-22T07:21:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi069.gfziwh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:21:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:21:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:12 smithi069 ceph-mon[83631]: Reconfiguring daemon mgr.smithi069.gfziwh on smithi069 2024-03-22T07:21:12.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:12 smithi063 ceph-mon[112784]: pgmap v23: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:12.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:12.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:12.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:12 smithi063 ceph-mon[112784]: Reconfiguring mgr.smithi069.gfziwh (monmap changed)... 2024-03-22T07:21:12.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi069.gfziwh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-22T07:21:12.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-22T07:21:12.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:12.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:12 smithi063 ceph-mon[112784]: Reconfiguring daemon mgr.smithi069.gfziwh on smithi069 2024-03-22T07:21:14.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:14 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:14.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:14 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:14.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:14 smithi069 ceph-mon[83631]: Reconfiguring mon.smithi069 (monmap changed)... 2024-03-22T07:21:14.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:14 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:21:14.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:14 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:21:14.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:14 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:14.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:14 smithi069 ceph-mon[83631]: Reconfiguring daemon mon.smithi069 on smithi069 2024-03-22T07:21:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:14 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:14 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:14 smithi063 ceph-mon[112784]: Reconfiguring mon.smithi069 (monmap changed)... 2024-03-22T07:21:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:14 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-22T07:21:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:14 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-22T07:21:14.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:14 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:14.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:14 smithi063 ceph-mon[112784]: Reconfiguring daemon mon.smithi069 on smithi069 2024-03-22T07:21:15.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:15 smithi069 ceph-mon[83631]: pgmap v24: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:15.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:15 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:15.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:15 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:15.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:15 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T07:21:15.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:15 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:15.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:15 smithi063 ceph-mon[112784]: pgmap v24: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:15.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:15 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:15.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:15 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:15.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:15 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T07:21:15.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:15 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:16.512 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:16 smithi069 ceph-mon[83631]: Reconfiguring osd.3 (monmap changed)... 2024-03-22T07:21:16.512 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:16 smithi069 ceph-mon[83631]: Reconfiguring daemon osd.3 on smithi069 2024-03-22T07:21:16.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:16 smithi063 ceph-mon[112784]: Reconfiguring osd.3 (monmap changed)... 2024-03-22T07:21:16.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:16 smithi063 ceph-mon[112784]: Reconfiguring daemon osd.3 on smithi069 2024-03-22T07:21:17.482 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:17 smithi069 ceph-mon[83631]: pgmap v25: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:17.482 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:17 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:21:17.482 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:17 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:17.482 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:17 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:17.482 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:17 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T07:21:17.482 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:17 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:17.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:17 smithi063 ceph-mon[112784]: pgmap v25: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:17.657 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:17 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:21:17.657 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:17 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:17.657 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:17 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:17.657 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:17 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T07:21:17.658 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:17 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:18.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:18 smithi069 ceph-mon[83631]: Reconfiguring osd.4 (monmap changed)... 2024-03-22T07:21:18.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:18 smithi069 ceph-mon[83631]: Reconfiguring daemon osd.4 on smithi069 2024-03-22T07:21:18.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:18 smithi063 ceph-mon[112784]: Reconfiguring osd.4 (monmap changed)... 2024-03-22T07:21:18.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:18 smithi063 ceph-mon[112784]: Reconfiguring daemon osd.4 on smithi069 2024-03-22T07:21:19.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:19 smithi069 ceph-mon[83631]: pgmap v26: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:19.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:19 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:19.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:19 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:19.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:19 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T07:21:19.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:19 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:19.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:19 smithi063 ceph-mon[112784]: pgmap v26: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:19.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:19 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:19.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:19 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:19.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:19 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T07:21:19.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:19 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:20.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:20 smithi069 ceph-mon[83631]: Reconfiguring osd.5 (monmap changed)... 2024-03-22T07:21:20.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:20 smithi069 ceph-mon[83631]: Reconfiguring daemon osd.5 on smithi069 2024-03-22T07:21:20.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:20 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:20.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:20 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:20.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:20 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi069.wwtsjn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:21:20.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:20 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:20.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:20 smithi063 ceph-mon[112784]: Reconfiguring osd.5 (monmap changed)... 2024-03-22T07:21:20.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:20 smithi063 ceph-mon[112784]: Reconfiguring daemon osd.5 on smithi069 2024-03-22T07:21:20.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:20 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:20.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:20 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:20.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:20 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi069.wwtsjn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:21:20.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:20 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:21.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:21 smithi069 ceph-mon[83631]: pgmap v27: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:21.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:21 smithi069 ceph-mon[83631]: Reconfiguring mds.cephfs.smithi069.wwtsjn (monmap changed)... 2024-03-22T07:21:21.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:21 smithi069 ceph-mon[83631]: Reconfiguring daemon mds.cephfs.smithi069.wwtsjn on smithi069 2024-03-22T07:21:21.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:21 smithi063 ceph-mon[112784]: pgmap v27: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:21.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:21 smithi063 ceph-mon[112784]: Reconfiguring mds.cephfs.smithi069.wwtsjn (monmap changed)... 2024-03-22T07:21:21.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:21 smithi063 ceph-mon[112784]: Reconfiguring daemon mds.cephfs.smithi069.wwtsjn on smithi069 2024-03-22T07:21:21.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.950+0000 7f3c49782700 1 -- 172.21.15.63:0/2540238669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c441037a0 msgr2=0x7f3c440fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:21.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.950+0000 7f3c49782700 1 --2- 172.21.15.63:0/2540238669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c441037a0 0x7f3c440fce10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c004940 tx=0x7f3c2c009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:21.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.950+0000 7f3c49782700 1 -- 172.21.15.63:0/2540238669 shutdown_connections 2024-03-22T07:21:21.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.950+0000 7f3c49782700 1 --2- 172.21.15.63:0/2540238669 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c441037a0 0x7f3c440fce10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:21.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.950+0000 7f3c49782700 1 --2- 172.21.15.63:0/2540238669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c44102e00 0x7f3c441031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:21.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.950+0000 7f3c49782700 1 -- 172.21.15.63:0/2540238669 >> 172.21.15.63:0/2540238669 conn(0x7f3c440f89b0 msgr2=0x7f3c440fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:21.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.950+0000 7f3c49782700 1 -- 172.21.15.63:0/2540238669 shutdown_connections 2024-03-22T07:21:21.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.950+0000 7f3c49782700 1 -- 172.21.15.63:0/2540238669 wait complete. 2024-03-22T07:21:21.952 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.951+0000 7f3c49782700 1 Processor -- start 2024-03-22T07:21:21.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.951+0000 7f3c49782700 1 -- start start 2024-03-22T07:21:21.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.951+0000 7f3c49782700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c44102e00 0x7f3c44113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:21.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.951+0000 7f3c49782700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c441037a0 0x7f3c44113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:21.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.951+0000 7f3c49782700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c441144a0 con 0x7f3c441037a0 2024-03-22T07:21:21.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.951+0000 7f3c49782700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4410daf0 con 0x7f3c44102e00 2024-03-22T07:21:21.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.951+0000 7f3c427fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c441037a0 0x7f3c44113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:21.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.951+0000 7f3c427fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c441037a0 0x7f3c44113e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37096/0 (socket says 172.21.15.63:37096) 2024-03-22T07:21:21.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.952+0000 7f3c427fc700 1 -- 172.21.15.63:0/877022517 learned_addr learned my addr 172.21.15.63:0/877022517 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:21:21.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.952+0000 7f3c42ffd700 1 --2- 172.21.15.63:0/877022517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c44102e00 0x7f3c44113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:21.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.952+0000 7f3c427fc700 1 -- 172.21.15.63:0/877022517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c44102e00 msgr2=0x7f3c44113910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:21.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.952+0000 7f3c427fc700 1 --2- 172.21.15.63:0/877022517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c44102e00 0x7f3c44113910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:21.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.952+0000 7f3c427fc700 1 -- 172.21.15.63:0/877022517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c005040 con 0x7f3c441037a0 2024-03-22T07:21:21.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.952+0000 7f3c427fc700 1 --2- 172.21.15.63:0/877022517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c441037a0 0x7f3c44113e50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c000c00 tx=0x7f3c2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:21.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.952+0000 7f3c3b7fe700 1 -- 172.21.15.63:0/877022517 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c2c021070 con 0x7f3c441037a0 2024-03-22T07:21:21.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.952+0000 7f3c49782700 1 -- 172.21.15.63:0/877022517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4410dd70 con 0x7f3c441037a0 2024-03-22T07:21:21.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.952+0000 7f3c3b7fe700 1 -- 172.21.15.63:0/877022517 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c2c013c00 con 0x7f3c441037a0 2024-03-22T07:21:21.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.952+0000 7f3c3b7fe700 1 -- 172.21.15.63:0/877022517 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c2c00e070 con 0x7f3c441037a0 2024-03-22T07:21:21.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.952+0000 7f3c49782700 1 -- 172.21.15.63:0/877022517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4410e250 con 0x7f3c441037a0 2024-03-22T07:21:21.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.953+0000 7f3c49782700 1 -- 172.21.15.63:0/877022517 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c441052d0 con 0x7f3c441037a0 2024-03-22T07:21:21.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.954+0000 7f3c3b7fe700 1 -- 172.21.15.63:0/877022517 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f3c2c01ca60 con 0x7f3c441037a0 2024-03-22T07:21:21.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.954+0000 7f3c3b7fe700 1 --2- 172.21.15.63:0/877022517 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f3c3006d4b0 0x7f3c3006f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:21.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.954+0000 7f3c3b7fe700 1 -- 172.21.15.63:0/877022517 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3c2c08df90 con 0x7f3c441037a0 2024-03-22T07:21:21.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.954+0000 7f3c42ffd700 1 --2- 172.21.15.63:0/877022517 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f3c3006d4b0 0x7f3c3006f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:21.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.955+0000 7f3c42ffd700 1 --2- 172.21.15.63:0/877022517 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f3c3006d4b0 0x7f3c3006f960 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009b80 tx=0x7f3c3c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:21.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:21.959+0000 7f3c3b7fe700 1 -- 172.21.15.63:0/877022517 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3c2c05b400 con 0x7f3c441037a0 2024-03-22T07:21:22.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.181+0000 7f3c49782700 1 -- 172.21.15.63:0/877022517 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c441135e0 con 0x7f3c3006d4b0 2024-03-22T07:21:22.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.183+0000 7f3c3b7fe700 1 -- 172.21.15.63:0/877022517 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f3c441135e0 con 0x7f3c3006d4b0 2024-03-22T07:21:22.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.184+0000 7f3c49782700 1 -- 172.21.15.63:0/877022517 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f3c3006d4b0 msgr2=0x7f3c3006f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:22.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.185+0000 7f3c49782700 1 --2- 172.21.15.63:0/877022517 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f3c3006d4b0 0x7f3c3006f960 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009b80 tx=0x7f3c3c00b040 comp rx=0 tx=0).stop 2024-03-22T07:21:22.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.185+0000 7f3c49782700 1 -- 172.21.15.63:0/877022517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c441037a0 msgr2=0x7f3c44113e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:22.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.185+0000 7f3c49782700 1 --2- 172.21.15.63:0/877022517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c441037a0 0x7f3c44113e50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c000c00 tx=0x7f3c2c009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:22.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.185+0000 7f3c49782700 1 -- 172.21.15.63:0/877022517 shutdown_connections 2024-03-22T07:21:22.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.185+0000 7f3c49782700 1 --2- 172.21.15.63:0/877022517 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f3c3006d4b0 0x7f3c3006f960 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.185+0000 7f3c49782700 1 --2- 172.21.15.63:0/877022517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c44102e00 0x7f3c44113910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.185+0000 7f3c49782700 1 --2- 172.21.15.63:0/877022517 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3c441037a0 0x7f3c44113e50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.185+0000 7f3c49782700 1 -- 172.21.15.63:0/877022517 >> 172.21.15.63:0/877022517 conn(0x7f3c440f89b0 msgr2=0x7f3c441021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:22.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.185+0000 7f3c49782700 1 -- 172.21.15.63:0/877022517 shutdown_connections 2024-03-22T07:21:22.185 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.185+0000 7f3c49782700 1 -- 172.21.15.63:0/877022517 wait complete. 2024-03-22T07:21:22.195 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:21:22.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.291+0000 7fca08683700 1 -- 172.21.15.63:0/3760111647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca00102710 msgr2=0x7fca000fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:22.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.291+0000 7fca08683700 1 --2- 172.21.15.63:0/3760111647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca00102710 0x7fca000fbd80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4004940 tx=0x7fc9f4009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:22.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.291+0000 7fca08683700 1 -- 172.21.15.63:0/3760111647 shutdown_connections 2024-03-22T07:21:22.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.291+0000 7fca08683700 1 --2- 172.21.15.63:0/3760111647 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca00102710 0x7fca000fbd80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.291+0000 7fca08683700 1 --2- 172.21.15.63:0/3760111647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca00101d70 0x7fca00102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.291+0000 7fca08683700 1 -- 172.21.15.63:0/3760111647 >> 172.21.15.63:0/3760111647 conn(0x7fca00072bc0 msgr2=0x7fca00072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:22.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.292+0000 7fca08683700 1 -- 172.21.15.63:0/3760111647 shutdown_connections 2024-03-22T07:21:22.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.292+0000 7fca08683700 1 -- 172.21.15.63:0/3760111647 wait complete. 2024-03-22T07:21:22.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.292+0000 7fca08683700 1 Processor -- start 2024-03-22T07:21:22.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.292+0000 7fca08683700 1 -- start start 2024-03-22T07:21:22.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.292+0000 7fca08683700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca00101d70 0x7fca000fd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:22.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.292+0000 7fca08683700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca00102710 0x7fca000fdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:22.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.292+0000 7fca08683700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca000fe470 con 0x7fca00101d70 2024-03-22T07:21:22.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.292+0000 7fca08683700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca000fe5e0 con 0x7fca00102710 2024-03-22T07:21:22.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.292+0000 7fca0641f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca00101d70 0x7fca000fd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:22.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.292+0000 7fca0641f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca00101d70 0x7fca000fd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37114/0 (socket says 172.21.15.63:37114) 2024-03-22T07:21:22.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.293+0000 7fca0641f700 1 -- 172.21.15.63:0/448981933 learned_addr learned my addr 172.21.15.63:0/448981933 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:21:22.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.293+0000 7fca05c1e700 1 --2- 172.21.15.63:0/448981933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca00102710 0x7fca000fdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:22.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.293+0000 7fca0641f700 1 -- 172.21.15.63:0/448981933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca00102710 msgr2=0x7fca000fdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:22.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.293+0000 7fca0641f700 1 --2- 172.21.15.63:0/448981933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca00102710 0x7fca000fdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.293+0000 7fca0641f700 1 -- 172.21.15.63:0/448981933 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f4005040 con 0x7fca00101d70 2024-03-22T07:21:22.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.293+0000 7fca0641f700 1 --2- 172.21.15.63:0/448981933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca00101d70 0x7fca000fd9f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0001d80 tx=0x7fc9f0009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:22.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.293+0000 7fc9feffd700 1 -- 172.21.15.63:0/448981933 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9f0007d10 con 0x7fca00101d70 2024-03-22T07:21:22.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.293+0000 7fc9feffd700 1 -- 172.21.15.63:0/448981933 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9f0001e80 con 0x7fca00101d70 2024-03-22T07:21:22.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.293+0000 7fca08683700 1 -- 172.21.15.63:0/448981933 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca00119e40 con 0x7fca00101d70 2024-03-22T07:21:22.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.293+0000 7fc9feffd700 1 -- 172.21.15.63:0/448981933 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9f0014640 con 0x7fca00101d70 2024-03-22T07:21:22.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.293+0000 7fca08683700 1 -- 172.21.15.63:0/448981933 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0011a1f0 con 0x7fca00101d70 2024-03-22T07:21:22.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.295+0000 7fc9feffd700 1 -- 172.21.15.63:0/448981933 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7fc9f000e960 con 0x7fca00101d70 2024-03-22T07:21:22.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.295+0000 7fca08683700 1 -- 172.21.15.63:0/448981933 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca000fd4c0 con 0x7fca00101d70 2024-03-22T07:21:22.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.295+0000 7fc9feffd700 1 --2- 172.21.15.63:0/448981933 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc9e806d4a0 0x7fc9e806f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:22.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.295+0000 7fc9feffd700 1 -- 172.21.15.63:0/448981933 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc9f008c7f0 con 0x7fca00101d70 2024-03-22T07:21:22.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.295+0000 7fca05c1e700 1 --2- 172.21.15.63:0/448981933 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc9e806d4a0 0x7fc9e806f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:22.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.296+0000 7fca05c1e700 1 --2- 172.21.15.63:0/448981933 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc9e806d4a0 0x7fc9e806f950 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4004890 tx=0x7fc9f4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:22.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.300+0000 7fc9feffd700 1 -- 172.21.15.63:0/448981933 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc9f0056270 con 0x7fca00101d70 2024-03-22T07:21:22.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.522+0000 7fca08683700 1 -- 172.21.15.63:0/448981933 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca0004bed0 con 0x7fc9e806d4a0 2024-03-22T07:21:22.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.524+0000 7fc9feffd700 1 -- 172.21.15.63:0/448981933 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fca0004bed0 con 0x7fc9e806d4a0 2024-03-22T07:21:22.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.526+0000 7fca08683700 1 -- 172.21.15.63:0/448981933 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc9e806d4a0 msgr2=0x7fc9e806f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:22.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.526+0000 7fca08683700 1 --2- 172.21.15.63:0/448981933 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc9e806d4a0 0x7fc9e806f950 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4004890 tx=0x7fc9f4009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:22.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.526+0000 7fca08683700 1 -- 172.21.15.63:0/448981933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca00101d70 msgr2=0x7fca000fd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:22.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.526+0000 7fca08683700 1 --2- 172.21.15.63:0/448981933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca00101d70 0x7fca000fd9f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0001d80 tx=0x7fc9f0009ec0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.526+0000 7fca08683700 1 -- 172.21.15.63:0/448981933 shutdown_connections 2024-03-22T07:21:22.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.526+0000 7fca08683700 1 --2- 172.21.15.63:0/448981933 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc9e806d4a0 0x7fc9e806f950 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.526+0000 7fca08683700 1 --2- 172.21.15.63:0/448981933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fca00101d70 0x7fca000fd9f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.526+0000 7fca08683700 1 --2- 172.21.15.63:0/448981933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca00102710 0x7fca000fdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.526+0000 7fca08683700 1 -- 172.21.15.63:0/448981933 >> 172.21.15.63:0/448981933 conn(0x7fca00072bc0 msgr2=0x7fca00065940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:22.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.526+0000 7fca08683700 1 -- 172.21.15.63:0/448981933 shutdown_connections 2024-03-22T07:21:22.527 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.526+0000 7fca08683700 1 -- 172.21.15.63:0/448981933 wait complete. 2024-03-22T07:21:22.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.628+0000 7f8c9e5ef700 1 -- 172.21.15.63:0/299454776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c98101d20 msgr2=0x7f8c981020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:22.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.628+0000 7f8c9e5ef700 1 --2- 172.21.15.63:0/299454776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c98101d20 0x7f8c981020f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8c88004940 tx=0x7f8c88009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:22.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.629+0000 7f8c9e5ef700 1 -- 172.21.15.63:0/299454776 shutdown_connections 2024-03-22T07:21:22.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.629+0000 7f8c9e5ef700 1 --2- 172.21.15.63:0/299454776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c981026c0 0x7f8c980fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.629+0000 7f8c9e5ef700 1 --2- 172.21.15.63:0/299454776 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c98101d20 0x7f8c981020f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.629+0000 7f8c9e5ef700 1 -- 172.21.15.63:0/299454776 >> 172.21.15.63:0/299454776 conn(0x7f8c980f7870 msgr2=0x7f8c980f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:22.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.629+0000 7f8c9e5ef700 1 -- 172.21.15.63:0/299454776 shutdown_connections 2024-03-22T07:21:22.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.629+0000 7f8c9e5ef700 1 -- 172.21.15.63:0/299454776 wait complete. 2024-03-22T07:21:22.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.629+0000 7f8c9e5ef700 1 Processor -- start 2024-03-22T07:21:22.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.629+0000 7f8c9e5ef700 1 -- start start 2024-03-22T07:21:22.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.630+0000 7f8c9e5ef700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c98101d20 0x7f8c98106bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:22.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.630+0000 7f8c9e5ef700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c981026c0 0x7f8c98107100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:22.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.630+0000 7f8c9e5ef700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98108b10 con 0x7f8c981026c0 2024-03-22T07:21:22.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.630+0000 7f8c9e5ef700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98107640 con 0x7f8c98101d20 2024-03-22T07:21:22.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.630+0000 7f8c977fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c981026c0 0x7f8c98107100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:22.631 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.630+0000 7f8c977fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c981026c0 0x7f8c98107100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37138/0 (socket says 172.21.15.63:37138) 2024-03-22T07:21:22.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.630+0000 7f8c977fe700 1 -- 172.21.15.63:0/1459112737 learned_addr learned my addr 172.21.15.63:0/1459112737 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:21:22.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.630+0000 7f8c97fff700 1 --2- 172.21.15.63:0/1459112737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c98101d20 0x7f8c98106bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:22.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.630+0000 7f8c977fe700 1 -- 172.21.15.63:0/1459112737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c98101d20 msgr2=0x7f8c98106bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:22.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.630+0000 7f8c977fe700 1 --2- 172.21.15.63:0/1459112737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c98101d20 0x7f8c98106bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.630+0000 7f8c977fe700 1 -- 172.21.15.63:0/1459112737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c88005040 con 0x7f8c981026c0 2024-03-22T07:21:22.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.630+0000 7f8c97fff700 1 --2- 172.21.15.63:0/1459112737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c98101d20 0x7f8c98106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:21:22.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.630+0000 7f8c977fe700 1 --2- 172.21.15.63:0/1459112737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c981026c0 0x7f8c98107100 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c002a80 tx=0x7f8c8c004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:22.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.630+0000 7f8c94ff9700 1 -- 172.21.15.63:0/1459112737 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c8c004e80 con 0x7f8c981026c0 2024-03-22T07:21:22.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.630+0000 7f8c94ff9700 1 -- 172.21.15.63:0/1459112737 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c8c00f040 con 0x7f8c981026c0 2024-03-22T07:21:22.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.630+0000 7f8c9e5ef700 1 -- 172.21.15.63:0/1459112737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c98107920 con 0x7f8c981026c0 2024-03-22T07:21:22.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.630+0000 7f8c94ff9700 1 -- 172.21.15.63:0/1459112737 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c8c00daa0 con 0x7f8c981026c0 2024-03-22T07:21:22.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.631+0000 7f8c9e5ef700 1 -- 172.21.15.63:0/1459112737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9811a190 con 0x7f8c981026c0 2024-03-22T07:21:22.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.632+0000 7f8c9e5ef700 1 -- 172.21.15.63:0/1459112737 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c980ff030 con 0x7f8c981026c0 2024-03-22T07:21:22.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.632+0000 7f8c94ff9700 1 -- 172.21.15.63:0/1459112737 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f8c8c007cc0 con 0x7f8c981026c0 2024-03-22T07:21:22.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.633+0000 7f8c94ff9700 1 --2- 172.21.15.63:0/1459112737 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f8c8006d4b0 0x7f8c8006f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:22.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.633+0000 7f8c94ff9700 1 -- 172.21.15.63:0/1459112737 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8c8c08c190 con 0x7f8c981026c0 2024-03-22T07:21:22.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.633+0000 7f8c97fff700 1 --2- 172.21.15.63:0/1459112737 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f8c8006d4b0 0x7f8c8006f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:22.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.633+0000 7f8c97fff700 1 --2- 172.21.15.63:0/1459112737 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f8c8006d4b0 0x7f8c8006f960 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8c88004560 tx=0x7f8c88009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:22.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.637+0000 7f8c94ff9700 1 -- 172.21.15.63:0/1459112737 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8c8c055a50 con 0x7f8c981026c0 2024-03-22T07:21:22.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.853+0000 7f8c9e5ef700 1 -- 172.21.15.63:0/1459112737 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c98109f40 con 0x7f8c8006d4b0 2024-03-22T07:21:22.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.863+0000 7f8c94ff9700 1 -- 172.21.15.63:0/1459112737 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8c98109f40 con 0x7f8c8006d4b0 2024-03-22T07:21:22.863 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:21:22.863 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (2m) 46s ago 13m 17.8M - 0.25.0 c8568f914cd2 06ef21da90e5 2024-03-22T07:21:22.863 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (13m) 46s ago 13m 6660k - 17.2.7-726-g9c4e0c34 837124da804f d6c6bbeb6904 2024-03-22T07:21:22.863 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (12m) 31s ago 12m 6665k - 17.2.7-726-g9c4e0c34 837124da804f 30992f8aa88c 2024-03-22T07:21:22.863 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (2m) 46s ago 12m 82.9M - 9.4.12 a0ea5e3c46d9 9015fae0d5e9 2024-03-22T07:21:22.863 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (7m) 46s ago 7m 74.1M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:21:22.863 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (7m) 46s ago 7m 124M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:21:22.863 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (7m) 31s ago 7m 63.0M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:21:22.863 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (7m) 31s ago 7m 129M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:21:22.864 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:8443,9283,8765 running (3m) 46s ago 14m 475M - 19.0.0-1667-gdb0330b1 4f325f8440e9 e49468f39991 2024-03-22T07:21:22.864 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283,8765 running (3m) 31s ago 12m 448M - 19.0.0-1667-gdb0330b1 4f325f8440e9 13c3f034ebb2 2024-03-22T07:21:22.864 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (60s) 46s ago 14m 30.8M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ab96538a07c1 2024-03-22T07:21:22.864 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (35s) 31s ago 12m 24.8M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 01a26e1fb1a1 2024-03-22T07:21:22.864 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (3m) 46s ago 13m 16.2M - 1.5.0 0da6a335fe13 0c825ee2a196 2024-03-22T07:21:22.864 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (2m) 31s ago 12m 16.5M - 1.5.0 0da6a335fe13 12edb2f35498 2024-03-22T07:21:22.864 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (11m) 46s ago 11m 218M 1557M 17.2.7-726-g9c4e0c34 837124da804f 580f22188a10 2024-03-22T07:21:22.864 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (11m) 46s ago 11m 243M 1557M 17.2.7-726-g9c4e0c34 837124da804f fe1ace35c88a 2024-03-22T07:21:22.864 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (10m) 46s ago 10m 199M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4cd995265dcf 2024-03-22T07:21:22.864 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (10m) 31s ago 10m 280M 2581M 17.2.7-726-g9c4e0c34 837124da804f 45c9a759b76d 2024-03-22T07:21:22.864 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (9m) 31s ago 9m 241M 2581M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:21:22.864 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (9m) 31s ago 9m 239M 2581M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:21:22.864 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (2m) 46s ago 12m 56.8M - 2.43.0 a07b618ecd1d 1d89b928f6e7 2024-03-22T07:21:22.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.865+0000 7f8c9e5ef700 1 -- 172.21.15.63:0/1459112737 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f8c8006d4b0 msgr2=0x7f8c8006f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:22.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.865+0000 7f8c9e5ef700 1 --2- 172.21.15.63:0/1459112737 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f8c8006d4b0 0x7f8c8006f960 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8c88004560 tx=0x7f8c88009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:22.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.865+0000 7f8c9e5ef700 1 -- 172.21.15.63:0/1459112737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c981026c0 msgr2=0x7f8c98107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:22.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.865+0000 7f8c9e5ef700 1 --2- 172.21.15.63:0/1459112737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c981026c0 0x7f8c98107100 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c002a80 tx=0x7f8c8c004630 comp rx=0 tx=0).stop 2024-03-22T07:21:22.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.865+0000 7f8c9e5ef700 1 -- 172.21.15.63:0/1459112737 shutdown_connections 2024-03-22T07:21:22.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.865+0000 7f8c9e5ef700 1 --2- 172.21.15.63:0/1459112737 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f8c8006d4b0 0x7f8c8006f960 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.865+0000 7f8c9e5ef700 1 --2- 172.21.15.63:0/1459112737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c98101d20 0x7f8c98106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.865+0000 7f8c9e5ef700 1 --2- 172.21.15.63:0/1459112737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c981026c0 0x7f8c98107100 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.865+0000 7f8c9e5ef700 1 -- 172.21.15.63:0/1459112737 >> 172.21.15.63:0/1459112737 conn(0x7f8c980f7870 msgr2=0x7f8c981062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:22.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.865+0000 7f8c9e5ef700 1 -- 172.21.15.63:0/1459112737 shutdown_connections 2024-03-22T07:21:22.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.865+0000 7f8c9e5ef700 1 -- 172.21.15.63:0/1459112737 wait complete. 2024-03-22T07:21:22.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.966+0000 7f1815bc0700 1 -- 172.21.15.63:0/669632488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18100fe780 msgr2=0x7f1810106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:22.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.966+0000 7f1815bc0700 1 --2- 172.21.15.63:0/669632488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18100fe780 0x7f1810106dc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1804004940 tx=0x7f1804009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:22.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.966+0000 7f1815bc0700 1 -- 172.21.15.63:0/669632488 shutdown_connections 2024-03-22T07:21:22.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.966+0000 7f1815bc0700 1 --2- 172.21.15.63:0/669632488 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18100fe780 0x7f1810106dc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.966+0000 7f1815bc0700 1 --2- 172.21.15.63:0/669632488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18100fdde0 0x7f18100fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.966+0000 7f1815bc0700 1 -- 172.21.15.63:0/669632488 >> 172.21.15.63:0/669632488 conn(0x7f18100f7890 msgr2=0x7f18100f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:22.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.967+0000 7f1815bc0700 1 -- 172.21.15.63:0/669632488 shutdown_connections 2024-03-22T07:21:22.967 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.967+0000 7f1815bc0700 1 -- 172.21.15.63:0/669632488 wait complete. 2024-03-22T07:21:22.967 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:22 smithi063 ceph-mon[112784]: pgmap v28: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:22.968 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:22 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:22.968 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:22 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:22.968 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:22 smithi063 ceph-mon[112784]: Reconfiguring mds.cephfs.smithi069.cijxah (monmap changed)... 2024-03-22T07:21:22.968 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:22 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi069.cijxah", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:21:22.968 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:22 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:22.968 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:22 smithi063 ceph-mon[112784]: Reconfiguring daemon mds.cephfs.smithi069.cijxah on smithi069 2024-03-22T07:21:22.968 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:22 smithi063 ceph-mon[112784]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:21:22.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.967+0000 7f1815bc0700 1 Processor -- start 2024-03-22T07:21:22.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.967+0000 7f1815bc0700 1 -- start start 2024-03-22T07:21:22.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.968+0000 7f1815bc0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18100fdde0 0x7f18101138b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:22.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.968+0000 7f1815bc0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18100fe780 0x7f1810113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:22.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.968+0000 7f1815bc0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1810114440 con 0x7f18100fe780 2024-03-22T07:21:22.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.968+0000 7f1815bc0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181010da40 con 0x7f18100fdde0 2024-03-22T07:21:22.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.968+0000 7f180effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18100fe780 0x7f1810113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:22.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.968+0000 7f180effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18100fe780 0x7f1810113df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37152/0 (socket says 172.21.15.63:37152) 2024-03-22T07:21:22.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.968+0000 7f180effd700 1 -- 172.21.15.63:0/625880900 learned_addr learned my addr 172.21.15.63:0/625880900 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:21:22.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.968+0000 7f180f7fe700 1 --2- 172.21.15.63:0/625880900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18100fdde0 0x7f18101138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:22.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.968+0000 7f180effd700 1 -- 172.21.15.63:0/625880900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18100fdde0 msgr2=0x7f18101138b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:22.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.968+0000 7f180effd700 1 --2- 172.21.15.63:0/625880900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18100fdde0 0x7f18101138b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:22.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.968+0000 7f180effd700 1 -- 172.21.15.63:0/625880900 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1804005040 con 0x7f18100fe780 2024-03-22T07:21:22.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.968+0000 7f180effd700 1 --2- 172.21.15.63:0/625880900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18100fe780 0x7f1810113df0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1804000c00 tx=0x7f1804009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:22.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.968+0000 7f17f7fff700 1 -- 172.21.15.63:0/625880900 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1804021070 con 0x7f18100fe780 2024-03-22T07:21:22.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.968+0000 7f17f7fff700 1 -- 172.21.15.63:0/625880900 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1804013c00 con 0x7f18100fe780 2024-03-22T07:21:22.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.968+0000 7f17f7fff700 1 -- 172.21.15.63:0/625880900 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f180400e070 con 0x7f18100fe780 2024-03-22T07:21:22.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.969+0000 7f1815bc0700 1 -- 172.21.15.63:0/625880900 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f181010dcc0 con 0x7f18100fe780 2024-03-22T07:21:22.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.969+0000 7f1815bc0700 1 -- 172.21.15.63:0/625880900 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f181010e1b0 con 0x7f18100fe780 2024-03-22T07:21:22.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.970+0000 7f1815bc0700 1 -- 172.21.15.63:0/625880900 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1810066bd0 con 0x7f18100fe780 2024-03-22T07:21:22.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.970+0000 7f17f7fff700 1 -- 172.21.15.63:0/625880900 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f180401c400 con 0x7f18100fe780 2024-03-22T07:21:22.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.971+0000 7f17f7fff700 1 --2- 172.21.15.63:0/625880900 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f17f806d4b0 0x7f17f806f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:22.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.971+0000 7f17f7fff700 1 -- 172.21.15.63:0/625880900 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f180408df50 con 0x7f18100fe780 2024-03-22T07:21:22.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.971+0000 7f180f7fe700 1 --2- 172.21.15.63:0/625880900 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f17f806d4b0 0x7f17f806f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:22.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.971+0000 7f180f7fe700 1 --2- 172.21.15.63:0/625880900 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f17f806d4b0 0x7f17f806f960 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1800009b80 tx=0x7f180000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:22.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:22.976+0000 7f17f7fff700 1 -- 172.21.15.63:0/625880900 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f18040c1d20 con 0x7f18100fe780 2024-03-22T07:21:22.991 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:22 smithi069 ceph-mon[83631]: pgmap v28: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:22.991 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:22 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:22.991 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:22 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:22.991 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:22 smithi069 ceph-mon[83631]: Reconfiguring mds.cephfs.smithi069.cijxah (monmap changed)... 2024-03-22T07:21:22.991 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:22 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi069.cijxah", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:21:22.991 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:22 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:22.991 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:22 smithi069 ceph-mon[83631]: Reconfiguring daemon mds.cephfs.smithi069.cijxah on smithi069 2024-03-22T07:21:22.991 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:22 smithi069 ceph-mon[83631]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:21:23.263 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.263+0000 7f1815bc0700 1 -- 172.21.15.63:0/625880900 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1810109e70 con 0x7f18100fe780 2024-03-22T07:21:23.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.264+0000 7f17f7fff700 1 -- 172.21.15.63:0/625880900 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f180405b3c0 con 0x7f18100fe780 2024-03-22T07:21:23.264 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:21:23.264 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:21:23.264 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:21:23.264 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:21:23.264 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:21:23.264 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:21:23.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:21:23.265 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:21:23.265 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6 2024-03-22T07:21:23.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:21:23.265 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:21:23.265 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:21:23.265 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:21:23.265 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:21:23.265 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 10, 2024-03-22T07:21:23.265 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-22T07:21:23.265 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:21:23.265 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:21:23.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.267+0000 7f1815bc0700 1 -- 172.21.15.63:0/625880900 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f17f806d4b0 msgr2=0x7f17f806f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:23.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.267+0000 7f1815bc0700 1 --2- 172.21.15.63:0/625880900 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f17f806d4b0 0x7f17f806f960 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1800009b80 tx=0x7f180000b040 comp rx=0 tx=0).stop 2024-03-22T07:21:23.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.267+0000 7f1815bc0700 1 -- 172.21.15.63:0/625880900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18100fe780 msgr2=0x7f1810113df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:23.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.267+0000 7f1815bc0700 1 --2- 172.21.15.63:0/625880900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18100fe780 0x7f1810113df0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1804000c00 tx=0x7f1804009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:23.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.267+0000 7f1815bc0700 1 -- 172.21.15.63:0/625880900 shutdown_connections 2024-03-22T07:21:23.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.267+0000 7f1815bc0700 1 --2- 172.21.15.63:0/625880900 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f17f806d4b0 0x7f17f806f960 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:23.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.267+0000 7f1815bc0700 1 --2- 172.21.15.63:0/625880900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f18100fdde0 0x7f18101138b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:23.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.267+0000 7f1815bc0700 1 --2- 172.21.15.63:0/625880900 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f18100fe780 0x7f1810113df0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:23.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.267+0000 7f1815bc0700 1 -- 172.21.15.63:0/625880900 >> 172.21.15.63:0/625880900 conn(0x7f18100f7890 msgr2=0x7f18100f8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:23.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.267+0000 7f1815bc0700 1 -- 172.21.15.63:0/625880900 shutdown_connections 2024-03-22T07:21:23.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.267+0000 7f1815bc0700 1 -- 172.21.15.63:0/625880900 wait complete. 2024-03-22T07:21:23.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.369+0000 7f9aba428700 1 -- 172.21.15.63:0/3502668691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ab40fde10 msgr2=0x7f9ab40fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:23.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.369+0000 7f9aba428700 1 --2- 172.21.15.63:0/3502668691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ab40fde10 0x7f9ab40fe1e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4004940 tx=0x7f9aa4009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:23.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.369+0000 7f9aba428700 1 -- 172.21.15.63:0/3502668691 shutdown_connections 2024-03-22T07:21:23.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.369+0000 7f9aba428700 1 --2- 172.21.15.63:0/3502668691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab40fe7b0 0x7f9ab4106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:23.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.369+0000 7f9aba428700 1 --2- 172.21.15.63:0/3502668691 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ab40fde10 0x7f9ab40fe1e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:23.369 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.369+0000 7f9aba428700 1 -- 172.21.15.63:0/3502668691 >> 172.21.15.63:0/3502668691 conn(0x7f9ab40f7880 msgr2=0x7f9ab40f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:23.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.369+0000 7f9aba428700 1 -- 172.21.15.63:0/3502668691 shutdown_connections 2024-03-22T07:21:23.370 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.369+0000 7f9aba428700 1 -- 172.21.15.63:0/3502668691 wait complete. 2024-03-22T07:21:23.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.370+0000 7f9aba428700 1 Processor -- start 2024-03-22T07:21:23.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.370+0000 7f9aba428700 1 -- start start 2024-03-22T07:21:23.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.370+0000 7f9aba428700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ab40fde10 0x7f9ab40fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:23.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.370+0000 7f9aba428700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab40fe7b0 0x7f9ab40fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:23.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.370+0000 7f9aba428700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab40fc710 con 0x7f9ab40fde10 2024-03-22T07:21:23.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.370+0000 7f9aba428700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab40fc850 con 0x7f9ab40fe7b0 2024-03-22T07:21:23.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.370+0000 7f9ab3fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ab40fde10 0x7f9ab40fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:23.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.370+0000 7f9ab3fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ab40fde10 0x7f9ab40fbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37160/0 (socket says 172.21.15.63:37160) 2024-03-22T07:21:23.371 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.370+0000 7f9ab37fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab40fe7b0 0x7f9ab40fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:23.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.370+0000 7f9ab3fff700 1 -- 172.21.15.63:0/1633521455 learned_addr learned my addr 172.21.15.63:0/1633521455 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:21:23.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.370+0000 7f9ab37fe700 1 -- 172.21.15.63:0/1633521455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ab40fde10 msgr2=0x7f9ab40fbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:23.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.370+0000 7f9ab37fe700 1 --2- 172.21.15.63:0/1633521455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ab40fde10 0x7f9ab40fbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:23.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.370+0000 7f9ab37fe700 1 -- 172.21.15.63:0/1633521455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa4005040 con 0x7f9ab40fe7b0 2024-03-22T07:21:23.372 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.371+0000 7f9ab37fe700 1 --2- 172.21.15.63:0/1633521455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab40fe7b0 0x7f9ab40fc180 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c002820 tx=0x7f9a9c004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:23.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.373+0000 7f9ab0ff9700 1 -- 172.21.15.63:0/1633521455 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a9c007cc0 con 0x7f9ab40fe7b0 2024-03-22T07:21:23.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.373+0000 7f9ab0ff9700 1 -- 172.21.15.63:0/1633521455 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a9c00f040 con 0x7f9ab40fe7b0 2024-03-22T07:21:23.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.373+0000 7f9ab0ff9700 1 -- 172.21.15.63:0/1633521455 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a9c013920 con 0x7f9ab40fe7b0 2024-03-22T07:21:23.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.373+0000 7f9aba428700 1 -- 172.21.15.63:0/1633521455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab411a1c0 con 0x7f9ab40fe7b0 2024-03-22T07:21:23.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.373+0000 7f9aba428700 1 -- 172.21.15.63:0/1633521455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab411a5c0 con 0x7f9ab40fe7b0 2024-03-22T07:21:23.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.374+0000 7f9aba428700 1 -- 172.21.15.63:0/1633521455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab4066770 con 0x7f9ab40fe7b0 2024-03-22T07:21:23.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.374+0000 7f9ab0ff9700 1 -- 172.21.15.63:0/1633521455 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f9a9c013ac0 con 0x7f9ab40fe7b0 2024-03-22T07:21:23.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.374+0000 7f9ab0ff9700 1 --2- 172.21.15.63:0/1633521455 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9aa006d4a0 0x7f9aa006f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:23.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.375+0000 7f9ab0ff9700 1 -- 172.21.15.63:0/1633521455 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9a9c08c820 con 0x7f9ab40fe7b0 2024-03-22T07:21:23.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.375+0000 7f9ab3fff700 1 --2- 172.21.15.63:0/1633521455 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9aa006d4a0 0x7f9aa006f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:23.375 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.375+0000 7f9ab3fff700 1 --2- 172.21.15.63:0/1633521455 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9aa006d4a0 0x7f9aa006f950 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009fd0 tx=0x7f9aa4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:23.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.379+0000 7f9ab0ff9700 1 -- 172.21.15.63:0/1633521455 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9a9c0560c0 con 0x7f9ab40fe7b0 2024-03-22T07:21:23.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.649+0000 7f9aba428700 1 -- 172.21.15.63:0/1633521455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9ab404bed0 con 0x7f9ab40fe7b0 2024-03-22T07:21:23.650 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.650+0000 7f9ab0ff9700 1 -- 172.21.15.63:0/1633521455 <== mon.1 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1868 (secure 0 0 0) 0x7f9a9c059780 con 0x7f9ab40fe7b0 2024-03-22T07:21:23.650 INFO:teuthology.orchestra.run.smithi063.stdout:e24 2024-03-22T07:21:23.650 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:21:23.650 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:21:23.650 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-03-22T07:21:23.651 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:21:23.651 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:21:23.651 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-03-22T07:21:23.651 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 24 2024-03-22T07:21:23.651 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-22T07:21:23.651 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-03-22T07:13:29.602905+0000 2024-03-22T07:21:23.651 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-03-22T07:20:24.253589+0000 2024-03-22T07:21:23.651 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-03-22T07:21:23.651 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-03-22T07:21:23.651 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-03-22T07:21:23.651 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-03-22T07:21:23.651 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-03-22T07:21:23.651 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-03-22T07:21:23.651 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-03-22T07:21:23.651 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-03-22T07:21:23.652 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-03-22T07:21:23.652 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:21:23.652 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 2 2024-03-22T07:21:23.652 INFO:teuthology.orchestra.run.smithi063.stdout:in 0,1 2024-03-22T07:21:23.652 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=14480,1=24285} 2024-03-22T07:21:23.652 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-03-22T07:21:23.652 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-03-22T07:21:23.652 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-03-22T07:21:23.652 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-03-22T07:21:23.652 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-03-22T07:21:23.652 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-03-22T07:21:23.652 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-03-22T07:21:23.652 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-03-22T07:21:23.652 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-03-22T07:21:23.652 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.xeelpk{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:21:23.653 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.oytapz{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3211595802,v1:172.21.15.63:6827/3211595802] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:21:23.653 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.wwtsjn{1:24285} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.69:6824/3798247699,v1:172.21.15.69:6825/3798247699] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:21:23.653 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.cijxah{1:24301} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.69:6826/3380719311,v1:172.21.15.69:6827/3380719311] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:21:23.653 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:21:23.653 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:21:23.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.652+0000 7f9aba428700 1 -- 172.21.15.63:0/1633521455 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9aa006d4a0 msgr2=0x7f9aa006f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:23.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.652+0000 7f9aba428700 1 --2- 172.21.15.63:0/1633521455 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9aa006d4a0 0x7f9aa006f950 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009fd0 tx=0x7f9aa4009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:23.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.652+0000 7f9aba428700 1 -- 172.21.15.63:0/1633521455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab40fe7b0 msgr2=0x7f9ab40fc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:23.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.652+0000 7f9aba428700 1 --2- 172.21.15.63:0/1633521455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab40fe7b0 0x7f9ab40fc180 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c002820 tx=0x7f9a9c004570 comp rx=0 tx=0).stop 2024-03-22T07:21:23.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.652+0000 7f9aba428700 1 -- 172.21.15.63:0/1633521455 shutdown_connections 2024-03-22T07:21:23.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.652+0000 7f9aba428700 1 --2- 172.21.15.63:0/1633521455 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9aa006d4a0 0x7f9aa006f950 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:23.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.652+0000 7f9aba428700 1 --2- 172.21.15.63:0/1633521455 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9ab40fde10 0x7f9ab40fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:23.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.652+0000 7f9aba428700 1 --2- 172.21.15.63:0/1633521455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab40fe7b0 0x7f9ab40fc180 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:23.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.652+0000 7f9aba428700 1 -- 172.21.15.63:0/1633521455 >> 172.21.15.63:0/1633521455 conn(0x7f9ab40f7880 msgr2=0x7f9ab40f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:23.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.652+0000 7f9aba428700 1 -- 172.21.15.63:0/1633521455 shutdown_connections 2024-03-22T07:21:23.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.652+0000 7f9aba428700 1 -- 172.21.15.63:0/1633521455 wait complete. 2024-03-22T07:21:23.654 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 24 2024-03-22T07:21:23.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.758+0000 7f38ed863700 1 -- 172.21.15.63:0/1732264392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38e8101d20 msgr2=0x7f38e81020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:23.758 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.758+0000 7f38ed863700 1 --2- 172.21.15.63:0/1732264392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38e8101d20 0x7f38e81020f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f38dc004940 tx=0x7f38dc009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:23.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.758+0000 7f38ed863700 1 -- 172.21.15.63:0/1732264392 shutdown_connections 2024-03-22T07:21:23.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.758+0000 7f38ed863700 1 --2- 172.21.15.63:0/1732264392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38e81026c0 0x7f38e80fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:23.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.758+0000 7f38ed863700 1 --2- 172.21.15.63:0/1732264392 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38e8101d20 0x7f38e81020f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:23.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.758+0000 7f38ed863700 1 -- 172.21.15.63:0/1732264392 >> 172.21.15.63:0/1732264392 conn(0x7f38e80f7870 msgr2=0x7f38e80f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:23.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.758+0000 7f38ed863700 1 -- 172.21.15.63:0/1732264392 shutdown_connections 2024-03-22T07:21:23.759 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.758+0000 7f38ed863700 1 -- 172.21.15.63:0/1732264392 wait complete. 2024-03-22T07:21:23.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.759+0000 7f38ed863700 1 Processor -- start 2024-03-22T07:21:23.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.759+0000 7f38ed863700 1 -- start start 2024-03-22T07:21:23.760 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.759+0000 7f38ed863700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38e8101d20 0x7f38e810ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.759+0000 7f38ed863700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38e81026c0 0x7f38e810efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.759+0000 7f38ed863700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e810f500 con 0x7f38e81026c0 2024-03-22T07:21:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.759+0000 7f38ed863700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e810f640 con 0x7f38e8101d20 2024-03-22T07:21:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.759+0000 7f38e67fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38e81026c0 0x7f38e810efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.759+0000 7f38e67fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38e81026c0 0x7f38e810efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37178/0 (socket says 172.21.15.63:37178) 2024-03-22T07:21:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.759+0000 7f38e67fc700 1 -- 172.21.15.63:0/2436469768 learned_addr learned my addr 172.21.15.63:0/2436469768 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:21:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.759+0000 7f38e6ffd700 1 --2- 172.21.15.63:0/2436469768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38e8101d20 0x7f38e810ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.760+0000 7f38e67fc700 1 -- 172.21.15.63:0/2436469768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38e8101d20 msgr2=0x7f38e810ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.760+0000 7f38e67fc700 1 --2- 172.21.15.63:0/2436469768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38e8101d20 0x7f38e810ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.760+0000 7f38e67fc700 1 -- 172.21.15.63:0/2436469768 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc005040 con 0x7f38e81026c0 2024-03-22T07:21:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.760+0000 7f38e67fc700 1 --2- 172.21.15.63:0/2436469768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38e81026c0 0x7f38e810efc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f38d800eb40 tx=0x7f38d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.760+0000 7f38cf7fe700 1 -- 172.21.15.63:0/2436469768 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38d8007cc0 con 0x7f38e81026c0 2024-03-22T07:21:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.760+0000 7f38cf7fe700 1 -- 172.21.15.63:0/2436469768 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38d800f040 con 0x7f38e81026c0 2024-03-22T07:21:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.760+0000 7f38cf7fe700 1 -- 172.21.15.63:0/2436469768 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38d800c9d0 con 0x7f38e81026c0 2024-03-22T07:21:23.761 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.760+0000 7f38ed863700 1 -- 172.21.15.63:0/2436469768 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e810f8c0 con 0x7f38e81026c0 2024-03-22T07:21:23.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.760+0000 7f38ed863700 1 -- 172.21.15.63:0/2436469768 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e8117e10 con 0x7f38e81026c0 2024-03-22T07:21:23.762 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.761+0000 7f38ed863700 1 -- 172.21.15.63:0/2436469768 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e80fd470 con 0x7f38e81026c0 2024-03-22T07:21:23.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.762+0000 7f38cf7fe700 1 -- 172.21.15.63:0/2436469768 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f38d800cb30 con 0x7f38e81026c0 2024-03-22T07:21:23.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.762+0000 7f38cf7fe700 1 --2- 172.21.15.63:0/2436469768 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f38d006d4b0 0x7f38d006f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:23.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.762+0000 7f38cf7fe700 1 -- 172.21.15.63:0/2436469768 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f38d808c610 con 0x7f38e81026c0 2024-03-22T07:21:23.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.762+0000 7f38e6ffd700 1 --2- 172.21.15.63:0/2436469768 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f38d006d4b0 0x7f38d006f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:23.763 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.763+0000 7f38e6ffd700 1 --2- 172.21.15.63:0/2436469768 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f38d006d4b0 0x7f38d006f960 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00b0d0 tx=0x7f38dc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:23.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.766+0000 7f38cf7fe700 1 -- 172.21.15.63:0/2436469768 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f38d80560a0 con 0x7f38e81026c0 2024-03-22T07:21:23.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.995+0000 7f38ed863700 1 -- 172.21.15.63:0/2436469768 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f38e80f8ef0 con 0x7f38d006d4b0 2024-03-22T07:21:23.996 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:23 smithi063 ceph-mon[112784]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:21:23.996 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:23 smithi063 ceph-mon[112784]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:21:23.996 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:23 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/625880900' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:21:23.996 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:23 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:23.996 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:23 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:23.996 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:23 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:21:23.997 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:23 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:21:23.997 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:23 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:21:23.997 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:23 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:23.997 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:23 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi063"}]: dispatch 2024-03-22T07:21:23.997 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:23 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi063"}]': finished 2024-03-22T07:21:23.997 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:23 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi069"}]: dispatch 2024-03-22T07:21:23.997 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:23 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi069"}]': finished 2024-03-22T07:21:23.997 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:23 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/1633521455' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:21:23.997 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.997+0000 7f38cf7fe700 1 -- 172.21.15.63:0/2436469768 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f38e80f8ef0 con 0x7f38d006d4b0 2024-03-22T07:21:23.998 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:21:23.998 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:21:23.998 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:21:23.998 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:21:23.998 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-03-22T07:21:23.998 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr", 2024-03-22T07:21:23.998 INFO:teuthology.orchestra.run.smithi063.stdout: "mon" 2024-03-22T07:21:23.998 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-03-22T07:21:23.998 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "4/21 daemons upgraded", 2024-03-22T07:21:23.998 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading crash daemons", 2024-03-22T07:21:23.998 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:21:23.998 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:21:23.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.999+0000 7f38ed863700 1 -- 172.21.15.63:0/2436469768 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f38d006d4b0 msgr2=0x7f38d006f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:23.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.999+0000 7f38ed863700 1 --2- 172.21.15.63:0/2436469768 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f38d006d4b0 0x7f38d006f960 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00b0d0 tx=0x7f38dc009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:23.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.999+0000 7f38ed863700 1 -- 172.21.15.63:0/2436469768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38e81026c0 msgr2=0x7f38e810efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:23.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.999+0000 7f38ed863700 1 --2- 172.21.15.63:0/2436469768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38e81026c0 0x7f38e810efc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f38d800eb40 tx=0x7f38d800b040 comp rx=0 tx=0).stop 2024-03-22T07:21:23.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.999+0000 7f38ed863700 1 -- 172.21.15.63:0/2436469768 shutdown_connections 2024-03-22T07:21:23.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.999+0000 7f38ed863700 1 --2- 172.21.15.63:0/2436469768 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f38d006d4b0 0x7f38d006f960 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:23.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.999+0000 7f38ed863700 1 --2- 172.21.15.63:0/2436469768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f38e8101d20 0x7f38e810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:23.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.999+0000 7f38ed863700 1 --2- 172.21.15.63:0/2436469768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f38e81026c0 0x7f38e810efc0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:23.999 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.999+0000 7f38ed863700 1 -- 172.21.15.63:0/2436469768 >> 172.21.15.63:0/2436469768 conn(0x7f38e80f7870 msgr2=0x7f38e80f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:24.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.999+0000 7f38ed863700 1 -- 172.21.15.63:0/2436469768 shutdown_connections 2024-03-22T07:21:24.000 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:23.999+0000 7f38ed863700 1 -- 172.21.15.63:0/2436469768 wait complete. 2024-03-22T07:21:24.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:23 smithi069 ceph-mon[83631]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:21:24.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:23 smithi069 ceph-mon[83631]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:21:24.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:23 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/625880900' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:21:24.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:23 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:24.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:23 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:24.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:23 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:21:24.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:23 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:21:24.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:23 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:21:24.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:23 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:24.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:23 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi063"}]: dispatch 2024-03-22T07:21:24.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:23 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi063"}]': finished 2024-03-22T07:21:24.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:23 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi069"}]: dispatch 2024-03-22T07:21:24.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:23 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi069"}]': finished 2024-03-22T07:21:24.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:23 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/1633521455' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:21:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.103+0000 7efc8d052700 1 -- 172.21.15.63:0/4144760737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc88070300 msgr2=0x7efc880706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.103+0000 7efc8d052700 1 --2- 172.21.15.63:0/4144760737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc88070300 0x7efc880706d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efc78004970 tx=0x7efc78009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.103+0000 7efc8d052700 1 -- 172.21.15.63:0/4144760737 shutdown_connections 2024-03-22T07:21:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.103+0000 7efc8d052700 1 --2- 172.21.15.63:0/4144760737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc88070c10 0x7efc88109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.103+0000 7efc8d052700 1 --2- 172.21.15.63:0/4144760737 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc88070300 0x7efc880706d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.103+0000 7efc8d052700 1 -- 172.21.15.63:0/4144760737 >> 172.21.15.63:0/4144760737 conn(0x7efc880f89a0 msgr2=0x7efc880fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.104+0000 7efc8d052700 1 -- 172.21.15.63:0/4144760737 shutdown_connections 2024-03-22T07:21:24.104 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.104+0000 7efc8d052700 1 -- 172.21.15.63:0/4144760737 wait complete. 2024-03-22T07:21:24.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.104+0000 7efc8d052700 1 Processor -- start 2024-03-22T07:21:24.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.104+0000 7efc8d052700 1 -- start start 2024-03-22T07:21:24.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.104+0000 7efc8d052700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc88070300 0x7efc88115e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:24.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.104+0000 7efc8d052700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc88070c10 0x7efc88116370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:24.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.104+0000 7efc8d052700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc881169c0 con 0x7efc88070c10 2024-03-22T07:21:24.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.104+0000 7efc8d052700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8810fd90 con 0x7efc88070300 2024-03-22T07:21:24.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.104+0000 7efc8659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc88070c10 0x7efc88116370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:24.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.105+0000 7efc8659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc88070c10 0x7efc88116370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:37198/0 (socket says 172.21.15.63:37198) 2024-03-22T07:21:24.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.105+0000 7efc8659c700 1 -- 172.21.15.63:0/3941475189 learned_addr learned my addr 172.21.15.63:0/3941475189 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:21:24.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.105+0000 7efc86d9d700 1 --2- 172.21.15.63:0/3941475189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc88070300 0x7efc88115e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:24.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.105+0000 7efc8659c700 1 -- 172.21.15.63:0/3941475189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc88070300 msgr2=0x7efc88115e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:24.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.105+0000 7efc8659c700 1 --2- 172.21.15.63:0/3941475189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc88070300 0x7efc88115e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:24.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.105+0000 7efc8659c700 1 -- 172.21.15.63:0/3941475189 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc78005040 con 0x7efc88070c10 2024-03-22T07:21:24.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.105+0000 7efc86d9d700 1 --2- 172.21.15.63:0/3941475189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc88070300 0x7efc88115e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:21:24.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.105+0000 7efc8659c700 1 --2- 172.21.15.63:0/3941475189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc88070c10 0x7efc88116370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00c8c0 tx=0x7efc7c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:24.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.105+0000 7efc6f7fe700 1 -- 172.21.15.63:0/3941475189 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc7c00a980 con 0x7efc88070c10 2024-03-22T07:21:24.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.105+0000 7efc8d052700 1 -- 172.21.15.63:0/3941475189 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc88110070 con 0x7efc88070c10 2024-03-22T07:21:24.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.105+0000 7efc6f7fe700 1 -- 172.21.15.63:0/3941475189 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc7c00ce60 con 0x7efc88070c10 2024-03-22T07:21:24.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.105+0000 7efc8d052700 1 -- 172.21.15.63:0/3941475189 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc881105b0 con 0x7efc88070c10 2024-03-22T07:21:24.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.105+0000 7efc6f7fe700 1 -- 172.21.15.63:0/3941475189 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc7c0115e0 con 0x7efc88070c10 2024-03-22T07:21:24.107 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.106+0000 7efc8d052700 1 -- 172.21.15.63:0/3941475189 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc88071a40 con 0x7efc88070c10 2024-03-22T07:21:24.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.107+0000 7efc6f7fe700 1 -- 172.21.15.63:0/3941475189 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7efc7c012400 con 0x7efc88070c10 2024-03-22T07:21:24.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.107+0000 7efc6f7fe700 1 --2- 172.21.15.63:0/3941475189 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7efc7006d4a0 0x7efc7006f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:24.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.107+0000 7efc6f7fe700 1 -- 172.21.15.63:0/3941475189 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6328+0+0 (secure 0 0 0) 0x7efc7c08cb10 con 0x7efc88070c10 2024-03-22T07:21:24.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.107+0000 7efc86d9d700 1 --2- 172.21.15.63:0/3941475189 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7efc7006d4a0 0x7efc7006f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:24.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.108+0000 7efc86d9d700 1 --2- 172.21.15.63:0/3941475189 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7efc7006d4a0 0x7efc7006f950 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efc7800b9c0 tx=0x7efc78009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:24.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.112+0000 7efc6f7fe700 1 -- 172.21.15.63:0/3941475189 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efc7c05a090 con 0x7efc88070c10 2024-03-22T07:21:24.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.407+0000 7efc8d052700 1 -- 172.21.15.63:0/3941475189 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efc8805f5f0 con 0x7efc88070c10 2024-03-22T07:21:24.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.407+0000 7efc6f7fe700 1 -- 172.21.15.63:0/3941475189 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7efc7c0126b0 con 0x7efc88070c10 2024-03-22T07:21:24.408 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-03-22T07:21:24.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.409+0000 7efc8d052700 1 -- 172.21.15.63:0/3941475189 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7efc7006d4a0 msgr2=0x7efc7006f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:24.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.409+0000 7efc8d052700 1 --2- 172.21.15.63:0/3941475189 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7efc7006d4a0 0x7efc7006f950 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efc7800b9c0 tx=0x7efc78009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:24.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.409+0000 7efc8d052700 1 -- 172.21.15.63:0/3941475189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc88070c10 msgr2=0x7efc88116370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:24.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.409+0000 7efc8d052700 1 --2- 172.21.15.63:0/3941475189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc88070c10 0x7efc88116370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00c8c0 tx=0x7efc7c0044c0 comp rx=0 tx=0).stop 2024-03-22T07:21:24.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.409+0000 7efc8d052700 1 -- 172.21.15.63:0/3941475189 shutdown_connections 2024-03-22T07:21:24.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.409+0000 7efc8d052700 1 --2- 172.21.15.63:0/3941475189 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7efc7006d4a0 0x7efc7006f950 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:24.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.409+0000 7efc8d052700 1 --2- 172.21.15.63:0/3941475189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc88070300 0x7efc88115e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:24.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.409+0000 7efc8d052700 1 --2- 172.21.15.63:0/3941475189 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efc88070c10 0x7efc88116370 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:24.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.409+0000 7efc8d052700 1 -- 172.21.15.63:0/3941475189 >> 172.21.15.63:0/3941475189 conn(0x7efc880f89a0 msgr2=0x7efc880f9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:24.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.410+0000 7efc8d052700 1 -- 172.21.15.63:0/3941475189 shutdown_connections 2024-03-22T07:21:24.410 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:24.410+0000 7efc8d052700 1 -- 172.21.15.63:0/3941475189 wait complete. 2024-03-22T07:21:25.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:24 smithi069 ceph-mon[83631]: Upgrade: Setting container_image for all mon 2024-03-22T07:21:25.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:24 smithi069 ceph-mon[83631]: pgmap v29: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:25.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:24 smithi069 ceph-mon[83631]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:21:25.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:24 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3941475189' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:21:25.030 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:24 smithi063 ceph-mon[112784]: Upgrade: Setting container_image for all mon 2024-03-22T07:21:25.030 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:24 smithi063 ceph-mon[112784]: pgmap v29: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:25.030 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:24 smithi063 ceph-mon[112784]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:21:25.030 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:24 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3941475189' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:21:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:26 smithi063 ceph-mon[112784]: Upgrade: Updating crash.smithi063 (1/2) 2024-03-22T07:21:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:26 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:26 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:21:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:26 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:26.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:26 smithi063 ceph-mon[112784]: Deploying daemon crash.smithi063 on smithi063 2024-03-22T07:21:26.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:26 smithi069 ceph-mon[83631]: Upgrade: Updating crash.smithi063 (1/2) 2024-03-22T07:21:26.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:26 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:26.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:26 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi063", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:21:26.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:26 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:26.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:26 smithi069 ceph-mon[83631]: Deploying daemon crash.smithi063 on smithi063 2024-03-22T07:21:27.330 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:27 smithi063 ceph-mon[112784]: pgmap v30: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:27.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:27 smithi069 ceph-mon[83631]: pgmap v30: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:29.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:29 smithi063 ceph-mon[112784]: pgmap v31: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:29.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:29 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:29.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:29 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:29.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:29 smithi069 ceph-mon[83631]: pgmap v31: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:29.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:29 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:29.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:29 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:31.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:31 smithi069 ceph-mon[83631]: pgmap v32: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:31.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:31 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:31.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:31 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi069", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:21:31.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:31 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:31.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:31 smithi063 ceph-mon[112784]: pgmap v32: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:31.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:31 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:31.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:31 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi069", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-22T07:21:31.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:31 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:32.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:32 smithi069 ceph-mon[83631]: Upgrade: Updating crash.smithi069 (2/2) 2024-03-22T07:21:32.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:32 smithi069 ceph-mon[83631]: Deploying daemon crash.smithi069 on smithi069 2024-03-22T07:21:32.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:32 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:21:32.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:32 smithi063 ceph-mon[112784]: Upgrade: Updating crash.smithi069 (2/2) 2024-03-22T07:21:32.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:32 smithi063 ceph-mon[112784]: Deploying daemon crash.smithi069 on smithi069 2024-03-22T07:21:32.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:21:33.501 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:33 smithi069 ceph-mon[83631]: pgmap v33: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:33.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:33 smithi063 ceph-mon[112784]: pgmap v33: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:35.474 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:35 smithi069 ceph-mon[83631]: pgmap v34: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:35.474 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:35 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:35.474 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:35 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:35.475 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:35 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:21:35.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:35 smithi063 ceph-mon[112784]: pgmap v34: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:35.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:35 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:35.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:35 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:35.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:35 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:21:37.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:37 smithi069 ceph-mon[83631]: pgmap v35: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:37.576 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:37 smithi063 ceph-mon[112784]: pgmap v35: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:38.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:38 smithi069 ceph-mon[83631]: pgmap v36: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:38.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:38 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:38.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:38 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:38.838 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:38 smithi063 ceph-mon[112784]: pgmap v36: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:38.839 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:38 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:38.839 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:38 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:39.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:39 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:39.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:39 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:39.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:39 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:39.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:39 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:39.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:39 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:39.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:39 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:39.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:39 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:39.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:39 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:40.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:40 smithi069 ceph-mon[83631]: pgmap v37: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:40.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:40 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:40.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:40 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:40.869 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:40 smithi063 ceph-mon[112784]: pgmap v37: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:40.870 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:40 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:40.870 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:40 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:43.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:42 smithi069 ceph-mon[83631]: pgmap v38: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:43.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:42 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:43.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:42 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:43.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:42 smithi063 ceph-mon[112784]: pgmap v38: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:43.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:42 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:43.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:42 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:45.243 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: pgmap v39: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:45.243 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:45.243 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:45.244 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:45.244 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:21:45.244 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:45.244 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:21:45.244 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:21:45.244 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:21:45.244 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:21:45.244 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: Upgrade: Setting container_image for all crash 2024-03-22T07:21:45.244 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:45.244 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi063"}]: dispatch 2024-03-22T07:21:45.244 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi063"}]': finished 2024-03-22T07:21:45.244 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi069"}]: dispatch 2024-03-22T07:21:45.244 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi069"}]': finished 2024-03-22T07:21:45.244 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-22T07:21:45.244 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-22T07:21:45.245 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:45 smithi063 ceph-mon[112784]: Upgrade: osd.0 is safe to restart 2024-03-22T07:21:45.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: pgmap v39: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:45.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:21:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:21:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:21:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:21:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:21:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: Upgrade: Setting container_image for all crash 2024-03-22T07:21:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi063"}]: dispatch 2024-03-22T07:21:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi063"}]': finished 2024-03-22T07:21:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi069"}]: dispatch 2024-03-22T07:21:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi069"}]': finished 2024-03-22T07:21:45.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-22T07:21:45.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-22T07:21:45.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:45 smithi069 ceph-mon[83631]: Upgrade: osd.0 is safe to restart 2024-03-22T07:21:47.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:46 smithi063 ceph-mon[112784]: pgmap v40: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:47.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:46 smithi063 ceph-mon[112784]: Upgrade: Updating osd.0 2024-03-22T07:21:47.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:46 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:47.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:46 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T07:21:47.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:46 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:47.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:46 smithi063 ceph-mon[112784]: Deploying daemon osd.0 on smithi063 2024-03-22T07:21:47.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:46 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:21:47.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:46 smithi069 ceph-mon[83631]: pgmap v40: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:47.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:46 smithi069 ceph-mon[83631]: Upgrade: Updating osd.0 2024-03-22T07:21:47.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:46 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:47.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:46 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-22T07:21:47.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:46 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:21:47.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:46 smithi069 ceph-mon[83631]: Deploying daemon osd.0 on smithi063 2024-03-22T07:21:47.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:46 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:21:49.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:48 smithi063 ceph-mon[112784]: pgmap v41: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:49.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:48 smithi069 ceph-mon[83631]: pgmap v41: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:21:49.525 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:49 smithi063 systemd[1]: Stopping Ceph osd.0 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:21:49.873 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:49 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0[47343]: 2024-03-22T07:21:49.523+0000 7f6c8a1e7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:21:49.873 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:49 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0[47343]: 2024-03-22T07:21:49.523+0000 7f6c8a1e7700 -1 osd.0 50 *** Got signal Terminated *** 2024-03-22T07:21:49.873 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:49 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0[47343]: 2024-03-22T07:21:49.523+0000 7f6c8a1e7700 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:21:50.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:49 smithi069 ceph-mon[83631]: osd.0 marked itself down and dead 2024-03-22T07:21:50.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:49 smithi063 ceph-mon[112784]: osd.0 marked itself down and dead 2024-03-22T07:21:50.668 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:50 smithi063 podman[123847]: 2024-03-22 07:21:50.382946407 +0000 UTC m=+0.982565740 container died 580f22188a109f76f1a18737ac601aadaabad1d9412681724b4751af311a9260 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-9c4e0c3) 2024-03-22T07:21:50.962 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:50 smithi063 podman[123847]: 2024-03-22 07:21:50.75881224 +0000 UTC m=+1.358431571 container cleanup 580f22188a109f76f1a18737ac601aadaabad1d9412681724b4751af311a9260 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=quincy-9c4e0c3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:21:50.962 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:50 smithi063 bash[123847]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0 2024-03-22T07:21:51.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:50 smithi069 ceph-mon[83631]: pgmap v42: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:51.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:50 smithi069 ceph-mon[83631]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:21:51.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:50 smithi069 ceph-mon[83631]: failed to encode map e51 with expected crc 2024-03-22T07:21:51.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:50 smithi069 ceph-mon[83631]: osdmap e51: 6 total, 5 up, 6 in 2024-03-22T07:21:51.338 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:51 smithi063 podman[123866]: 2024-03-22 07:21:51.035314571 +0000 UTC m=+0.642991764 container remove 580f22188a109f76f1a18737ac601aadaabad1d9412681724b4751af311a9260 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=quincy-9c4e0c3, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319) 2024-03-22T07:21:51.339 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:50 smithi063 ceph-mon[112784]: pgmap v42: 65 pgs: 65 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:21:51.339 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:50 smithi063 ceph-mon[112784]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:21:51.339 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:50 smithi063 ceph-mon[112784]: failed to encode map e51 with expected crc 2024-03-22T07:21:51.339 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:50 smithi063 ceph-mon[112784]: osdmap e51: 6 total, 5 up, 6 in 2024-03-22T07:21:51.597 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:51 smithi063 podman[123962]: 2024-03-22 07:21:51.242058013 +0000 UTC m=+0.017676565 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:21:51.597 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:51 smithi063 podman[123962]: 2024-03-22 07:21:51.352858098 +0000 UTC m=+0.128476650 container create f02e892cd440d2f088f44fb4d5ca088b62e932a06c93585ed8959a841286ca05 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-db0330b) 2024-03-22T07:21:51.873 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:51 smithi063 podman[123962]: 2024-03-22 07:21:51.597697081 +0000 UTC m=+0.373315649 container init f02e892cd440d2f088f44fb4d5ca088b62e932a06c93585ed8959a841286ca05 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-22T07:21:51.873 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:51 smithi063 podman[123962]: 2024-03-22 07:21:51.60237798 +0000 UTC m=+0.377996530 container start f02e892cd440d2f088f44fb4d5ca088b62e932a06c93585ed8959a841286ca05 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b) 2024-03-22T07:21:51.873 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:51 smithi063 podman[123962]: 2024-03-22 07:21:51.688669244 +0000 UTC m=+0.464287804 container attach f02e892cd440d2f088f44fb4d5ca088b62e932a06c93585ed8959a841286ca05 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=squid-db0330b) 2024-03-22T07:21:52.225 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:51 smithi063 podman[123962]: 2024-03-22 07:21:51.907045245 +0000 UTC m=+0.682663808 container died f02e892cd440d2f088f44fb4d5ca088b62e932a06c93585ed8959a841286ca05 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-22T07:21:52.225 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:51 smithi063 ceph-mon[112784]: failed to encode map e52 with expected crc 2024-03-22T07:21:52.225 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:51 smithi063 ceph-mon[112784]: osdmap e52: 6 total, 5 up, 6 in 2024-03-22T07:21:52.225 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:51 smithi063 ceph-mon[112784]: failed to encode map e52 with expected crc 2024-03-22T07:21:52.225 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:51 smithi063 ceph-mon[112784]: failed to encode map e52 with expected crc 2024-03-22T07:21:52.225 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:51 smithi063 ceph-mon[112784]: failed to encode map e52 with expected crc 2024-03-22T07:21:52.225 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:51 smithi063 ceph-mon[112784]: failed to encode map e52 with expected crc 2024-03-22T07:21:52.226 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:51 smithi063 ceph-mon[112784]: failed to encode map e52 with expected crc 2024-03-22T07:21:52.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:51 smithi069 ceph-mon[83631]: failed to encode map e52 with expected crc 2024-03-22T07:21:52.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:51 smithi069 ceph-mon[83631]: osdmap e52: 6 total, 5 up, 6 in 2024-03-22T07:21:52.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:51 smithi069 ceph-mon[83631]: failed to encode map e52 with expected crc 2024-03-22T07:21:52.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:51 smithi069 ceph-mon[83631]: failed to encode map e52 with expected crc 2024-03-22T07:21:52.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:51 smithi069 ceph-mon[83631]: failed to encode map e52 with expected crc 2024-03-22T07:21:52.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:51 smithi069 ceph-mon[83631]: failed to encode map e52 with expected crc 2024-03-22T07:21:52.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:51 smithi069 ceph-mon[83631]: failed to encode map e52 with expected crc 2024-03-22T07:21:52.774 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:52 smithi063 podman[123989]: 2024-03-22 07:21:52.509690631 +0000 UTC m=+0.597578838 container remove f02e892cd440d2f088f44fb4d5ca088b62e932a06c93585ed8959a841286ca05 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:21:52.775 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:52 smithi063 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.0.service: Deactivated successfully. 2024-03-22T07:21:52.775 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:52 smithi063 systemd[1]: Stopped Ceph osd.0 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:21:52.775 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:52 smithi063 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.0.service: Consumed 54.954s CPU time. 2024-03-22T07:21:52.775 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:52 smithi063 systemd[1]: Starting Ceph osd.0 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:21:53.093 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:52 smithi063 ceph-mon[112784]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-22T07:21:53.093 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:52 smithi063 podman[124096]: 2024-03-22 07:21:52.854538757 +0000 UTC m=+0.017941511 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:21:53.093 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:52 smithi063 podman[124096]: 2024-03-22 07:21:52.96934514 +0000 UTC m=+0.132747956 container create 2cc82369e4bf776944552f3fa46f6827c6aa8eb88ca57c24bbebeb4689fcaa02 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-activate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:21:53.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:52 smithi069 ceph-mon[83631]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-22T07:21:53.373 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:53 smithi063 podman[124096]: 2024-03-22 07:21:53.222005209 +0000 UTC m=+0.385407960 container init 2cc82369e4bf776944552f3fa46f6827c6aa8eb88ca57c24bbebeb4689fcaa02 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-activate, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-03-22T07:21:53.373 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:53 smithi063 podman[124096]: 2024-03-22 07:21:53.225562582 +0000 UTC m=+0.388965347 container start 2cc82369e4bf776944552f3fa46f6827c6aa8eb88ca57c24bbebeb4689fcaa02 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:21:53.373 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:53 smithi063 podman[124096]: 2024-03-22 07:21:53.305044468 +0000 UTC m=+0.468447225 container attach 2cc82369e4bf776944552f3fa46f6827c6aa8eb88ca57c24bbebeb4689fcaa02 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-activate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-22T07:21:54.017 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:53 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-activate[124111]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-22T07:21:54.017 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:53 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-activate[124111]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-22T07:21:54.017 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:53 smithi063 bash[124096]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-22T07:21:54.017 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:53 smithi063 bash[124096]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-22T07:21:54.017 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:53 smithi063 bash[124096]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-22T07:21:54.017 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:53 smithi063 bash[124096]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-22T07:21:54.017 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:53 smithi063 bash[124096]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-22T07:21:54.017 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:53 smithi063 bash[124096]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-22T07:21:54.017 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:53 smithi063 bash[124096]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-22T07:21:54.017 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:53 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-activate[124111]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-22T07:21:54.018 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:53 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-activate[124111]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-22T07:21:54.018 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:53 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-activate[124111]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-22T07:21:54.018 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:53 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-activate[124111]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-22T07:21:54.018 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:53 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-activate[124111]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-22T07:21:54.018 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:53 smithi063 podman[124096]: 2024-03-22 07:21:53.69770823 +0000 UTC m=+0.861110982 container died 2cc82369e4bf776944552f3fa46f6827c6aa8eb88ca57c24bbebeb4689fcaa02 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-activate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-22T07:21:54.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:54 smithi069 ceph-mon[83631]: Health check failed: Degraded data redundancy: 69/354 objects degraded (19.492%), 20 pgs degraded (PG_DEGRADED) 2024-03-22T07:21:54.317 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:54 smithi063 ceph-mon[112784]: Health check failed: Degraded data redundancy: 69/354 objects degraded (19.492%), 20 pgs degraded (PG_DEGRADED) 2024-03-22T07:21:54.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.517+0000 7fbbd7046700 1 -- 172.21.15.63:0/3535512955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbd0101d70 msgr2=0x7fbbd0102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:54.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.517+0000 7fbbd7046700 1 --2- 172.21.15.63:0/3535512955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbd0101d70 0x7fbbd0102140 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4004940 tx=0x7fbbc4009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:54.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.517+0000 7fbbd7046700 1 -- 172.21.15.63:0/3535512955 shutdown_connections 2024-03-22T07:21:54.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.517+0000 7fbbd7046700 1 --2- 172.21.15.63:0/3535512955 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbd0102710 0x7fbbd00fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:54.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.517+0000 7fbbd7046700 1 --2- 172.21.15.63:0/3535512955 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbd0101d70 0x7fbbd0102140 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:54.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.517+0000 7fbbd7046700 1 -- 172.21.15.63:0/3535512955 >> 172.21.15.63:0/3535512955 conn(0x7fbbd0072bc0 msgr2=0x7fbbd0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:54.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.517+0000 7fbbd7046700 1 -- 172.21.15.63:0/3535512955 shutdown_connections 2024-03-22T07:21:54.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.517+0000 7fbbd7046700 1 -- 172.21.15.63:0/3535512955 wait complete. 2024-03-22T07:21:54.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.518+0000 7fbbd7046700 1 Processor -- start 2024-03-22T07:21:54.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.518+0000 7fbbd7046700 1 -- start start 2024-03-22T07:21:54.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.518+0000 7fbbd7046700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbd0101d70 0x7fbbd0113b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:54.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.518+0000 7fbbd7046700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbd0102710 0x7fbbd0114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:54.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.518+0000 7fbbd7046700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd0114730 con 0x7fbbd0101d70 2024-03-22T07:21:54.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.518+0000 7fbbd7046700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd010da90 con 0x7fbbd0102710 2024-03-22T07:21:54.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.518+0000 7fbbd4de2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbd0101d70 0x7fbbd0113b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:54.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.518+0000 7fbbd4de2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbd0101d70 0x7fbbd0113b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:41176/0 (socket says 172.21.15.63:41176) 2024-03-22T07:21:54.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.518+0000 7fbbd4de2700 1 -- 172.21.15.63:0/2621375405 learned_addr learned my addr 172.21.15.63:0/2621375405 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:21:54.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.518+0000 7fbbcffff700 1 --2- 172.21.15.63:0/2621375405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbd0102710 0x7fbbd0114050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:54.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.518+0000 7fbbd4de2700 1 -- 172.21.15.63:0/2621375405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbd0102710 msgr2=0x7fbbd0114050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:54.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.518+0000 7fbbd4de2700 1 --2- 172.21.15.63:0/2621375405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbd0102710 0x7fbbd0114050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:54.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.518+0000 7fbbd4de2700 1 -- 172.21.15.63:0/2621375405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc4005040 con 0x7fbbd0101d70 2024-03-22T07:21:54.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.519+0000 7fbbd4de2700 1 --2- 172.21.15.63:0/2621375405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbd0101d70 0x7fbbd0113b10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400b3b0 tx=0x7fbbc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:54.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.519+0000 7fbbcd7fa700 1 -- 172.21.15.63:0/2621375405 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbc400e040 con 0x7fbbd0101d70 2024-03-22T07:21:54.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.519+0000 7fbbcd7fa700 1 -- 172.21.15.63:0/2621375405 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbbc400bbc0 con 0x7fbbd0101d70 2024-03-22T07:21:54.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.519+0000 7fbbd7046700 1 -- 172.21.15.63:0/2621375405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd010dd10 con 0x7fbbd0101d70 2024-03-22T07:21:54.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.519+0000 7fbbcd7fa700 1 -- 172.21.15.63:0/2621375405 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbc401c3f0 con 0x7fbbd0101d70 2024-03-22T07:21:54.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.519+0000 7fbbd7046700 1 -- 172.21.15.63:0/2621375405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd010e2d0 con 0x7fbbd0101d70 2024-03-22T07:21:54.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.520+0000 7fbbd7046700 1 -- 172.21.15.63:0/2621375405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd00ff080 con 0x7fbbd0101d70 2024-03-22T07:21:54.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.520+0000 7fbbcd7fa700 1 -- 172.21.15.63:0/2621375405 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7fbbc4013420 con 0x7fbbd0101d70 2024-03-22T07:21:54.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.521+0000 7fbbcd7fa700 1 --2- 172.21.15.63:0/2621375405 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbbc006d4a0 0x7fbbc006f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:54.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.521+0000 7fbbcffff700 1 --2- 172.21.15.63:0/2621375405 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbbc006d4a0 0x7fbbc006f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:54.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.521+0000 7fbbcd7fa700 1 -- 172.21.15.63:0/2621375405 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbbc408de30 con 0x7fbbd0101d70 2024-03-22T07:21:54.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.521+0000 7fbbcffff700 1 --2- 172.21.15.63:0/2621375405 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbbc006d4a0 0x7fbbc006f950 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc006fd0 tx=0x7fbbbc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:54.526 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.526+0000 7fbbcd7fa700 1 -- 172.21.15.63:0/2621375405 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbbc405b220 con 0x7fbbd0101d70 2024-03-22T07:21:54.623 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:54 smithi063 podman[124251]: 2024-03-22 07:21:54.317504722 +0000 UTC m=+0.615012805 container remove 2cc82369e4bf776944552f3fa46f6827c6aa8eb88ca57c24bbebeb4689fcaa02 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-22T07:21:54.623 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:54 smithi063 podman[124315]: 2024-03-22 07:21:54.434615365 +0000 UTC m=+0.019577362 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:21:54.623 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:54 smithi063 podman[124315]: 2024-03-22 07:21:54.543536242 +0000 UTC m=+0.128498220 container create fdbc8b34ab78f500d2b4ca6183e7c2d10e5761e5d5bad0b88a265d676eed1fdc (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-22T07:21:54.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.751+0000 7fbbd7046700 1 -- 172.21.15.63:0/2621375405 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbd010f200 con 0x7fbbc006d4a0 2024-03-22T07:21:54.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.752+0000 7fbbcd7fa700 1 -- 172.21.15.63:0/2621375405 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbbd010f200 con 0x7fbbc006d4a0 2024-03-22T07:21:54.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.754+0000 7fbbd7046700 1 -- 172.21.15.63:0/2621375405 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbbc006d4a0 msgr2=0x7fbbc006f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:54.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.754+0000 7fbbd7046700 1 --2- 172.21.15.63:0/2621375405 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbbc006d4a0 0x7fbbc006f950 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc006fd0 tx=0x7fbbbc008040 comp rx=0 tx=0).stop 2024-03-22T07:21:54.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.754+0000 7fbbd7046700 1 -- 172.21.15.63:0/2621375405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbd0101d70 msgr2=0x7fbbd0113b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:54.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.754+0000 7fbbd7046700 1 --2- 172.21.15.63:0/2621375405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbd0101d70 0x7fbbd0113b10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400b3b0 tx=0x7fbbc4009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:54.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.754+0000 7fbbd7046700 1 -- 172.21.15.63:0/2621375405 shutdown_connections 2024-03-22T07:21:54.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.754+0000 7fbbd7046700 1 --2- 172.21.15.63:0/2621375405 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbbc006d4a0 0x7fbbc006f950 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:54.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.754+0000 7fbbd7046700 1 --2- 172.21.15.63:0/2621375405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbbd0101d70 0x7fbbd0113b10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:54.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.754+0000 7fbbd7046700 1 --2- 172.21.15.63:0/2621375405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbd0102710 0x7fbbd0114050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:54.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.754+0000 7fbbd7046700 1 -- 172.21.15.63:0/2621375405 >> 172.21.15.63:0/2621375405 conn(0x7fbbd0072bc0 msgr2=0x7fbbd00fae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:54.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.754+0000 7fbbd7046700 1 -- 172.21.15.63:0/2621375405 shutdown_connections 2024-03-22T07:21:54.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.754+0000 7fbbd7046700 1 -- 172.21.15.63:0/2621375405 wait complete. 2024-03-22T07:21:54.766 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:21:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.860+0000 7f475c29c700 1 -- 172.21.15.63:0/3711389883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47540fb540 msgr2=0x7f47540fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.860+0000 7f475c29c700 1 --2- 172.21.15.63:0/3711389883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47540fb540 0x7f47540fb990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4744004940 tx=0x7f4744009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.860+0000 7f475c29c700 1 -- 172.21.15.63:0/3711389883 shutdown_connections 2024-03-22T07:21:54.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.860+0000 7f475c29c700 1 --2- 172.21.15.63:0/3711389883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f47540fb540 0x7f47540fb990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.860+0000 7f475c29c700 1 --2- 172.21.15.63:0/3711389883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4754102fd0 0x7f47541033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.860+0000 7f475c29c700 1 -- 172.21.15.63:0/3711389883 >> 172.21.15.63:0/3711389883 conn(0x7f4754071530 msgr2=0x7f4754071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.860+0000 7f475c29c700 1 -- 172.21.15.63:0/3711389883 shutdown_connections 2024-03-22T07:21:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.860+0000 7f475c29c700 1 -- 172.21.15.63:0/3711389883 wait complete. 2024-03-22T07:21:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.861+0000 7f475c29c700 1 Processor -- start 2024-03-22T07:21:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.861+0000 7f475c29c700 1 -- start start 2024-03-22T07:21:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.861+0000 7f475c29c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47540fb540 0x7f47540fc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.861+0000 7f475c29c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4754102fd0 0x7f47540fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.861+0000 7f475c29c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47540fe670 con 0x7f4754102fd0 2024-03-22T07:21:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.861+0000 7f475c29c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47540fd1a0 con 0x7f47540fb540 2024-03-22T07:21:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.861+0000 7f4759837700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4754102fd0 0x7f47540fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.861+0000 7f4759837700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4754102fd0 0x7f47540fcc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39766/0 (socket says 172.21.15.63:39766) 2024-03-22T07:21:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.861+0000 7f4759837700 1 -- 172.21.15.63:0/2150751774 learned_addr learned my addr 172.21.15.63:0/2150751774 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:21:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.862+0000 7f475a038700 1 --2- 172.21.15.63:0/2150751774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47540fb540 0x7f47540fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:54.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.862+0000 7f4759837700 1 -- 172.21.15.63:0/2150751774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47540fb540 msgr2=0x7f47540fc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:54.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.862+0000 7f4759837700 1 --2- 172.21.15.63:0/2150751774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47540fb540 0x7f47540fc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:54.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.862+0000 7f4759837700 1 -- 172.21.15.63:0/2150751774 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4744005040 con 0x7f4754102fd0 2024-03-22T07:21:54.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.862+0000 7f4759837700 1 --2- 172.21.15.63:0/2150751774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4754102fd0 0x7f47540fcc60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4744009180 tx=0x7f4744009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:54.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.862+0000 7f474affd700 1 -- 172.21.15.63:0/2150751774 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f474400e040 con 0x7f4754102fd0 2024-03-22T07:21:54.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.862+0000 7f474affd700 1 -- 172.21.15.63:0/2150751774 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f474400bbc0 con 0x7f4754102fd0 2024-03-22T07:21:54.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.862+0000 7f475c29c700 1 -- 172.21.15.63:0/2150751774 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47540fd420 con 0x7f4754102fd0 2024-03-22T07:21:54.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.862+0000 7f474affd700 1 -- 172.21.15.63:0/2150751774 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f474401c630 con 0x7f4754102fd0 2024-03-22T07:21:54.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.862+0000 7f475c29c700 1 -- 172.21.15.63:0/2150751774 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475411a0a0 con 0x7f4754102fd0 2024-03-22T07:21:54.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.863+0000 7f475c29c700 1 -- 172.21.15.63:0/2150751774 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47541053b0 con 0x7f4754102fd0 2024-03-22T07:21:54.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.864+0000 7f474affd700 1 -- 172.21.15.63:0/2150751774 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f474400bd30 con 0x7f4754102fd0 2024-03-22T07:21:54.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.864+0000 7f474affd700 1 --2- 172.21.15.63:0/2150751774 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f474006d4b0 0x7f474006f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:54.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.864+0000 7f474affd700 1 -- 172.21.15.63:0/2150751774 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f474408df00 con 0x7f4754102fd0 2024-03-22T07:21:54.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.864+0000 7f475a038700 1 --2- 172.21.15.63:0/2150751774 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f474006d4b0 0x7f474006f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:54.866 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.864+0000 7f475a038700 1 --2- 172.21.15.63:0/2150751774 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f474006d4b0 0x7f474006f960 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4750007fd0 tx=0x7f4750009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:54.869 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:54.869+0000 7f474affd700 1 -- 172.21.15.63:0/2150751774 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4744057840 con 0x7f4754102fd0 2024-03-22T07:21:55.025 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:54 smithi063 podman[124315]: 2024-03-22 07:21:54.796295494 +0000 UTC m=+0.381257496 container init fdbc8b34ab78f500d2b4ca6183e7c2d10e5761e5d5bad0b88a265d676eed1fdc (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-22T07:21:55.025 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:54 smithi063 podman[124315]: 2024-03-22 07:21:54.799709589 +0000 UTC m=+0.384671576 container start fdbc8b34ab78f500d2b4ca6183e7c2d10e5761e5d5bad0b88a265d676eed1fdc (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-22T07:21:55.025 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:54 smithi063 ceph-osd[124362]: -- 172.21.15.63:0/2046707657 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d73f8271e0 con 0x55d73f7f9000 2024-03-22T07:21:55.025 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:54 smithi063 bash[124315]: fdbc8b34ab78f500d2b4ca6183e7c2d10e5761e5d5bad0b88a265d676eed1fdc 2024-03-22T07:21:55.025 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:54 smithi063 systemd[1]: Started Ceph osd.0 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:21:55.105 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.103+0000 7f475c29c700 1 -- 172.21.15.63:0/2150751774 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f475410a010 con 0x7f474006d4b0 2024-03-22T07:21:55.106 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.105+0000 7f474affd700 1 -- 172.21.15.63:0/2150751774 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f475410a010 con 0x7f474006d4b0 2024-03-22T07:21:55.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.107+0000 7f475c29c700 1 -- 172.21.15.63:0/2150751774 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f474006d4b0 msgr2=0x7f474006f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:55.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.107+0000 7f475c29c700 1 --2- 172.21.15.63:0/2150751774 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f474006d4b0 0x7f474006f960 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4750007fd0 tx=0x7f4750009040 comp rx=0 tx=0).stop 2024-03-22T07:21:55.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.107+0000 7f475c29c700 1 -- 172.21.15.63:0/2150751774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4754102fd0 msgr2=0x7f47540fcc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:55.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.107+0000 7f475c29c700 1 --2- 172.21.15.63:0/2150751774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4754102fd0 0x7f47540fcc60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4744009180 tx=0x7f4744009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:55.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.107+0000 7f475c29c700 1 -- 172.21.15.63:0/2150751774 shutdown_connections 2024-03-22T07:21:55.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.107+0000 7f475c29c700 1 --2- 172.21.15.63:0/2150751774 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f474006d4b0 0x7f474006f960 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:55.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.107+0000 7f475c29c700 1 --2- 172.21.15.63:0/2150751774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f47540fb540 0x7f47540fc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:55.108 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.107+0000 7f475c29c700 1 --2- 172.21.15.63:0/2150751774 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4754102fd0 0x7f47540fcc60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:55.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.107+0000 7f475c29c700 1 -- 172.21.15.63:0/2150751774 >> 172.21.15.63:0/2150751774 conn(0x7f4754071530 msgr2=0x7f47540fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:55.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.107+0000 7f475c29c700 1 -- 172.21.15.63:0/2150751774 shutdown_connections 2024-03-22T07:21:55.109 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.107+0000 7f475c29c700 1 -- 172.21.15.63:0/2150751774 wait complete. 2024-03-22T07:21:55.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.212+0000 7f8d1e258700 1 -- 172.21.15.63:0/527736038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d181026b0 msgr2=0x7f8d180fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:55.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.212+0000 7f8d1e258700 1 --2- 172.21.15.63:0/527736038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d181026b0 0x7f8d180fbd20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c004940 tx=0x7f8d0c009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:55.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.212+0000 7f8d1e258700 1 -- 172.21.15.63:0/527736038 shutdown_connections 2024-03-22T07:21:55.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.212+0000 7f8d1e258700 1 --2- 172.21.15.63:0/527736038 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d181026b0 0x7f8d180fbd20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:55.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.212+0000 7f8d1e258700 1 --2- 172.21.15.63:0/527736038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d18101d10 0x7f8d181020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:55.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.212+0000 7f8d1e258700 1 -- 172.21.15.63:0/527736038 >> 172.21.15.63:0/527736038 conn(0x7f8d180f7880 msgr2=0x7f8d180f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:55.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.212+0000 7f8d1e258700 1 -- 172.21.15.63:0/527736038 shutdown_connections 2024-03-22T07:21:55.214 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.212+0000 7f8d1e258700 1 -- 172.21.15.63:0/527736038 wait complete. 2024-03-22T07:21:55.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.213+0000 7f8d1e258700 1 Processor -- start 2024-03-22T07:21:55.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.213+0000 7f8d1e258700 1 -- start start 2024-03-22T07:21:55.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.213+0000 7f8d1e258700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d18101d10 0x7f8d1810efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:55.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.213+0000 7f8d1e258700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d181026b0 0x7f8d1810f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:55.215 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.213+0000 7f8d1e258700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1810fa30 con 0x7f8d18101d10 2024-03-22T07:21:55.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.213+0000 7f8d1e258700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1810fb70 con 0x7f8d181026b0 2024-03-22T07:21:55.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.213+0000 7f8d177fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d18101d10 0x7f8d1810efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:55.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.213+0000 7f8d177fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d18101d10 0x7f8d1810efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39780/0 (socket says 172.21.15.63:39780) 2024-03-22T07:21:55.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.213+0000 7f8d177fe700 1 -- 172.21.15.63:0/2633829427 learned_addr learned my addr 172.21.15.63:0/2633829427 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:21:55.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.213+0000 7f8d16ffd700 1 --2- 172.21.15.63:0/2633829427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d181026b0 0x7f8d1810f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:55.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.214+0000 7f8d177fe700 1 -- 172.21.15.63:0/2633829427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d181026b0 msgr2=0x7f8d1810f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:55.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.214+0000 7f8d177fe700 1 --2- 172.21.15.63:0/2633829427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d181026b0 0x7f8d1810f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:55.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.214+0000 7f8d177fe700 1 -- 172.21.15.63:0/2633829427 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c005040 con 0x7f8d18101d10 2024-03-22T07:21:55.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.214+0000 7f8d177fe700 1 --2- 172.21.15.63:0/2633829427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d18101d10 0x7f8d1810efb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800c830 tx=0x7f8d08004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:55.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.214+0000 7f8cfffff700 1 -- 172.21.15.63:0/2633829427 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d080106f0 con 0x7f8d18101d10 2024-03-22T07:21:55.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.214+0000 7f8cfffff700 1 -- 172.21.15.63:0/2633829427 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d08010d20 con 0x7f8d18101d10 2024-03-22T07:21:55.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.214+0000 7f8d1e258700 1 -- 172.21.15.63:0/2633829427 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d18117e60 con 0x7f8d18101d10 2024-03-22T07:21:55.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.214+0000 7f8cfffff700 1 -- 172.21.15.63:0/2633829427 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d080113a0 con 0x7f8d18101d10 2024-03-22T07:21:55.216 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.214+0000 7f8d1e258700 1 -- 172.21.15.63:0/2633829427 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d181183f0 con 0x7f8d18101d10 2024-03-22T07:21:55.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.215+0000 7f8d1e258700 1 -- 172.21.15.63:0/2633829427 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d180fd460 con 0x7f8d18101d10 2024-03-22T07:21:55.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.216+0000 7f8cfffff700 1 -- 172.21.15.63:0/2633829427 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f8d08012400 con 0x7f8d18101d10 2024-03-22T07:21:55.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.216+0000 7f8cfffff700 1 --2- 172.21.15.63:0/2633829427 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f8d0006d4a0 0x7f8d0006f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:55.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.216+0000 7f8cfffff700 1 -- 172.21.15.63:0/2633829427 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8d0808c9e0 con 0x7f8d18101d10 2024-03-22T07:21:55.217 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.216+0000 7f8d16ffd700 1 --2- 172.21.15.63:0/2633829427 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f8d0006d4a0 0x7f8d0006f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:55.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.217+0000 7f8d16ffd700 1 --2- 172.21.15.63:0/2633829427 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f8d0006d4a0 0x7f8d0006f950 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8d180668f0 tx=0x7f8d0c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:55.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.221+0000 7f8cfffff700 1 -- 172.21.15.63:0/2633829427 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8d08056430 con 0x7f8d18101d10 2024-03-22T07:21:55.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:55 smithi069 ceph-mon[83631]: pgmap v46: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 69/354 objects degraded (19.492%) 2024-03-22T07:21:55.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:55 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:55.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:55 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:55.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:55 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:21:55.337 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:55 smithi063 ceph-mon[112784]: pgmap v46: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 69/354 objects degraded (19.492%) 2024-03-22T07:21:55.338 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:55 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:55.338 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:55 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:21:55.338 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:55 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:21:55.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.440+0000 7f8d1e258700 1 -- 172.21.15.63:0/2633829427 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d1804bed0 con 0x7f8d0006d4a0 2024-03-22T07:21:55.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.450+0000 7f8cfffff700 1 -- 172.21.15.63:0/2633829427 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8d1804bed0 con 0x7f8d0006d4a0 2024-03-22T07:21:55.451 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:21:55.452 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (2m) 16s ago 13m 26.6M - 0.25.0 c8568f914cd2 06ef21da90e5 2024-03-22T07:21:55.452 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (26s) 16s ago 13m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 224c291ab774 2024-03-22T07:21:55.452 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (21s) 17s ago 12m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 7eabb7d42631 2024-03-22T07:21:55.452 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (2m) 16s ago 13m 81.0M - 9.4.12 a0ea5e3c46d9 9015fae0d5e9 2024-03-22T07:21:55.452 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (8m) 16s ago 8m 74.1M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:21:55.452 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (8m) 16s ago 8m 124M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:21:55.452 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (8m) 17s ago 8m 63.0M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:21:55.452 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (8m) 17s ago 8m 129M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:21:55.452 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:8443,9283,8765 running (4m) 16s ago 15m 497M - 19.0.0-1667-gdb0330b1 4f325f8440e9 e49468f39991 2024-03-22T07:21:55.452 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283,8765 running (3m) 17s ago 12m 448M - 19.0.0-1667-gdb0330b1 4f325f8440e9 13c3f034ebb2 2024-03-22T07:21:55.452 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (93s) 16s ago 15m 43.1M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ab96538a07c1 2024-03-22T07:21:55.452 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (67s) 17s ago 12m 35.5M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 01a26e1fb1a1 2024-03-22T07:21:55.452 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (3m) 16s ago 13m 16.2M - 1.5.0 0da6a335fe13 0c825ee2a196 2024-03-22T07:21:55.452 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (3m) 17s ago 12m 16.9M - 1.5.0 0da6a335fe13 12edb2f35498 2024-03-22T07:21:55.452 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 starting - - - 1557M 2024-03-22T07:21:55.453 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (11m) 16s ago 11m 244M 1557M 17.2.7-726-g9c4e0c34 837124da804f fe1ace35c88a 2024-03-22T07:21:55.453 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (11m) 16s ago 11m 200M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4cd995265dcf 2024-03-22T07:21:55.453 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (10m) 17s ago 10m 281M 2581M 17.2.7-726-g9c4e0c34 837124da804f 45c9a759b76d 2024-03-22T07:21:55.453 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (10m) 17s ago 10m 241M 2581M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:21:55.453 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (9m) 17s ago 9m 240M 2581M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:21:55.453 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (3m) 16s ago 13m 61.1M - 2.43.0 a07b618ecd1d 1d89b928f6e7 2024-03-22T07:21:55.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.453+0000 7f8d1e258700 1 -- 172.21.15.63:0/2633829427 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f8d0006d4a0 msgr2=0x7f8d0006f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:55.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.453+0000 7f8d1e258700 1 --2- 172.21.15.63:0/2633829427 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f8d0006d4a0 0x7f8d0006f950 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8d180668f0 tx=0x7f8d0c009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:55.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.453+0000 7f8d1e258700 1 -- 172.21.15.63:0/2633829427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d18101d10 msgr2=0x7f8d1810efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:55.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.453+0000 7f8d1e258700 1 --2- 172.21.15.63:0/2633829427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d18101d10 0x7f8d1810efb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800c830 tx=0x7f8d08004470 comp rx=0 tx=0).stop 2024-03-22T07:21:55.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.453+0000 7f8d1e258700 1 -- 172.21.15.63:0/2633829427 shutdown_connections 2024-03-22T07:21:55.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.453+0000 7f8d1e258700 1 --2- 172.21.15.63:0/2633829427 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f8d0006d4a0 0x7f8d0006f950 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:55.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.453+0000 7f8d1e258700 1 --2- 172.21.15.63:0/2633829427 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8d18101d10 0x7f8d1810efb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:55.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.453+0000 7f8d1e258700 1 --2- 172.21.15.63:0/2633829427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d181026b0 0x7f8d1810f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:55.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.453+0000 7f8d1e258700 1 -- 172.21.15.63:0/2633829427 >> 172.21.15.63:0/2633829427 conn(0x7f8d180f7880 msgr2=0x7f8d180f8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:55.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.453+0000 7f8d1e258700 1 -- 172.21.15.63:0/2633829427 shutdown_connections 2024-03-22T07:21:55.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.453+0000 7f8d1e258700 1 -- 172.21.15.63:0/2633829427 wait complete. 2024-03-22T07:21:55.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.572+0000 7ff0bf9f2700 1 -- 172.21.15.63:0/1754008436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0b8101d70 msgr2=0x7ff0b8102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:55.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.572+0000 7ff0bf9f2700 1 --2- 172.21.15.63:0/1754008436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0b8101d70 0x7ff0b8102140 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4004940 tx=0x7ff0b4009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:55.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.572+0000 7ff0bf9f2700 1 -- 172.21.15.63:0/1754008436 shutdown_connections 2024-03-22T07:21:55.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.572+0000 7ff0bf9f2700 1 --2- 172.21.15.63:0/1754008436 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0b80fc5b0 0x7ff0b80fca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:55.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.572+0000 7ff0bf9f2700 1 --2- 172.21.15.63:0/1754008436 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0b8101d70 0x7ff0b8102140 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:55.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.572+0000 7ff0bf9f2700 1 -- 172.21.15.63:0/1754008436 >> 172.21.15.63:0/1754008436 conn(0x7ff0b80714c0 msgr2=0x7ff0b80718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:55.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.572+0000 7ff0bf9f2700 1 -- 172.21.15.63:0/1754008436 shutdown_connections 2024-03-22T07:21:55.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.572+0000 7ff0bf9f2700 1 -- 172.21.15.63:0/1754008436 wait complete. 2024-03-22T07:21:55.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.573+0000 7ff0bf9f2700 1 Processor -- start 2024-03-22T07:21:55.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.573+0000 7ff0bf9f2700 1 -- start start 2024-03-22T07:21:55.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.573+0000 7ff0bf9f2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0b80fc5b0 0x7ff0b810eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:55.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.573+0000 7ff0bf9f2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0b8101d70 0x7ff0b810f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:55.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.573+0000 7ff0bf9f2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b810f6e0 con 0x7ff0b80fc5b0 2024-03-22T07:21:55.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.573+0000 7ff0bf9f2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b810f820 con 0x7ff0b8101d70 2024-03-22T07:21:55.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.573+0000 7ff0bd78e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0b80fc5b0 0x7ff0b810eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:55.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.573+0000 7ff0bd78e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0b80fc5b0 0x7ff0b810eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39792/0 (socket says 172.21.15.63:39792) 2024-03-22T07:21:55.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.573+0000 7ff0bd78e700 1 -- 172.21.15.63:0/3947315058 learned_addr learned my addr 172.21.15.63:0/3947315058 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:21:55.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.573+0000 7ff0bcf8d700 1 --2- 172.21.15.63:0/3947315058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0b8101d70 0x7ff0b810f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:55.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.573+0000 7ff0bd78e700 1 -- 172.21.15.63:0/3947315058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0b8101d70 msgr2=0x7ff0b810f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:55.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.574+0000 7ff0bd78e700 1 --2- 172.21.15.63:0/3947315058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0b8101d70 0x7ff0b810f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:55.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.574+0000 7ff0bd78e700 1 -- 172.21.15.63:0/3947315058 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b4005040 con 0x7ff0b80fc5b0 2024-03-22T07:21:55.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.574+0000 7ff0bd78e700 1 --2- 172.21.15.63:0/3947315058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0b80fc5b0 0x7ff0b810eb50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009180 tx=0x7ff0b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:55.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.574+0000 7ff0adffb700 1 -- 172.21.15.63:0/3947315058 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0b400e040 con 0x7ff0b80fc5b0 2024-03-22T07:21:55.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.574+0000 7ff0adffb700 1 -- 172.21.15.63:0/3947315058 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0b400bbc0 con 0x7ff0b80fc5b0 2024-03-22T07:21:55.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.574+0000 7ff0bf9f2700 1 -- 172.21.15.63:0/3947315058 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b8117bc0 con 0x7ff0b80fc5b0 2024-03-22T07:21:55.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.574+0000 7ff0adffb700 1 -- 172.21.15.63:0/3947315058 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0b401c3f0 con 0x7ff0b80fc5b0 2024-03-22T07:21:55.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.574+0000 7ff0bf9f2700 1 -- 172.21.15.63:0/3947315058 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b81180d0 con 0x7ff0b80fc5b0 2024-03-22T07:21:55.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.575+0000 7ff0bf9f2700 1 -- 172.21.15.63:0/3947315058 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b80fd5c0 con 0x7ff0b80fc5b0 2024-03-22T07:21:55.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.576+0000 7ff0adffb700 1 -- 172.21.15.63:0/3947315058 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7ff0b400bd30 con 0x7ff0b80fc5b0 2024-03-22T07:21:55.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.576+0000 7ff0adffb700 1 --2- 172.21.15.63:0/3947315058 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff0a406d4a0 0x7ff0a406f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:55.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.577+0000 7ff0adffb700 1 -- 172.21.15.63:0/3947315058 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff0b408dd60 con 0x7ff0b80fc5b0 2024-03-22T07:21:55.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.577+0000 7ff0bcf8d700 1 --2- 172.21.15.63:0/3947315058 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff0a406d4a0 0x7ff0a406f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:55.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.577+0000 7ff0bcf8d700 1 --2- 172.21.15.63:0/3947315058 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff0a406d4a0 0x7ff0a406f950 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8007fd0 tx=0x7ff0a8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:55.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.581+0000 7ff0adffb700 1 -- 172.21.15.63:0/3947315058 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff0b4057740 con 0x7ff0b80fc5b0 2024-03-22T07:21:55.623 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:55 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0[124353]: 2024-03-22T07:21:55.336+0000 7f5f0f6727c0 -1 Falling back to public interface 2024-03-22T07:21:55.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.877+0000 7ff0bf9f2700 1 -- 172.21.15.63:0/3947315058 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff0b804bed0 con 0x7ff0b80fc5b0 2024-03-22T07:21:55.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.878+0000 7ff0adffb700 1 -- 172.21.15.63:0/3947315058 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7ff0b405ae00 con 0x7ff0b80fc5b0 2024-03-22T07:21:55.879 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:21:55.879 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:21:55.879 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:21:55.879 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:21:55.879 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:21:55.879 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:21:55.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:21:55.880 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:21:55.880 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 5 2024-03-22T07:21:55.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:21:55.880 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:21:55.880 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:21:55.880 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:21:55.880 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:21:55.880 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 9, 2024-03-22T07:21:55.880 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-22T07:21:55.880 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:21:55.880 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:21:55.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.880+0000 7ff0bf9f2700 1 -- 172.21.15.63:0/3947315058 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff0a406d4a0 msgr2=0x7ff0a406f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:55.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.880+0000 7ff0bf9f2700 1 --2- 172.21.15.63:0/3947315058 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff0a406d4a0 0x7ff0a406f950 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8007fd0 tx=0x7ff0a8009040 comp rx=0 tx=0).stop 2024-03-22T07:21:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.880+0000 7ff0bf9f2700 1 -- 172.21.15.63:0/3947315058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0b80fc5b0 msgr2=0x7ff0b810eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.880+0000 7ff0bf9f2700 1 --2- 172.21.15.63:0/3947315058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0b80fc5b0 0x7ff0b810eb50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009180 tx=0x7ff0b4009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.880+0000 7ff0bf9f2700 1 -- 172.21.15.63:0/3947315058 shutdown_connections 2024-03-22T07:21:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.880+0000 7ff0bf9f2700 1 --2- 172.21.15.63:0/3947315058 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff0a406d4a0 0x7ff0a406f950 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.880+0000 7ff0bf9f2700 1 --2- 172.21.15.63:0/3947315058 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff0b80fc5b0 0x7ff0b810eb50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.880+0000 7ff0bf9f2700 1 --2- 172.21.15.63:0/3947315058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0b8101d70 0x7ff0b810f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.880+0000 7ff0bf9f2700 1 -- 172.21.15.63:0/3947315058 >> 172.21.15.63:0/3947315058 conn(0x7ff0b80714c0 msgr2=0x7ff0b8104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.881+0000 7ff0bf9f2700 1 -- 172.21.15.63:0/3947315058 shutdown_connections 2024-03-22T07:21:55.882 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.881+0000 7ff0bf9f2700 1 -- 172.21.15.63:0/3947315058 wait complete. 2024-03-22T07:21:55.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.989+0000 7f5bec63b700 1 -- 172.21.15.63:0/2060153379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be4102e10 msgr2=0x7f5be41031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:55.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.989+0000 7f5bec63b700 1 --2- 172.21.15.63:0/2060153379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be4102e10 0x7f5be41031e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5be0004940 tx=0x7f5be0009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:55.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.990+0000 7f5bec63b700 1 -- 172.21.15.63:0/2060153379 shutdown_connections 2024-03-22T07:21:55.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.990+0000 7f5bec63b700 1 --2- 172.21.15.63:0/2060153379 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5be41037b0 0x7f5be40fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:55.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.990+0000 7f5bec63b700 1 --2- 172.21.15.63:0/2060153379 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be4102e10 0x7f5be41031e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:55.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.990+0000 7f5bec63b700 1 -- 172.21.15.63:0/2060153379 >> 172.21.15.63:0/2060153379 conn(0x7f5be40f8980 msgr2=0x7f5be40fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:55.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.990+0000 7f5bec63b700 1 -- 172.21.15.63:0/2060153379 shutdown_connections 2024-03-22T07:21:55.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.990+0000 7f5bec63b700 1 -- 172.21.15.63:0/2060153379 wait complete. 2024-03-22T07:21:55.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.990+0000 7f5bec63b700 1 Processor -- start 2024-03-22T07:21:55.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.990+0000 7f5bec63b700 1 -- start start 2024-03-22T07:21:55.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.990+0000 7f5bec63b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5be4102e10 0x7f5be4115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:55.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.990+0000 7f5bec63b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be41037b0 0x7f5be4116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:55.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.990+0000 7f5bec63b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be4116950 con 0x7f5be41037b0 2024-03-22T07:21:55.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.990+0000 7f5bec63b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be410fd70 con 0x7f5be4102e10 2024-03-22T07:21:55.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.991+0000 7f5be9bd6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be41037b0 0x7f5be4116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:55.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.991+0000 7f5bea3d7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5be4102e10 0x7f5be4115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:55.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.991+0000 7f5be9bd6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be41037b0 0x7f5be4116300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39810/0 (socket says 172.21.15.63:39810) 2024-03-22T07:21:55.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.991+0000 7f5be9bd6700 1 -- 172.21.15.63:0/3100247918 learned_addr learned my addr 172.21.15.63:0/3100247918 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:21:55.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.991+0000 7f5bea3d7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5be4102e10 0x7f5be4115dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.63:60434/0 (socket says 172.21.15.63:60434) 2024-03-22T07:21:55.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.991+0000 7f5be9bd6700 1 -- 172.21.15.63:0/3100247918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5be4102e10 msgr2=0x7f5be4115dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:55.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.991+0000 7f5be9bd6700 1 --2- 172.21.15.63:0/3100247918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5be4102e10 0x7f5be4115dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:55.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.991+0000 7f5be9bd6700 1 -- 172.21.15.63:0/3100247918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be0005040 con 0x7f5be41037b0 2024-03-22T07:21:55.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.991+0000 7f5be9bd6700 1 --2- 172.21.15.63:0/3100247918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be41037b0 0x7f5be4116300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4002a50 tx=0x7f5bd4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:55.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.991+0000 7f5bdaffd700 1 -- 172.21.15.63:0/3100247918 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bd4007cc0 con 0x7f5be41037b0 2024-03-22T07:21:55.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.991+0000 7f5bdaffd700 1 -- 172.21.15.63:0/3100247918 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5bd400f040 con 0x7f5be41037b0 2024-03-22T07:21:55.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.991+0000 7f5bec63b700 1 -- 172.21.15.63:0/3100247918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be4110050 con 0x7f5be41037b0 2024-03-22T07:21:55.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.991+0000 7f5bdaffd700 1 -- 172.21.15.63:0/3100247918 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bd400d9c0 con 0x7f5be41037b0 2024-03-22T07:21:55.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.991+0000 7f5bec63b700 1 -- 172.21.15.63:0/3100247918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be4110610 con 0x7f5be41037b0 2024-03-22T07:21:55.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.992+0000 7f5bec63b700 1 -- 172.21.15.63:0/3100247918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be4106ab0 con 0x7f5be41037b0 2024-03-22T07:21:55.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.993+0000 7f5bdaffd700 1 -- 172.21.15.63:0/3100247918 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f5bd4009730 con 0x7f5be41037b0 2024-03-22T07:21:55.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.993+0000 7f5bdaffd700 1 --2- 172.21.15.63:0/3100247918 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f5bd006d4a0 0x7f5bd006f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:55.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.994+0000 7f5bea3d7700 1 --2- 172.21.15.63:0/3100247918 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f5bd006d4a0 0x7f5bd006f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:55.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.994+0000 7f5bdaffd700 1 -- 172.21.15.63:0/3100247918 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5bd408c7c0 con 0x7f5be41037b0 2024-03-22T07:21:55.995 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.994+0000 7f5bea3d7700 1 --2- 172.21.15.63:0/3100247918 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f5bd006d4a0 0x7f5bd006f950 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5be0004f90 tx=0x7f5be0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:56.002 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:55.998+0000 7f5bdaffd700 1 -- 172.21.15.63:0/3100247918 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5bd4059bb0 con 0x7f5be41037b0 2024-03-22T07:21:56.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:56 smithi063 ceph-mon[112784]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:21:56.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:56 smithi063 ceph-mon[112784]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:21:56.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:56 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3947315058' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:21:56.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.253+0000 7f5bec63b700 1 -- 172.21.15.63:0/3100247918 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5be4111470 con 0x7f5be41037b0 2024-03-22T07:21:56.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.253+0000 7f5bdaffd700 1 -- 172.21.15.63:0/3100247918 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1868 (secure 0 0 0) 0x7f5bd4059740 con 0x7f5be41037b0 2024-03-22T07:21:56.255 INFO:teuthology.orchestra.run.smithi063.stdout:e24 2024-03-22T07:21:56.255 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:21:56.255 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:21:56.255 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-03-22T07:21:56.255 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:21:56.255 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:21:56.255 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-03-22T07:21:56.255 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 24 2024-03-22T07:21:56.255 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-22T07:21:56.255 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-03-22T07:13:29.602905+0000 2024-03-22T07:21:56.255 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-03-22T07:20:24.253589+0000 2024-03-22T07:21:56.255 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-03-22T07:21:56.255 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-03-22T07:21:56.256 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-03-22T07:21:56.256 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-03-22T07:21:56.256 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-03-22T07:21:56.256 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-03-22T07:21:56.256 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-03-22T07:21:56.256 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-03-22T07:21:56.256 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-03-22T07:21:56.256 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:21:56.256 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 2 2024-03-22T07:21:56.256 INFO:teuthology.orchestra.run.smithi063.stdout:in 0,1 2024-03-22T07:21:56.256 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=14480,1=24285} 2024-03-22T07:21:56.256 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-03-22T07:21:56.256 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-03-22T07:21:56.256 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-03-22T07:21:56.257 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-03-22T07:21:56.257 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-03-22T07:21:56.257 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-03-22T07:21:56.257 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-03-22T07:21:56.257 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-03-22T07:21:56.257 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-03-22T07:21:56.257 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.xeelpk{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:21:56.257 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.oytapz{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3211595802,v1:172.21.15.63:6827/3211595802] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:21:56.257 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.wwtsjn{1:24285} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.69:6824/3798247699,v1:172.21.15.69:6825/3798247699] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:21:56.257 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.cijxah{1:24301} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.69:6826/3380719311,v1:172.21.15.69:6827/3380719311] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:21:56.257 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:21:56.257 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:21:56.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.255+0000 7f5bec63b700 1 -- 172.21.15.63:0/3100247918 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f5bd006d4a0 msgr2=0x7f5bd006f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:56.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.255+0000 7f5bec63b700 1 --2- 172.21.15.63:0/3100247918 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f5bd006d4a0 0x7f5bd006f950 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5be0004f90 tx=0x7f5be0009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:56.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.255+0000 7f5bec63b700 1 -- 172.21.15.63:0/3100247918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be41037b0 msgr2=0x7f5be4116300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:56.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.255+0000 7f5bec63b700 1 --2- 172.21.15.63:0/3100247918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be41037b0 0x7f5be4116300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4002a50 tx=0x7f5bd4004600 comp rx=0 tx=0).stop 2024-03-22T07:21:56.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.255+0000 7f5bec63b700 1 -- 172.21.15.63:0/3100247918 shutdown_connections 2024-03-22T07:21:56.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.255+0000 7f5bec63b700 1 --2- 172.21.15.63:0/3100247918 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f5bd006d4a0 0x7f5bd006f950 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:56.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.255+0000 7f5bec63b700 1 --2- 172.21.15.63:0/3100247918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5be4102e10 0x7f5be4115dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:56.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.255+0000 7f5bec63b700 1 --2- 172.21.15.63:0/3100247918 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5be41037b0 0x7f5be4116300 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:56.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.255+0000 7f5bec63b700 1 -- 172.21.15.63:0/3100247918 >> 172.21.15.63:0/3100247918 conn(0x7f5be40f8980 msgr2=0x7f5be4107400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:56.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.255+0000 7f5bec63b700 1 -- 172.21.15.63:0/3100247918 shutdown_connections 2024-03-22T07:21:56.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.255+0000 7f5bec63b700 1 -- 172.21.15.63:0/3100247918 wait complete. 2024-03-22T07:21:56.259 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 24 2024-03-22T07:21:56.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.374+0000 7f7fc6c14700 1 -- 172.21.15.63:0/3227491324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fc0102f70 msgr2=0x7f7fc0103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:56.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.374+0000 7f7fc6c14700 1 --2- 172.21.15.63:0/3227491324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fc0102f70 0x7f7fc0103340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4004940 tx=0x7f7fb4009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:56.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.374+0000 7f7fc6c14700 1 -- 172.21.15.63:0/3227491324 shutdown_connections 2024-03-22T07:21:56.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.374+0000 7f7fc6c14700 1 --2- 172.21.15.63:0/3227491324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fc00fb3b0 0x7f7fc00fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:56.376 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.375+0000 7f7fc6c14700 1 --2- 172.21.15.63:0/3227491324 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fc0102f70 0x7f7fc0103340 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:56.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.375+0000 7f7fc6c14700 1 -- 172.21.15.63:0/3227491324 >> 172.21.15.63:0/3227491324 conn(0x7f7fc0072310 msgr2=0x7f7fc0072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:56.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.375+0000 7f7fc6c14700 1 -- 172.21.15.63:0/3227491324 shutdown_connections 2024-03-22T07:21:56.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.375+0000 7f7fc6c14700 1 -- 172.21.15.63:0/3227491324 wait complete. 2024-03-22T07:21:56.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.375+0000 7f7fc6c14700 1 Processor -- start 2024-03-22T07:21:56.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.375+0000 7f7fc6c14700 1 -- start start 2024-03-22T07:21:56.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.376+0000 7f7fc6c14700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fc00fb3b0 0x7f7fc0113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:56.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.376+0000 7f7fc6c14700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fc0102f70 0x7f7fc0113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:56.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.376+0000 7f7fc6c14700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc01144b0 con 0x7f7fc0102f70 2024-03-22T07:21:56.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.376+0000 7f7fc6c14700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc010dab0 con 0x7f7fc00fb3b0 2024-03-22T07:21:56.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.376+0000 7f7fbffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fc0102f70 0x7f7fc0113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:56.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.376+0000 7f7fbffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fc0102f70 0x7f7fc0113e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39820/0 (socket says 172.21.15.63:39820) 2024-03-22T07:21:56.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.376+0000 7f7fbffff700 1 -- 172.21.15.63:0/3190899601 learned_addr learned my addr 172.21.15.63:0/3190899601 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:21:56.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.376+0000 7f7fc49b0700 1 --2- 172.21.15.63:0/3190899601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fc00fb3b0 0x7f7fc0113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:56.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.376+0000 7f7fbffff700 1 -- 172.21.15.63:0/3190899601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fc00fb3b0 msgr2=0x7f7fc0113920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:56.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.376+0000 7f7fbffff700 1 --2- 172.21.15.63:0/3190899601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fc00fb3b0 0x7f7fc0113920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:56.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.376+0000 7f7fbffff700 1 -- 172.21.15.63:0/3190899601 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb4005040 con 0x7f7fc0102f70 2024-03-22T07:21:56.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.376+0000 7f7fbffff700 1 --2- 172.21.15.63:0/3190899601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fc0102f70 0x7f7fc0113e60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00eb70 tx=0x7f7fac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:56.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.376+0000 7f7fbd7fa700 1 -- 172.21.15.63:0/3190899601 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fac007cc0 con 0x7f7fc0102f70 2024-03-22T07:21:56.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.377+0000 7f7fbd7fa700 1 -- 172.21.15.63:0/3190899601 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fac00f040 con 0x7f7fc0102f70 2024-03-22T07:21:56.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.377+0000 7f7fc6c14700 1 -- 172.21.15.63:0/3190899601 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc010dd30 con 0x7f7fc0102f70 2024-03-22T07:21:56.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.377+0000 7f7fbd7fa700 1 -- 172.21.15.63:0/3190899601 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fac00ca90 con 0x7f7fc0102f70 2024-03-22T07:21:56.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.377+0000 7f7fc6c14700 1 -- 172.21.15.63:0/3190899601 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc010e270 con 0x7f7fc0102f70 2024-03-22T07:21:56.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.378+0000 7f7fc6c14700 1 -- 172.21.15.63:0/3190899601 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc00fe6d0 con 0x7f7fc0102f70 2024-03-22T07:21:56.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.378+0000 7f7fbd7fa700 1 -- 172.21.15.63:0/3190899601 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f7fac00cbf0 con 0x7f7fc0102f70 2024-03-22T07:21:56.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.379+0000 7f7fbd7fa700 1 --2- 172.21.15.63:0/3190899601 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7fb006d4b0 0x7f7fb006f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:56.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.379+0000 7f7fbd7fa700 1 -- 172.21.15.63:0/3190899601 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7fac08c680 con 0x7f7fc0102f70 2024-03-22T07:21:56.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.379+0000 7f7fc49b0700 1 --2- 172.21.15.63:0/3190899601 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7fb006d4b0 0x7f7fb006f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:56.380 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.379+0000 7f7fc49b0700 1 --2- 172.21.15.63:0/3190899601 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7fb006d4b0 0x7f7fb006f960 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009b30 tx=0x7f7fb4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:56.384 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.383+0000 7f7fbd7fa700 1 -- 172.21.15.63:0/3190899601 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7fac059a70 con 0x7f7fc0102f70 2024-03-22T07:21:56.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:56 smithi069 ceph-mon[83631]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:21:56.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:56 smithi069 ceph-mon[83631]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:21:56.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:56 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3947315058' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:21:56.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.622+0000 7f7fc6c14700 1 -- 172.21.15.63:0/3190899601 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fc010a020 con 0x7f7fb006d4b0 2024-03-22T07:21:56.624 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.624+0000 7f7fbd7fa700 1 -- 172.21.15.63:0/3190899601 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7fc010a020 con 0x7f7fb006d4b0 2024-03-22T07:21:56.625 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:21:56.625 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:21:56.625 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:21:56.625 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:21:56.625 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-03-22T07:21:56.625 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-03-22T07:21:56.625 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr", 2024-03-22T07:21:56.625 INFO:teuthology.orchestra.run.smithi063.stdout: "mon" 2024-03-22T07:21:56.625 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-03-22T07:21:56.625 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "6/21 daemons upgraded", 2024-03-22T07:21:56.625 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T07:21:56.626 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:21:56.626 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:21:56.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.626+0000 7f7fc6c14700 1 -- 172.21.15.63:0/3190899601 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7fb006d4b0 msgr2=0x7f7fb006f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:56.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.626+0000 7f7fc6c14700 1 --2- 172.21.15.63:0/3190899601 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7fb006d4b0 0x7f7fb006f960 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009b30 tx=0x7f7fb4009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:56.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.626+0000 7f7fc6c14700 1 -- 172.21.15.63:0/3190899601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fc0102f70 msgr2=0x7f7fc0113e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:56.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.626+0000 7f7fc6c14700 1 --2- 172.21.15.63:0/3190899601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fc0102f70 0x7f7fc0113e60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00eb70 tx=0x7f7fac00b040 comp rx=0 tx=0).stop 2024-03-22T07:21:56.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.626+0000 7f7fc6c14700 1 -- 172.21.15.63:0/3190899601 shutdown_connections 2024-03-22T07:21:56.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.626+0000 7f7fc6c14700 1 --2- 172.21.15.63:0/3190899601 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7fb006d4b0 0x7f7fb006f960 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:56.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.626+0000 7f7fc6c14700 1 --2- 172.21.15.63:0/3190899601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fc00fb3b0 0x7f7fc0113920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:56.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.626+0000 7f7fc6c14700 1 --2- 172.21.15.63:0/3190899601 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7fc0102f70 0x7f7fc0113e60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:56.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.626+0000 7f7fc6c14700 1 -- 172.21.15.63:0/3190899601 >> 172.21.15.63:0/3190899601 conn(0x7f7fc0072310 msgr2=0x7f7fc00f9580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:56.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.626+0000 7f7fc6c14700 1 -- 172.21.15.63:0/3190899601 shutdown_connections 2024-03-22T07:21:56.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.626+0000 7f7fc6c14700 1 -- 172.21.15.63:0/3190899601 wait complete. 2024-03-22T07:21:56.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.733+0000 7f201fe76700 1 -- 172.21.15.63:0/4167786489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20181037b0 msgr2=0x7f20180fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:56.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.733+0000 7f201fe76700 1 --2- 172.21.15.63:0/4167786489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20181037b0 0x7f20180fce20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2008004940 tx=0x7f2008009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:56.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.733+0000 7f201fe76700 1 -- 172.21.15.63:0/4167786489 shutdown_connections 2024-03-22T07:21:56.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.733+0000 7f201fe76700 1 --2- 172.21.15.63:0/4167786489 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20181037b0 0x7f20180fce20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:56.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.733+0000 7f201fe76700 1 --2- 172.21.15.63:0/4167786489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2018102e10 0x7f20181031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:56.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.733+0000 7f201fe76700 1 -- 172.21.15.63:0/4167786489 >> 172.21.15.63:0/4167786489 conn(0x7f20180f8980 msgr2=0x7f20180fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:56.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.733+0000 7f201fe76700 1 -- 172.21.15.63:0/4167786489 shutdown_connections 2024-03-22T07:21:56.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.734+0000 7f201fe76700 1 -- 172.21.15.63:0/4167786489 wait complete. 2024-03-22T07:21:56.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.734+0000 7f201fe76700 1 Processor -- start 2024-03-22T07:21:56.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.734+0000 7f201fe76700 1 -- start start 2024-03-22T07:21:56.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.734+0000 7f201fe76700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2018102e10 0x7f2018113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:56.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.734+0000 7f201fe76700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20181037b0 0x7f20181140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:56.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.734+0000 7f201fe76700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2018114700 con 0x7f20181037b0 2024-03-22T07:21:56.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.734+0000 7f201fe76700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201810db40 con 0x7f2018102e10 2024-03-22T07:21:56.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.734+0000 7f201d411700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20181037b0 0x7f20181140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:56.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.734+0000 7f201d411700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20181037b0 0x7f20181140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:39840/0 (socket says 172.21.15.63:39840) 2024-03-22T07:21:56.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.735+0000 7f201d411700 1 -- 172.21.15.63:0/1752580166 learned_addr learned my addr 172.21.15.63:0/1752580166 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:21:56.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.735+0000 7f201dc12700 1 --2- 172.21.15.63:0/1752580166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2018102e10 0x7f2018113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:56.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.735+0000 7f201d411700 1 -- 172.21.15.63:0/1752580166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2018102e10 msgr2=0x7f2018113b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:56.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.735+0000 7f201d411700 1 --2- 172.21.15.63:0/1752580166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2018102e10 0x7f2018113b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:56.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.735+0000 7f201d411700 1 -- 172.21.15.63:0/1752580166 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2008005040 con 0x7f20181037b0 2024-03-22T07:21:56.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.735+0000 7f201dc12700 1 --2- 172.21.15.63:0/1752580166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2018102e10 0x7f2018113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:21:56.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.735+0000 7f201d411700 1 --2- 172.21.15.63:0/1752580166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20181037b0 0x7f20181140b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2008009100 tx=0x7f2008009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:56.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.735+0000 7f200e7fc700 1 -- 172.21.15.63:0/1752580166 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f200800e040 con 0x7f20181037b0 2024-03-22T07:21:56.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.735+0000 7f201fe76700 1 -- 172.21.15.63:0/1752580166 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f201810ddc0 con 0x7f20181037b0 2024-03-22T07:21:56.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.735+0000 7f200e7fc700 1 -- 172.21.15.63:0/1752580166 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2008004290 con 0x7f20181037b0 2024-03-22T07:21:56.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.736+0000 7f201fe76700 1 -- 172.21.15.63:0/1752580166 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f201810e380 con 0x7f20181037b0 2024-03-22T07:21:56.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.736+0000 7f200e7fc700 1 -- 172.21.15.63:0/1752580166 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f200801c3f0 con 0x7f20181037b0 2024-03-22T07:21:56.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.737+0000 7f200e7fc700 1 -- 172.21.15.63:0/1752580166 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 39) v1 ==== 91269+0+0 (secure 0 0 0) 0x7f200801c6b0 con 0x7f20181037b0 2024-03-22T07:21:56.739 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.737+0000 7f201fe76700 1 -- 172.21.15.63:0/1752580166 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20181052e0 con 0x7f20181037b0 2024-03-22T07:21:56.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.737+0000 7f200e7fc700 1 --2- 172.21.15.63:0/1752580166 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f200406d4a0 0x7f200406f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:21:56.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.737+0000 7f200e7fc700 1 -- 172.21.15.63:0/1752580166 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f200808de20 con 0x7f20181037b0 2024-03-22T07:21:56.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.737+0000 7f201dc12700 1 --2- 172.21.15.63:0/1752580166 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f200406d4a0 0x7f200406f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:21:56.740 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.738+0000 7f201dc12700 1 --2- 172.21.15.63:0/1752580166 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f200406d4a0 0x7f200406f950 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2014009fd0 tx=0x7f201400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:21:56.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:56.742+0000 7f200e7fc700 1 -- 172.21.15.63:0/1752580166 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f200805b300 con 0x7f20181037b0 2024-03-22T07:21:57.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:57.030+0000 7f201fe76700 1 -- 172.21.15.63:0/1752580166 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f20180fe560 con 0x7f20181037b0 2024-03-22T07:21:57.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:57.032+0000 7f200e7fc700 1 -- 172.21.15.63:0/1752580166 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1383 (secure 0 0 0) 0x7f200805ae90 con 0x7f20181037b0 2024-03-22T07:21:57.033 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 69/354 objects degraded (19.492%), 20 pgs degraded 2024-03-22T07:21:57.034 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-22T07:21:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: osd.0 (root=default,host=smithi063) is down 2024-03-22T07:21:57.034 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 69/354 objects degraded (19.492%), 20 pgs degraded 2024-03-22T07:21:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-03-22T07:21:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-03-22T07:21:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-03-22T07:21:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-03-22T07:21:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-03-22T07:21:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-03-22T07:21:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-03-22T07:21:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-03-22T07:21:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.c is active+undersized+degraded, acting [2,3] 2024-03-22T07:21:57.034 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-03-22T07:21:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-03-22T07:21:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-03-22T07:21:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.12 is active+undersized+degraded, acting [3,1] 2024-03-22T07:21:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-03-22T07:21:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-03-22T07:21:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-03-22T07:21:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-03-22T07:21:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-03-22T07:21:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-03-22T07:21:57.035 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-03-22T07:21:57.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:57.034+0000 7f201fe76700 1 -- 172.21.15.63:0/1752580166 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f200406d4a0 msgr2=0x7f200406f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:57.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:57.034+0000 7f201fe76700 1 --2- 172.21.15.63:0/1752580166 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f200406d4a0 0x7f200406f950 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2014009fd0 tx=0x7f201400b040 comp rx=0 tx=0).stop 2024-03-22T07:21:57.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:57.034+0000 7f201fe76700 1 -- 172.21.15.63:0/1752580166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20181037b0 msgr2=0x7f20181140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:21:57.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:57.034+0000 7f201fe76700 1 --2- 172.21.15.63:0/1752580166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20181037b0 0x7f20181140b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2008009100 tx=0x7f2008009e30 comp rx=0 tx=0).stop 2024-03-22T07:21:57.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:57.034+0000 7f201fe76700 1 -- 172.21.15.63:0/1752580166 shutdown_connections 2024-03-22T07:21:57.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:57.034+0000 7f201fe76700 1 --2- 172.21.15.63:0/1752580166 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f200406d4a0 0x7f200406f950 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:57.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:57.034+0000 7f201fe76700 1 --2- 172.21.15.63:0/1752580166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2018102e10 0x7f2018113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:57.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:57.034+0000 7f201fe76700 1 --2- 172.21.15.63:0/1752580166 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f20181037b0 0x7f20181140b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:21:57.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:57.034+0000 7f201fe76700 1 -- 172.21.15.63:0/1752580166 >> 172.21.15.63:0/1752580166 conn(0x7f20180f8980 msgr2=0x7f20181021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:21:57.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:57.034+0000 7f201fe76700 1 -- 172.21.15.63:0/1752580166 shutdown_connections 2024-03-22T07:21:57.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:21:57.034+0000 7f201fe76700 1 -- 172.21.15.63:0/1752580166 wait complete. 2024-03-22T07:21:57.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:57 smithi063 ceph-mon[112784]: pgmap v47: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 69/354 objects degraded (19.492%) 2024-03-22T07:21:57.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:57 smithi063 ceph-mon[112784]: from='client.34198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:21:57.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:57 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3100247918' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:21:57.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:57 smithi069 ceph-mon[83631]: pgmap v47: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 69/354 objects degraded (19.492%) 2024-03-22T07:21:57.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:57 smithi069 ceph-mon[83631]: from='client.34198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:21:57.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:57 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3100247918' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:21:58.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:58 smithi063 ceph-mon[112784]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:21:58.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:58 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/1752580166' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:21:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:58 smithi069 ceph-mon[83631]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:21:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:58 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/1752580166' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:21:59.276 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:21:59 smithi063 ceph-mon[112784]: pgmap v48: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 69/354 objects degraded (19.492%) 2024-03-22T07:21:59.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:21:59 smithi069 ceph-mon[83631]: pgmap v48: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 69/354 objects degraded (19.492%) 2024-03-22T07:21:59.623 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:59 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0[124353]: 2024-03-22T07:21:59.274+0000 7f5f0f6727c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-03-22T07:22:00.010 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:21:59 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0[124353]: 2024-03-22T07:21:59.853+0000 7f5f0f6727c0 -1 osd.0 50 log_to_monitors true 2024-03-22T07:22:00.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:00 smithi063 ceph-mon[112784]: from='osd.0 [v2:172.21.15.63:6802/108390664,v1:172.21.15.63:6803/108390664]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T07:22:00.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:00 smithi069 ceph-mon[83631]: from='osd.0 [v2:172.21.15.63:6802/108390664,v1:172.21.15.63:6803/108390664]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-22T07:22:01.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:01 smithi069 ceph-mon[83631]: pgmap v49: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 69/354 objects degraded (19.492%) 2024-03-22T07:22:01.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:01 smithi069 ceph-mon[83631]: from='osd.0 [v2:172.21.15.63:6802/108390664,v1:172.21.15.63:6803/108390664]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T07:22:01.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:01 smithi069 ceph-mon[83631]: failed to encode map e53 with expected crc 2024-03-22T07:22:01.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:01 smithi069 ceph-mon[83631]: osdmap e53: 6 total, 5 up, 6 in 2024-03-22T07:22:01.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:01 smithi069 ceph-mon[83631]: from='osd.0 [v2:172.21.15.63:6802/108390664,v1:172.21.15.63:6803/108390664]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-03-22T07:22:01.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:01.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:01.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:01 smithi069 ceph-mon[83631]: from='osd.0 [v2:172.21.15.63:6802/108390664,v1:172.21.15.63:6803/108390664]' entity='osd.0' 2024-03-22T07:22:01.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:01 smithi069 ceph-mon[83631]: failed to encode map e53 with expected crc 2024-03-22T07:22:01.373 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:22:00 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0[124353]: 2024-03-22T07:22:00.926+0000 7f5efcac2700 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:22:01.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:01 smithi063 ceph-mon[112784]: pgmap v49: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 299 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 69/354 objects degraded (19.492%) 2024-03-22T07:22:01.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:01 smithi063 ceph-mon[112784]: from='osd.0 [v2:172.21.15.63:6802/108390664,v1:172.21.15.63:6803/108390664]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-22T07:22:01.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:01 smithi063 ceph-mon[112784]: failed to encode map e53 with expected crc 2024-03-22T07:22:01.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:01 smithi063 ceph-mon[112784]: osdmap e53: 6 total, 5 up, 6 in 2024-03-22T07:22:01.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:01 smithi063 ceph-mon[112784]: from='osd.0 [v2:172.21.15.63:6802/108390664,v1:172.21.15.63:6803/108390664]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-03-22T07:22:01.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:01.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:01.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:01 smithi063 ceph-mon[112784]: from='osd.0 [v2:172.21.15.63:6802/108390664,v1:172.21.15.63:6803/108390664]' entity='osd.0' 2024-03-22T07:22:01.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:01 smithi063 ceph-mon[112784]: failed to encode map e53 with expected crc 2024-03-22T07:22:02.531 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:02 smithi063 ceph-mon[112784]: pgmap v51: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 299 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 69/354 objects degraded (19.492%) 2024-03-22T07:22:02.532 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:02.532 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:22:02.532 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:02.532 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:02.532 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:02 smithi063 ceph-mon[112784]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:22:02.532 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:02 smithi063 ceph-mon[112784]: failed to encode map e54 with expected crc 2024-03-22T07:22:02.532 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:02 smithi063 ceph-mon[112784]: osd.0 [v2:172.21.15.63:6802/108390664,v1:172.21.15.63:6803/108390664] boot 2024-03-22T07:22:02.532 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:02 smithi063 ceph-mon[112784]: osdmap e54: 6 total, 6 up, 6 in 2024-03-22T07:22:02.532 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:22:02.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:02 smithi069 ceph-mon[83631]: pgmap v51: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 299 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 69/354 objects degraded (19.492%) 2024-03-22T07:22:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:22:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:02 smithi069 ceph-mon[83631]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:22:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:02 smithi069 ceph-mon[83631]: failed to encode map e54 with expected crc 2024-03-22T07:22:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:02 smithi069 ceph-mon[83631]: osd.0 [v2:172.21.15.63:6802/108390664,v1:172.21.15.63:6803/108390664] boot 2024-03-22T07:22:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:02 smithi069 ceph-mon[83631]: osdmap e54: 6 total, 6 up, 6 in 2024-03-22T07:22:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-22T07:22:04.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:03 smithi063 ceph-mon[112784]: mgrmap e40: smithi063.mnxayh(active, since 91s), standbys: smithi069.gfziwh 2024-03-22T07:22:04.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:03 smithi063 ceph-mon[112784]: failed to encode map e55 with expected crc 2024-03-22T07:22:04.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:03 smithi063 ceph-mon[112784]: osdmap e55: 6 total, 6 up, 6 in 2024-03-22T07:22:04.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:03 smithi063 ceph-mon[112784]: failed to encode map e55 with expected crc 2024-03-22T07:22:04.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:03 smithi063 ceph-mon[112784]: failed to encode map e55 with expected crc 2024-03-22T07:22:04.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:03 smithi063 ceph-mon[112784]: failed to encode map e55 with expected crc 2024-03-22T07:22:04.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:03 smithi063 ceph-mon[112784]: failed to encode map e55 with expected crc 2024-03-22T07:22:04.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:03 smithi063 ceph-mon[112784]: failed to encode map e55 with expected crc 2024-03-22T07:22:04.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:03 smithi069 ceph-mon[83631]: mgrmap e40: smithi063.mnxayh(active, since 91s), standbys: smithi069.gfziwh 2024-03-22T07:22:04.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:03 smithi069 ceph-mon[83631]: failed to encode map e55 with expected crc 2024-03-22T07:22:04.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:03 smithi069 ceph-mon[83631]: osdmap e55: 6 total, 6 up, 6 in 2024-03-22T07:22:04.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:03 smithi069 ceph-mon[83631]: failed to encode map e55 with expected crc 2024-03-22T07:22:04.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:03 smithi069 ceph-mon[83631]: failed to encode map e55 with expected crc 2024-03-22T07:22:04.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:03 smithi069 ceph-mon[83631]: failed to encode map e55 with expected crc 2024-03-22T07:22:04.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:03 smithi069 ceph-mon[83631]: failed to encode map e55 with expected crc 2024-03-22T07:22:04.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:03 smithi069 ceph-mon[83631]: failed to encode map e55 with expected crc 2024-03-22T07:22:05.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:04 smithi063 ceph-mon[112784]: pgmap v54: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 299 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 4 op/s; 12/354 objects degraded (3.390%) 2024-03-22T07:22:05.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:04 smithi063 ceph-mon[112784]: Health check update: Degraded data redundancy: 12/354 objects degraded (3.390%), 3 pgs degraded (PG_DEGRADED) 2024-03-22T07:22:05.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:04 smithi069 ceph-mon[83631]: pgmap v54: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 299 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 4 op/s; 12/354 objects degraded (3.390%) 2024-03-22T07:22:05.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:04 smithi069 ceph-mon[83631]: Health check update: Degraded data redundancy: 12/354 objects degraded (3.390%), 3 pgs degraded (PG_DEGRADED) 2024-03-22T07:22:06.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:06 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:06.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:06 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:06.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:06 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:22:06.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:06 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:22:06.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:06 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:06.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:06 smithi063 ceph-mon[112784]: pgmap v55: 65 pgs: 25 peering, 40 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-22T07:22:06.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:06 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:22:06.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:06 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:22:06.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:06 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:22:06.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:06 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:22:06.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:06 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T07:22:06.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:06 smithi063 ceph-mon[112784]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T07:22:06.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:06 smithi063 ceph-mon[112784]: Upgrade: unsafe to stop osd(s) at this time (13 PGs are or would become offline) 2024-03-22T07:22:06.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:06 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:06.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:06 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:06.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:06 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:22:06.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:06 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:22:06.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:06 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:06.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:06 smithi069 ceph-mon[83631]: pgmap v55: 65 pgs: 25 peering, 40 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-22T07:22:06.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:06 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:22:06.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:06 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:22:06.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:06 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:22:06.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:06 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:22:06.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:06 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T07:22:06.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:06 smithi069 ceph-mon[83631]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T07:22:06.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:06 smithi069 ceph-mon[83631]: Upgrade: unsafe to stop osd(s) at this time (13 PGs are or would become offline) 2024-03-22T07:22:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:07 smithi063 ceph-mon[112784]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/354 objects degraded (3.390%), 3 pgs degraded) 2024-03-22T07:22:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:07 smithi063 ceph-mon[112784]: Cluster is now healthy 2024-03-22T07:22:07.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:07 smithi069 ceph-mon[83631]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/354 objects degraded (3.390%), 3 pgs degraded) 2024-03-22T07:22:07.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:07 smithi069 ceph-mon[83631]: Cluster is now healthy 2024-03-22T07:22:08.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:08 smithi069 ceph-mon[83631]: pgmap v56: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 833 B/s wr, 5 op/s; 38 B/s, 0 objects/s recovering 2024-03-22T07:22:08.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:08 smithi063 ceph-mon[112784]: pgmap v56: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 833 B/s wr, 5 op/s; 38 B/s, 0 objects/s recovering 2024-03-22T07:22:10.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:10 smithi069 ceph-mon[83631]: pgmap v57: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 5 op/s; 35 B/s, 0 objects/s recovering 2024-03-22T07:22:10.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:10 smithi063 ceph-mon[112784]: pgmap v57: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 5 op/s; 35 B/s, 0 objects/s recovering 2024-03-22T07:22:12.774 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:12 smithi063 ceph-mon[112784]: pgmap v58: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 647 B/s wr, 4 op/s; 29 B/s, 0 objects/s recovering 2024-03-22T07:22:12.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:12 smithi069 ceph-mon[83631]: pgmap v58: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 647 B/s wr, 4 op/s; 29 B/s, 0 objects/s recovering 2024-03-22T07:22:14.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:14 smithi069 ceph-mon[83631]: pgmap v59: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 26 B/s, 0 objects/s recovering 2024-03-22T07:22:14.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:14 smithi063 ceph-mon[112784]: pgmap v59: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 26 B/s, 0 objects/s recovering 2024-03-22T07:22:16.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:16 smithi069 ceph-mon[83631]: pgmap v60: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 23 B/s, 0 objects/s recovering 2024-03-22T07:22:16.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:16 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:16.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:16 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:22:16.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:16 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:16.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:16 smithi063 ceph-mon[112784]: pgmap v60: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 23 B/s, 0 objects/s recovering 2024-03-22T07:22:16.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:16 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:16.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:16 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:22:16.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:16 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:18.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:18 smithi069 ceph-mon[83631]: pgmap v61: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 23 B/s, 0 objects/s recovering 2024-03-22T07:22:18.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:18 smithi063 ceph-mon[112784]: pgmap v61: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 23 B/s, 0 objects/s recovering 2024-03-22T07:22:20.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:20 smithi069 ceph-mon[83631]: pgmap v62: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:22:20.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:20 smithi063 ceph-mon[112784]: pgmap v62: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:22:21.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:21 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T07:22:21.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:21 smithi069 ceph-mon[83631]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T07:22:21.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:21 smithi069 ceph-mon[83631]: Upgrade: osd.1 is safe to restart 2024-03-22T07:22:21.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:21 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T07:22:21.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:21 smithi063 ceph-mon[112784]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-22T07:22:21.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:21 smithi063 ceph-mon[112784]: Upgrade: osd.1 is safe to restart 2024-03-22T07:22:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:23 smithi063 ceph-mon[112784]: pgmap v63: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:22:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:23 smithi063 ceph-mon[112784]: Upgrade: Updating osd.1 2024-03-22T07:22:23.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:23 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:23.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:23 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T07:22:23.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:23 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:22:23.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:23 smithi063 ceph-mon[112784]: Deploying daemon osd.1 on smithi063 2024-03-22T07:22:23.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:23 smithi069 ceph-mon[83631]: pgmap v63: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:22:23.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:23 smithi069 ceph-mon[83631]: Upgrade: Updating osd.1 2024-03-22T07:22:23.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:23 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:23.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:23 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-22T07:22:23.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:23 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:22:23.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:23 smithi069 ceph-mon[83631]: Deploying daemon osd.1 on smithi063 2024-03-22T07:22:25.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:25 smithi063 ceph-mon[112784]: pgmap v64: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:22:25.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:25 smithi069 ceph-mon[83631]: pgmap v64: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:22:25.873 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:25 smithi063 systemd[1]: Stopping Ceph osd.1 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:22:25.873 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:25 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1[54588]: 2024-03-22T07:22:25.729+0000 7f81e2eee700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:22:25.873 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:25 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1[54588]: 2024-03-22T07:22:25.729+0000 7f81e2eee700 -1 osd.1 55 *** Got signal Terminated *** 2024-03-22T07:22:25.873 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:25 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1[54588]: 2024-03-22T07:22:25.729+0000 7f81e2eee700 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:22:26.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:26 smithi069 ceph-mon[83631]: osd.1 marked itself down and dead 2024-03-22T07:22:26.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:26 smithi063 ceph-mon[112784]: osd.1 marked itself down and dead 2024-03-22T07:22:26.966 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:26 smithi063 podman[129274]: 2024-03-22 07:22:26.651025056 +0000 UTC m=+1.023467618 container died fe1ace35c88a450a20a6764b5de7eab083412d2326ad9c5806d72b75ad20a76b (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=quincy-9c4e0c3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:22:27.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.153+0000 7fcea7899700 1 -- 172.21.15.63:0/1457330402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcea00fe830 msgr2=0x7fcea0106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:27.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.153+0000 7fcea7899700 1 --2- 172.21.15.63:0/1457330402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcea00fe830 0x7fcea0106f00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce90004940 tx=0x7fce90009e30 comp rx=0 tx=0).stop 2024-03-22T07:22:27.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.153+0000 7fcea7899700 1 -- 172.21.15.63:0/1457330402 shutdown_connections 2024-03-22T07:22:27.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.153+0000 7fcea7899700 1 --2- 172.21.15.63:0/1457330402 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcea00fe830 0x7fcea0106f00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:27.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.153+0000 7fcea7899700 1 --2- 172.21.15.63:0/1457330402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea00fde90 0x7fcea00fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:27.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.153+0000 7fcea7899700 1 -- 172.21.15.63:0/1457330402 >> 172.21.15.63:0/1457330402 conn(0x7fcea0072bc0 msgr2=0x7fcea0072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:22:27.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.153+0000 7fcea7899700 1 -- 172.21.15.63:0/1457330402 shutdown_connections 2024-03-22T07:22:27.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.154+0000 7fcea7899700 1 -- 172.21.15.63:0/1457330402 wait complete. 2024-03-22T07:22:27.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.154+0000 7fcea7899700 1 Processor -- start 2024-03-22T07:22:27.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.154+0000 7fcea7899700 1 -- start start 2024-03-22T07:22:27.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.154+0000 7fcea7899700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcea00fde90 0x7fcea0113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:27.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.154+0000 7fcea7899700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea00fe830 0x7fcea01140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:27.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.154+0000 7fcea7899700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea0114730 con 0x7fcea00fde90 2024-03-22T07:22:27.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.154+0000 7fcea7899700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea010db20 con 0x7fcea00fe830 2024-03-22T07:22:27.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.154+0000 7fcea5635700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcea00fde90 0x7fcea0113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:27.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.154+0000 7fcea5635700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcea00fde90 0x7fcea0113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53834/0 (socket says 172.21.15.63:53834) 2024-03-22T07:22:27.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.154+0000 7fcea5635700 1 -- 172.21.15.63:0/889477768 learned_addr learned my addr 172.21.15.63:0/889477768 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:22:27.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.155+0000 7fcea4e34700 1 --2- 172.21.15.63:0/889477768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea00fe830 0x7fcea01140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:27.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.155+0000 7fcea5635700 1 -- 172.21.15.63:0/889477768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea00fe830 msgr2=0x7fcea01140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:27.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.155+0000 7fcea5635700 1 --2- 172.21.15.63:0/889477768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea00fe830 0x7fcea01140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:27.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.155+0000 7fcea5635700 1 -- 172.21.15.63:0/889477768 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce90005040 con 0x7fcea00fde90 2024-03-22T07:22:27.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.155+0000 7fcea4e34700 1 --2- 172.21.15.63:0/889477768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea00fe830 0x7fcea01140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:22:27.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.155+0000 7fcea5635700 1 --2- 172.21.15.63:0/889477768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcea00fde90 0x7fcea0113ba0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fce9c002a50 tx=0x7fce9c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:22:27.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.155+0000 7fce95ffb700 1 -- 172.21.15.63:0/889477768 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce9c007cc0 con 0x7fcea00fde90 2024-03-22T07:22:27.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.155+0000 7fcea7899700 1 -- 172.21.15.63:0/889477768 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea010de00 con 0x7fcea00fde90 2024-03-22T07:22:27.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.155+0000 7fce95ffb700 1 -- 172.21.15.63:0/889477768 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce9c00f040 con 0x7fcea00fde90 2024-03-22T07:22:27.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.155+0000 7fce95ffb700 1 -- 172.21.15.63:0/889477768 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce9c009640 con 0x7fcea00fde90 2024-03-22T07:22:27.158 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.155+0000 7fcea7899700 1 -- 172.21.15.63:0/889477768 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea010e340 con 0x7fcea00fde90 2024-03-22T07:22:27.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.156+0000 7fcea7899700 1 -- 172.21.15.63:0/889477768 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea0065e30 con 0x7fcea00fde90 2024-03-22T07:22:27.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.157+0000 7fce95ffb700 1 -- 172.21.15.63:0/889477768 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fce9c0097a0 con 0x7fcea00fde90 2024-03-22T07:22:27.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.157+0000 7fce95ffb700 1 --2- 172.21.15.63:0/889477768 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fce8c06d380 0x7fce8c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:27.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.157+0000 7fce95ffb700 1 -- 172.21.15.63:0/889477768 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fce9c08c670 con 0x7fcea00fde90 2024-03-22T07:22:27.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.157+0000 7fcea4e34700 1 --2- 172.21.15.63:0/889477768 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fce8c06d380 0x7fce8c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:27.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.158+0000 7fcea4e34700 1 --2- 172.21.15.63:0/889477768 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fce8c06d380 0x7fce8c06f830 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fce9000b3b0 tx=0x7fce90009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:22:27.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.162+0000 7fce95ffb700 1 -- 172.21.15.63:0/889477768 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fce9c059c80 con 0x7fcea00fde90 2024-03-22T07:22:27.271 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:27 smithi063 podman[129274]: 2024-03-22 07:22:27.07099601 +0000 UTC m=+1.443438571 container cleanup fe1ace35c88a450a20a6764b5de7eab083412d2326ad9c5806d72b75ad20a76b (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=quincy-9c4e0c3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-22T07:22:27.271 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:27 smithi063 bash[129274]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1 2024-03-22T07:22:27.411 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.410+0000 7fcea7899700 1 -- 172.21.15.63:0/889477768 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcea0109f10 con 0x7fce8c06d380 2024-03-22T07:22:27.412 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.412+0000 7fce95ffb700 1 -- 172.21.15.63:0/889477768 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fcea0109f10 con 0x7fce8c06d380 2024-03-22T07:22:27.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.414+0000 7fcea7899700 1 -- 172.21.15.63:0/889477768 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fce8c06d380 msgr2=0x7fce8c06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:27.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.414+0000 7fcea7899700 1 --2- 172.21.15.63:0/889477768 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fce8c06d380 0x7fce8c06f830 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fce9000b3b0 tx=0x7fce90009e30 comp rx=0 tx=0).stop 2024-03-22T07:22:27.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.414+0000 7fcea7899700 1 -- 172.21.15.63:0/889477768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcea00fde90 msgr2=0x7fcea0113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:27.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.414+0000 7fcea7899700 1 --2- 172.21.15.63:0/889477768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcea00fde90 0x7fcea0113ba0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fce9c002a50 tx=0x7fce9c0046b0 comp rx=0 tx=0).stop 2024-03-22T07:22:27.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.414+0000 7fcea7899700 1 -- 172.21.15.63:0/889477768 shutdown_connections 2024-03-22T07:22:27.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.414+0000 7fcea7899700 1 --2- 172.21.15.63:0/889477768 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fce8c06d380 0x7fce8c06f830 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:27.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.414+0000 7fcea7899700 1 --2- 172.21.15.63:0/889477768 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcea00fde90 0x7fcea0113ba0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:27.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.414+0000 7fcea7899700 1 --2- 172.21.15.63:0/889477768 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcea00fe830 0x7fcea01140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:27.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.414+0000 7fcea7899700 1 -- 172.21.15.63:0/889477768 >> 172.21.15.63:0/889477768 conn(0x7fcea0072bc0 msgr2=0x7fcea01037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:22:27.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.414+0000 7fcea7899700 1 -- 172.21.15.63:0/889477768 shutdown_connections 2024-03-22T07:22:27.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.414+0000 7fcea7899700 1 -- 172.21.15.63:0/889477768 wait complete. 2024-03-22T07:22:27.428 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:22:27.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:27 smithi069 ceph-mon[83631]: pgmap v65: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:22:27.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:27 smithi069 ceph-mon[83631]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:22:27.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:27 smithi069 ceph-mon[83631]: failed to encode map e56 with expected crc 2024-03-22T07:22:27.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:27 smithi069 ceph-mon[83631]: failed to encode map e56 with expected crc 2024-03-22T07:22:27.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:27 smithi069 ceph-mon[83631]: failed to encode map e56 with expected crc 2024-03-22T07:22:27.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:27 smithi069 ceph-mon[83631]: failed to encode map e56 with expected crc 2024-03-22T07:22:27.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:27 smithi069 ceph-mon[83631]: osdmap e56: 6 total, 5 up, 6 in 2024-03-22T07:22:27.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.537+0000 7f1988d68700 1 -- 172.21.15.63:0/2071884973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19840fb540 msgr2=0x7f19840fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:27.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.537+0000 7f1988d68700 1 --2- 172.21.15.63:0/2071884973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19840fb540 0x7f19840fb990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1978004940 tx=0x7f1978009e30 comp rx=0 tx=0).stop 2024-03-22T07:22:27.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.538+0000 7f1988d68700 1 -- 172.21.15.63:0/2071884973 shutdown_connections 2024-03-22T07:22:27.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.538+0000 7f1988d68700 1 --2- 172.21.15.63:0/2071884973 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f19840fb540 0x7f19840fb990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:27.539 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:27 smithi063 podman[129293]: 2024-03-22 07:22:27.305910919 +0000 UTC m=+0.654502733 container remove fe1ace35c88a450a20a6764b5de7eab083412d2326ad9c5806d72b75ad20a76b (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-9c4e0c3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-22T07:22:27.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.538+0000 7f1988d68700 1 --2- 172.21.15.63:0/2071884973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1984102fd0 0x7f19841033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:27.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.538+0000 7f1988d68700 1 -- 172.21.15.63:0/2071884973 >> 172.21.15.63:0/2071884973 conn(0x7f1984071530 msgr2=0x7f1984071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:22:27.539 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:27 smithi063 ceph-mon[112784]: pgmap v65: 65 pgs: 65 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:22:27.540 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:27 smithi063 ceph-mon[112784]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:22:27.540 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:27 smithi063 ceph-mon[112784]: failed to encode map e56 with expected crc 2024-03-22T07:22:27.540 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:27 smithi063 ceph-mon[112784]: failed to encode map e56 with expected crc 2024-03-22T07:22:27.540 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:27 smithi063 ceph-mon[112784]: failed to encode map e56 with expected crc 2024-03-22T07:22:27.540 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:27 smithi063 ceph-mon[112784]: failed to encode map e56 with expected crc 2024-03-22T07:22:27.540 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:27 smithi063 ceph-mon[112784]: osdmap e56: 6 total, 5 up, 6 in 2024-03-22T07:22:27.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.538+0000 7f1988d68700 1 -- 172.21.15.63:0/2071884973 shutdown_connections 2024-03-22T07:22:27.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.538+0000 7f1988d68700 1 -- 172.21.15.63:0/2071884973 wait complete. 2024-03-22T07:22:27.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.539+0000 7f1988d68700 1 Processor -- start 2024-03-22T07:22:27.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.539+0000 7f1988d68700 1 -- start start 2024-03-22T07:22:27.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.539+0000 7f1988d68700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19840fb540 0x7f19840fc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:27.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.539+0000 7f1988d68700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1984102fd0 0x7f19840fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:27.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.539+0000 7f1988d68700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19840fe650 con 0x7f1984102fd0 2024-03-22T07:22:27.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.539+0000 7f1988d68700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19840fd1a0 con 0x7f19840fb540 2024-03-22T07:22:27.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.539+0000 7f198259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19840fb540 0x7f19840fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:27.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.539+0000 7f198259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19840fb540 0x7f19840fc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.63:59608/0 (socket says 172.21.15.63:59608) 2024-03-22T07:22:27.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.539+0000 7f198259c700 1 -- 172.21.15.63:0/2653135179 learned_addr learned my addr 172.21.15.63:0/2653135179 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:22:27.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.539+0000 7f1981d9b700 1 --2- 172.21.15.63:0/2653135179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1984102fd0 0x7f19840fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:27.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.539+0000 7f198259c700 1 -- 172.21.15.63:0/2653135179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1984102fd0 msgr2=0x7f19840fcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:27.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.539+0000 7f198259c700 1 --2- 172.21.15.63:0/2653135179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1984102fd0 0x7f19840fcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:27.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.539+0000 7f198259c700 1 -- 172.21.15.63:0/2653135179 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1978005040 con 0x7f19840fb540 2024-03-22T07:22:27.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.540+0000 7f198259c700 1 --2- 172.21.15.63:0/2653135179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19840fb540 0x7f19840fc720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1974002a50 tx=0x7f19740046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:22:27.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.540+0000 7f1972ffd700 1 -- 172.21.15.63:0/2653135179 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1974007cc0 con 0x7f19840fb540 2024-03-22T07:22:27.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.540+0000 7f1988d68700 1 -- 172.21.15.63:0/2653135179 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19840fd480 con 0x7f19840fb540 2024-03-22T07:22:27.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.540+0000 7f1972ffd700 1 -- 172.21.15.63:0/2653135179 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f197400f040 con 0x7f19840fb540 2024-03-22T07:22:27.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.540+0000 7f1972ffd700 1 -- 172.21.15.63:0/2653135179 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f197400da30 con 0x7f19840fb540 2024-03-22T07:22:27.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.540+0000 7f1988d68700 1 -- 172.21.15.63:0/2653135179 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198411a0d0 con 0x7f19840fb540 2024-03-22T07:22:27.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.541+0000 7f1988d68700 1 -- 172.21.15.63:0/2653135179 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19841053b0 con 0x7f19840fb540 2024-03-22T07:22:27.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.542+0000 7f1972ffd700 1 -- 172.21.15.63:0/2653135179 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f197400db90 con 0x7f19840fb540 2024-03-22T07:22:27.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.542+0000 7f1972ffd700 1 --2- 172.21.15.63:0/2653135179 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f196c06d380 0x7f196c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:27.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.542+0000 7f1972ffd700 1 -- 172.21.15.63:0/2653135179 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f197408c6c0 con 0x7f19840fb540 2024-03-22T07:22:27.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.542+0000 7f1981d9b700 1 --2- 172.21.15.63:0/2653135179 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f196c06d380 0x7f196c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:27.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.543+0000 7f1981d9b700 1 --2- 172.21.15.63:0/2653135179 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f196c06d380 0x7f196c06f830 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f19840f9de0 tx=0x7f1978009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:22:27.547 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.547+0000 7f1972ffd700 1 -- 172.21.15.63:0/2653135179 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1974056100 con 0x7f19840fb540 2024-03-22T07:22:27.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.780+0000 7f1988d68700 1 -- 172.21.15.63:0/2653135179 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f19840fe830 con 0x7f196c06d380 2024-03-22T07:22:27.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.781+0000 7f1972ffd700 1 -- 172.21.15.63:0/2653135179 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f19840fe830 con 0x7f196c06d380 2024-03-22T07:22:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.783+0000 7f1988d68700 1 -- 172.21.15.63:0/2653135179 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f196c06d380 msgr2=0x7f196c06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.783+0000 7f1988d68700 1 --2- 172.21.15.63:0/2653135179 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f196c06d380 0x7f196c06f830 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f19840f9de0 tx=0x7f1978009e30 comp rx=0 tx=0).stop 2024-03-22T07:22:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.783+0000 7f1988d68700 1 -- 172.21.15.63:0/2653135179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19840fb540 msgr2=0x7f19840fc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.783+0000 7f1988d68700 1 --2- 172.21.15.63:0/2653135179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19840fb540 0x7f19840fc720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1974002a50 tx=0x7f19740046b0 comp rx=0 tx=0).stop 2024-03-22T07:22:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.784+0000 7f1988d68700 1 -- 172.21.15.63:0/2653135179 shutdown_connections 2024-03-22T07:22:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.784+0000 7f1988d68700 1 --2- 172.21.15.63:0/2653135179 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f196c06d380 0x7f196c06f830 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.784+0000 7f1988d68700 1 --2- 172.21.15.63:0/2653135179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f19840fb540 0x7f19840fc720 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.784+0000 7f1988d68700 1 --2- 172.21.15.63:0/2653135179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1984102fd0 0x7f19840fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:27.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.784+0000 7f1988d68700 1 -- 172.21.15.63:0/2653135179 >> 172.21.15.63:0/2653135179 conn(0x7f1984071530 msgr2=0x7f19840fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:22:27.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.784+0000 7f1988d68700 1 -- 172.21.15.63:0/2653135179 shutdown_connections 2024-03-22T07:22:27.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.784+0000 7f1988d68700 1 -- 172.21.15.63:0/2653135179 wait complete. 2024-03-22T07:22:27.789 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:27 smithi063 podman[129431]: 2024-03-22 07:22:27.672120276 +0000 UTC m=+0.018122904 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:22:27.789 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:27 smithi063 podman[129431]: 2024-03-22 07:22:27.79033931 +0000 UTC m=+0.136341919 container create b916e5713e99dfae104479feace8f5adcd3af217d1621ad883898d1a731f9359 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-22T07:22:27.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.887+0000 7f26ef349700 1 -- 172.21.15.63:0/1577533942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e8101d10 msgr2=0x7f26e81020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:27.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.887+0000 7f26ef349700 1 --2- 172.21.15.63:0/1577533942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e8101d10 0x7f26e81020e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f26e4004940 tx=0x7f26e4009e30 comp rx=0 tx=0).stop 2024-03-22T07:22:27.889 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.887+0000 7f26ef349700 1 -- 172.21.15.63:0/1577533942 shutdown_connections 2024-03-22T07:22:27.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.887+0000 7f26ef349700 1 --2- 172.21.15.63:0/1577533942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26e81026b0 0x7f26e80fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:27.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.887+0000 7f26ef349700 1 --2- 172.21.15.63:0/1577533942 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e8101d10 0x7f26e81020e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:27.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.887+0000 7f26ef349700 1 -- 172.21.15.63:0/1577533942 >> 172.21.15.63:0/1577533942 conn(0x7f26e80f7880 msgr2=0x7f26e80f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:22:27.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.888+0000 7f26ef349700 1 -- 172.21.15.63:0/1577533942 shutdown_connections 2024-03-22T07:22:27.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.888+0000 7f26ef349700 1 -- 172.21.15.63:0/1577533942 wait complete. 2024-03-22T07:22:27.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.888+0000 7f26ef349700 1 Processor -- start 2024-03-22T07:22:27.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.888+0000 7f26ef349700 1 -- start start 2024-03-22T07:22:27.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.888+0000 7f26ef349700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26e8101d10 0x7f26e8106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:27.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.888+0000 7f26ef349700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e81026b0 0x7f26e81070f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:27.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.888+0000 7f26ef349700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e8107680 con 0x7f26e81026b0 2024-03-22T07:22:27.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.888+0000 7f26ef349700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e81077c0 con 0x7f26e8101d10 2024-03-22T07:22:27.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.888+0000 7f26ec8e4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e81026b0 0x7f26e81070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:27.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.888+0000 7f26ec8e4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e81026b0 0x7f26e81070f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53860/0 (socket says 172.21.15.63:53860) 2024-03-22T07:22:27.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.888+0000 7f26ec8e4700 1 -- 172.21.15.63:0/3746345078 learned_addr learned my addr 172.21.15.63:0/3746345078 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:22:27.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.889+0000 7f26ed0e5700 1 --2- 172.21.15.63:0/3746345078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26e8101d10 0x7f26e8106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:27.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.889+0000 7f26ec8e4700 1 -- 172.21.15.63:0/3746345078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26e8101d10 msgr2=0x7f26e8106bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:27.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.889+0000 7f26ec8e4700 1 --2- 172.21.15.63:0/3746345078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26e8101d10 0x7f26e8106bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:27.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.889+0000 7f26ec8e4700 1 -- 172.21.15.63:0/3746345078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e4005040 con 0x7f26e81026b0 2024-03-22T07:22:27.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.889+0000 7f26ec8e4700 1 --2- 172.21.15.63:0/3746345078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e81026b0 0x7f26e81070f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f26d8002a50 tx=0x7f26d8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:22:27.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.889+0000 7f26ddffb700 1 -- 172.21.15.63:0/3746345078 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26d8007cc0 con 0x7f26e81026b0 2024-03-22T07:22:27.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.889+0000 7f26ddffb700 1 -- 172.21.15.63:0/3746345078 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26d800f040 con 0x7f26e81026b0 2024-03-22T07:22:27.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.889+0000 7f26ef349700 1 -- 172.21.15.63:0/3746345078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e811a1a0 con 0x7f26e81026b0 2024-03-22T07:22:27.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.889+0000 7f26ddffb700 1 -- 172.21.15.63:0/3746345078 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26d800d9c0 con 0x7f26e81026b0 2024-03-22T07:22:27.891 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.889+0000 7f26ef349700 1 -- 172.21.15.63:0/3746345078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e811a620 con 0x7f26e81026b0 2024-03-22T07:22:27.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.890+0000 7f26ef349700 1 -- 172.21.15.63:0/3746345078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e80ff020 con 0x7f26e81026b0 2024-03-22T07:22:27.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.891+0000 7f26ddffb700 1 -- 172.21.15.63:0/3746345078 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f26d8009730 con 0x7f26e81026b0 2024-03-22T07:22:27.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.891+0000 7f26ddffb700 1 --2- 172.21.15.63:0/3746345078 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f26d406d380 0x7f26d406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:27.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.891+0000 7f26ddffb700 1 -- 172.21.15.63:0/3746345078 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f26d808c620 con 0x7f26e81026b0 2024-03-22T07:22:27.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.891+0000 7f26ed0e5700 1 --2- 172.21.15.63:0/3746345078 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f26d406d380 0x7f26d406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:27.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.891+0000 7f26ed0e5700 1 --2- 172.21.15.63:0/3746345078 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f26d406d380 0x7f26d406f830 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f26e4004f90 tx=0x7f26e4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:22:27.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:27.897+0000 7f26ddffb700 1 -- 172.21.15.63:0/3746345078 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f26d8056100 con 0x7f26e81026b0 2024-03-22T07:22:28.118 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:28 smithi063 podman[129431]: 2024-03-22 07:22:28.034631406 +0000 UTC m=+0.380634028 container init b916e5713e99dfae104479feace8f5adcd3af217d1621ad883898d1a731f9359 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-deactivate, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:22:28.118 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:28 smithi063 podman[129431]: 2024-03-22 07:22:28.037948075 +0000 UTC m=+0.383950695 container start b916e5713e99dfae104479feace8f5adcd3af217d1621ad883898d1a731f9359 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:22:28.131 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.131+0000 7f26ef349700 1 -- 172.21.15.63:0/3746345078 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f26e8109f00 con 0x7f26d406d380 2024-03-22T07:22:28.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.141+0000 7f26ddffb700 1 -- 172.21.15.63:0/3746345078 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f26e8109f00 con 0x7f26d406d380 2024-03-22T07:22:28.141 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:22:28.141 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (3m) 27s ago 14m 26.6M - 0.25.0 c8568f914cd2 06ef21da90e5 2024-03-22T07:22:28.142 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (59s) 27s ago 14m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 224c291ab774 2024-03-22T07:22:28.142 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (54s) 50s ago 13m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 7eabb7d42631 2024-03-22T07:22:28.142 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (3m) 27s ago 14m 83.1M - 9.4.12 a0ea5e3c46d9 9015fae0d5e9 2024-03-22T07:22:28.142 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (8m) 27s ago 8m 74.1M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:22:28.142 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (8m) 27s ago 8m 124M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:22:28.142 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (8m) 50s ago 8m 63.0M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:22:28.142 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (8m) 50s ago 8m 129M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:22:28.142 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:8443,9283,8765 running (4m) 27s ago 15m 499M - 19.0.0-1667-gdb0330b1 4f325f8440e9 e49468f39991 2024-03-22T07:22:28.142 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283,8765 running (4m) 50s ago 13m 448M - 19.0.0-1667-gdb0330b1 4f325f8440e9 13c3f034ebb2 2024-03-22T07:22:28.142 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (2m) 27s ago 15m 44.8M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ab96538a07c1 2024-03-22T07:22:28.142 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (100s) 50s ago 13m 35.5M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 01a26e1fb1a1 2024-03-22T07:22:28.142 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (4m) 27s ago 14m 16.4M - 1.5.0 0da6a335fe13 0c825ee2a196 2024-03-22T07:22:28.142 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (4m) 50s ago 13m 16.9M - 1.5.0 0da6a335fe13 12edb2f35498 2024-03-22T07:22:28.143 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (33s) 27s ago 12m 15.2M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 fdbc8b34ab78 2024-03-22T07:22:28.143 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (12m) 27s ago 12m 244M 1557M 17.2.7-726-g9c4e0c34 837124da804f fe1ace35c88a 2024-03-22T07:22:28.143 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (11m) 27s ago 11m 200M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4cd995265dcf 2024-03-22T07:22:28.143 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (11m) 50s ago 11m 281M 2581M 17.2.7-726-g9c4e0c34 837124da804f 45c9a759b76d 2024-03-22T07:22:28.143 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (10m) 50s ago 10m 241M 2581M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:22:28.143 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (10m) 50s ago 10m 240M 2581M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:22:28.143 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (3m) 27s ago 13m 61.1M - 2.43.0 a07b618ecd1d 1d89b928f6e7 2024-03-22T07:22:28.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.145+0000 7f26ef349700 1 -- 172.21.15.63:0/3746345078 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f26d406d380 msgr2=0x7f26d406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:28.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.145+0000 7f26ef349700 1 --2- 172.21.15.63:0/3746345078 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f26d406d380 0x7f26d406f830 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f26e4004f90 tx=0x7f26e4009e30 comp rx=0 tx=0).stop 2024-03-22T07:22:28.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.145+0000 7f26ef349700 1 -- 172.21.15.63:0/3746345078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e81026b0 msgr2=0x7f26e81070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:28.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.145+0000 7f26ef349700 1 --2- 172.21.15.63:0/3746345078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e81026b0 0x7f26e81070f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f26d8002a50 tx=0x7f26d8004600 comp rx=0 tx=0).stop 2024-03-22T07:22:28.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.145+0000 7f26ef349700 1 -- 172.21.15.63:0/3746345078 shutdown_connections 2024-03-22T07:22:28.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.145+0000 7f26ef349700 1 --2- 172.21.15.63:0/3746345078 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f26d406d380 0x7f26d406f830 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:28.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.145+0000 7f26ef349700 1 --2- 172.21.15.63:0/3746345078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26e8101d10 0x7f26e8106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:28.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.145+0000 7f26ef349700 1 --2- 172.21.15.63:0/3746345078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f26e81026b0 0x7f26e81070f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:28.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.145+0000 7f26ef349700 1 -- 172.21.15.63:0/3746345078 >> 172.21.15.63:0/3746345078 conn(0x7f26e80f7880 msgr2=0x7f26e81062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:22:28.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.145+0000 7f26ef349700 1 -- 172.21.15.63:0/3746345078 shutdown_connections 2024-03-22T07:22:28.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.145+0000 7f26ef349700 1 -- 172.21.15.63:0/3746345078 wait complete. 2024-03-22T07:22:28.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.252+0000 7fb9876d0700 1 -- 172.21.15.63:0/2689656728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9800fe830 msgr2=0x7fb980106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:28.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.252+0000 7fb9876d0700 1 --2- 172.21.15.63:0/2689656728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9800fe830 0x7fb980106f00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb970004940 tx=0x7fb970009e30 comp rx=0 tx=0).stop 2024-03-22T07:22:28.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.252+0000 7fb9876d0700 1 -- 172.21.15.63:0/2689656728 shutdown_connections 2024-03-22T07:22:28.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.252+0000 7fb9876d0700 1 --2- 172.21.15.63:0/2689656728 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9800fe830 0x7fb980106f00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:28.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.252+0000 7fb9876d0700 1 --2- 172.21.15.63:0/2689656728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9800fde90 0x7fb9800fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:28.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.252+0000 7fb9876d0700 1 -- 172.21.15.63:0/2689656728 >> 172.21.15.63:0/2689656728 conn(0x7fb980072bc0 msgr2=0x7fb980072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:22:28.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.252+0000 7fb9876d0700 1 -- 172.21.15.63:0/2689656728 shutdown_connections 2024-03-22T07:22:28.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.252+0000 7fb9876d0700 1 -- 172.21.15.63:0/2689656728 wait complete. 2024-03-22T07:22:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.253+0000 7fb9876d0700 1 Processor -- start 2024-03-22T07:22:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.253+0000 7fb9876d0700 1 -- start start 2024-03-22T07:22:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.253+0000 7fb9876d0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9800fde90 0x7fb980113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.253+0000 7fb9876d0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9800fe830 0x7fb9801140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.253+0000 7fb9876d0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb980114730 con 0x7fb9800fe830 2024-03-22T07:22:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.253+0000 7fb9876d0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98010db20 con 0x7fb9800fde90 2024-03-22T07:22:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.253+0000 7fb984c6b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9800fe830 0x7fb9801140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.254+0000 7fb984c6b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9800fe830 0x7fb9801140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53880/0 (socket says 172.21.15.63:53880) 2024-03-22T07:22:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.254+0000 7fb984c6b700 1 -- 172.21.15.63:0/2717085636 learned_addr learned my addr 172.21.15.63:0/2717085636 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:22:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.254+0000 7fb98546c700 1 --2- 172.21.15.63:0/2717085636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9800fde90 0x7fb980113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.254+0000 7fb984c6b700 1 -- 172.21.15.63:0/2717085636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9800fde90 msgr2=0x7fb980113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:28.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.254+0000 7fb984c6b700 1 --2- 172.21.15.63:0/2717085636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9800fde90 0x7fb980113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:28.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.254+0000 7fb984c6b700 1 -- 172.21.15.63:0/2717085636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb970005040 con 0x7fb9800fe830 2024-03-22T07:22:28.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.254+0000 7fb984c6b700 1 --2- 172.21.15.63:0/2717085636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9800fe830 0x7fb9801140e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb970009380 tx=0x7fb970009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:22:28.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.254+0000 7fb975ffb700 1 -- 172.21.15.63:0/2717085636 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb97000e040 con 0x7fb9800fe830 2024-03-22T07:22:28.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.254+0000 7fb975ffb700 1 -- 172.21.15.63:0/2717085636 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb970004290 con 0x7fb9800fe830 2024-03-22T07:22:28.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.254+0000 7fb9876d0700 1 -- 172.21.15.63:0/2717085636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98010dda0 con 0x7fb9800fe830 2024-03-22T07:22:28.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.254+0000 7fb975ffb700 1 -- 172.21.15.63:0/2717085636 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb97001c3f0 con 0x7fb9800fe830 2024-03-22T07:22:28.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.254+0000 7fb9876d0700 1 -- 172.21.15.63:0/2717085636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98010e2e0 con 0x7fb9800fe830 2024-03-22T07:22:28.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.255+0000 7fb9876d0700 1 -- 172.21.15.63:0/2717085636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb980065e30 con 0x7fb9800fe830 2024-03-22T07:22:28.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.256+0000 7fb975ffb700 1 -- 172.21.15.63:0/2717085636 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fb970013420 con 0x7fb9800fe830 2024-03-22T07:22:28.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.256+0000 7fb975ffb700 1 --2- 172.21.15.63:0/2717085636 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fb96c06d390 0x7fb96c06f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:28.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.256+0000 7fb975ffb700 1 -- 172.21.15.63:0/2717085636 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb97008dad0 con 0x7fb9800fe830 2024-03-22T07:22:28.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.256+0000 7fb98546c700 1 --2- 172.21.15.63:0/2717085636 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fb96c06d390 0x7fb96c06f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:28.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.257+0000 7fb98546c700 1 --2- 172.21.15.63:0/2717085636 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fb96c06d390 0x7fb96c06f840 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb97c004990 tx=0x7fb97c009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:22:28.265 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.265+0000 7fb975ffb700 1 -- 172.21.15.63:0/2717085636 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb97005b200 con 0x7fb9800fe830 2024-03-22T07:22:28.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:28 smithi063 ceph-mon[112784]: failed to encode map e57 with expected crc 2024-03-22T07:22:28.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:28 smithi063 ceph-mon[112784]: osdmap e57: 6 total, 5 up, 6 in 2024-03-22T07:22:28.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:28 smithi063 ceph-mon[112784]: failed to encode map e57 with expected crc 2024-03-22T07:22:28.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:28 smithi063 ceph-mon[112784]: failed to encode map e57 with expected crc 2024-03-22T07:22:28.369 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:28 smithi063 ceph-mon[112784]: failed to encode map e57 with expected crc 2024-03-22T07:22:28.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:28 smithi063 ceph-mon[112784]: failed to encode map e57 with expected crc 2024-03-22T07:22:28.370 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:28 smithi063 ceph-mon[112784]: failed to encode map e57 with expected crc 2024-03-22T07:22:28.370 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:28 smithi063 podman[129431]: 2024-03-22 07:22:28.118062448 +0000 UTC m=+0.464065068 container attach b916e5713e99dfae104479feace8f5adcd3af217d1621ad883898d1a731f9359 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-22T07:22:28.370 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:28 smithi063 podman[129431]: 2024-03-22 07:22:28.351988878 +0000 UTC m=+0.697991487 container died b916e5713e99dfae104479feace8f5adcd3af217d1621ad883898d1a731f9359 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-22T07:22:28.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.562+0000 7fb9876d0700 1 -- 172.21.15.63:0/2717085636 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb980109f10 con 0x7fb9800fe830 2024-03-22T07:22:28.563 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.563+0000 7fb975ffb700 1 -- 172.21.15.63:0/2717085636 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fb97005ad90 con 0x7fb9800fe830 2024-03-22T07:22:28.563 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:22:28.564 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:22:28.564 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:22:28.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:22:28.564 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:22:28.564 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:22:28.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:22:28.564 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:22:28.564 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4, 2024-03-22T07:22:28.564 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-22T07:22:28.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:22:28.564 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:22:28.564 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:22:28.564 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:22:28.565 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:22:28.565 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 8, 2024-03-22T07:22:28.565 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 5 2024-03-22T07:22:28.565 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:22:28.565 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:22:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.565+0000 7fb9876d0700 1 -- 172.21.15.63:0/2717085636 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fb96c06d390 msgr2=0x7fb96c06f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.565+0000 7fb9876d0700 1 --2- 172.21.15.63:0/2717085636 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fb96c06d390 0x7fb96c06f840 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb97c004990 tx=0x7fb97c009ee0 comp rx=0 tx=0).stop 2024-03-22T07:22:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.565+0000 7fb9876d0700 1 -- 172.21.15.63:0/2717085636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9800fe830 msgr2=0x7fb9801140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.565+0000 7fb9876d0700 1 --2- 172.21.15.63:0/2717085636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9800fe830 0x7fb9801140e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb970009380 tx=0x7fb970009e30 comp rx=0 tx=0).stop 2024-03-22T07:22:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.565+0000 7fb9876d0700 1 -- 172.21.15.63:0/2717085636 shutdown_connections 2024-03-22T07:22:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.565+0000 7fb9876d0700 1 --2- 172.21.15.63:0/2717085636 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fb96c06d390 0x7fb96c06f840 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.565+0000 7fb9876d0700 1 --2- 172.21.15.63:0/2717085636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb9800fde90 0x7fb980113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.565+0000 7fb9876d0700 1 --2- 172.21.15.63:0/2717085636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fb9800fe830 0x7fb9801140e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.565+0000 7fb9876d0700 1 -- 172.21.15.63:0/2717085636 >> 172.21.15.63:0/2717085636 conn(0x7fb980072bc0 msgr2=0x7fb9801037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:22:28.566 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.565+0000 7fb9876d0700 1 -- 172.21.15.63:0/2717085636 shutdown_connections 2024-03-22T07:22:28.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.565+0000 7fb9876d0700 1 -- 172.21.15.63:0/2717085636 wait complete. 2024-03-22T07:22:28.674 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.674+0000 7fbc510bf700 1 -- 172.21.15.63:0/1163257456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc4c1026c0 msgr2=0x7fbc4c0fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:28.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.674+0000 7fbc510bf700 1 --2- 172.21.15.63:0/1163257456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc4c1026c0 0x7fbc4c0fbd30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbc40004940 tx=0x7fbc40009e30 comp rx=0 tx=0).stop 2024-03-22T07:22:28.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.674+0000 7fbc510bf700 1 -- 172.21.15.63:0/1163257456 shutdown_connections 2024-03-22T07:22:28.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.674+0000 7fbc510bf700 1 --2- 172.21.15.63:0/1163257456 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc4c1026c0 0x7fbc4c0fbd30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:28.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.674+0000 7fbc510bf700 1 --2- 172.21.15.63:0/1163257456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc4c101d20 0x7fbc4c1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:28.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.674+0000 7fbc510bf700 1 -- 172.21.15.63:0/1163257456 >> 172.21.15.63:0/1163257456 conn(0x7fbc4c0f78b0 msgr2=0x7fbc4c0f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:22:28.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.674+0000 7fbc510bf700 1 -- 172.21.15.63:0/1163257456 shutdown_connections 2024-03-22T07:22:28.675 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.674+0000 7fbc510bf700 1 -- 172.21.15.63:0/1163257456 wait complete. 2024-03-22T07:22:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.675+0000 7fbc510bf700 1 Processor -- start 2024-03-22T07:22:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.675+0000 7fbc510bf700 1 -- start start 2024-03-22T07:22:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.675+0000 7fbc510bf700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc4c101d20 0x7fbc4c112870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.675+0000 7fbc510bf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc4c1026c0 0x7fbc4c112db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.675+0000 7fbc510bf700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c113400 con 0x7fbc4c101d20 2024-03-22T07:22:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.675+0000 7fbc510bf700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c10ca00 con 0x7fbc4c1026c0 2024-03-22T07:22:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.675+0000 7fbc4ad9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc4c101d20 0x7fbc4c112870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.675+0000 7fbc4ad9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc4c101d20 0x7fbc4c112870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53898/0 (socket says 172.21.15.63:53898) 2024-03-22T07:22:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.676+0000 7fbc4ad9d700 1 -- 172.21.15.63:0/3226754931 learned_addr learned my addr 172.21.15.63:0/3226754931 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:22:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.676+0000 7fbc4a59c700 1 --2- 172.21.15.63:0/3226754931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc4c1026c0 0x7fbc4c112db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.676+0000 7fbc4ad9d700 1 -- 172.21.15.63:0/3226754931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc4c1026c0 msgr2=0x7fbc4c112db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.676+0000 7fbc4ad9d700 1 --2- 172.21.15.63:0/3226754931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc4c1026c0 0x7fbc4c112db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:28.677 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.676+0000 7fbc4ad9d700 1 -- 172.21.15.63:0/3226754931 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc40005040 con 0x7fbc4c101d20 2024-03-22T07:22:28.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.676+0000 7fbc4ad9d700 1 --2- 172.21.15.63:0/3226754931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc4c101d20 0x7fbc4c112870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c002a50 tx=0x7fbc3c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:22:28.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.676+0000 7fbc337fe700 1 -- 172.21.15.63:0/3226754931 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc3c007cc0 con 0x7fbc4c101d20 2024-03-22T07:22:28.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.676+0000 7fbc510bf700 1 -- 172.21.15.63:0/3226754931 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4c10cce0 con 0x7fbc4c101d20 2024-03-22T07:22:28.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.676+0000 7fbc337fe700 1 -- 172.21.15.63:0/3226754931 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc3c00f040 con 0x7fbc4c101d20 2024-03-22T07:22:28.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.676+0000 7fbc510bf700 1 -- 172.21.15.63:0/3226754931 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4c10d270 con 0x7fbc4c101d20 2024-03-22T07:22:28.678 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.676+0000 7fbc337fe700 1 -- 172.21.15.63:0/3226754931 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc3c00da30 con 0x7fbc4c101d20 2024-03-22T07:22:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.677+0000 7fbc510bf700 1 -- 172.21.15.63:0/3226754931 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc4c0fd470 con 0x7fbc4c101d20 2024-03-22T07:22:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.678+0000 7fbc337fe700 1 -- 172.21.15.63:0/3226754931 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fbc3c00dc70 con 0x7fbc4c101d20 2024-03-22T07:22:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.679+0000 7fbc337fe700 1 --2- 172.21.15.63:0/3226754931 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbc3406d380 0x7fbc3406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.679+0000 7fbc337fe700 1 -- 172.21.15.63:0/3226754931 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbc3c08c580 con 0x7fbc4c101d20 2024-03-22T07:22:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.679+0000 7fbc4a59c700 1 --2- 172.21.15.63:0/3226754931 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbc3406d380 0x7fbc3406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:28.679 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.679+0000 7fbc4a59c700 1 --2- 172.21.15.63:0/3226754931 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbc3406d380 0x7fbc3406f830 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbc40004890 tx=0x7fbc40009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:22:28.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.683+0000 7fbc337fe700 1 -- 172.21.15.63:0/3226754931 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbc3c059c10 con 0x7fbc4c101d20 2024-03-22T07:22:28.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:28 smithi069 ceph-mon[83631]: failed to encode map e57 with expected crc 2024-03-22T07:22:28.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:28 smithi069 ceph-mon[83631]: osdmap e57: 6 total, 5 up, 6 in 2024-03-22T07:22:28.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:28 smithi069 ceph-mon[83631]: failed to encode map e57 with expected crc 2024-03-22T07:22:28.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:28 smithi069 ceph-mon[83631]: failed to encode map e57 with expected crc 2024-03-22T07:22:28.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:28 smithi069 ceph-mon[83631]: failed to encode map e57 with expected crc 2024-03-22T07:22:28.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:28 smithi069 ceph-mon[83631]: failed to encode map e57 with expected crc 2024-03-22T07:22:28.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:28 smithi069 ceph-mon[83631]: failed to encode map e57 with expected crc 2024-03-22T07:22:28.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.934+0000 7fbc510bf700 1 -- 172.21.15.63:0/3226754931 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbc4c108e00 con 0x7fbc4c101d20 2024-03-22T07:22:28.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.934+0000 7fbc337fe700 1 -- 172.21.15.63:0/3226754931 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1868 (secure 0 0 0) 0x7fbc3c0597a0 con 0x7fbc4c101d20 2024-03-22T07:22:28.935 INFO:teuthology.orchestra.run.smithi063.stdout:e24 2024-03-22T07:22:28.935 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:22:28.935 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:22:28.935 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-03-22T07:22:28.935 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:22:28.935 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:22:28.935 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-03-22T07:22:28.935 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 24 2024-03-22T07:22:28.935 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-22T07:22:28.935 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-03-22T07:13:29.602905+0000 2024-03-22T07:22:28.935 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-03-22T07:20:24.253589+0000 2024-03-22T07:22:28.935 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-03-22T07:22:28.936 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-03-22T07:22:28.936 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-03-22T07:22:28.936 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-03-22T07:22:28.936 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-03-22T07:22:28.936 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-03-22T07:22:28.936 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-03-22T07:22:28.936 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-03-22T07:22:28.936 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-03-22T07:22:28.936 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:22:28.936 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 2 2024-03-22T07:22:28.936 INFO:teuthology.orchestra.run.smithi063.stdout:in 0,1 2024-03-22T07:22:28.936 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=14480,1=24285} 2024-03-22T07:22:28.936 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-03-22T07:22:28.936 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-03-22T07:22:28.937 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-03-22T07:22:28.937 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-03-22T07:22:28.937 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-03-22T07:22:28.937 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-03-22T07:22:28.937 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-03-22T07:22:28.937 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-03-22T07:22:28.937 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-03-22T07:22:28.937 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.xeelpk{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:22:28.937 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.oytapz{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3211595802,v1:172.21.15.63:6827/3211595802] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:22:28.937 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.wwtsjn{1:24285} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.69:6824/3798247699,v1:172.21.15.69:6825/3798247699] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:22:28.937 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.cijxah{1:24301} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.69:6826/3380719311,v1:172.21.15.69:6827/3380719311] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:22:28.937 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:22:28.937 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:22:28.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.936+0000 7fbc510bf700 1 -- 172.21.15.63:0/3226754931 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbc3406d380 msgr2=0x7fbc3406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:28.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.936+0000 7fbc510bf700 1 --2- 172.21.15.63:0/3226754931 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbc3406d380 0x7fbc3406f830 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbc40004890 tx=0x7fbc40009e30 comp rx=0 tx=0).stop 2024-03-22T07:22:28.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.936+0000 7fbc510bf700 1 -- 172.21.15.63:0/3226754931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc4c101d20 msgr2=0x7fbc4c112870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:28.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.936+0000 7fbc510bf700 1 --2- 172.21.15.63:0/3226754931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc4c101d20 0x7fbc4c112870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c002a50 tx=0x7fbc3c0046b0 comp rx=0 tx=0).stop 2024-03-22T07:22:28.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.936+0000 7fbc510bf700 1 -- 172.21.15.63:0/3226754931 shutdown_connections 2024-03-22T07:22:28.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.936+0000 7fbc510bf700 1 --2- 172.21.15.63:0/3226754931 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbc3406d380 0x7fbc3406f830 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:28.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.936+0000 7fbc510bf700 1 --2- 172.21.15.63:0/3226754931 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc4c101d20 0x7fbc4c112870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:28.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.936+0000 7fbc510bf700 1 --2- 172.21.15.63:0/3226754931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc4c1026c0 0x7fbc4c112db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:28.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.936+0000 7fbc510bf700 1 -- 172.21.15.63:0/3226754931 >> 172.21.15.63:0/3226754931 conn(0x7fbc4c0f78b0 msgr2=0x7fbc4c0f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:22:28.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.937+0000 7fbc510bf700 1 -- 172.21.15.63:0/3226754931 shutdown_connections 2024-03-22T07:22:28.939 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:28.937+0000 7fbc510bf700 1 -- 172.21.15.63:0/3226754931 wait complete. 2024-03-22T07:22:28.939 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 24 2024-03-22T07:22:29.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.044+0000 7fa315aef700 1 -- 172.21.15.63:0/419710464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa310102720 msgr2=0x7fa3100fbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:29.044 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.044+0000 7fa315aef700 1 --2- 172.21.15.63:0/419710464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa310102720 0x7fa3100fbd90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa304004940 tx=0x7fa304009e30 comp rx=0 tx=0).stop 2024-03-22T07:22:29.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.044+0000 7fa315aef700 1 -- 172.21.15.63:0/419710464 shutdown_connections 2024-03-22T07:22:29.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.044+0000 7fa315aef700 1 --2- 172.21.15.63:0/419710464 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa310102720 0x7fa3100fbd90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:29.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.044+0000 7fa315aef700 1 --2- 172.21.15.63:0/419710464 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa310101d80 0x7fa310102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:29.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.044+0000 7fa315aef700 1 -- 172.21.15.63:0/419710464 >> 172.21.15.63:0/419710464 conn(0x7fa3100f78f0 msgr2=0x7fa3100f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:22:29.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.044+0000 7fa315aef700 1 -- 172.21.15.63:0/419710464 shutdown_connections 2024-03-22T07:22:29.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.044+0000 7fa315aef700 1 -- 172.21.15.63:0/419710464 wait complete. 2024-03-22T07:22:29.045 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.044+0000 7fa315aef700 1 Processor -- start 2024-03-22T07:22:29.046 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.045+0000 7fa315aef700 1 -- start start 2024-03-22T07:22:29.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.045+0000 7fa315aef700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa310101d80 0x7fa310115b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:29.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.045+0000 7fa315aef700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa310102720 0x7fa310116060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:29.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.045+0000 7fa315aef700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3101166b0 con 0x7fa310101d80 2024-03-22T07:22:29.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.045+0000 7fa315aef700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31010fcd0 con 0x7fa310102720 2024-03-22T07:22:29.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.045+0000 7fa30f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa310101d80 0x7fa310115b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:29.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.045+0000 7fa30f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa310101d80 0x7fa310115b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53914/0 (socket says 172.21.15.63:53914) 2024-03-22T07:22:29.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.045+0000 7fa30f7fe700 1 -- 172.21.15.63:0/3963642712 learned_addr learned my addr 172.21.15.63:0/3963642712 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:22:29.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.045+0000 7fa30effd700 1 --2- 172.21.15.63:0/3963642712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa310102720 0x7fa310116060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:29.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.045+0000 7fa30f7fe700 1 -- 172.21.15.63:0/3963642712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa310102720 msgr2=0x7fa310116060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:29.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.045+0000 7fa30f7fe700 1 --2- 172.21.15.63:0/3963642712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa310102720 0x7fa310116060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:29.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.045+0000 7fa30f7fe700 1 -- 172.21.15.63:0/3963642712 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa304005040 con 0x7fa310101d80 2024-03-22T07:22:29.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.045+0000 7fa30f7fe700 1 --2- 172.21.15.63:0/3963642712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa310101d80 0x7fa310115b20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa300002a50 tx=0x7fa3000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:22:29.047 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.046+0000 7fa2f7fff700 1 -- 172.21.15.63:0/3963642712 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa300007cc0 con 0x7fa310101d80 2024-03-22T07:22:29.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.046+0000 7fa315aef700 1 -- 172.21.15.63:0/3963642712 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31010ffb0 con 0x7fa310101d80 2024-03-22T07:22:29.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.046+0000 7fa315aef700 1 -- 172.21.15.63:0/3963642712 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa310110540 con 0x7fa310101d80 2024-03-22T07:22:29.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.046+0000 7fa2f7fff700 1 -- 172.21.15.63:0/3963642712 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa30000f040 con 0x7fa310101d80 2024-03-22T07:22:29.048 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.046+0000 7fa2f7fff700 1 -- 172.21.15.63:0/3963642712 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa30000da30 con 0x7fa310101d80 2024-03-22T07:22:29.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.047+0000 7fa315aef700 1 -- 172.21.15.63:0/3963642712 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2fc0052e0 con 0x7fa310101d80 2024-03-22T07:22:29.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.047+0000 7fa2f7fff700 1 -- 172.21.15.63:0/3963642712 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fa3000097c0 con 0x7fa310101d80 2024-03-22T07:22:29.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.048+0000 7fa2f7fff700 1 --2- 172.21.15.63:0/3963642712 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fa2f806d380 0x7fa2f806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:29.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.048+0000 7fa2f7fff700 1 -- 172.21.15.63:0/3963642712 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa30008c5e0 con 0x7fa310101d80 2024-03-22T07:22:29.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.048+0000 7fa30effd700 1 --2- 172.21.15.63:0/3963642712 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fa2f806d380 0x7fa2f806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:29.049 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.048+0000 7fa30effd700 1 --2- 172.21.15.63:0/3963642712 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fa2f806d380 0x7fa2f806f830 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa304004890 tx=0x7fa304009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:22:29.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.052+0000 7fa2f7fff700 1 -- 172.21.15.63:0/3963642712 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa3000560c0 con 0x7fa310101d80 2024-03-22T07:22:29.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.282+0000 7fa315aef700 1 -- 172.21.15.63:0/3963642712 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2fc000bf0 con 0x7fa2f806d380 2024-03-22T07:22:29.284 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:29 smithi063 podman[129503]: 2024-03-22 07:22:29.064973838 +0000 UTC m=+0.703647074 container remove b916e5713e99dfae104479feace8f5adcd3af217d1621ad883898d1a731f9359 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-03-22T07:22:29.284 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:29 smithi063 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-22T07:22:29.285 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:29 smithi063 systemd[1]: Stopped Ceph osd.1 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:22:29.285 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:29 smithi063 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.1.service: Consumed 1min 1.047s CPU time. 2024-03-22T07:22:29.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.284+0000 7fa2f7fff700 1 -- 172.21.15.63:0/3963642712 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa2fc000bf0 con 0x7fa2f806d380 2024-03-22T07:22:29.285 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:22:29.285 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:22:29.285 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:22:29.285 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:22:29.285 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-03-22T07:22:29.285 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-03-22T07:22:29.286 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr", 2024-03-22T07:22:29.286 INFO:teuthology.orchestra.run.smithi063.stdout: "mon" 2024-03-22T07:22:29.286 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-03-22T07:22:29.286 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "7/21 daemons upgraded", 2024-03-22T07:22:29.286 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T07:22:29.286 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:22:29.286 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:22:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.287+0000 7fa315aef700 1 -- 172.21.15.63:0/3963642712 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fa2f806d380 msgr2=0x7fa2f806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.287+0000 7fa315aef700 1 --2- 172.21.15.63:0/3963642712 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fa2f806d380 0x7fa2f806f830 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa304004890 tx=0x7fa304009e30 comp rx=0 tx=0).stop 2024-03-22T07:22:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.287+0000 7fa315aef700 1 -- 172.21.15.63:0/3963642712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa310101d80 msgr2=0x7fa310115b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.287+0000 7fa315aef700 1 --2- 172.21.15.63:0/3963642712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa310101d80 0x7fa310115b20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa300002a50 tx=0x7fa3000046b0 comp rx=0 tx=0).stop 2024-03-22T07:22:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.287+0000 7fa315aef700 1 -- 172.21.15.63:0/3963642712 shutdown_connections 2024-03-22T07:22:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.287+0000 7fa315aef700 1 --2- 172.21.15.63:0/3963642712 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fa2f806d380 0x7fa2f806f830 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.287+0000 7fa315aef700 1 --2- 172.21.15.63:0/3963642712 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa310101d80 0x7fa310115b20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.287+0000 7fa315aef700 1 --2- 172.21.15.63:0/3963642712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa310102720 0x7fa310116060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.287+0000 7fa315aef700 1 -- 172.21.15.63:0/3963642712 >> 172.21.15.63:0/3963642712 conn(0x7fa3100f78f0 msgr2=0x7fa310101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:22:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.288+0000 7fa315aef700 1 -- 172.21.15.63:0/3963642712 shutdown_connections 2024-03-22T07:22:29.288 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.288+0000 7fa315aef700 1 -- 172.21.15.63:0/3963642712 wait complete. 2024-03-22T07:22:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.403+0000 7f70acdf2700 1 -- 172.21.15.63:0/1476703114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a81014f0 msgr2=0x7f70a81018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.403+0000 7f70acdf2700 1 --2- 172.21.15.63:0/1476703114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a81014f0 0x7f70a81018c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7098004a50 tx=0x7f709800af40 comp rx=0 tx=0).stop 2024-03-22T07:22:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.403+0000 7f70acdf2700 1 -- 172.21.15.63:0/1476703114 shutdown_connections 2024-03-22T07:22:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.403+0000 7f70acdf2700 1 --2- 172.21.15.63:0/1476703114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70a80fb4a0 0x7f70a80fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.403+0000 7f70acdf2700 1 --2- 172.21.15.63:0/1476703114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a81014f0 0x7f70a81018c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.403+0000 7f70acdf2700 1 -- 172.21.15.63:0/1476703114 >> 172.21.15.63:0/1476703114 conn(0x7f70a80714c0 msgr2=0x7f70a80718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:22:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.403+0000 7f70acdf2700 1 -- 172.21.15.63:0/1476703114 shutdown_connections 2024-03-22T07:22:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.403+0000 7f70acdf2700 1 -- 172.21.15.63:0/1476703114 wait complete. 2024-03-22T07:22:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.404+0000 7f70acdf2700 1 Processor -- start 2024-03-22T07:22:29.404 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.404+0000 7f70acdf2700 1 -- start start 2024-03-22T07:22:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.404+0000 7f70acdf2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a80fb4a0 0x7f70a80fd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.404+0000 7f70acdf2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70a81014f0 0x7f70a80fd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.404+0000 7f70acdf2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a80fdb80 con 0x7f70a80fb4a0 2024-03-22T07:22:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.404+0000 7f70acdf2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a80fdcf0 con 0x7f70a81014f0 2024-03-22T07:22:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.404+0000 7f70a659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a80fb4a0 0x7f70a80fd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.404+0000 7f70a659c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a80fb4a0 0x7f70a80fd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:53930/0 (socket says 172.21.15.63:53930) 2024-03-22T07:22:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.404+0000 7f70a659c700 1 -- 172.21.15.63:0/3994086384 learned_addr learned my addr 172.21.15.63:0/3994086384 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:22:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.404+0000 7f70a5d9b700 1 --2- 172.21.15.63:0/3994086384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70a81014f0 0x7f70a80fd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.405+0000 7f70a659c700 1 -- 172.21.15.63:0/3994086384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70a81014f0 msgr2=0x7f70a80fd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:29.406 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.405+0000 7f70a659c700 1 --2- 172.21.15.63:0/3994086384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70a81014f0 0x7f70a80fd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.405+0000 7f70a659c700 1 -- 172.21.15.63:0/3994086384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7098005040 con 0x7f70a80fb4a0 2024-03-22T07:22:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.405+0000 7f70a659c700 1 --2- 172.21.15.63:0/3994086384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a80fb4a0 0x7f70a80fd0b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f709800c970 tx=0x7f709800af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:22:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.405+0000 7f7096ffd700 1 -- 172.21.15.63:0/3994086384 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f709800f050 con 0x7f70a80fb4a0 2024-03-22T07:22:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.405+0000 7f70acdf2700 1 -- 172.21.15.63:0/3994086384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a8119ec0 con 0x7f70a80fb4a0 2024-03-22T07:22:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.405+0000 7f7096ffd700 1 -- 172.21.15.63:0/3994086384 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f709800a4c0 con 0x7f70a80fb4a0 2024-03-22T07:22:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.405+0000 7f7096ffd700 1 -- 172.21.15.63:0/3994086384 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7098014760 con 0x7f70a80fb4a0 2024-03-22T07:22:29.407 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.405+0000 7f70acdf2700 1 -- 172.21.15.63:0/3994086384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a811a210 con 0x7f70a80fb4a0 2024-03-22T07:22:29.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.406+0000 7f70acdf2700 1 -- 172.21.15.63:0/3994086384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a8104ff0 con 0x7f70a80fb4a0 2024-03-22T07:22:29.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.407+0000 7f7096ffd700 1 -- 172.21.15.63:0/3994086384 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f709801db70 con 0x7f70a80fb4a0 2024-03-22T07:22:29.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.407+0000 7f7096ffd700 1 --2- 172.21.15.63:0/3994086384 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f709006d380 0x7f709006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:29.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.407+0000 7f7096ffd700 1 -- 172.21.15.63:0/3994086384 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f709808e440 con 0x7f70a80fb4a0 2024-03-22T07:22:29.408 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.408+0000 7f70a5d9b700 1 --2- 172.21.15.63:0/3994086384 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f709006d380 0x7f709006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:29.409 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.408+0000 7f70a5d9b700 1 --2- 172.21.15.63:0/3994086384 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f709006d380 0x7f709006f830 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f70a80fe9a0 tx=0x7f709c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:22:29.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.415+0000 7f7096ffd700 1 -- 172.21.15.63:0/3994086384 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f709805bad0 con 0x7f70a80fb4a0 2024-03-22T07:22:29.538 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:29 smithi063 systemd[1]: Starting Ceph osd.1 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:22:29.539 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:29 smithi063 ceph-mon[112784]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:22:29.539 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:29 smithi063 ceph-mon[112784]: pgmap v68: 65 pgs: 14 active+undersized+degraded, 3 stale+active+clean, 15 active+undersized, 33 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 48/354 objects degraded (13.559%) 2024-03-22T07:22:29.539 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:29 smithi063 ceph-mon[112784]: from='client.44145 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:22:29.539 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:29 smithi063 ceph-mon[112784]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:22:29.539 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:29 smithi063 ceph-mon[112784]: Health check failed: Degraded data redundancy: 48/354 objects degraded (13.559%), 14 pgs degraded (PG_DEGRADED) 2024-03-22T07:22:29.539 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:29 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2717085636' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:22:29.539 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:29 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3226754931' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:22:29.709 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.709+0000 7f70acdf2700 1 -- 172.21.15.63:0/3994086384 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f70a804bed0 con 0x7f70a80fb4a0 2024-03-22T07:22:29.710 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.709+0000 7f7096ffd700 1 -- 172.21.15.63:0/3994086384 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1049 (secure 0 0 0) 0x7f709805b660 con 0x7f70a80fb4a0 2024-03-22T07:22:29.710 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 48/354 objects degraded (13.559%), 14 pgs degraded 2024-03-22T07:22:29.710 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-22T07:22:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: osd.1 (root=default,host=smithi063) is down 2024-03-22T07:22:29.710 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 48/354 objects degraded (13.559%), 14 pgs degraded 2024-03-22T07:22:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-03-22T07:22:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-03-22T07:22:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-03-22T07:22:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-03-22T07:22:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-03-22T07:22:29.710 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-03-22T07:22:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-03-22T07:22:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-03-22T07:22:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-03-22T07:22:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-03-22T07:22:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.11 is active+undersized+degraded, acting [4,3] 2024-03-22T07:22:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.12 is active+undersized+degraded, acting [3,0] 2024-03-22T07:22:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-03-22T07:22:29.711 INFO:teuthology.orchestra.run.smithi063.stdout: pg 2.17 is active+undersized+degraded, acting [5,2] 2024-03-22T07:22:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.711+0000 7f70acdf2700 1 -- 172.21.15.63:0/3994086384 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f709006d380 msgr2=0x7f709006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.711+0000 7f70acdf2700 1 --2- 172.21.15.63:0/3994086384 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f709006d380 0x7f709006f830 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f70a80fe9a0 tx=0x7f709c00b040 comp rx=0 tx=0).stop 2024-03-22T07:22:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.711+0000 7f70acdf2700 1 -- 172.21.15.63:0/3994086384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a80fb4a0 msgr2=0x7f70a80fd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.711+0000 7f70acdf2700 1 --2- 172.21.15.63:0/3994086384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a80fb4a0 0x7f70a80fd0b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f709800c970 tx=0x7f709800af40 comp rx=0 tx=0).stop 2024-03-22T07:22:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.711+0000 7f70acdf2700 1 -- 172.21.15.63:0/3994086384 shutdown_connections 2024-03-22T07:22:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.711+0000 7f70acdf2700 1 --2- 172.21.15.63:0/3994086384 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f709006d380 0x7f709006f830 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.711+0000 7f70acdf2700 1 --2- 172.21.15.63:0/3994086384 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f70a80fb4a0 0x7f70a80fd0b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.712+0000 7f70acdf2700 1 --2- 172.21.15.63:0/3994086384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70a81014f0 0x7f70a80fd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.712+0000 7f70acdf2700 1 -- 172.21.15.63:0/3994086384 >> 172.21.15.63:0/3994086384 conn(0x7f70a80714c0 msgr2=0x7f70a8102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:22:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.712+0000 7f70acdf2700 1 -- 172.21.15.63:0/3994086384 shutdown_connections 2024-03-22T07:22:29.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:29.712+0000 7f70acdf2700 1 -- 172.21.15.63:0/3994086384 wait complete. 2024-03-22T07:22:29.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:29 smithi069 ceph-mon[83631]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:22:29.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:29 smithi069 ceph-mon[83631]: pgmap v68: 65 pgs: 14 active+undersized+degraded, 3 stale+active+clean, 15 active+undersized, 33 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 48/354 objects degraded (13.559%) 2024-03-22T07:22:29.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:29 smithi069 ceph-mon[83631]: from='client.44145 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:22:29.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:29 smithi069 ceph-mon[83631]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:22:29.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:29 smithi069 ceph-mon[83631]: Health check failed: Degraded data redundancy: 48/354 objects degraded (13.559%), 14 pgs degraded (PG_DEGRADED) 2024-03-22T07:22:29.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:29 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2717085636' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:22:29.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:29 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3226754931' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:22:29.873 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:29 smithi063 podman[129681]: 2024-03-22 07:22:29.442312725 +0000 UTC m=+0.020060671 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:22:29.873 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:29 smithi063 podman[129681]: 2024-03-22 07:22:29.641663007 +0000 UTC m=+0.219410946 container create eba0c5e6176f0a17619dfd6be81eda55285d162a6d74186cd583d12e583130ba (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-activate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-22T07:22:30.288 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:29 smithi063 podman[129681]: 2024-03-22 07:22:29.877611331 +0000 UTC m=+0.455359270 container init eba0c5e6176f0a17619dfd6be81eda55285d162a6d74186cd583d12e583130ba (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-22T07:22:30.289 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:29 smithi063 podman[129681]: 2024-03-22 07:22:29.880690805 +0000 UTC m=+0.458438744 container start eba0c5e6176f0a17619dfd6be81eda55285d162a6d74186cd583d12e583130ba (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:22:30.289 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:29 smithi063 podman[129681]: 2024-03-22 07:22:29.960947055 +0000 UTC m=+0.538695005 container attach eba0c5e6176f0a17619dfd6be81eda55285d162a6d74186cd583d12e583130ba (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-activate, GIT_CLEAN=True, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:22:30.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:30 smithi063 ceph-mon[112784]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:22:30.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:30 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3994086384' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:22:30.623 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:30 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-activate[129700]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-22T07:22:30.623 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:30 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-activate[129700]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-22T07:22:30.624 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:30 smithi063 bash[129681]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-22T07:22:30.624 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:30 smithi063 bash[129681]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-22T07:22:30.624 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:30 smithi063 bash[129681]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-22T07:22:30.624 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:30 smithi063 bash[129681]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-22T07:22:30.624 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:30 smithi063 bash[129681]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-22T07:22:30.624 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:30 smithi063 bash[129681]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-22T07:22:30.624 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:30 smithi063 bash[129681]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-22T07:22:30.624 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:30 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-activate[129700]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-22T07:22:30.624 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:30 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-activate[129700]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-22T07:22:30.624 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:30 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-activate[129700]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-22T07:22:30.624 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:30 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-activate[129700]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-22T07:22:30.624 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:30 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-activate[129700]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-22T07:22:30.624 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:30 smithi063 podman[129681]: 2024-03-22 07:22:30.355950511 +0000 UTC m=+0.933698440 container died eba0c5e6176f0a17619dfd6be81eda55285d162a6d74186cd583d12e583130ba (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-22T07:22:30.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:30 smithi069 ceph-mon[83631]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:22:30.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:30 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3994086384' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:22:31.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:31 smithi063 ceph-mon[112784]: pgmap v69: 65 pgs: 14 active+undersized+degraded, 3 stale+active+clean, 15 active+undersized, 33 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 48/354 objects degraded (13.559%) 2024-03-22T07:22:31.373 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:31 smithi063 podman[129838]: 2024-03-22 07:22:31.098670424 +0000 UTC m=+0.737246568 container remove eba0c5e6176f0a17619dfd6be81eda55285d162a6d74186cd583d12e583130ba (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-activate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-22T07:22:31.373 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:31 smithi063 podman[129902]: 2024-03-22 07:22:31.219196198 +0000 UTC m=+0.017849288 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:22:31.373 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:31 smithi063 podman[129902]: 2024-03-22 07:22:31.324667619 +0000 UTC m=+0.123320701 container create 5200fd89304678d089ed611e9904bc6ca1262d795265cf60f9ee46ab4ca39b70 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:22:31.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:31 smithi069 ceph-mon[83631]: pgmap v69: 65 pgs: 14 active+undersized+degraded, 3 stale+active+clean, 15 active+undersized, 33 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 48/354 objects degraded (13.559%) 2024-03-22T07:22:31.873 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:31 smithi063 podman[129902]: 2024-03-22 07:22:31.510458896 +0000 UTC m=+0.309111976 container init 5200fd89304678d089ed611e9904bc6ca1262d795265cf60f9ee46ab4ca39b70 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:22:31.873 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:31 smithi063 podman[129902]: 2024-03-22 07:22:31.513528702 +0000 UTC m=+0.312181788 container start 5200fd89304678d089ed611e9904bc6ca1262d795265cf60f9ee46ab4ca39b70 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:22:31.873 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:31 smithi063 ceph-osd[129922]: -- 172.21.15.63:0/1976752761 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e1a33031e0 con 0x55e1a32d5000 2024-03-22T07:22:31.873 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:31 smithi063 bash[129902]: 5200fd89304678d089ed611e9904bc6ca1262d795265cf60f9ee46ab4ca39b70 2024-03-22T07:22:31.873 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:31 smithi063 systemd[1]: Started Ceph osd.1 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:22:32.373 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:32 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1[129918]: 2024-03-22T07:22:32.055+0000 7f6d9902f7c0 -1 Falling back to public interface 2024-03-22T07:22:32.774 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:32 smithi063 ceph-mon[112784]: pgmap v70: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 55/354 objects degraded (15.537%) 2024-03-22T07:22:32.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:32.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:22:32.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:32.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:32.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:22:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:32 smithi069 ceph-mon[83631]: pgmap v70: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 55/354 objects degraded (15.537%) 2024-03-22T07:22:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:32 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:32 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:22:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:32 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:32 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:32 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:22:34.750 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:34 smithi063 ceph-mon[112784]: pgmap v71: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 55/354 objects degraded (15.537%) 2024-03-22T07:22:34.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:34 smithi069 ceph-mon[83631]: pgmap v71: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 55/354 objects degraded (15.537%) 2024-03-22T07:22:36.758 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:36 smithi063 ceph-mon[112784]: pgmap v72: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 55/354 objects degraded (15.537%) 2024-03-22T07:22:36.758 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:36 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1[129918]: 2024-03-22T07:22:36.538+0000 7f6d9902f7c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-03-22T07:22:36.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:36 smithi069 ceph-mon[83631]: pgmap v72: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 55/354 objects degraded (15.537%) 2024-03-22T07:22:37.373 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:37 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1[129918]: 2024-03-22T07:22:37.115+0000 7f6d9902f7c0 -1 osd.1 55 log_to_monitors true 2024-03-22T07:22:37.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:37 smithi069 ceph-mon[83631]: from='osd.1 [v2:172.21.15.63:6810/3855758242,v1:172.21.15.63:6811/3855758242]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T07:22:37.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:37 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:37.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:37 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:37 smithi063 ceph-mon[112784]: from='osd.1 [v2:172.21.15.63:6810/3855758242,v1:172.21.15.63:6811/3855758242]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-22T07:22:37.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:37 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:37.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:37 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:38.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:38 smithi069 ceph-mon[83631]: pgmap v73: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 55/354 objects degraded (15.537%) 2024-03-22T07:22:38.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:38 smithi069 ceph-mon[83631]: from='osd.1 [v2:172.21.15.63:6810/3855758242,v1:172.21.15.63:6811/3855758242]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T07:22:38.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:38 smithi069 ceph-mon[83631]: osdmap e58: 6 total, 5 up, 6 in 2024-03-22T07:22:38.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:38 smithi069 ceph-mon[83631]: from='osd.1 [v2:172.21.15.63:6810/3855758242,v1:172.21.15.63:6811/3855758242]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-03-22T07:22:38.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:38 smithi069 ceph-mon[83631]: failed to encode map e58 with expected crc 2024-03-22T07:22:38.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:38 smithi069 ceph-mon[83631]: Health check update: Degraded data redundancy: 55/354 objects degraded (15.537%), 16 pgs degraded (PG_DEGRADED) 2024-03-22T07:22:38.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:38 smithi069 ceph-mon[83631]: from='osd.1 [v2:172.21.15.63:6810/3855758242,v1:172.21.15.63:6811/3855758242]' entity='osd.1' 2024-03-22T07:22:38.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:38 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:38.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:38 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:38.873 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:22:38 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1[129918]: 2024-03-22T07:22:38.374+0000 7f6d8647f700 -1 osd.1 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:22:38.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:38 smithi063 ceph-mon[112784]: pgmap v73: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 299 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 55/354 objects degraded (15.537%) 2024-03-22T07:22:38.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:38 smithi063 ceph-mon[112784]: from='osd.1 [v2:172.21.15.63:6810/3855758242,v1:172.21.15.63:6811/3855758242]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-22T07:22:38.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:38 smithi063 ceph-mon[112784]: osdmap e58: 6 total, 5 up, 6 in 2024-03-22T07:22:38.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:38 smithi063 ceph-mon[112784]: from='osd.1 [v2:172.21.15.63:6810/3855758242,v1:172.21.15.63:6811/3855758242]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-03-22T07:22:38.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:38 smithi063 ceph-mon[112784]: failed to encode map e58 with expected crc 2024-03-22T07:22:38.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:38 smithi063 ceph-mon[112784]: Health check update: Degraded data redundancy: 55/354 objects degraded (15.537%), 16 pgs degraded (PG_DEGRADED) 2024-03-22T07:22:38.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:38 smithi063 ceph-mon[112784]: from='osd.1 [v2:172.21.15.63:6810/3855758242,v1:172.21.15.63:6811/3855758242]' entity='osd.1' 2024-03-22T07:22:38.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:38 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:38.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:38 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:39.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:39 smithi063 ceph-mon[112784]: failed to encode map e58 with expected crc 2024-03-22T07:22:39.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:39 smithi063 ceph-mon[112784]: failed to encode map e58 with expected crc 2024-03-22T07:22:39.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:39 smithi063 ceph-mon[112784]: failed to encode map e58 with expected crc 2024-03-22T07:22:39.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:39 smithi063 ceph-mon[112784]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:22:39.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:39 smithi063 ceph-mon[112784]: osd.1 [v2:172.21.15.63:6810/3855758242,v1:172.21.15.63:6811/3855758242] boot 2024-03-22T07:22:39.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:39 smithi063 ceph-mon[112784]: osdmap e59: 6 total, 6 up, 6 in 2024-03-22T07:22:39.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:39 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:22:39.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:39 smithi069 ceph-mon[83631]: failed to encode map e58 with expected crc 2024-03-22T07:22:39.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:39 smithi069 ceph-mon[83631]: failed to encode map e58 with expected crc 2024-03-22T07:22:39.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:39 smithi069 ceph-mon[83631]: failed to encode map e58 with expected crc 2024-03-22T07:22:39.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:39 smithi069 ceph-mon[83631]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:22:39.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:39 smithi069 ceph-mon[83631]: osd.1 [v2:172.21.15.63:6810/3855758242,v1:172.21.15.63:6811/3855758242] boot 2024-03-22T07:22:39.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:39 smithi069 ceph-mon[83631]: osdmap e59: 6 total, 6 up, 6 in 2024-03-22T07:22:39.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:39 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-22T07:22:40.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:40 smithi063 ceph-mon[112784]: failed to encode map e59 with expected crc 2024-03-22T07:22:40.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:40 smithi063 ceph-mon[112784]: pgmap v76: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 55/354 objects degraded (15.537%) 2024-03-22T07:22:40.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:40 smithi063 ceph-mon[112784]: osdmap e60: 6 total, 6 up, 6 in 2024-03-22T07:22:40.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:40 smithi069 ceph-mon[83631]: failed to encode map e59 with expected crc 2024-03-22T07:22:40.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:40 smithi069 ceph-mon[83631]: pgmap v76: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 55/354 objects degraded (15.537%) 2024-03-22T07:22:40.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:40 smithi069 ceph-mon[83631]: osdmap e60: 6 total, 6 up, 6 in 2024-03-22T07:22:41.588 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:41 smithi063 ceph-mon[112784]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.588 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:41 smithi063 ceph-mon[112784]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.588 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:41 smithi063 ceph-mon[112784]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.588 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:41 smithi063 ceph-mon[112784]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.588 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:41 smithi063 ceph-mon[112784]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.588 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:41 smithi063 ceph-mon[112784]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.589 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:41 smithi063 ceph-mon[112784]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.589 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:41 smithi063 ceph-mon[112784]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.589 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:41 smithi063 ceph-mon[112784]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.589 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:41 smithi063 ceph-mon[112784]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.589 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:41 smithi063 ceph-mon[112784]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:41 smithi069 ceph-mon[83631]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:41 smithi069 ceph-mon[83631]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:41 smithi069 ceph-mon[83631]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:41 smithi069 ceph-mon[83631]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:41 smithi069 ceph-mon[83631]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:41 smithi069 ceph-mon[83631]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:41 smithi069 ceph-mon[83631]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:41 smithi069 ceph-mon[83631]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:41 smithi069 ceph-mon[83631]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:41 smithi069 ceph-mon[83631]: failed to encode map e60 with expected crc 2024-03-22T07:22:41.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:41 smithi069 ceph-mon[83631]: failed to encode map e60 with expected crc 2024-03-22T07:22:43.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:43 smithi063 ceph-mon[112784]: pgmap v78: 65 pgs: 2 peering, 16 active+undersized+degraded, 16 active+undersized, 31 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 55/354 objects degraded (15.537%) 2024-03-22T07:22:43.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:43.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:43.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:22:43.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:22:43.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:43.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:22:43.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:22:43.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:22:43.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:22:43.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T07:22:43.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:43 smithi063 ceph-mon[112784]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T07:22:43.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:43 smithi063 ceph-mon[112784]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-03-22T07:22:43.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:43 smithi069 ceph-mon[83631]: pgmap v78: 65 pgs: 2 peering, 16 active+undersized+degraded, 16 active+undersized, 31 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 55/354 objects degraded (15.537%) 2024-03-22T07:22:43.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:43 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:43.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:43 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:43.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:43 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:22:43.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:43 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:22:43.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:43 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:43.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:43 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:22:43.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:43 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:22:43.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:43 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:22:43.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:43 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:22:43.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:43 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T07:22:43.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:43 smithi069 ceph-mon[83631]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T07:22:43.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:43 smithi069 ceph-mon[83631]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-03-22T07:22:44.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:44 smithi063 ceph-mon[112784]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 55/354 objects degraded (15.537%), 16 pgs degraded) 2024-03-22T07:22:44.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:44 smithi063 ceph-mon[112784]: Cluster is now healthy 2024-03-22T07:22:44.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:44 smithi069 ceph-mon[83631]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 55/354 objects degraded (15.537%), 16 pgs degraded) 2024-03-22T07:22:44.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:44 smithi069 ceph-mon[83631]: Cluster is now healthy 2024-03-22T07:22:45.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:45 smithi069 ceph-mon[83631]: pgmap v79: 65 pgs: 2 peering, 63 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-22T07:22:45.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:45 smithi063 ceph-mon[112784]: pgmap v79: 65 pgs: 2 peering, 63 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-22T07:22:47.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:47 smithi069 ceph-mon[83631]: pgmap v80: 65 pgs: 65 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 902 B/s rd, 0 op/s 2024-03-22T07:22:47.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:47 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:47.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:47 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:22:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:47 smithi063 ceph-mon[112784]: pgmap v80: 65 pgs: 65 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 902 B/s rd, 0 op/s 2024-03-22T07:22:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:47 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:47 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:22:49.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:49 smithi069 ceph-mon[83631]: pgmap v81: 65 pgs: 65 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 2 op/s 2024-03-22T07:22:49.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:49 smithi063 ceph-mon[112784]: pgmap v81: 65 pgs: 65 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 2 op/s 2024-03-22T07:22:51.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:51 smithi069 ceph-mon[83631]: pgmap v82: 65 pgs: 65 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-03-22T07:22:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:51 smithi063 ceph-mon[112784]: pgmap v82: 65 pgs: 65 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-03-22T07:22:52.774 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:52 smithi063 ceph-mon[112784]: pgmap v83: 65 pgs: 65 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-22T07:22:52.774 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:52 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:52.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:52 smithi069 ceph-mon[83631]: pgmap v83: 65 pgs: 65 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-22T07:22:52.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:52 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:22:54.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:54 smithi069 ceph-mon[83631]: pgmap v84: 65 pgs: 65 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-03-22T07:22:54.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:54 smithi063 ceph-mon[112784]: pgmap v84: 65 pgs: 65 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-03-22T07:22:56.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:56 smithi069 ceph-mon[83631]: pgmap v85: 65 pgs: 65 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-22T07:22:56.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:56 smithi063 ceph-mon[112784]: pgmap v85: 65 pgs: 65 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-22T07:22:57.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:57 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T07:22:57.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:57 smithi069 ceph-mon[83631]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T07:22:57.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:57 smithi069 ceph-mon[83631]: Upgrade: osd.2 is safe to restart 2024-03-22T07:22:57.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:57 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T07:22:57.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:57 smithi063 ceph-mon[112784]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-22T07:22:57.812 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:57 smithi063 ceph-mon[112784]: Upgrade: osd.2 is safe to restart 2024-03-22T07:22:58.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:58 smithi069 ceph-mon[83631]: pgmap v86: 65 pgs: 65 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-22T07:22:58.830 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:58 smithi063 ceph-mon[112784]: pgmap v86: 65 pgs: 65 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-03-22T07:22:59.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.821+0000 7f564023e700 1 -- 172.21.15.63:0/96769630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56381037b0 msgr2=0x7f56380fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:59.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.821+0000 7f564023e700 1 --2- 172.21.15.63:0/96769630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56381037b0 0x7f56380fce20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5628004940 tx=0x7f5628009e30 comp rx=0 tx=0).stop 2024-03-22T07:22:59.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.821+0000 7f564023e700 1 -- 172.21.15.63:0/96769630 shutdown_connections 2024-03-22T07:22:59.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.821+0000 7f564023e700 1 --2- 172.21.15.63:0/96769630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f56381037b0 0x7f56380fce20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:59.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.821+0000 7f564023e700 1 --2- 172.21.15.63:0/96769630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5638102e10 0x7f56381031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:59.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.821+0000 7f564023e700 1 -- 172.21.15.63:0/96769630 >> 172.21.15.63:0/96769630 conn(0x7f56380f8980 msgr2=0x7f56380fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:22:59.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.821+0000 7f564023e700 1 -- 172.21.15.63:0/96769630 shutdown_connections 2024-03-22T07:22:59.822 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.821+0000 7f564023e700 1 -- 172.21.15.63:0/96769630 wait complete. 2024-03-22T07:22:59.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.822+0000 7f564023e700 1 Processor -- start 2024-03-22T07:22:59.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.822+0000 7f564023e700 1 -- start start 2024-03-22T07:22:59.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.822+0000 7f564023e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5638102e10 0x7f5638113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:59.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.822+0000 7f564023e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56381037b0 0x7f56381140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:59.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.822+0000 7f564023e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5638114700 con 0x7f5638102e10 2024-03-22T07:22:59.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.822+0000 7f564023e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563810db40 con 0x7f56381037b0 2024-03-22T07:22:59.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.822+0000 7f563dfda700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5638102e10 0x7f5638113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:59.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.822+0000 7f563dfda700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5638102e10 0x7f5638113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47688/0 (socket says 172.21.15.63:47688) 2024-03-22T07:22:59.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.822+0000 7f563dfda700 1 -- 172.21.15.63:0/1881989486 learned_addr learned my addr 172.21.15.63:0/1881989486 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:22:59.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.823+0000 7f563d7d9700 1 --2- 172.21.15.63:0/1881989486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56381037b0 0x7f56381140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:59.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.823+0000 7f563dfda700 1 -- 172.21.15.63:0/1881989486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56381037b0 msgr2=0x7f56381140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:22:59.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.823+0000 7f563dfda700 1 --2- 172.21.15.63:0/1881989486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56381037b0 0x7f56381140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:22:59.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.823+0000 7f563dfda700 1 -- 172.21.15.63:0/1881989486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5628005040 con 0x7f5638102e10 2024-03-22T07:22:59.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.823+0000 7f563d7d9700 1 --2- 172.21.15.63:0/1881989486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56381037b0 0x7f56381140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:22:59.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.823+0000 7f563dfda700 1 --2- 172.21.15.63:0/1881989486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5638102e10 0x7f5638113b70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5634002a50 tx=0x7f56340046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:22:59.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.823+0000 7f562e7fc700 1 -- 172.21.15.63:0/1881989486 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5634007cc0 con 0x7f5638102e10 2024-03-22T07:22:59.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.823+0000 7f564023e700 1 -- 172.21.15.63:0/1881989486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563810de20 con 0x7f5638102e10 2024-03-22T07:22:59.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.823+0000 7f562e7fc700 1 -- 172.21.15.63:0/1881989486 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f563400f040 con 0x7f5638102e10 2024-03-22T07:22:59.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.823+0000 7f562e7fc700 1 -- 172.21.15.63:0/1881989486 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5634009640 con 0x7f5638102e10 2024-03-22T07:22:59.825 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.823+0000 7f564023e700 1 -- 172.21.15.63:0/1881989486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563810e3e0 con 0x7f5638102e10 2024-03-22T07:22:59.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.824+0000 7f564023e700 1 -- 172.21.15.63:0/1881989486 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56381052e0 con 0x7f5638102e10 2024-03-22T07:22:59.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.825+0000 7f562e7fc700 1 -- 172.21.15.63:0/1881989486 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f56340097c0 con 0x7f5638102e10 2024-03-22T07:22:59.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.825+0000 7f562e7fc700 1 --2- 172.21.15.63:0/1881989486 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f562406d380 0x7f562406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:22:59.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.825+0000 7f562e7fc700 1 -- 172.21.15.63:0/1881989486 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f563408c5d0 con 0x7f5638102e10 2024-03-22T07:22:59.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.825+0000 7f563d7d9700 1 --2- 172.21.15.63:0/1881989486 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f562406d380 0x7f562406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:22:59.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.826+0000 7f563d7d9700 1 --2- 172.21.15.63:0/1881989486 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f562406d380 0x7f562406f830 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f563810ecb0 tx=0x7f5628009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:22:59.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:22:59.831+0000 7f562e7fc700 1 -- 172.21.15.63:0/1881989486 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5634059c00 con 0x7f5638102e10 2024-03-22T07:23:00.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.054+0000 7f564023e700 1 -- 172.21.15.63:0/1881989486 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f563810f1c0 con 0x7f562406d380 2024-03-22T07:23:00.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.056+0000 7f562e7fc700 1 -- 172.21.15.63:0/1881989486 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f563810f1c0 con 0x7f562406d380 2024-03-22T07:23:00.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.058+0000 7f564023e700 1 -- 172.21.15.63:0/1881989486 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f562406d380 msgr2=0x7f562406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:00.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.058+0000 7f564023e700 1 --2- 172.21.15.63:0/1881989486 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f562406d380 0x7f562406f830 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f563810ecb0 tx=0x7f5628009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:00.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.058+0000 7f564023e700 1 -- 172.21.15.63:0/1881989486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5638102e10 msgr2=0x7f5638113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:00.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.058+0000 7f564023e700 1 --2- 172.21.15.63:0/1881989486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5638102e10 0x7f5638113b70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5634002a50 tx=0x7f56340046b0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.058+0000 7f564023e700 1 -- 172.21.15.63:0/1881989486 shutdown_connections 2024-03-22T07:23:00.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.058+0000 7f564023e700 1 --2- 172.21.15.63:0/1881989486 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f562406d380 0x7f562406f830 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.058+0000 7f564023e700 1 --2- 172.21.15.63:0/1881989486 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f5638102e10 0x7f5638113b70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.058+0000 7f564023e700 1 --2- 172.21.15.63:0/1881989486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56381037b0 0x7f56381140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.058+0000 7f564023e700 1 -- 172.21.15.63:0/1881989486 >> 172.21.15.63:0/1881989486 conn(0x7f56380f8980 msgr2=0x7f56381021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:00.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.058+0000 7f564023e700 1 -- 172.21.15.63:0/1881989486 shutdown_connections 2024-03-22T07:23:00.059 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.058+0000 7f564023e700 1 -- 172.21.15.63:0/1881989486 wait complete. 2024-03-22T07:23:00.068 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:23:00.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.162+0000 7f74f1411700 1 -- 172.21.15.63:0/3532753010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74ec101d70 msgr2=0x7f74ec102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:00.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.162+0000 7f74f1411700 1 --2- 172.21.15.63:0/3532753010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74ec101d70 0x7f74ec102140 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f74dc004940 tx=0x7f74dc009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:00.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.162+0000 7f74f1411700 1 -- 172.21.15.63:0/3532753010 shutdown_connections 2024-03-22T07:23:00.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.162+0000 7f74f1411700 1 --2- 172.21.15.63:0/3532753010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74ec102710 0x7f74ec0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.162+0000 7f74f1411700 1 --2- 172.21.15.63:0/3532753010 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74ec101d70 0x7f74ec102140 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.162+0000 7f74f1411700 1 -- 172.21.15.63:0/3532753010 >> 172.21.15.63:0/3532753010 conn(0x7f74ec072bc0 msgr2=0x7f74ec072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:00.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.162+0000 7f74f1411700 1 -- 172.21.15.63:0/3532753010 shutdown_connections 2024-03-22T07:23:00.163 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:59 smithi063 ceph-mon[112784]: Upgrade: Updating osd.2 2024-03-22T07:23:00.163 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:00.163 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T07:23:00.163 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:23:00.163 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:22:59 smithi063 ceph-mon[112784]: Deploying daemon osd.2 on smithi063 2024-03-22T07:23:00.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.162+0000 7f74f1411700 1 -- 172.21.15.63:0/3532753010 wait complete. 2024-03-22T07:23:00.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.163+0000 7f74f1411700 1 Processor -- start 2024-03-22T07:23:00.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.163+0000 7f74f1411700 1 -- start start 2024-03-22T07:23:00.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.163+0000 7f74f1411700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74ec101d70 0x7f74ec102bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:00.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.163+0000 7f74f1411700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74ec102710 0x7f74ec103130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:00.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.163+0000 7f74eaffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74ec101d70 0x7f74ec102bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:00.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.163+0000 7f74f1411700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec1049e0 con 0x7f74ec101d70 2024-03-22T07:23:00.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.163+0000 7f74f1411700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec104b50 con 0x7f74ec102710 2024-03-22T07:23:00.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.163+0000 7f74eaffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74ec101d70 0x7f74ec102bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47706/0 (socket says 172.21.15.63:47706) 2024-03-22T07:23:00.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.163+0000 7f74eaffd700 1 -- 172.21.15.63:0/2340128986 learned_addr learned my addr 172.21.15.63:0/2340128986 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:23:00.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.163+0000 7f74eaffd700 1 -- 172.21.15.63:0/2340128986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74ec102710 msgr2=0x7f74ec103130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:00.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.163+0000 7f74eaffd700 1 --2- 172.21.15.63:0/2340128986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74ec102710 0x7f74ec103130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.163+0000 7f74eaffd700 1 -- 172.21.15.63:0/2340128986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74dc005040 con 0x7f74ec101d70 2024-03-22T07:23:00.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.164+0000 7f74eaffd700 1 --2- 172.21.15.63:0/2340128986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74ec101d70 0x7f74ec102bf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00b8b0 tx=0x7f74dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:00.165 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.164+0000 7f74d37fe700 1 -- 172.21.15.63:0/2340128986 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74dc00e040 con 0x7f74ec101d70 2024-03-22T07:23:00.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.164+0000 7f74f1411700 1 -- 172.21.15.63:0/2340128986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ec103730 con 0x7f74ec101d70 2024-03-22T07:23:00.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.164+0000 7f74d37fe700 1 -- 172.21.15.63:0/2340128986 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74dc004290 con 0x7f74ec101d70 2024-03-22T07:23:00.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.164+0000 7f74f1411700 1 -- 172.21.15.63:0/2340128986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ec11e790 con 0x7f74ec101d70 2024-03-22T07:23:00.166 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.164+0000 7f74d37fe700 1 -- 172.21.15.63:0/2340128986 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74dc01c3f0 con 0x7f74ec101d70 2024-03-22T07:23:00.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.165+0000 7f74f1411700 1 -- 172.21.15.63:0/2340128986 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ec0fd4c0 con 0x7f74ec101d70 2024-03-22T07:23:00.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.166+0000 7f74d37fe700 1 -- 172.21.15.63:0/2340128986 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f74dc0137b0 con 0x7f74ec101d70 2024-03-22T07:23:00.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.167+0000 7f74d37fe700 1 --2- 172.21.15.63:0/2340128986 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f74d406d380 0x7f74d406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:00.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.167+0000 7f74d37fe700 1 -- 172.21.15.63:0/2340128986 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f74dc08dc10 con 0x7f74ec101d70 2024-03-22T07:23:00.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.167+0000 7f74ea7fc700 1 --2- 172.21.15.63:0/2340128986 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f74d406d380 0x7f74d406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:00.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.167+0000 7f74ea7fc700 1 --2- 172.21.15.63:0/2340128986 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f74d406d380 0x7f74d406f830 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f74ec0fa670 tx=0x7f74e0007cc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:00.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.172+0000 7f74d37fe700 1 -- 172.21.15.63:0/2340128986 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f74dc057770 con 0x7f74ec101d70 2024-03-22T07:23:00.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:59 smithi069 ceph-mon[83631]: Upgrade: Updating osd.2 2024-03-22T07:23:00.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:00.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-22T07:23:00.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:23:00.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:22:59 smithi069 ceph-mon[83631]: Deploying daemon osd.2 on smithi063 2024-03-22T07:23:00.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.395+0000 7f74f1411700 1 -- 172.21.15.63:0/2340128986 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f74ec108e70 con 0x7f74d406d380 2024-03-22T07:23:00.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.397+0000 7f74d37fe700 1 -- 172.21.15.63:0/2340128986 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f74ec108e70 con 0x7f74d406d380 2024-03-22T07:23:00.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.399+0000 7f74f1411700 1 -- 172.21.15.63:0/2340128986 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f74d406d380 msgr2=0x7f74d406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:00.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.399+0000 7f74f1411700 1 --2- 172.21.15.63:0/2340128986 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f74d406d380 0x7f74d406f830 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f74ec0fa670 tx=0x7f74e0007cc0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.399+0000 7f74f1411700 1 -- 172.21.15.63:0/2340128986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74ec101d70 msgr2=0x7f74ec102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:00.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.399+0000 7f74f1411700 1 --2- 172.21.15.63:0/2340128986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74ec101d70 0x7f74ec102bf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00b8b0 tx=0x7f74dc009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:00.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.399+0000 7f74f1411700 1 -- 172.21.15.63:0/2340128986 shutdown_connections 2024-03-22T07:23:00.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.399+0000 7f74f1411700 1 --2- 172.21.15.63:0/2340128986 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f74d406d380 0x7f74d406f830 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.399+0000 7f74f1411700 1 --2- 172.21.15.63:0/2340128986 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f74ec101d70 0x7f74ec102bf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.399+0000 7f74f1411700 1 --2- 172.21.15.63:0/2340128986 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74ec102710 0x7f74ec103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.399+0000 7f74f1411700 1 -- 172.21.15.63:0/2340128986 >> 172.21.15.63:0/2340128986 conn(0x7f74ec072bc0 msgr2=0x7f74ec065940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:00.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.399+0000 7f74f1411700 1 -- 172.21.15.63:0/2340128986 shutdown_connections 2024-03-22T07:23:00.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.399+0000 7f74f1411700 1 -- 172.21.15.63:0/2340128986 wait complete. 2024-03-22T07:23:00.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.502+0000 7fbc46656700 1 -- 172.21.15.63:0/3746336976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc400fb570 msgr2=0x7fbc400fb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:00.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.502+0000 7fbc46656700 1 --2- 172.21.15.63:0/3746336976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc400fb570 0x7fbc400fb9e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbc34004940 tx=0x7fbc34009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:00.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.502+0000 7fbc46656700 1 -- 172.21.15.63:0/3746336976 shutdown_connections 2024-03-22T07:23:00.503 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.502+0000 7fbc46656700 1 --2- 172.21.15.63:0/3746336976 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc400fb570 0x7fbc400fb9e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.502+0000 7fbc46656700 1 --2- 172.21.15.63:0/3746336976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc40101530 0x7fbc40101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.502+0000 7fbc46656700 1 -- 172.21.15.63:0/3746336976 >> 172.21.15.63:0/3746336976 conn(0x7fbc400714c0 msgr2=0x7fbc400718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:00.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.502+0000 7fbc46656700 1 -- 172.21.15.63:0/3746336976 shutdown_connections 2024-03-22T07:23:00.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.502+0000 7fbc46656700 1 -- 172.21.15.63:0/3746336976 wait complete. 2024-03-22T07:23:00.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.502+0000 7fbc46656700 1 Processor -- start 2024-03-22T07:23:00.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.503+0000 7fbc46656700 1 -- start start 2024-03-22T07:23:00.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.503+0000 7fbc46656700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc400fb570 0x7fbc4010eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:00.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.503+0000 7fbc46656700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc40101530 0x7fbc4010f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:00.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.503+0000 7fbc46656700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4010f5a0 con 0x7fbc40101530 2024-03-22T07:23:00.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.503+0000 7fbc46656700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4010f6e0 con 0x7fbc400fb570 2024-03-22T07:23:00.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.503+0000 7fbc3f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc40101530 0x7fbc4010f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:00.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.503+0000 7fbc3f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc40101530 0x7fbc4010f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47724/0 (socket says 172.21.15.63:47724) 2024-03-22T07:23:00.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.503+0000 7fbc3f7fe700 1 -- 172.21.15.63:0/3296023406 learned_addr learned my addr 172.21.15.63:0/3296023406 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:23:00.504 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.503+0000 7fbc3ffff700 1 --2- 172.21.15.63:0/3296023406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc400fb570 0x7fbc4010eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:00.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.503+0000 7fbc3f7fe700 1 -- 172.21.15.63:0/3296023406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc400fb570 msgr2=0x7fbc4010eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:00.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.503+0000 7fbc3f7fe700 1 --2- 172.21.15.63:0/3296023406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc400fb570 0x7fbc4010eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.505 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.503+0000 7fbc3f7fe700 1 -- 172.21.15.63:0/3296023406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc34005040 con 0x7fbc40101530 2024-03-22T07:23:00.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.504+0000 7fbc3f7fe700 1 --2- 172.21.15.63:0/3296023406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc40101530 0x7fbc4010f060 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400b8b0 tx=0x7fbc34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:00.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.504+0000 7fbc3cff9700 1 -- 172.21.15.63:0/3296023406 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc3400e040 con 0x7fbc40101530 2024-03-22T07:23:00.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.504+0000 7fbc3cff9700 1 -- 172.21.15.63:0/3296023406 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc34004290 con 0x7fbc40101530 2024-03-22T07:23:00.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.504+0000 7fbc46656700 1 -- 172.21.15.63:0/3296023406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4010f960 con 0x7fbc40101530 2024-03-22T07:23:00.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.504+0000 7fbc3cff9700 1 -- 172.21.15.63:0/3296023406 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc3401c3f0 con 0x7fbc40101530 2024-03-22T07:23:00.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.504+0000 7fbc46656700 1 -- 172.21.15.63:0/3296023406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc40117e90 con 0x7fbc40101530 2024-03-22T07:23:00.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.505+0000 7fbc46656700 1 -- 172.21.15.63:0/3296023406 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc401039e0 con 0x7fbc40101530 2024-03-22T07:23:00.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.505+0000 7fbc3cff9700 1 -- 172.21.15.63:0/3296023406 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fbc34013420 con 0x7fbc40101530 2024-03-22T07:23:00.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.506+0000 7fbc3cff9700 1 --2- 172.21.15.63:0/3296023406 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbc2806d380 0x7fbc2806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:00.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.506+0000 7fbc3ffff700 1 --2- 172.21.15.63:0/3296023406 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbc2806d380 0x7fbc2806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:00.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.506+0000 7fbc3cff9700 1 -- 172.21.15.63:0/3296023406 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbc3408dab0 con 0x7fbc40101530 2024-03-22T07:23:00.507 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.506+0000 7fbc3ffff700 1 --2- 172.21.15.63:0/3296023406 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbc2806d380 0x7fbc2806f830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc30007fd0 tx=0x7fbc30009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:00.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.510+0000 7fbc3cff9700 1 -- 172.21.15.63:0/3296023406 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbc34057610 con 0x7fbc40101530 2024-03-22T07:23:00.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.727+0000 7fbc46656700 1 -- 172.21.15.63:0/3296023406 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc40106880 con 0x7fbc2806d380 2024-03-22T07:23:00.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.738+0000 7fbc3cff9700 1 -- 172.21.15.63:0/3296023406 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fbc40106880 con 0x7fbc2806d380 2024-03-22T07:23:00.738 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:23:00.738 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (4m) 23s ago 14m 26.6M - 0.25.0 c8568f914cd2 06ef21da90e5 2024-03-22T07:23:00.738 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (92s) 23s ago 15m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 224c291ab774 2024-03-22T07:23:00.739 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (86s) 83s ago 14m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 7eabb7d42631 2024-03-22T07:23:00.739 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (3m) 23s ago 14m 83.7M - 9.4.12 a0ea5e3c46d9 9015fae0d5e9 2024-03-22T07:23:00.739 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (9m) 23s ago 9m 74.1M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:23:00.739 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (9m) 23s ago 9m 123M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:23:00.739 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (9m) 83s ago 9m 63.0M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:23:00.739 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (9m) 83s ago 9m 129M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:23:00.739 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:8443,9283,8765 running (5m) 23s ago 16m 505M - 19.0.0-1667-gdb0330b1 4f325f8440e9 e49468f39991 2024-03-22T07:23:00.739 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283,8765 running (4m) 83s ago 13m 448M - 19.0.0-1667-gdb0330b1 4f325f8440e9 13c3f034ebb2 2024-03-22T07:23:00.739 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (2m) 23s ago 16m 46.8M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ab96538a07c1 2024-03-22T07:23:00.739 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (2m) 83s ago 13m 35.5M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 01a26e1fb1a1 2024-03-22T07:23:00.739 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (4m) 23s ago 14m 16.5M - 1.5.0 0da6a335fe13 0c825ee2a196 2024-03-22T07:23:00.739 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (4m) 83s ago 14m 16.9M - 1.5.0 0da6a335fe13 12edb2f35498 2024-03-22T07:23:00.739 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (66s) 23s ago 13m 101M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 fdbc8b34ab78 2024-03-22T07:23:00.740 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (29s) 23s ago 12m 11.7M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 5200fd893046 2024-03-22T07:23:00.740 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (12m) 23s ago 12m 202M 1557M 17.2.7-726-g9c4e0c34 837124da804f 4cd995265dcf 2024-03-22T07:23:00.740 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (11m) 83s ago 11m 281M 2581M 17.2.7-726-g9c4e0c34 837124da804f 45c9a759b76d 2024-03-22T07:23:00.740 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (11m) 83s ago 11m 241M 2581M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:23:00.740 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (10m) 83s ago 10m 240M 2581M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:23:00.740 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (4m) 23s ago 14m 61.1M - 2.43.0 a07b618ecd1d 1d89b928f6e7 2024-03-22T07:23:00.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.742+0000 7fbc46656700 1 -- 172.21.15.63:0/3296023406 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbc2806d380 msgr2=0x7fbc2806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:00.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.742+0000 7fbc46656700 1 --2- 172.21.15.63:0/3296023406 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbc2806d380 0x7fbc2806f830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc30007fd0 tx=0x7fbc30009040 comp rx=0 tx=0).stop 2024-03-22T07:23:00.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.742+0000 7fbc46656700 1 -- 172.21.15.63:0/3296023406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc40101530 msgr2=0x7fbc4010f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:00.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.742+0000 7fbc46656700 1 --2- 172.21.15.63:0/3296023406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc40101530 0x7fbc4010f060 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400b8b0 tx=0x7fbc34009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:00.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.742+0000 7fbc46656700 1 -- 172.21.15.63:0/3296023406 shutdown_connections 2024-03-22T07:23:00.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.742+0000 7fbc46656700 1 --2- 172.21.15.63:0/3296023406 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbc2806d380 0x7fbc2806f830 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.742+0000 7fbc46656700 1 --2- 172.21.15.63:0/3296023406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc400fb570 0x7fbc4010eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.742+0000 7fbc46656700 1 --2- 172.21.15.63:0/3296023406 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc40101530 0x7fbc4010f060 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.742+0000 7fbc46656700 1 -- 172.21.15.63:0/3296023406 >> 172.21.15.63:0/3296023406 conn(0x7fbc400714c0 msgr2=0x7fbc400fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:00.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.742+0000 7fbc46656700 1 -- 172.21.15.63:0/3296023406 shutdown_connections 2024-03-22T07:23:00.743 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.742+0000 7fbc46656700 1 -- 172.21.15.63:0/3296023406 wait complete. 2024-03-22T07:23:00.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.847+0000 7f7ef2c9b700 1 -- 172.21.15.63:0/2538138323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7eec102e00 msgr2=0x7f7eec1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:00.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.847+0000 7f7ef2c9b700 1 --2- 172.21.15.63:0/2538138323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7eec102e00 0x7f7eec1031d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0004940 tx=0x7f7ee0009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:00.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.847+0000 7f7ef2c9b700 1 -- 172.21.15.63:0/2538138323 shutdown_connections 2024-03-22T07:23:00.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.847+0000 7f7ef2c9b700 1 --2- 172.21.15.63:0/2538138323 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7eec1037a0 0x7f7eec0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.848 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.847+0000 7f7ef2c9b700 1 --2- 172.21.15.63:0/2538138323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7eec102e00 0x7f7eec1031d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.847+0000 7f7ef2c9b700 1 -- 172.21.15.63:0/2538138323 >> 172.21.15.63:0/2538138323 conn(0x7f7eec0f89b0 msgr2=0x7f7eec0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:00.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.847+0000 7f7ef2c9b700 1 -- 172.21.15.63:0/2538138323 shutdown_connections 2024-03-22T07:23:00.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.848+0000 7f7ef2c9b700 1 -- 172.21.15.63:0/2538138323 wait complete. 2024-03-22T07:23:00.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.848+0000 7f7ef2c9b700 1 Processor -- start 2024-03-22T07:23:00.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.848+0000 7f7ef2c9b700 1 -- start start 2024-03-22T07:23:00.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.849+0000 7f7ef2c9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7eec102e00 0x7f7eec0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:00.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.849+0000 7f7ef2c9b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7eec1037a0 0x7f7eec070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:00.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.849+0000 7f7ef2c9b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec0721c0 con 0x7f7eec102e00 2024-03-22T07:23:00.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.849+0000 7f7ef2c9b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec070d40 con 0x7f7eec1037a0 2024-03-22T07:23:00.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.849+0000 7f7ef0a37700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7eec102e00 0x7f7eec0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:00.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.849+0000 7f7ef0a37700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7eec102e00 0x7f7eec0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47740/0 (socket says 172.21.15.63:47740) 2024-03-22T07:23:00.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.849+0000 7f7ef0a37700 1 -- 172.21.15.63:0/651879467 learned_addr learned my addr 172.21.15.63:0/651879467 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:23:00.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.849+0000 7f7eebfff700 1 --2- 172.21.15.63:0/651879467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7eec1037a0 0x7f7eec070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:00.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.849+0000 7f7ef0a37700 1 -- 172.21.15.63:0/651879467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7eec1037a0 msgr2=0x7f7eec070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:00.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.849+0000 7f7ef0a37700 1 --2- 172.21.15.63:0/651879467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7eec1037a0 0x7f7eec070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:00.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.849+0000 7f7ef0a37700 1 -- 172.21.15.63:0/651879467 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee0005040 con 0x7f7eec102e00 2024-03-22T07:23:00.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.849+0000 7f7eebfff700 1 --2- 172.21.15.63:0/651879467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7eec1037a0 0x7f7eec070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:23:00.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.849+0000 7f7ef0a37700 1 --2- 172.21.15.63:0/651879467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7eec102e00 0x7f7eec0702c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009b30 tx=0x7f7ee0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:00.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.849+0000 7f7ee97fa700 1 -- 172.21.15.63:0/651879467 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ee0021070 con 0x7f7eec102e00 2024-03-22T07:23:00.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.849+0000 7f7ee97fa700 1 -- 172.21.15.63:0/651879467 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ee0013c00 con 0x7f7eec102e00 2024-03-22T07:23:00.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.849+0000 7f7ef2c9b700 1 -- 172.21.15.63:0/651879467 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eec070fc0 con 0x7f7eec102e00 2024-03-22T07:23:00.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.850+0000 7f7ee97fa700 1 -- 172.21.15.63:0/651879467 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ee000e070 con 0x7f7eec102e00 2024-03-22T07:23:00.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.850+0000 7f7ef2c9b700 1 -- 172.21.15.63:0/651879467 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eec11a010 con 0x7f7eec102e00 2024-03-22T07:23:00.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.850+0000 7f7ef2c9b700 1 -- 172.21.15.63:0/651879467 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eec100110 con 0x7f7eec102e00 2024-03-22T07:23:00.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.851+0000 7f7ee97fa700 1 -- 172.21.15.63:0/651879467 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f7ee001cad0 con 0x7f7eec102e00 2024-03-22T07:23:00.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.851+0000 7f7ee97fa700 1 --2- 172.21.15.63:0/651879467 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7edc06d380 0x7f7edc06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:00.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.852+0000 7f7ee97fa700 1 -- 172.21.15.63:0/651879467 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7ee008dc80 con 0x7f7eec102e00 2024-03-22T07:23:00.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.852+0000 7f7eebfff700 1 --2- 172.21.15.63:0/651879467 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7edc06d380 0x7f7edc06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:00.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.852+0000 7f7eebfff700 1 --2- 172.21.15.63:0/651879467 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7edc06d380 0x7f7edc06f830 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0719a0 tx=0x7f7ed800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:00.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:00.856+0000 7f7ee97fa700 1 -- 172.21.15.63:0/651879467 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7ee00577e0 con 0x7f7eec102e00 2024-03-22T07:23:01.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:00 smithi063 ceph-mon[112784]: pgmap v87: 65 pgs: 65 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-22T07:23:01.108 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:00 smithi063 ceph-mon[112784]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:23:01.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.149+0000 7f7ef2c9b700 1 -- 172.21.15.63:0/651879467 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7eec109f30 con 0x7f7eec102e00 2024-03-22T07:23:01.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.150+0000 7f7ee97fa700 1 -- 172.21.15.63:0/651879467 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f7ee0004490 con 0x7f7eec102e00 2024-03-22T07:23:01.150 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:23:01.150 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:23:01.150 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:23:01.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:23:01.150 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:23:01.150 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:23:01.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:23:01.150 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:23:01.150 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4, 2024-03-22T07:23:01.150 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:23:01.150 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:23:01.151 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:23:01.151 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:23:01.151 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:23:01.151 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:23:01.151 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 8, 2024-03-22T07:23:01.151 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 6 2024-03-22T07:23:01.151 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:23:01.151 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:23:01.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.152+0000 7f7ef2c9b700 1 -- 172.21.15.63:0/651879467 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7edc06d380 msgr2=0x7f7edc06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:01.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.152+0000 7f7ef2c9b700 1 --2- 172.21.15.63:0/651879467 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7edc06d380 0x7f7edc06f830 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0719a0 tx=0x7f7ed800b040 comp rx=0 tx=0).stop 2024-03-22T07:23:01.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.152+0000 7f7ef2c9b700 1 -- 172.21.15.63:0/651879467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7eec102e00 msgr2=0x7f7eec0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:01.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.152+0000 7f7ef2c9b700 1 --2- 172.21.15.63:0/651879467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7eec102e00 0x7f7eec0702c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009b30 tx=0x7f7ee0009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:01.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.152+0000 7f7ef2c9b700 1 -- 172.21.15.63:0/651879467 shutdown_connections 2024-03-22T07:23:01.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.152+0000 7f7ef2c9b700 1 --2- 172.21.15.63:0/651879467 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7edc06d380 0x7f7edc06f830 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.152+0000 7f7ef2c9b700 1 --2- 172.21.15.63:0/651879467 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7eec102e00 0x7f7eec0702c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.152+0000 7f7ef2c9b700 1 --2- 172.21.15.63:0/651879467 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7eec1037a0 0x7f7eec070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.152+0000 7f7ef2c9b700 1 -- 172.21.15.63:0/651879467 >> 172.21.15.63:0/651879467 conn(0x7f7eec0f89b0 msgr2=0x7f7eec107350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:01.152 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.152+0000 7f7ef2c9b700 1 -- 172.21.15.63:0/651879467 shutdown_connections 2024-03-22T07:23:01.153 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.152+0000 7f7ef2c9b700 1 -- 172.21.15.63:0/651879467 wait complete. 2024-03-22T07:23:01.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.256+0000 7fd7ffb01700 1 -- 172.21.15.63:0/238686408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7f8102710 msgr2=0x7fd7f80fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:01.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.256+0000 7fd7ffb01700 1 --2- 172.21.15.63:0/238686408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7f8102710 0x7fd7f80fbd80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8004940 tx=0x7fd7e8009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:01.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.256+0000 7fd7ffb01700 1 -- 172.21.15.63:0/238686408 shutdown_connections 2024-03-22T07:23:01.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.256+0000 7fd7ffb01700 1 --2- 172.21.15.63:0/238686408 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7f8102710 0x7fd7f80fbd80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.256+0000 7fd7ffb01700 1 --2- 172.21.15.63:0/238686408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7f8101d70 0x7fd7f8102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.256+0000 7fd7ffb01700 1 -- 172.21.15.63:0/238686408 >> 172.21.15.63:0/238686408 conn(0x7fd7f8072bc0 msgr2=0x7fd7f8072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:01.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.256+0000 7fd7ffb01700 1 -- 172.21.15.63:0/238686408 shutdown_connections 2024-03-22T07:23:01.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.256+0000 7fd7ffb01700 1 -- 172.21.15.63:0/238686408 wait complete. 2024-03-22T07:23:01.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.256+0000 7fd7ffb01700 1 Processor -- start 2024-03-22T07:23:01.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.256+0000 7fd7ffb01700 1 -- start start 2024-03-22T07:23:01.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.257+0000 7fd7ffb01700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7f8101d70 0x7fd7f8113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:01.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.257+0000 7fd7ffb01700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7f8102710 0x7fd7f81140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:01.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.257+0000 7fd7ffb01700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f81146f0 con 0x7fd7f8101d70 2024-03-22T07:23:01.258 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.257+0000 7fd7ffb01700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f810dae0 con 0x7fd7f8102710 2024-03-22T07:23:01.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.257+0000 7fd7fd89d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7f8101d70 0x7fd7f8113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:01.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.257+0000 7fd7fd89d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7f8101d70 0x7fd7f8113b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47756/0 (socket says 172.21.15.63:47756) 2024-03-22T07:23:01.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.257+0000 7fd7fd89d700 1 -- 172.21.15.63:0/3145655175 learned_addr learned my addr 172.21.15.63:0/3145655175 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:23:01.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.257+0000 7fd7fd09c700 1 --2- 172.21.15.63:0/3145655175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7f8102710 0x7fd7f81140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:01.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.257+0000 7fd7fd89d700 1 -- 172.21.15.63:0/3145655175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7f8102710 msgr2=0x7fd7f81140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:01.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.257+0000 7fd7fd89d700 1 --2- 172.21.15.63:0/3145655175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7f8102710 0x7fd7f81140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.257+0000 7fd7fd89d700 1 -- 172.21.15.63:0/3145655175 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e8005040 con 0x7fd7f8101d70 2024-03-22T07:23:01.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.257+0000 7fd7fd89d700 1 --2- 172.21.15.63:0/3145655175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7f8101d70 0x7fd7f8113b60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4002a50 tx=0x7fd7f40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:01.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.257+0000 7fd7ee7fc700 1 -- 172.21.15.63:0/3145655175 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7f4007cc0 con 0x7fd7f8101d70 2024-03-22T07:23:01.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.258+0000 7fd7ffb01700 1 -- 172.21.15.63:0/3145655175 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f810ddc0 con 0x7fd7f8101d70 2024-03-22T07:23:01.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.258+0000 7fd7ee7fc700 1 -- 172.21.15.63:0/3145655175 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7f400f040 con 0x7fd7f8101d70 2024-03-22T07:23:01.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.258+0000 7fd7ee7fc700 1 -- 172.21.15.63:0/3145655175 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7f400da30 con 0x7fd7f8101d70 2024-03-22T07:23:01.259 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.258+0000 7fd7ffb01700 1 -- 172.21.15.63:0/3145655175 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f810e350 con 0x7fd7f8101d70 2024-03-22T07:23:01.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.259+0000 7fd7ffb01700 1 -- 172.21.15.63:0/3145655175 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f80fd4c0 con 0x7fd7f8101d70 2024-03-22T07:23:01.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.259+0000 7fd7ee7fc700 1 -- 172.21.15.63:0/3145655175 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fd7f400db90 con 0x7fd7f8101d70 2024-03-22T07:23:01.260 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.259+0000 7fd7ee7fc700 1 --2- 172.21.15.63:0/3145655175 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd7e406d380 0x7fd7e406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:01.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.260+0000 7fd7ee7fc700 1 -- 172.21.15.63:0/3145655175 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd7f408c650 con 0x7fd7f8101d70 2024-03-22T07:23:01.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.260+0000 7fd7fd09c700 1 --2- 172.21.15.63:0/3145655175 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd7e406d380 0x7fd7e406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:01.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.260+0000 7fd7fd09c700 1 --2- 172.21.15.63:0/3145655175 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd7e406d380 0x7fd7e406f830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8004890 tx=0x7fd7e8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:01.264 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.264+0000 7fd7ee7fc700 1 -- 172.21.15.63:0/3145655175 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd7f4059c60 con 0x7fd7f8101d70 2024-03-22T07:23:01.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:00 smithi069 ceph-mon[83631]: pgmap v87: 65 pgs: 65 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-22T07:23:01.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:00 smithi069 ceph-mon[83631]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:23:01.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.518+0000 7fd7ffb01700 1 -- 172.21.15.63:0/3145655175 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd7f8108ea0 con 0x7fd7f8101d70 2024-03-22T07:23:01.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.519+0000 7fd7ee7fc700 1 -- 172.21.15.63:0/3145655175 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1868 (secure 0 0 0) 0x7fd7f40597f0 con 0x7fd7f8101d70 2024-03-22T07:23:01.519 INFO:teuthology.orchestra.run.smithi063.stdout:e24 2024-03-22T07:23:01.519 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:23:01.519 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:23:01.519 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-03-22T07:23:01.520 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:23:01.520 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:23:01.520 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-03-22T07:23:01.520 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 24 2024-03-22T07:23:01.520 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-22T07:23:01.520 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-03-22T07:13:29.602905+0000 2024-03-22T07:23:01.520 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-03-22T07:20:24.253589+0000 2024-03-22T07:23:01.520 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-03-22T07:23:01.520 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-03-22T07:23:01.520 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-03-22T07:23:01.520 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-03-22T07:23:01.520 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-03-22T07:23:01.520 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-03-22T07:23:01.520 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-03-22T07:23:01.520 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-03-22T07:23:01.521 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-03-22T07:23:01.521 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:23:01.521 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 2 2024-03-22T07:23:01.521 INFO:teuthology.orchestra.run.smithi063.stdout:in 0,1 2024-03-22T07:23:01.521 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=14480,1=24285} 2024-03-22T07:23:01.521 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-03-22T07:23:01.521 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-03-22T07:23:01.521 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-03-22T07:23:01.521 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-03-22T07:23:01.521 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-03-22T07:23:01.521 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-03-22T07:23:01.521 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-03-22T07:23:01.521 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-03-22T07:23:01.522 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-03-22T07:23:01.522 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.xeelpk{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:23:01.522 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.oytapz{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3211595802,v1:172.21.15.63:6827/3211595802] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:23:01.522 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.wwtsjn{1:24285} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.69:6824/3798247699,v1:172.21.15.69:6825/3798247699] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:23:01.522 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.cijxah{1:24301} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.69:6826/3380719311,v1:172.21.15.69:6827/3380719311] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:23:01.522 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:23:01.522 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:23:01.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.521+0000 7fd7ffb01700 1 -- 172.21.15.63:0/3145655175 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd7e406d380 msgr2=0x7fd7e406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:01.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.521+0000 7fd7ffb01700 1 --2- 172.21.15.63:0/3145655175 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd7e406d380 0x7fd7e406f830 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8004890 tx=0x7fd7e8009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:01.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.521+0000 7fd7ffb01700 1 -- 172.21.15.63:0/3145655175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7f8101d70 msgr2=0x7fd7f8113b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:01.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.521+0000 7fd7ffb01700 1 --2- 172.21.15.63:0/3145655175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7f8101d70 0x7fd7f8113b60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4002a50 tx=0x7fd7f40046b0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.521+0000 7fd7ffb01700 1 -- 172.21.15.63:0/3145655175 shutdown_connections 2024-03-22T07:23:01.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.521+0000 7fd7ffb01700 1 --2- 172.21.15.63:0/3145655175 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd7e406d380 0x7fd7e406f830 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.521+0000 7fd7ffb01700 1 --2- 172.21.15.63:0/3145655175 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd7f8101d70 0x7fd7f8113b60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.523 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.521+0000 7fd7ffb01700 1 --2- 172.21.15.63:0/3145655175 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd7f8102710 0x7fd7f81140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.521+0000 7fd7ffb01700 1 -- 172.21.15.63:0/3145655175 >> 172.21.15.63:0/3145655175 conn(0x7fd7f8072bc0 msgr2=0x7fd7f80659a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:01.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.521+0000 7fd7ffb01700 1 -- 172.21.15.63:0/3145655175 shutdown_connections 2024-03-22T07:23:01.524 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.521+0000 7fd7ffb01700 1 -- 172.21.15.63:0/3145655175 wait complete. 2024-03-22T07:23:01.524 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 24 2024-03-22T07:23:01.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.624+0000 7f62a92f6700 1 -- 172.21.15.63:0/3829687519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62a40fe830 msgr2=0x7f62a4106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:01.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.624+0000 7f62a92f6700 1 --2- 172.21.15.63:0/3829687519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62a40fe830 0x7f62a4106f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6298004940 tx=0x7f6298009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:01.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.624+0000 7f62a92f6700 1 -- 172.21.15.63:0/3829687519 shutdown_connections 2024-03-22T07:23:01.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.624+0000 7f62a92f6700 1 --2- 172.21.15.63:0/3829687519 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62a40fe830 0x7f62a4106f00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.624+0000 7f62a92f6700 1 --2- 172.21.15.63:0/3829687519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62a40fde90 0x7f62a40fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.624+0000 7f62a92f6700 1 -- 172.21.15.63:0/3829687519 >> 172.21.15.63:0/3829687519 conn(0x7f62a4072bc0 msgr2=0x7f62a4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:01.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.624+0000 7f62a92f6700 1 -- 172.21.15.63:0/3829687519 shutdown_connections 2024-03-22T07:23:01.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.624+0000 7f62a92f6700 1 -- 172.21.15.63:0/3829687519 wait complete. 2024-03-22T07:23:01.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.625+0000 7f62a92f6700 1 Processor -- start 2024-03-22T07:23:01.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.625+0000 7f62a92f6700 1 -- start start 2024-03-22T07:23:01.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.625+0000 7f62a92f6700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62a40fde90 0x7f62a40fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:01.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.625+0000 7f62a92f6700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62a40fe830 0x7f62a40fc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:01.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.625+0000 7f62a92f6700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a40fc820 con 0x7f62a40fe830 2024-03-22T07:23:01.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.625+0000 7f62a92f6700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a40fc960 con 0x7f62a40fde90 2024-03-22T07:23:01.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.625+0000 7f62a27fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62a40fe830 0x7f62a40fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:01.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.625+0000 7f62a27fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62a40fe830 0x7f62a40fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47760/0 (socket says 172.21.15.63:47760) 2024-03-22T07:23:01.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.625+0000 7f62a27fc700 1 -- 172.21.15.63:0/930883639 learned_addr learned my addr 172.21.15.63:0/930883639 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:23:01.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.625+0000 7f62a2ffd700 1 --2- 172.21.15.63:0/930883639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62a40fde90 0x7f62a40fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:01.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.626+0000 7f62a27fc700 1 -- 172.21.15.63:0/930883639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62a40fde90 msgr2=0x7f62a40fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:01.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.626+0000 7f62a27fc700 1 --2- 172.21.15.63:0/930883639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62a40fde90 0x7f62a40fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.626+0000 7f62a27fc700 1 -- 172.21.15.63:0/930883639 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6298005040 con 0x7f62a40fe830 2024-03-22T07:23:01.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.626+0000 7f62a27fc700 1 --2- 172.21.15.63:0/930883639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62a40fe830 0x7f62a40fc290 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6298009360 tx=0x7f6298009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:01.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.626+0000 7f628b7fe700 1 -- 172.21.15.63:0/930883639 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f629800e040 con 0x7f62a40fe830 2024-03-22T07:23:01.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.626+0000 7f628b7fe700 1 -- 172.21.15.63:0/930883639 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6298013420 con 0x7f62a40fe830 2024-03-22T07:23:01.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.626+0000 7f62a92f6700 1 -- 172.21.15.63:0/930883639 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a40fcbb0 con 0x7f62a40fe830 2024-03-22T07:23:01.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.626+0000 7f628b7fe700 1 -- 172.21.15.63:0/930883639 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f629801c630 con 0x7f62a40fe830 2024-03-22T07:23:01.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.626+0000 7f62a92f6700 1 -- 172.21.15.63:0/930883639 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a411a1d0 con 0x7f62a40fe830 2024-03-22T07:23:01.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.627+0000 7f62a92f6700 1 -- 172.21.15.63:0/930883639 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a4065e30 con 0x7f62a40fe830 2024-03-22T07:23:01.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.627+0000 7f628b7fe700 1 -- 172.21.15.63:0/930883639 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f6298013a90 con 0x7f62a40fe830 2024-03-22T07:23:01.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.628+0000 7f628b7fe700 1 --2- 172.21.15.63:0/930883639 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f628c06d380 0x7f628c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:01.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.628+0000 7f62a2ffd700 1 --2- 172.21.15.63:0/930883639 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f628c06d380 0x7f628c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:01.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.628+0000 7f628b7fe700 1 -- 172.21.15.63:0/930883639 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f629808d910 con 0x7f62a40fe830 2024-03-22T07:23:01.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.628+0000 7f62a2ffd700 1 --2- 172.21.15.63:0/930883639 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f628c06d380 0x7f628c06f830 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6294009fd0 tx=0x7f629400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:01.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.632+0000 7f628b7fe700 1 -- 172.21.15.63:0/930883639 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6298057510 con 0x7f62a40fe830 2024-03-22T07:23:01.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.860+0000 7f62a92f6700 1 -- 172.21.15.63:0/930883639 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f62a4109f10 con 0x7f628c06d380 2024-03-22T07:23:01.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.862+0000 7f628b7fe700 1 -- 172.21.15.63:0/930883639 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f62a4109f10 con 0x7f628c06d380 2024-03-22T07:23:01.862 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:23:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:23:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:23:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:23:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-03-22T07:23:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-03-22T07:23:01.862 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr", 2024-03-22T07:23:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: "mon" 2024-03-22T07:23:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-03-22T07:23:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "8/21 daemons upgraded", 2024-03-22T07:23:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T07:23:01.863 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:23:01.863 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:23:01.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.864+0000 7f62a92f6700 1 -- 172.21.15.63:0/930883639 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f628c06d380 msgr2=0x7f628c06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:01.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.864+0000 7f62a92f6700 1 --2- 172.21.15.63:0/930883639 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f628c06d380 0x7f628c06f830 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6294009fd0 tx=0x7f629400b040 comp rx=0 tx=0).stop 2024-03-22T07:23:01.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.864+0000 7f62a92f6700 1 -- 172.21.15.63:0/930883639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62a40fe830 msgr2=0x7f62a40fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:01.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.864+0000 7f62a92f6700 1 --2- 172.21.15.63:0/930883639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62a40fe830 0x7f62a40fc290 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6298009360 tx=0x7f6298009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:01.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.864+0000 7f62a92f6700 1 -- 172.21.15.63:0/930883639 shutdown_connections 2024-03-22T07:23:01.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.864+0000 7f62a92f6700 1 --2- 172.21.15.63:0/930883639 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f628c06d380 0x7f628c06f830 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.864 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.864+0000 7f62a92f6700 1 --2- 172.21.15.63:0/930883639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f62a40fde90 0x7f62a40fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.864+0000 7f62a92f6700 1 --2- 172.21.15.63:0/930883639 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f62a40fe830 0x7f62a40fc290 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.864+0000 7f62a92f6700 1 -- 172.21.15.63:0/930883639 >> 172.21.15.63:0/930883639 conn(0x7f62a4072bc0 msgr2=0x7f62a4103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:01.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.864+0000 7f62a92f6700 1 -- 172.21.15.63:0/930883639 shutdown_connections 2024-03-22T07:23:01.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.864+0000 7f62a92f6700 1 -- 172.21.15.63:0/930883639 wait complete. 2024-03-22T07:23:01.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.969+0000 7f02dda6e700 1 -- 172.21.15.63:0/2543376054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02d81026b0 msgr2=0x7f02d80fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:01.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.969+0000 7f02dda6e700 1 --2- 172.21.15.63:0/2543376054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02d81026b0 0x7f02d80fbd20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f02cc004940 tx=0x7f02cc009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:01.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.969+0000 7f02dda6e700 1 -- 172.21.15.63:0/2543376054 shutdown_connections 2024-03-22T07:23:01.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.969+0000 7f02dda6e700 1 --2- 172.21.15.63:0/2543376054 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02d81026b0 0x7f02d80fbd20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.969+0000 7f02dda6e700 1 --2- 172.21.15.63:0/2543376054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02d8101d10 0x7f02d81020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.969+0000 7f02dda6e700 1 -- 172.21.15.63:0/2543376054 >> 172.21.15.63:0/2543376054 conn(0x7f02d80f78e0 msgr2=0x7f02d80f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:01.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.969+0000 7f02dda6e700 1 -- 172.21.15.63:0/2543376054 shutdown_connections 2024-03-22T07:23:01.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.969+0000 7f02dda6e700 1 -- 172.21.15.63:0/2543376054 wait complete. 2024-03-22T07:23:01.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.970+0000 7f02dda6e700 1 Processor -- start 2024-03-22T07:23:01.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.970+0000 7f02dda6e700 1 -- start start 2024-03-22T07:23:01.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.970+0000 7f02dda6e700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02d8101d10 0x7f02d810eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:01.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.970+0000 7f02dda6e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02d81026b0 0x7f02d810f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:01.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.970+0000 7f02dda6e700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d810f5a0 con 0x7f02d81026b0 2024-03-22T07:23:01.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.970+0000 7f02dda6e700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d810f6e0 con 0x7f02d8101d10 2024-03-22T07:23:01.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.971+0000 7f02d6ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02d81026b0 0x7f02d810f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:01.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.971+0000 7f02d6ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02d81026b0 0x7f02d810f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47774/0 (socket says 172.21.15.63:47774) 2024-03-22T07:23:01.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.971+0000 7f02d6ffd700 1 -- 172.21.15.63:0/2554966541 learned_addr learned my addr 172.21.15.63:0/2554966541 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:23:01.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.971+0000 7f02d77fe700 1 --2- 172.21.15.63:0/2554966541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02d8101d10 0x7f02d810eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:01.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.971+0000 7f02d6ffd700 1 -- 172.21.15.63:0/2554966541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02d8101d10 msgr2=0x7f02d810eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:01.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.971+0000 7f02d6ffd700 1 --2- 172.21.15.63:0/2554966541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02d8101d10 0x7f02d810eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:01.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.971+0000 7f02d6ffd700 1 -- 172.21.15.63:0/2554966541 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02cc005040 con 0x7f02d81026b0 2024-03-22T07:23:01.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.971+0000 7f02d6ffd700 1 --2- 172.21.15.63:0/2554966541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02d81026b0 0x7f02d810f060 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009100 tx=0x7f02cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:01.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.971+0000 7f02bffff700 1 -- 172.21.15.63:0/2554966541 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02cc00e040 con 0x7f02d81026b0 2024-03-22T07:23:01.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.971+0000 7f02dda6e700 1 -- 172.21.15.63:0/2554966541 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d810f960 con 0x7f02d81026b0 2024-03-22T07:23:01.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.971+0000 7f02bffff700 1 -- 172.21.15.63:0/2554966541 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02cc009af0 con 0x7f02d81026b0 2024-03-22T07:23:01.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.971+0000 7f02bffff700 1 -- 172.21.15.63:0/2554966541 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02cc01c3f0 con 0x7f02d81026b0 2024-03-22T07:23:01.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.971+0000 7f02dda6e700 1 -- 172.21.15.63:0/2554966541 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d8117e90 con 0x7f02d81026b0 2024-03-22T07:23:01.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.972+0000 7f02dda6e700 1 -- 172.21.15.63:0/2554966541 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d80fd460 con 0x7f02d81026b0 2024-03-22T07:23:01.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.973+0000 7f02bffff700 1 -- 172.21.15.63:0/2554966541 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f02cc00baf0 con 0x7f02d81026b0 2024-03-22T07:23:01.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.973+0000 7f02bffff700 1 --2- 172.21.15.63:0/2554966541 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f02c006d380 0x7f02c006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:01.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.973+0000 7f02bffff700 1 -- 172.21.15.63:0/2554966541 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f02cc08dae0 con 0x7f02d81026b0 2024-03-22T07:23:01.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.973+0000 7f02d77fe700 1 --2- 172.21.15.63:0/2554966541 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f02c006d380 0x7f02c006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:01.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.974+0000 7f02d77fe700 1 --2- 172.21.15.63:0/2554966541 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f02c006d380 0x7f02c006f830 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f02c8004eb0 tx=0x7f02c800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:01.978 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:01.978+0000 7f02bffff700 1 -- 172.21.15.63:0/2554966541 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f02cc057640 con 0x7f02d81026b0 2024-03-22T07:23:02.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:01 smithi063 ceph-mon[112784]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:23:02.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:01 smithi063 ceph-mon[112784]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:23:02.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:01 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/651879467' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:23:02.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:23:02.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:01 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3145655175' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:23:02.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:02.266+0000 7f02dda6e700 1 -- 172.21.15.63:0/2554966541 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f02d8108e40 con 0x7f02d81026b0 2024-03-22T07:23:02.267 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:02.266+0000 7f02bffff700 1 -- 172.21.15.63:0/2554966541 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f02cc05ad00 con 0x7f02d81026b0 2024-03-22T07:23:02.267 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-03-22T07:23:02.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:02.268+0000 7f02dda6e700 1 -- 172.21.15.63:0/2554966541 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f02c006d380 msgr2=0x7f02c006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:02.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:02.268+0000 7f02dda6e700 1 --2- 172.21.15.63:0/2554966541 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f02c006d380 0x7f02c006f830 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f02c8004eb0 tx=0x7f02c800b040 comp rx=0 tx=0).stop 2024-03-22T07:23:02.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:02.268+0000 7f02dda6e700 1 -- 172.21.15.63:0/2554966541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02d81026b0 msgr2=0x7f02d810f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:02.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:02.268+0000 7f02dda6e700 1 --2- 172.21.15.63:0/2554966541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02d81026b0 0x7f02d810f060 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009100 tx=0x7f02cc009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:02.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:02.269+0000 7f02dda6e700 1 -- 172.21.15.63:0/2554966541 shutdown_connections 2024-03-22T07:23:02.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:02.269+0000 7f02dda6e700 1 --2- 172.21.15.63:0/2554966541 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f02c006d380 0x7f02c006f830 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:02.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:02.269+0000 7f02dda6e700 1 --2- 172.21.15.63:0/2554966541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02d8101d10 0x7f02d810eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:02.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:02.269+0000 7f02dda6e700 1 --2- 172.21.15.63:0/2554966541 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f02d81026b0 0x7f02d810f060 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:02.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:02.269+0000 7f02dda6e700 1 -- 172.21.15.63:0/2554966541 >> 172.21.15.63:0/2554966541 conn(0x7f02d80f78e0 msgr2=0x7f02d80f88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:02.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:02.269+0000 7f02dda6e700 1 -- 172.21.15.63:0/2554966541 shutdown_connections 2024-03-22T07:23:02.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:02.269+0000 7f02dda6e700 1 -- 172.21.15.63:0/2554966541 wait complete. 2024-03-22T07:23:02.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:01 smithi069 ceph-mon[83631]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:23:02.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:01 smithi069 ceph-mon[83631]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:23:02.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:01 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/651879467' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:23:02.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:23:02.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:01 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3145655175' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:23:02.832 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:02 smithi063 systemd[1]: Stopping Ceph osd.2 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:23:03.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:02 smithi063 ceph-mon[112784]: pgmap v88: 65 pgs: 65 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-22T07:23:03.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:02 smithi063 ceph-mon[112784]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:23:03.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:02 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2554966541' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:23:03.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:02 smithi063 ceph-mon[112784]: osd.2 marked itself down and dead 2024-03-22T07:23:03.123 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:02 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2[62359]: 2024-03-22T07:23:02.831+0000 7f153ad27700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:23:03.124 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:02 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2[62359]: 2024-03-22T07:23:02.831+0000 7f153ad27700 -1 osd.2 60 *** Got signal Terminated *** 2024-03-22T07:23:03.124 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:02 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2[62359]: 2024-03-22T07:23:02.831+0000 7f153ad27700 -1 osd.2 60 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:23:03.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:02 smithi069 ceph-mon[83631]: pgmap v88: 65 pgs: 65 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-03-22T07:23:03.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:02 smithi069 ceph-mon[83631]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:23:03.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:02 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2554966541' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:23:03.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:02 smithi069 ceph-mon[83631]: osd.2 marked itself down and dead 2024-03-22T07:23:03.607 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:03 smithi063 podman[135293]: 2024-03-22 07:23:03.311262267 +0000 UTC m=+0.613992817 container died 4cd995265dcf123902e09a215c55c177eb742487eb42697dfc42a901a1a5641b (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2, RELEASE=quincy-9c4e0c3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-22T07:23:03.873 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:03 smithi063 podman[135293]: 2024-03-22 07:23:03.697038893 +0000 UTC m=+0.999769443 container cleanup 4cd995265dcf123902e09a215c55c177eb742487eb42697dfc42a901a1a5641b (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2, RELEASE=quincy-9c4e0c3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-22T07:23:03.873 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:03 smithi063 bash[135293]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2 2024-03-22T07:23:04.274 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:03 smithi063 podman[135313]: 2024-03-22 07:23:03.923196424 +0000 UTC m=+0.615547759 container remove 4cd995265dcf123902e09a215c55c177eb742487eb42697dfc42a901a1a5641b (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=quincy-9c4e0c3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:23:04.274 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:03 smithi063 ceph-mon[112784]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:23:04.274 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:03 smithi063 ceph-mon[112784]: failed to encode map e61 with expected crc 2024-03-22T07:23:04.274 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:03 smithi063 ceph-mon[112784]: osdmap e61: 6 total, 5 up, 6 in 2024-03-22T07:23:04.274 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:03 smithi063 ceph-mon[112784]: failed to encode map e61 with expected crc 2024-03-22T07:23:04.274 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:03 smithi063 ceph-mon[112784]: failed to encode map e61 with expected crc 2024-03-22T07:23:04.274 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:03 smithi063 ceph-mon[112784]: osdmap e62: 6 total, 5 up, 6 in 2024-03-22T07:23:04.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:03 smithi069 ceph-mon[83631]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:23:04.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:03 smithi069 ceph-mon[83631]: failed to encode map e61 with expected crc 2024-03-22T07:23:04.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:03 smithi069 ceph-mon[83631]: osdmap e61: 6 total, 5 up, 6 in 2024-03-22T07:23:04.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:03 smithi069 ceph-mon[83631]: failed to encode map e61 with expected crc 2024-03-22T07:23:04.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:03 smithi069 ceph-mon[83631]: failed to encode map e61 with expected crc 2024-03-22T07:23:04.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:03 smithi069 ceph-mon[83631]: osdmap e62: 6 total, 5 up, 6 in 2024-03-22T07:23:04.576 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:04 smithi063 podman[135407]: 2024-03-22 07:23:04.178955659 +0000 UTC m=+0.018872456 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:23:04.577 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:04 smithi063 podman[135407]: 2024-03-22 07:23:04.295486888 +0000 UTC m=+0.135403673 container create 0e2cf01ae00451b945f586412624e022127e0c2746817b83881681ea3dff5c2f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-22T07:23:04.577 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:04 smithi063 podman[135407]: 2024-03-22 07:23:04.577146961 +0000 UTC m=+0.417063748 container init 0e2cf01ae00451b945f586412624e022127e0c2746817b83881681ea3dff5c2f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-db0330b) 2024-03-22T07:23:04.873 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:04 smithi063 podman[135407]: 2024-03-22 07:23:04.580367397 +0000 UTC m=+0.420284191 container start 0e2cf01ae00451b945f586412624e022127e0c2746817b83881681ea3dff5c2f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-deactivate, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-22T07:23:04.873 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:04 smithi063 podman[135407]: 2024-03-22 07:23:04.685542033 +0000 UTC m=+0.525458832 container attach 0e2cf01ae00451b945f586412624e022127e0c2746817b83881681ea3dff5c2f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-db0330b) 2024-03-22T07:23:05.248 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:04 smithi063 ceph-mon[112784]: pgmap v90: 65 pgs: 9 stale+active+clean, 56 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-22T07:23:05.248 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:04 smithi063 ceph-mon[112784]: failed to encode map e62 with expected crc 2024-03-22T07:23:05.248 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:04 smithi063 ceph-mon[112784]: failed to encode map e62 with expected crc 2024-03-22T07:23:05.249 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:04 smithi063 ceph-mon[112784]: failed to encode map e62 with expected crc 2024-03-22T07:23:05.249 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:04 smithi063 ceph-mon[112784]: failed to encode map e62 with expected crc 2024-03-22T07:23:05.249 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:04 smithi063 ceph-mon[112784]: failed to encode map e62 with expected crc 2024-03-22T07:23:05.249 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:04 smithi063 ceph-mon[112784]: failed to encode map e62 with expected crc 2024-03-22T07:23:05.249 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:04 smithi063 ceph-mon[112784]: failed to encode map e62 with expected crc 2024-03-22T07:23:05.249 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:04 smithi063 podman[135407]: 2024-03-22 07:23:04.885291311 +0000 UTC m=+0.725208098 container died 0e2cf01ae00451b945f586412624e022127e0c2746817b83881681ea3dff5c2f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-22T07:23:05.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:04 smithi069 ceph-mon[83631]: pgmap v90: 65 pgs: 9 stale+active+clean, 56 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-22T07:23:05.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:04 smithi069 ceph-mon[83631]: failed to encode map e62 with expected crc 2024-03-22T07:23:05.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:04 smithi069 ceph-mon[83631]: failed to encode map e62 with expected crc 2024-03-22T07:23:05.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:04 smithi069 ceph-mon[83631]: failed to encode map e62 with expected crc 2024-03-22T07:23:05.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:04 smithi069 ceph-mon[83631]: failed to encode map e62 with expected crc 2024-03-22T07:23:05.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:04 smithi069 ceph-mon[83631]: failed to encode map e62 with expected crc 2024-03-22T07:23:05.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:04 smithi069 ceph-mon[83631]: failed to encode map e62 with expected crc 2024-03-22T07:23:05.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:04 smithi069 ceph-mon[83631]: failed to encode map e62 with expected crc 2024-03-22T07:23:05.873 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:05 smithi063 podman[135433]: 2024-03-22 07:23:05.581614716 +0000 UTC m=+0.691616270 container remove 0e2cf01ae00451b945f586412624e022127e0c2746817b83881681ea3dff5c2f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307) 2024-03-22T07:23:05.873 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:05 smithi063 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.2.service: Deactivated successfully. 2024-03-22T07:23:05.873 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:05 smithi063 systemd[1]: Stopped Ceph osd.2 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:23:05.873 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:05 smithi063 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.2.service: Consumed 50.647s CPU time. 2024-03-22T07:23:05.874 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:05 smithi063 systemd[1]: Starting Ceph osd.2 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:23:06.282 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:06 smithi069 ceph-mon[83631]: Health check failed: Degraded data redundancy: 12/354 objects degraded (3.390%), 4 pgs degraded (PG_DEGRADED) 2024-03-22T07:23:06.343 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:06 smithi063 ceph-mon[112784]: Health check failed: Degraded data redundancy: 12/354 objects degraded (3.390%), 4 pgs degraded (PG_DEGRADED) 2024-03-22T07:23:06.343 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 podman[135539]: 2024-03-22 07:23:05.929083111 +0000 UTC m=+0.017735102 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:23:06.343 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 podman[135539]: 2024-03-22 07:23:06.032975762 +0000 UTC m=+0.121627723 container create 827901dbac46151e4438a1919ec257cd3fbffb4b66ff3f03abd9ff65fb74de55 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-22T07:23:06.343 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 podman[135539]: 2024-03-22 07:23:06.260388212 +0000 UTC m=+0.349040173 container init 827901dbac46151e4438a1919ec257cd3fbffb4b66ff3f03abd9ff65fb74de55 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-activate, ceph=True, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307) 2024-03-22T07:23:06.343 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 podman[135539]: 2024-03-22 07:23:06.264478067 +0000 UTC m=+0.353130038 container start 827901dbac46151e4438a1919ec257cd3fbffb4b66ff3f03abd9ff65fb74de55 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-activate, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-22T07:23:06.343 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 podman[135539]: 2024-03-22 07:23:06.343623059 +0000 UTC m=+0.432275034 container attach 827901dbac46151e4438a1919ec257cd3fbffb4b66ff3f03abd9ff65fb74de55 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-activate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-22T07:23:07.017 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-activate[135554]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-22T07:23:07.017 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-activate[135554]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-22T07:23:07.017 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 bash[135539]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-22T07:23:07.017 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 bash[135539]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-22T07:23:07.017 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 bash[135539]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-22T07:23:07.017 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 bash[135539]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-22T07:23:07.018 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 bash[135539]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-22T07:23:07.018 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 bash[135539]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-22T07:23:07.018 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 bash[135539]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-22T07:23:07.018 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-activate[135554]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-22T07:23:07.018 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-activate[135554]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-22T07:23:07.018 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-activate[135554]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-22T07:23:07.018 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-activate[135554]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-22T07:23:07.018 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-activate[135554]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-22T07:23:07.018 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:06 smithi063 podman[135539]: 2024-03-22 07:23:06.736538557 +0000 UTC m=+0.825190519 container died 827901dbac46151e4438a1919ec257cd3fbffb4b66ff3f03abd9ff65fb74de55 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-activate, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:23:07.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:07 smithi069 ceph-mon[83631]: pgmap v92: 65 pgs: 4 active+undersized+degraded, 6 stale+active+clean, 4 active+undersized, 51 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 12/354 objects degraded (3.390%) 2024-03-22T07:23:07.304 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:07 smithi063 ceph-mon[112784]: pgmap v92: 65 pgs: 4 active+undersized+degraded, 6 stale+active+clean, 4 active+undersized, 51 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 12/354 objects degraded (3.390%) 2024-03-22T07:23:07.873 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:07 smithi063 podman[135678]: 2024-03-22 07:23:07.597844636 +0000 UTC m=+0.853130706 container remove 827901dbac46151e4438a1919ec257cd3fbffb4b66ff3f03abd9ff65fb74de55 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-activate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, ceph=True) 2024-03-22T07:23:07.873 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:07 smithi063 podman[135742]: 2024-03-22 07:23:07.720533631 +0000 UTC m=+0.018365051 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:23:07.873 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:07 smithi063 podman[135742]: 2024-03-22 07:23:07.823738252 +0000 UTC m=+0.121569667 container create 918925ee69f35536ff347a6a3fbef9c0d43c1c824da02c1e7c40f58e9feae5b3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-22T07:23:08.373 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:08 smithi063 podman[135742]: 2024-03-22 07:23:08.092975021 +0000 UTC m=+0.390806441 container init 918925ee69f35536ff347a6a3fbef9c0d43c1c824da02c1e7c40f58e9feae5b3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5) 2024-03-22T07:23:08.373 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:08 smithi063 podman[135742]: 2024-03-22 07:23:08.096149035 +0000 UTC m=+0.393980453 container start 918925ee69f35536ff347a6a3fbef9c0d43c1c824da02c1e7c40f58e9feae5b3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:23:08.373 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:08 smithi063 ceph-osd[135761]: -- 172.21.15.63:0/4046514939 <== mon.0 v2:172.21.15.63:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555a10f051e0 con 0x555a10ed6c00 2024-03-22T07:23:08.374 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:08 smithi063 bash[135742]: 918925ee69f35536ff347a6a3fbef9c0d43c1c824da02c1e7c40f58e9feae5b3 2024-03-22T07:23:08.374 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:08 smithi063 systemd[1]: Started Ceph osd.2 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:23:09.095 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:08 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2[135757]: 2024-03-22T07:23:08.639+0000 7f8db52dc7c0 -1 Falling back to public interface 2024-03-22T07:23:09.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:09 smithi063 ceph-mon[112784]: pgmap v93: 65 pgs: 10 active+undersized+degraded, 4 stale+active+clean, 10 active+undersized, 41 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 29/354 objects degraded (8.192%) 2024-03-22T07:23:09.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:09 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:09.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:09 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:09.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:09 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:23:09.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:09 smithi069 ceph-mon[83631]: pgmap v93: 65 pgs: 10 active+undersized+degraded, 4 stale+active+clean, 10 active+undersized, 41 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 29/354 objects degraded (8.192%) 2024-03-22T07:23:09.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:09 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:09.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:09 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:09.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:09 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:23:11.489 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:11 smithi063 ceph-mon[112784]: pgmap v94: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 42/354 objects degraded (11.864%) 2024-03-22T07:23:11.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:11 smithi069 ceph-mon[83631]: pgmap v94: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 42/354 objects degraded (11.864%) 2024-03-22T07:23:12.873 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:12 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2[135757]: 2024-03-22T07:23:12.597+0000 7f8db52dc7c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-03-22T07:23:13.244 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:13 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2[135757]: 2024-03-22T07:23:13.019+0000 7f8db52dc7c0 -1 osd.2 60 log_to_monitors true 2024-03-22T07:23:13.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:13 smithi069 ceph-mon[83631]: pgmap v95: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 42/354 objects degraded (11.864%) 2024-03-22T07:23:13.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:13 smithi069 ceph-mon[83631]: Health check update: Degraded data redundancy: 42/354 objects degraded (11.864%), 13 pgs degraded (PG_DEGRADED) 2024-03-22T07:23:13.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:13 smithi069 ceph-mon[83631]: from='osd.2 [v2:172.21.15.63:6818/2063458996,v1:172.21.15.63:6819/2063458996]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T07:23:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:13 smithi063 ceph-mon[112784]: pgmap v95: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 299 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 42/354 objects degraded (11.864%) 2024-03-22T07:23:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:13 smithi063 ceph-mon[112784]: Health check update: Degraded data redundancy: 42/354 objects degraded (11.864%), 13 pgs degraded (PG_DEGRADED) 2024-03-22T07:23:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:13 smithi063 ceph-mon[112784]: from='osd.2 [v2:172.21.15.63:6818/2063458996,v1:172.21.15.63:6819/2063458996]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-22T07:23:14.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:14 smithi069 ceph-mon[83631]: from='osd.2 [v2:172.21.15.63:6818/2063458996,v1:172.21.15.63:6819/2063458996]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T07:23:14.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:14 smithi069 ceph-mon[83631]: osdmap e63: 6 total, 5 up, 6 in 2024-03-22T07:23:14.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:14 smithi069 ceph-mon[83631]: from='osd.2 [v2:172.21.15.63:6818/2063458996,v1:172.21.15.63:6819/2063458996]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-03-22T07:23:14.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:14 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:14.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:14 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:14.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:14 smithi069 ceph-mon[83631]: from='osd.2 [v2:172.21.15.63:6818/2063458996,v1:172.21.15.63:6819/2063458996]' entity='osd.2' 2024-03-22T07:23:14.623 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:23:14 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2[135757]: 2024-03-22T07:23:14.139+0000 7f8da272c700 -1 osd.2 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:23:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:14 smithi063 ceph-mon[112784]: from='osd.2 [v2:172.21.15.63:6818/2063458996,v1:172.21.15.63:6819/2063458996]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-22T07:23:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:14 smithi063 ceph-mon[112784]: osdmap e63: 6 total, 5 up, 6 in 2024-03-22T07:23:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:14 smithi063 ceph-mon[112784]: from='osd.2 [v2:172.21.15.63:6818/2063458996,v1:172.21.15.63:6819/2063458996]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi063", "root=default"]}]: dispatch 2024-03-22T07:23:14.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:14 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:14.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:14 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:14.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:14 smithi063 ceph-mon[112784]: from='osd.2 [v2:172.21.15.63:6818/2063458996,v1:172.21.15.63:6819/2063458996]' entity='osd.2' 2024-03-22T07:23:15.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:15 smithi069 ceph-mon[83631]: pgmap v97: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 42/354 objects degraded (11.864%) 2024-03-22T07:23:15.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:15 smithi069 ceph-mon[83631]: failed to encode map e63 with expected crc 2024-03-22T07:23:15.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:15 smithi069 ceph-mon[83631]: failed to encode map e63 with expected crc 2024-03-22T07:23:15.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:15 smithi069 ceph-mon[83631]: failed to encode map e63 with expected crc 2024-03-22T07:23:15.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:15 smithi069 ceph-mon[83631]: failed to encode map e63 with expected crc 2024-03-22T07:23:15.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:15 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:15.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:15 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:15.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:15 smithi069 ceph-mon[83631]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:23:15.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:15 smithi069 ceph-mon[83631]: osd.2 [v2:172.21.15.63:6818/2063458996,v1:172.21.15.63:6819/2063458996] boot 2024-03-22T07:23:15.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:15 smithi069 ceph-mon[83631]: osdmap e64: 6 total, 6 up, 6 in 2024-03-22T07:23:15.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:15 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:23:15.588 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:15 smithi063 ceph-mon[112784]: pgmap v97: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 42/354 objects degraded (11.864%) 2024-03-22T07:23:15.588 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:15 smithi063 ceph-mon[112784]: failed to encode map e63 with expected crc 2024-03-22T07:23:15.589 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:15 smithi063 ceph-mon[112784]: failed to encode map e63 with expected crc 2024-03-22T07:23:15.589 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:15 smithi063 ceph-mon[112784]: failed to encode map e63 with expected crc 2024-03-22T07:23:15.589 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:15 smithi063 ceph-mon[112784]: failed to encode map e63 with expected crc 2024-03-22T07:23:15.589 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:15 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:15.589 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:15 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:15.589 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:15 smithi063 ceph-mon[112784]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:23:15.589 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:15 smithi063 ceph-mon[112784]: osd.2 [v2:172.21.15.63:6818/2063458996,v1:172.21.15.63:6819/2063458996] boot 2024-03-22T07:23:15.589 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:15 smithi063 ceph-mon[112784]: osdmap e64: 6 total, 6 up, 6 in 2024-03-22T07:23:15.589 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:15 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-22T07:23:16.509 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:16 smithi063 ceph-mon[112784]: failed to encode map e64 with expected crc 2024-03-22T07:23:16.509 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:16 smithi063 ceph-mon[112784]: failed to encode map e64 with expected crc 2024-03-22T07:23:16.509 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:16 smithi063 ceph-mon[112784]: failed to encode map e64 with expected crc 2024-03-22T07:23:16.509 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:16 smithi063 ceph-mon[112784]: osdmap e65: 6 total, 6 up, 6 in 2024-03-22T07:23:16.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:16 smithi069 ceph-mon[83631]: failed to encode map e64 with expected crc 2024-03-22T07:23:16.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:16 smithi069 ceph-mon[83631]: failed to encode map e64 with expected crc 2024-03-22T07:23:16.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:16 smithi069 ceph-mon[83631]: failed to encode map e64 with expected crc 2024-03-22T07:23:16.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:16 smithi069 ceph-mon[83631]: osdmap e65: 6 total, 6 up, 6 in 2024-03-22T07:23:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:17 smithi063 ceph-mon[112784]: pgmap v99: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 42/354 objects degraded (11.864%) 2024-03-22T07:23:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:17 smithi063 ceph-mon[112784]: failed to encode map e65 with expected crc 2024-03-22T07:23:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:17 smithi063 ceph-mon[112784]: failed to encode map e65 with expected crc 2024-03-22T07:23:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:17 smithi063 ceph-mon[112784]: failed to encode map e65 with expected crc 2024-03-22T07:23:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:17 smithi063 ceph-mon[112784]: failed to encode map e65 with expected crc 2024-03-22T07:23:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:17 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:17.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:17 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:23:17.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:17 smithi069 ceph-mon[83631]: pgmap v99: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 42/354 objects degraded (11.864%) 2024-03-22T07:23:17.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:17 smithi069 ceph-mon[83631]: failed to encode map e65 with expected crc 2024-03-22T07:23:17.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:17 smithi069 ceph-mon[83631]: failed to encode map e65 with expected crc 2024-03-22T07:23:17.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:17 smithi069 ceph-mon[83631]: failed to encode map e65 with expected crc 2024-03-22T07:23:17.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:17 smithi069 ceph-mon[83631]: failed to encode map e65 with expected crc 2024-03-22T07:23:17.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:17 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:17.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:17 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:23:18.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:18 smithi063 ceph-mon[112784]: Health check update: Degraded data redundancy: 26/354 objects degraded (7.345%), 8 pgs degraded (PG_DEGRADED) 2024-03-22T07:23:18.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:18 smithi069 ceph-mon[83631]: Health check update: Degraded data redundancy: 26/354 objects degraded (7.345%), 8 pgs degraded (PG_DEGRADED) 2024-03-22T07:23:19.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:19 smithi063 ceph-mon[112784]: pgmap v101: 65 pgs: 8 active+undersized+degraded, 9 active+undersized, 48 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 4 op/s; 26/354 objects degraded (7.345%) 2024-03-22T07:23:19.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:19 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:19.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:19 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:19.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:19 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:23:19.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:19 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:23:19.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:19 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:19.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:19 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:23:19.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:19 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:23:19.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:19 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:23:19.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:19 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:23:19.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:19 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T07:23:19.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:19 smithi069 ceph-mon[83631]: pgmap v101: 65 pgs: 8 active+undersized+degraded, 9 active+undersized, 48 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 4 op/s; 26/354 objects degraded (7.345%) 2024-03-22T07:23:19.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:19 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:19.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:19 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:19.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:19 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:23:19.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:19 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:23:19.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:19 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:19.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:19 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:23:19.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:19 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:23:19.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:19 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:23:19.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:19 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:23:19.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:19 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T07:23:20.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:20 smithi063 ceph-mon[112784]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T07:23:20.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:20 smithi063 ceph-mon[112784]: Upgrade: unsafe to stop osd(s) at this time (6 PGs are or would become offline) 2024-03-22T07:23:20.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:20 smithi069 ceph-mon[83631]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T07:23:20.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:20 smithi069 ceph-mon[83631]: Upgrade: unsafe to stop osd(s) at this time (6 PGs are or would become offline) 2024-03-22T07:23:21.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:21 smithi063 ceph-mon[112784]: pgmap v102: 65 pgs: 2 active+undersized+degraded, 3 active+undersized, 60 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 3 op/s; 8/354 objects degraded (2.260%) 2024-03-22T07:23:21.680 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:21 smithi069 ceph-mon[83631]: pgmap v102: 65 pgs: 2 active+undersized+degraded, 3 active+undersized, 60 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 3 op/s; 8/354 objects degraded (2.260%) 2024-03-22T07:23:22.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:22 smithi063 ceph-mon[112784]: pgmap v103: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s 2024-03-22T07:23:22.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:22 smithi063 ceph-mon[112784]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/354 objects degraded (2.260%), 2 pgs degraded) 2024-03-22T07:23:22.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:22 smithi063 ceph-mon[112784]: Cluster is now healthy 2024-03-22T07:23:22.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:22 smithi069 ceph-mon[83631]: pgmap v103: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s 2024-03-22T07:23:22.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:22 smithi069 ceph-mon[83631]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/354 objects degraded (2.260%), 2 pgs degraded) 2024-03-22T07:23:22.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:22 smithi069 ceph-mon[83631]: Cluster is now healthy 2024-03-22T07:23:24.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:24 smithi069 ceph-mon[83631]: pgmap v104: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-03-22T07:23:24.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:24 smithi063 ceph-mon[112784]: pgmap v104: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-03-22T07:23:26.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:26 smithi069 ceph-mon[83631]: pgmap v105: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-22T07:23:26.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:26 smithi063 ceph-mon[112784]: pgmap v105: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-03-22T07:23:27.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:27 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:27.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:27 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:28.774 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:28 smithi063 ceph-mon[112784]: pgmap v106: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 3 op/s 2024-03-22T07:23:28.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:28 smithi069 ceph-mon[83631]: pgmap v106: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 3 op/s 2024-03-22T07:23:30.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:30 smithi069 ceph-mon[83631]: pgmap v107: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:23:30.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:30 smithi063 ceph-mon[112784]: pgmap v107: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:23:32.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.385+0000 7f2fde745700 1 -- 172.21.15.63:0/2391881872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fd8102e00 msgr2=0x7f2fd81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:32.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.385+0000 7f2fde745700 1 --2- 172.21.15.63:0/2391881872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fd8102e00 0x7f2fd81031d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc004940 tx=0x7f2fcc009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:32.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.385+0000 7f2fde745700 1 -- 172.21.15.63:0/2391881872 shutdown_connections 2024-03-22T07:23:32.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.385+0000 7f2fde745700 1 --2- 172.21.15.63:0/2391881872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fd81037a0 0x7f2fd80fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:32.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.385+0000 7f2fde745700 1 --2- 172.21.15.63:0/2391881872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fd8102e00 0x7f2fd81031d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:32.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.385+0000 7f2fde745700 1 -- 172.21.15.63:0/2391881872 >> 172.21.15.63:0/2391881872 conn(0x7f2fd80f89b0 msgr2=0x7f2fd80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:32.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.385+0000 7f2fde745700 1 -- 172.21.15.63:0/2391881872 shutdown_connections 2024-03-22T07:23:32.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.385+0000 7f2fde745700 1 -- 172.21.15.63:0/2391881872 wait complete. 2024-03-22T07:23:32.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.386+0000 7f2fde745700 1 Processor -- start 2024-03-22T07:23:32.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.386+0000 7f2fde745700 1 -- start start 2024-03-22T07:23:32.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.386+0000 7f2fde745700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fd8102e00 0x7f2fd80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:32.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.386+0000 7f2fde745700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fd81037a0 0x7f2fd8070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:32.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.386+0000 7f2fde745700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd8070d40 con 0x7f2fd8102e00 2024-03-22T07:23:32.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.386+0000 7f2fde745700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd8070e80 con 0x7f2fd81037a0 2024-03-22T07:23:32.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.387+0000 7f2fd7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fd8102e00 0x7f2fd80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:32.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.387+0000 7f2fd7fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fd8102e00 0x7f2fd80702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51296/0 (socket says 172.21.15.63:51296) 2024-03-22T07:23:32.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.387+0000 7f2fd7fff700 1 -- 172.21.15.63:0/3041607653 learned_addr learned my addr 172.21.15.63:0/3041607653 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:23:32.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.387+0000 7f2fd77fe700 1 --2- 172.21.15.63:0/3041607653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fd81037a0 0x7f2fd8070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:32.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.387+0000 7f2fd7fff700 1 -- 172.21.15.63:0/3041607653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fd81037a0 msgr2=0x7f2fd8070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:32.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.387+0000 7f2fd7fff700 1 --2- 172.21.15.63:0/3041607653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fd81037a0 0x7f2fd8070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:32.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.387+0000 7f2fd7fff700 1 -- 172.21.15.63:0/3041607653 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc005040 con 0x7f2fd8102e00 2024-03-22T07:23:32.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.387+0000 7f2fd7fff700 1 --2- 172.21.15.63:0/3041607653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fd8102e00 0x7f2fd80702c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00b3b0 tx=0x7f2fcc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:32.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.387+0000 7f2fd4ff9700 1 -- 172.21.15.63:0/3041607653 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fcc00e040 con 0x7f2fd8102e00 2024-03-22T07:23:32.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.387+0000 7f2fd4ff9700 1 -- 172.21.15.63:0/3041607653 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2fcc00bbc0 con 0x7f2fd8102e00 2024-03-22T07:23:32.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.387+0000 7f2fde745700 1 -- 172.21.15.63:0/3041607653 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd80710d0 con 0x7f2fd8102e00 2024-03-22T07:23:32.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.387+0000 7f2fd4ff9700 1 -- 172.21.15.63:0/3041607653 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fcc01c3f0 con 0x7f2fd8102e00 2024-03-22T07:23:32.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.387+0000 7f2fde745700 1 -- 172.21.15.63:0/3041607653 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd811a2a0 con 0x7f2fd8102e00 2024-03-22T07:23:32.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.388+0000 7f2fde745700 1 -- 172.21.15.63:0/3041607653 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd8100110 con 0x7f2fd8102e00 2024-03-22T07:23:32.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.389+0000 7f2fd4ff9700 1 -- 172.21.15.63:0/3041607653 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f2fcc00bd30 con 0x7f2fd8102e00 2024-03-22T07:23:32.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.389+0000 7f2fd4ff9700 1 --2- 172.21.15.63:0/3041607653 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2fc006d380 0x7f2fc006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:32.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.389+0000 7f2fd4ff9700 1 -- 172.21.15.63:0/3041607653 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2fcc08dab0 con 0x7f2fd8102e00 2024-03-22T07:23:32.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.389+0000 7f2fd77fe700 1 --2- 172.21.15.63:0/3041607653 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2fc006d380 0x7f2fc006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:32.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.390+0000 7f2fd77fe700 1 --2- 172.21.15.63:0/3041607653 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2fc006d380 0x7f2fc006f830 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8007fd0 tx=0x7f2fc8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:32.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.394+0000 7f2fd4ff9700 1 -- 172.21.15.63:0/3041607653 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2fcc057590 con 0x7f2fd8102e00 2024-03-22T07:23:32.619 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.618+0000 7f2fde745700 1 -- 172.21.15.63:0/3041607653 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fd804bed0 con 0x7f2fc006d380 2024-03-22T07:23:32.620 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.619+0000 7f2fd4ff9700 1 -- 172.21.15.63:0/3041607653 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f2fd804bed0 con 0x7f2fc006d380 2024-03-22T07:23:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.621+0000 7f2fde745700 1 -- 172.21.15.63:0/3041607653 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2fc006d380 msgr2=0x7f2fc006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.621+0000 7f2fde745700 1 --2- 172.21.15.63:0/3041607653 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2fc006d380 0x7f2fc006f830 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8007fd0 tx=0x7f2fc8009040 comp rx=0 tx=0).stop 2024-03-22T07:23:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.621+0000 7f2fde745700 1 -- 172.21.15.63:0/3041607653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fd8102e00 msgr2=0x7f2fd80702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.621+0000 7f2fde745700 1 --2- 172.21.15.63:0/3041607653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fd8102e00 0x7f2fd80702c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00b3b0 tx=0x7f2fcc009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.622+0000 7f2fde745700 1 -- 172.21.15.63:0/3041607653 shutdown_connections 2024-03-22T07:23:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.622+0000 7f2fde745700 1 --2- 172.21.15.63:0/3041607653 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2fc006d380 0x7f2fc006f830 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.622+0000 7f2fde745700 1 --2- 172.21.15.63:0/3041607653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2fd8102e00 0x7f2fd80702c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.622+0000 7f2fde745700 1 --2- 172.21.15.63:0/3041607653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2fd81037a0 0x7f2fd8070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.622+0000 7f2fde745700 1 -- 172.21.15.63:0/3041607653 >> 172.21.15.63:0/3041607653 conn(0x7f2fd80f89b0 msgr2=0x7f2fd81073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.622+0000 7f2fde745700 1 -- 172.21.15.63:0/3041607653 shutdown_connections 2024-03-22T07:23:32.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.622+0000 7f2fde745700 1 -- 172.21.15.63:0/3041607653 wait complete. 2024-03-22T07:23:32.633 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:23:32.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.725+0000 7fbf8915b700 1 -- 172.21.15.63:0/1847960432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf840fbc80 msgr2=0x7fbf840fc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:32.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.725+0000 7fbf8915b700 1 --2- 172.21.15.63:0/1847960432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf840fbc80 0x7fbf840fc050 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c004940 tx=0x7fbf7c009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:32.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.725+0000 7fbf8915b700 1 -- 172.21.15.63:0/1847960432 shutdown_connections 2024-03-22T07:23:32.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.725+0000 7fbf8915b700 1 --2- 172.21.15.63:0/1847960432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf840fc590 0x7fbf841092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:32.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.725+0000 7fbf8915b700 1 --2- 172.21.15.63:0/1847960432 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf840fbc80 0x7fbf840fc050 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:32.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.725+0000 7fbf8915b700 1 -- 172.21.15.63:0/1847960432 >> 172.21.15.63:0/1847960432 conn(0x7fbf84072bc0 msgr2=0x7fbf84072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:32.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.725+0000 7fbf8915b700 1 -- 172.21.15.63:0/1847960432 shutdown_connections 2024-03-22T07:23:32.726 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.725+0000 7fbf8915b700 1 -- 172.21.15.63:0/1847960432 wait complete. 2024-03-22T07:23:32.727 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:32 smithi063 ceph-mon[112784]: pgmap v108: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:23:32.727 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:32.727 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:23:32.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.725+0000 7fbf8915b700 1 Processor -- start 2024-03-22T07:23:32.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.725+0000 7fbf8915b700 1 -- start start 2024-03-22T07:23:32.728 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.726+0000 7fbf8915b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf840fbc80 0x7fbf84114c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:32.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.726+0000 7fbf8915b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf840fc590 0x7fbf841151a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:32.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.726+0000 7fbf8915b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf841157a0 con 0x7fbf840fc590 2024-03-22T07:23:32.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.726+0000 7fbf8915b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8410fd70 con 0x7fbf840fbc80 2024-03-22T07:23:32.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.726+0000 7fbf8259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf840fc590 0x7fbf841151a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:32.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.726+0000 7fbf8259c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf840fc590 0x7fbf841151a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51318/0 (socket says 172.21.15.63:51318) 2024-03-22T07:23:32.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.726+0000 7fbf8259c700 1 -- 172.21.15.63:0/1867081286 learned_addr learned my addr 172.21.15.63:0/1867081286 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:23:32.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.726+0000 7fbf82d9d700 1 --2- 172.21.15.63:0/1867081286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf840fbc80 0x7fbf84114c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:32.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.726+0000 7fbf8259c700 1 -- 172.21.15.63:0/1867081286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf840fbc80 msgr2=0x7fbf84114c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:32.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.726+0000 7fbf8259c700 1 --2- 172.21.15.63:0/1867081286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf840fbc80 0x7fbf84114c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:32.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.726+0000 7fbf8259c700 1 -- 172.21.15.63:0/1867081286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c005040 con 0x7fbf840fc590 2024-03-22T07:23:32.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.726+0000 7fbf8259c700 1 --2- 172.21.15.63:0/1867081286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf840fc590 0x7fbf841151a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c002a50 tx=0x7fbf6c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:32.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.726+0000 7fbf7b7fe700 1 -- 172.21.15.63:0/1867081286 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf6c007cc0 con 0x7fbf840fc590 2024-03-22T07:23:32.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.726+0000 7fbf8915b700 1 -- 172.21.15.63:0/1867081286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf84110050 con 0x7fbf840fc590 2024-03-22T07:23:32.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.726+0000 7fbf7b7fe700 1 -- 172.21.15.63:0/1867081286 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf6c00f040 con 0x7fbf840fc590 2024-03-22T07:23:32.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.726+0000 7fbf7b7fe700 1 -- 172.21.15.63:0/1867081286 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf6c00da30 con 0x7fbf840fc590 2024-03-22T07:23:32.729 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.726+0000 7fbf8915b700 1 -- 172.21.15.63:0/1867081286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf84110610 con 0x7fbf840fc590 2024-03-22T07:23:32.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.727+0000 7fbf8915b700 1 -- 172.21.15.63:0/1867081286 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf84065e30 con 0x7fbf840fc590 2024-03-22T07:23:32.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.728+0000 7fbf7b7fe700 1 -- 172.21.15.63:0/1867081286 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fbf6c00dc70 con 0x7fbf840fc590 2024-03-22T07:23:32.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.728+0000 7fbf7b7fe700 1 --2- 172.21.15.63:0/1867081286 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbf7006d380 0x7fbf7006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:32.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.728+0000 7fbf7b7fe700 1 -- 172.21.15.63:0/1867081286 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbf6c08c650 con 0x7fbf840fc590 2024-03-22T07:23:32.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.728+0000 7fbf82d9d700 1 --2- 172.21.15.63:0/1867081286 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbf7006d380 0x7fbf7006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:32.730 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.729+0000 7fbf82d9d700 1 --2- 172.21.15.63:0/1867081286 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbf7006d380 0x7fbf7006f830 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c004890 tx=0x7fbf7c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:32.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.733+0000 7fbf7b7fe700 1 -- 172.21.15.63:0/1867081286 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbf6c056130 con 0x7fbf840fc590 2024-03-22T07:23:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:32 smithi069 ceph-mon[83631]: pgmap v108: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:23:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:32 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:32 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:23:32.957 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.956+0000 7fbf8915b700 1 -- 172.21.15.63:0/1867081286 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf84111410 con 0x7fbf7006d380 2024-03-22T07:23:32.958 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.957+0000 7fbf7b7fe700 1 -- 172.21.15.63:0/1867081286 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbf84111410 con 0x7fbf7006d380 2024-03-22T07:23:32.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.959+0000 7fbf8915b700 1 -- 172.21.15.63:0/1867081286 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbf7006d380 msgr2=0x7fbf7006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:32.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.959+0000 7fbf8915b700 1 --2- 172.21.15.63:0/1867081286 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbf7006d380 0x7fbf7006f830 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c004890 tx=0x7fbf7c009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:32.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.959+0000 7fbf8915b700 1 -- 172.21.15.63:0/1867081286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf840fc590 msgr2=0x7fbf841151a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:32.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.959+0000 7fbf8915b700 1 --2- 172.21.15.63:0/1867081286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf840fc590 0x7fbf841151a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c002a50 tx=0x7fbf6c0046b0 comp rx=0 tx=0).stop 2024-03-22T07:23:32.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.959+0000 7fbf8915b700 1 -- 172.21.15.63:0/1867081286 shutdown_connections 2024-03-22T07:23:32.960 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.959+0000 7fbf8915b700 1 --2- 172.21.15.63:0/1867081286 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbf7006d380 0x7fbf7006f830 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:32.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.959+0000 7fbf8915b700 1 --2- 172.21.15.63:0/1867081286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf840fbc80 0x7fbf84114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:32.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.959+0000 7fbf8915b700 1 --2- 172.21.15.63:0/1867081286 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbf840fc590 0x7fbf841151a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:32.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.959+0000 7fbf8915b700 1 -- 172.21.15.63:0/1867081286 >> 172.21.15.63:0/1867081286 conn(0x7fbf84072bc0 msgr2=0x7fbf84105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:32.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.959+0000 7fbf8915b700 1 -- 172.21.15.63:0/1867081286 shutdown_connections 2024-03-22T07:23:32.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:32.959+0000 7fbf8915b700 1 -- 172.21.15.63:0/1867081286 wait complete. 2024-03-22T07:23:33.062 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.061+0000 7fbe95c63700 1 -- 172.21.15.63:0/3188295283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe901037c0 msgr2=0x7fbe900fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:33.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.061+0000 7fbe95c63700 1 --2- 172.21.15.63:0/3188295283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe901037c0 0x7fbe900fce30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbe78004940 tx=0x7fbe78009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:33.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.061+0000 7fbe95c63700 1 -- 172.21.15.63:0/3188295283 shutdown_connections 2024-03-22T07:23:33.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.061+0000 7fbe95c63700 1 --2- 172.21.15.63:0/3188295283 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe901037c0 0x7fbe900fce30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:33.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.061+0000 7fbe95c63700 1 --2- 172.21.15.63:0/3188295283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe90102e20 0x7fbe901031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:33.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.061+0000 7fbe95c63700 1 -- 172.21.15.63:0/3188295283 >> 172.21.15.63:0/3188295283 conn(0x7fbe900f89b0 msgr2=0x7fbe900fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:33.063 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.061+0000 7fbe95c63700 1 -- 172.21.15.63:0/3188295283 shutdown_connections 2024-03-22T07:23:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.062+0000 7fbe95c63700 1 -- 172.21.15.63:0/3188295283 wait complete. 2024-03-22T07:23:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.062+0000 7fbe95c63700 1 Processor -- start 2024-03-22T07:23:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.063+0000 7fbe95c63700 1 -- start start 2024-03-22T07:23:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.063+0000 7fbe95c63700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe90102e20 0x7fbe90113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.063+0000 7fbe95c63700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe901037c0 0x7fbe90113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.063+0000 7fbe95c63700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe901144c0 con 0x7fbe90102e20 2024-03-22T07:23:33.064 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.063+0000 7fbe95c63700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9010db10 con 0x7fbe901037c0 2024-03-22T07:23:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.063+0000 7fbe8f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe90102e20 0x7fbe90113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.063+0000 7fbe8f7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe90102e20 0x7fbe90113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51340/0 (socket says 172.21.15.63:51340) 2024-03-22T07:23:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.063+0000 7fbe8f7fe700 1 -- 172.21.15.63:0/590305440 learned_addr learned my addr 172.21.15.63:0/590305440 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:23:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.063+0000 7fbe8effd700 1 --2- 172.21.15.63:0/590305440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe901037c0 0x7fbe90113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.063+0000 7fbe8f7fe700 1 -- 172.21.15.63:0/590305440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe901037c0 msgr2=0x7fbe90113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.063+0000 7fbe8f7fe700 1 --2- 172.21.15.63:0/590305440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe901037c0 0x7fbe90113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:33.065 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.063+0000 7fbe8f7fe700 1 -- 172.21.15.63:0/590305440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe78005040 con 0x7fbe90102e20 2024-03-22T07:23:33.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.063+0000 7fbe8f7fe700 1 --2- 172.21.15.63:0/590305440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe90102e20 0x7fbe90113930 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbe8000ec90 tx=0x7fbe8000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:33.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.064+0000 7fbe87fff700 1 -- 172.21.15.63:0/590305440 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe80007b30 con 0x7fbe90102e20 2024-03-22T07:23:33.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.064+0000 7fbe87fff700 1 -- 172.21.15.63:0/590305440 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe8000f040 con 0x7fbe90102e20 2024-03-22T07:23:33.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.064+0000 7fbe87fff700 1 -- 172.21.15.63:0/590305440 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe8000cc80 con 0x7fbe90102e20 2024-03-22T07:23:33.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.064+0000 7fbe95c63700 1 -- 172.21.15.63:0/590305440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9010ddf0 con 0x7fbe90102e20 2024-03-22T07:23:33.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.064+0000 7fbe95c63700 1 -- 172.21.15.63:0/590305440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9010e300 con 0x7fbe90102e20 2024-03-22T07:23:33.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.065+0000 7fbe95c63700 1 -- 172.21.15.63:0/590305440 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe901052f0 con 0x7fbe90102e20 2024-03-22T07:23:33.066 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.065+0000 7fbe87fff700 1 -- 172.21.15.63:0/590305440 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fbe80007c90 con 0x7fbe90102e20 2024-03-22T07:23:33.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.066+0000 7fbe87fff700 1 --2- 172.21.15.63:0/590305440 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbe7c06d380 0x7fbe7c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:33.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.066+0000 7fbe8effd700 1 --2- 172.21.15.63:0/590305440 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbe7c06d380 0x7fbe7c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:33.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.066+0000 7fbe87fff700 1 -- 172.21.15.63:0/590305440 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbe8008ccf0 con 0x7fbe90102e20 2024-03-22T07:23:33.067 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.066+0000 7fbe8effd700 1 --2- 172.21.15.63:0/590305440 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbe7c06d380 0x7fbe7c06f830 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe9010eda0 tx=0x7fbe78009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:33.072 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.071+0000 7fbe87fff700 1 -- 172.21.15.63:0/590305440 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe8005a300 con 0x7fbe90102e20 2024-03-22T07:23:33.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.282+0000 7fbe95c63700 1 -- 172.21.15.63:0/590305440 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe9010ef80 con 0x7fbe7c06d380 2024-03-22T07:23:33.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.292+0000 7fbe87fff700 1 -- 172.21.15.63:0/590305440 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fbe9010ef80 con 0x7fbe7c06d380 2024-03-22T07:23:33.293 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:23:33.293 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (4m) 19s ago 15m 26.3M - 0.25.0 c8568f914cd2 06ef21da90e5 2024-03-22T07:23:33.293 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (2m) 19s ago 15m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 224c291ab774 2024-03-22T07:23:33.293 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (119s) 115s ago 14m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 7eabb7d42631 2024-03-22T07:23:33.293 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (4m) 19s ago 15m 86.0M - 9.4.12 a0ea5e3c46d9 9015fae0d5e9 2024-03-22T07:23:33.293 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (10m) 19s ago 9m 74.1M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:23:33.293 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (9m) 19s ago 9m 123M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:23:33.293 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (9m) 115s ago 9m 63.0M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:23:33.294 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (9m) 115s ago 9m 129M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:23:33.294 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:8443,9283,8765 running (5m) 19s ago 16m 506M - 19.0.0-1667-gdb0330b1 4f325f8440e9 e49468f39991 2024-03-22T07:23:33.294 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283,8765 running (5m) 115s ago 14m 448M - 19.0.0-1667-gdb0330b1 4f325f8440e9 13c3f034ebb2 2024-03-22T07:23:33.294 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (3m) 19s ago 16m 47.7M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ab96538a07c1 2024-03-22T07:23:33.294 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (2m) 115s ago 14m 35.5M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 01a26e1fb1a1 2024-03-22T07:23:33.294 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (5m) 19s ago 15m 16.0M - 1.5.0 0da6a335fe13 0c825ee2a196 2024-03-22T07:23:33.294 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (5m) 115s ago 14m 16.9M - 1.5.0 0da6a335fe13 12edb2f35498 2024-03-22T07:23:33.294 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (98s) 19s ago 13m 102M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 fdbc8b34ab78 2024-03-22T07:23:33.294 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (62s) 19s ago 13m 98.3M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 5200fd893046 2024-03-22T07:23:33.294 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (25s) 19s ago 12m 11.7M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 918925ee69f3 2024-03-22T07:23:33.294 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (12m) 115s ago 12m 281M 2581M 17.2.7-726-g9c4e0c34 837124da804f 45c9a759b76d 2024-03-22T07:23:33.294 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (11m) 115s ago 11m 241M 2581M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:23:33.294 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (11m) 115s ago 11m 240M 2581M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:23:33.294 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (4m) 19s ago 14m 61.1M - 2.43.0 a07b618ecd1d 1d89b928f6e7 2024-03-22T07:23:33.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.294+0000 7fbe95c63700 1 -- 172.21.15.63:0/590305440 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbe7c06d380 msgr2=0x7fbe7c06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:33.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.294+0000 7fbe95c63700 1 --2- 172.21.15.63:0/590305440 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbe7c06d380 0x7fbe7c06f830 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe9010eda0 tx=0x7fbe78009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:33.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.294+0000 7fbe95c63700 1 -- 172.21.15.63:0/590305440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe90102e20 msgr2=0x7fbe90113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:33.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.294+0000 7fbe95c63700 1 --2- 172.21.15.63:0/590305440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe90102e20 0x7fbe90113930 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbe8000ec90 tx=0x7fbe8000b040 comp rx=0 tx=0).stop 2024-03-22T07:23:33.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.294+0000 7fbe95c63700 1 -- 172.21.15.63:0/590305440 shutdown_connections 2024-03-22T07:23:33.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.294+0000 7fbe95c63700 1 --2- 172.21.15.63:0/590305440 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbe7c06d380 0x7fbe7c06f830 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:33.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.294+0000 7fbe95c63700 1 --2- 172.21.15.63:0/590305440 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbe90102e20 0x7fbe90113930 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:33.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.294+0000 7fbe95c63700 1 --2- 172.21.15.63:0/590305440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbe901037c0 0x7fbe90113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:33.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.294+0000 7fbe95c63700 1 -- 172.21.15.63:0/590305440 >> 172.21.15.63:0/590305440 conn(0x7fbe900f89b0 msgr2=0x7fbe901021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:33.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.294+0000 7fbe95c63700 1 -- 172.21.15.63:0/590305440 shutdown_connections 2024-03-22T07:23:33.296 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.294+0000 7fbe95c63700 1 -- 172.21.15.63:0/590305440 wait complete. 2024-03-22T07:23:33.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.395+0000 7fcae2152700 1 -- 172.21.15.63:0/3640526792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcadc101d20 msgr2=0x7fcadc1020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:33.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.395+0000 7fcae2152700 1 --2- 172.21.15.63:0/3640526792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcadc101d20 0x7fcadc1020f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcacc004940 tx=0x7fcacc009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:33.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.395+0000 7fcae2152700 1 -- 172.21.15.63:0/3640526792 shutdown_connections 2024-03-22T07:23:33.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.395+0000 7fcae2152700 1 --2- 172.21.15.63:0/3640526792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcadc1026c0 0x7fcadc0fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:33.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.395+0000 7fcae2152700 1 --2- 172.21.15.63:0/3640526792 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcadc101d20 0x7fcadc1020f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:33.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.395+0000 7fcae2152700 1 -- 172.21.15.63:0/3640526792 >> 172.21.15.63:0/3640526792 conn(0x7fcadc0f78b0 msgr2=0x7fcadc0f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:33.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.396+0000 7fcae2152700 1 -- 172.21.15.63:0/3640526792 shutdown_connections 2024-03-22T07:23:33.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.396+0000 7fcae2152700 1 -- 172.21.15.63:0/3640526792 wait complete. 2024-03-22T07:23:33.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.396+0000 7fcae2152700 1 Processor -- start 2024-03-22T07:23:33.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.396+0000 7fcae2152700 1 -- start start 2024-03-22T07:23:33.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.396+0000 7fcae2152700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcadc101d20 0x7fcadc10eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:33.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.396+0000 7fcae2152700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcadc1026c0 0x7fcadc10f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:33.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.396+0000 7fcae2152700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc10f560 con 0x7fcadc101d20 2024-03-22T07:23:33.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.396+0000 7fcae2152700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc10f6a0 con 0x7fcadc1026c0 2024-03-22T07:23:33.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.396+0000 7fcadb7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcadc101d20 0x7fcadc10eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:33.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.397+0000 7fcadb7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcadc101d20 0x7fcadc10eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51352/0 (socket says 172.21.15.63:51352) 2024-03-22T07:23:33.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.397+0000 7fcadb7fe700 1 -- 172.21.15.63:0/215875983 learned_addr learned my addr 172.21.15.63:0/215875983 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:23:33.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.397+0000 7fcadaffd700 1 --2- 172.21.15.63:0/215875983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcadc1026c0 0x7fcadc10f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:33.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.397+0000 7fcadb7fe700 1 -- 172.21.15.63:0/215875983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcadc1026c0 msgr2=0x7fcadc10f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:33.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.397+0000 7fcadb7fe700 1 --2- 172.21.15.63:0/215875983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcadc1026c0 0x7fcadc10f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:33.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.397+0000 7fcadb7fe700 1 -- 172.21.15.63:0/215875983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc005040 con 0x7fcadc101d20 2024-03-22T07:23:33.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.397+0000 7fcadb7fe700 1 --2- 172.21.15.63:0/215875983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcadc101d20 0x7fcadc10eae0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009150 tx=0x7fcacc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:33.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.397+0000 7fcac3fff700 1 -- 172.21.15.63:0/215875983 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcacc00e040 con 0x7fcadc101d20 2024-03-22T07:23:33.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.397+0000 7fcac3fff700 1 -- 172.21.15.63:0/215875983 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcacc013420 con 0x7fcadc101d20 2024-03-22T07:23:33.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.397+0000 7fcae2152700 1 -- 172.21.15.63:0/215875983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcadc10f920 con 0x7fcadc101d20 2024-03-22T07:23:33.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.397+0000 7fcac3fff700 1 -- 172.21.15.63:0/215875983 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcacc01c630 con 0x7fcadc101d20 2024-03-22T07:23:33.400 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.397+0000 7fcae2152700 1 -- 172.21.15.63:0/215875983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcadc117ef0 con 0x7fcadc101d20 2024-03-22T07:23:33.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.398+0000 7fcae2152700 1 -- 172.21.15.63:0/215875983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcadc0fd470 con 0x7fcadc101d20 2024-03-22T07:23:33.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.399+0000 7fcac3fff700 1 -- 172.21.15.63:0/215875983 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fcacc013590 con 0x7fcadc101d20 2024-03-22T07:23:33.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.399+0000 7fcac3fff700 1 --2- 172.21.15.63:0/215875983 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fcac406d380 0x7fcac406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:33.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.399+0000 7fcac3fff700 1 -- 172.21.15.63:0/215875983 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcacc08da70 con 0x7fcadc101d20 2024-03-22T07:23:33.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.399+0000 7fcadaffd700 1 --2- 172.21.15.63:0/215875983 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fcac406d380 0x7fcac406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:33.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.400+0000 7fcadaffd700 1 --2- 172.21.15.63:0/215875983 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fcac406d380 0x7fcac406f830 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcad0007fd0 tx=0x7fcad0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:33.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.404+0000 7fcac3fff700 1 -- 172.21.15.63:0/215875983 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcacc057670 con 0x7fcadc101d20 2024-03-22T07:23:33.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.693+0000 7fcae2152700 1 -- 172.21.15.63:0/215875983 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcadc0668c0 con 0x7fcadc101d20 2024-03-22T07:23:33.694 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:33 smithi063 ceph-mon[112784]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:23:33.694 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:33 smithi063 ceph-mon[112784]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:23:33.694 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:33 smithi063 ceph-mon[112784]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:23:33.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.693+0000 7fcac3fff700 1 -- 172.21.15.63:0/215875983 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fcacc05ad30 con 0x7fcadc101d20 2024-03-22T07:23:33.695 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:23:33.695 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:23:33.695 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:23:33.695 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:23:33.695 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:23:33.695 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:23:33.695 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:23:33.695 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:23:33.695 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 3, 2024-03-22T07:23:33.695 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 3 2024-03-22T07:23:33.696 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:23:33.696 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:23:33.696 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:23:33.696 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:23:33.696 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:23:33.696 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 7, 2024-03-22T07:23:33.696 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 7 2024-03-22T07:23:33.696 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:23:33.696 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:23:33.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.695+0000 7fcae2152700 1 -- 172.21.15.63:0/215875983 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fcac406d380 msgr2=0x7fcac406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:33.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.695+0000 7fcae2152700 1 --2- 172.21.15.63:0/215875983 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fcac406d380 0x7fcac406f830 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcad0007fd0 tx=0x7fcad0009040 comp rx=0 tx=0).stop 2024-03-22T07:23:33.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.695+0000 7fcae2152700 1 -- 172.21.15.63:0/215875983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcadc101d20 msgr2=0x7fcadc10eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:33.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.695+0000 7fcae2152700 1 --2- 172.21.15.63:0/215875983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcadc101d20 0x7fcadc10eae0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009150 tx=0x7fcacc009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:33.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.695+0000 7fcae2152700 1 -- 172.21.15.63:0/215875983 shutdown_connections 2024-03-22T07:23:33.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.695+0000 7fcae2152700 1 --2- 172.21.15.63:0/215875983 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fcac406d380 0x7fcac406f830 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:33.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.695+0000 7fcae2152700 1 --2- 172.21.15.63:0/215875983 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcadc101d20 0x7fcadc10eae0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:33.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.695+0000 7fcae2152700 1 --2- 172.21.15.63:0/215875983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcadc1026c0 0x7fcadc10f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:33.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.695+0000 7fcae2152700 1 -- 172.21.15.63:0/215875983 >> 172.21.15.63:0/215875983 conn(0x7fcadc0f78b0 msgr2=0x7fcadc104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:33.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.696+0000 7fcae2152700 1 -- 172.21.15.63:0/215875983 shutdown_connections 2024-03-22T07:23:33.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.696+0000 7fcae2152700 1 -- 172.21.15.63:0/215875983 wait complete. 2024-03-22T07:23:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:33 smithi069 ceph-mon[83631]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:23:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:33 smithi069 ceph-mon[83631]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:23:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:33 smithi069 ceph-mon[83631]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:23:33.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.798+0000 7f96e1548700 1 -- 172.21.15.63:0/1917342365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96dc101d20 msgr2=0x7f96dc1020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:33.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.798+0000 7f96e1548700 1 --2- 172.21.15.63:0/1917342365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96dc101d20 0x7f96dc1020f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f96cc004940 tx=0x7f96cc009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:33.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.798+0000 7f96e1548700 1 -- 172.21.15.63:0/1917342365 shutdown_connections 2024-03-22T07:23:33.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.798+0000 7f96e1548700 1 --2- 172.21.15.63:0/1917342365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96dc1026c0 0x7f96dc0fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:33.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.798+0000 7f96e1548700 1 --2- 172.21.15.63:0/1917342365 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96dc101d20 0x7f96dc1020f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:33.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.798+0000 7f96e1548700 1 -- 172.21.15.63:0/1917342365 >> 172.21.15.63:0/1917342365 conn(0x7f96dc0f7870 msgr2=0x7f96dc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:33.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.799+0000 7f96e1548700 1 -- 172.21.15.63:0/1917342365 shutdown_connections 2024-03-22T07:23:33.800 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.799+0000 7f96e1548700 1 -- 172.21.15.63:0/1917342365 wait complete. 2024-03-22T07:23:33.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.799+0000 7f96e1548700 1 Processor -- start 2024-03-22T07:23:33.801 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.799+0000 7f96e1548700 1 -- start start 2024-03-22T07:23:33.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.799+0000 7f96e1548700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96dc101d20 0x7f96dc113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:33.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.799+0000 7f96e1548700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96dc1026c0 0x7f96dc113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:33.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.799+0000 7f96e1548700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc1144b0 con 0x7f96dc101d20 2024-03-22T07:23:33.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.799+0000 7f96e1548700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc10dab0 con 0x7f96dc1026c0 2024-03-22T07:23:33.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.800+0000 7f96daffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96dc101d20 0x7f96dc113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:33.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.800+0000 7f96daffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96dc101d20 0x7f96dc113920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51358/0 (socket says 172.21.15.63:51358) 2024-03-22T07:23:33.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.800+0000 7f96daffd700 1 -- 172.21.15.63:0/2474660211 learned_addr learned my addr 172.21.15.63:0/2474660211 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:23:33.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.800+0000 7f96da7fc700 1 --2- 172.21.15.63:0/2474660211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96dc1026c0 0x7f96dc113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:33.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.800+0000 7f96daffd700 1 -- 172.21.15.63:0/2474660211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96dc1026c0 msgr2=0x7f96dc113e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:33.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.800+0000 7f96daffd700 1 --2- 172.21.15.63:0/2474660211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96dc1026c0 0x7f96dc113e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:33.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.800+0000 7f96daffd700 1 -- 172.21.15.63:0/2474660211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc005040 con 0x7f96dc101d20 2024-03-22T07:23:33.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.800+0000 7f96daffd700 1 --2- 172.21.15.63:0/2474660211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96dc101d20 0x7f96dc113920 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009b30 tx=0x7f96cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:33.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.800+0000 7f96c37fe700 1 -- 172.21.15.63:0/2474660211 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96cc021070 con 0x7f96dc101d20 2024-03-22T07:23:33.802 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.800+0000 7f96c37fe700 1 -- 172.21.15.63:0/2474660211 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f96cc013c00 con 0x7f96dc101d20 2024-03-22T07:23:33.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.800+0000 7f96e1548700 1 -- 172.21.15.63:0/2474660211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96dc10dd30 con 0x7f96dc101d20 2024-03-22T07:23:33.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.800+0000 7f96c37fe700 1 -- 172.21.15.63:0/2474660211 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96cc00e070 con 0x7f96dc101d20 2024-03-22T07:23:33.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.800+0000 7f96e1548700 1 -- 172.21.15.63:0/2474660211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96dc10e210 con 0x7f96dc101d20 2024-03-22T07:23:33.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.801+0000 7f96e1548700 1 -- 172.21.15.63:0/2474660211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96dc0fd470 con 0x7f96dc101d20 2024-03-22T07:23:33.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.802+0000 7f96c37fe700 1 -- 172.21.15.63:0/2474660211 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f96cc01cad0 con 0x7f96dc101d20 2024-03-22T07:23:33.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.802+0000 7f96c37fe700 1 --2- 172.21.15.63:0/2474660211 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f96c406d380 0x7f96c406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:33.803 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.802+0000 7f96da7fc700 1 --2- 172.21.15.63:0/2474660211 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f96c406d380 0x7f96c406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:33.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.802+0000 7f96c37fe700 1 -- 172.21.15.63:0/2474660211 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f96cc08dcf0 con 0x7f96dc101d20 2024-03-22T07:23:33.804 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.803+0000 7f96da7fc700 1 --2- 172.21.15.63:0/2474660211 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f96c406d380 0x7f96c406f830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f96dc10ec70 tx=0x7f96d000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:33.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:33.807+0000 7f96c37fe700 1 -- 172.21.15.63:0/2474660211 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f96cc05b380 con 0x7f96dc101d20 2024-03-22T07:23:34.053 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.052+0000 7f96e1548700 1 -- 172.21.15.63:0/2474660211 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f96dc0fd470 con 0x7f96dc101d20 2024-03-22T07:23:34.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.053+0000 7f96c37fe700 1 -- 172.21.15.63:0/2474660211 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1868 (secure 0 0 0) 0x7f96cc004490 con 0x7f96dc101d20 2024-03-22T07:23:34.054 INFO:teuthology.orchestra.run.smithi063.stdout:e24 2024-03-22T07:23:34.054 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:23:34.054 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:23:34.054 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-03-22T07:23:34.054 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:23:34.054 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:23:34.054 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-03-22T07:23:34.054 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 24 2024-03-22T07:23:34.055 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-22T07:23:34.055 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-03-22T07:13:29.602905+0000 2024-03-22T07:23:34.055 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-03-22T07:20:24.253589+0000 2024-03-22T07:23:34.055 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-03-22T07:23:34.055 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-03-22T07:23:34.055 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-03-22T07:23:34.055 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-03-22T07:23:34.055 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-03-22T07:23:34.055 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-03-22T07:23:34.055 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-03-22T07:23:34.055 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-03-22T07:23:34.055 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-03-22T07:23:34.055 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:23:34.055 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 2 2024-03-22T07:23:34.055 INFO:teuthology.orchestra.run.smithi063.stdout:in 0,1 2024-03-22T07:23:34.055 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=14480,1=24285} 2024-03-22T07:23:34.055 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-03-22T07:23:34.056 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-03-22T07:23:34.056 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-03-22T07:23:34.056 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-03-22T07:23:34.056 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-03-22T07:23:34.056 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-03-22T07:23:34.056 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-03-22T07:23:34.056 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-03-22T07:23:34.056 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-03-22T07:23:34.056 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.xeelpk{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:23:34.056 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.oytapz{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3211595802,v1:172.21.15.63:6827/3211595802] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:23:34.056 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.wwtsjn{1:24285} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.69:6824/3798247699,v1:172.21.15.69:6825/3798247699] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:23:34.056 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.cijxah{1:24301} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.69:6826/3380719311,v1:172.21.15.69:6827/3380719311] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:23:34.056 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:23:34.056 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:23:34.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.055+0000 7f96e1548700 1 -- 172.21.15.63:0/2474660211 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f96c406d380 msgr2=0x7f96c406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:34.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.055+0000 7f96e1548700 1 --2- 172.21.15.63:0/2474660211 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f96c406d380 0x7f96c406f830 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f96dc10ec70 tx=0x7f96d000b040 comp rx=0 tx=0).stop 2024-03-22T07:23:34.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.055+0000 7f96e1548700 1 -- 172.21.15.63:0/2474660211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96dc101d20 msgr2=0x7f96dc113920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:34.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.055+0000 7f96e1548700 1 --2- 172.21.15.63:0/2474660211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96dc101d20 0x7f96dc113920 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009b30 tx=0x7f96cc009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:34.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.055+0000 7f96e1548700 1 -- 172.21.15.63:0/2474660211 shutdown_connections 2024-03-22T07:23:34.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.055+0000 7f96e1548700 1 --2- 172.21.15.63:0/2474660211 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f96c406d380 0x7f96c406f830 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:34.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.055+0000 7f96e1548700 1 --2- 172.21.15.63:0/2474660211 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f96dc101d20 0x7f96dc113920 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:34.057 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.055+0000 7f96e1548700 1 --2- 172.21.15.63:0/2474660211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96dc1026c0 0x7f96dc113e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:34.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.055+0000 7f96e1548700 1 -- 172.21.15.63:0/2474660211 >> 172.21.15.63:0/2474660211 conn(0x7f96dc0f7870 msgr2=0x7f96dc0f8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:34.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.055+0000 7f96e1548700 1 -- 172.21.15.63:0/2474660211 shutdown_connections 2024-03-22T07:23:34.058 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.055+0000 7f96e1548700 1 -- 172.21.15.63:0/2474660211 wait complete. 2024-03-22T07:23:34.058 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 24 2024-03-22T07:23:34.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.158+0000 7f0520db9700 1 -- 172.21.15.63:0/1380036201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f051c1037a0 msgr2=0x7f051c0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:34.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.158+0000 7f0520db9700 1 --2- 172.21.15.63:0/1380036201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f051c1037a0 0x7f051c0fce10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f050c004940 tx=0x7f050c009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:34.159 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.158+0000 7f0520db9700 1 -- 172.21.15.63:0/1380036201 shutdown_connections 2024-03-22T07:23:34.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.158+0000 7f0520db9700 1 --2- 172.21.15.63:0/1380036201 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f051c1037a0 0x7f051c0fce10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:34.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.158+0000 7f0520db9700 1 --2- 172.21.15.63:0/1380036201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f051c102e00 0x7f051c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:34.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.158+0000 7f0520db9700 1 -- 172.21.15.63:0/1380036201 >> 172.21.15.63:0/1380036201 conn(0x7f051c0f89f0 msgr2=0x7f051c0fae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:34.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.158+0000 7f0520db9700 1 -- 172.21.15.63:0/1380036201 shutdown_connections 2024-03-22T07:23:34.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.158+0000 7f0520db9700 1 -- 172.21.15.63:0/1380036201 wait complete. 2024-03-22T07:23:34.161 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.159+0000 7f0520db9700 1 Processor -- start 2024-03-22T07:23:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.159+0000 7f0520db9700 1 -- start start 2024-03-22T07:23:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.159+0000 7f0520db9700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f051c102e00 0x7f051c113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.159+0000 7f0520db9700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f051c1037a0 0x7f051c113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.159+0000 7f0520db9700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051c1144b0 con 0x7f051c1037a0 2024-03-22T07:23:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.159+0000 7f0520db9700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051c10dab0 con 0x7f051c102e00 2024-03-22T07:23:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.159+0000 7f0519d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f051c1037a0 0x7f051c113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.159+0000 7f0519d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f051c1037a0 0x7f051c113e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51370/0 (socket says 172.21.15.63:51370) 2024-03-22T07:23:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.159+0000 7f0519d9b700 1 -- 172.21.15.63:0/2165044449 learned_addr learned my addr 172.21.15.63:0/2165044449 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:23:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.159+0000 7f051a59c700 1 --2- 172.21.15.63:0/2165044449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f051c102e00 0x7f051c113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.159+0000 7f0519d9b700 1 -- 172.21.15.63:0/2165044449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f051c102e00 msgr2=0x7f051c113920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.159+0000 7f0519d9b700 1 --2- 172.21.15.63:0/2165044449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f051c102e00 0x7f051c113920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.160+0000 7f0519d9b700 1 -- 172.21.15.63:0/2165044449 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f050c005040 con 0x7f051c1037a0 2024-03-22T07:23:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.160+0000 7f0519d9b700 1 --2- 172.21.15.63:0/2165044449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f051c1037a0 0x7f051c113e60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f050c009100 tx=0x7f050c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.160+0000 7f0512ffd700 1 -- 172.21.15.63:0/2165044449 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f050c00e040 con 0x7f051c1037a0 2024-03-22T07:23:34.162 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.160+0000 7f0512ffd700 1 -- 172.21.15.63:0/2165044449 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f050c009af0 con 0x7f051c1037a0 2024-03-22T07:23:34.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.160+0000 7f0520db9700 1 -- 172.21.15.63:0/2165044449 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f051c10dd30 con 0x7f051c1037a0 2024-03-22T07:23:34.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.160+0000 7f0512ffd700 1 -- 172.21.15.63:0/2165044449 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f050c01c3f0 con 0x7f051c1037a0 2024-03-22T07:23:34.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.160+0000 7f0520db9700 1 -- 172.21.15.63:0/2165044449 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f051c10e2f0 con 0x7f051c1037a0 2024-03-22T07:23:34.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.161+0000 7f0520db9700 1 -- 172.21.15.63:0/2165044449 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f051c1052d0 con 0x7f051c1037a0 2024-03-22T07:23:34.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.161+0000 7f0512ffd700 1 -- 172.21.15.63:0/2165044449 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f050c00baf0 con 0x7f051c1037a0 2024-03-22T07:23:34.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.162+0000 7f0512ffd700 1 --2- 172.21.15.63:0/2165044449 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f050806d380 0x7f050806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:34.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.162+0000 7f0512ffd700 1 -- 172.21.15.63:0/2165044449 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f050c08dae0 con 0x7f051c1037a0 2024-03-22T07:23:34.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.162+0000 7f051a59c700 1 --2- 172.21.15.63:0/2165044449 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f050806d380 0x7f050806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:34.164 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.162+0000 7f051a59c700 1 --2- 172.21.15.63:0/2165044449 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f050806d380 0x7f050806f830 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0504009fd0 tx=0x7f050400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:34.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.166+0000 7f0512ffd700 1 -- 172.21.15.63:0/2165044449 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f050c05b170 con 0x7f051c1037a0 2024-03-22T07:23:34.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.390+0000 7f0520db9700 1 -- 172.21.15.63:0/2165044449 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f051c04c780 con 0x7f050806d380 2024-03-22T07:23:34.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.391+0000 7f0512ffd700 1 -- 172.21.15.63:0/2165044449 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f051c04c780 con 0x7f050806d380 2024-03-22T07:23:34.392 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:23:34.392 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:23:34.392 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:23:34.392 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:23:34.392 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-03-22T07:23:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-03-22T07:23:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr", 2024-03-22T07:23:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: "mon" 2024-03-22T07:23:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-03-22T07:23:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "9/21 daemons upgraded", 2024-03-22T07:23:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T07:23:34.393 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:23:34.393 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:23:34.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.393+0000 7f0520db9700 1 -- 172.21.15.63:0/2165044449 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f050806d380 msgr2=0x7f050806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:34.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.393+0000 7f0520db9700 1 --2- 172.21.15.63:0/2165044449 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f050806d380 0x7f050806f830 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0504009fd0 tx=0x7f050400b040 comp rx=0 tx=0).stop 2024-03-22T07:23:34.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.393+0000 7f0520db9700 1 -- 172.21.15.63:0/2165044449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f051c1037a0 msgr2=0x7f051c113e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:34.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.393+0000 7f0520db9700 1 --2- 172.21.15.63:0/2165044449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f051c1037a0 0x7f051c113e60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f050c009100 tx=0x7f050c009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:34.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.393+0000 7f0520db9700 1 -- 172.21.15.63:0/2165044449 shutdown_connections 2024-03-22T07:23:34.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.393+0000 7f0520db9700 1 --2- 172.21.15.63:0/2165044449 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f050806d380 0x7f050806f830 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:34.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.393+0000 7f0520db9700 1 --2- 172.21.15.63:0/2165044449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f051c102e00 0x7f051c113920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:34.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.393+0000 7f0520db9700 1 --2- 172.21.15.63:0/2165044449 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f051c1037a0 0x7f051c113e60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:34.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.393+0000 7f0520db9700 1 -- 172.21.15.63:0/2165044449 >> 172.21.15.63:0/2165044449 conn(0x7f051c0f89f0 msgr2=0x7f051c0fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:34.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.393+0000 7f0520db9700 1 -- 172.21.15.63:0/2165044449 shutdown_connections 2024-03-22T07:23:34.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.393+0000 7f0520db9700 1 -- 172.21.15.63:0/2165044449 wait complete. 2024-03-22T07:23:34.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.496+0000 7fce51057700 1 -- 172.21.15.63:0/3149725132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce4c101d20 msgr2=0x7fce4c1020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:34.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.496+0000 7fce51057700 1 --2- 172.21.15.63:0/3149725132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce4c101d20 0x7fce4c1020f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fce3c004940 tx=0x7fce3c009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:34.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.496+0000 7fce51057700 1 -- 172.21.15.63:0/3149725132 shutdown_connections 2024-03-22T07:23:34.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.496+0000 7fce51057700 1 --2- 172.21.15.63:0/3149725132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce4c1026c0 0x7fce4c0fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:34.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.496+0000 7fce51057700 1 --2- 172.21.15.63:0/3149725132 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce4c101d20 0x7fce4c1020f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:34.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.496+0000 7fce51057700 1 -- 172.21.15.63:0/3149725132 >> 172.21.15.63:0/3149725132 conn(0x7fce4c0f7870 msgr2=0x7fce4c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:34.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.496+0000 7fce51057700 1 -- 172.21.15.63:0/3149725132 shutdown_connections 2024-03-22T07:23:34.498 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.497+0000 7fce51057700 1 -- 172.21.15.63:0/3149725132 wait complete. 2024-03-22T07:23:34.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.497+0000 7fce51057700 1 Processor -- start 2024-03-22T07:23:34.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.497+0000 7fce51057700 1 -- start start 2024-03-22T07:23:34.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.497+0000 7fce51057700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce4c101d20 0x7fce4c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:34.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.497+0000 7fce51057700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce4c1026c0 0x7fce4c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:34.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.497+0000 7fce51057700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c114590 con 0x7fce4c101d20 2024-03-22T07:23:34.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.497+0000 7fce51057700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c10db00 con 0x7fce4c1026c0 2024-03-22T07:23:34.500 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.498+0000 7fce4ad9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce4c101d20 0x7fce4c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:34.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.498+0000 7fce4ad9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce4c101d20 0x7fce4c113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:51396/0 (socket says 172.21.15.63:51396) 2024-03-22T07:23:34.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.498+0000 7fce4ad9d700 1 -- 172.21.15.63:0/3122332508 learned_addr learned my addr 172.21.15.63:0/3122332508 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:23:34.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.498+0000 7fce4a59c700 1 --2- 172.21.15.63:0/3122332508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce4c1026c0 0x7fce4c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:34.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.498+0000 7fce4ad9d700 1 -- 172.21.15.63:0/3122332508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce4c1026c0 msgr2=0x7fce4c113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:34.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.498+0000 7fce4ad9d700 1 --2- 172.21.15.63:0/3122332508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce4c1026c0 0x7fce4c113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:34.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.498+0000 7fce4ad9d700 1 -- 172.21.15.63:0/3122332508 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c005040 con 0x7fce4c101d20 2024-03-22T07:23:34.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.498+0000 7fce4ad9d700 1 --2- 172.21.15.63:0/3122332508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce4c101d20 0x7fce4c113970 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00b3b0 tx=0x7fce3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:34.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.498+0000 7fce337fe700 1 -- 172.21.15.63:0/3122332508 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce3c00e040 con 0x7fce4c101d20 2024-03-22T07:23:34.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.498+0000 7fce337fe700 1 -- 172.21.15.63:0/3122332508 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce3c00bbc0 con 0x7fce4c101d20 2024-03-22T07:23:34.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.498+0000 7fce51057700 1 -- 172.21.15.63:0/3122332508 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4c10dd80 con 0x7fce4c101d20 2024-03-22T07:23:34.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.498+0000 7fce337fe700 1 -- 172.21.15.63:0/3122332508 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce3c01c3f0 con 0x7fce4c101d20 2024-03-22T07:23:34.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.498+0000 7fce51057700 1 -- 172.21.15.63:0/3122332508 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c10e340 con 0x7fce4c101d20 2024-03-22T07:23:34.501 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.499+0000 7fce51057700 1 -- 172.21.15.63:0/3122332508 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4c106420 con 0x7fce4c101d20 2024-03-22T07:23:34.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.500+0000 7fce337fe700 1 -- 172.21.15.63:0/3122332508 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fce3c00bd30 con 0x7fce4c101d20 2024-03-22T07:23:34.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.500+0000 7fce337fe700 1 --2- 172.21.15.63:0/3122332508 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fce3406d380 0x7fce3406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:23:34.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.500+0000 7fce337fe700 1 -- 172.21.15.63:0/3122332508 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fce3c08da50 con 0x7fce4c101d20 2024-03-22T07:23:34.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.500+0000 7fce4a59c700 1 --2- 172.21.15.63:0/3122332508 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fce3406d380 0x7fce3406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:23:34.502 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.501+0000 7fce4a59c700 1 --2- 172.21.15.63:0/3122332508 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fce3406d380 0x7fce3406f830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fce40006fd0 tx=0x7fce40008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:23:34.506 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.505+0000 7fce337fe700 1 -- 172.21.15.63:0/3122332508 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fce3c05b0e0 con 0x7fce4c101d20 2024-03-22T07:23:34.609 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:34 smithi069 ceph-mon[83631]: pgmap v109: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:23:34.609 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:34 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/215875983' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:23:34.609 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:34 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T07:23:34.609 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:34 smithi069 ceph-mon[83631]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T07:23:34.609 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:34 smithi069 ceph-mon[83631]: Upgrade: osd.3 is safe to restart 2024-03-22T07:23:34.609 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:34 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2474660211' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:23:34.795 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.794+0000 7fce51057700 1 -- 172.21.15.63:0/3122332508 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fce4c10efc0 con 0x7fce4c101d20 2024-03-22T07:23:34.796 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:34 smithi063 ceph-mon[112784]: pgmap v109: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:23:34.796 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:34 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/215875983' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:23:34.796 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:34 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T07:23:34.796 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:34 smithi063 ceph-mon[112784]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-22T07:23:34.796 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:34 smithi063 ceph-mon[112784]: Upgrade: osd.3 is safe to restart 2024-03-22T07:23:34.796 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:34 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2474660211' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:23:34.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.795+0000 7fce337fe700 1 -- 172.21.15.63:0/3122332508 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fce3c05ac70 con 0x7fce4c101d20 2024-03-22T07:23:34.796 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-03-22T07:23:34.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.797+0000 7fce51057700 1 -- 172.21.15.63:0/3122332508 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fce3406d380 msgr2=0x7fce3406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:34.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.797+0000 7fce51057700 1 --2- 172.21.15.63:0/3122332508 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fce3406d380 0x7fce3406f830 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fce40006fd0 tx=0x7fce40008040 comp rx=0 tx=0).stop 2024-03-22T07:23:34.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.797+0000 7fce51057700 1 -- 172.21.15.63:0/3122332508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce4c101d20 msgr2=0x7fce4c113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:23:34.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.797+0000 7fce51057700 1 --2- 172.21.15.63:0/3122332508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce4c101d20 0x7fce4c113970 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00b3b0 tx=0x7fce3c009e30 comp rx=0 tx=0).stop 2024-03-22T07:23:34.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.797+0000 7fce51057700 1 -- 172.21.15.63:0/3122332508 shutdown_connections 2024-03-22T07:23:34.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.797+0000 7fce51057700 1 --2- 172.21.15.63:0/3122332508 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fce3406d380 0x7fce3406f830 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:34.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.797+0000 7fce51057700 1 --2- 172.21.15.63:0/3122332508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fce4c101d20 0x7fce4c113970 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:34.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.797+0000 7fce51057700 1 --2- 172.21.15.63:0/3122332508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fce4c1026c0 0x7fce4c113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:23:34.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.797+0000 7fce51057700 1 -- 172.21.15.63:0/3122332508 >> 172.21.15.63:0/3122332508 conn(0x7fce4c0f7870 msgr2=0x7fce4c066b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:23:34.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.797+0000 7fce51057700 1 -- 172.21.15.63:0/3122332508 shutdown_connections 2024-03-22T07:23:34.799 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:23:34.797+0000 7fce51057700 1 -- 172.21.15.63:0/3122332508 wait complete. 2024-03-22T07:23:35.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:35 smithi069 ceph-mon[83631]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:23:35.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:35 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3122332508' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:23:35.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:35 smithi069 ceph-mon[83631]: Upgrade: Updating osd.3 2024-03-22T07:23:35.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:35 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:35.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:35 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T07:23:35.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:35 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:23:35.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:35 smithi069 ceph-mon[83631]: Deploying daemon osd.3 on smithi069 2024-03-22T07:23:35.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:35 smithi063 ceph-mon[112784]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:23:35.893 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:35 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3122332508' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:23:35.893 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:35 smithi063 ceph-mon[112784]: Upgrade: Updating osd.3 2024-03-22T07:23:35.893 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:35 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:35.893 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:35 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-22T07:23:35.893 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:35 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:23:35.893 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:35 smithi063 ceph-mon[112784]: Deploying daemon osd.3 on smithi069 2024-03-22T07:23:37.467 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:37 smithi069 ceph-mon[83631]: pgmap v110: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:23:37.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:37 smithi063 ceph-mon[112784]: pgmap v110: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:23:38.743 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:38 smithi069 systemd[1]: Stopping Ceph osd.3 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:23:39.029 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:38 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3[41863]: 2024-03-22T07:23:38.765+0000 7f14b6952700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:23:39.029 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:38 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3[41863]: 2024-03-22T07:23:38.765+0000 7f14b6952700 -1 osd.3 65 *** Got signal Terminated *** 2024-03-22T07:23:39.029 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:38 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3[41863]: 2024-03-22T07:23:38.765+0000 7f14b6952700 -1 osd.3 65 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:23:39.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:39 smithi063 ceph-mon[112784]: pgmap v111: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:23:39.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:39 smithi063 ceph-mon[112784]: osd.3 marked itself down and dead 2024-03-22T07:23:39.670 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:39 smithi069 ceph-mon[83631]: pgmap v111: 65 pgs: 65 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:23:39.670 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:39 smithi069 ceph-mon[83631]: osd.3 marked itself down and dead 2024-03-22T07:23:39.957 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:39 smithi069 podman[92404]: 2024-03-22 07:23:39.670512787 +0000 UTC m=+1.037087562 container died 45c9a759b76d37d1cbcf610caad000ebc36113a240c264802b45f1f3c106809e (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3, RELEASE=quincy-9c4e0c3, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-03-22T07:23:40.279 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:40 smithi069 podman[92404]: 2024-03-22 07:23:40.056896097 +0000 UTC m=+1.423470915 container cleanup 45c9a759b76d37d1cbcf610caad000ebc36113a240c264802b45f1f3c106809e (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3, io.buildah.version=1.33.5, RELEASE=quincy-9c4e0c3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-22T07:23:40.279 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:40 smithi069 bash[92404]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3 2024-03-22T07:23:40.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:40 smithi063 ceph-mon[112784]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:23:40.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:40 smithi063 ceph-mon[112784]: failed to encode map e66 with expected crc 2024-03-22T07:23:40.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:40 smithi063 ceph-mon[112784]: osdmap e66: 6 total, 5 up, 6 in 2024-03-22T07:23:40.672 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:40 smithi069 ceph-mon[83631]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:23:40.672 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:40 smithi069 ceph-mon[83631]: failed to encode map e66 with expected crc 2024-03-22T07:23:40.672 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:40 smithi069 ceph-mon[83631]: osdmap e66: 6 total, 5 up, 6 in 2024-03-22T07:23:40.672 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:40 smithi069 podman[92434]: 2024-03-22 07:23:40.316423861 +0000 UTC m=+0.646224121 container remove 45c9a759b76d37d1cbcf610caad000ebc36113a240c264802b45f1f3c106809e (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=quincy-9c4e0c3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-22T07:23:40.955 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:40 smithi069 podman[92526]: 2024-03-22 07:23:40.575944391 +0000 UTC m=+0.017843398 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:23:40.955 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:40 smithi069 podman[92526]: 2024-03-22 07:23:40.712824127 +0000 UTC m=+0.154723132 container create a41d1a7738de37e3b369f1d0c72e30478aad67b5f4b92fc95eab8adf8f745c4c (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=) 2024-03-22T07:23:40.955 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:40 smithi069 podman[92526]: 2024-03-22 07:23:40.955762066 +0000 UTC m=+0.397661080 container init a41d1a7738de37e3b369f1d0c72e30478aad67b5f4b92fc95eab8adf8f745c4c (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux ) 2024-03-22T07:23:41.245 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:40 smithi069 podman[92526]: 2024-03-22 07:23:40.959087304 +0000 UTC m=+0.400986316 container start a41d1a7738de37e3b369f1d0c72e30478aad67b5f4b92fc95eab8adf8f745c4c (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:23:41.245 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:41 smithi069 podman[92526]: 2024-03-22 07:23:41.055537399 +0000 UTC m=+0.497436419 container attach a41d1a7738de37e3b369f1d0c72e30478aad67b5f4b92fc95eab8adf8f745c4c (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:23:41.529 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:41 smithi069 podman[92526]: 2024-03-22 07:23:41.245632744 +0000 UTC m=+0.687531760 container died a41d1a7738de37e3b369f1d0c72e30478aad67b5f4b92fc95eab8adf8f745c4c (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-22T07:23:41.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:41 smithi069 ceph-mon[83631]: pgmap v113: 65 pgs: 16 stale+active+clean, 49 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-22T07:23:41.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:41 smithi069 ceph-mon[83631]: osdmap e67: 6 total, 5 up, 6 in 2024-03-22T07:23:41.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:41 smithi069 ceph-mon[83631]: failed to encode map e67 with expected crc 2024-03-22T07:23:41.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:41 smithi069 ceph-mon[83631]: failed to encode map e67 with expected crc 2024-03-22T07:23:41.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:41 smithi069 ceph-mon[83631]: failed to encode map e67 with expected crc 2024-03-22T07:23:41.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:41 smithi069 ceph-mon[83631]: failed to encode map e67 with expected crc 2024-03-22T07:23:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:41 smithi063 ceph-mon[112784]: pgmap v113: 65 pgs: 16 stale+active+clean, 49 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-22T07:23:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:41 smithi063 ceph-mon[112784]: osdmap e67: 6 total, 5 up, 6 in 2024-03-22T07:23:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:41 smithi063 ceph-mon[112784]: failed to encode map e67 with expected crc 2024-03-22T07:23:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:41 smithi063 ceph-mon[112784]: failed to encode map e67 with expected crc 2024-03-22T07:23:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:41 smithi063 ceph-mon[112784]: failed to encode map e67 with expected crc 2024-03-22T07:23:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:41 smithi063 ceph-mon[112784]: failed to encode map e67 with expected crc 2024-03-22T07:23:42.280 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:41 smithi069 podman[92552]: 2024-03-22 07:23:41.925911827 +0000 UTC m=+0.670213602 container remove a41d1a7738de37e3b369f1d0c72e30478aad67b5f4b92fc95eab8adf8f745c4c (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, ceph=True) 2024-03-22T07:23:42.280 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:41 smithi069 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.3.service: Deactivated successfully. 2024-03-22T07:23:42.280 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:41 smithi069 systemd[1]: Stopped Ceph osd.3 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:23:42.280 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:41 smithi069 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.3.service: Consumed 1min 10.477s CPU time. 2024-03-22T07:23:42.280 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:42 smithi069 systemd[1]: Starting Ceph osd.3 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:23:42.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:42 smithi063 ceph-mon[112784]: pgmap v115: 65 pgs: 7 peering, 13 stale+active+clean, 45 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:23:42.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:42 smithi063 ceph-mon[112784]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-03-22T07:23:42.670 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:42 smithi069 ceph-mon[83631]: pgmap v115: 65 pgs: 7 peering, 13 stale+active+clean, 45 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:23:42.671 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:42 smithi069 ceph-mon[83631]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-03-22T07:23:42.671 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:42 smithi069 podman[92655]: 2024-03-22 07:23:42.368795239 +0000 UTC m=+0.112190999 container create 1e2e8dd92428588492ac15358dee813d7e8a883ba225416d0d77354cfece1501 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, GIT_BRANCH=HEAD) 2024-03-22T07:23:42.671 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:42 smithi069 podman[92655]: 2024-03-22 07:23:42.275386148 +0000 UTC m=+0.018781923 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:23:42.671 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:42 smithi069 podman[92655]: 2024-03-22 07:23:42.588013901 +0000 UTC m=+0.331409677 container init 1e2e8dd92428588492ac15358dee813d7e8a883ba225416d0d77354cfece1501 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-22T07:23:42.671 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:42 smithi069 podman[92655]: 2024-03-22 07:23:42.591054994 +0000 UTC m=+0.334450754 container start 1e2e8dd92428588492ac15358dee813d7e8a883ba225416d0d77354cfece1501 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-22T07:23:42.671 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:42 smithi069 podman[92655]: 2024-03-22 07:23:42.671326064 +0000 UTC m=+0.414721851 container attach 1e2e8dd92428588492ac15358dee813d7e8a883ba225416d0d77354cfece1501 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-activate, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-22T07:23:43.349 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:43 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-activate[92670]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-22T07:23:43.349 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:43 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-activate[92670]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-22T07:23:43.349 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:43 smithi069 bash[92655]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-22T07:23:43.349 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:43 smithi069 bash[92655]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-22T07:23:43.349 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:43 smithi069 bash[92655]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-22T07:23:43.349 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:43 smithi069 bash[92655]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-22T07:23:43.349 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:43 smithi069 bash[92655]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-22T07:23:43.349 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:43 smithi069 bash[92655]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-22T07:23:43.350 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:43 smithi069 bash[92655]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-22T07:23:43.350 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:43 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-activate[92670]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-22T07:23:43.350 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:43 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-activate[92670]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-22T07:23:43.350 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:43 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-activate[92670]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-22T07:23:43.350 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:43 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-activate[92670]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-22T07:23:43.350 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:43 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-activate[92670]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-22T07:23:43.350 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:43 smithi069 podman[92655]: 2024-03-22 07:23:43.055837979 +0000 UTC m=+0.799233739 container died 1e2e8dd92428588492ac15358dee813d7e8a883ba225416d0d77354cfece1501 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-22T07:23:43.633 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:43 smithi069 ceph-mon[83631]: Health check failed: Degraded data redundancy: 65/354 objects degraded (18.362%), 17 pgs degraded (PG_DEGRADED) 2024-03-22T07:23:43.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:43 smithi063 ceph-mon[112784]: Health check failed: Degraded data redundancy: 65/354 objects degraded (18.362%), 17 pgs degraded (PG_DEGRADED) 2024-03-22T07:23:43.931 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:43 smithi069 podman[92790]: 2024-03-22 07:23:43.633369287 +0000 UTC m=+0.571864443 container remove 1e2e8dd92428588492ac15358dee813d7e8a883ba225416d0d77354cfece1501 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-activate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-22T07:23:43.931 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:43 smithi069 podman[92854]: 2024-03-22 07:23:43.842413854 +0000 UTC m=+0.112849979 container create 652a1e62339b36d44f435412d1a8b4726030d04d135111b7caff57c6871a7054 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-22T07:23:43.931 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:43 smithi069 podman[92854]: 2024-03-22 07:23:43.748360466 +0000 UTC m=+0.018796594 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:23:44.279 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:44 smithi069 podman[92854]: 2024-03-22 07:23:44.075547928 +0000 UTC m=+0.345984054 container init 652a1e62339b36d44f435412d1a8b4726030d04d135111b7caff57c6871a7054 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-22T07:23:44.279 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:44 smithi069 podman[92854]: 2024-03-22 07:23:44.078691896 +0000 UTC m=+0.349128021 container start 652a1e62339b36d44f435412d1a8b4726030d04d135111b7caff57c6871a7054 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux ) 2024-03-22T07:23:44.280 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:44 smithi069 ceph-osd[92873]: -- 172.21.15.69:0/2372535643 <== mon.1 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55eb073cd1e0 con 0x55eb0739ec00 2024-03-22T07:23:44.280 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:44 smithi069 bash[92854]: 652a1e62339b36d44f435412d1a8b4726030d04d135111b7caff57c6871a7054 2024-03-22T07:23:44.280 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:44 smithi069 systemd[1]: Started Ceph osd.3 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:23:45.016 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:44 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3[92869]: 2024-03-22T07:23:44.617+0000 7fe057d8c7c0 -1 Falling back to public interface 2024-03-22T07:23:45.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:45 smithi069 ceph-mon[83631]: pgmap v116: 65 pgs: 17 active+undersized+degraded, 7 peering, 14 active+undersized, 27 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 65/354 objects degraded (18.362%) 2024-03-22T07:23:45.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:45 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:45.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:45 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:45.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:45 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:23:45.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:45 smithi063 ceph-mon[112784]: pgmap v116: 65 pgs: 17 active+undersized+degraded, 7 peering, 14 active+undersized, 27 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 65/354 objects degraded (18.362%) 2024-03-22T07:23:45.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:45.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:45.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:45 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:23:46.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:46 smithi063 ceph-mon[112784]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-03-22T07:23:46.703 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:46 smithi069 ceph-mon[83631]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-03-22T07:23:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:47 smithi063 ceph-mon[112784]: pgmap v117: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 74/354 objects degraded (20.904%) 2024-03-22T07:23:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:47 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:47 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:23:47.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:47 smithi069 ceph-mon[83631]: pgmap v117: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 74/354 objects degraded (20.904%) 2024-03-22T07:23:47.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:47 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:47.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:47 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:23:48.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:48 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:48.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:48 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:48.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:48 smithi063 ceph-mon[112784]: pgmap v118: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 74/354 objects degraded (20.904%) 2024-03-22T07:23:48.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:48 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:48.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:48 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:48.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:48 smithi069 ceph-mon[83631]: pgmap v118: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 74/354 objects degraded (20.904%) 2024-03-22T07:23:49.409 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:49 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3[92869]: 2024-03-22T07:23:49.207+0000 7fe057d8c7c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-03-22T07:23:49.758 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:49 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:49.758 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:49 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:49.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:49 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:49.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:49 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:50.029 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:49 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3[92869]: 2024-03-22T07:23:49.861+0000 7fe057d8c7c0 -1 osd.3 65 log_to_monitors true 2024-03-22T07:23:50.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:50 smithi069 ceph-mon[83631]: pgmap v119: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 74/354 objects degraded (20.904%) 2024-03-22T07:23:50.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:50 smithi069 ceph-mon[83631]: from='osd.3 [v2:172.21.15.69:6800/2989645370,v1:172.21.15.69:6801/2989645370]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T07:23:50.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:50 smithi069 ceph-mon[83631]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T07:23:50.780 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:23:50 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3[92869]: 2024-03-22T07:23:50.556+0000 7fe0491e4700 -1 osd.3 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:23:50.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:50 smithi063 ceph-mon[112784]: pgmap v119: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 299 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 74/354 objects degraded (20.904%) 2024-03-22T07:23:50.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:50 smithi063 ceph-mon[112784]: from='osd.3 [v2:172.21.15.69:6800/2989645370,v1:172.21.15.69:6801/2989645370]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T07:23:50.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:50 smithi063 ceph-mon[112784]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-22T07:23:51.653 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:51 smithi069 ceph-mon[83631]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T07:23:51.653 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:51 smithi069 ceph-mon[83631]: osdmap e68: 6 total, 5 up, 6 in 2024-03-22T07:23:51.653 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:51 smithi069 ceph-mon[83631]: from='osd.3 [v2:172.21.15.69:6800/2989645370,v1:172.21.15.69:6801/2989645370]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:23:51.654 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:51 smithi069 ceph-mon[83631]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:23:51.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:51 smithi063 ceph-mon[112784]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-22T07:23:51.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:51 smithi063 ceph-mon[112784]: osdmap e68: 6 total, 5 up, 6 in 2024-03-22T07:23:51.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:51 smithi063 ceph-mon[112784]: from='osd.3 [v2:172.21.15.69:6800/2989645370,v1:172.21.15.69:6801/2989645370]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:23:51.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:51 smithi063 ceph-mon[112784]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:23:52.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: failed to encode map e68 with expected crc 2024-03-22T07:23:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: failed to encode map e68 with expected crc 2024-03-22T07:23:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: pgmap v121: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 74/354 objects degraded (20.904%) 2024-03-22T07:23:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:23:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: failed to encode map e69 with expected crc 2024-03-22T07:23:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: osd.3 [v2:172.21.15.69:6800/2989645370,v1:172.21.15.69:6801/2989645370] boot 2024-03-22T07:23:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: osdmap e69: 6 total, 6 up, 6 in 2024-03-22T07:23:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:23:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:23:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:23:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:23:52.874 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:23:52.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:23:52.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:23:52.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T07:23:52.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T07:23:52.875 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:52 smithi063 ceph-mon[112784]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-03-22T07:23:53.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: failed to encode map e68 with expected crc 2024-03-22T07:23:53.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: failed to encode map e68 with expected crc 2024-03-22T07:23:53.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: pgmap v121: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 74/354 objects degraded (20.904%) 2024-03-22T07:23:53.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:23:53.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: failed to encode map e69 with expected crc 2024-03-22T07:23:53.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: osd.3 [v2:172.21.15.69:6800/2989645370,v1:172.21.15.69:6801/2989645370] boot 2024-03-22T07:23:53.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: osdmap e69: 6 total, 6 up, 6 in 2024-03-22T07:23:53.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-22T07:23:53.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:53.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:53.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:23:53.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:23:53.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:23:53.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:23:53.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:23:53.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:23:53.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:23:53.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T07:23:53.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T07:23:53.031 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:52 smithi069 ceph-mon[83631]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-03-22T07:23:53.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:53 smithi069 ceph-mon[83631]: failed to encode map e70 with expected crc 2024-03-22T07:23:53.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:53 smithi069 ceph-mon[83631]: osdmap e70: 6 total, 6 up, 6 in 2024-03-22T07:23:53.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:53 smithi069 ceph-mon[83631]: failed to encode map e70 with expected crc 2024-03-22T07:23:53.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:53 smithi069 ceph-mon[83631]: failed to encode map e70 with expected crc 2024-03-22T07:23:53.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:53 smithi069 ceph-mon[83631]: failed to encode map e70 with expected crc 2024-03-22T07:23:53.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:53 smithi069 ceph-mon[83631]: Health check update: Degraded data redundancy: 74/354 objects degraded (20.904%), 20 pgs degraded (PG_DEGRADED) 2024-03-22T07:23:53.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:53 smithi063 ceph-mon[112784]: failed to encode map e70 with expected crc 2024-03-22T07:23:53.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:53 smithi063 ceph-mon[112784]: osdmap e70: 6 total, 6 up, 6 in 2024-03-22T07:23:53.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:53 smithi063 ceph-mon[112784]: failed to encode map e70 with expected crc 2024-03-22T07:23:53.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:53 smithi063 ceph-mon[112784]: failed to encode map e70 with expected crc 2024-03-22T07:23:53.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:53 smithi063 ceph-mon[112784]: failed to encode map e70 with expected crc 2024-03-22T07:23:53.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:53 smithi063 ceph-mon[112784]: Health check update: Degraded data redundancy: 74/354 objects degraded (20.904%), 20 pgs degraded (PG_DEGRADED) 2024-03-22T07:23:54.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:54 smithi069 ceph-mon[83631]: pgmap v124: 65 pgs: 15 peering, 11 active+undersized+degraded, 9 active+undersized, 30 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 3 op/s; 48/354 objects degraded (13.559%) 2024-03-22T07:23:54.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:54 smithi063 ceph-mon[112784]: pgmap v124: 65 pgs: 15 peering, 11 active+undersized+degraded, 9 active+undersized, 30 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 3 op/s; 48/354 objects degraded (13.559%) 2024-03-22T07:23:56.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:56 smithi063 ceph-mon[112784]: pgmap v125: 65 pgs: 15 peering, 10 active+undersized+degraded, 6 active+undersized, 34 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 44/354 objects degraded (12.429%) 2024-03-22T07:23:57.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:56 smithi069 ceph-mon[83631]: pgmap v125: 65 pgs: 15 peering, 10 active+undersized+degraded, 6 active+undersized, 34 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 2 op/s; 44/354 objects degraded (12.429%) 2024-03-22T07:23:57.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:57 smithi063 ceph-mon[112784]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 44/354 objects degraded (12.429%), 10 pgs degraded) 2024-03-22T07:23:57.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:57 smithi063 ceph-mon[112784]: Cluster is now healthy 2024-03-22T07:23:58.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:57 smithi069 ceph-mon[83631]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 44/354 objects degraded (12.429%), 10 pgs degraded) 2024-03-22T07:23:58.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:57 smithi069 ceph-mon[83631]: Cluster is now healthy 2024-03-22T07:23:58.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:23:58 smithi063 ceph-mon[112784]: pgmap v126: 65 pgs: 7 peering, 58 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-03-22T07:23:59.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:23:58 smithi069 ceph-mon[83631]: pgmap v126: 65 pgs: 7 peering, 58 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-03-22T07:24:00.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:00 smithi063 ceph-mon[112784]: pgmap v127: 65 pgs: 65 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-03-22T07:24:01.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:00 smithi069 ceph-mon[83631]: pgmap v127: 65 pgs: 65 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-03-22T07:24:02.774 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:02 smithi063 ceph-mon[112784]: pgmap v128: 65 pgs: 65 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-22T07:24:02.774 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:02.774 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:24:02.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:02 smithi069 ceph-mon[83631]: pgmap v128: 65 pgs: 65 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-22T07:24:02.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:02.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:24:04.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:04 smithi069 ceph-mon[83631]: pgmap v129: 65 pgs: 65 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 4 op/s 2024-03-22T07:24:04.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:04 smithi063 ceph-mon[112784]: pgmap v129: 65 pgs: 65 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 4 op/s 2024-03-22T07:24:04.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.915+0000 7f3f9f4c3700 1 -- 172.21.15.63:0/639244813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f980fb4d0 msgr2=0x7f3f980fb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:04.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.915+0000 7f3f9f4c3700 1 --2- 172.21.15.63:0/639244813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f980fb4d0 0x7f3f980fb940 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3f88004940 tx=0x7f3f88009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:04.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.915+0000 7f3f9f4c3700 1 -- 172.21.15.63:0/639244813 shutdown_connections 2024-03-22T07:24:04.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.915+0000 7f3f9f4c3700 1 --2- 172.21.15.63:0/639244813 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f980fb4d0 0x7f3f980fb940 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:04.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.915+0000 7f3f9f4c3700 1 --2- 172.21.15.63:0/639244813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f98102f50 0x7f3f98103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:04.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.915+0000 7f3f9f4c3700 1 -- 172.21.15.63:0/639244813 >> 172.21.15.63:0/639244813 conn(0x7f3f980714c0 msgr2=0x7f3f980718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:04.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.915+0000 7f3f9f4c3700 1 -- 172.21.15.63:0/639244813 shutdown_connections 2024-03-22T07:24:04.915 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.915+0000 7f3f9f4c3700 1 -- 172.21.15.63:0/639244813 wait complete. 2024-03-22T07:24:04.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.915+0000 7f3f9f4c3700 1 Processor -- start 2024-03-22T07:24:04.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.916+0000 7f3f9f4c3700 1 -- start start 2024-03-22T07:24:04.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.916+0000 7f3f9f4c3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f980fb4d0 0x7f3f981139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:04.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.916+0000 7f3f9f4c3700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f98102f50 0x7f3f98113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:04.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.916+0000 7f3f9f4c3700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f98114530 con 0x7f3f980fb4d0 2024-03-22T07:24:04.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.916+0000 7f3f9f4c3700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9810db50 con 0x7f3f98102f50 2024-03-22T07:24:04.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.916+0000 7f3f9d25f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f980fb4d0 0x7f3f981139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:04.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.916+0000 7f3f9d25f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f980fb4d0 0x7f3f981139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58848/0 (socket says 172.21.15.63:58848) 2024-03-22T07:24:04.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.916+0000 7f3f9d25f700 1 -- 172.21.15.63:0/665492849 learned_addr learned my addr 172.21.15.63:0/665492849 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:24:04.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.916+0000 7f3f9ca5e700 1 --2- 172.21.15.63:0/665492849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f98102f50 0x7f3f98113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:04.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.916+0000 7f3f9d25f700 1 -- 172.21.15.63:0/665492849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f98102f50 msgr2=0x7f3f98113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:04.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.916+0000 7f3f9d25f700 1 --2- 172.21.15.63:0/665492849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f98102f50 0x7f3f98113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:04.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.916+0000 7f3f9d25f700 1 -- 172.21.15.63:0/665492849 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f88005040 con 0x7f3f980fb4d0 2024-03-22T07:24:04.917 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.917+0000 7f3f9d25f700 1 --2- 172.21.15.63:0/665492849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f980fb4d0 0x7f3f981139a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3f94002a80 tx=0x7f3f94004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:04.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.917+0000 7f3f8dffb700 1 -- 172.21.15.63:0/665492849 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f94007cc0 con 0x7f3f980fb4d0 2024-03-22T07:24:04.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.917+0000 7f3f8dffb700 1 -- 172.21.15.63:0/665492849 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f9400f040 con 0x7f3f980fb4d0 2024-03-22T07:24:04.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.917+0000 7f3f9f4c3700 1 -- 172.21.15.63:0/665492849 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f9810de30 con 0x7f3f980fb4d0 2024-03-22T07:24:04.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.917+0000 7f3f8dffb700 1 -- 172.21.15.63:0/665492849 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f9400da30 con 0x7f3f980fb4d0 2024-03-22T07:24:04.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.917+0000 7f3f9f4c3700 1 -- 172.21.15.63:0/665492849 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f9810e370 con 0x7f3f980fb4d0 2024-03-22T07:24:04.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.918+0000 7f3f9f4c3700 1 -- 172.21.15.63:0/665492849 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f98105330 con 0x7f3f980fb4d0 2024-03-22T07:24:04.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.918+0000 7f3f8dffb700 1 -- 172.21.15.63:0/665492849 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f3f9400db90 con 0x7f3f980fb4d0 2024-03-22T07:24:04.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.919+0000 7f3f8dffb700 1 --2- 172.21.15.63:0/665492849 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f3f8406d380 0x7f3f8406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:04.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.919+0000 7f3f8dffb700 1 -- 172.21.15.63:0/665492849 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3f9408c600 con 0x7f3f980fb4d0 2024-03-22T07:24:04.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.919+0000 7f3f9ca5e700 1 --2- 172.21.15.63:0/665492849 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f3f8406d380 0x7f3f8406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:04.919 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.919+0000 7f3f9ca5e700 1 --2- 172.21.15.63:0/665492849 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f3f8406d380 0x7f3f8406f830 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f9810f240 tx=0x7f3f88009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:04.923 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:04.924+0000 7f3f8dffb700 1 -- 172.21.15.63:0/665492849 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3f940560e0 con 0x7f3f980fb4d0 2024-03-22T07:24:05.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.146+0000 7f3f9f4c3700 1 -- 172.21.15.63:0/665492849 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f9810a110 con 0x7f3f8406d380 2024-03-22T07:24:05.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.147+0000 7f3f8dffb700 1 -- 172.21.15.63:0/665492849 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f3f9810a110 con 0x7f3f8406d380 2024-03-22T07:24:05.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.149+0000 7f3f9f4c3700 1 -- 172.21.15.63:0/665492849 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f3f8406d380 msgr2=0x7f3f8406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:05.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.149+0000 7f3f9f4c3700 1 --2- 172.21.15.63:0/665492849 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f3f8406d380 0x7f3f8406f830 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f9810f240 tx=0x7f3f88009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:05.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.149+0000 7f3f9f4c3700 1 -- 172.21.15.63:0/665492849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f980fb4d0 msgr2=0x7f3f981139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:05.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.149+0000 7f3f9f4c3700 1 --2- 172.21.15.63:0/665492849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f980fb4d0 0x7f3f981139a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3f94002a80 tx=0x7f3f94004600 comp rx=0 tx=0).stop 2024-03-22T07:24:05.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.149+0000 7f3f9f4c3700 1 -- 172.21.15.63:0/665492849 shutdown_connections 2024-03-22T07:24:05.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.149+0000 7f3f9f4c3700 1 --2- 172.21.15.63:0/665492849 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f3f8406d380 0x7f3f8406f830 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.149+0000 7f3f9f4c3700 1 --2- 172.21.15.63:0/665492849 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3f980fb4d0 0x7f3f981139a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.149+0000 7f3f9f4c3700 1 --2- 172.21.15.63:0/665492849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f98102f50 0x7f3f98113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.149+0000 7f3f9f4c3700 1 -- 172.21.15.63:0/665492849 >> 172.21.15.63:0/665492849 conn(0x7f3f980714c0 msgr2=0x7f3f980fa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:05.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.149+0000 7f3f9f4c3700 1 -- 172.21.15.63:0/665492849 shutdown_connections 2024-03-22T07:24:05.150 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.149+0000 7f3f9f4c3700 1 -- 172.21.15.63:0/665492849 wait complete. 2024-03-22T07:24:05.160 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:24:05.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.253+0000 7fcd8e9a2700 1 -- 172.21.15.63:0/3533476771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd881035d0 msgr2=0x7fcd880fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:05.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.253+0000 7fcd8e9a2700 1 --2- 172.21.15.63:0/3533476771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd881035d0 0x7fcd880fcc40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c004940 tx=0x7fcd7c009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:05.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.253+0000 7fcd8e9a2700 1 -- 172.21.15.63:0/3533476771 shutdown_connections 2024-03-22T07:24:05.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.253+0000 7fcd8e9a2700 1 --2- 172.21.15.63:0/3533476771 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd881035d0 0x7fcd880fcc40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.253+0000 7fcd8e9a2700 1 --2- 172.21.15.63:0/3533476771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd88102c30 0x7fcd88103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.253+0000 7fcd8e9a2700 1 -- 172.21.15.63:0/3533476771 >> 172.21.15.63:0/3533476771 conn(0x7fcd880f8760 msgr2=0x7fcd880fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:05.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.253+0000 7fcd8e9a2700 1 -- 172.21.15.63:0/3533476771 shutdown_connections 2024-03-22T07:24:05.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.253+0000 7fcd8e9a2700 1 -- 172.21.15.63:0/3533476771 wait complete. 2024-03-22T07:24:05.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.253+0000 7fcd8e9a2700 1 Processor -- start 2024-03-22T07:24:05.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.254+0000 7fcd8e9a2700 1 -- start start 2024-03-22T07:24:05.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.254+0000 7fcd8e9a2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd88102c30 0x7fcd88113a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:05.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.254+0000 7fcd8e9a2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd881035d0 0x7fcd88113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:05.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.254+0000 7fcd8e9a2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd88114590 con 0x7fcd881035d0 2024-03-22T07:24:05.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.254+0000 7fcd8e9a2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8810d980 con 0x7fcd88102c30 2024-03-22T07:24:05.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.254+0000 7fcd8d19f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd881035d0 0x7fcd88113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:05.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.254+0000 7fcd8d19f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd881035d0 0x7fcd88113f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58866/0 (socket says 172.21.15.63:58866) 2024-03-22T07:24:05.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.254+0000 7fcd8d19f700 1 -- 172.21.15.63:0/1569560125 learned_addr learned my addr 172.21.15.63:0/1569560125 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:24:05.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.254+0000 7fcd8d9a0700 1 --2- 172.21.15.63:0/1569560125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd88102c30 0x7fcd88113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:05.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.254+0000 7fcd8d19f700 1 -- 172.21.15.63:0/1569560125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd88102c30 msgr2=0x7fcd88113a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:05.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.254+0000 7fcd8d19f700 1 --2- 172.21.15.63:0/1569560125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd88102c30 0x7fcd88113a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.255 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.254+0000 7fcd8d19f700 1 -- 172.21.15.63:0/1569560125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c005040 con 0x7fcd881035d0 2024-03-22T07:24:05.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.254+0000 7fcd8d19f700 1 --2- 172.21.15.63:0/1569560125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd881035d0 0x7fcd88113f40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009100 tx=0x7fcd7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:05.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.255+0000 7fcd867fc700 1 -- 172.21.15.63:0/1569560125 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd7c00e040 con 0x7fcd881035d0 2024-03-22T07:24:05.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.255+0000 7fcd867fc700 1 -- 172.21.15.63:0/1569560125 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd7c004290 con 0x7fcd881035d0 2024-03-22T07:24:05.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.255+0000 7fcd8e9a2700 1 -- 172.21.15.63:0/1569560125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8810dc00 con 0x7fcd881035d0 2024-03-22T07:24:05.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.255+0000 7fcd867fc700 1 -- 172.21.15.63:0/1569560125 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd7c01c3f0 con 0x7fcd881035d0 2024-03-22T07:24:05.256 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.255+0000 7fcd8e9a2700 1 -- 172.21.15.63:0/1569560125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8810e1c0 con 0x7fcd881035d0 2024-03-22T07:24:05.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.256+0000 7fcd8e9a2700 1 -- 172.21.15.63:0/1569560125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd880fe380 con 0x7fcd881035d0 2024-03-22T07:24:05.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.256+0000 7fcd867fc700 1 -- 172.21.15.63:0/1569560125 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fcd7c013690 con 0x7fcd881035d0 2024-03-22T07:24:05.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.257+0000 7fcd867fc700 1 --2- 172.21.15.63:0/1569560125 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fcd7006d380 0x7fcd7006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:05.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.257+0000 7fcd867fc700 1 -- 172.21.15.63:0/1569560125 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcd7c08da30 con 0x7fcd881035d0 2024-03-22T07:24:05.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.257+0000 7fcd8d9a0700 1 --2- 172.21.15.63:0/1569560125 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fcd7006d380 0x7fcd7006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:05.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.257+0000 7fcd8d9a0700 1 --2- 172.21.15.63:0/1569560125 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fcd7006d380 0x7fcd7006f830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcd80001450 tx=0x7fcd8000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:05.261 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.261+0000 7fcd867fc700 1 -- 172.21.15.63:0/1569560125 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcd7c05b0c0 con 0x7fcd881035d0 2024-03-22T07:24:05.486 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.486+0000 7fcd8e9a2700 1 -- 172.21.15.63:0/1569560125 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd88109d90 con 0x7fcd7006d380 2024-03-22T07:24:05.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.487+0000 7fcd867fc700 1 -- 172.21.15.63:0/1569560125 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fcd88109d90 con 0x7fcd7006d380 2024-03-22T07:24:05.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.489+0000 7fcd8e9a2700 1 -- 172.21.15.63:0/1569560125 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fcd7006d380 msgr2=0x7fcd7006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:05.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.489+0000 7fcd8e9a2700 1 --2- 172.21.15.63:0/1569560125 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fcd7006d380 0x7fcd7006f830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcd80001450 tx=0x7fcd8000b040 comp rx=0 tx=0).stop 2024-03-22T07:24:05.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.489+0000 7fcd8e9a2700 1 -- 172.21.15.63:0/1569560125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd881035d0 msgr2=0x7fcd88113f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:05.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.489+0000 7fcd8e9a2700 1 --2- 172.21.15.63:0/1569560125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd881035d0 0x7fcd88113f40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009100 tx=0x7fcd7c009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:05.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.489+0000 7fcd8e9a2700 1 -- 172.21.15.63:0/1569560125 shutdown_connections 2024-03-22T07:24:05.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.489+0000 7fcd8e9a2700 1 --2- 172.21.15.63:0/1569560125 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fcd7006d380 0x7fcd7006f830 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.489+0000 7fcd8e9a2700 1 --2- 172.21.15.63:0/1569560125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcd88102c30 0x7fcd88113a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.489+0000 7fcd8e9a2700 1 --2- 172.21.15.63:0/1569560125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcd881035d0 0x7fcd88113f40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.489+0000 7fcd8e9a2700 1 -- 172.21.15.63:0/1569560125 >> 172.21.15.63:0/1569560125 conn(0x7fcd880f8760 msgr2=0x7fcd880f9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:05.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.489+0000 7fcd8e9a2700 1 -- 172.21.15.63:0/1569560125 shutdown_connections 2024-03-22T07:24:05.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.489+0000 7fcd8e9a2700 1 -- 172.21.15.63:0/1569560125 wait complete. 2024-03-22T07:24:05.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.592+0000 7f7328899700 1 -- 172.21.15.63:0/2366951442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73241037b0 msgr2=0x7f73240fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:05.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.592+0000 7f7328899700 1 --2- 172.21.15.63:0/2366951442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73241037b0 0x7f73240fce20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f730c004940 tx=0x7f730c009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:05.592 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:05 smithi063 ceph-mon[112784]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:24:05.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.592+0000 7f7328899700 1 -- 172.21.15.63:0/2366951442 shutdown_connections 2024-03-22T07:24:05.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.592+0000 7f7328899700 1 --2- 172.21.15.63:0/2366951442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73241037b0 0x7f73240fce20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.592+0000 7f7328899700 1 --2- 172.21.15.63:0/2366951442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7324102e10 0x7f73241031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.592+0000 7f7328899700 1 -- 172.21.15.63:0/2366951442 >> 172.21.15.63:0/2366951442 conn(0x7f73240f8980 msgr2=0x7f73240fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:05.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.592+0000 7f7328899700 1 -- 172.21.15.63:0/2366951442 shutdown_connections 2024-03-22T07:24:05.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.592+0000 7f7328899700 1 -- 172.21.15.63:0/2366951442 wait complete. 2024-03-22T07:24:05.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.593+0000 7f7328899700 1 Processor -- start 2024-03-22T07:24:05.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.593+0000 7f7328899700 1 -- start start 2024-03-22T07:24:05.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.593+0000 7f7328899700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7324102e10 0x7f7324113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:05.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.593+0000 7f7328899700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73241037b0 0x7f73241140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:05.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.593+0000 7f7328899700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7324114710 con 0x7f73241037b0 2024-03-22T07:24:05.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.593+0000 7f7328899700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732410db00 con 0x7f7324102e10 2024-03-22T07:24:05.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.593+0000 7f7321d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73241037b0 0x7f73241140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:05.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.593+0000 7f7321d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73241037b0 0x7f73241140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58882/0 (socket says 172.21.15.63:58882) 2024-03-22T07:24:05.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.593+0000 7f7321d9b700 1 -- 172.21.15.63:0/2972866763 learned_addr learned my addr 172.21.15.63:0/2972866763 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:24:05.594 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.593+0000 7f732259c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7324102e10 0x7f7324113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:05.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.594+0000 7f7321d9b700 1 -- 172.21.15.63:0/2972866763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7324102e10 msgr2=0x7f7324113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:05.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.594+0000 7f7321d9b700 1 --2- 172.21.15.63:0/2972866763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7324102e10 0x7f7324113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.594+0000 7f7321d9b700 1 -- 172.21.15.63:0/2972866763 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f730c005040 con 0x7f73241037b0 2024-03-22T07:24:05.595 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.594+0000 7f732259c700 1 --2- 172.21.15.63:0/2972866763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7324102e10 0x7f7324113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:24:05.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.594+0000 7f7321d9b700 1 --2- 172.21.15.63:0/2972866763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73241037b0 0x7f73241140c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f730c009100 tx=0x7f730c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:05.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.594+0000 7f731affd700 1 -- 172.21.15.63:0/2972866763 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f730c00e040 con 0x7f73241037b0 2024-03-22T07:24:05.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.594+0000 7f731affd700 1 -- 172.21.15.63:0/2972866763 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f730c009af0 con 0x7f73241037b0 2024-03-22T07:24:05.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.594+0000 7f7328899700 1 -- 172.21.15.63:0/2972866763 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732410dd80 con 0x7f73241037b0 2024-03-22T07:24:05.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.594+0000 7f731affd700 1 -- 172.21.15.63:0/2972866763 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f730c01c3f0 con 0x7f73241037b0 2024-03-22T07:24:05.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.594+0000 7f7328899700 1 -- 172.21.15.63:0/2972866763 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732410e340 con 0x7f73241037b0 2024-03-22T07:24:05.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.595+0000 7f7328899700 1 -- 172.21.15.63:0/2972866763 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73241052e0 con 0x7f73241037b0 2024-03-22T07:24:05.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.596+0000 7f731affd700 1 -- 172.21.15.63:0/2972866763 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f730c00baf0 con 0x7f73241037b0 2024-03-22T07:24:05.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.596+0000 7f731affd700 1 --2- 172.21.15.63:0/2972866763 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f731006d380 0x7f731006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:05.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.596+0000 7f731affd700 1 -- 172.21.15.63:0/2972866763 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f730c08d890 con 0x7f73241037b0 2024-03-22T07:24:05.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.596+0000 7f732259c700 1 --2- 172.21.15.63:0/2972866763 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f731006d380 0x7f731006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:05.597 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.597+0000 7f732259c700 1 --2- 172.21.15.63:0/2972866763 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f731006d380 0x7f731006f830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f731c007d40 tx=0x7f731c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:05.601 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.601+0000 7f731affd700 1 -- 172.21.15.63:0/2972866763 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f730c05af20 con 0x7f73241037b0 2024-03-22T07:24:05.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:05 smithi069 ceph-mon[83631]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:24:05.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.814+0000 7f7328899700 1 -- 172.21.15.63:0/2972866763 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f732405f660 con 0x7f731006d380 2024-03-22T07:24:05.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.824+0000 7f731affd700 1 -- 172.21.15.63:0/2972866763 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f732405f660 con 0x7f731006d380 2024-03-22T07:24:05.824 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:24:05.824 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (5m) 52s ago 15m 26.3M - 0.25.0 c8568f914cd2 06ef21da90e5 2024-03-22T07:24:05.824 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (2m) 52s ago 16m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 224c291ab774 2024-03-22T07:24:05.824 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (2m) 18s ago 15m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 7eabb7d42631 2024-03-22T07:24:05.824 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (4m) 52s ago 15m 86.0M - 9.4.12 a0ea5e3c46d9 9015fae0d5e9 2024-03-22T07:24:05.824 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (10m) 52s ago 10m 74.1M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:24:05.824 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (10m) 52s ago 10m 123M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:24:05.824 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (10m) 18s ago 10m 63.1M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:24:05.824 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (10m) 18s ago 10m 129M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:24:05.824 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:8443,9283,8765 running (6m) 52s ago 17m 506M - 19.0.0-1667-gdb0330b1 4f325f8440e9 e49468f39991 2024-03-22T07:24:05.825 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283,8765 running (5m) 18s ago 15m 450M - 19.0.0-1667-gdb0330b1 4f325f8440e9 13c3f034ebb2 2024-03-22T07:24:05.825 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (3m) 52s ago 17m 47.7M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ab96538a07c1 2024-03-22T07:24:05.825 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (3m) 18s ago 14m 37.4M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 01a26e1fb1a1 2024-03-22T07:24:05.825 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (5m) 52s ago 16m 16.0M - 1.5.0 0da6a335fe13 0c825ee2a196 2024-03-22T07:24:05.825 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (5m) 18s ago 15m 16.4M - 1.5.0 0da6a335fe13 12edb2f35498 2024-03-22T07:24:05.825 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (2m) 52s ago 14m 102M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 fdbc8b34ab78 2024-03-22T07:24:05.825 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (94s) 52s ago 13m 98.3M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 5200fd893046 2024-03-22T07:24:05.825 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (58s) 52s ago 13m 11.7M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 918925ee69f3 2024-03-22T07:24:05.825 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (22s) 18s ago 12m 14.8M 2581M 19.0.0-1667-gdb0330b1 4f325f8440e9 652a1e62339b 2024-03-22T07:24:05.825 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (12m) 18s ago 12m 243M 2581M 17.2.7-726-g9c4e0c34 837124da804f 76bc35d01de0 2024-03-22T07:24:05.825 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (12m) 18s ago 12m 236M 2581M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:24:05.825 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (5m) 52s ago 15m 61.1M - 2.43.0 a07b618ecd1d 1d89b928f6e7 2024-03-22T07:24:05.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.826+0000 7f7328899700 1 -- 172.21.15.63:0/2972866763 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f731006d380 msgr2=0x7f731006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:05.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.826+0000 7f7328899700 1 --2- 172.21.15.63:0/2972866763 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f731006d380 0x7f731006f830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f731c007d40 tx=0x7f731c009da0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.826+0000 7f7328899700 1 -- 172.21.15.63:0/2972866763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73241037b0 msgr2=0x7f73241140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:05.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.826+0000 7f7328899700 1 --2- 172.21.15.63:0/2972866763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73241037b0 0x7f73241140c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f730c009100 tx=0x7f730c009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:05.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.826+0000 7f7328899700 1 -- 172.21.15.63:0/2972866763 shutdown_connections 2024-03-22T07:24:05.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.826+0000 7f7328899700 1 --2- 172.21.15.63:0/2972866763 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f731006d380 0x7f731006f830 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.826 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.826+0000 7f7328899700 1 --2- 172.21.15.63:0/2972866763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7324102e10 0x7f7324113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.826+0000 7f7328899700 1 --2- 172.21.15.63:0/2972866763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f73241037b0 0x7f73241140c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.826+0000 7f7328899700 1 -- 172.21.15.63:0/2972866763 >> 172.21.15.63:0/2972866763 conn(0x7f73240f8980 msgr2=0x7f73241021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:05.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.826+0000 7f7328899700 1 -- 172.21.15.63:0/2972866763 shutdown_connections 2024-03-22T07:24:05.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.826+0000 7f7328899700 1 -- 172.21.15.63:0/2972866763 wait complete. 2024-03-22T07:24:05.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.927+0000 7f7c973fa700 1 -- 172.21.15.63:0/2308937840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c90103780 msgr2=0x7f7c900fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:05.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.927+0000 7f7c973fa700 1 --2- 172.21.15.63:0/2308937840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c90103780 0x7f7c900fcdf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009ce0 tx=0x7f7c84004d10 comp rx=0 tx=0).stop 2024-03-22T07:24:05.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.927+0000 7f7c973fa700 1 -- 172.21.15.63:0/2308937840 shutdown_connections 2024-03-22T07:24:05.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.927+0000 7f7c973fa700 1 --2- 172.21.15.63:0/2308937840 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c90103780 0x7f7c900fcdf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.927+0000 7f7c973fa700 1 --2- 172.21.15.63:0/2308937840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c90102de0 0x7f7c901031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.927+0000 7f7c973fa700 1 -- 172.21.15.63:0/2308937840 >> 172.21.15.63:0/2308937840 conn(0x7f7c900f8970 msgr2=0x7f7c900fad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:05.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.927+0000 7f7c973fa700 1 -- 172.21.15.63:0/2308937840 shutdown_connections 2024-03-22T07:24:05.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.927+0000 7f7c973fa700 1 -- 172.21.15.63:0/2308937840 wait complete. 2024-03-22T07:24:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.928+0000 7f7c973fa700 1 Processor -- start 2024-03-22T07:24:05.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.928+0000 7f7c973fa700 1 -- start start 2024-03-22T07:24:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.928+0000 7f7c973fa700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c90102de0 0x7f7c90113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.928+0000 7f7c973fa700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c90103780 0x7f7c901140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.928+0000 7f7c973fa700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c901146f0 con 0x7f7c90102de0 2024-03-22T07:24:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.928+0000 7f7c973fa700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9010dae0 con 0x7f7c90103780 2024-03-22T07:24:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.928+0000 7f7c95196700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c90102de0 0x7f7c90113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.928+0000 7f7c95196700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c90102de0 0x7f7c90113b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58898/0 (socket says 172.21.15.63:58898) 2024-03-22T07:24:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.928+0000 7f7c95196700 1 -- 172.21.15.63:0/975473441 learned_addr learned my addr 172.21.15.63:0/975473441 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:24:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.929+0000 7f7c94995700 1 --2- 172.21.15.63:0/975473441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c90103780 0x7f7c901140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.929+0000 7f7c95196700 1 -- 172.21.15.63:0/975473441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c90103780 msgr2=0x7f7c901140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.929+0000 7f7c95196700 1 --2- 172.21.15.63:0/975473441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c90103780 0x7f7c901140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.929+0000 7f7c95196700 1 -- 172.21.15.63:0/975473441 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c84009940 con 0x7f7c90102de0 2024-03-22T07:24:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.929+0000 7f7c95196700 1 --2- 172.21.15.63:0/975473441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c90102de0 0x7f7c90113b60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000eb40 tx=0x7f7c8000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.929+0000 7f7c8dffb700 1 -- 172.21.15.63:0/975473441 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c80007cc0 con 0x7f7c90102de0 2024-03-22T07:24:05.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.929+0000 7f7c8dffb700 1 -- 172.21.15.63:0/975473441 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c8000f040 con 0x7f7c90102de0 2024-03-22T07:24:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.929+0000 7f7c973fa700 1 -- 172.21.15.63:0/975473441 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9010dd60 con 0x7f7c90102de0 2024-03-22T07:24:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.929+0000 7f7c8dffb700 1 -- 172.21.15.63:0/975473441 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c80009620 con 0x7f7c90102de0 2024-03-22T07:24:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.929+0000 7f7c973fa700 1 -- 172.21.15.63:0/975473441 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c9010e2f0 con 0x7f7c90102de0 2024-03-22T07:24:05.931 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.930+0000 7f7c973fa700 1 -- 172.21.15.63:0/975473441 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c900fe530 con 0x7f7c90102de0 2024-03-22T07:24:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.931+0000 7f7c8dffb700 1 -- 172.21.15.63:0/975473441 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f7c80009780 con 0x7f7c90102de0 2024-03-22T07:24:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.931+0000 7f7c8dffb700 1 --2- 172.21.15.63:0/975473441 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7c7806d380 0x7f7c7806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.931+0000 7f7c8dffb700 1 -- 172.21.15.63:0/975473441 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7c8008c550 con 0x7f7c90102de0 2024-03-22T07:24:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.931+0000 7f7c94995700 1 --2- 172.21.15.63:0/975473441 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7c7806d380 0x7f7c7806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:05.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.932+0000 7f7c94995700 1 --2- 172.21.15.63:0/975473441 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7c7806d380 0x7f7c7806f830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7c84004f80 tx=0x7f7c84004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:05.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:05.936+0000 7f7c8dffb700 1 -- 172.21.15.63:0/975473441 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7c80059c10 con 0x7f7c90102de0 2024-03-22T07:24:06.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.222+0000 7f7c973fa700 1 -- 172.21.15.63:0/975473441 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7c90109f10 con 0x7f7c90102de0 2024-03-22T07:24:06.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.222+0000 7f7c8dffb700 1 -- 172.21.15.63:0/975473441 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f7c800597a0 con 0x7f7c90102de0 2024-03-22T07:24:06.222 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:24:06.222 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:24:06.222 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:24:06.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:24:06.223 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:24:06.223 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:24:06.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:24:06.223 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:24:06.223 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 2, 2024-03-22T07:24:06.223 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-22T07:24:06.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:24:06.223 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:24:06.223 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:24:06.223 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:24:06.223 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:24:06.223 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 6, 2024-03-22T07:24:06.223 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 8 2024-03-22T07:24:06.223 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:24:06.223 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:24:06.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.224+0000 7f7c973fa700 1 -- 172.21.15.63:0/975473441 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7c7806d380 msgr2=0x7f7c7806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:06.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.224+0000 7f7c973fa700 1 --2- 172.21.15.63:0/975473441 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7c7806d380 0x7f7c7806f830 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7c84004f80 tx=0x7f7c84004d10 comp rx=0 tx=0).stop 2024-03-22T07:24:06.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.224+0000 7f7c973fa700 1 -- 172.21.15.63:0/975473441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c90102de0 msgr2=0x7f7c90113b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:06.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.224+0000 7f7c973fa700 1 --2- 172.21.15.63:0/975473441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c90102de0 0x7f7c90113b60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000eb40 tx=0x7f7c8000b040 comp rx=0 tx=0).stop 2024-03-22T07:24:06.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.225+0000 7f7c973fa700 1 -- 172.21.15.63:0/975473441 shutdown_connections 2024-03-22T07:24:06.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.225+0000 7f7c973fa700 1 --2- 172.21.15.63:0/975473441 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7c7806d380 0x7f7c7806f830 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:06.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.225+0000 7f7c973fa700 1 --2- 172.21.15.63:0/975473441 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7c90102de0 0x7f7c90113b60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:06.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.225+0000 7f7c973fa700 1 --2- 172.21.15.63:0/975473441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7c90103780 0x7f7c901140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:06.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.225+0000 7f7c973fa700 1 -- 172.21.15.63:0/975473441 >> 172.21.15.63:0/975473441 conn(0x7f7c900f8970 msgr2=0x7f7c900f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:06.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.225+0000 7f7c973fa700 1 -- 172.21.15.63:0/975473441 shutdown_connections 2024-03-22T07:24:06.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.225+0000 7f7c973fa700 1 -- 172.21.15.63:0/975473441 wait complete. 2024-03-22T07:24:06.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.327+0000 7eff85e07700 1 -- 172.21.15.63:0/2398887512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff800fe700 msgr2=0x7eff80106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:06.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.327+0000 7eff85e07700 1 --2- 172.21.15.63:0/2398887512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff800fe700 0x7eff80106dd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7eff74004940 tx=0x7eff74009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:06.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.327+0000 7eff85e07700 1 -- 172.21.15.63:0/2398887512 shutdown_connections 2024-03-22T07:24:06.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.327+0000 7eff85e07700 1 --2- 172.21.15.63:0/2398887512 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff800fe700 0x7eff80106dd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:06.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.327+0000 7eff85e07700 1 --2- 172.21.15.63:0/2398887512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff800fddf0 0x7eff800fe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:06.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.327+0000 7eff85e07700 1 -- 172.21.15.63:0/2398887512 >> 172.21.15.63:0/2398887512 conn(0x7eff800f7820 msgr2=0x7eff800f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:06.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.327+0000 7eff85e07700 1 -- 172.21.15.63:0/2398887512 shutdown_connections 2024-03-22T07:24:06.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.327+0000 7eff85e07700 1 -- 172.21.15.63:0/2398887512 wait complete. 2024-03-22T07:24:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.327+0000 7eff85e07700 1 Processor -- start 2024-03-22T07:24:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.328+0000 7eff85e07700 1 -- start start 2024-03-22T07:24:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.328+0000 7eff85e07700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff800fddf0 0x7eff80113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.328+0000 7eff85e07700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff800fe700 0x7eff801140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.328+0000 7eff85e07700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff801146f0 con 0x7eff800fe700 2024-03-22T07:24:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.328+0000 7eff85e07700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8010dae0 con 0x7eff800fddf0 2024-03-22T07:24:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.328+0000 7eff7effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff800fe700 0x7eff801140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.328+0000 7eff7effd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff800fe700 0x7eff801140a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58926/0 (socket says 172.21.15.63:58926) 2024-03-22T07:24:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.328+0000 7eff7effd700 1 -- 172.21.15.63:0/1935787459 learned_addr learned my addr 172.21.15.63:0/1935787459 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:24:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.328+0000 7eff7f7fe700 1 --2- 172.21.15.63:0/1935787459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff800fddf0 0x7eff80113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.328+0000 7eff7effd700 1 -- 172.21.15.63:0/1935787459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff800fddf0 msgr2=0x7eff80113b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.328+0000 7eff7effd700 1 --2- 172.21.15.63:0/1935787459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff800fddf0 0x7eff80113b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.328+0000 7eff7effd700 1 -- 172.21.15.63:0/1935787459 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff74005040 con 0x7eff800fe700 2024-03-22T07:24:06.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.328+0000 7eff7f7fe700 1 --2- 172.21.15.63:0/1935787459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff800fddf0 0x7eff80113b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:24:06.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.328+0000 7eff7effd700 1 --2- 172.21.15.63:0/1935787459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff800fe700 0x7eff801140a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7eff7400b3b0 tx=0x7eff74009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:06.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.328+0000 7eff67fff700 1 -- 172.21.15.63:0/1935787459 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff7400e040 con 0x7eff800fe700 2024-03-22T07:24:06.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.329+0000 7eff67fff700 1 -- 172.21.15.63:0/1935787459 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff7400bbc0 con 0x7eff800fe700 2024-03-22T07:24:06.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.329+0000 7eff85e07700 1 -- 172.21.15.63:0/1935787459 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8010dd60 con 0x7eff800fe700 2024-03-22T07:24:06.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.329+0000 7eff67fff700 1 -- 172.21.15.63:0/1935787459 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff7401c3f0 con 0x7eff800fe700 2024-03-22T07:24:06.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.329+0000 7eff85e07700 1 -- 172.21.15.63:0/1935787459 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8010e250 con 0x7eff800fe700 2024-03-22T07:24:06.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.330+0000 7eff85e07700 1 -- 172.21.15.63:0/1935787459 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff800669d0 con 0x7eff800fe700 2024-03-22T07:24:06.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.330+0000 7eff67fff700 1 -- 172.21.15.63:0/1935787459 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7eff7400bd30 con 0x7eff800fe700 2024-03-22T07:24:06.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.330+0000 7eff67fff700 1 --2- 172.21.15.63:0/1935787459 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7eff6806d380 0x7eff6806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:06.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.331+0000 7eff67fff700 1 -- 172.21.15.63:0/1935787459 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6328+0+0 (secure 0 0 0) 0x7eff7408db60 con 0x7eff800fe700 2024-03-22T07:24:06.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.331+0000 7eff7f7fe700 1 --2- 172.21.15.63:0/1935787459 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7eff6806d380 0x7eff6806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:06.331 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.331+0000 7eff7f7fe700 1 --2- 172.21.15.63:0/1935787459 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7eff6806d380 0x7eff6806f830 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7eff70007fd0 tx=0x7eff70009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:06.335 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.335+0000 7eff67fff700 1 -- 172.21.15.63:0/1935787459 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7eff7405b170 con 0x7eff800fe700 2024-03-22T07:24:06.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.583+0000 7eff85e07700 1 -- 172.21.15.63:0/1935787459 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7eff80109e80 con 0x7eff800fe700 2024-03-22T07:24:06.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.584+0000 7eff67fff700 1 -- 172.21.15.63:0/1935787459 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1868 (secure 0 0 0) 0x7eff7405ad00 con 0x7eff800fe700 2024-03-22T07:24:06.623 INFO:teuthology.orchestra.run.smithi063.stdout:e24 2024-03-22T07:24:06.623 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:24:06.623 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:24:06.623 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-03-22T07:24:06.623 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:24:06.623 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:24:06.623 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-03-22T07:24:06.624 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 24 2024-03-22T07:24:06.624 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-22T07:24:06.624 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-03-22T07:13:29.602905+0000 2024-03-22T07:24:06.624 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-03-22T07:20:24.253589+0000 2024-03-22T07:24:06.624 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-03-22T07:24:06.624 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-03-22T07:24:06.624 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-03-22T07:24:06.624 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-03-22T07:24:06.624 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-03-22T07:24:06.624 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-03-22T07:24:06.624 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-03-22T07:24:06.624 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-03-22T07:24:06.624 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-03-22T07:24:06.624 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:24:06.624 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 2 2024-03-22T07:24:06.624 INFO:teuthology.orchestra.run.smithi063.stdout:in 0,1 2024-03-22T07:24:06.625 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=14480,1=24285} 2024-03-22T07:24:06.625 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-03-22T07:24:06.625 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-03-22T07:24:06.625 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-03-22T07:24:06.625 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-03-22T07:24:06.625 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-03-22T07:24:06.625 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-03-22T07:24:06.625 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-03-22T07:24:06.625 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-03-22T07:24:06.625 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-03-22T07:24:06.625 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.xeelpk{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:24:06.625 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.oytapz{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3211595802,v1:172.21.15.63:6827/3211595802] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:24:06.625 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.wwtsjn{1:24285} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.69:6824/3798247699,v1:172.21.15.69:6825/3798247699] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:24:06.625 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.cijxah{1:24301} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.69:6826/3380719311,v1:172.21.15.69:6827/3380719311] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:24:06.625 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:24:06.625 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:24:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.586+0000 7eff85e07700 1 -- 172.21.15.63:0/1935787459 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7eff6806d380 msgr2=0x7eff6806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.586+0000 7eff85e07700 1 --2- 172.21.15.63:0/1935787459 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7eff6806d380 0x7eff6806f830 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7eff70007fd0 tx=0x7eff70009040 comp rx=0 tx=0).stop 2024-03-22T07:24:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.586+0000 7eff85e07700 1 -- 172.21.15.63:0/1935787459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff800fe700 msgr2=0x7eff801140a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.586+0000 7eff85e07700 1 --2- 172.21.15.63:0/1935787459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff800fe700 0x7eff801140a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7eff7400b3b0 tx=0x7eff74009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.586+0000 7eff85e07700 1 -- 172.21.15.63:0/1935787459 shutdown_connections 2024-03-22T07:24:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.586+0000 7eff85e07700 1 --2- 172.21.15.63:0/1935787459 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7eff6806d380 0x7eff6806f830 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.586+0000 7eff85e07700 1 --2- 172.21.15.63:0/1935787459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7eff800fddf0 0x7eff80113b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:06.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.586+0000 7eff85e07700 1 --2- 172.21.15.63:0/1935787459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7eff800fe700 0x7eff801140a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:06.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.586+0000 7eff85e07700 1 -- 172.21.15.63:0/1935787459 >> 172.21.15.63:0/1935787459 conn(0x7eff800f7820 msgr2=0x7eff800f8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:06.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.586+0000 7eff85e07700 1 -- 172.21.15.63:0/1935787459 shutdown_connections 2024-03-22T07:24:06.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.586+0000 7eff85e07700 1 -- 172.21.15.63:0/1935787459 wait complete. 2024-03-22T07:24:06.627 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 24 2024-03-22T07:24:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.689+0000 7f9c59768700 1 -- 172.21.15.63:0/3131082760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c54102e00 msgr2=0x7f9c541031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.689+0000 7f9c59768700 1 --2- 172.21.15.63:0/3131082760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c54102e00 0x7f9c541031d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c004940 tx=0x7f9c4c009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.690+0000 7f9c59768700 1 -- 172.21.15.63:0/3131082760 shutdown_connections 2024-03-22T07:24:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.690+0000 7f9c59768700 1 --2- 172.21.15.63:0/3131082760 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c541037a0 0x7f9c540fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.690+0000 7f9c59768700 1 --2- 172.21.15.63:0/3131082760 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c54102e00 0x7f9c541031d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.690+0000 7f9c59768700 1 -- 172.21.15.63:0/3131082760 >> 172.21.15.63:0/3131082760 conn(0x7f9c540f89b0 msgr2=0x7f9c540fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.690+0000 7f9c59768700 1 -- 172.21.15.63:0/3131082760 shutdown_connections 2024-03-22T07:24:06.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.690+0000 7f9c59768700 1 -- 172.21.15.63:0/3131082760 wait complete. 2024-03-22T07:24:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.690+0000 7f9c59768700 1 Processor -- start 2024-03-22T07:24:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.690+0000 7f9c59768700 1 -- start start 2024-03-22T07:24:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.690+0000 7f9c59768700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c54102e00 0x7f9c54113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.690+0000 7f9c59768700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c541037a0 0x7f9c54113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.690+0000 7f9c59768700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c54114520 con 0x7f9c54102e00 2024-03-22T07:24:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.690+0000 7f9c59768700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5410db70 con 0x7f9c541037a0 2024-03-22T07:24:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.691+0000 7f9c52ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c54102e00 0x7f9c54113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.691+0000 7f9c52ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c54102e00 0x7f9c54113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58948/0 (socket says 172.21.15.63:58948) 2024-03-22T07:24:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.691+0000 7f9c52ffd700 1 -- 172.21.15.63:0/912035020 learned_addr learned my addr 172.21.15.63:0/912035020 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:24:06.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.691+0000 7f9c527fc700 1 --2- 172.21.15.63:0/912035020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c541037a0 0x7f9c54113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:06.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.691+0000 7f9c52ffd700 1 -- 172.21.15.63:0/912035020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c541037a0 msgr2=0x7f9c54113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:06.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.691+0000 7f9c52ffd700 1 --2- 172.21.15.63:0/912035020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c541037a0 0x7f9c54113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:06.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.691+0000 7f9c52ffd700 1 -- 172.21.15.63:0/912035020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c005040 con 0x7f9c54102e00 2024-03-22T07:24:06.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.691+0000 7f9c52ffd700 1 --2- 172.21.15.63:0/912035020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c54102e00 0x7f9c54113990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00b3b0 tx=0x7f9c4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:06.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.691+0000 7f9c4b7fe700 1 -- 172.21.15.63:0/912035020 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c4c00e040 con 0x7f9c54102e00 2024-03-22T07:24:06.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.691+0000 7f9c59768700 1 -- 172.21.15.63:0/912035020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5410ddf0 con 0x7f9c54102e00 2024-03-22T07:24:06.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.691+0000 7f9c4b7fe700 1 -- 172.21.15.63:0/912035020 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c4c013420 con 0x7f9c54102e00 2024-03-22T07:24:06.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.691+0000 7f9c4b7fe700 1 -- 172.21.15.63:0/912035020 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c4c01c630 con 0x7f9c54102e00 2024-03-22T07:24:06.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.691+0000 7f9c59768700 1 -- 172.21.15.63:0/912035020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5410e3b0 con 0x7f9c54102e00 2024-03-22T07:24:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.692+0000 7f9c59768700 1 -- 172.21.15.63:0/912035020 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c54100110 con 0x7f9c54102e00 2024-03-22T07:24:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.693+0000 7f9c4b7fe700 1 -- 172.21.15.63:0/912035020 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f9c4c013590 con 0x7f9c54102e00 2024-03-22T07:24:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.693+0000 7f9c4b7fe700 1 --2- 172.21.15.63:0/912035020 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9c4006d380 0x7f9c4006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.693+0000 7f9c4b7fe700 1 -- 172.21.15.63:0/912035020 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9c4c08dac0 con 0x7f9c54102e00 2024-03-22T07:24:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.693+0000 7f9c527fc700 1 --2- 172.21.15.63:0/912035020 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9c4006d380 0x7f9c4006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.694+0000 7f9c527fc700 1 --2- 172.21.15.63:0/912035020 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9c4006d380 0x7f9c4006f830 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c5410ece0 tx=0x7f9c3c0043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:06.698 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.698+0000 7f9c4b7fe700 1 -- 172.21.15.63:0/912035020 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9c4c05b150 con 0x7f9c54102e00 2024-03-22T07:24:06.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:06 smithi069 ceph-mon[83631]: pgmap v130: 65 pgs: 65 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:24:06.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:06 smithi069 ceph-mon[83631]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:24:06.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:06 smithi069 ceph-mon[83631]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:24:06.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:06 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/975473441' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:24:06.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:06 smithi063 ceph-mon[112784]: pgmap v130: 65 pgs: 65 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:24:06.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:06 smithi063 ceph-mon[112784]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:24:06.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:06 smithi063 ceph-mon[112784]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:24:06.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:06 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/975473441' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:24:06.924 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.924+0000 7f9c59768700 1 -- 172.21.15.63:0/912035020 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c5410f050 con 0x7f9c4006d380 2024-03-22T07:24:06.925 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.925+0000 7f9c4b7fe700 1 -- 172.21.15.63:0/912035020 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9c5410f050 con 0x7f9c4006d380 2024-03-22T07:24:06.925 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:24:06.925 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:24:06.926 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:24:06.926 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:24:06.926 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-03-22T07:24:06.926 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-03-22T07:24:06.926 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr", 2024-03-22T07:24:06.926 INFO:teuthology.orchestra.run.smithi063.stdout: "mon" 2024-03-22T07:24:06.926 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-03-22T07:24:06.926 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "10/21 daemons upgraded", 2024-03-22T07:24:06.926 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T07:24:06.926 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:24:06.926 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:24:06.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.927+0000 7f9c59768700 1 -- 172.21.15.63:0/912035020 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9c4006d380 msgr2=0x7f9c4006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:06.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.927+0000 7f9c59768700 1 --2- 172.21.15.63:0/912035020 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9c4006d380 0x7f9c4006f830 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c5410ece0 tx=0x7f9c3c0043c0 comp rx=0 tx=0).stop 2024-03-22T07:24:06.927 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.927+0000 7f9c59768700 1 -- 172.21.15.63:0/912035020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c54102e00 msgr2=0x7f9c54113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:06.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.927+0000 7f9c59768700 1 --2- 172.21.15.63:0/912035020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c54102e00 0x7f9c54113990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00b3b0 tx=0x7f9c4c009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:06.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.927+0000 7f9c59768700 1 -- 172.21.15.63:0/912035020 shutdown_connections 2024-03-22T07:24:06.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.927+0000 7f9c59768700 1 --2- 172.21.15.63:0/912035020 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9c4006d380 0x7f9c4006f830 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:06.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.927+0000 7f9c59768700 1 --2- 172.21.15.63:0/912035020 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c54102e00 0x7f9c54113990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:06.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.927+0000 7f9c59768700 1 --2- 172.21.15.63:0/912035020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c541037a0 0x7f9c54113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:06.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.927+0000 7f9c59768700 1 -- 172.21.15.63:0/912035020 >> 172.21.15.63:0/912035020 conn(0x7f9c540f89b0 msgr2=0x7f9c541073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:06.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.927+0000 7f9c59768700 1 -- 172.21.15.63:0/912035020 shutdown_connections 2024-03-22T07:24:06.928 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:06.927+0000 7f9c59768700 1 -- 172.21.15.63:0/912035020 wait complete. 2024-03-22T07:24:07.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.030+0000 7f2d9357c700 1 -- 172.21.15.63:0/162464444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d8c101d70 msgr2=0x7f2d8c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:07.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.030+0000 7f2d9357c700 1 --2- 172.21.15.63:0/162464444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d8c101d70 0x7f2d8c102140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c004940 tx=0x7f2d7c009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:07.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.030+0000 7f2d9357c700 1 -- 172.21.15.63:0/162464444 shutdown_connections 2024-03-22T07:24:07.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.030+0000 7f2d9357c700 1 --2- 172.21.15.63:0/162464444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d8c102710 0x7f2d8c0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:07.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.030+0000 7f2d9357c700 1 --2- 172.21.15.63:0/162464444 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d8c101d70 0x7f2d8c102140 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:07.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.030+0000 7f2d9357c700 1 -- 172.21.15.63:0/162464444 >> 172.21.15.63:0/162464444 conn(0x7f2d8c072bc0 msgr2=0x7f2d8c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:07.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.030+0000 7f2d9357c700 1 -- 172.21.15.63:0/162464444 shutdown_connections 2024-03-22T07:24:07.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.030+0000 7f2d9357c700 1 -- 172.21.15.63:0/162464444 wait complete. 2024-03-22T07:24:07.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.030+0000 7f2d9357c700 1 Processor -- start 2024-03-22T07:24:07.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.030+0000 7f2d9357c700 1 -- start start 2024-03-22T07:24:07.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.031+0000 7f2d9357c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d8c101d70 0x7f2d8c112a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:07.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.031+0000 7f2d9357c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d8c102710 0x7f2d8c112f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:07.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.031+0000 7f2d9357c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c113630 con 0x7f2d8c102710 2024-03-22T07:24:07.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.031+0000 7f2d9357c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c10c990 con 0x7f2d8c101d70 2024-03-22T07:24:07.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.031+0000 7f2d90b17700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d8c102710 0x7f2d8c112f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:07.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.031+0000 7f2d90b17700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d8c102710 0x7f2d8c112f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58966/0 (socket says 172.21.15.63:58966) 2024-03-22T07:24:07.032 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.031+0000 7f2d90b17700 1 -- 172.21.15.63:0/1617933581 learned_addr learned my addr 172.21.15.63:0/1617933581 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:24:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.031+0000 7f2d91318700 1 --2- 172.21.15.63:0/1617933581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d8c101d70 0x7f2d8c112a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.031+0000 7f2d90b17700 1 -- 172.21.15.63:0/1617933581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d8c101d70 msgr2=0x7f2d8c112a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.031+0000 7f2d90b17700 1 --2- 172.21.15.63:0/1617933581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d8c101d70 0x7f2d8c112a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.031+0000 7f2d90b17700 1 -- 172.21.15.63:0/1617933581 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c005040 con 0x7f2d8c102710 2024-03-22T07:24:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.031+0000 7f2d90b17700 1 --2- 172.21.15.63:0/1617933581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d8c102710 0x7f2d8c112f50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800b790 tx=0x7f2d88009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.032+0000 7f2d81ffb700 1 -- 172.21.15.63:0/1617933581 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d8800f6f0 con 0x7f2d8c102710 2024-03-22T07:24:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.032+0000 7f2d81ffb700 1 -- 172.21.15.63:0/1617933581 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d8800fd20 con 0x7f2d8c102710 2024-03-22T07:24:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.032+0000 7f2d9357c700 1 -- 172.21.15.63:0/1617933581 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8c10cc70 con 0x7f2d8c102710 2024-03-22T07:24:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.032+0000 7f2d81ffb700 1 -- 172.21.15.63:0/1617933581 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d880103a0 con 0x7f2d8c102710 2024-03-22T07:24:07.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.032+0000 7f2d9357c700 1 -- 172.21.15.63:0/1617933581 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8c10d230 con 0x7f2d8c102710 2024-03-22T07:24:07.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.033+0000 7f2d9357c700 1 -- 172.21.15.63:0/1617933581 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8c0fd4c0 con 0x7f2d8c102710 2024-03-22T07:24:07.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.033+0000 7f2d81ffb700 1 -- 172.21.15.63:0/1617933581 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f2d88011400 con 0x7f2d8c102710 2024-03-22T07:24:07.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.033+0000 7f2d81ffb700 1 --2- 172.21.15.63:0/1617933581 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2d7806f590 0x7f2d78071a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:07.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.034+0000 7f2d81ffb700 1 -- 172.21.15.63:0/1617933581 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2d8808c680 con 0x7f2d8c102710 2024-03-22T07:24:07.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.034+0000 7f2d91318700 1 --2- 172.21.15.63:0/1617933581 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2d7806f590 0x7f2d78071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:07.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.034+0000 7f2d91318700 1 --2- 172.21.15.63:0/1617933581 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2d7806f590 0x7f2d78071a40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c004f90 tx=0x7f2d7c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:07.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.038+0000 7f2d81ffb700 1 -- 172.21.15.63:0/1617933581 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2d88059d10 con 0x7f2d8c102710 2024-03-22T07:24:07.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.325+0000 7f2d9357c700 1 -- 172.21.15.63:0/1617933581 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2d8c05f5f0 con 0x7f2d8c102710 2024-03-22T07:24:07.325 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.325+0000 7f2d81ffb700 1 -- 172.21.15.63:0/1617933581 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f2d880598a0 con 0x7f2d8c102710 2024-03-22T07:24:07.325 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-03-22T07:24:07.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.328+0000 7f2d9357c700 1 -- 172.21.15.63:0/1617933581 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2d7806f590 msgr2=0x7f2d78071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:07.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.328+0000 7f2d9357c700 1 --2- 172.21.15.63:0/1617933581 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2d7806f590 0x7f2d78071a40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c004f90 tx=0x7f2d7c009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:07.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.328+0000 7f2d9357c700 1 -- 172.21.15.63:0/1617933581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d8c102710 msgr2=0x7f2d8c112f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:07.328 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.328+0000 7f2d9357c700 1 --2- 172.21.15.63:0/1617933581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d8c102710 0x7f2d8c112f50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800b790 tx=0x7f2d88009220 comp rx=0 tx=0).stop 2024-03-22T07:24:07.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.328+0000 7f2d9357c700 1 -- 172.21.15.63:0/1617933581 shutdown_connections 2024-03-22T07:24:07.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.328+0000 7f2d9357c700 1 --2- 172.21.15.63:0/1617933581 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2d7806f590 0x7f2d78071a40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:07.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.328+0000 7f2d9357c700 1 --2- 172.21.15.63:0/1617933581 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d8c101d70 0x7f2d8c112a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:07.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.328+0000 7f2d9357c700 1 --2- 172.21.15.63:0/1617933581 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2d8c102710 0x7f2d8c112f50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:07.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.328+0000 7f2d9357c700 1 -- 172.21.15.63:0/1617933581 >> 172.21.15.63:0/1617933581 conn(0x7f2d8c072bc0 msgr2=0x7f2d8c0fae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:07.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.328+0000 7f2d9357c700 1 -- 172.21.15.63:0/1617933581 shutdown_connections 2024-03-22T07:24:07.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:07.328+0000 7f2d9357c700 1 -- 172.21.15.63:0/1617933581 wait complete. 2024-03-22T07:24:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:07 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:07 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/1935787459' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:24:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:07 smithi063 ceph-mon[112784]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:24:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:07 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T07:24:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:07 smithi063 ceph-mon[112784]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T07:24:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:07 smithi063 ceph-mon[112784]: Upgrade: osd.4 is safe to restart 2024-03-22T07:24:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:07 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/1617933581' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:24:07.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:07 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:07.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:07 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/1935787459' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:24:07.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:07 smithi069 ceph-mon[83631]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:24:07.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:07 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T07:24:07.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:07 smithi069 ceph-mon[83631]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-22T07:24:07.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:07 smithi069 ceph-mon[83631]: Upgrade: osd.4 is safe to restart 2024-03-22T07:24:07.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:07 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/1617933581' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:24:08.726 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:08 smithi069 ceph-mon[83631]: pgmap v131: 65 pgs: 65 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:24:08.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:08 smithi063 ceph-mon[112784]: pgmap v131: 65 pgs: 65 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:24:10.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:09 smithi069 ceph-mon[83631]: Upgrade: Updating osd.4 2024-03-22T07:24:10.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:09 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:10.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:09 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T07:24:10.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:09 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:24:10.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:09 smithi069 ceph-mon[83631]: Deploying daemon osd.4 on smithi069 2024-03-22T07:24:10.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:09 smithi063 ceph-mon[112784]: Upgrade: Updating osd.4 2024-03-22T07:24:10.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:09 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:10.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:09 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-22T07:24:10.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:09 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:24:10.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:09 smithi063 ceph-mon[112784]: Deploying daemon osd.4 on smithi069 2024-03-22T07:24:11.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:10 smithi069 ceph-mon[83631]: pgmap v132: 65 pgs: 65 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:24:11.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:10 smithi063 ceph-mon[112784]: pgmap v132: 65 pgs: 65 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:24:12.331 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:12 smithi069 systemd[1]: Stopping Ceph osd.4 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:24:12.779 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:12 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[48724]: 2024-03-22T07:24:12.353+0000 7f8a93232700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:24:12.779 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:12 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[48724]: 2024-03-22T07:24:12.353+0000 7f8a93232700 -1 osd.4 70 *** Got signal Terminated *** 2024-03-22T07:24:12.780 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:12 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[48724]: 2024-03-22T07:24:12.353+0000 7f8a93232700 -1 osd.4 70 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:24:13.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:12 smithi063 ceph-mon[112784]: pgmap v133: 65 pgs: 65 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-22T07:24:13.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:12 smithi063 ceph-mon[112784]: osd.4 marked itself down and dead 2024-03-22T07:24:13.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:12 smithi069 ceph-mon[83631]: pgmap v133: 65 pgs: 65 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-22T07:24:13.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:12 smithi069 ceph-mon[83631]: osd.4 marked itself down and dead 2024-03-22T07:24:13.605 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:13 smithi069 podman[97587]: 2024-03-22 07:24:13.284757168 +0000 UTC m=+1.049188932 container died 76bc35d01de0e406ff0543c6713e85ac16cd8029f8588616326a4ed8aefd4369 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=quincy-9c4e0c3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:24:13.946 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:13 smithi069 ceph-mon[83631]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:24:13.946 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:13 smithi069 ceph-mon[83631]: failed to encode map e71 with expected crc 2024-03-22T07:24:13.946 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:13 smithi069 ceph-mon[83631]: osdmap e71: 6 total, 5 up, 6 in 2024-03-22T07:24:13.946 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:13 smithi069 podman[97587]: 2024-03-22 07:24:13.712513001 +0000 UTC m=+1.476944766 container cleanup 76bc35d01de0e406ff0543c6713e85ac16cd8029f8588616326a4ed8aefd4369 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=quincy-9c4e0c3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:24:13.946 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:13 smithi069 bash[97587]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4 2024-03-22T07:24:14.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:13 smithi063 ceph-mon[112784]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:24:14.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:13 smithi063 ceph-mon[112784]: failed to encode map e71 with expected crc 2024-03-22T07:24:14.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:13 smithi063 ceph-mon[112784]: osdmap e71: 6 total, 5 up, 6 in 2024-03-22T07:24:14.279 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:13 smithi069 podman[97605]: 2024-03-22 07:24:13.946496751 +0000 UTC m=+0.657103888 container remove 76bc35d01de0e406ff0543c6713e85ac16cd8029f8588616326a4ed8aefd4369 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4, GIT_BRANCH=HEAD, RELEASE=quincy-9c4e0c3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-22T07:24:14.650 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:14 smithi069 podman[97697]: 2024-03-22 07:24:14.347239794 +0000 UTC m=+0.106456366 container create 974f5ec44d186c0345aa59219d375e99ec2952eacb06a5d1e4d80bca4b2f1935 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-db0330b, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-22T07:24:14.650 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:14 smithi069 podman[97697]: 2024-03-22 07:24:14.258758214 +0000 UTC m=+0.017974810 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:24:14.650 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:14 smithi069 podman[97697]: 2024-03-22 07:24:14.566655798 +0000 UTC m=+0.325872371 container init 974f5ec44d186c0345aa59219d375e99ec2952eacb06a5d1e4d80bca4b2f1935 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-22T07:24:14.650 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:14 smithi069 podman[97697]: 2024-03-22 07:24:14.569826544 +0000 UTC m=+0.329043126 container start 974f5ec44d186c0345aa59219d375e99ec2952eacb06a5d1e4d80bca4b2f1935 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-22T07:24:15.029 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:14 smithi069 podman[97697]: 2024-03-22 07:24:14.649779955 +0000 UTC m=+0.408996533 container attach 974f5ec44d186c0345aa59219d375e99ec2952eacb06a5d1e4d80bca4b2f1935 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-22T07:24:15.030 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:14 smithi069 podman[97697]: 2024-03-22 07:24:14.86488976 +0000 UTC m=+0.624106326 container died 974f5ec44d186c0345aa59219d375e99ec2952eacb06a5d1e4d80bca4b2f1935 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-22T07:24:15.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:14 smithi069 ceph-mon[83631]: pgmap v135: 65 pgs: 2 peering, 5 stale+active+clean, 58 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-22T07:24:15.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:14 smithi069 ceph-mon[83631]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-22T07:24:15.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:14 smithi069 ceph-mon[83631]: osdmap e72: 6 total, 5 up, 6 in 2024-03-22T07:24:15.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:14 smithi069 ceph-mon[83631]: failed to encode map e72 with expected crc 2024-03-22T07:24:15.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:14 smithi069 ceph-mon[83631]: failed to encode map e72 with expected crc 2024-03-22T07:24:15.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:14 smithi063 ceph-mon[112784]: pgmap v135: 65 pgs: 2 peering, 5 stale+active+clean, 58 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-03-22T07:24:15.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:14 smithi063 ceph-mon[112784]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-22T07:24:15.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:14 smithi063 ceph-mon[112784]: osdmap e72: 6 total, 5 up, 6 in 2024-03-22T07:24:15.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:14 smithi063 ceph-mon[112784]: failed to encode map e72 with expected crc 2024-03-22T07:24:15.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:14 smithi063 ceph-mon[112784]: failed to encode map e72 with expected crc 2024-03-22T07:24:15.780 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:15 smithi069 podman[97723]: 2024-03-22 07:24:15.461819345 +0000 UTC m=+0.591780562 container remove 974f5ec44d186c0345aa59219d375e99ec2952eacb06a5d1e4d80bca4b2f1935 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:24:15.780 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:15 smithi069 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-22T07:24:15.780 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:15 smithi069 systemd[1]: Stopped Ceph osd.4 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:24:15.780 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:15 smithi069 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.4.service: Consumed 58.814s CPU time. 2024-03-22T07:24:15.780 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:15 smithi069 systemd[1]: Starting Ceph osd.4 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:24:16.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:15 smithi063 ceph-mon[112784]: Health check failed: Degraded data redundancy: 14/354 objects degraded (3.955%), 4 pgs degraded (PG_DEGRADED) 2024-03-22T07:24:16.182 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:15 smithi069 ceph-mon[83631]: Health check failed: Degraded data redundancy: 14/354 objects degraded (3.955%), 4 pgs degraded (PG_DEGRADED) 2024-03-22T07:24:16.182 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:15 smithi069 podman[97825]: 2024-03-22 07:24:15.800840171 +0000 UTC m=+0.017360299 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:24:16.182 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:15 smithi069 podman[97825]: 2024-03-22 07:24:15.92142578 +0000 UTC m=+0.137945942 container create 1715b2c759054724899cae417696c3f01cdab7be4cfaf9d1e4f428836873523b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-activate, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-03-22T07:24:16.529 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:16 smithi069 podman[97825]: 2024-03-22 07:24:16.182566906 +0000 UTC m=+0.399087030 container init 1715b2c759054724899cae417696c3f01cdab7be4cfaf9d1e4f428836873523b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-03-22T07:24:16.529 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:16 smithi069 podman[97825]: 2024-03-22 07:24:16.185861653 +0000 UTC m=+0.402381780 container start 1715b2c759054724899cae417696c3f01cdab7be4cfaf9d1e4f428836873523b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-22T07:24:16.529 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:16 smithi069 podman[97825]: 2024-03-22 07:24:16.276137621 +0000 UTC m=+0.492657745 container attach 1715b2c759054724899cae417696c3f01cdab7be4cfaf9d1e4f428836873523b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-activate, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-db0330b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-22T07:24:16.986 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:16 smithi069 ceph-mon[83631]: pgmap v137: 65 pgs: 4 active+undersized+degraded, 2 peering, 3 stale+active+clean, 4 active+undersized, 52 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 14/354 objects degraded (3.955%) 2024-03-22T07:24:16.986 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:16 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:16.986 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:16 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:24:16.986 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:16 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-activate[97840]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-22T07:24:16.986 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:16 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-activate[97840]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-22T07:24:16.986 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:16 smithi069 bash[97825]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-22T07:24:16.987 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:16 smithi069 bash[97825]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-22T07:24:16.987 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:16 smithi069 bash[97825]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-22T07:24:16.987 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:16 smithi069 bash[97825]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-22T07:24:16.987 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:16 smithi069 bash[97825]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-22T07:24:16.987 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:16 smithi069 bash[97825]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-22T07:24:16.987 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:16 smithi069 bash[97825]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-22T07:24:16.987 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:16 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-activate[97840]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-22T07:24:16.987 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:16 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-activate[97840]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-22T07:24:16.987 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:16 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-activate[97840]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-22T07:24:16.987 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:16 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-activate[97840]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-22T07:24:16.987 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:16 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-activate[97840]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-22T07:24:16.987 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:16 smithi069 podman[97825]: 2024-03-22 07:24:16.646519778 +0000 UTC m=+0.863039905 container died 1715b2c759054724899cae417696c3f01cdab7be4cfaf9d1e4f428836873523b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-activate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b) 2024-03-22T07:24:17.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:16 smithi063 ceph-mon[112784]: pgmap v137: 65 pgs: 4 active+undersized+degraded, 2 peering, 3 stale+active+clean, 4 active+undersized, 52 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 14/354 objects degraded (3.955%) 2024-03-22T07:24:17.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:16 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:17.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:16 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:24:17.529 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:17 smithi069 podman[97959]: 2024-03-22 07:24:17.269378439 +0000 UTC m=+0.617831937 container remove 1715b2c759054724899cae417696c3f01cdab7be4cfaf9d1e4f428836873523b (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-22T07:24:17.530 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:17 smithi069 podman[98022]: 2024-03-22 07:24:17.382968781 +0000 UTC m=+0.018448695 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:24:17.530 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:17 smithi069 podman[98022]: 2024-03-22 07:24:17.495365389 +0000 UTC m=+0.130845331 container create d6340a0e1c6c9bcedef5ef216271acd3ec4f2a8f4328362bcd4bc8bcd975ec83 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-22T07:24:18.029 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:17 smithi069 podman[98022]: 2024-03-22 07:24:17.680816652 +0000 UTC m=+0.316296561 container init d6340a0e1c6c9bcedef5ef216271acd3ec4f2a8f4328362bcd4bc8bcd975ec83 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-22T07:24:18.030 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:17 smithi069 podman[98022]: 2024-03-22 07:24:17.683681897 +0000 UTC m=+0.319161797 container start d6340a0e1c6c9bcedef5ef216271acd3ec4f2a8f4328362bcd4bc8bcd975ec83 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-22T07:24:18.030 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:17 smithi069 ceph-osd[98041]: -- 172.21.15.69:0/1185651958 <== mon.1 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562ac91391e0 con 0x562ac910b000 2024-03-22T07:24:18.030 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:17 smithi069 bash[98022]: d6340a0e1c6c9bcedef5ef216271acd3ec4f2a8f4328362bcd4bc8bcd975ec83 2024-03-22T07:24:18.030 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:17 smithi069 systemd[1]: Started Ceph osd.4 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:24:18.529 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:18 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[98037]: 2024-03-22T07:24:18.231+0000 7fe8951317c0 -1 Falling back to public interface 2024-03-22T07:24:19.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:18 smithi069 ceph-mon[83631]: pgmap v138: 65 pgs: 12 active+undersized+degraded, 2 peering, 1 stale+active+clean, 12 active+undersized, 38 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 47/354 objects degraded (13.277%) 2024-03-22T07:24:19.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:18 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:19.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:18 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:19.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:18 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:24:19.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:18 smithi063 ceph-mon[112784]: pgmap v138: 65 pgs: 12 active+undersized+degraded, 2 peering, 1 stale+active+clean, 12 active+undersized, 38 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 47/354 objects degraded (13.277%) 2024-03-22T07:24:19.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:18 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:19.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:18 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:19.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:18 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:24:20.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:19 smithi063 ceph-mon[112784]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-22T07:24:20.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:19 smithi069 ceph-mon[83631]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-22T07:24:21.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:20 smithi069 ceph-mon[83631]: pgmap v139: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 60/354 objects degraded (16.949%) 2024-03-22T07:24:21.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:20 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:21.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:20 smithi063 ceph-mon[112784]: pgmap v139: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 60/354 objects degraded (16.949%) 2024-03-22T07:24:21.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:20 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:22.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:21 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:22.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:21 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:22.919 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:22 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[98037]: 2024-03-22T07:24:22.797+0000 7fe8951317c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-03-22T07:24:23.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:22 smithi063 ceph-mon[112784]: pgmap v140: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 60/354 objects degraded (16.949%) 2024-03-22T07:24:23.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:22 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:23.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:22 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:23.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:22 smithi063 ceph-mon[112784]: Health check update: Degraded data redundancy: 60/354 objects degraded (16.949%), 16 pgs degraded (PG_DEGRADED) 2024-03-22T07:24:23.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:22 smithi069 ceph-mon[83631]: pgmap v140: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 60/354 objects degraded (16.949%) 2024-03-22T07:24:23.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:22 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:23.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:22 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:23.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:22 smithi069 ceph-mon[83631]: Health check update: Degraded data redundancy: 60/354 objects degraded (16.949%), 16 pgs degraded (PG_DEGRADED) 2024-03-22T07:24:23.595 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:23 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[98037]: 2024-03-22T07:24:23.321+0000 7fe8951317c0 -1 osd.4 70 log_to_monitors true 2024-03-22T07:24:24.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:23 smithi069 ceph-mon[83631]: from='osd.4 [v2:172.21.15.69:6808/2629704021,v1:172.21.15.69:6809/2629704021]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T07:24:24.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:23 smithi069 ceph-mon[83631]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T07:24:24.280 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:24:23 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[98037]: 2024-03-22T07:24:23.990+0000 7fe886589700 -1 osd.4 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:24:24.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:23 smithi063 ceph-mon[112784]: from='osd.4 [v2:172.21.15.69:6808/2629704021,v1:172.21.15.69:6809/2629704021]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T07:24:24.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:23 smithi063 ceph-mon[112784]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-22T07:24:25.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:24 smithi069 ceph-mon[83631]: pgmap v141: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 60/354 objects degraded (16.949%) 2024-03-22T07:24:25.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:24 smithi069 ceph-mon[83631]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T07:24:25.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:24 smithi069 ceph-mon[83631]: failed to encode map e73 with expected crc 2024-03-22T07:24:25.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:24 smithi069 ceph-mon[83631]: osdmap e73: 6 total, 5 up, 6 in 2024-03-22T07:24:25.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:24 smithi069 ceph-mon[83631]: from='osd.4 [v2:172.21.15.69:6808/2629704021,v1:172.21.15.69:6809/2629704021]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:24:25.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:24 smithi069 ceph-mon[83631]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:24:25.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:24 smithi063 ceph-mon[112784]: pgmap v141: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 299 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 60/354 objects degraded (16.949%) 2024-03-22T07:24:25.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:24 smithi063 ceph-mon[112784]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-22T07:24:25.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:24 smithi063 ceph-mon[112784]: failed to encode map e73 with expected crc 2024-03-22T07:24:25.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:24 smithi063 ceph-mon[112784]: osdmap e73: 6 total, 5 up, 6 in 2024-03-22T07:24:25.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:24 smithi063 ceph-mon[112784]: from='osd.4 [v2:172.21.15.69:6808/2629704021,v1:172.21.15.69:6809/2629704021]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:24:25.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:24 smithi063 ceph-mon[112784]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:24:26.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:25 smithi069 ceph-mon[83631]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:24:26.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:25 smithi069 ceph-mon[83631]: failed to encode map e74 with expected crc 2024-03-22T07:24:26.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:25 smithi069 ceph-mon[83631]: osd.4 [v2:172.21.15.69:6808/2629704021,v1:172.21.15.69:6809/2629704021] boot 2024-03-22T07:24:26.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:25 smithi069 ceph-mon[83631]: osdmap e74: 6 total, 6 up, 6 in 2024-03-22T07:24:26.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:25 smithi069 ceph-mon[83631]: failed to encode map e74 with expected crc 2024-03-22T07:24:26.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:25 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:24:26.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:25 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:26.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:25 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:26.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:25 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:24:26.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:25 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:24:26.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:25 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:26.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:25 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:24:26.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:25 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:24:26.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:25 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:24:26.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:25 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:24:26.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:25 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T07:24:26.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:25 smithi069 ceph-mon[83631]: osdmap e75: 6 total, 6 up, 6 in 2024-03-22T07:24:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:25 smithi063 ceph-mon[112784]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:24:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:25 smithi063 ceph-mon[112784]: failed to encode map e74 with expected crc 2024-03-22T07:24:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:25 smithi063 ceph-mon[112784]: osd.4 [v2:172.21.15.69:6808/2629704021,v1:172.21.15.69:6809/2629704021] boot 2024-03-22T07:24:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:25 smithi063 ceph-mon[112784]: osdmap e74: 6 total, 6 up, 6 in 2024-03-22T07:24:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:25 smithi063 ceph-mon[112784]: failed to encode map e74 with expected crc 2024-03-22T07:24:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:25 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-22T07:24:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:25 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:26.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:25 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:26.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:25 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:24:26.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:25 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:24:26.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:25 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:26.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:25 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:24:26.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:25 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:24:26.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:25 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:24:26.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:25 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:24:26.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:25 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T07:24:26.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:25 smithi063 ceph-mon[112784]: osdmap e75: 6 total, 6 up, 6 in 2024-03-22T07:24:27.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:26 smithi069 ceph-mon[83631]: pgmap v144: 65 pgs: 6 peering, 13 active+undersized+degraded, 15 active+undersized, 31 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 48/354 objects degraded (13.559%) 2024-03-22T07:24:27.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:26 smithi069 ceph-mon[83631]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T07:24:27.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:26 smithi069 ceph-mon[83631]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-22T07:24:27.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:26 smithi069 ceph-mon[83631]: failed to encode map e75 with expected crc 2024-03-22T07:24:27.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:26 smithi069 ceph-mon[83631]: failed to encode map e75 with expected crc 2024-03-22T07:24:27.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:26 smithi069 ceph-mon[83631]: failed to encode map e75 with expected crc 2024-03-22T07:24:27.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:26 smithi063 ceph-mon[112784]: pgmap v144: 65 pgs: 6 peering, 13 active+undersized+degraded, 15 active+undersized, 31 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 48/354 objects degraded (13.559%) 2024-03-22T07:24:27.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:26 smithi063 ceph-mon[112784]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T07:24:27.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:26 smithi063 ceph-mon[112784]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-03-22T07:24:27.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:26 smithi063 ceph-mon[112784]: failed to encode map e75 with expected crc 2024-03-22T07:24:27.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:26 smithi063 ceph-mon[112784]: failed to encode map e75 with expected crc 2024-03-22T07:24:27.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:26 smithi063 ceph-mon[112784]: failed to encode map e75 with expected crc 2024-03-22T07:24:28.274 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:27 smithi063 ceph-mon[112784]: Health check update: Degraded data redundancy: 48/354 objects degraded (13.559%), 13 pgs degraded (PG_DEGRADED) 2024-03-22T07:24:28.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:27 smithi069 ceph-mon[83631]: Health check update: Degraded data redundancy: 48/354 objects degraded (13.559%), 13 pgs degraded (PG_DEGRADED) 2024-03-22T07:24:29.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:28 smithi069 ceph-mon[83631]: pgmap v146: 65 pgs: 16 peering, 7 active+undersized+degraded, 7 active+undersized, 35 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s; 19/354 objects degraded (5.367%) 2024-03-22T07:24:29.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:28 smithi063 ceph-mon[112784]: pgmap v146: 65 pgs: 16 peering, 7 active+undersized+degraded, 7 active+undersized, 35 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s; 19/354 objects degraded (5.367%) 2024-03-22T07:24:30.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:30 smithi063 ceph-mon[112784]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 19/354 objects degraded (5.367%), 7 pgs degraded) 2024-03-22T07:24:30.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:30 smithi063 ceph-mon[112784]: Cluster is now healthy 2024-03-22T07:24:30.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:30 smithi069 ceph-mon[83631]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 19/354 objects degraded (5.367%), 7 pgs degraded) 2024-03-22T07:24:30.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:30 smithi069 ceph-mon[83631]: Cluster is now healthy 2024-03-22T07:24:31.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:31 smithi063 ceph-mon[112784]: pgmap v147: 65 pgs: 16 peering, 49 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-22T07:24:31.504 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:31 smithi069 ceph-mon[83631]: pgmap v147: 65 pgs: 16 peering, 49 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-22T07:24:32.773 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:32 smithi063 ceph-mon[112784]: pgmap v148: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-22T07:24:32.774 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:32.774 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:24:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:32 smithi069 ceph-mon[83631]: pgmap v148: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-22T07:24:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:32 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:32 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:24:34.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:34 smithi069 ceph-mon[83631]: pgmap v149: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-03-22T07:24:34.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:34 smithi063 ceph-mon[112784]: pgmap v149: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-03-22T07:24:36.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:36 smithi069 ceph-mon[83631]: pgmap v150: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-22T07:24:36.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:36 smithi063 ceph-mon[112784]: pgmap v150: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-22T07:24:37.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.446+0000 7fa868ed0700 1 -- 172.21.15.63:0/1215194139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa864102fd0 msgr2=0x7fa8641033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:37.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.446+0000 7fa868ed0700 1 --2- 172.21.15.63:0/1215194139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa864102fd0 0x7fa8641033a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa85c004940 tx=0x7fa85c009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:37.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.446+0000 7fa868ed0700 1 -- 172.21.15.63:0/1215194139 shutdown_connections 2024-03-22T07:24:37.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.446+0000 7fa868ed0700 1 --2- 172.21.15.63:0/1215194139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8640fb540 0x7fa8640fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:37.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.446+0000 7fa868ed0700 1 --2- 172.21.15.63:0/1215194139 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa864102fd0 0x7fa8641033a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:37.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.446+0000 7fa868ed0700 1 -- 172.21.15.63:0/1215194139 >> 172.21.15.63:0/1215194139 conn(0x7fa864071530 msgr2=0x7fa864071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:37.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.446+0000 7fa868ed0700 1 -- 172.21.15.63:0/1215194139 shutdown_connections 2024-03-22T07:24:37.447 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.446+0000 7fa868ed0700 1 -- 172.21.15.63:0/1215194139 wait complete. 2024-03-22T07:24:37.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.447+0000 7fa868ed0700 1 Processor -- start 2024-03-22T07:24:37.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.447+0000 7fa868ed0700 1 -- start start 2024-03-22T07:24:37.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.447+0000 7fa868ed0700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8640fb540 0x7fa8640fc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:37.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.447+0000 7fa868ed0700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa864102fd0 0x7fa8640fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:37.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.447+0000 7fa868ed0700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8640fe6b0 con 0x7fa864102fd0 2024-03-22T07:24:37.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.447+0000 7fa868ed0700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8640fd1a0 con 0x7fa8640fb540 2024-03-22T07:24:37.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.447+0000 7fa861d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa864102fd0 0x7fa8640fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:37.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.447+0000 7fa861d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa864102fd0 0x7fa8640fcc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44556/0 (socket says 172.21.15.63:44556) 2024-03-22T07:24:37.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.447+0000 7fa861d9b700 1 -- 172.21.15.63:0/896360428 learned_addr learned my addr 172.21.15.63:0/896360428 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:24:37.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.447+0000 7fa86259c700 1 --2- 172.21.15.63:0/896360428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8640fb540 0x7fa8640fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:37.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.447+0000 7fa861d9b700 1 -- 172.21.15.63:0/896360428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8640fb540 msgr2=0x7fa8640fc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:37.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.447+0000 7fa861d9b700 1 --2- 172.21.15.63:0/896360428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8640fb540 0x7fa8640fc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:37.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.447+0000 7fa861d9b700 1 -- 172.21.15.63:0/896360428 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c005040 con 0x7fa864102fd0 2024-03-22T07:24:37.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.448+0000 7fa861d9b700 1 --2- 172.21.15.63:0/896360428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa864102fd0 0x7fa8640fcc60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00b790 tx=0x7fa84c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:37.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.448+0000 7fa85affd700 1 -- 172.21.15.63:0/896360428 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa84c00f770 con 0x7fa864102fd0 2024-03-22T07:24:37.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.448+0000 7fa85affd700 1 -- 172.21.15.63:0/896360428 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa84c00fda0 con 0x7fa864102fd0 2024-03-22T07:24:37.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.448+0000 7fa868ed0700 1 -- 172.21.15.63:0/896360428 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8640fd480 con 0x7fa864102fd0 2024-03-22T07:24:37.449 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.448+0000 7fa85affd700 1 -- 172.21.15.63:0/896360428 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa84c00d310 con 0x7fa864102fd0 2024-03-22T07:24:37.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.448+0000 7fa868ed0700 1 -- 172.21.15.63:0/896360428 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86411a1b0 con 0x7fa864102fd0 2024-03-22T07:24:37.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.449+0000 7fa868ed0700 1 -- 172.21.15.63:0/896360428 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa864100270 con 0x7fa864102fd0 2024-03-22T07:24:37.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.449+0000 7fa85affd700 1 -- 172.21.15.63:0/896360428 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fa84c010420 con 0x7fa864102fd0 2024-03-22T07:24:37.450 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.450+0000 7fa85affd700 1 --2- 172.21.15.63:0/896360428 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fa85006d380 0x7fa85006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:37.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.450+0000 7fa85affd700 1 -- 172.21.15.63:0/896360428 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa84c08c8e0 con 0x7fa864102fd0 2024-03-22T07:24:37.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.450+0000 7fa86259c700 1 --2- 172.21.15.63:0/896360428 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fa85006d380 0x7fa85006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:37.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.450+0000 7fa86259c700 1 --2- 172.21.15.63:0/896360428 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fa85006d380 0x7fa85006f830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa85c004f90 tx=0x7fa85c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:37.454 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.455+0000 7fa85affd700 1 -- 172.21.15.63:0/896360428 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa84c056440 con 0x7fa864102fd0 2024-03-22T07:24:37.682 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.682+0000 7fa868ed0700 1 -- 172.21.15.63:0/896360428 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8640fe7f0 con 0x7fa85006d380 2024-03-22T07:24:37.683 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.683+0000 7fa85affd700 1 -- 172.21.15.63:0/896360428 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fa8640fe7f0 con 0x7fa85006d380 2024-03-22T07:24:37.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.686+0000 7fa868ed0700 1 -- 172.21.15.63:0/896360428 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fa85006d380 msgr2=0x7fa85006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:37.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.686+0000 7fa868ed0700 1 --2- 172.21.15.63:0/896360428 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fa85006d380 0x7fa85006f830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa85c004f90 tx=0x7fa85c009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:37.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.686+0000 7fa868ed0700 1 -- 172.21.15.63:0/896360428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa864102fd0 msgr2=0x7fa8640fcc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:37.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.686+0000 7fa868ed0700 1 --2- 172.21.15.63:0/896360428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa864102fd0 0x7fa8640fcc60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00b790 tx=0x7fa84c009250 comp rx=0 tx=0).stop 2024-03-22T07:24:37.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.686+0000 7fa868ed0700 1 -- 172.21.15.63:0/896360428 shutdown_connections 2024-03-22T07:24:37.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.686+0000 7fa868ed0700 1 --2- 172.21.15.63:0/896360428 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fa85006d380 0x7fa85006f830 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:37.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.686+0000 7fa868ed0700 1 --2- 172.21.15.63:0/896360428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8640fb540 0x7fa8640fc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:37.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.686+0000 7fa868ed0700 1 --2- 172.21.15.63:0/896360428 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fa864102fd0 0x7fa8640fcc60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:37.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.686+0000 7fa868ed0700 1 -- 172.21.15.63:0/896360428 >> 172.21.15.63:0/896360428 conn(0x7fa864071530 msgr2=0x7fa8640fa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:37.686 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.686+0000 7fa868ed0700 1 -- 172.21.15.63:0/896360428 shutdown_connections 2024-03-22T07:24:37.687 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.686+0000 7fa868ed0700 1 -- 172.21.15.63:0/896360428 wait complete. 2024-03-22T07:24:37.696 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:24:37.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:37 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:37.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.789+0000 7f7842225700 1 -- 172.21.15.63:0/4031286314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f783c1026c0 msgr2=0x7f783c0fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:37.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.789+0000 7f7842225700 1 --2- 172.21.15.63:0/4031286314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f783c1026c0 0x7f783c0fbd30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7830004940 tx=0x7f7830009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:37.790 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:37 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:37.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.789+0000 7f7842225700 1 -- 172.21.15.63:0/4031286314 shutdown_connections 2024-03-22T07:24:37.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.789+0000 7f7842225700 1 --2- 172.21.15.63:0/4031286314 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f783c1026c0 0x7f783c0fbd30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:37.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.789+0000 7f7842225700 1 --2- 172.21.15.63:0/4031286314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f783c101d20 0x7f783c1020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:37.791 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.789+0000 7f7842225700 1 -- 172.21.15.63:0/4031286314 >> 172.21.15.63:0/4031286314 conn(0x7f783c0f78b0 msgr2=0x7f783c0f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.789+0000 7f7842225700 1 -- 172.21.15.63:0/4031286314 shutdown_connections 2024-03-22T07:24:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.789+0000 7f7842225700 1 -- 172.21.15.63:0/4031286314 wait complete. 2024-03-22T07:24:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.790+0000 7f7842225700 1 Processor -- start 2024-03-22T07:24:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.790+0000 7f7842225700 1 -- start start 2024-03-22T07:24:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.790+0000 7f7842225700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f783c101d20 0x7f783c113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.790+0000 7f7842225700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f783c1026c0 0x7f783c113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.790+0000 7f7842225700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c114520 con 0x7f783c101d20 2024-03-22T07:24:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.790+0000 7f7842225700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c10db20 con 0x7f783c1026c0 2024-03-22T07:24:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.790+0000 7f783b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f783c101d20 0x7f783c113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.790+0000 7f783b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f783c101d20 0x7f783c113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44574/0 (socket says 172.21.15.63:44574) 2024-03-22T07:24:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.790+0000 7f783b7fe700 1 -- 172.21.15.63:0/4044961385 learned_addr learned my addr 172.21.15.63:0/4044961385 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:24:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.791+0000 7f783affd700 1 --2- 172.21.15.63:0/4044961385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f783c1026c0 0x7f783c113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.791+0000 7f783b7fe700 1 -- 172.21.15.63:0/4044961385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f783c1026c0 msgr2=0x7f783c113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.791+0000 7f783b7fe700 1 --2- 172.21.15.63:0/4044961385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f783c1026c0 0x7f783c113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:37.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.791+0000 7f783b7fe700 1 -- 172.21.15.63:0/4044961385 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7830005040 con 0x7f783c101d20 2024-03-22T07:24:37.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.791+0000 7f783affd700 1 --2- 172.21.15.63:0/4044961385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f783c1026c0 0x7f783c113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:24:37.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.791+0000 7f783b7fe700 1 --2- 172.21.15.63:0/4044961385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f783c101d20 0x7f783c113990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f782c00c8c0 tx=0x7f782c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:37.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.791+0000 7f7823fff700 1 -- 172.21.15.63:0/4044961385 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f782c00a980 con 0x7f783c101d20 2024-03-22T07:24:37.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.791+0000 7f7823fff700 1 -- 172.21.15.63:0/4044961385 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f782c00ce60 con 0x7f783c101d20 2024-03-22T07:24:37.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.791+0000 7f7823fff700 1 -- 172.21.15.63:0/4044961385 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f782c0115e0 con 0x7f783c101d20 2024-03-22T07:24:37.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.791+0000 7f7842225700 1 -- 172.21.15.63:0/4044961385 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783c10de00 con 0x7f783c101d20 2024-03-22T07:24:37.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.791+0000 7f7842225700 1 -- 172.21.15.63:0/4044961385 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783c10e370 con 0x7f783c101d20 2024-03-22T07:24:37.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.793+0000 7f7842225700 1 -- 172.21.15.63:0/4044961385 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f783c0fd470 con 0x7f783c101d20 2024-03-22T07:24:37.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.793+0000 7f7823fff700 1 -- 172.21.15.63:0/4044961385 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f782c012400 con 0x7f783c101d20 2024-03-22T07:24:37.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.793+0000 7f7823fff700 1 --2- 172.21.15.63:0/4044961385 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f782406d380 0x7f782406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:37.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.793+0000 7f7823fff700 1 -- 172.21.15.63:0/4044961385 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f782c08c990 con 0x7f783c101d20 2024-03-22T07:24:37.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.794+0000 7f783affd700 1 --2- 172.21.15.63:0/4044961385 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f782406d380 0x7f782406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:37.794 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.794+0000 7f783affd700 1 --2- 172.21.15.63:0/4044961385 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f782406d380 0x7f782406f830 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f783c10ece0 tx=0x7f7830009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:37.798 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:37.798+0000 7f7823fff700 1 -- 172.21.15.63:0/4044961385 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f782c05a020 con 0x7f783c101d20 2024-03-22T07:24:38.018 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.018+0000 7f7842225700 1 -- 172.21.15.63:0/4044961385 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f783c108e00 con 0x7f782406d380 2024-03-22T07:24:38.019 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.019+0000 7f7823fff700 1 -- 172.21.15.63:0/4044961385 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f783c108e00 con 0x7f782406d380 2024-03-22T07:24:38.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.021+0000 7f7842225700 1 -- 172.21.15.63:0/4044961385 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f782406d380 msgr2=0x7f782406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:38.021 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.021+0000 7f7842225700 1 --2- 172.21.15.63:0/4044961385 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f782406d380 0x7f782406f830 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f783c10ece0 tx=0x7f7830009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:38.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.021+0000 7f7842225700 1 -- 172.21.15.63:0/4044961385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f783c101d20 msgr2=0x7f783c113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:38.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.021+0000 7f7842225700 1 --2- 172.21.15.63:0/4044961385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f783c101d20 0x7f783c113990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f782c00c8c0 tx=0x7f782c0044c0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.021+0000 7f7842225700 1 -- 172.21.15.63:0/4044961385 shutdown_connections 2024-03-22T07:24:38.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.021+0000 7f7842225700 1 --2- 172.21.15.63:0/4044961385 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f782406d380 0x7f782406f830 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.021+0000 7f7842225700 1 --2- 172.21.15.63:0/4044961385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f783c101d20 0x7f783c113990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.021+0000 7f7842225700 1 --2- 172.21.15.63:0/4044961385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f783c1026c0 0x7f783c113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.021+0000 7f7842225700 1 -- 172.21.15.63:0/4044961385 >> 172.21.15.63:0/4044961385 conn(0x7f783c0f78b0 msgr2=0x7f783c0662c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:38.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.021+0000 7f7842225700 1 -- 172.21.15.63:0/4044961385 shutdown_connections 2024-03-22T07:24:38.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.021+0000 7f7842225700 1 -- 172.21.15.63:0/4044961385 wait complete. 2024-03-22T07:24:38.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.124+0000 7f1571f4d700 1 -- 172.21.15.63:0/2828706094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f156c101d70 msgr2=0x7f156c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:38.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.124+0000 7f1571f4d700 1 --2- 172.21.15.63:0/2828706094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f156c101d70 0x7f156c102140 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1560004940 tx=0x7f1560009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:38.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.124+0000 7f1571f4d700 1 -- 172.21.15.63:0/2828706094 shutdown_connections 2024-03-22T07:24:38.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.124+0000 7f1571f4d700 1 --2- 172.21.15.63:0/2828706094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f156c102710 0x7f156c0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.124+0000 7f1571f4d700 1 --2- 172.21.15.63:0/2828706094 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f156c101d70 0x7f156c102140 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.124 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.124+0000 7f1571f4d700 1 -- 172.21.15.63:0/2828706094 >> 172.21.15.63:0/2828706094 conn(0x7f156c072bc0 msgr2=0x7f156c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:38.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.124+0000 7f1571f4d700 1 -- 172.21.15.63:0/2828706094 shutdown_connections 2024-03-22T07:24:38.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.124+0000 7f1571f4d700 1 -- 172.21.15.63:0/2828706094 wait complete. 2024-03-22T07:24:38.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.125+0000 7f1571f4d700 1 Processor -- start 2024-03-22T07:24:38.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.125+0000 7f1571f4d700 1 -- start start 2024-03-22T07:24:38.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.125+0000 7f1571f4d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f156c101d70 0x7f156c102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:38.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.125+0000 7f1571f4d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f156c102710 0x7f156c103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:38.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.125+0000 7f1571f4d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c1036a0 con 0x7f156c102710 2024-03-22T07:24:38.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.125+0000 7f1571f4d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c1037e0 con 0x7f156c101d70 2024-03-22T07:24:38.126 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.125+0000 7f156affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f156c102710 0x7f156c103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:38.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.125+0000 7f156affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f156c102710 0x7f156c103110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44592/0 (socket says 172.21.15.63:44592) 2024-03-22T07:24:38.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.125+0000 7f156affd700 1 -- 172.21.15.63:0/2663449677 learned_addr learned my addr 172.21.15.63:0/2663449677 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:24:38.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.125+0000 7f156b7fe700 1 --2- 172.21.15.63:0/2663449677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f156c101d70 0x7f156c102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:38.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.126+0000 7f156affd700 1 -- 172.21.15.63:0/2663449677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f156c101d70 msgr2=0x7f156c102bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:38.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.126+0000 7f156affd700 1 --2- 172.21.15.63:0/2663449677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f156c101d70 0x7f156c102bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.126+0000 7f156affd700 1 -- 172.21.15.63:0/2663449677 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1560005040 con 0x7f156c102710 2024-03-22T07:24:38.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.126+0000 7f156affd700 1 --2- 172.21.15.63:0/2663449677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f156c102710 0x7f156c103110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f155c002a50 tx=0x7f155c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:38.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.126+0000 7f1553fff700 1 -- 172.21.15.63:0/2663449677 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f155c007cc0 con 0x7f156c102710 2024-03-22T07:24:38.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.126+0000 7f1553fff700 1 -- 172.21.15.63:0/2663449677 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f155c00f040 con 0x7f156c102710 2024-03-22T07:24:38.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.126+0000 7f1571f4d700 1 -- 172.21.15.63:0/2663449677 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c119db0 con 0x7f156c102710 2024-03-22T07:24:38.127 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.126+0000 7f1571f4d700 1 -- 172.21.15.63:0/2663449677 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c11a230 con 0x7f156c102710 2024-03-22T07:24:38.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.126+0000 7f1553fff700 1 -- 172.21.15.63:0/2663449677 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f155c00db30 con 0x7f156c102710 2024-03-22T07:24:38.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.127+0000 7f1571f4d700 1 -- 172.21.15.63:0/2663449677 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156c0fd4c0 con 0x7f156c102710 2024-03-22T07:24:38.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.128+0000 7f1553fff700 1 -- 172.21.15.63:0/2663449677 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f155c00dc90 con 0x7f156c102710 2024-03-22T07:24:38.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.128+0000 7f1553fff700 1 --2- 172.21.15.63:0/2663449677 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f155406d380 0x7f155406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:38.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.128+0000 7f1553fff700 1 -- 172.21.15.63:0/2663449677 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f155c08c640 con 0x7f156c102710 2024-03-22T07:24:38.128 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.128+0000 7f156b7fe700 1 --2- 172.21.15.63:0/2663449677 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f155406d380 0x7f155406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:38.129 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.129+0000 7f156b7fe700 1 --2- 172.21.15.63:0/2663449677 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f155406d380 0x7f155406f830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1560004f90 tx=0x7f1560009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:38.133 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.133+0000 7f1553fff700 1 -- 172.21.15.63:0/2663449677 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f155c056120 con 0x7f156c102710 2024-03-22T07:24:38.345 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.344+0000 7f1571f4d700 1 -- 172.21.15.63:0/2663449677 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f156c103970 con 0x7f155406d380 2024-03-22T07:24:38.355 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.355+0000 7f1553fff700 1 -- 172.21.15.63:0/2663449677 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f156c103970 con 0x7f155406d380 2024-03-22T07:24:38.355 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:24:38.355 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (5m) 84s ago 16m 26.3M - 0.25.0 c8568f914cd2 06ef21da90e5 2024-03-22T07:24:38.355 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (3m) 84s ago 16m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 224c291ab774 2024-03-22T07:24:38.355 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (3m) 17s ago 15m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 7eabb7d42631 2024-03-22T07:24:38.355 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (5m) 84s ago 16m 86.0M - 9.4.12 a0ea5e3c46d9 9015fae0d5e9 2024-03-22T07:24:38.355 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (11m) 84s ago 11m 74.1M - 17.2.7-726-g9c4e0c34 837124da804f 6a2b52d49228 2024-03-22T07:24:38.355 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (10m) 84s ago 10m 123M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:24:38.355 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (10m) 17s ago 10m 63.0M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:24:38.356 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (11m) 17s ago 11m 129M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:24:38.356 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:8443,9283,8765 running (7m) 84s ago 17m 506M - 19.0.0-1667-gdb0330b1 4f325f8440e9 e49468f39991 2024-03-22T07:24:38.356 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283,8765 running (6m) 17s ago 15m 450M - 19.0.0-1667-gdb0330b1 4f325f8440e9 13c3f034ebb2 2024-03-22T07:24:38.356 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (4m) 84s ago 18m 47.7M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ab96538a07c1 2024-03-22T07:24:38.356 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (3m) 17s ago 15m 38.5M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 01a26e1fb1a1 2024-03-22T07:24:38.356 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (6m) 84s ago 16m 16.0M - 1.5.0 0da6a335fe13 0c825ee2a196 2024-03-22T07:24:38.356 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (6m) 17s ago 15m 16.6M - 1.5.0 0da6a335fe13 12edb2f35498 2024-03-22T07:24:38.356 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (2m) 84s ago 14m 102M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 fdbc8b34ab78 2024-03-22T07:24:38.356 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (2m) 84s ago 14m 98.3M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 5200fd893046 2024-03-22T07:24:38.356 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (90s) 84s ago 13m 11.7M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 918925ee69f3 2024-03-22T07:24:38.356 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (54s) 17s ago 13m 99.2M 2581M 19.0.0-1667-gdb0330b1 4f325f8440e9 652a1e62339b 2024-03-22T07:24:38.356 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (20s) 17s ago 13m 12.7M 2581M 19.0.0-1667-gdb0330b1 4f325f8440e9 d6340a0e1c6c 2024-03-22T07:24:38.356 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (12m) 17s ago 12m 237M 2581M 17.2.7-726-g9c4e0c34 837124da804f 08e82e141487 2024-03-22T07:24:38.356 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (5m) 84s ago 16m 61.1M - 2.43.0 a07b618ecd1d 1d89b928f6e7 2024-03-22T07:24:38.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.357+0000 7f1571f4d700 1 -- 172.21.15.63:0/2663449677 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f155406d380 msgr2=0x7f155406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:38.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.357+0000 7f1571f4d700 1 --2- 172.21.15.63:0/2663449677 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f155406d380 0x7f155406f830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1560004f90 tx=0x7f1560009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:38.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.357+0000 7f1571f4d700 1 -- 172.21.15.63:0/2663449677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f156c102710 msgr2=0x7f156c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:38.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.357+0000 7f1571f4d700 1 --2- 172.21.15.63:0/2663449677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f156c102710 0x7f156c103110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f155c002a50 tx=0x7f155c0046b0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.357+0000 7f1571f4d700 1 -- 172.21.15.63:0/2663449677 shutdown_connections 2024-03-22T07:24:38.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.357+0000 7f1571f4d700 1 --2- 172.21.15.63:0/2663449677 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f155406d380 0x7f155406f830 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.357 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.357+0000 7f1571f4d700 1 --2- 172.21.15.63:0/2663449677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f156c101d70 0x7f156c102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.357+0000 7f1571f4d700 1 --2- 172.21.15.63:0/2663449677 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f156c102710 0x7f156c103110 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.357+0000 7f1571f4d700 1 -- 172.21.15.63:0/2663449677 >> 172.21.15.63:0/2663449677 conn(0x7f156c072bc0 msgr2=0x7f156c0658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:38.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.357+0000 7f1571f4d700 1 -- 172.21.15.63:0/2663449677 shutdown_connections 2024-03-22T07:24:38.358 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.357+0000 7f1571f4d700 1 -- 172.21.15.63:0/2663449677 wait complete. 2024-03-22T07:24:38.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.458+0000 7f237e874700 1 -- 172.21.15.63:0/1761111733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2378070bf0 msgr2=0x7f2378109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:38.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.458+0000 7f237e874700 1 --2- 172.21.15.63:0/1761111733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2378070bf0 0x7f2378109190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f236c004940 tx=0x7f236c009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:38.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.458+0000 7f237e874700 1 -- 172.21.15.63:0/1761111733 shutdown_connections 2024-03-22T07:24:38.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.458+0000 7f237e874700 1 --2- 172.21.15.63:0/1761111733 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2378070bf0 0x7f2378109190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.459 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.458+0000 7f237e874700 1 --2- 172.21.15.63:0/1761111733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23780702e0 0x7f23780706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.458+0000 7f237e874700 1 -- 172.21.15.63:0/1761111733 >> 172.21.15.63:0/1761111733 conn(0x7f23780f89b0 msgr2=0x7f23780fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:38.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.458+0000 7f237e874700 1 -- 172.21.15.63:0/1761111733 shutdown_connections 2024-03-22T07:24:38.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.458+0000 7f237e874700 1 -- 172.21.15.63:0/1761111733 wait complete. 2024-03-22T07:24:38.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.459+0000 7f237e874700 1 Processor -- start 2024-03-22T07:24:38.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.459+0000 7f237e874700 1 -- start start 2024-03-22T07:24:38.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.459+0000 7f237e874700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23780702e0 0x7f2378115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:38.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.459+0000 7f237e874700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2378070bf0 0x7f2378116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:38.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.459+0000 7f237e874700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2378116780 con 0x7f2378070bf0 2024-03-22T07:24:38.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.459+0000 7f237e874700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237810fd60 con 0x7f23780702e0 2024-03-22T07:24:38.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.459+0000 7f23777fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2378070bf0 0x7f2378116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:38.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.459+0000 7f23777fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2378070bf0 0x7f2378116130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44624/0 (socket says 172.21.15.63:44624) 2024-03-22T07:24:38.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.459+0000 7f23777fe700 1 -- 172.21.15.63:0/3986682761 learned_addr learned my addr 172.21.15.63:0/3986682761 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:24:38.460 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.459+0000 7f2377fff700 1 --2- 172.21.15.63:0/3986682761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23780702e0 0x7f2378115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:38.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.460+0000 7f23777fe700 1 -- 172.21.15.63:0/3986682761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23780702e0 msgr2=0x7f2378115bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:38.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.460+0000 7f23777fe700 1 --2- 172.21.15.63:0/3986682761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23780702e0 0x7f2378115bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.460+0000 7f23777fe700 1 -- 172.21.15.63:0/3986682761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c005040 con 0x7f2378070bf0 2024-03-22T07:24:38.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.460+0000 7f23777fe700 1 --2- 172.21.15.63:0/3986682761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2378070bf0 0x7f2378116130 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f236c00b3b0 tx=0x7f236c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:38.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.460+0000 7f2374ff9700 1 -- 172.21.15.63:0/3986682761 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f236c00e040 con 0x7f2378070bf0 2024-03-22T07:24:38.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.460+0000 7f237e874700 1 -- 172.21.15.63:0/3986682761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237810ffe0 con 0x7f2378070bf0 2024-03-22T07:24:38.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.460+0000 7f2374ff9700 1 -- 172.21.15.63:0/3986682761 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f236c013420 con 0x7f2378070bf0 2024-03-22T07:24:38.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.460+0000 7f2374ff9700 1 -- 172.21.15.63:0/3986682761 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f236c01c630 con 0x7f2378070bf0 2024-03-22T07:24:38.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.460+0000 7f237e874700 1 -- 172.21.15.63:0/3986682761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2378110520 con 0x7f2378070bf0 2024-03-22T07:24:38.462 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.461+0000 7f237e874700 1 -- 172.21.15.63:0/3986682761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2378071a20 con 0x7f2378070bf0 2024-03-22T07:24:38.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.461+0000 7f2374ff9700 1 -- 172.21.15.63:0/3986682761 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f236c00baf0 con 0x7f2378070bf0 2024-03-22T07:24:38.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.462+0000 7f2374ff9700 1 --2- 172.21.15.63:0/3986682761 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f236006d380 0x7f236006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:38.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.462+0000 7f2374ff9700 1 -- 172.21.15.63:0/3986682761 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f236c08d9d0 con 0x7f2378070bf0 2024-03-22T07:24:38.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.462+0000 7f2377fff700 1 --2- 172.21.15.63:0/3986682761 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f236006d380 0x7f236006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:38.463 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.462+0000 7f2377fff700 1 --2- 172.21.15.63:0/3986682761 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f236006d380 0x7f236006f830 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2368004f60 tx=0x7f236800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:38.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.467+0000 7f2374ff9700 1 -- 172.21.15.63:0/3986682761 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f236c05b100 con 0x7f2378070bf0 2024-03-22T07:24:38.752 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.751+0000 7f237e874700 1 -- 172.21.15.63:0/3986682761 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f237804c710 con 0x7f2378070bf0 2024-03-22T07:24:38.752 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:38 smithi063 ceph-mon[112784]: pgmap v151: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-03-22T07:24:38.752 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:38 smithi063 ceph-mon[112784]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:24:38.752 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:38 smithi063 ceph-mon[112784]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:24:38.752 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:38 smithi063 ceph-mon[112784]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:24:38.753 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.752+0000 7f2374ff9700 1 -- 172.21.15.63:0/3986682761 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f236c05ac90 con 0x7f2378070bf0 2024-03-22T07:24:38.753 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:24:38.753 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:24:38.753 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:24:38.753 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:24:38.753 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:24:38.753 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:24:38.753 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:24:38.753 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:24:38.753 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T07:24:38.753 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 5 2024-03-22T07:24:38.753 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:24:38.754 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:24:38.754 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 4 2024-03-22T07:24:38.754 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:24:38.754 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:24:38.754 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 5, 2024-03-22T07:24:38.754 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 9 2024-03-22T07:24:38.754 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:24:38.754 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:24:38.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.755+0000 7f237e874700 1 -- 172.21.15.63:0/3986682761 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f236006d380 msgr2=0x7f236006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:38.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.755+0000 7f237e874700 1 --2- 172.21.15.63:0/3986682761 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f236006d380 0x7f236006f830 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2368004f60 tx=0x7f236800b040 comp rx=0 tx=0).stop 2024-03-22T07:24:38.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.755+0000 7f237e874700 1 -- 172.21.15.63:0/3986682761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2378070bf0 msgr2=0x7f2378116130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:38.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.755+0000 7f237e874700 1 --2- 172.21.15.63:0/3986682761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2378070bf0 0x7f2378116130 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f236c00b3b0 tx=0x7f236c009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:38.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.755+0000 7f237e874700 1 -- 172.21.15.63:0/3986682761 shutdown_connections 2024-03-22T07:24:38.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.755+0000 7f237e874700 1 --2- 172.21.15.63:0/3986682761 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f236006d380 0x7f236006f830 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.755+0000 7f237e874700 1 --2- 172.21.15.63:0/3986682761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f23780702e0 0x7f2378115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.755+0000 7f237e874700 1 --2- 172.21.15.63:0/3986682761 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2378070bf0 0x7f2378116130 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.755+0000 7f237e874700 1 -- 172.21.15.63:0/3986682761 >> 172.21.15.63:0/3986682761 conn(0x7f23780f89b0 msgr2=0x7f23780f9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:38.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.755+0000 7f237e874700 1 -- 172.21.15.63:0/3986682761 shutdown_connections 2024-03-22T07:24:38.755 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.755+0000 7f237e874700 1 -- 172.21.15.63:0/3986682761 wait complete. 2024-03-22T07:24:38.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:38 smithi069 ceph-mon[83631]: pgmap v151: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-03-22T07:24:38.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:38 smithi069 ceph-mon[83631]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:24:38.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:38 smithi069 ceph-mon[83631]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:24:38.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:38 smithi069 ceph-mon[83631]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:24:38.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.856+0000 7f7b714f7700 1 -- 172.21.15.63:0/732144773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b6c101d30 msgr2=0x7f7b6c102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:38.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.856+0000 7f7b714f7700 1 --2- 172.21.15.63:0/732144773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b6c101d30 0x7f7b6c102100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009ce0 tx=0x7f7b64004d10 comp rx=0 tx=0).stop 2024-03-22T07:24:38.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.857+0000 7f7b714f7700 1 -- 172.21.15.63:0/732144773 shutdown_connections 2024-03-22T07:24:38.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.857+0000 7f7b714f7700 1 --2- 172.21.15.63:0/732144773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b6c1026d0 0x7f7b6c0fbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.857+0000 7f7b714f7700 1 --2- 172.21.15.63:0/732144773 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b6c101d30 0x7f7b6c102100 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.857+0000 7f7b714f7700 1 -- 172.21.15.63:0/732144773 >> 172.21.15.63:0/732144773 conn(0x7f7b6c0f7860 msgr2=0x7f7b6c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:38.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.857+0000 7f7b714f7700 1 -- 172.21.15.63:0/732144773 shutdown_connections 2024-03-22T07:24:38.858 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.857+0000 7f7b714f7700 1 -- 172.21.15.63:0/732144773 wait complete. 2024-03-22T07:24:38.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.857+0000 7f7b714f7700 1 Processor -- start 2024-03-22T07:24:38.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.857+0000 7f7b714f7700 1 -- start start 2024-03-22T07:24:38.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.858+0000 7f7b714f7700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b6c101d30 0x7f7b6c113bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:38.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.858+0000 7f7b714f7700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b6c1026d0 0x7f7b6c114100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:38.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.858+0000 7f7b714f7700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c114750 con 0x7f7b6c101d30 2024-03-22T07:24:38.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.858+0000 7f7b714f7700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c10dd70 con 0x7f7b6c1026d0 2024-03-22T07:24:38.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.858+0000 7f7b6affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b6c101d30 0x7f7b6c113bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:38.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.858+0000 7f7b6affd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b6c101d30 0x7f7b6c113bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44638/0 (socket says 172.21.15.63:44638) 2024-03-22T07:24:38.861 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.858+0000 7f7b6affd700 1 -- 172.21.15.63:0/1780819068 learned_addr learned my addr 172.21.15.63:0/1780819068 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:24:38.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.858+0000 7f7b6a7fc700 1 --2- 172.21.15.63:0/1780819068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b6c1026d0 0x7f7b6c114100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:38.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.858+0000 7f7b6affd700 1 -- 172.21.15.63:0/1780819068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b6c1026d0 msgr2=0x7f7b6c114100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:38.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.858+0000 7f7b6affd700 1 --2- 172.21.15.63:0/1780819068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b6c1026d0 0x7f7b6c114100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:38.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.858+0000 7f7b6affd700 1 -- 172.21.15.63:0/1780819068 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b64009940 con 0x7f7b6c101d30 2024-03-22T07:24:38.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.858+0000 7f7b6a7fc700 1 --2- 172.21.15.63:0/1780819068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b6c1026d0 0x7f7b6c114100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:24:38.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.858+0000 7f7b6affd700 1 --2- 172.21.15.63:0/1780819068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b6c101d30 0x7f7b6c113bc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7b64003400 tx=0x7f7b64004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:38.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.858+0000 7f7b637fe700 1 -- 172.21.15.63:0/1780819068 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b6400e040 con 0x7f7b6c101d30 2024-03-22T07:24:38.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.858+0000 7f7b637fe700 1 -- 172.21.15.63:0/1780819068 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b64004610 con 0x7f7b6c101d30 2024-03-22T07:24:38.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.858+0000 7f7b714f7700 1 -- 172.21.15.63:0/1780819068 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6c10dff0 con 0x7f7b6c101d30 2024-03-22T07:24:38.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.858+0000 7f7b637fe700 1 -- 172.21.15.63:0/1780819068 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b6401c6d0 con 0x7f7b6c101d30 2024-03-22T07:24:38.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.858+0000 7f7b714f7700 1 -- 172.21.15.63:0/1780819068 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6c10e530 con 0x7f7b6c101d30 2024-03-22T07:24:38.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.860+0000 7f7b714f7700 1 -- 172.21.15.63:0/1780819068 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b6c0fd480 con 0x7f7b6c101d30 2024-03-22T07:24:38.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.860+0000 7f7b637fe700 1 -- 172.21.15.63:0/1780819068 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f7b64012410 con 0x7f7b6c101d30 2024-03-22T07:24:38.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.860+0000 7f7b637fe700 1 --2- 172.21.15.63:0/1780819068 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7b5806d380 0x7f7b5806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:38.862 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.860+0000 7f7b637fe700 1 -- 172.21.15.63:0/1780819068 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7b6408e360 con 0x7f7b6c101d30 2024-03-22T07:24:38.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.860+0000 7f7b6a7fc700 1 --2- 172.21.15.63:0/1780819068 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7b5806d380 0x7f7b5806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:38.863 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.861+0000 7f7b6a7fc700 1 --2- 172.21.15.63:0/1780819068 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7b5806d380 0x7f7b5806f830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009fd0 tx=0x7f7b54009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:38.865 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:38.865+0000 7f7b637fe700 1 -- 172.21.15.63:0/1780819068 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7b64057e40 con 0x7f7b6c101d30 2024-03-22T07:24:39.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.111+0000 7f7b714f7700 1 -- 172.21.15.63:0/1780819068 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7b6c10f250 con 0x7f7b6c101d30 2024-03-22T07:24:39.112 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.112+0000 7f7b637fe700 1 -- 172.21.15.63:0/1780819068 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+1868 (secure 0 0 0) 0x7f7b64004e10 con 0x7f7b6c101d30 2024-03-22T07:24:39.112 INFO:teuthology.orchestra.run.smithi063.stdout:e24 2024-03-22T07:24:39.112 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:24:39.112 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:24:39.112 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-03-22T07:24:39.112 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:24:39.112 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:24:39.112 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-03-22T07:24:39.113 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 24 2024-03-22T07:24:39.113 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-22T07:24:39.113 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-03-22T07:13:29.602905+0000 2024-03-22T07:24:39.113 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-03-22T07:20:24.253589+0000 2024-03-22T07:24:39.113 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-03-22T07:24:39.113 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-03-22T07:24:39.113 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-03-22T07:24:39.113 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-03-22T07:24:39.113 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-03-22T07:24:39.113 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-03-22T07:24:39.113 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-03-22T07:24:39.113 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-03-22T07:24:39.113 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 47 2024-03-22T07:24:39.113 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:24:39.113 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 2 2024-03-22T07:24:39.114 INFO:teuthology.orchestra.run.smithi063.stdout:in 0,1 2024-03-22T07:24:39.114 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=14480,1=24285} 2024-03-22T07:24:39.114 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-03-22T07:24:39.114 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-03-22T07:24:39.114 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-03-22T07:24:39.114 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-03-22T07:24:39.114 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-03-22T07:24:39.114 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-03-22T07:24:39.114 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-03-22T07:24:39.114 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-03-22T07:24:39.114 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-03-22T07:24:39.114 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.xeelpk{0:14480} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.63:6828/1373266298,v1:172.21.15.63:6829/1373266298] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:24:39.114 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.oytapz{0:14492} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/3211595802,v1:172.21.15.63:6827/3211595802] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:24:39.114 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.wwtsjn{1:24285} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.69:6824/3798247699,v1:172.21.15.69:6825/3798247699] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:24:39.115 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.cijxah{1:24301} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.69:6826/3380719311,v1:172.21.15.69:6827/3380719311] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:24:39.115 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:24:39.115 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:24:39.115 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.114+0000 7f7b714f7700 1 -- 172.21.15.63:0/1780819068 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7b5806d380 msgr2=0x7f7b5806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:39.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.114+0000 7f7b714f7700 1 --2- 172.21.15.63:0/1780819068 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7b5806d380 0x7f7b5806f830 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009fd0 tx=0x7f7b54009ee0 comp rx=0 tx=0).stop 2024-03-22T07:24:39.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.114+0000 7f7b714f7700 1 -- 172.21.15.63:0/1780819068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b6c101d30 msgr2=0x7f7b6c113bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:39.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.114+0000 7f7b714f7700 1 --2- 172.21.15.63:0/1780819068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b6c101d30 0x7f7b6c113bc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7b64003400 tx=0x7f7b64004d10 comp rx=0 tx=0).stop 2024-03-22T07:24:39.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.114+0000 7f7b714f7700 1 -- 172.21.15.63:0/1780819068 shutdown_connections 2024-03-22T07:24:39.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.114+0000 7f7b714f7700 1 --2- 172.21.15.63:0/1780819068 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7b5806d380 0x7f7b5806f830 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:39.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.114+0000 7f7b714f7700 1 --2- 172.21.15.63:0/1780819068 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7b6c101d30 0x7f7b6c113bc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:39.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.114+0000 7f7b714f7700 1 --2- 172.21.15.63:0/1780819068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b6c1026d0 0x7f7b6c114100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:39.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.114+0000 7f7b714f7700 1 -- 172.21.15.63:0/1780819068 >> 172.21.15.63:0/1780819068 conn(0x7f7b6c0f7860 msgr2=0x7f7b6c104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:39.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.114+0000 7f7b714f7700 1 -- 172.21.15.63:0/1780819068 shutdown_connections 2024-03-22T07:24:39.116 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.114+0000 7f7b714f7700 1 -- 172.21.15.63:0/1780819068 wait complete. 2024-03-22T07:24:39.116 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 24 2024-03-22T07:24:39.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.217+0000 7f9c0ae6a700 1 -- 172.21.15.63:0/232478366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c041037b0 msgr2=0x7f9c040fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:39.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.217+0000 7f9c0ae6a700 1 --2- 172.21.15.63:0/232478366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c041037b0 0x7f9c040fce20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0004940 tx=0x7f9bf0009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:39.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.218+0000 7f9c0ae6a700 1 -- 172.21.15.63:0/232478366 shutdown_connections 2024-03-22T07:24:39.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.218+0000 7f9c0ae6a700 1 --2- 172.21.15.63:0/232478366 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c041037b0 0x7f9c040fce20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:39.218 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.218+0000 7f9c0ae6a700 1 --2- 172.21.15.63:0/232478366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c04102e10 0x7f9c041031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:39.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.218+0000 7f9c0ae6a700 1 -- 172.21.15.63:0/232478366 >> 172.21.15.63:0/232478366 conn(0x7f9c040f8980 msgr2=0x7f9c040fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:39.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.218+0000 7f9c0ae6a700 1 -- 172.21.15.63:0/232478366 shutdown_connections 2024-03-22T07:24:39.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.218+0000 7f9c0ae6a700 1 -- 172.21.15.63:0/232478366 wait complete. 2024-03-22T07:24:39.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.218+0000 7f9c0ae6a700 1 Processor -- start 2024-03-22T07:24:39.219 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.218+0000 7f9c0ae6a700 1 -- start start 2024-03-22T07:24:39.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.218+0000 7f9c0ae6a700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c04102e10 0x7f9c04113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:39.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.218+0000 7f9c0ae6a700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c041037b0 0x7f9c041140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:39.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.218+0000 7f9c0ae6a700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c04114710 con 0x7f9c04102e10 2024-03-22T07:24:39.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.218+0000 7f9c0ae6a700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0410db00 con 0x7f9c041037b0 2024-03-22T07:24:39.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.219+0000 7f9c08c06700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c04102e10 0x7f9c04113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:39.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.219+0000 7f9c08c06700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c04102e10 0x7f9c04113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44658/0 (socket says 172.21.15.63:44658) 2024-03-22T07:24:39.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.219+0000 7f9c08c06700 1 -- 172.21.15.63:0/286891399 learned_addr learned my addr 172.21.15.63:0/286891399 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:24:39.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.219+0000 7f9c03fff700 1 --2- 172.21.15.63:0/286891399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c041037b0 0x7f9c041140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:39.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.219+0000 7f9c08c06700 1 -- 172.21.15.63:0/286891399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c041037b0 msgr2=0x7f9c041140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:39.220 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.219+0000 7f9c08c06700 1 --2- 172.21.15.63:0/286891399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c041037b0 0x7f9c041140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:39.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.219+0000 7f9c08c06700 1 -- 172.21.15.63:0/286891399 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf0005040 con 0x7f9c04102e10 2024-03-22T07:24:39.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.219+0000 7f9c08c06700 1 --2- 172.21.15.63:0/286891399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c04102e10 0x7f9c04113b80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8002a50 tx=0x7f9bf80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:39.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.219+0000 7f9c017fa700 1 -- 172.21.15.63:0/286891399 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bf8007cc0 con 0x7f9c04102e10 2024-03-22T07:24:39.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.219+0000 7f9c017fa700 1 -- 172.21.15.63:0/286891399 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bf800f040 con 0x7f9c04102e10 2024-03-22T07:24:39.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.219+0000 7f9c0ae6a700 1 -- 172.21.15.63:0/286891399 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c0410dde0 con 0x7f9c04102e10 2024-03-22T07:24:39.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.219+0000 7f9c017fa700 1 -- 172.21.15.63:0/286891399 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bf800da30 con 0x7f9c04102e10 2024-03-22T07:24:39.221 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.219+0000 7f9c0ae6a700 1 -- 172.21.15.63:0/286891399 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0410e370 con 0x7f9c04102e10 2024-03-22T07:24:39.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.220+0000 7f9c0ae6a700 1 -- 172.21.15.63:0/286891399 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c041052e0 con 0x7f9c04102e10 2024-03-22T07:24:39.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.221+0000 7f9c017fa700 1 -- 172.21.15.63:0/286891399 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f9bf800db90 con 0x7f9c04102e10 2024-03-22T07:24:39.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.221+0000 7f9c017fa700 1 --2- 172.21.15.63:0/286891399 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9bf406d380 0x7f9bf406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:39.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.221+0000 7f9c017fa700 1 -- 172.21.15.63:0/286891399 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9bf808c650 con 0x7f9c04102e10 2024-03-22T07:24:39.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.221+0000 7f9c03fff700 1 --2- 172.21.15.63:0/286891399 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9bf406d380 0x7f9bf406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:39.222 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.222+0000 7f9c03fff700 1 --2- 172.21.15.63:0/286891399 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9bf406d380 0x7f9bf406f830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0004890 tx=0x7f9bf0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:39.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.226+0000 7f9c017fa700 1 -- 172.21.15.63:0/286891399 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9bf8059c60 con 0x7f9c04102e10 2024-03-22T07:24:39.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.447+0000 7f9c0ae6a700 1 -- 172.21.15.63:0/286891399 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c0410f1d0 con 0x7f9bf406d380 2024-03-22T07:24:39.448 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.448+0000 7f9c017fa700 1 -- 172.21.15.63:0/286891399 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9c0410f1d0 con 0x7f9bf406d380 2024-03-22T07:24:39.448 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:24:39.448 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:24:39.448 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:24:39.449 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:24:39.449 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-03-22T07:24:39.449 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-03-22T07:24:39.449 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr", 2024-03-22T07:24:39.449 INFO:teuthology.orchestra.run.smithi063.stdout: "mon" 2024-03-22T07:24:39.449 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-03-22T07:24:39.449 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "11/21 daemons upgraded", 2024-03-22T07:24:39.449 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading osd daemons", 2024-03-22T07:24:39.449 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:24:39.449 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:24:39.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.450+0000 7f9c0ae6a700 1 -- 172.21.15.63:0/286891399 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9bf406d380 msgr2=0x7f9bf406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:39.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.450+0000 7f9c0ae6a700 1 --2- 172.21.15.63:0/286891399 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9bf406d380 0x7f9bf406f830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0004890 tx=0x7f9bf0009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:39.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.450+0000 7f9c0ae6a700 1 -- 172.21.15.63:0/286891399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c04102e10 msgr2=0x7f9c04113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:39.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.450+0000 7f9c0ae6a700 1 --2- 172.21.15.63:0/286891399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c04102e10 0x7f9c04113b80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8002a50 tx=0x7f9bf80046b0 comp rx=0 tx=0).stop 2024-03-22T07:24:39.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.450+0000 7f9c0ae6a700 1 -- 172.21.15.63:0/286891399 shutdown_connections 2024-03-22T07:24:39.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.450+0000 7f9c0ae6a700 1 --2- 172.21.15.63:0/286891399 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9bf406d380 0x7f9bf406f830 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:39.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.450+0000 7f9c0ae6a700 1 --2- 172.21.15.63:0/286891399 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9c04102e10 0x7f9c04113b80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:39.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.450+0000 7f9c0ae6a700 1 --2- 172.21.15.63:0/286891399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c041037b0 0x7f9c041140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:39.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.450+0000 7f9c0ae6a700 1 -- 172.21.15.63:0/286891399 >> 172.21.15.63:0/286891399 conn(0x7f9c040f8980 msgr2=0x7f9c041021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:39.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.450+0000 7f9c0ae6a700 1 -- 172.21.15.63:0/286891399 shutdown_connections 2024-03-22T07:24:39.451 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.450+0000 7f9c0ae6a700 1 -- 172.21.15.63:0/286891399 wait complete. 2024-03-22T07:24:39.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.553+0000 7fbc620b1700 1 -- 172.21.15.63:0/4099203248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5c101d70 msgr2=0x7fbc5c102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:39.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.553+0000 7fbc620b1700 1 --2- 172.21.15.63:0/4099203248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5c101d70 0x7fbc5c102140 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbc50004940 tx=0x7fbc50009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:39.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.553+0000 7fbc620b1700 1 -- 172.21.15.63:0/4099203248 shutdown_connections 2024-03-22T07:24:39.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.553+0000 7fbc620b1700 1 --2- 172.21.15.63:0/4099203248 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc5c102710 0x7fbc5c0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:39.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.553+0000 7fbc620b1700 1 --2- 172.21.15.63:0/4099203248 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5c101d70 0x7fbc5c102140 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:39.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.553+0000 7fbc620b1700 1 -- 172.21.15.63:0/4099203248 >> 172.21.15.63:0/4099203248 conn(0x7fbc5c072bc0 msgr2=0x7fbc5c072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:39.554 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.554+0000 7fbc620b1700 1 -- 172.21.15.63:0/4099203248 shutdown_connections 2024-03-22T07:24:39.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.554+0000 7fbc620b1700 1 -- 172.21.15.63:0/4099203248 wait complete. 2024-03-22T07:24:39.555 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.554+0000 7fbc620b1700 1 Processor -- start 2024-03-22T07:24:39.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.554+0000 7fbc620b1700 1 -- start start 2024-03-22T07:24:39.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.554+0000 7fbc620b1700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5c101d70 0x7fbc5c112a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:39.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.554+0000 7fbc620b1700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc5c102710 0x7fbc5c112f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:39.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.554+0000 7fbc620b1700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c1135d0 con 0x7fbc5c101d70 2024-03-22T07:24:39.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.554+0000 7fbc620b1700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c10c9c0 con 0x7fbc5c102710 2024-03-22T07:24:39.557 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.555+0000 7fbc5b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5c101d70 0x7fbc5c112a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:39.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.555+0000 7fbc5b7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5c101d70 0x7fbc5c112a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44670/0 (socket says 172.21.15.63:44670) 2024-03-22T07:24:39.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.555+0000 7fbc5b7fe700 1 -- 172.21.15.63:0/3990629228 learned_addr learned my addr 172.21.15.63:0/3990629228 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:24:39.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.555+0000 7fbc5affd700 1 --2- 172.21.15.63:0/3990629228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc5c102710 0x7fbc5c112f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:39.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.555+0000 7fbc5b7fe700 1 -- 172.21.15.63:0/3990629228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc5c102710 msgr2=0x7fbc5c112f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:39.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.555+0000 7fbc5b7fe700 1 --2- 172.21.15.63:0/3990629228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc5c102710 0x7fbc5c112f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:39.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.555+0000 7fbc5b7fe700 1 -- 172.21.15.63:0/3990629228 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc50005040 con 0x7fbc5c101d70 2024-03-22T07:24:39.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.555+0000 7fbc5b7fe700 1 --2- 172.21.15.63:0/3990629228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5c101d70 0x7fbc5c112a40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000b8b0 tx=0x7fbc50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:39.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.555+0000 7fbc43fff700 1 -- 172.21.15.63:0/3990629228 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc5000e040 con 0x7fbc5c101d70 2024-03-22T07:24:39.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.555+0000 7fbc620b1700 1 -- 172.21.15.63:0/3990629228 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5c10cc40 con 0x7fbc5c101d70 2024-03-22T07:24:39.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.555+0000 7fbc43fff700 1 -- 172.21.15.63:0/3990629228 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc50004290 con 0x7fbc5c101d70 2024-03-22T07:24:39.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.555+0000 7fbc43fff700 1 -- 172.21.15.63:0/3990629228 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc5001c3f0 con 0x7fbc5c101d70 2024-03-22T07:24:39.558 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.555+0000 7fbc620b1700 1 -- 172.21.15.63:0/3990629228 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5c10d200 con 0x7fbc5c101d70 2024-03-22T07:24:39.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.556+0000 7fbc620b1700 1 -- 172.21.15.63:0/3990629228 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5c0ff080 con 0x7fbc5c101d70 2024-03-22T07:24:39.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.557+0000 7fbc43fff700 1 -- 172.21.15.63:0/3990629228 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fbc50013420 con 0x7fbc5c101d70 2024-03-22T07:24:39.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.557+0000 7fbc43fff700 1 --2- 172.21.15.63:0/3990629228 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbc4406d380 0x7fbc4406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:24:39.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.557+0000 7fbc43fff700 1 -- 172.21.15.63:0/3990629228 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbc5008dae0 con 0x7fbc5c101d70 2024-03-22T07:24:39.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.557+0000 7fbc5affd700 1 --2- 172.21.15.63:0/3990629228 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbc4406d380 0x7fbc4406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:24:39.559 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.558+0000 7fbc5affd700 1 --2- 172.21.15.63:0/3990629228 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbc4406d380 0x7fbc4406f830 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c10db80 tx=0x7fbc4c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:24:39.562 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.562+0000 7fbc43fff700 1 -- 172.21.15.63:0/3990629228 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbc5005b170 con 0x7fbc5c101d70 2024-03-22T07:24:39.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.849+0000 7fbc620b1700 1 -- 172.21.15.63:0/3990629228 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbc5c10e090 con 0x7fbc5c101d70 2024-03-22T07:24:39.850 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:39 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3986682761' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:24:39.850 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:39 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/1780819068' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:24:39.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.850+0000 7fbc43fff700 1 -- 172.21.15.63:0/3990629228 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fbc5005ad00 con 0x7fbc5c101d70 2024-03-22T07:24:39.851 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-03-22T07:24:39.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.852+0000 7fbc620b1700 1 -- 172.21.15.63:0/3990629228 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbc4406d380 msgr2=0x7fbc4406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:39.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.852+0000 7fbc620b1700 1 --2- 172.21.15.63:0/3990629228 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbc4406d380 0x7fbc4406f830 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c10db80 tx=0x7fbc4c009040 comp rx=0 tx=0).stop 2024-03-22T07:24:39.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.852+0000 7fbc620b1700 1 -- 172.21.15.63:0/3990629228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5c101d70 msgr2=0x7fbc5c112a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:24:39.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.852+0000 7fbc620b1700 1 --2- 172.21.15.63:0/3990629228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5c101d70 0x7fbc5c112a40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000b8b0 tx=0x7fbc50009e30 comp rx=0 tx=0).stop 2024-03-22T07:24:39.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.852+0000 7fbc620b1700 1 -- 172.21.15.63:0/3990629228 shutdown_connections 2024-03-22T07:24:39.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.852+0000 7fbc620b1700 1 --2- 172.21.15.63:0/3990629228 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbc4406d380 0x7fbc4406f830 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:39.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.852+0000 7fbc620b1700 1 --2- 172.21.15.63:0/3990629228 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbc5c101d70 0x7fbc5c112a40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:39.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.852+0000 7fbc620b1700 1 --2- 172.21.15.63:0/3990629228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc5c102710 0x7fbc5c112f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:24:39.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.852+0000 7fbc620b1700 1 -- 172.21.15.63:0/3990629228 >> 172.21.15.63:0/3990629228 conn(0x7fbc5c072bc0 msgr2=0x7fbc5c0fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:24:39.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.852+0000 7fbc620b1700 1 -- 172.21.15.63:0/3990629228 shutdown_connections 2024-03-22T07:24:39.853 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:24:39.852+0000 7fbc620b1700 1 -- 172.21.15.63:0/3990629228 wait complete. 2024-03-22T07:24:40.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:39 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3986682761' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:24:40.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:39 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/1780819068' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:24:40.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:40 smithi063 ceph-mon[112784]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:24:40.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:40 smithi063 ceph-mon[112784]: pgmap v152: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:24:40.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:40 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3990629228' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:24:41.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:40 smithi069 ceph-mon[83631]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:24:41.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:40 smithi069 ceph-mon[83631]: pgmap v152: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:24:41.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:40 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3990629228' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:24:41.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:41 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T07:24:41.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:41 smithi069 ceph-mon[83631]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T07:24:41.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:41 smithi069 ceph-mon[83631]: Upgrade: osd.5 is safe to restart 2024-03-22T07:24:41.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:41 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T07:24:41.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:41 smithi063 ceph-mon[112784]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-22T07:24:41.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:41 smithi063 ceph-mon[112784]: Upgrade: osd.5 is safe to restart 2024-03-22T07:24:43.485 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:43 smithi069 ceph-mon[83631]: pgmap v153: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-22T07:24:43.485 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:43 smithi069 ceph-mon[83631]: Upgrade: Updating osd.5 2024-03-22T07:24:43.485 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:43 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:43.485 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:43 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T07:24:43.485 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:43 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:24:43.486 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:43 smithi069 ceph-mon[83631]: Deploying daemon osd.5 on smithi069 2024-03-22T07:24:43.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:43 smithi063 ceph-mon[112784]: pgmap v153: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-03-22T07:24:43.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:43 smithi063 ceph-mon[112784]: Upgrade: Updating osd.5 2024-03-22T07:24:43.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:43.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-22T07:24:43.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:43 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:24:43.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:43 smithi063 ceph-mon[112784]: Deploying daemon osd.5 on smithi069 2024-03-22T07:24:45.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:45 smithi069 ceph-mon[83631]: pgmap v154: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-22T07:24:45.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:45 smithi063 ceph-mon[112784]: pgmap v154: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-03-22T07:24:45.824 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:45 smithi069 systemd[1]: Stopping Ceph osd.5 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:24:46.243 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:45 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[55195]: 2024-03-22T07:24:45.829+0000 7f50d6a93700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:24:46.243 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:45 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[55195]: 2024-03-22T07:24:45.829+0000 7f50d6a93700 -1 osd.5 75 *** Got signal Terminated *** 2024-03-22T07:24:46.243 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:45 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[55195]: 2024-03-22T07:24:45.829+0000 7f50d6a93700 -1 osd.5 75 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:24:46.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:46 smithi069 ceph-mon[83631]: osd.5 marked itself down and dead 2024-03-22T07:24:46.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:46 smithi063 ceph-mon[112784]: osd.5 marked itself down and dead 2024-03-22T07:24:46.914 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:46 smithi069 podman[102849]: 2024-03-22 07:24:46.602563701 +0000 UTC m=+0.886773352 container died 08e82e141487c187abccdedba3f0d98ce79200c35fc028e35658ce24237b6894 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=quincy-9c4e0c3) 2024-03-22T07:24:47.223 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:46 smithi069 podman[102849]: 2024-03-22 07:24:46.997202003 +0000 UTC m=+1.281411688 container cleanup 08e82e141487c187abccdedba3f0d98ce79200c35fc028e35658ce24237b6894 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=quincy-9c4e0c3, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:24:47.223 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:46 smithi069 bash[102849]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5 2024-03-22T07:24:47.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:47 smithi069 podman[102878]: 2024-03-22 07:24:47.223145052 +0000 UTC m=+0.617568618 container remove 08e82e141487c187abccdedba3f0d98ce79200c35fc028e35658ce24237b6894 (image=quay.ceph.io/ceph-ci/ceph@sha256:e2ff0b9ff70b30131a16fd0ac4779c04eb8f80752618b4f4b744a02826313232, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5, RELEASE=quincy-9c4e0c3, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-22T07:24:47.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:47 smithi069 ceph-mon[83631]: pgmap v155: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:24:47.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:47 smithi069 ceph-mon[83631]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:24:47.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:47 smithi069 ceph-mon[83631]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-22T07:24:47.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:47 smithi069 ceph-mon[83631]: osdmap e76: 6 total, 5 up, 6 in 2024-03-22T07:24:47.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:47 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:47.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:47 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:24:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:47 smithi063 ceph-mon[112784]: pgmap v155: 65 pgs: 65 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:24:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:47 smithi063 ceph-mon[112784]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-22T07:24:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:47 smithi063 ceph-mon[112784]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-22T07:24:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:47 smithi063 ceph-mon[112784]: osdmap e76: 6 total, 5 up, 6 in 2024-03-22T07:24:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:47 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:47.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:47 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:24:47.843 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:47 smithi069 podman[102970]: 2024-03-22 07:24:47.459156322 +0000 UTC m=+0.018060369 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:24:47.844 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:47 smithi069 podman[102970]: 2024-03-22 07:24:47.591079484 +0000 UTC m=+0.149983520 container create f1b88076b4ebab490ed014862a3ef99a9c310c27213d42e3a8b68cb366648d11 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-deactivate, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-03-22T07:24:47.844 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:47 smithi069 podman[102970]: 2024-03-22 07:24:47.84344836 +0000 UTC m=+0.402352407 container init f1b88076b4ebab490ed014862a3ef99a9c310c27213d42e3a8b68cb366648d11 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:24:48.137 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:47 smithi069 podman[102970]: 2024-03-22 07:24:47.846750234 +0000 UTC m=+0.405654272 container start f1b88076b4ebab490ed014862a3ef99a9c310c27213d42e3a8b68cb366648d11 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-22T07:24:48.137 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:47 smithi069 podman[102970]: 2024-03-22 07:24:47.935071483 +0000 UTC m=+0.493975521 container attach f1b88076b4ebab490ed014862a3ef99a9c310c27213d42e3a8b68cb366648d11 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-deactivate, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:24:48.479 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:48 smithi069 ceph-mon[83631]: osdmap e77: 6 total, 5 up, 6 in 2024-03-22T07:24:48.479 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:48 smithi069 podman[102970]: 2024-03-22 07:24:48.137518056 +0000 UTC m=+0.696422106 container died f1b88076b4ebab490ed014862a3ef99a9c310c27213d42e3a8b68cb366648d11 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-22T07:24:48.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:48 smithi063 ceph-mon[112784]: osdmap e77: 6 total, 5 up, 6 in 2024-03-22T07:24:49.227 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:48 smithi069 podman[102996]: 2024-03-22 07:24:48.797124971 +0000 UTC m=+0.649923920 container remove f1b88076b4ebab490ed014862a3ef99a9c310c27213d42e3a8b68cb366648d11 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-db0330b) 2024-03-22T07:24:49.227 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:48 smithi069 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-22T07:24:49.227 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:48 smithi069 systemd[1]: Stopped Ceph osd.5 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:24:49.227 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:48 smithi069 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.5.service: Consumed 1min 817ms CPU time. 2024-03-22T07:24:49.227 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 systemd[1]: Starting Ceph osd.5 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:24:49.509 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:49 smithi069 ceph-mon[83631]: pgmap v158: 65 pgs: 13 stale+active+clean, 52 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-22T07:24:49.510 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 podman[103099]: 2024-03-22 07:24:49.129634091 +0000 UTC m=+0.017114372 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:24:49.510 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 podman[103099]: 2024-03-22 07:24:49.251387045 +0000 UTC m=+0.138867331 container create 1850fe2fc6df97b03a292dad815173eb52861eb46bb7635b32b042c8ba987d25 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-activate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-22T07:24:49.510 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 podman[103099]: 2024-03-22 07:24:49.509912851 +0000 UTC m=+0.397393141 container init 1850fe2fc6df97b03a292dad815173eb52861eb46bb7635b32b042c8ba987d25 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-activate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-22T07:24:49.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:49 smithi063 ceph-mon[112784]: pgmap v158: 65 pgs: 13 stale+active+clean, 52 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-03-22T07:24:49.779 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 podman[103099]: 2024-03-22 07:24:49.513409281 +0000 UTC m=+0.400889572 container start 1850fe2fc6df97b03a292dad815173eb52861eb46bb7635b32b042c8ba987d25 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-activate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-db0330b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-03-22T07:24:49.779 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 podman[103099]: 2024-03-22 07:24:49.642901369 +0000 UTC m=+0.530381666 container attach 1850fe2fc6df97b03a292dad815173eb52861eb46bb7635b32b042c8ba987d25 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-activate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True) 2024-03-22T07:24:50.280 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-activate[103115]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-22T07:24:50.280 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-activate[103115]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-22T07:24:50.280 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 bash[103099]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-22T07:24:50.280 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 bash[103099]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-22T07:24:50.280 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 bash[103099]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-22T07:24:50.280 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 bash[103099]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-22T07:24:50.280 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 bash[103099]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-22T07:24:50.280 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 bash[103099]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-22T07:24:50.280 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 bash[103099]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-22T07:24:50.280 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-activate[103115]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-22T07:24:50.280 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-activate[103115]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-22T07:24:50.280 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-activate[103115]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-22T07:24:50.281 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-activate[103115]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-22T07:24:50.281 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-activate[103115]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-22T07:24:50.281 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:49 smithi069 podman[103099]: 2024-03-22 07:24:49.975490848 +0000 UTC m=+0.862971144 container died 1850fe2fc6df97b03a292dad815173eb52861eb46bb7635b32b042c8ba987d25 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-activate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, io.buildah.version=1.33.5) 2024-03-22T07:24:50.571 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:50 smithi069 ceph-mon[83631]: Health check failed: Degraded data redundancy: 23/354 objects degraded (6.497%), 8 pgs degraded (PG_DEGRADED) 2024-03-22T07:24:50.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:50 smithi063 ceph-mon[112784]: Health check failed: Degraded data redundancy: 23/354 objects degraded (6.497%), 8 pgs degraded (PG_DEGRADED) 2024-03-22T07:24:50.936 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:50 smithi069 podman[103235]: 2024-03-22 07:24:50.57138548 +0000 UTC m=+0.590781275 container remove 1850fe2fc6df97b03a292dad815173eb52861eb46bb7635b32b042c8ba987d25 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-activate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-22T07:24:50.936 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:50 smithi069 podman[103298]: 2024-03-22 07:24:50.685599133 +0000 UTC m=+0.018707269 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:24:50.936 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:50 smithi069 podman[103298]: 2024-03-22 07:24:50.814335365 +0000 UTC m=+0.147443526 container create c21401aa8e199da02b539cb6a7e4b2e995f82d5757aab61f96425a417bec4eab (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-22T07:24:51.280 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:51 smithi069 podman[103298]: 2024-03-22 07:24:51.041421894 +0000 UTC m=+0.374530023 container init c21401aa8e199da02b539cb6a7e4b2e995f82d5757aab61f96425a417bec4eab (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307) 2024-03-22T07:24:51.280 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:51 smithi069 podman[103298]: 2024-03-22 07:24:51.044410161 +0000 UTC m=+0.377518281 container start c21401aa8e199da02b539cb6a7e4b2e995f82d5757aab61f96425a417bec4eab (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:24:51.280 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:51 smithi069 ceph-osd[103317]: -- 172.21.15.69:0/2044887495 <== mon.1 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561ed19331e0 con 0x561ed1904c00 2024-03-22T07:24:51.280 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:51 smithi069 bash[103298]: c21401aa8e199da02b539cb6a7e4b2e995f82d5757aab61f96425a417bec4eab 2024-03-22T07:24:51.280 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:51 smithi069 systemd[1]: Started Ceph osd.5 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:24:51.587 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:51 smithi069 ceph-mon[83631]: pgmap v159: 65 pgs: 8 active+undersized+degraded, 5 stale+active+clean, 7 active+undersized, 45 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23/354 objects degraded (6.497%) 2024-03-22T07:24:51.587 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:51 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:51.587 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:51 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:51.587 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:51 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:24:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:51 smithi063 ceph-mon[112784]: pgmap v159: 65 pgs: 8 active+undersized+degraded, 5 stale+active+clean, 7 active+undersized, 45 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23/354 objects degraded (6.497%) 2024-03-22T07:24:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:51 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:51 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:51.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:51 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:24:51.971 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:51 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:24:51.586+0000 7f2cbc27d7c0 -1 Falling back to public interface 2024-03-22T07:24:53.616 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:53 smithi069 ceph-mon[83631]: pgmap v160: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 54/354 objects degraded (15.254%) 2024-03-22T07:24:53.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:53 smithi063 ceph-mon[112784]: pgmap v160: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 54/354 objects degraded (15.254%) 2024-03-22T07:24:55.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:55 smithi069 ceph-mon[83631]: pgmap v161: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 54/354 objects degraded (15.254%) 2024-03-22T07:24:55.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:55 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:55.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:55 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:55.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:55 smithi063 ceph-mon[112784]: pgmap v161: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 54/354 objects degraded (15.254%) 2024-03-22T07:24:55.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:55 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:55.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:55 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:56.129 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:55 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:24:55.784+0000 7f2cbc27d7c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-03-22T07:24:56.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:56 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:56.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:56 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:56.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:56 smithi069 ceph-mon[83631]: from='osd.5 [v2:172.21.15.69:6816/1480046180,v1:172.21.15.69:6817/1480046180]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T07:24:56.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:56 smithi069 ceph-mon[83631]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T07:24:56.530 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:56 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:24:56.211+0000 7f2cbc27d7c0 -1 osd.5 75 log_to_monitors true 2024-03-22T07:24:56.531 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:24:56 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:24:56.279+0000 7f2cad6d5700 -1 osd.5 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-22T07:24:56.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:56 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:56.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:56 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:56.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:56 smithi063 ceph-mon[112784]: from='osd.5 [v2:172.21.15.69:6816/1480046180,v1:172.21.15.69:6817/1480046180]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T07:24:56.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:56 smithi063 ceph-mon[112784]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-22T07:24:57.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:57 smithi069 ceph-mon[83631]: pgmap v162: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 54/354 objects degraded (15.254%) 2024-03-22T07:24:57.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:57 smithi069 ceph-mon[83631]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T07:24:57.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:57 smithi069 ceph-mon[83631]: osdmap e78: 6 total, 5 up, 6 in 2024-03-22T07:24:57.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:57 smithi069 ceph-mon[83631]: from='osd.5 [v2:172.21.15.69:6816/1480046180,v1:172.21.15.69:6817/1480046180]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:24:57.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:57 smithi069 ceph-mon[83631]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:24:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:57 smithi063 ceph-mon[112784]: pgmap v162: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 299 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 54/354 objects degraded (15.254%) 2024-03-22T07:24:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:57 smithi063 ceph-mon[112784]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-22T07:24:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:57 smithi063 ceph-mon[112784]: osdmap e78: 6 total, 5 up, 6 in 2024-03-22T07:24:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:57 smithi063 ceph-mon[112784]: from='osd.5 [v2:172.21.15.69:6816/1480046180,v1:172.21.15.69:6817/1480046180]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:24:57.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:57 smithi063 ceph-mon[112784]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-03-22T07:24:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:58 smithi069 ceph-mon[83631]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:24:58.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:58 smithi069 ceph-mon[83631]: osd.5 [v2:172.21.15.69:6816/1480046180,v1:172.21.15.69:6817/1480046180] boot 2024-03-22T07:24:58.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:58 smithi069 ceph-mon[83631]: osdmap e79: 6 total, 6 up, 6 in 2024-03-22T07:24:58.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:58 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:24:58.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:58 smithi069 ceph-mon[83631]: Health check update: Degraded data redundancy: 54/354 objects degraded (15.254%), 14 pgs degraded (PG_DEGRADED) 2024-03-22T07:24:58.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:58 smithi069 ceph-mon[83631]: osdmap e80: 6 total, 6 up, 6 in 2024-03-22T07:24:58.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:58 smithi063 ceph-mon[112784]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-22T07:24:58.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:58 smithi063 ceph-mon[112784]: osd.5 [v2:172.21.15.69:6816/1480046180,v1:172.21.15.69:6817/1480046180] boot 2024-03-22T07:24:58.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:58 smithi063 ceph-mon[112784]: osdmap e79: 6 total, 6 up, 6 in 2024-03-22T07:24:58.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:58 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-22T07:24:58.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:58 smithi063 ceph-mon[112784]: Health check update: Degraded data redundancy: 54/354 objects degraded (15.254%), 14 pgs degraded (PG_DEGRADED) 2024-03-22T07:24:58.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:58 smithi063 ceph-mon[112784]: osdmap e80: 6 total, 6 up, 6 in 2024-03-22T07:24:59.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: pgmap v165: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 54/354 objects degraded (15.254%) 2024-03-22T07:24:59.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:59.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:59.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:24:59.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:24:59.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:59.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:24:59.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:24:59.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:24:59.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:24:59.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:24:59.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:59.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-22T07:24:59.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-22T07:24:59.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-22T07:24:59.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-22T07:24:59.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-22T07:24:59.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-22T07:24:59.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-22T07:24:59.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-22T07:24:59.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-22T07:24:59.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-22T07:24:59.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-22T07:24:59.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-22T07:24:59.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:24:59 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-22T07:24:59.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: pgmap v165: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 54/354 objects degraded (15.254%) 2024-03-22T07:24:59.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:59.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:59.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:24:59.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:24:59.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:59.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:24:59.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:24:59.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:24:59.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:24:59.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:24:59.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:24:59.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-22T07:24:59.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-22T07:24:59.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-22T07:24:59.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-22T07:24:59.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-22T07:24:59.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-22T07:24:59.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-22T07:24:59.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-22T07:24:59.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-22T07:24:59.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-22T07:24:59.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-22T07:24:59.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-22T07:24:59.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:24:59 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-22T07:25:00.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:00 smithi063 ceph-mon[112784]: Upgrade: Setting container_image for all osd 2024-03-22T07:25:00.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:00 smithi063 ceph-mon[112784]: Upgrade: Setting require_osd_release to 19 squid 2024-03-22T07:25:00.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:00 smithi063 ceph-mon[112784]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-22T07:25:00.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:00 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-22T07:25:00.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:00 smithi063 ceph-mon[112784]: osdmap e81: 6 total, 6 up, 6 in 2024-03-22T07:25:00.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:00 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:00.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:00 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-03-22T07:25:00.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:00 smithi069 ceph-mon[83631]: Upgrade: Setting container_image for all osd 2024-03-22T07:25:00.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:00 smithi069 ceph-mon[83631]: Upgrade: Setting require_osd_release to 19 squid 2024-03-22T07:25:00.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:00 smithi069 ceph-mon[83631]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-22T07:25:00.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:00 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-22T07:25:00.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:00 smithi069 ceph-mon[83631]: osdmap e81: 6 total, 6 up, 6 in 2024-03-22T07:25:00.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:00 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:00.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:00 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-03-22T07:25:01.292 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:00 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063[112780]: 2024-03-22T07:25:00.990+0000 7f52b5c9f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T07:25:01.593 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: pgmap v167: 65 pgs: 2 activating, 2 peering, 10 active+undersized+degraded, 11 active+undersized, 40 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 44/354 objects degraded (12.429%) 2024-03-22T07:25:01.593 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-03-22T07:25:01.593 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: osdmap e82: 6 total, 6 up, 6 in 2024-03-22T07:25:01.593 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-22T07:25:01.593 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-03-22T07:25:01.593 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: fsmap cephfs:2 {0=cephfs.smithi063.xeelpk=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2024-03-22T07:25:01.593 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:25:01.593 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:25:01.593 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:25:01.593 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:01.593 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:25:01.593 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:01.593 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:01.594 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:01.596 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:01.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-22T07:25:01.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T07:25:01.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T07:25:01.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-22T07:25:01.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-22T07:25:01.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: osdmap e83: 6 total, 6 up, 6 in 2024-03-22T07:25:01.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: mds.? [v2:172.21.15.69:6826/675976207,v1:172.21.15.69:6827/675976207] up:boot 2024-03-22T07:25:01.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: mds.? [v2:172.21.15.63:6826/1673237445,v1:172.21.15.63:6827/1673237445] up:boot 2024-03-22T07:25:01.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-03-22T07:25:01.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.oytapz"}]: dispatch 2024-03-22T07:25:01.597 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.cijxah"}]: dispatch 2024-03-22T07:25:01.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: pgmap v167: 65 pgs: 2 activating, 2 peering, 10 active+undersized+degraded, 11 active+undersized, 40 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 44/354 objects degraded (12.429%) 2024-03-22T07:25:01.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-03-22T07:25:01.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: osdmap e82: 6 total, 6 up, 6 in 2024-03-22T07:25:01.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-22T07:25:01.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-03-22T07:25:01.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: fsmap cephfs:2 {0=cephfs.smithi063.xeelpk=up:active,1=cephfs.smithi069.wwtsjn=up:active} 2024-03-22T07:25:01.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:25:01.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:25:01.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:25:01.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:01.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:25:01.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:01.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:01.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:01.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:01.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-22T07:25:01.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-22T07:25:01.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-22T07:25:01.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-03-22T07:25:01.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-22T07:25:01.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: osdmap e83: 6 total, 6 up, 6 in 2024-03-22T07:25:01.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: mds.? [v2:172.21.15.69:6826/675976207,v1:172.21.15.69:6827/675976207] up:boot 2024-03-22T07:25:01.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: mds.? [v2:172.21.15.63:6826/1673237445,v1:172.21.15.63:6827/1673237445] up:boot 2024-03-22T07:25:01.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-03-22T07:25:01.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.oytapz"}]: dispatch 2024-03-22T07:25:01.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.cijxah"}]: dispatch 2024-03-22T07:25:02.613 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:02 smithi063 ceph-mon[112784]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-03-22T07:25:02.613 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:02.613 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:25:02.614 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:02 smithi063 ceph-mon[112784]: mds.? [v2:172.21.15.69:6824/2785357568,v1:172.21.15.69:6825/2785357568] up:boot 2024-03-22T07:25:02.614 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:02 smithi063 ceph-mon[112784]: mds.? [v2:172.21.15.63:6828/1202791208,v1:172.21.15.63:6829/1202791208] up:boot 2024-03-22T07:25:02.614 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:02 smithi063 ceph-mon[112784]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:25:02.614 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.xeelpk"}]: dispatch 2024-03-22T07:25:02.614 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.wwtsjn"}]: dispatch 2024-03-22T07:25:02.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:02 smithi069 ceph-mon[83631]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-03-22T07:25:02.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:02.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:25:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:02 smithi069 ceph-mon[83631]: mds.? [v2:172.21.15.69:6824/2785357568,v1:172.21.15.69:6825/2785357568] up:boot 2024-03-22T07:25:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:02 smithi069 ceph-mon[83631]: mds.? [v2:172.21.15.63:6828/1202791208,v1:172.21.15.63:6829/1202791208] up:boot 2024-03-22T07:25:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:02 smithi069 ceph-mon[83631]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:25:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.xeelpk"}]: dispatch 2024-03-22T07:25:02.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.wwtsjn"}]: dispatch 2024-03-22T07:25:03.550 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:03 smithi063 ceph-mon[112784]: pgmap v171: 65 pgs: 2 activating, 2 peering, 61 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-03-22T07:25:03.550 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:03 smithi063 ceph-mon[112784]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 44/354 objects degraded (12.429%), 10 pgs degraded) 2024-03-22T07:25:03.550 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:03 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:03.550 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:03 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.oytapz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:25:03.551 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:03 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:25:03.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:03 smithi069 ceph-mon[83631]: pgmap v171: 65 pgs: 2 activating, 2 peering, 61 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-03-22T07:25:03.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:03 smithi069 ceph-mon[83631]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 44/354 objects degraded (12.429%), 10 pgs degraded) 2024-03-22T07:25:03.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:03 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:03.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:03 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.oytapz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:25:03.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:03 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:25:04.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:04 smithi063 ceph-mon[112784]: Upgrade: Updating mds.cephfs.smithi063.oytapz 2024-03-22T07:25:04.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:04 smithi063 ceph-mon[112784]: Deploying daemon mds.cephfs.smithi063.oytapz on smithi063 2024-03-22T07:25:04.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:04 smithi069 ceph-mon[83631]: Upgrade: Updating mds.cephfs.smithi063.oytapz 2024-03-22T07:25:04.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:04 smithi069 ceph-mon[83631]: Deploying daemon mds.cephfs.smithi063.oytapz on smithi063 2024-03-22T07:25:05.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:05 smithi063 ceph-mon[112784]: pgmap v172: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-03-22T07:25:05.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:05 smithi069 ceph-mon[83631]: pgmap v172: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s 2024-03-22T07:25:06.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:06 smithi063 ceph-mon[112784]: osdmap e84: 6 total, 6 up, 6 in 2024-03-22T07:25:06.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:06 smithi063 ceph-mon[112784]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-22T07:25:06.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:06 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.oytapz"}]: dispatch 2024-03-22T07:25:06.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:06 smithi069 ceph-mon[83631]: osdmap e84: 6 total, 6 up, 6 in 2024-03-22T07:25:06.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:06 smithi069 ceph-mon[83631]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-22T07:25:06.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:06 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.oytapz"}]: dispatch 2024-03-22T07:25:07.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:07 smithi063 ceph-mon[112784]: pgmap v174: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-03-22T07:25:07.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:07 smithi069 ceph-mon[83631]: pgmap v174: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-03-22T07:25:09.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:09 smithi063 ceph-mon[112784]: pgmap v175: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-03-22T07:25:09.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:09 smithi069 ceph-mon[83631]: pgmap v175: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-03-22T07:25:09.961 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.958+0000 7f1321876700 1 -- 172.21.15.63:0/2078802395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f131c102e10 msgr2=0x7f131c1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.958+0000 7f1321876700 1 --2- 172.21.15.63:0/2078802395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f131c102e10 0x7f131c1031e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1304004940 tx=0x7f1304009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.958+0000 7f1321876700 1 -- 172.21.15.63:0/2078802395 shutdown_connections 2024-03-22T07:25:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.958+0000 7f1321876700 1 --2- 172.21.15.63:0/2078802395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f131c1037b0 0x7f131c0fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.958+0000 7f1321876700 1 --2- 172.21.15.63:0/2078802395 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f131c102e10 0x7f131c1031e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.958+0000 7f1321876700 1 -- 172.21.15.63:0/2078802395 >> 172.21.15.63:0/2078802395 conn(0x7f131c0f89a0 msgr2=0x7f131c0fadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.958+0000 7f1321876700 1 -- 172.21.15.63:0/2078802395 shutdown_connections 2024-03-22T07:25:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.958+0000 7f1321876700 1 -- 172.21.15.63:0/2078802395 wait complete. 2024-03-22T07:25:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.959+0000 7f1321876700 1 Processor -- start 2024-03-22T07:25:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.959+0000 7f1321876700 1 -- start start 2024-03-22T07:25:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.959+0000 7f1321876700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f131c102e10 0x7f131c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.959+0000 7f1321876700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f131c1037b0 0x7f131c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:09.962 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.959+0000 7f1321876700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c070d40 con 0x7f131c1037b0 2024-03-22T07:25:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.959+0000 7f1321876700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c070e80 con 0x7f131c102e10 2024-03-22T07:25:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.959+0000 7f131affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f131c102e10 0x7f131c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.959+0000 7f131affd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f131c102e10 0x7f131c0702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.63:36874/0 (socket says 172.21.15.63:36874) 2024-03-22T07:25:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.959+0000 7f131affd700 1 -- 172.21.15.63:0/2576776293 learned_addr learned my addr 172.21.15.63:0/2576776293 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:25:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.960+0000 7f131a7fc700 1 --2- 172.21.15.63:0/2576776293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f131c1037b0 0x7f131c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.960+0000 7f131affd700 1 -- 172.21.15.63:0/2576776293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f131c1037b0 msgr2=0x7f131c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.960+0000 7f131affd700 1 --2- 172.21.15.63:0/2576776293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f131c1037b0 0x7f131c070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.960+0000 7f131affd700 1 -- 172.21.15.63:0/2576776293 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1304005040 con 0x7f131c102e10 2024-03-22T07:25:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.960+0000 7f131affd700 1 --2- 172.21.15.63:0/2576776293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f131c102e10 0x7f131c0702c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1304009150 tx=0x7f1304009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.960+0000 7f13137fe700 1 -- 172.21.15.63:0/2576776293 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130400e040 con 0x7f131c102e10 2024-03-22T07:25:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.960+0000 7f13137fe700 1 -- 172.21.15.63:0/2576776293 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1304013420 con 0x7f131c102e10 2024-03-22T07:25:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.960+0000 7f1321876700 1 -- 172.21.15.63:0/2576776293 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131c0710d0 con 0x7f131c102e10 2024-03-22T07:25:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.960+0000 7f13137fe700 1 -- 172.21.15.63:0/2576776293 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130401c630 con 0x7f131c102e10 2024-03-22T07:25:09.963 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.960+0000 7f1321876700 1 -- 172.21.15.63:0/2576776293 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131c11a2f0 con 0x7f131c102e10 2024-03-22T07:25:09.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.961+0000 7f1321876700 1 -- 172.21.15.63:0/2576776293 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f131c100120 con 0x7f131c102e10 2024-03-22T07:25:09.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.962+0000 7f13137fe700 1 -- 172.21.15.63:0/2576776293 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f1304013590 con 0x7f131c102e10 2024-03-22T07:25:09.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.962+0000 7f13137fe700 1 --2- 172.21.15.63:0/2576776293 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f130806d390 0x7f130806f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:09.964 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.963+0000 7f13137fe700 1 -- 172.21.15.63:0/2576776293 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f130408dc00 con 0x7f131c102e10 2024-03-22T07:25:09.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.963+0000 7f131a7fc700 1 --2- 172.21.15.63:0/2576776293 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f130806d390 0x7f130806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:09.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.963+0000 7f131a7fc700 1 --2- 172.21.15.63:0/2576776293 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f130806d390 0x7f130806f840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f130c007fd0 tx=0x7f130c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:09.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:09.967+0000 7f13137fe700 1 -- 172.21.15.63:0/2576776293 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1304057610 con 0x7f131c102e10 2024-03-22T07:25:10.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.204+0000 7f1321876700 1 -- 172.21.15.63:0/2576776293 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f131c04bed0 con 0x7f130806d390 2024-03-22T07:25:10.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.206+0000 7f13137fe700 1 -- 172.21.15.63:0/2576776293 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f131c04bed0 con 0x7f130806d390 2024-03-22T07:25:10.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.208+0000 7f1321876700 1 -- 172.21.15.63:0/2576776293 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f130806d390 msgr2=0x7f130806f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:10.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.208+0000 7f1321876700 1 --2- 172.21.15.63:0/2576776293 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f130806d390 0x7f130806f840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f130c007fd0 tx=0x7f130c009040 comp rx=0 tx=0).stop 2024-03-22T07:25:10.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.208+0000 7f1321876700 1 -- 172.21.15.63:0/2576776293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f131c102e10 msgr2=0x7f131c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:10.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.208+0000 7f1321876700 1 --2- 172.21.15.63:0/2576776293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f131c102e10 0x7f131c0702c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1304009150 tx=0x7f1304009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:10.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.208+0000 7f1321876700 1 -- 172.21.15.63:0/2576776293 shutdown_connections 2024-03-22T07:25:10.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.208+0000 7f1321876700 1 --2- 172.21.15.63:0/2576776293 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f130806d390 0x7f130806f840 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:10.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.208+0000 7f1321876700 1 --2- 172.21.15.63:0/2576776293 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f131c102e10 0x7f131c0702c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:10.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.208+0000 7f1321876700 1 --2- 172.21.15.63:0/2576776293 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f131c1037b0 0x7f131c070800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:10.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.208+0000 7f1321876700 1 -- 172.21.15.63:0/2576776293 >> 172.21.15.63:0/2576776293 conn(0x7f131c0f89a0 msgr2=0x7f131c0facf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:10.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.208+0000 7f1321876700 1 -- 172.21.15.63:0/2576776293 shutdown_connections 2024-03-22T07:25:10.210 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.208+0000 7f1321876700 1 -- 172.21.15.63:0/2576776293 wait complete. 2024-03-22T07:25:10.219 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:25:10.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.312+0000 7f402aef2700 1 -- 172.21.15.63:0/2961167389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4024102e00 msgr2=0x7f40241031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:10.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.312+0000 7f402aef2700 1 --2- 172.21.15.63:0/2961167389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4024102e00 0x7f40241031d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4018004940 tx=0x7f4018009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:10.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.312+0000 7f402aef2700 1 -- 172.21.15.63:0/2961167389 shutdown_connections 2024-03-22T07:25:10.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.312+0000 7f402aef2700 1 --2- 172.21.15.63:0/2961167389 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40241037a0 0x7f40240fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:10.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.312+0000 7f402aef2700 1 --2- 172.21.15.63:0/2961167389 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4024102e00 0x7f40241031d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:10.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.312+0000 7f402aef2700 1 -- 172.21.15.63:0/2961167389 >> 172.21.15.63:0/2961167389 conn(0x7f40240f89b0 msgr2=0x7f40240fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:10.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.312+0000 7f402aef2700 1 -- 172.21.15.63:0/2961167389 shutdown_connections 2024-03-22T07:25:10.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.312+0000 7f402aef2700 1 -- 172.21.15.63:0/2961167389 wait complete. 2024-03-22T07:25:10.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.312+0000 7f402aef2700 1 Processor -- start 2024-03-22T07:25:10.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.312+0000 7f402aef2700 1 -- start start 2024-03-22T07:25:10.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.313+0000 7f402aef2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4024102e00 0x7f40240702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:10.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.313+0000 7f402aef2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40241037a0 0x7f4024070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:10.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.313+0000 7f402aef2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40240721c0 con 0x7f40241037a0 2024-03-22T07:25:10.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.313+0000 7f402aef2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4024070d40 con 0x7f4024102e00 2024-03-22T07:25:10.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.313+0000 7f4023fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40241037a0 0x7f4024070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:10.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.313+0000 7f4023fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40241037a0 0x7f4024070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58178/0 (socket says 172.21.15.63:58178) 2024-03-22T07:25:10.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.313+0000 7f4023fff700 1 -- 172.21.15.63:0/2086294173 learned_addr learned my addr 172.21.15.63:0/2086294173 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:25:10.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.313+0000 7f4028c8e700 1 --2- 172.21.15.63:0/2086294173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4024102e00 0x7f40240702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:10.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.313+0000 7f4023fff700 1 -- 172.21.15.63:0/2086294173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4024102e00 msgr2=0x7f40240702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:10.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.313+0000 7f4023fff700 1 --2- 172.21.15.63:0/2086294173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4024102e00 0x7f40240702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:10.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.313+0000 7f4023fff700 1 -- 172.21.15.63:0/2086294173 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4018005040 con 0x7f40241037a0 2024-03-22T07:25:10.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.313+0000 7f4023fff700 1 --2- 172.21.15.63:0/2086294173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40241037a0 0x7f4024070800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f401000c890 tx=0x7f40100044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:10.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.313+0000 7f40217fa700 1 -- 172.21.15.63:0/2086294173 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f401000a700 con 0x7f40241037a0 2024-03-22T07:25:10.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.313+0000 7f40217fa700 1 -- 172.21.15.63:0/2086294173 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f401000ad30 con 0x7f40241037a0 2024-03-22T07:25:10.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.313+0000 7f402aef2700 1 -- 172.21.15.63:0/2086294173 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4024071020 con 0x7f40241037a0 2024-03-22T07:25:10.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.313+0000 7f40217fa700 1 -- 172.21.15.63:0/2086294173 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40100113a0 con 0x7f40241037a0 2024-03-22T07:25:10.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.313+0000 7f402aef2700 1 -- 172.21.15.63:0/2086294173 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402411a180 con 0x7f40241037a0 2024-03-22T07:25:10.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.314+0000 7f402aef2700 1 -- 172.21.15.63:0/2086294173 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4024100110 con 0x7f40241037a0 2024-03-22T07:25:10.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.315+0000 7f40217fa700 1 -- 172.21.15.63:0/2086294173 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f4010012400 con 0x7f40241037a0 2024-03-22T07:25:10.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.315+0000 7f40217fa700 1 --2- 172.21.15.63:0/2086294173 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f401406d380 0x7f401406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:10.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.315+0000 7f4028c8e700 1 --2- 172.21.15.63:0/2086294173 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f401406d380 0x7f401406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:10.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.315+0000 7f40217fa700 1 -- 172.21.15.63:0/2086294173 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f401008c9c0 con 0x7f40241037a0 2024-03-22T07:25:10.317 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.316+0000 7f4028c8e700 1 --2- 172.21.15.63:0/2086294173 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f401406d380 0x7f401406f830 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4018009fd0 tx=0x7f4018009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:10.321 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.320+0000 7f40217fa700 1 -- 172.21.15.63:0/2086294173 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4010056470 con 0x7f40241037a0 2024-03-22T07:25:10.549 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.548+0000 7f402aef2700 1 -- 172.21.15.63:0/2086294173 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f40240717c0 con 0x7f401406d380 2024-03-22T07:25:10.550 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.549+0000 7f40217fa700 1 -- 172.21.15.63:0/2086294173 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f40240717c0 con 0x7f401406d380 2024-03-22T07:25:10.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.551+0000 7f402aef2700 1 -- 172.21.15.63:0/2086294173 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f401406d380 msgr2=0x7f401406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:10.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.551+0000 7f402aef2700 1 --2- 172.21.15.63:0/2086294173 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f401406d380 0x7f401406f830 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4018009fd0 tx=0x7f4018009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:10.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.551+0000 7f402aef2700 1 -- 172.21.15.63:0/2086294173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40241037a0 msgr2=0x7f4024070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:10.552 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.551+0000 7f402aef2700 1 --2- 172.21.15.63:0/2086294173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40241037a0 0x7f4024070800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f401000c890 tx=0x7f40100044c0 comp rx=0 tx=0).stop 2024-03-22T07:25:10.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.551+0000 7f402aef2700 1 -- 172.21.15.63:0/2086294173 shutdown_connections 2024-03-22T07:25:10.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.551+0000 7f402aef2700 1 --2- 172.21.15.63:0/2086294173 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f401406d380 0x7f401406f830 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:10.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.551+0000 7f402aef2700 1 --2- 172.21.15.63:0/2086294173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4024102e00 0x7f40240702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:10.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.551+0000 7f402aef2700 1 --2- 172.21.15.63:0/2086294173 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f40241037a0 0x7f4024070800 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:10.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.551+0000 7f402aef2700 1 -- 172.21.15.63:0/2086294173 >> 172.21.15.63:0/2086294173 conn(0x7f40240f89b0 msgr2=0x7f40241073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:10.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.551+0000 7f402aef2700 1 -- 172.21.15.63:0/2086294173 shutdown_connections 2024-03-22T07:25:10.553 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.551+0000 7f402aef2700 1 -- 172.21.15.63:0/2086294173 wait complete. 2024-03-22T07:25:10.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.658+0000 7f310261f700 1 -- 172.21.15.63:0/4213154485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30fc0fb540 msgr2=0x7f30fc0fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:10.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.658+0000 7f310261f700 1 --2- 172.21.15.63:0/4213154485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30fc0fb540 0x7f30fc0fb990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f30f0004940 tx=0x7f30f0009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:10.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.658+0000 7f310261f700 1 -- 172.21.15.63:0/4213154485 shutdown_connections 2024-03-22T07:25:10.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.658+0000 7f310261f700 1 --2- 172.21.15.63:0/4213154485 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30fc0fb540 0x7f30fc0fb990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:10.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.658+0000 7f310261f700 1 --2- 172.21.15.63:0/4213154485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc102fd0 0x7f30fc1033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:10.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.658+0000 7f310261f700 1 -- 172.21.15.63:0/4213154485 >> 172.21.15.63:0/4213154485 conn(0x7f30fc071530 msgr2=0x7f30fc071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:10.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.658+0000 7f310261f700 1 -- 172.21.15.63:0/4213154485 shutdown_connections 2024-03-22T07:25:10.660 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.659+0000 7f310261f700 1 -- 172.21.15.63:0/4213154485 wait complete. 2024-03-22T07:25:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.659+0000 7f310261f700 1 Processor -- start 2024-03-22T07:25:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.659+0000 7f310261f700 1 -- start start 2024-03-22T07:25:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.660+0000 7f310261f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30fc0fb540 0x7f30fc0fc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.660+0000 7f310261f700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc102fd0 0x7f30fc0fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.660+0000 7f310261f700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc0fe670 con 0x7f30fc0fb540 2024-03-22T07:25:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.660+0000 7f310261f700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc0fd1a0 con 0x7f30fc102fd0 2024-03-22T07:25:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.660+0000 7f30fbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30fc0fb540 0x7f30fc0fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.660+0000 7f30fbfff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30fc0fb540 0x7f30fc0fc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58214/0 (socket says 172.21.15.63:58214) 2024-03-22T07:25:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.660+0000 7f30fbfff700 1 -- 172.21.15.63:0/3783424779 learned_addr learned my addr 172.21.15.63:0/3783424779 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:25:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.660+0000 7f30fb7fe700 1 --2- 172.21.15.63:0/3783424779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc102fd0 0x7f30fc0fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.660+0000 7f30fbfff700 1 -- 172.21.15.63:0/3783424779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc102fd0 msgr2=0x7f30fc0fcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:10.663 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.660+0000 7f30fbfff700 1 --2- 172.21.15.63:0/3783424779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc102fd0 0x7f30fc0fcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:10.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.660+0000 7f30fbfff700 1 -- 172.21.15.63:0/3783424779 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f0005040 con 0x7f30fc0fb540 2024-03-22T07:25:10.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.660+0000 7f30fbfff700 1 --2- 172.21.15.63:0/3783424779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30fc0fb540 0x7f30fc0fc720 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f30ec002a50 tx=0x7f30ec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:10.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.660+0000 7f30f8ff9700 1 -- 172.21.15.63:0/3783424779 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30ec007cc0 con 0x7f30fc0fb540 2024-03-22T07:25:10.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.660+0000 7f30f8ff9700 1 -- 172.21.15.63:0/3783424779 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30ec00f040 con 0x7f30fc0fb540 2024-03-22T07:25:10.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.660+0000 7f30f8ff9700 1 -- 172.21.15.63:0/3783424779 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30ec00da30 con 0x7f30fc0fb540 2024-03-22T07:25:10.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.660+0000 7f310261f700 1 -- 172.21.15.63:0/3783424779 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30fc0fd480 con 0x7f30fc0fb540 2024-03-22T07:25:10.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.661+0000 7f310261f700 1 -- 172.21.15.63:0/3783424779 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30fc11a100 con 0x7f30fc0fb540 2024-03-22T07:25:10.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.662+0000 7f310261f700 1 -- 172.21.15.63:0/3783424779 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30fc1053b0 con 0x7f30fc0fb540 2024-03-22T07:25:10.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.662+0000 7f30f8ff9700 1 -- 172.21.15.63:0/3783424779 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f30ec00db90 con 0x7f30fc0fb540 2024-03-22T07:25:10.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.663+0000 7f30f8ff9700 1 --2- 172.21.15.63:0/3783424779 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f30e406d380 0x7f30e406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:10.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.663+0000 7f30f8ff9700 1 -- 172.21.15.63:0/3783424779 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f30ec08c5f0 con 0x7f30fc0fb540 2024-03-22T07:25:10.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.663+0000 7f30fb7fe700 1 --2- 172.21.15.63:0/3783424779 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f30e406d380 0x7f30e406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:10.665 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.664+0000 7f30fb7fe700 1 --2- 172.21.15.63:0/3783424779 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f30e406d380 0x7f30e406f830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0fde50 tx=0x7f30f0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:10.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.668+0000 7f30f8ff9700 1 -- 172.21.15.63:0/3783424779 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f30ec0560a0 con 0x7f30fc0fb540 2024-03-22T07:25:10.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.891+0000 7f310261f700 1 -- 172.21.15.63:0/3783424779 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f30fc0fdc00 con 0x7f30e406d380 2024-03-22T07:25:10.903 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.902+0000 7f30f8ff9700 1 -- 172.21.15.63:0/3783424779 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f30fc0fdc00 con 0x7f30e406d380 2024-03-22T07:25:10.903 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:25:10.903 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (6m) 117s ago 16m 26.3M - 0.25.0 c8568f914cd2 06ef21da90e5 2024-03-22T07:25:10.903 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (3m) 117s ago 17m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 224c291ab774 2024-03-22T07:25:10.903 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (3m) 16s ago 16m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 7eabb7d42631 2024-03-22T07:25:10.903 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (5m) 117s ago 16m 86.0M - 9.4.12 a0ea5e3c46d9 9015fae0d5e9 2024-03-22T07:25:10.903 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 starting - - - - 2024-03-22T07:25:10.903 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (11m) 117s ago 11m 123M - 17.2.7-726-g9c4e0c34 837124da804f 5d3b4eb3e28e 2024-03-22T07:25:10.903 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (11m) 16s ago 11m 63.0M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:25:10.904 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (11m) 16s ago 11m 129M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:25:10.904 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:8443,9283,8765 running (7m) 117s ago 18m 506M - 19.0.0-1667-gdb0330b1 4f325f8440e9 e49468f39991 2024-03-22T07:25:10.904 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283,8765 running (7m) 16s ago 16m 450M - 19.0.0-1667-gdb0330b1 4f325f8440e9 13c3f034ebb2 2024-03-22T07:25:10.904 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (4m) 117s ago 18m 47.7M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ab96538a07c1 2024-03-22T07:25:10.904 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (4m) 16s ago 16m 39.5M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 01a26e1fb1a1 2024-03-22T07:25:10.904 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (6m) 117s ago 17m 16.0M - 1.5.0 0da6a335fe13 0c825ee2a196 2024-03-22T07:25:10.904 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (6m) 16s ago 16m 16.5M - 1.5.0 0da6a335fe13 12edb2f35498 2024-03-22T07:25:10.904 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (3m) 117s ago 15m 102M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 fdbc8b34ab78 2024-03-22T07:25:10.904 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (2m) 117s ago 14m 98.3M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 5200fd893046 2024-03-22T07:25:10.904 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (2m) 117s ago 14m 11.7M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 918925ee69f3 2024-03-22T07:25:10.904 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (87s) 16s ago 13m 101M 2581M 19.0.0-1667-gdb0330b1 4f325f8440e9 652a1e62339b 2024-03-22T07:25:10.904 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (53s) 16s ago 13m 94.4M 2581M 19.0.0-1667-gdb0330b1 4f325f8440e9 d6340a0e1c6c 2024-03-22T07:25:10.904 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (20s) 16s ago 13m 12.7M 2581M 19.0.0-1667-gdb0330b1 4f325f8440e9 c21401aa8e19 2024-03-22T07:25:10.904 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (6m) 117s ago 16m 61.1M - 2.43.0 a07b618ecd1d 1d89b928f6e7 2024-03-22T07:25:10.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.904+0000 7f310261f700 1 -- 172.21.15.63:0/3783424779 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f30e406d380 msgr2=0x7f30e406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:10.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.904+0000 7f310261f700 1 --2- 172.21.15.63:0/3783424779 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f30e406d380 0x7f30e406f830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0fde50 tx=0x7f30f0009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:10.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.904+0000 7f310261f700 1 -- 172.21.15.63:0/3783424779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30fc0fb540 msgr2=0x7f30fc0fc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:10.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.904+0000 7f310261f700 1 --2- 172.21.15.63:0/3783424779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30fc0fb540 0x7f30fc0fc720 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f30ec002a50 tx=0x7f30ec0046b0 comp rx=0 tx=0).stop 2024-03-22T07:25:10.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.904+0000 7f310261f700 1 -- 172.21.15.63:0/3783424779 shutdown_connections 2024-03-22T07:25:10.905 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.904+0000 7f310261f700 1 --2- 172.21.15.63:0/3783424779 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f30e406d380 0x7f30e406f830 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:10.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.904+0000 7f310261f700 1 --2- 172.21.15.63:0/3783424779 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f30fc0fb540 0x7f30fc0fc720 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:10.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.904+0000 7f310261f700 1 --2- 172.21.15.63:0/3783424779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30fc102fd0 0x7f30fc0fcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:10.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.904+0000 7f310261f700 1 -- 172.21.15.63:0/3783424779 >> 172.21.15.63:0/3783424779 conn(0x7f30fc071530 msgr2=0x7f30fc0fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:10.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.904+0000 7f310261f700 1 -- 172.21.15.63:0/3783424779 shutdown_connections 2024-03-22T07:25:10.906 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:10.904+0000 7f310261f700 1 -- 172.21.15.63:0/3783424779 wait complete. 2024-03-22T07:25:11.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.007+0000 7f52b17fe700 1 -- 172.21.15.63:0/2660165763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52ac101d70 msgr2=0x7f52ac102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:11.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.007+0000 7f52b17fe700 1 --2- 172.21.15.63:0/2660165763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52ac101d70 0x7f52ac102140 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f529c004940 tx=0x7f529c009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:11.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.007+0000 7f52b17fe700 1 -- 172.21.15.63:0/2660165763 shutdown_connections 2024-03-22T07:25:11.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.007+0000 7f52b17fe700 1 --2- 172.21.15.63:0/2660165763 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52ac102710 0x7f52ac0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:11.009 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.007+0000 7f52b17fe700 1 --2- 172.21.15.63:0/2660165763 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52ac101d70 0x7f52ac102140 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:11.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.008+0000 7f52b17fe700 1 -- 172.21.15.63:0/2660165763 >> 172.21.15.63:0/2660165763 conn(0x7f52ac072bc0 msgr2=0x7f52ac072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:11.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.008+0000 7f52b17fe700 1 -- 172.21.15.63:0/2660165763 shutdown_connections 2024-03-22T07:25:11.010 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.008+0000 7f52b17fe700 1 -- 172.21.15.63:0/2660165763 wait complete. 2024-03-22T07:25:11.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.008+0000 7f52b17fe700 1 Processor -- start 2024-03-22T07:25:11.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.008+0000 7f52b17fe700 1 -- start start 2024-03-22T07:25:11.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.008+0000 7f52b17fe700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52ac101d70 0x7f52ac113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:11.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.008+0000 7f52b17fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52ac102710 0x7f52ac1140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:11.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.008+0000 7f52b17fe700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac1147b0 con 0x7f52ac102710 2024-03-22T07:25:11.012 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.008+0000 7f52b17fe700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac10db10 con 0x7f52ac101d70 2024-03-22T07:25:11.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.009+0000 7f52aa7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52ac102710 0x7f52ac1140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:11.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.009+0000 7f52aa7fc700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52ac102710 0x7f52ac1140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58226/0 (socket says 172.21.15.63:58226) 2024-03-22T07:25:11.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.009+0000 7f52aa7fc700 1 -- 172.21.15.63:0/1906446171 learned_addr learned my addr 172.21.15.63:0/1906446171 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:25:11.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.009+0000 7f52aaffd700 1 --2- 172.21.15.63:0/1906446171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52ac101d70 0x7f52ac113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:11.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.009+0000 7f52aa7fc700 1 -- 172.21.15.63:0/1906446171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52ac101d70 msgr2=0x7f52ac113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:11.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.009+0000 7f52aa7fc700 1 --2- 172.21.15.63:0/1906446171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52ac101d70 0x7f52ac113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:11.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.009+0000 7f52aa7fc700 1 -- 172.21.15.63:0/1906446171 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c005040 con 0x7f52ac102710 2024-03-22T07:25:11.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.009+0000 7f52aaffd700 1 --2- 172.21.15.63:0/1906446171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52ac101d70 0x7f52ac113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:25:11.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.009+0000 7f52aa7fc700 1 --2- 172.21.15.63:0/1906446171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52ac102710 0x7f52ac1140d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f52a000b790 tx=0x7f52a0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:11.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.009+0000 7f52937fe700 1 -- 172.21.15.63:0/1906446171 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52a000f6f0 con 0x7f52ac102710 2024-03-22T07:25:11.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.009+0000 7f52937fe700 1 -- 172.21.15.63:0/1906446171 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52a000fd20 con 0x7f52ac102710 2024-03-22T07:25:11.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.009+0000 7f52b17fe700 1 -- 172.21.15.63:0/1906446171 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ac10ddf0 con 0x7f52ac102710 2024-03-22T07:25:11.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.009+0000 7f52937fe700 1 -- 172.21.15.63:0/1906446171 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52a00103a0 con 0x7f52ac102710 2024-03-22T07:25:11.013 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.009+0000 7f52b17fe700 1 -- 172.21.15.63:0/1906446171 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ac10e3b0 con 0x7f52ac102710 2024-03-22T07:25:11.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.011+0000 7f52b17fe700 1 -- 172.21.15.63:0/1906446171 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ac0fd4c0 con 0x7f52ac102710 2024-03-22T07:25:11.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.011+0000 7f52937fe700 1 -- 172.21.15.63:0/1906446171 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f52a0011400 con 0x7f52ac102710 2024-03-22T07:25:11.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.012+0000 7f52937fe700 1 --2- 172.21.15.63:0/1906446171 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f529406d380 0x7f529406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:11.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.012+0000 7f52937fe700 1 -- 172.21.15.63:0/1906446171 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f52a008c7e0 con 0x7f52ac102710 2024-03-22T07:25:11.014 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.012+0000 7f52aaffd700 1 --2- 172.21.15.63:0/1906446171 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f529406d380 0x7f529406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:11.015 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.012+0000 7f52aaffd700 1 --2- 172.21.15.63:0/1906446171 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f529406d380 0x7f529406f830 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f529c004560 tx=0x7f529c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:11.017 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.016+0000 7f52937fe700 1 -- 172.21.15.63:0/1906446171 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f52a0059d40 con 0x7f52ac102710 2024-03-22T07:25:11.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.318+0000 7f52b17fe700 1 -- 172.21.15.63:0/1906446171 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f52ac108ee0 con 0x7f52ac102710 2024-03-22T07:25:11.320 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.319+0000 7f52937fe700 1 -- 172.21.15.63:0/1906446171 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f52a00598d0 con 0x7f52ac102710 2024-03-22T07:25:11.320 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:25:11.320 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:25:11.320 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:25:11.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:25:11.320 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:25:11.320 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:25:11.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:25:11.320 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:25:11.320 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 6 2024-03-22T07:25:11.320 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:25:11.321 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:25:11.321 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 3, 2024-03-22T07:25:11.321 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 1 2024-03-22T07:25:11.321 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:25:11.321 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:25:11.321 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 3, 2024-03-22T07:25:11.321 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 11 2024-03-22T07:25:11.321 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:25:11.321 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:25:11.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.321+0000 7f52b17fe700 1 -- 172.21.15.63:0/1906446171 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f529406d380 msgr2=0x7f529406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:11.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.321+0000 7f52b17fe700 1 --2- 172.21.15.63:0/1906446171 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f529406d380 0x7f529406f830 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f529c004560 tx=0x7f529c009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:11.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.321+0000 7f52b17fe700 1 -- 172.21.15.63:0/1906446171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52ac102710 msgr2=0x7f52ac1140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:11.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.321+0000 7f52b17fe700 1 --2- 172.21.15.63:0/1906446171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52ac102710 0x7f52ac1140d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f52a000b790 tx=0x7f52a0009220 comp rx=0 tx=0).stop 2024-03-22T07:25:11.322 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.321+0000 7f52b17fe700 1 -- 172.21.15.63:0/1906446171 shutdown_connections 2024-03-22T07:25:11.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.321+0000 7f52b17fe700 1 --2- 172.21.15.63:0/1906446171 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f529406d380 0x7f529406f830 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:11.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.321+0000 7f52b17fe700 1 --2- 172.21.15.63:0/1906446171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52ac101d70 0x7f52ac113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:11.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.321+0000 7f52b17fe700 1 --2- 172.21.15.63:0/1906446171 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52ac102710 0x7f52ac1140d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:11.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.321+0000 7f52b17fe700 1 -- 172.21.15.63:0/1906446171 >> 172.21.15.63:0/1906446171 conn(0x7f52ac072bc0 msgr2=0x7f52ac065900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:11.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.321+0000 7f52b17fe700 1 -- 172.21.15.63:0/1906446171 shutdown_connections 2024-03-22T07:25:11.323 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.321+0000 7f52b17fe700 1 -- 172.21.15.63:0/1906446171 wait complete. 2024-03-22T07:25:11.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.427+0000 7ff62f747700 1 -- 172.21.15.63:0/3430284114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6280fb4a0 msgr2=0x7ff6280fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:11.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.427+0000 7ff62f747700 1 --2- 172.21.15.63:0/3430284114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6280fb4a0 0x7ff6280fb910 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff618004940 tx=0x7ff618009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:11.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.427+0000 7ff62f747700 1 -- 172.21.15.63:0/3430284114 shutdown_connections 2024-03-22T07:25:11.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.427+0000 7ff62f747700 1 --2- 172.21.15.63:0/3430284114 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6280fb4a0 0x7ff6280fb910 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:11.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.427+0000 7ff62f747700 1 --2- 172.21.15.63:0/3430284114 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff628102e70 0x7ff628103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:11.428 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.427+0000 7ff62f747700 1 -- 172.21.15.63:0/3430284114 >> 172.21.15.63:0/3430284114 conn(0x7ff6280714c0 msgr2=0x7ff6280718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:11.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.427+0000 7ff62f747700 1 -- 172.21.15.63:0/3430284114 shutdown_connections 2024-03-22T07:25:11.429 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.427+0000 7ff62f747700 1 -- 172.21.15.63:0/3430284114 wait complete. 2024-03-22T07:25:11.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.428+0000 7ff62f747700 1 Processor -- start 2024-03-22T07:25:11.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.428+0000 7ff62f747700 1 -- start start 2024-03-22T07:25:11.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.428+0000 7ff62f747700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6280fb4a0 0x7ff628113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:11.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.428+0000 7ff62f747700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff628102e70 0x7ff6281140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:11.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.428+0000 7ff62f747700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff628114700 con 0x7ff6280fb4a0 2024-03-22T07:25:11.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.428+0000 7ff62f747700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62810daf0 con 0x7ff628102e70 2024-03-22T07:25:11.430 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.428+0000 7ff62d4e3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6280fb4a0 0x7ff628113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.428+0000 7ff62d4e3700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6280fb4a0 0x7ff628113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58244/0 (socket says 172.21.15.63:58244) 2024-03-22T07:25:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.428+0000 7ff62d4e3700 1 -- 172.21.15.63:0/1815470355 learned_addr learned my addr 172.21.15.63:0/1815470355 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:25:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.428+0000 7ff62cce2700 1 --2- 172.21.15.63:0/1815470355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff628102e70 0x7ff6281140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.428+0000 7ff62d4e3700 1 -- 172.21.15.63:0/1815470355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff628102e70 msgr2=0x7ff6281140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.428+0000 7ff62d4e3700 1 --2- 172.21.15.63:0/1815470355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff628102e70 0x7ff6281140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.428+0000 7ff62d4e3700 1 -- 172.21.15.63:0/1815470355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff618005040 con 0x7ff6280fb4a0 2024-03-22T07:25:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.428+0000 7ff62d4e3700 1 --2- 172.21.15.63:0/1815470355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6280fb4a0 0x7ff628113b70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c8c0 tx=0x7ff6240044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.429+0000 7ff61dffb700 1 -- 172.21.15.63:0/1815470355 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff62400a980 con 0x7ff6280fb4a0 2024-03-22T07:25:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.429+0000 7ff62f747700 1 -- 172.21.15.63:0/1815470355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff62810ddd0 con 0x7ff6280fb4a0 2024-03-22T07:25:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.429+0000 7ff61dffb700 1 -- 172.21.15.63:0/1815470355 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff62400ce60 con 0x7ff6280fb4a0 2024-03-22T07:25:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.429+0000 7ff61dffb700 1 -- 172.21.15.63:0/1815470355 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6240115e0 con 0x7ff6280fb4a0 2024-03-22T07:25:11.431 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.429+0000 7ff62f747700 1 -- 172.21.15.63:0/1815470355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62810e390 con 0x7ff6280fb4a0 2024-03-22T07:25:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.430+0000 7ff62f747700 1 -- 172.21.15.63:0/1815470355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff628100110 con 0x7ff6280fb4a0 2024-03-22T07:25:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.430+0000 7ff61dffb700 1 -- 172.21.15.63:0/1815470355 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7ff624012400 con 0x7ff6280fb4a0 2024-03-22T07:25:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.431+0000 7ff61dffb700 1 --2- 172.21.15.63:0/1815470355 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff61406d380 0x7ff61406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.431+0000 7ff61dffb700 1 -- 172.21.15.63:0/1815470355 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff62408ca40 con 0x7ff6280fb4a0 2024-03-22T07:25:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.431+0000 7ff62cce2700 1 --2- 172.21.15.63:0/1815470355 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff61406d380 0x7ff61406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:11.432 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.431+0000 7ff62cce2700 1 --2- 172.21.15.63:0/1815470355 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff61406d380 0x7ff61406f830 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff618000c00 tx=0x7ff618009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:11.436 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.435+0000 7ff61dffb700 1 -- 172.21.15.63:0/1815470355 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff62405a020 con 0x7ff6280fb4a0 2024-03-22T07:25:11.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:11 smithi063 ceph-mon[112784]: pgmap v176: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-03-22T07:25:11.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:11 smithi063 ceph-mon[112784]: from='client.44285 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:25:11.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:11 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:11.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:11 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:11.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:11 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:25:11.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:11 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/1906446171' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:11.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.702+0000 7ff62f747700 1 -- 172.21.15.63:0/1815470355 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff62810f1c0 con 0x7ff6280fb4a0 2024-03-22T07:25:11.703 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.702+0000 7ff61dffb700 1 -- 172.21.15.63:0/1815470355 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1835 (secure 0 0 0) 0x7ff6240126b0 con 0x7ff6280fb4a0 2024-03-22T07:25:11.703 INFO:teuthology.orchestra.run.smithi063.stdout:e29 2024-03-22T07:25:11.704 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:25:11.704 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:25:11.704 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-03-22T07:25:11.704 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:25:11.704 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:25:11.704 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-03-22T07:25:11.704 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 26 2024-03-22T07:25:11.704 INFO:teuthology.orchestra.run.smithi063.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-22T07:25:11.704 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-03-22T07:13:29.602905+0000 2024-03-22T07:25:11.704 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-03-22T07:25:00.991999+0000 2024-03-22T07:25:11.704 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-03-22T07:25:11.704 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-03-22T07:25:11.704 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-03-22T07:25:11.704 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-03-22T07:25:11.704 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-03-22T07:25:11.704 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-03-22T07:25:11.705 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-03-22T07:25:11.705 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-03-22T07:25:11.705 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 83 2024-03-22T07:25:11.705 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:25:11.705 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 2 2024-03-22T07:25:11.705 INFO:teuthology.orchestra.run.smithi063.stdout:in 0,1 2024-03-22T07:25:11.705 INFO:teuthology.orchestra.run.smithi063.stdout:up {} 2024-03-22T07:25:11.705 INFO:teuthology.orchestra.run.smithi063.stdout:failed 0,1 2024-03-22T07:25:11.705 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-03-22T07:25:11.705 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-03-22T07:25:11.705 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-03-22T07:25:11.705 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-03-22T07:25:11.705 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-03-22T07:25:11.705 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-03-22T07:25:11.705 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-03-22T07:25:11.705 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-03-22T07:25:11.705 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:25:11.706 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:25:11.706 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-03-22T07:25:11.706 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:25:11.706 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.xeelpk{-1:34388} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6828/1202791208,v1:172.21.15.63:6829/1202791208] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:25:11.706 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.oytapz{-1:34400} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1171423403,v1:172.21.15.63:6827/1171423403] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T07:25:11.706 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.cijxah{-1:44273} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.69:6826/675976207,v1:172.21.15.69:6827/675976207] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:25:11.706 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.wwtsjn{-1:44279} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.69:6824/2785357568,v1:172.21.15.69:6825/2785357568] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:25:11.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.704+0000 7ff62f747700 1 -- 172.21.15.63:0/1815470355 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff61406d380 msgr2=0x7ff61406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:11.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.704+0000 7ff62f747700 1 --2- 172.21.15.63:0/1815470355 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff61406d380 0x7ff61406f830 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff618000c00 tx=0x7ff618009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:11.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.704+0000 7ff62f747700 1 -- 172.21.15.63:0/1815470355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6280fb4a0 msgr2=0x7ff628113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:11.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.704+0000 7ff62f747700 1 --2- 172.21.15.63:0/1815470355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6280fb4a0 0x7ff628113b70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c8c0 tx=0x7ff6240044c0 comp rx=0 tx=0).stop 2024-03-22T07:25:11.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.704+0000 7ff62f747700 1 -- 172.21.15.63:0/1815470355 shutdown_connections 2024-03-22T07:25:11.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.704+0000 7ff62f747700 1 --2- 172.21.15.63:0/1815470355 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff61406d380 0x7ff61406f830 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:11.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.704+0000 7ff62f747700 1 --2- 172.21.15.63:0/1815470355 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff6280fb4a0 0x7ff628113b70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:11.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.704+0000 7ff62f747700 1 --2- 172.21.15.63:0/1815470355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff628102e70 0x7ff6281140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:11.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.704+0000 7ff62f747700 1 -- 172.21.15.63:0/1815470355 >> 172.21.15.63:0/1815470355 conn(0x7ff6280714c0 msgr2=0x7ff6280fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:11.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.705+0000 7ff62f747700 1 -- 172.21.15.63:0/1815470355 shutdown_connections 2024-03-22T07:25:11.707 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.705+0000 7ff62f747700 1 -- 172.21.15.63:0/1815470355 wait complete. 2024-03-22T07:25:11.707 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 29 2024-03-22T07:25:11.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:11 smithi069 ceph-mon[83631]: pgmap v176: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-03-22T07:25:11.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:11 smithi069 ceph-mon[83631]: from='client.44285 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:25:11.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:11 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:11.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:11 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:11.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:11 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:25:11.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:11 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/1906446171' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:11.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.812+0000 7ff2cb97b700 1 -- 172.21.15.63:0/2083117630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2c4102e70 msgr2=0x7ff2c4103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:11.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.812+0000 7ff2cb97b700 1 --2- 172.21.15.63:0/2083117630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2c4102e70 0x7ff2c4103240 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0004940 tx=0x7ff2c0009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:11.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.812+0000 7ff2cb97b700 1 -- 172.21.15.63:0/2083117630 shutdown_connections 2024-03-22T07:25:11.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.812+0000 7ff2cb97b700 1 --2- 172.21.15.63:0/2083117630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2c40fb4a0 0x7ff2c40fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:11.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.812+0000 7ff2cb97b700 1 --2- 172.21.15.63:0/2083117630 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2c4102e70 0x7ff2c4103240 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:11.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.812+0000 7ff2cb97b700 1 -- 172.21.15.63:0/2083117630 >> 172.21.15.63:0/2083117630 conn(0x7ff2c40714c0 msgr2=0x7ff2c40718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:11.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.812+0000 7ff2cb97b700 1 -- 172.21.15.63:0/2083117630 shutdown_connections 2024-03-22T07:25:11.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.812+0000 7ff2cb97b700 1 -- 172.21.15.63:0/2083117630 wait complete. 2024-03-22T07:25:11.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.812+0000 7ff2cb97b700 1 Processor -- start 2024-03-22T07:25:11.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.812+0000 7ff2cb97b700 1 -- start start 2024-03-22T07:25:11.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.812+0000 7ff2cb97b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2c40fb4a0 0x7ff2c4113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:11.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.812+0000 7ff2cb97b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2c4102e70 0x7ff2c4114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:11.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.813+0000 7ff2cb97b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c41146e0 con 0x7ff2c4102e70 2024-03-22T07:25:11.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.813+0000 7ff2cb97b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c410dad0 con 0x7ff2c40fb4a0 2024-03-22T07:25:11.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.813+0000 7ff2c8f16700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2c4102e70 0x7ff2c4114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:11.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.813+0000 7ff2c8f16700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2c4102e70 0x7ff2c4114090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58262/0 (socket says 172.21.15.63:58262) 2024-03-22T07:25:11.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.813+0000 7ff2c8f16700 1 -- 172.21.15.63:0/1448790121 learned_addr learned my addr 172.21.15.63:0/1448790121 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:25:11.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.813+0000 7ff2c9717700 1 --2- 172.21.15.63:0/1448790121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2c40fb4a0 0x7ff2c4113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:11.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.813+0000 7ff2c8f16700 1 -- 172.21.15.63:0/1448790121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2c40fb4a0 msgr2=0x7ff2c4113b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:11.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.813+0000 7ff2c8f16700 1 --2- 172.21.15.63:0/1448790121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2c40fb4a0 0x7ff2c4113b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:11.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.813+0000 7ff2c8f16700 1 -- 172.21.15.63:0/1448790121 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c0005040 con 0x7ff2c4102e70 2024-03-22T07:25:11.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.813+0000 7ff2c8f16700 1 --2- 172.21.15.63:0/1448790121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2c4102e70 0x7ff2c4114090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400c830 tx=0x7ff2b4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:11.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.813+0000 7ff2b9ffb700 1 -- 172.21.15.63:0/1448790121 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2b40106f0 con 0x7ff2c4102e70 2024-03-22T07:25:11.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.813+0000 7ff2b9ffb700 1 -- 172.21.15.63:0/1448790121 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2b4010d20 con 0x7ff2c4102e70 2024-03-22T07:25:11.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.814+0000 7ff2cb97b700 1 -- 172.21.15.63:0/1448790121 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c410ddb0 con 0x7ff2c4102e70 2024-03-22T07:25:11.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.814+0000 7ff2b9ffb700 1 -- 172.21.15.63:0/1448790121 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2b40113a0 con 0x7ff2c4102e70 2024-03-22T07:25:11.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.814+0000 7ff2cb97b700 1 -- 172.21.15.63:0/1448790121 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c410e370 con 0x7ff2c4102e70 2024-03-22T07:25:11.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.815+0000 7ff2cb97b700 1 -- 172.21.15.63:0/1448790121 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c4100110 con 0x7ff2c4102e70 2024-03-22T07:25:11.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.815+0000 7ff2b9ffb700 1 -- 172.21.15.63:0/1448790121 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7ff2b4010850 con 0x7ff2c4102e70 2024-03-22T07:25:11.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.816+0000 7ff2b9ffb700 1 --2- 172.21.15.63:0/1448790121 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff2b006d390 0x7ff2b006f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:11.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.816+0000 7ff2b9ffb700 1 -- 172.21.15.63:0/1448790121 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff2b408c820 con 0x7ff2c4102e70 2024-03-22T07:25:11.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.816+0000 7ff2c9717700 1 --2- 172.21.15.63:0/1448790121 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff2b006d390 0x7ff2b006f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:11.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.816+0000 7ff2c9717700 1 --2- 172.21.15.63:0/1448790121 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff2b006d390 0x7ff2b006f840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000b3b0 tx=0x7ff2c0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:11.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:11.820+0000 7ff2b9ffb700 1 -- 172.21.15.63:0/1448790121 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff2b4059ea0 con 0x7ff2c4102e70 2024-03-22T07:25:12.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.050+0000 7ff2cb97b700 1 -- 172.21.15.63:0/1448790121 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2c4113820 con 0x7ff2b006d390 2024-03-22T07:25:12.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.052+0000 7ff2b9ffb700 1 -- 172.21.15.63:0/1448790121 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff2c4113820 con 0x7ff2b006d390 2024-03-22T07:25:12.053 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:25:12.053 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:25:12.053 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:25:12.053 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:25:12.053 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-03-22T07:25:12.053 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-03-22T07:25:12.053 INFO:teuthology.orchestra.run.smithi063.stdout: "osd", 2024-03-22T07:25:12.053 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr", 2024-03-22T07:25:12.053 INFO:teuthology.orchestra.run.smithi063.stdout: "mon" 2024-03-22T07:25:12.053 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-03-22T07:25:12.053 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "12/21 daemons upgraded", 2024-03-22T07:25:12.053 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mds daemons", 2024-03-22T07:25:12.053 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:25:12.053 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:25:12.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.054+0000 7ff2cb97b700 1 -- 172.21.15.63:0/1448790121 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff2b006d390 msgr2=0x7ff2b006f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:12.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.054+0000 7ff2cb97b700 1 --2- 172.21.15.63:0/1448790121 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff2b006d390 0x7ff2b006f840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000b3b0 tx=0x7ff2c0009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:12.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.054+0000 7ff2cb97b700 1 -- 172.21.15.63:0/1448790121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2c4102e70 msgr2=0x7ff2c4114090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:12.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.054+0000 7ff2cb97b700 1 --2- 172.21.15.63:0/1448790121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2c4102e70 0x7ff2c4114090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400c830 tx=0x7ff2b4004470 comp rx=0 tx=0).stop 2024-03-22T07:25:12.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.054+0000 7ff2cb97b700 1 -- 172.21.15.63:0/1448790121 shutdown_connections 2024-03-22T07:25:12.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.054+0000 7ff2cb97b700 1 --2- 172.21.15.63:0/1448790121 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff2b006d390 0x7ff2b006f840 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:12.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.054+0000 7ff2cb97b700 1 --2- 172.21.15.63:0/1448790121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff2c40fb4a0 0x7ff2c4113b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:12.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.054+0000 7ff2cb97b700 1 --2- 172.21.15.63:0/1448790121 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff2c4102e70 0x7ff2c4114090 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:12.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.054+0000 7ff2cb97b700 1 -- 172.21.15.63:0/1448790121 >> 172.21.15.63:0/1448790121 conn(0x7ff2c40714c0 msgr2=0x7ff2c40fa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:12.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.054+0000 7ff2cb97b700 1 -- 172.21.15.63:0/1448790121 shutdown_connections 2024-03-22T07:25:12.056 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.054+0000 7ff2cb97b700 1 -- 172.21.15.63:0/1448790121 wait complete. 2024-03-22T07:25:12.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.178+0000 7f833c93b700 1 -- 172.21.15.63:0/3722662442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8338102e10 msgr2=0x7f83381031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:12.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.178+0000 7f833c93b700 1 --2- 172.21.15.63:0/3722662442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8338102e10 0x7f83381031e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f832c004940 tx=0x7f832c009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:12.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.178+0000 7f833c93b700 1 -- 172.21.15.63:0/3722662442 shutdown_connections 2024-03-22T07:25:12.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.178+0000 7f833c93b700 1 --2- 172.21.15.63:0/3722662442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83381037b0 0x7f83380fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:12.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.178+0000 7f833c93b700 1 --2- 172.21.15.63:0/3722662442 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8338102e10 0x7f83381031e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:12.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.178+0000 7f833c93b700 1 -- 172.21.15.63:0/3722662442 >> 172.21.15.63:0/3722662442 conn(0x7f83380f8980 msgr2=0x7f83380fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:12.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.178+0000 7f833c93b700 1 -- 172.21.15.63:0/3722662442 shutdown_connections 2024-03-22T07:25:12.180 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.179+0000 7f833c93b700 1 -- 172.21.15.63:0/3722662442 wait complete. 2024-03-22T07:25:12.181 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.179+0000 7f833c93b700 1 Processor -- start 2024-03-22T07:25:12.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.179+0000 7f833c93b700 1 -- start start 2024-03-22T07:25:12.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.179+0000 7f833c93b700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8338102e10 0x7f8338113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:12.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.179+0000 7f833c93b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83381037b0 0x7f8338114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:12.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.179+0000 7f833c93b700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83381146b0 con 0x7f83381037b0 2024-03-22T07:25:12.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.179+0000 7f833c93b700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833810daa0 con 0x7f8338102e10 2024-03-22T07:25:12.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.180+0000 7f8335d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83381037b0 0x7f8338114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:12.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.180+0000 7f8335d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83381037b0 0x7f8338114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:58288/0 (socket says 172.21.15.63:58288) 2024-03-22T07:25:12.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.180+0000 7f8335d9b700 1 -- 172.21.15.63:0/1763243078 learned_addr learned my addr 172.21.15.63:0/1763243078 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:25:12.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.180+0000 7f833659c700 1 --2- 172.21.15.63:0/1763243078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8338102e10 0x7f8338113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:12.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.180+0000 7f8335d9b700 1 -- 172.21.15.63:0/1763243078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8338102e10 msgr2=0x7f8338113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:12.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.180+0000 7f8335d9b700 1 --2- 172.21.15.63:0/1763243078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8338102e10 0x7f8338113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:12.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.180+0000 7f8335d9b700 1 -- 172.21.15.63:0/1763243078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c005040 con 0x7f83381037b0 2024-03-22T07:25:12.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.180+0000 7f8335d9b700 1 --2- 172.21.15.63:0/1763243078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83381037b0 0x7f8338114060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f832800eb40 tx=0x7f832800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:12.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.180+0000 7f8326ffd700 1 -- 172.21.15.63:0/1763243078 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8328007cc0 con 0x7f83381037b0 2024-03-22T07:25:12.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.180+0000 7f833c93b700 1 -- 172.21.15.63:0/1763243078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833810dd20 con 0x7f83381037b0 2024-03-22T07:25:12.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.180+0000 7f8326ffd700 1 -- 172.21.15.63:0/1763243078 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f832800f040 con 0x7f83381037b0 2024-03-22T07:25:12.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.180+0000 7f8326ffd700 1 -- 172.21.15.63:0/1763243078 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83280095e0 con 0x7f83381037b0 2024-03-22T07:25:12.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.180+0000 7f833c93b700 1 -- 172.21.15.63:0/1763243078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833810e260 con 0x7f83381037b0 2024-03-22T07:25:12.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.181+0000 7f833c93b700 1 -- 172.21.15.63:0/1763243078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8338100120 con 0x7f83381037b0 2024-03-22T07:25:12.183 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.182+0000 7f8326ffd700 1 -- 172.21.15.63:0/1763243078 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f8328009740 con 0x7f83381037b0 2024-03-22T07:25:12.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.182+0000 7f8326ffd700 1 --2- 172.21.15.63:0/1763243078 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f832006d380 0x7f832006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:12.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.183+0000 7f8326ffd700 1 -- 172.21.15.63:0/1763243078 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f832808c430 con 0x7f83381037b0 2024-03-22T07:25:12.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.183+0000 7f833659c700 1 --2- 172.21.15.63:0/1763243078 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f832006d380 0x7f832006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:12.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.183+0000 7f833659c700 1 --2- 172.21.15.63:0/1763243078 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f832006d380 0x7f832006f830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f832c00b0d0 tx=0x7f832c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:12.188 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.187+0000 7f8326ffd700 1 -- 172.21.15.63:0/1763243078 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8328059a40 con 0x7f83381037b0 2024-03-22T07:25:12.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.487+0000 7f833c93b700 1 -- 172.21.15.63:0/1763243078 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f833810f170 con 0x7f83381037b0 2024-03-22T07:25:12.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.488+0000 7f8326ffd700 1 -- 172.21.15.63:0/1763243078 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f83280595d0 con 0x7f83381037b0 2024-03-22T07:25:12.489 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-22T07:25:12.489 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-22T07:25:12.489 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs is degraded 2024-03-22T07:25:12.489 INFO:teuthology.orchestra.run.smithi063.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-22T07:25:12.490 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-22T07:25:12.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.490+0000 7f833c93b700 1 -- 172.21.15.63:0/1763243078 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f832006d380 msgr2=0x7f832006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:12.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.490+0000 7f833c93b700 1 --2- 172.21.15.63:0/1763243078 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f832006d380 0x7f832006f830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f832c00b0d0 tx=0x7f832c009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:12.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.490+0000 7f833c93b700 1 -- 172.21.15.63:0/1763243078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83381037b0 msgr2=0x7f8338114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:12.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.490+0000 7f833c93b700 1 --2- 172.21.15.63:0/1763243078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83381037b0 0x7f8338114060 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f832800eb40 tx=0x7f832800b040 comp rx=0 tx=0).stop 2024-03-22T07:25:12.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.490+0000 7f833c93b700 1 -- 172.21.15.63:0/1763243078 shutdown_connections 2024-03-22T07:25:12.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.490+0000 7f833c93b700 1 --2- 172.21.15.63:0/1763243078 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f832006d380 0x7f832006f830 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:12.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.490+0000 7f833c93b700 1 --2- 172.21.15.63:0/1763243078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8338102e10 0x7f8338113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:12.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.490+0000 7f833c93b700 1 --2- 172.21.15.63:0/1763243078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f83381037b0 0x7f8338114060 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:12.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.490+0000 7f833c93b700 1 -- 172.21.15.63:0/1763243078 >> 172.21.15.63:0/1763243078 conn(0x7f83380f8980 msgr2=0x7f83381073b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:12.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.490+0000 7f833c93b700 1 -- 172.21.15.63:0/1763243078 shutdown_connections 2024-03-22T07:25:12.492 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:12.490+0000 7f833c93b700 1 -- 172.21.15.63:0/1763243078 wait complete. 2024-03-22T07:25:12.613 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:12 smithi063 ceph-mon[112784]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:25:12.613 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:12 smithi063 ceph-mon[112784]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:25:12.613 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:12 smithi063 ceph-mon[112784]: mds.? [v2:172.21.15.63:6826/1171423403,v1:172.21.15.63:6827/1171423403] up:boot 2024-03-22T07:25:12.613 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:12 smithi063 ceph-mon[112784]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:25:12.613 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.oytapz"}]: dispatch 2024-03-22T07:25:12.613 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:12.614 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:12 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/1815470355' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:25:12.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:12 smithi069 ceph-mon[83631]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:25:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:12 smithi069 ceph-mon[83631]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:25:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:12 smithi069 ceph-mon[83631]: mds.? [v2:172.21.15.63:6826/1171423403,v1:172.21.15.63:6827/1171423403] up:boot 2024-03-22T07:25:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:12 smithi069 ceph-mon[83631]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:25:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.oytapz"}]: dispatch 2024-03-22T07:25:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:12 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/1815470355' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:25:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:13 smithi063 ceph-mon[112784]: pgmap v177: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-03-22T07:25:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:13 smithi063 ceph-mon[112784]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:25:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:13 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/1763243078' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:25:13.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:13 smithi069 ceph-mon[83631]: pgmap v177: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-03-22T07:25:13.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:13 smithi069 ceph-mon[83631]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:25:13.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:13 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/1763243078' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:25:15.611 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:15 smithi063 ceph-mon[112784]: pgmap v178: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:15.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:15 smithi069 ceph-mon[83631]: pgmap v178: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:17.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:16 smithi069 ceph-mon[83631]: pgmap v179: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:17.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:16 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:17.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:16 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:17.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:16 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:17.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:16 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:25:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:16 smithi063 ceph-mon[112784]: pgmap v179: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:16 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:16 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:16 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:16 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:25:18.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:18 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:18.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:18 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:18.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:18 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:18.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:18 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:19.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:19 smithi063 ceph-mon[112784]: pgmap v180: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:19.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:19 smithi069 ceph-mon[83631]: pgmap v180: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:21.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:21 smithi063 ceph-mon[112784]: pgmap v181: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:21.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:21 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:21.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:21 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:21.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:21 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:25:21.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:21 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:25:21.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:21 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:21.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:21 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:25:21.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:21 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:21.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:21 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:21.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:21 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:21.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:21 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:21.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:21 smithi069 ceph-mon[83631]: pgmap v181: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:21.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:21 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:21.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:21 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:21.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:21 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:25:21.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:21 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:25:21.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:21 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:21.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:21 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:25:21.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:21 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:21.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:21 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:21.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:21 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:21.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:21 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:23.310 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:23 smithi063 ceph-mon[112784]: pgmap v182: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:23.310 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:23 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:23.310 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:23 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.xeelpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:25:23.310 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:23 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:25:23.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:23 smithi069 ceph-mon[83631]: pgmap v182: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:23.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:23 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:23.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:23 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi063.xeelpk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:25:23.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:23 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:25:24.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:24 smithi063 ceph-mon[112784]: Upgrade: Updating mds.cephfs.smithi063.xeelpk 2024-03-22T07:25:24.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:24 smithi063 ceph-mon[112784]: Deploying daemon mds.cephfs.smithi063.xeelpk on smithi063 2024-03-22T07:25:24.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:24 smithi069 ceph-mon[83631]: Upgrade: Updating mds.cephfs.smithi063.xeelpk 2024-03-22T07:25:24.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:24 smithi069 ceph-mon[83631]: Deploying daemon mds.cephfs.smithi063.xeelpk on smithi063 2024-03-22T07:25:25.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:25 smithi063 ceph-mon[112784]: pgmap v183: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:25.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:25 smithi069 ceph-mon[83631]: pgmap v183: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:26 smithi063 ceph-mon[112784]: osdmap e85: 6 total, 6 up, 6 in 2024-03-22T07:25:26.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:26 smithi063 ceph-mon[112784]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-22T07:25:26.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:26 smithi069 ceph-mon[83631]: osdmap e85: 6 total, 6 up, 6 in 2024-03-22T07:25:26.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:26 smithi069 ceph-mon[83631]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-22T07:25:27.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:27 smithi063 ceph-mon[112784]: pgmap v185: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:27.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:27 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.xeelpk"}]: dispatch 2024-03-22T07:25:27.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:27 smithi069 ceph-mon[83631]: pgmap v185: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:27.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:27 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.xeelpk"}]: dispatch 2024-03-22T07:25:29.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:29 smithi063 ceph-mon[112784]: pgmap v186: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:29.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:29 smithi069 ceph-mon[83631]: pgmap v186: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:31.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:31 smithi063 ceph-mon[112784]: pgmap v187: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:31.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:31 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:31.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:31 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:31.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:31 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:25:31.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:31 smithi063 ceph-mon[112784]: mds.? [v2:172.21.15.63:6828/1190308435,v1:172.21.15.63:6829/1190308435] up:boot 2024-03-22T07:25:31.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:31 smithi063 ceph-mon[112784]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:25:31.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:31 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.xeelpk"}]: dispatch 2024-03-22T07:25:31.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:31 smithi069 ceph-mon[83631]: pgmap v187: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:31.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:31 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:31.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:31 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:31.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:31 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:25:31.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:31 smithi069 ceph-mon[83631]: mds.? [v2:172.21.15.63:6828/1190308435,v1:172.21.15.63:6829/1190308435] up:boot 2024-03-22T07:25:31.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:31 smithi069 ceph-mon[83631]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:25:31.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:31 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi063.xeelpk"}]: dispatch 2024-03-22T07:25:32.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:32 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:25:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:32 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:25:33.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:33 smithi063 ceph-mon[112784]: pgmap v188: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:33.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:33 smithi069 ceph-mon[83631]: pgmap v188: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:35.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:35 smithi069 ceph-mon[83631]: pgmap v189: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:35.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:35 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:35.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:35 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:35.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:35 smithi063 ceph-mon[112784]: pgmap v189: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:35.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:35 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:35.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:35 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:37.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:36 smithi069 ceph-mon[83631]: pgmap v190: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:37.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:36 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:37.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:36 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:37.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:36 smithi063 ceph-mon[112784]: pgmap v190: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:37.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:36 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:37.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:36 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:39.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:38 smithi069 ceph-mon[83631]: pgmap v191: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:39.295 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:38 smithi063 ceph-mon[112784]: pgmap v191: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:40.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:40 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:40.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:40 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:40.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:40 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:25:40.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:40 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:25:40.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:40 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:40.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:40 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:25:40.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:40 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:40.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:40 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:40.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:40 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:40.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:40 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:40.728 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:40 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:40.728 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:40 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:40.728 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:40 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:25:40.728 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:40 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:25:40.728 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:40 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:40.728 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:40 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:25:40.728 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:40 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:40.728 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:40 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:40.728 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:40 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:40.728 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:40 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:41 smithi063 ceph-mon[112784]: pgmap v192: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:41 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:41 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi069.wwtsjn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:25:41.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:41 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:25:41.629 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:41 smithi069 ceph-mon[83631]: pgmap v192: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:41.629 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:41 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:41.629 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:41 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi069.wwtsjn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:25:41.629 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:41 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:25:42.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.608+0000 7fadcd81c700 1 -- 172.21.15.63:0/412795984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc81037c0 msgr2=0x7fadc80fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:42.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.608+0000 7fadcd81c700 1 --2- 172.21.15.63:0/412795984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc81037c0 0x7fadc80fce30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fadb0004940 tx=0x7fadb0009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:42.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.608+0000 7fadcd81c700 1 -- 172.21.15.63:0/412795984 shutdown_connections 2024-03-22T07:25:42.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.608+0000 7fadcd81c700 1 --2- 172.21.15.63:0/412795984 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc81037c0 0x7fadc80fce30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:42.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.608+0000 7fadcd81c700 1 --2- 172.21.15.63:0/412795984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadc8102e20 0x7fadc81031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:42.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.608+0000 7fadcd81c700 1 -- 172.21.15.63:0/412795984 >> 172.21.15.63:0/412795984 conn(0x7fadc80f89d0 msgr2=0x7fadc80fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:42.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.608+0000 7fadcd81c700 1 -- 172.21.15.63:0/412795984 shutdown_connections 2024-03-22T07:25:42.609 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.608+0000 7fadcd81c700 1 -- 172.21.15.63:0/412795984 wait complete. 2024-03-22T07:25:42.610 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.609+0000 7fadcd81c700 1 Processor -- start 2024-03-22T07:25:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.609+0000 7fadcd81c700 1 -- start start 2024-03-22T07:25:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.609+0000 7fadcd81c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc8102e20 0x7fadc8113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.609+0000 7fadcd81c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadc81037c0 0x7fadc81140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.609+0000 7fadcd81c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc8114740 con 0x7fadc8102e20 2024-03-22T07:25:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.609+0000 7fadcd81c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc810db30 con 0x7fadc81037c0 2024-03-22T07:25:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.610+0000 7fadc6ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc8102e20 0x7fadc8113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.610+0000 7fadc6ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc8102e20 0x7fadc8113bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44138/0 (socket says 172.21.15.63:44138) 2024-03-22T07:25:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.610+0000 7fadc6ffd700 1 -- 172.21.15.63:0/2056223857 learned_addr learned my addr 172.21.15.63:0/2056223857 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:25:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.610+0000 7fadc67fc700 1 --2- 172.21.15.63:0/2056223857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadc81037c0 0x7fadc81140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.610+0000 7fadc6ffd700 1 -- 172.21.15.63:0/2056223857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadc81037c0 msgr2=0x7fadc81140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.610+0000 7fadc6ffd700 1 --2- 172.21.15.63:0/2056223857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadc81037c0 0x7fadc81140f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.610+0000 7fadc6ffd700 1 -- 172.21.15.63:0/2056223857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb0005040 con 0x7fadc8102e20 2024-03-22T07:25:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.610+0000 7fadc6ffd700 1 --2- 172.21.15.63:0/2056223857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc8102e20 0x7fadc8113bb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fadc000cb10 tx=0x7fadc0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.610+0000 7fadbf7fe700 1 -- 172.21.15.63:0/2056223857 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc0004200 con 0x7fadc8102e20 2024-03-22T07:25:42.611 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.610+0000 7fadbf7fe700 1 -- 172.21.15.63:0/2056223857 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadc000ac70 con 0x7fadc8102e20 2024-03-22T07:25:42.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.610+0000 7fadcd81c700 1 -- 172.21.15.63:0/2056223857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc810de10 con 0x7fadc8102e20 2024-03-22T07:25:42.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.610+0000 7fadbf7fe700 1 -- 172.21.15.63:0/2056223857 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc0003d30 con 0x7fadc8102e20 2024-03-22T07:25:42.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.610+0000 7fadcd81c700 1 -- 172.21.15.63:0/2056223857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc810e3a0 con 0x7fadc8102e20 2024-03-22T07:25:42.612 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.611+0000 7fadcd81c700 1 -- 172.21.15.63:0/2056223857 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc81052f0 con 0x7fadc8102e20 2024-03-22T07:25:42.612 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:42 smithi063 ceph-mon[112784]: Upgrade: Updating mds.cephfs.smithi069.wwtsjn 2024-03-22T07:25:42.612 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:42 smithi063 ceph-mon[112784]: Deploying daemon mds.cephfs.smithi069.wwtsjn on smithi069 2024-03-22T07:25:42.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.612+0000 7fadbf7fe700 1 -- 172.21.15.63:0/2056223857 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fadc000a7a0 con 0x7fadc8102e20 2024-03-22T07:25:42.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.613+0000 7fadbf7fe700 1 --2- 172.21.15.63:0/2056223857 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fadb406d390 0x7fadb406f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:42.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.613+0000 7fadbf7fe700 1 -- 172.21.15.63:0/2056223857 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fadc008ce40 con 0x7fadc8102e20 2024-03-22T07:25:42.613 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.613+0000 7fadc67fc700 1 --2- 172.21.15.63:0/2056223857 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fadb406d390 0x7fadb406f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:42.614 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.614+0000 7fadc67fc700 1 --2- 172.21.15.63:0/2056223857 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fadb406d390 0x7fadb406f840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fadc810ee10 tx=0x7fadb0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:42.617 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.618+0000 7fadbf7fe700 1 -- 172.21.15.63:0/2056223857 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fadc005a3a0 con 0x7fadc8102e20 2024-03-22T07:25:42.632 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:42 smithi069 ceph-mon[83631]: Upgrade: Updating mds.cephfs.smithi069.wwtsjn 2024-03-22T07:25:42.632 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:42 smithi069 ceph-mon[83631]: Deploying daemon mds.cephfs.smithi069.wwtsjn on smithi069 2024-03-22T07:25:42.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.843+0000 7fadcd81c700 1 -- 172.21.15.63:0/2056223857 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fadc8109f00 con 0x7fadb406d390 2024-03-22T07:25:42.844 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.844+0000 7fadbf7fe700 1 -- 172.21.15.63:0/2056223857 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fadc8109f00 con 0x7fadb406d390 2024-03-22T07:25:42.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.846+0000 7fadcd81c700 1 -- 172.21.15.63:0/2056223857 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fadb406d390 msgr2=0x7fadb406f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:42.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.846+0000 7fadcd81c700 1 --2- 172.21.15.63:0/2056223857 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fadb406d390 0x7fadb406f840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fadc810ee10 tx=0x7fadb0009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:42.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.846+0000 7fadcd81c700 1 -- 172.21.15.63:0/2056223857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc8102e20 msgr2=0x7fadc8113bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:42.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.846+0000 7fadcd81c700 1 --2- 172.21.15.63:0/2056223857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc8102e20 0x7fadc8113bb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fadc000cb10 tx=0x7fadc0007f90 comp rx=0 tx=0).stop 2024-03-22T07:25:42.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.846+0000 7fadcd81c700 1 -- 172.21.15.63:0/2056223857 shutdown_connections 2024-03-22T07:25:42.846 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.846+0000 7fadcd81c700 1 --2- 172.21.15.63:0/2056223857 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fadb406d390 0x7fadb406f840 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:42.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.846+0000 7fadcd81c700 1 --2- 172.21.15.63:0/2056223857 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fadc8102e20 0x7fadc8113bb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:42.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.846+0000 7fadcd81c700 1 --2- 172.21.15.63:0/2056223857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadc81037c0 0x7fadc81140f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:42.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.846+0000 7fadcd81c700 1 -- 172.21.15.63:0/2056223857 >> 172.21.15.63:0/2056223857 conn(0x7fadc80f89d0 msgr2=0x7fadc80fac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:42.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.846+0000 7fadcd81c700 1 -- 172.21.15.63:0/2056223857 shutdown_connections 2024-03-22T07:25:42.847 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.846+0000 7fadcd81c700 1 -- 172.21.15.63:0/2056223857 wait complete. 2024-03-22T07:25:42.857 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:25:42.950 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.950+0000 7fe52ebb8700 1 -- 172.21.15.63:0/1374201110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5281026c0 msgr2=0x7fe5280fbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:42.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.950+0000 7fe52ebb8700 1 --2- 172.21.15.63:0/1374201110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5281026c0 0x7fe5280fbd30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe518004940 tx=0x7fe518009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:42.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.950+0000 7fe52ebb8700 1 -- 172.21.15.63:0/1374201110 shutdown_connections 2024-03-22T07:25:42.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.950+0000 7fe52ebb8700 1 --2- 172.21.15.63:0/1374201110 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe5281026c0 0x7fe5280fbd30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:42.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.950+0000 7fe52ebb8700 1 --2- 172.21.15.63:0/1374201110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe528101d20 0x7fe5281020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:42.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.950+0000 7fe52ebb8700 1 -- 172.21.15.63:0/1374201110 >> 172.21.15.63:0/1374201110 conn(0x7fe5280f7870 msgr2=0x7fe5280f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:42.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.950+0000 7fe52ebb8700 1 -- 172.21.15.63:0/1374201110 shutdown_connections 2024-03-22T07:25:42.951 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.951+0000 7fe52ebb8700 1 -- 172.21.15.63:0/1374201110 wait complete. 2024-03-22T07:25:42.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.951+0000 7fe52ebb8700 1 Processor -- start 2024-03-22T07:25:42.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.951+0000 7fe52ebb8700 1 -- start start 2024-03-22T07:25:42.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.951+0000 7fe52ebb8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe528101d20 0x7fe528102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:42.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.951+0000 7fe52ebb8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5281026c0 0x7fe5281030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:42.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.951+0000 7fe52ebb8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe528104b60 con 0x7fe528101d20 2024-03-22T07:25:42.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.951+0000 7fe52ebb8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe528103600 con 0x7fe5281026c0 2024-03-22T07:25:42.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.951+0000 7fe52c954700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe528101d20 0x7fe528102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:42.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.952+0000 7fe52c954700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe528101d20 0x7fe528102b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44166/0 (socket says 172.21.15.63:44166) 2024-03-22T07:25:42.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.952+0000 7fe52c954700 1 -- 172.21.15.63:0/4136631342 learned_addr learned my addr 172.21.15.63:0/4136631342 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:25:42.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.952+0000 7fe51ffff700 1 --2- 172.21.15.63:0/4136631342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5281026c0 0x7fe5281030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:42.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.952+0000 7fe52c954700 1 -- 172.21.15.63:0/4136631342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5281026c0 msgr2=0x7fe5281030c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:42.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.952+0000 7fe52c954700 1 --2- 172.21.15.63:0/4136631342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5281026c0 0x7fe5281030c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:42.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.952+0000 7fe52c954700 1 -- 172.21.15.63:0/4136631342 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe518005040 con 0x7fe528101d20 2024-03-22T07:25:42.953 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.952+0000 7fe52c954700 1 --2- 172.21.15.63:0/4136631342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe528101d20 0x7fe528102b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe52000bbb0 tx=0x7fe520009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:42.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.952+0000 7fe51d7fa700 1 -- 172.21.15.63:0/4136631342 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5200092e0 con 0x7fe528101d20 2024-03-22T07:25:42.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.952+0000 7fe51d7fa700 1 -- 172.21.15.63:0/4136631342 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe52000cb20 con 0x7fe528101d20 2024-03-22T07:25:42.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.952+0000 7fe52ebb8700 1 -- 172.21.15.63:0/4136631342 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5281038e0 con 0x7fe528101d20 2024-03-22T07:25:42.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.952+0000 7fe51d7fa700 1 -- 172.21.15.63:0/4136631342 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe520015d70 con 0x7fe528101d20 2024-03-22T07:25:42.954 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.952+0000 7fe52ebb8700 1 -- 172.21.15.63:0/4136631342 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52811a190 con 0x7fe528101d20 2024-03-22T07:25:42.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.953+0000 7fe52ebb8700 1 -- 172.21.15.63:0/4136631342 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5280fd470 con 0x7fe528101d20 2024-03-22T07:25:42.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.954+0000 7fe51d7fa700 1 -- 172.21.15.63:0/4136631342 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fe520012070 con 0x7fe528101d20 2024-03-22T07:25:42.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.954+0000 7fe51d7fa700 1 --2- 172.21.15.63:0/4136631342 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe51006d380 0x7fe51006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:42.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.954+0000 7fe51d7fa700 1 -- 172.21.15.63:0/4136631342 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe52008c650 con 0x7fe528101d20 2024-03-22T07:25:42.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.954+0000 7fe51ffff700 1 --2- 172.21.15.63:0/4136631342 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe51006d380 0x7fe51006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:42.955 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.955+0000 7fe51ffff700 1 --2- 172.21.15.63:0/4136631342 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe51006d380 0x7fe51006f830 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe518004890 tx=0x7fe518009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:42.959 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:42.959+0000 7fe51d7fa700 1 -- 172.21.15.63:0/4136631342 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe520056080 con 0x7fe528101d20 2024-03-22T07:25:43.182 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.182+0000 7fe52ebb8700 1 -- 172.21.15.63:0/4136631342 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe528104160 con 0x7fe51006d380 2024-03-22T07:25:43.184 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.184+0000 7fe51d7fa700 1 -- 172.21.15.63:0/4136631342 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe51800b700 con 0x7fe51006d380 2024-03-22T07:25:43.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.186+0000 7fe52ebb8700 1 -- 172.21.15.63:0/4136631342 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe51006d380 msgr2=0x7fe51006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:43.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.186+0000 7fe52ebb8700 1 --2- 172.21.15.63:0/4136631342 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe51006d380 0x7fe51006f830 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe518004890 tx=0x7fe518009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:43.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.186+0000 7fe52ebb8700 1 -- 172.21.15.63:0/4136631342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe528101d20 msgr2=0x7fe528102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:43.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.186+0000 7fe52ebb8700 1 --2- 172.21.15.63:0/4136631342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe528101d20 0x7fe528102b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe52000bbb0 tx=0x7fe520009840 comp rx=0 tx=0).stop 2024-03-22T07:25:43.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.186+0000 7fe52ebb8700 1 -- 172.21.15.63:0/4136631342 shutdown_connections 2024-03-22T07:25:43.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.186+0000 7fe52ebb8700 1 --2- 172.21.15.63:0/4136631342 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe51006d380 0x7fe51006f830 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:43.186 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.186+0000 7fe52ebb8700 1 --2- 172.21.15.63:0/4136631342 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe528101d20 0x7fe528102b80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:43.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.186+0000 7fe52ebb8700 1 --2- 172.21.15.63:0/4136631342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe5281026c0 0x7fe5281030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:43.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.186+0000 7fe52ebb8700 1 -- 172.21.15.63:0/4136631342 >> 172.21.15.63:0/4136631342 conn(0x7fe5280f7870 msgr2=0x7fe5280f8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:43.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.186+0000 7fe52ebb8700 1 -- 172.21.15.63:0/4136631342 shutdown_connections 2024-03-22T07:25:43.187 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.186+0000 7fe52ebb8700 1 -- 172.21.15.63:0/4136631342 wait complete. 2024-03-22T07:25:43.291 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.289+0000 7f1e84eb4700 1 -- 172.21.15.63:0/3625570418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e800fb3b0 msgr2=0x7f1e800fb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:43.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.289+0000 7f1e84eb4700 1 --2- 172.21.15.63:0/3625570418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e800fb3b0 0x7f1e800fb820 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1e70004940 tx=0x7f1e70009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:43.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.289+0000 7f1e84eb4700 1 -- 172.21.15.63:0/3625570418 shutdown_connections 2024-03-22T07:25:43.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.289+0000 7f1e84eb4700 1 --2- 172.21.15.63:0/3625570418 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e800fb3b0 0x7f1e800fb820 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:43.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.289+0000 7f1e84eb4700 1 --2- 172.21.15.63:0/3625570418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e80102f70 0x7f1e80103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:43.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.289+0000 7f1e84eb4700 1 -- 172.21.15.63:0/3625570418 >> 172.21.15.63:0/3625570418 conn(0x7f1e80072310 msgr2=0x7f1e80072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:43.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.289+0000 7f1e84eb4700 1 -- 172.21.15.63:0/3625570418 shutdown_connections 2024-03-22T07:25:43.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.290+0000 7f1e84eb4700 1 -- 172.21.15.63:0/3625570418 wait complete. 2024-03-22T07:25:43.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.290+0000 7f1e84eb4700 1 Processor -- start 2024-03-22T07:25:43.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.290+0000 7f1e84eb4700 1 -- start start 2024-03-22T07:25:43.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.290+0000 7f1e84eb4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e800fb3b0 0x7f1e80113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:43.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.290+0000 7f1e84eb4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e80102f70 0x7f1e80113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:43.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.290+0000 7f1e84eb4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e801144f0 con 0x7f1e80102f70 2024-03-22T07:25:43.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.290+0000 7f1e84eb4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8010daf0 con 0x7f1e800fb3b0 2024-03-22T07:25:43.292 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.290+0000 7f1e7dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e80102f70 0x7f1e80113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.290+0000 7f1e7dd9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e80102f70 0x7f1e80113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44186/0 (socket says 172.21.15.63:44186) 2024-03-22T07:25:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.290+0000 7f1e7dd9b700 1 -- 172.21.15.63:0/1121013853 learned_addr learned my addr 172.21.15.63:0/1121013853 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:25:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.291+0000 7f1e7e59c700 1 --2- 172.21.15.63:0/1121013853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e800fb3b0 0x7f1e80113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.291+0000 7f1e7dd9b700 1 -- 172.21.15.63:0/1121013853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e800fb3b0 msgr2=0x7f1e80113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.291+0000 7f1e7dd9b700 1 --2- 172.21.15.63:0/1121013853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e800fb3b0 0x7f1e80113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.291+0000 7f1e7dd9b700 1 -- 172.21.15.63:0/1121013853 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e70005040 con 0x7f1e80102f70 2024-03-22T07:25:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.291+0000 7f1e7e59c700 1 --2- 172.21.15.63:0/1121013853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e800fb3b0 0x7f1e80113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:25:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.291+0000 7f1e7dd9b700 1 --2- 172.21.15.63:0/1121013853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e80102f70 0x7f1e80113ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000b3b0 tx=0x7f1e70009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.291+0000 7f1e76ffd700 1 -- 172.21.15.63:0/1121013853 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e7000e040 con 0x7f1e80102f70 2024-03-22T07:25:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.291+0000 7f1e76ffd700 1 -- 172.21.15.63:0/1121013853 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e7000bbc0 con 0x7f1e80102f70 2024-03-22T07:25:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.291+0000 7f1e76ffd700 1 -- 172.21.15.63:0/1121013853 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e7001c3f0 con 0x7f1e80102f70 2024-03-22T07:25:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.291+0000 7f1e84eb4700 1 -- 172.21.15.63:0/1121013853 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8010dd70 con 0x7f1e80102f70 2024-03-22T07:25:43.293 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.291+0000 7f1e84eb4700 1 -- 172.21.15.63:0/1121013853 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8010e330 con 0x7f1e80102f70 2024-03-22T07:25:43.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.292+0000 7f1e84eb4700 1 -- 172.21.15.63:0/1121013853 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e80105350 con 0x7f1e80102f70 2024-03-22T07:25:43.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.293+0000 7f1e76ffd700 1 -- 172.21.15.63:0/1121013853 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f1e7000bd30 con 0x7f1e80102f70 2024-03-22T07:25:43.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.293+0000 7f1e76ffd700 1 --2- 172.21.15.63:0/1121013853 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1e6c06d380 0x7f1e6c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:43.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.293+0000 7f1e76ffd700 1 -- 172.21.15.63:0/1121013853 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1e7008dc10 con 0x7f1e80102f70 2024-03-22T07:25:43.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.293+0000 7f1e7e59c700 1 --2- 172.21.15.63:0/1121013853 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1e6c06d380 0x7f1e6c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:43.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.294+0000 7f1e7e59c700 1 --2- 172.21.15.63:0/1121013853 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1e6c06d380 0x7f1e6c06f830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009fd0 tx=0x7f1e68009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:43.300 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.300+0000 7f1e76ffd700 1 -- 172.21.15.63:0/1121013853 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1e7005b170 con 0x7f1e80102f70 2024-03-22T07:25:43.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.518+0000 7f1e84eb4700 1 -- 172.21.15.63:0/1121013853 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e8010f020 con 0x7f1e6c06d380 2024-03-22T07:25:43.529 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.529+0000 7f1e76ffd700 1 -- 172.21.15.63:0/1121013853 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f1e8010f020 con 0x7f1e6c06d380 2024-03-22T07:25:43.529 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:25:43.529 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (6m) 8s ago 17m 25.6M - 0.25.0 c8568f914cd2 06ef21da90e5 2024-03-22T07:25:43.529 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (4m) 8s ago 17m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 224c291ab774 2024-03-22T07:25:43.529 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (4m) 49s ago 16m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 7eabb7d42631 2024-03-22T07:25:43.529 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (6m) 8s ago 17m 86.5M - 9.4.12 a0ea5e3c46d9 9015fae0d5e9 2024-03-22T07:25:43.529 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (33s) 8s ago 12m 14.0M - 19.0.0-1667-gdb0330b1 4f325f8440e9 58fd92473761 2024-03-22T07:25:43.529 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (13s) 8s ago 12m 13.7M - 19.0.0-1667-gdb0330b1 4f325f8440e9 0946db898d5d 2024-03-22T07:25:43.530 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (12m) 49s ago 12m 63.0M - 17.2.7-726-g9c4e0c34 837124da804f 58a5425e395c 2024-03-22T07:25:43.530 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (12m) 49s ago 12m 129M - 17.2.7-726-g9c4e0c34 837124da804f 7c1e389d2069 2024-03-22T07:25:43.530 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:8443,9283,8765 running (8m) 8s ago 18m 510M - 19.0.0-1667-gdb0330b1 4f325f8440e9 e49468f39991 2024-03-22T07:25:43.530 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283,8765 running (7m) 49s ago 16m 450M - 19.0.0-1667-gdb0330b1 4f325f8440e9 13c3f034ebb2 2024-03-22T07:25:43.530 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (5m) 8s ago 19m 54.5M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ab96538a07c1 2024-03-22T07:25:43.530 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (4m) 49s ago 16m 39.5M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 01a26e1fb1a1 2024-03-22T07:25:43.530 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (7m) 8s ago 17m 16.4M - 1.5.0 0da6a335fe13 0c825ee2a196 2024-03-22T07:25:43.530 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (7m) 49s ago 16m 16.5M - 1.5.0 0da6a335fe13 12edb2f35498 2024-03-22T07:25:43.530 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (3m) 8s ago 15m 109M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 fdbc8b34ab78 2024-03-22T07:25:43.530 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (3m) 8s ago 15m 101M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 5200fd893046 2024-03-22T07:25:43.530 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (2m) 8s ago 14m 88.3M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 918925ee69f3 2024-03-22T07:25:43.530 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (119s) 49s ago 14m 101M 2581M 19.0.0-1667-gdb0330b1 4f325f8440e9 652a1e62339b 2024-03-22T07:25:43.530 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (86s) 49s ago 14m 94.4M 2581M 19.0.0-1667-gdb0330b1 4f325f8440e9 d6340a0e1c6c 2024-03-22T07:25:43.530 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (52s) 49s ago 13m 12.7M 2581M 19.0.0-1667-gdb0330b1 4f325f8440e9 c21401aa8e19 2024-03-22T07:25:43.531 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (7m) 8s ago 17m 63.3M - 2.43.0 a07b618ecd1d 1d89b928f6e7 2024-03-22T07:25:43.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.531+0000 7f1e84eb4700 1 -- 172.21.15.63:0/1121013853 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1e6c06d380 msgr2=0x7f1e6c06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:43.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.531+0000 7f1e84eb4700 1 --2- 172.21.15.63:0/1121013853 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1e6c06d380 0x7f1e6c06f830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009fd0 tx=0x7f1e68009ee0 comp rx=0 tx=0).stop 2024-03-22T07:25:43.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.531+0000 7f1e84eb4700 1 -- 172.21.15.63:0/1121013853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e80102f70 msgr2=0x7f1e80113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:43.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.531+0000 7f1e84eb4700 1 --2- 172.21.15.63:0/1121013853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e80102f70 0x7f1e80113ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000b3b0 tx=0x7f1e70009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:43.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.531+0000 7f1e84eb4700 1 -- 172.21.15.63:0/1121013853 shutdown_connections 2024-03-22T07:25:43.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.531+0000 7f1e84eb4700 1 --2- 172.21.15.63:0/1121013853 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1e6c06d380 0x7f1e6c06f830 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:43.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.531+0000 7f1e84eb4700 1 --2- 172.21.15.63:0/1121013853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e800fb3b0 0x7f1e80113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:43.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.531+0000 7f1e84eb4700 1 --2- 172.21.15.63:0/1121013853 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e80102f70 0x7f1e80113ea0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:43.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.531+0000 7f1e84eb4700 1 -- 172.21.15.63:0/1121013853 >> 172.21.15.63:0/1121013853 conn(0x7f1e80072310 msgr2=0x7f1e800f9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:43.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.531+0000 7f1e84eb4700 1 -- 172.21.15.63:0/1121013853 shutdown_connections 2024-03-22T07:25:43.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.531+0000 7f1e84eb4700 1 -- 172.21.15.63:0/1121013853 wait complete. 2024-03-22T07:25:43.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:43 smithi063 ceph-mon[112784]: pgmap v193: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:43.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.633+0000 7fce02716700 1 -- 172.21.15.63:0/3046055727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdfc0fe7c0 msgr2=0x7fcdfc106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:43.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.633+0000 7fce02716700 1 --2- 172.21.15.63:0/3046055727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdfc0fe7c0 0x7fcdfc106e00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0004940 tx=0x7fcdf0009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:43.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.633+0000 7fce02716700 1 -- 172.21.15.63:0/3046055727 shutdown_connections 2024-03-22T07:25:43.633 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.633+0000 7fce02716700 1 --2- 172.21.15.63:0/3046055727 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdfc0fe7c0 0x7fcdfc106e00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:43.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.633+0000 7fce02716700 1 --2- 172.21.15.63:0/3046055727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdfc0fde20 0x7fcdfc0fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:43.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.633+0000 7fce02716700 1 -- 172.21.15.63:0/3046055727 >> 172.21.15.63:0/3046055727 conn(0x7fcdfc0f7870 msgr2=0x7fcdfc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:43.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.633+0000 7fce02716700 1 -- 172.21.15.63:0/3046055727 shutdown_connections 2024-03-22T07:25:43.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.633+0000 7fce02716700 1 -- 172.21.15.63:0/3046055727 wait complete. 2024-03-22T07:25:43.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.634+0000 7fce02716700 1 Processor -- start 2024-03-22T07:25:43.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.634+0000 7fce02716700 1 -- start start 2024-03-22T07:25:43.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.634+0000 7fce02716700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdfc0fde20 0x7fcdfc115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:43.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.634+0000 7fce02716700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdfc0fe7c0 0x7fcdfc1160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:43.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.634+0000 7fce02716700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc116740 con 0x7fcdfc0fe7c0 2024-03-22T07:25:43.635 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.634+0000 7fce02716700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc10fd20 con 0x7fcdfc0fde20 2024-03-22T07:25:43.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.634+0000 7fcdfb7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdfc0fe7c0 0x7fcdfc1160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:43.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.634+0000 7fcdfb7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdfc0fe7c0 0x7fcdfc1160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44194/0 (socket says 172.21.15.63:44194) 2024-03-22T07:25:43.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.634+0000 7fcdfb7fe700 1 -- 172.21.15.63:0/2150575781 learned_addr learned my addr 172.21.15.63:0/2150575781 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:25:43.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.634+0000 7fcdfbfff700 1 --2- 172.21.15.63:0/2150575781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdfc0fde20 0x7fcdfc115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:43.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.635+0000 7fcdfb7fe700 1 -- 172.21.15.63:0/2150575781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdfc0fde20 msgr2=0x7fcdfc115bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:43.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.635+0000 7fcdfb7fe700 1 --2- 172.21.15.63:0/2150575781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdfc0fde20 0x7fcdfc115bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:43.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.635+0000 7fcdfb7fe700 1 -- 172.21.15.63:0/2150575781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf0005040 con 0x7fcdfc0fe7c0 2024-03-22T07:25:43.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.635+0000 7fcdfb7fe700 1 --2- 172.21.15.63:0/2150575781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdfc0fe7c0 0x7fcdfc1160f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0000c00 tx=0x7fcdf0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:43.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.635+0000 7fcdf8ff9700 1 -- 172.21.15.63:0/2150575781 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdf0021070 con 0x7fcdfc0fe7c0 2024-03-22T07:25:43.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.635+0000 7fcdf8ff9700 1 -- 172.21.15.63:0/2150575781 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdf0013c00 con 0x7fcdfc0fe7c0 2024-03-22T07:25:43.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.635+0000 7fce02716700 1 -- 172.21.15.63:0/2150575781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdfc10ffa0 con 0x7fcdfc0fe7c0 2024-03-22T07:25:43.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.635+0000 7fcdf8ff9700 1 -- 172.21.15.63:0/2150575781 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdf000e070 con 0x7fcdfc0fe7c0 2024-03-22T07:25:43.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.635+0000 7fce02716700 1 -- 172.21.15.63:0/2150575781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdfc1104e0 con 0x7fcdfc0fe7c0 2024-03-22T07:25:43.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.636+0000 7fce02716700 1 -- 172.21.15.63:0/2150575781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdfc066b70 con 0x7fcdfc0fe7c0 2024-03-22T07:25:43.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.637+0000 7fcdf8ff9700 1 -- 172.21.15.63:0/2150575781 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fcdf001c400 con 0x7fcdfc0fe7c0 2024-03-22T07:25:43.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.637+0000 7fcdf8ff9700 1 --2- 172.21.15.63:0/2150575781 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fcde406d380 0x7fcde406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:43.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.637+0000 7fcdf8ff9700 1 -- 172.21.15.63:0/2150575781 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fcdf008ddb0 con 0x7fcdfc0fe7c0 2024-03-22T07:25:43.637 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.637+0000 7fcdfbfff700 1 --2- 172.21.15.63:0/2150575781 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fcde406d380 0x7fcde406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:43.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.637+0000 7fcdfbfff700 1 --2- 172.21.15.63:0/2150575781 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fcde406d380 0x7fcde406f830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009b80 tx=0x7fcdec00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:43.642 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.642+0000 7fcdf8ff9700 1 -- 172.21.15.63:0/2150575781 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcdf005b390 con 0x7fcdfc0fe7c0 2024-03-22T07:25:43.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:43 smithi069 ceph-mon[83631]: pgmap v193: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:43.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.926+0000 7fce02716700 1 -- 172.21.15.63:0/2150575781 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcdfc109ed0 con 0x7fcdfc0fe7c0 2024-03-22T07:25:43.926 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.926+0000 7fcdf8ff9700 1 -- 172.21.15.63:0/2150575781 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fcdf005af20 con 0x7fcdfc0fe7c0 2024-03-22T07:25:43.927 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:25:43.927 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:25:43.927 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:25:43.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:25:43.927 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:25:43.927 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:25:43.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:25:43.927 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:25:43.927 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 6 2024-03-22T07:25:43.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:25:43.927 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:25:43.927 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T07:25:43.927 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:25:43.927 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:25:43.928 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:25:43.928 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 17.2.7-726-g9c4e0c34 (9c4e0c34f1949e7d705fe7ff26a4f4b729df7a45) quincy (stable)": 1, 2024-03-22T07:25:43.928 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 12 2024-03-22T07:25:43.928 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:25:43.928 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:25:43.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.929+0000 7fce02716700 1 -- 172.21.15.63:0/2150575781 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fcde406d380 msgr2=0x7fcde406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:43.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.929+0000 7fce02716700 1 --2- 172.21.15.63:0/2150575781 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fcde406d380 0x7fcde406f830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009b80 tx=0x7fcdec00b040 comp rx=0 tx=0).stop 2024-03-22T07:25:43.929 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.929+0000 7fce02716700 1 -- 172.21.15.63:0/2150575781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdfc0fe7c0 msgr2=0x7fcdfc1160f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:43.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.929+0000 7fce02716700 1 --2- 172.21.15.63:0/2150575781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdfc0fe7c0 0x7fcdfc1160f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0000c00 tx=0x7fcdf0009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:43.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.929+0000 7fce02716700 1 -- 172.21.15.63:0/2150575781 shutdown_connections 2024-03-22T07:25:43.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.929+0000 7fce02716700 1 --2- 172.21.15.63:0/2150575781 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fcde406d380 0x7fcde406f830 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:43.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.929+0000 7fce02716700 1 --2- 172.21.15.63:0/2150575781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcdfc0fde20 0x7fcdfc115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:43.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.929+0000 7fce02716700 1 --2- 172.21.15.63:0/2150575781 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fcdfc0fe7c0 0x7fcdfc1160f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:43.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.929+0000 7fce02716700 1 -- 172.21.15.63:0/2150575781 >> 172.21.15.63:0/2150575781 conn(0x7fcdfc0f7870 msgr2=0x7fcdfc0f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:43.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.929+0000 7fce02716700 1 -- 172.21.15.63:0/2150575781 shutdown_connections 2024-03-22T07:25:43.930 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:43.929+0000 7fce02716700 1 -- 172.21.15.63:0/2150575781 wait complete. 2024-03-22T07:25:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.032+0000 7fc7276d4700 1 -- 172.21.15.63:0/147547928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7200fde10 msgr2=0x7fc7200fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.032+0000 7fc7276d4700 1 --2- 172.21.15.63:0/147547928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7200fde10 0x7fc7200fe1e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc710004940 tx=0x7fc710009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.032+0000 7fc7276d4700 1 -- 172.21.15.63:0/147547928 shutdown_connections 2024-03-22T07:25:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.032+0000 7fc7276d4700 1 --2- 172.21.15.63:0/147547928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7200fe7b0 0x7fc720106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:44.033 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.032+0000 7fc7276d4700 1 --2- 172.21.15.63:0/147547928 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7200fde10 0x7fc7200fe1e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:44.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.032+0000 7fc7276d4700 1 -- 172.21.15.63:0/147547928 >> 172.21.15.63:0/147547928 conn(0x7fc7200f7870 msgr2=0x7fc7200f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:44.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.032+0000 7fc7276d4700 1 -- 172.21.15.63:0/147547928 shutdown_connections 2024-03-22T07:25:44.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.032+0000 7fc7276d4700 1 -- 172.21.15.63:0/147547928 wait complete. 2024-03-22T07:25:44.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.033+0000 7fc7276d4700 1 Processor -- start 2024-03-22T07:25:44.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.033+0000 7fc7276d4700 1 -- start start 2024-03-22T07:25:44.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.033+0000 7fc7276d4700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7200fde10 0x7fc720113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:44.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.033+0000 7fc7276d4700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7200fe7b0 0x7fc720113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:44.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.033+0000 7fc7276d4700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc720114520 con 0x7fc7200fde10 2024-03-22T07:25:44.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.033+0000 7fc7276d4700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72010db40 con 0x7fc7200fe7b0 2024-03-22T07:25:44.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.033+0000 7fc725470700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7200fde10 0x7fc720113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:44.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.033+0000 7fc725470700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7200fde10 0x7fc720113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44212/0 (socket says 172.21.15.63:44212) 2024-03-22T07:25:44.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.033+0000 7fc725470700 1 -- 172.21.15.63:0/2500613545 learned_addr learned my addr 172.21.15.63:0/2500613545 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:25:44.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.033+0000 7fc724c6f700 1 --2- 172.21.15.63:0/2500613545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7200fe7b0 0x7fc720113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:44.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.034+0000 7fc725470700 1 -- 172.21.15.63:0/2500613545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7200fe7b0 msgr2=0x7fc720113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:44.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.034+0000 7fc725470700 1 --2- 172.21.15.63:0/2500613545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7200fe7b0 0x7fc720113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:44.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.034+0000 7fc725470700 1 -- 172.21.15.63:0/2500613545 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc710005040 con 0x7fc7200fde10 2024-03-22T07:25:44.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.034+0000 7fc725470700 1 --2- 172.21.15.63:0/2500613545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7200fde10 0x7fc720113990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc710004890 tx=0x7fc710009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:44.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.034+0000 7fc715ffb700 1 -- 172.21.15.63:0/2500613545 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc71000e040 con 0x7fc7200fde10 2024-03-22T07:25:44.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.034+0000 7fc715ffb700 1 -- 172.21.15.63:0/2500613545 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc710009af0 con 0x7fc7200fde10 2024-03-22T07:25:44.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.034+0000 7fc7276d4700 1 -- 172.21.15.63:0/2500613545 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72010ddc0 con 0x7fc7200fde10 2024-03-22T07:25:44.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.034+0000 7fc715ffb700 1 -- 172.21.15.63:0/2500613545 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc71001c3f0 con 0x7fc7200fde10 2024-03-22T07:25:44.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.034+0000 7fc7276d4700 1 -- 172.21.15.63:0/2500613545 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc72010e300 con 0x7fc7200fde10 2024-03-22T07:25:44.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.035+0000 7fc7276d4700 1 -- 172.21.15.63:0/2500613545 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc720066ad0 con 0x7fc7200fde10 2024-03-22T07:25:44.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.036+0000 7fc715ffb700 1 -- 172.21.15.63:0/2500613545 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fc71000baf0 con 0x7fc7200fde10 2024-03-22T07:25:44.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.036+0000 7fc715ffb700 1 --2- 172.21.15.63:0/2500613545 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc70c06d380 0x7fc70c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:44.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.036+0000 7fc715ffb700 1 -- 172.21.15.63:0/2500613545 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc71008db20 con 0x7fc7200fde10 2024-03-22T07:25:44.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.036+0000 7fc724c6f700 1 --2- 172.21.15.63:0/2500613545 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc70c06d380 0x7fc70c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:44.037 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.037+0000 7fc724c6f700 1 --2- 172.21.15.63:0/2500613545 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc70c06d380 0x7fc70c06f830 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc72010f230 tx=0x7fc71c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:44.040 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.041+0000 7fc715ffb700 1 -- 172.21.15.63:0/2500613545 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc7100575d0 con 0x7fc7200fde10 2024-03-22T07:25:44.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.289+0000 7fc7276d4700 1 -- 172.21.15.63:0/2500613545 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc720109ea0 con 0x7fc7200fde10 2024-03-22T07:25:44.290 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.290+0000 7fc715ffb700 1 -- 172.21.15.63:0/2500613545 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1659 (secure 0 0 0) 0x7fc71005ac90 con 0x7fc7200fde10 2024-03-22T07:25:44.290 INFO:teuthology.orchestra.run.smithi063.stdout:e32 2024-03-22T07:25:44.290 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:25:44.290 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:25:44.290 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-03-22T07:25:44.290 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:25:44.290 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:25:44.291 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-03-22T07:25:44.291 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 26 2024-03-22T07:25:44.291 INFO:teuthology.orchestra.run.smithi063.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-03-22T07:25:44.291 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-03-22T07:13:29.602905+0000 2024-03-22T07:25:44.291 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-03-22T07:25:00.991999+0000 2024-03-22T07:25:44.291 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-03-22T07:25:44.291 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-03-22T07:25:44.291 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-03-22T07:25:44.291 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-03-22T07:25:44.291 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-03-22T07:25:44.291 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-03-22T07:25:44.291 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-03-22T07:25:44.291 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-03-22T07:25:44.291 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 83 2024-03-22T07:25:44.291 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:25:44.292 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 2 2024-03-22T07:25:44.292 INFO:teuthology.orchestra.run.smithi063.stdout:in 0,1 2024-03-22T07:25:44.292 INFO:teuthology.orchestra.run.smithi063.stdout:up {} 2024-03-22T07:25:44.292 INFO:teuthology.orchestra.run.smithi063.stdout:failed 0,1 2024-03-22T07:25:44.292 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-03-22T07:25:44.292 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-03-22T07:25:44.292 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-03-22T07:25:44.292 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-03-22T07:25:44.292 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-03-22T07:25:44.292 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-03-22T07:25:44.292 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-03-22T07:25:44.292 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-03-22T07:25:44.293 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:25:44.293 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:25:44.293 INFO:teuthology.orchestra.run.smithi063.stdout:Standby daemons: 2024-03-22T07:25:44.293 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:25:44.293 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.oytapz{-1:34400} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6826/1171423403,v1:172.21.15.63:6827/1171423403] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T07:25:44.293 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.xeelpk{-1:34424} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.63:6828/1190308435,v1:172.21.15.63:6829/1190308435] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T07:25:44.293 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.cijxah{-1:44273} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.69:6826/675976207,v1:172.21.15.69:6827/675976207] compat {c=[1],r=[1],i=[7ff]}] 2024-03-22T07:25:44.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.293+0000 7fc7276d4700 1 -- 172.21.15.63:0/2500613545 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc70c06d380 msgr2=0x7fc70c06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:44.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.293+0000 7fc7276d4700 1 --2- 172.21.15.63:0/2500613545 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc70c06d380 0x7fc70c06f830 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc72010f230 tx=0x7fc71c009040 comp rx=0 tx=0).stop 2024-03-22T07:25:44.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.293+0000 7fc7276d4700 1 -- 172.21.15.63:0/2500613545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7200fde10 msgr2=0x7fc720113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:44.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.293+0000 7fc7276d4700 1 --2- 172.21.15.63:0/2500613545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7200fde10 0x7fc720113990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc710004890 tx=0x7fc710009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:44.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.293+0000 7fc7276d4700 1 -- 172.21.15.63:0/2500613545 shutdown_connections 2024-03-22T07:25:44.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.293+0000 7fc7276d4700 1 --2- 172.21.15.63:0/2500613545 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc70c06d380 0x7fc70c06f830 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:44.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.293+0000 7fc7276d4700 1 --2- 172.21.15.63:0/2500613545 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7200fde10 0x7fc720113990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:44.294 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.293+0000 7fc7276d4700 1 --2- 172.21.15.63:0/2500613545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7200fe7b0 0x7fc720113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:44.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.293+0000 7fc7276d4700 1 -- 172.21.15.63:0/2500613545 >> 172.21.15.63:0/2500613545 conn(0x7fc7200f7870 msgr2=0x7fc7200f8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:44.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.293+0000 7fc7276d4700 1 -- 172.21.15.63:0/2500613545 shutdown_connections 2024-03-22T07:25:44.295 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.293+0000 7fc7276d4700 1 -- 172.21.15.63:0/2500613545 wait complete. 2024-03-22T07:25:44.295 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 32 2024-03-22T07:25:44.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.397+0000 7f7d9ecf2700 1 -- 172.21.15.63:0/42245484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d980fde90 msgr2=0x7f7d980fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:44.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.397+0000 7f7d9ecf2700 1 --2- 172.21.15.63:0/42245484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d980fde90 0x7f7d980fe260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c004940 tx=0x7f7d8c009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:44.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.397+0000 7f7d9ecf2700 1 -- 172.21.15.63:0/42245484 shutdown_connections 2024-03-22T07:25:44.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.397+0000 7f7d9ecf2700 1 --2- 172.21.15.63:0/42245484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d980fe830 0x7f7d98106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:44.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.397+0000 7f7d9ecf2700 1 --2- 172.21.15.63:0/42245484 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d980fde90 0x7f7d980fe260 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:44.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.397+0000 7f7d9ecf2700 1 -- 172.21.15.63:0/42245484 >> 172.21.15.63:0/42245484 conn(0x7f7d98072bc0 msgr2=0x7f7d98072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:44.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.397+0000 7f7d9ecf2700 1 -- 172.21.15.63:0/42245484 shutdown_connections 2024-03-22T07:25:44.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.397+0000 7f7d9ecf2700 1 -- 172.21.15.63:0/42245484 wait complete. 2024-03-22T07:25:44.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.397+0000 7f7d9ecf2700 1 Processor -- start 2024-03-22T07:25:44.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.397+0000 7f7d9ecf2700 1 -- start start 2024-03-22T07:25:44.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.398+0000 7f7d9ecf2700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d980fde90 0x7f7d98113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:44.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.398+0000 7f7d9ecf2700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d980fe830 0x7f7d981140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:44.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.398+0000 7f7d9ecf2700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d98114720 con 0x7f7d980fde90 2024-03-22T07:25:44.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.398+0000 7f7d9ecf2700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9810db30 con 0x7f7d980fe830 2024-03-22T07:25:44.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.398+0000 7f7d9ca8e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d980fde90 0x7f7d98113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:44.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.398+0000 7f7d9ca8e700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d980fde90 0x7f7d98113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:44238/0 (socket says 172.21.15.63:44238) 2024-03-22T07:25:44.399 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.398+0000 7f7d9ca8e700 1 -- 172.21.15.63:0/81377783 learned_addr learned my addr 172.21.15.63:0/81377783 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:25:44.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.398+0000 7f7d97fff700 1 --2- 172.21.15.63:0/81377783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d980fe830 0x7f7d981140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:44.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.398+0000 7f7d9ca8e700 1 -- 172.21.15.63:0/81377783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d980fe830 msgr2=0x7f7d981140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:44.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.398+0000 7f7d9ca8e700 1 --2- 172.21.15.63:0/81377783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d980fe830 0x7f7d981140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:44.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.398+0000 7f7d9ca8e700 1 -- 172.21.15.63:0/81377783 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8c005040 con 0x7f7d980fde90 2024-03-22T07:25:44.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.398+0000 7f7d9ca8e700 1 --2- 172.21.15.63:0/81377783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d980fde90 0x7f7d98113b90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0093b0 tx=0x7f7d8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:44.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.399+0000 7f7d957fa700 1 -- 172.21.15.63:0/81377783 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d8c00e040 con 0x7f7d980fde90 2024-03-22T07:25:44.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.399+0000 7f7d957fa700 1 -- 172.21.15.63:0/81377783 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d8c004290 con 0x7f7d980fde90 2024-03-22T07:25:44.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.399+0000 7f7d9ecf2700 1 -- 172.21.15.63:0/81377783 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9810ddb0 con 0x7f7d980fde90 2024-03-22T07:25:44.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.399+0000 7f7d957fa700 1 -- 172.21.15.63:0/81377783 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d8c01c3f0 con 0x7f7d980fde90 2024-03-22T07:25:44.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.399+0000 7f7d9ecf2700 1 -- 172.21.15.63:0/81377783 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d9810e2f0 con 0x7f7d980fde90 2024-03-22T07:25:44.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.400+0000 7f7d9ecf2700 1 -- 172.21.15.63:0/81377783 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d98101980 con 0x7f7d980fde90 2024-03-22T07:25:44.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.400+0000 7f7d957fa700 1 -- 172.21.15.63:0/81377783 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f7d8c013890 con 0x7f7d980fde90 2024-03-22T07:25:44.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.400+0000 7f7d957fa700 1 --2- 172.21.15.63:0/81377783 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7d8806d390 0x7f7d8806f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:44.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.401+0000 7f7d97fff700 1 --2- 172.21.15.63:0/81377783 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7d8806d390 0x7f7d8806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:44.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.401+0000 7f7d957fa700 1 -- 172.21.15.63:0/81377783 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f7d8c08dc20 con 0x7f7d980fde90 2024-03-22T07:25:44.402 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.401+0000 7f7d97fff700 1 --2- 172.21.15.63:0/81377783 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7d8806d390 0x7f7d8806f840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7d9810f430 tx=0x7f7d84009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:44.405 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.405+0000 7f7d957fa700 1 -- 172.21.15.63:0/81377783 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7d8c0576d0 con 0x7f7d980fde90 2024-03-22T07:25:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:44 smithi063 ceph-mon[112784]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:25:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:44 smithi063 ceph-mon[112784]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:25:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:44 smithi063 ceph-mon[112784]: osdmap e86: 6 total, 6 up, 6 in 2024-03-22T07:25:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:44 smithi063 ceph-mon[112784]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-22T07:25:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:44 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2150575781' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:44.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:44 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2500613545' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:25:44.634 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.634+0000 7f7d9ecf2700 1 -- 172.21.15.63:0/81377783 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d98109f00 con 0x7f7d8806d390 2024-03-22T07:25:44.636 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.636+0000 7f7d957fa700 1 -- 172.21.15.63:0/81377783 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7d98109f00 con 0x7f7d8806d390 2024-03-22T07:25:44.636 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:25:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10", 2024-03-22T07:25:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": true, 2024-03-22T07:25:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-22T07:25:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [ 2024-03-22T07:25:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "crash", 2024-03-22T07:25:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "osd", 2024-03-22T07:25:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr", 2024-03-22T07:25:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: "mon" 2024-03-22T07:25:44.636 INFO:teuthology.orchestra.run.smithi063.stdout: ], 2024-03-22T07:25:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": "14/21 daemons upgraded", 2024-03-22T07:25:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "Currently upgrading mds daemons", 2024-03-22T07:25:44.637 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:25:44.637 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:25:44.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.638+0000 7f7d9ecf2700 1 -- 172.21.15.63:0/81377783 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7d8806d390 msgr2=0x7f7d8806f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:44.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.638+0000 7f7d9ecf2700 1 --2- 172.21.15.63:0/81377783 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7d8806d390 0x7f7d8806f840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7d9810f430 tx=0x7f7d84009040 comp rx=0 tx=0).stop 2024-03-22T07:25:44.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.638+0000 7f7d9ecf2700 1 -- 172.21.15.63:0/81377783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d980fde90 msgr2=0x7f7d98113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:44.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.638+0000 7f7d9ecf2700 1 --2- 172.21.15.63:0/81377783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d980fde90 0x7f7d98113b90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0093b0 tx=0x7f7d8c009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:44.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.638+0000 7f7d9ecf2700 1 -- 172.21.15.63:0/81377783 shutdown_connections 2024-03-22T07:25:44.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.638+0000 7f7d9ecf2700 1 --2- 172.21.15.63:0/81377783 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7d8806d390 0x7f7d8806f840 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:44.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.638+0000 7f7d9ecf2700 1 --2- 172.21.15.63:0/81377783 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7d980fde90 0x7f7d98113b90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:44.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.638+0000 7f7d9ecf2700 1 --2- 172.21.15.63:0/81377783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d980fe830 0x7f7d981140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:44.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.638+0000 7f7d9ecf2700 1 -- 172.21.15.63:0/81377783 >> 172.21.15.63:0/81377783 conn(0x7f7d98072bc0 msgr2=0x7f7d980ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:44.638 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.638+0000 7f7d9ecf2700 1 -- 172.21.15.63:0/81377783 shutdown_connections 2024-03-22T07:25:44.639 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.638+0000 7f7d9ecf2700 1 -- 172.21.15.63:0/81377783 wait complete. 2024-03-22T07:25:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.741+0000 7f7f0cb9d700 1 -- 172.21.15.63:0/2395569877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f081024f0 msgr2=0x7f7f080fbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.741+0000 7f7f0cb9d700 1 --2- 172.21.15.63:0/2395569877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f081024f0 0x7f7f080fbb60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7efc004940 tx=0x7f7efc009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.741+0000 7f7f0cb9d700 1 -- 172.21.15.63:0/2395569877 shutdown_connections 2024-03-22T07:25:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.741+0000 7f7f0cb9d700 1 --2- 172.21.15.63:0/2395569877 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f081024f0 0x7f7f080fbb60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.741+0000 7f7f0cb9d700 1 --2- 172.21.15.63:0/2395569877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f08101b50 0x7f7f08101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.741+0000 7f7f0cb9d700 1 -- 172.21.15.63:0/2395569877 >> 172.21.15.63:0/2395569877 conn(0x7f7f080f76a0 msgr2=0x7f7f080f9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.741+0000 7f7f0cb9d700 1 -- 172.21.15.63:0/2395569877 shutdown_connections 2024-03-22T07:25:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.741+0000 7f7f0cb9d700 1 -- 172.21.15.63:0/2395569877 wait complete. 2024-03-22T07:25:44.742 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.742+0000 7f7f0cb9d700 1 Processor -- start 2024-03-22T07:25:44.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.742+0000 7f7f0cb9d700 1 -- start start 2024-03-22T07:25:44.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.742+0000 7f7f0cb9d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f08101b50 0x7f7f08113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:44.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.742+0000 7f7f0cb9d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f081024f0 0x7f7f08113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:44.744 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.742+0000 7f7f0cb9d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f08114500 con 0x7f7f081024f0 2024-03-22T07:25:44.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.742+0000 7f7f0cb9d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0810d8f0 con 0x7f7f08101b50 2024-03-22T07:25:44.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.742+0000 7f7f06ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f081024f0 0x7f7f08113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:44.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.742+0000 7f7f06ffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f081024f0 0x7f7f08113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:34932/0 (socket says 172.21.15.63:34932) 2024-03-22T07:25:44.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.742+0000 7f7f06ffd700 1 -- 172.21.15.63:0/3091608901 learned_addr learned my addr 172.21.15.63:0/3091608901 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:25:44.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.743+0000 7f7f077fe700 1 --2- 172.21.15.63:0/3091608901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f08101b50 0x7f7f08113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:44.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.743+0000 7f7f06ffd700 1 -- 172.21.15.63:0/3091608901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f08101b50 msgr2=0x7f7f08113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:44.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.743+0000 7f7f06ffd700 1 --2- 172.21.15.63:0/3091608901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f08101b50 0x7f7f08113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:44.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.743+0000 7f7f06ffd700 1 -- 172.21.15.63:0/3091608901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc005040 con 0x7f7f081024f0 2024-03-22T07:25:44.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.743+0000 7f7f06ffd700 1 --2- 172.21.15.63:0/3091608901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f081024f0 0x7f7f08113eb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009100 tx=0x7f7efc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:44.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.743+0000 7f7eeffff700 1 -- 172.21.15.63:0/3091608901 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7efc00e040 con 0x7f7f081024f0 2024-03-22T07:25:44.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.743+0000 7f7f0cb9d700 1 -- 172.21.15.63:0/3091608901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f0810db70 con 0x7f7f081024f0 2024-03-22T07:25:44.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.743+0000 7f7eeffff700 1 -- 172.21.15.63:0/3091608901 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7efc013420 con 0x7f7f081024f0 2024-03-22T07:25:44.745 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.743+0000 7f7eeffff700 1 -- 172.21.15.63:0/3091608901 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7efc01c610 con 0x7f7f081024f0 2024-03-22T07:25:44.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.743+0000 7f7f0cb9d700 1 -- 172.21.15.63:0/3091608901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0810e130 con 0x7f7f081024f0 2024-03-22T07:25:44.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.744+0000 7f7f0cb9d700 1 -- 172.21.15.63:0/3091608901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f080fd2a0 con 0x7f7f081024f0 2024-03-22T07:25:44.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.745+0000 7f7eeffff700 1 -- 172.21.15.63:0/3091608901 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f7efc013590 con 0x7f7f081024f0 2024-03-22T07:25:44.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.745+0000 7f7eeffff700 1 --2- 172.21.15.63:0/3091608901 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7ef006d380 0x7f7ef006f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:25:44.746 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.745+0000 7f7eeffff700 1 -- 172.21.15.63:0/3091608901 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f7efc08dbe0 con 0x7f7f081024f0 2024-03-22T07:25:44.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.745+0000 7f7f077fe700 1 --2- 172.21.15.63:0/3091608901 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7ef006d380 0x7f7ef006f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:25:44.747 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.745+0000 7f7f077fe700 1 --2- 172.21.15.63:0/3091608901 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7ef006d380 0x7f7ef006f830 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8004cc0 tx=0x7f7ef800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:25:44.749 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:44.749+0000 7f7eeffff700 1 -- 172.21.15.63:0/3091608901 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7efc05b1c0 con 0x7f7f081024f0 2024-03-22T07:25:44.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:44 smithi069 ceph-mon[83631]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:25:44.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:44 smithi069 ceph-mon[83631]: from='client.34432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:25:44.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:44 smithi069 ceph-mon[83631]: osdmap e86: 6 total, 6 up, 6 in 2024-03-22T07:25:44.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:44 smithi069 ceph-mon[83631]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-22T07:25:44.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:44 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2150575781' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:44.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:44 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2500613545' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:25:45.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:45.035+0000 7f7f0cb9d700 1 -- 172.21.15.63:0/3091608901 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7f0810dd00 con 0x7f7f081024f0 2024-03-22T07:25:45.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:45.035+0000 7f7eeffff700 1 -- 172.21.15.63:0/3091608901 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f7f0810dd00 con 0x7f7f081024f0 2024-03-22T07:25:45.036 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-03-22T07:25:45.036 INFO:teuthology.orchestra.run.smithi063.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-03-22T07:25:45.036 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs is degraded 2024-03-22T07:25:45.036 INFO:teuthology.orchestra.run.smithi063.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-03-22T07:25:45.036 INFO:teuthology.orchestra.run.smithi063.stdout: fs cephfs is offline because no MDS is active for it. 2024-03-22T07:25:45.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:45.038+0000 7f7f0cb9d700 1 -- 172.21.15.63:0/3091608901 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7ef006d380 msgr2=0x7f7ef006f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:45.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:45.038+0000 7f7f0cb9d700 1 --2- 172.21.15.63:0/3091608901 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7ef006d380 0x7f7ef006f830 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8004cc0 tx=0x7f7ef800b040 comp rx=0 tx=0).stop 2024-03-22T07:25:45.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:45.038+0000 7f7f0cb9d700 1 -- 172.21.15.63:0/3091608901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f081024f0 msgr2=0x7f7f08113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:25:45.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:45.038+0000 7f7f0cb9d700 1 --2- 172.21.15.63:0/3091608901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f081024f0 0x7f7f08113eb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009100 tx=0x7f7efc009e30 comp rx=0 tx=0).stop 2024-03-22T07:25:45.038 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:45.038+0000 7f7f0cb9d700 1 -- 172.21.15.63:0/3091608901 shutdown_connections 2024-03-22T07:25:45.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:45.038+0000 7f7f0cb9d700 1 --2- 172.21.15.63:0/3091608901 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f7ef006d380 0x7f7ef006f830 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:45.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:45.038+0000 7f7f0cb9d700 1 --2- 172.21.15.63:0/3091608901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f08101b50 0x7f7f08113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:45.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:45.038+0000 7f7f0cb9d700 1 --2- 172.21.15.63:0/3091608901 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7f081024f0 0x7f7f08113eb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:25:45.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:45.038+0000 7f7f0cb9d700 1 -- 172.21.15.63:0/3091608901 >> 172.21.15.63:0/3091608901 conn(0x7f7f080f76a0 msgr2=0x7f7f08066070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:25:45.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:45.038+0000 7f7f0cb9d700 1 -- 172.21.15.63:0/3091608901 shutdown_connections 2024-03-22T07:25:45.039 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:25:45.038+0000 7f7f0cb9d700 1 -- 172.21.15.63:0/3091608901 wait complete. 2024-03-22T07:25:45.774 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:45 smithi069 ceph-mon[83631]: pgmap v195: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:45.774 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:45 smithi069 ceph-mon[83631]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:25:45.774 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:45 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3091608901' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:25:45.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:45 smithi063 ceph-mon[112784]: pgmap v195: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:45.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:45 smithi063 ceph-mon[112784]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:25:45.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:45 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3091608901' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:25:46.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:46 smithi069 ceph-mon[83631]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:25:46.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:46 smithi063 ceph-mon[112784]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:25:47.713 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:47 smithi069 ceph-mon[83631]: pgmap v196: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:47.713 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:47 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:25:47.713 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:47 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:47.713 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:47 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:47.713 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:47 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:25:47.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:47 smithi063 ceph-mon[112784]: pgmap v196: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:47.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:47 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:25:47.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:47 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:47.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:47 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:47.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:47 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:25:48.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:48 smithi069 ceph-mon[83631]: mds.? [v2:172.21.15.69:6824/653947973,v1:172.21.15.69:6825/653947973] up:boot 2024-03-22T07:25:48.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:48 smithi069 ceph-mon[83631]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:25:48.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:48 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.wwtsjn"}]: dispatch 2024-03-22T07:25:48.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:48 smithi063 ceph-mon[112784]: mds.? [v2:172.21.15.69:6824/653947973,v1:172.21.15.69:6825/653947973] up:boot 2024-03-22T07:25:48.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:48 smithi063 ceph-mon[112784]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:25:48.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:48 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.wwtsjn"}]: dispatch 2024-03-22T07:25:49.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:49 smithi069 ceph-mon[83631]: pgmap v197: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:49.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:49 smithi063 ceph-mon[112784]: pgmap v197: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:50.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:50 smithi063 ceph-mon[112784]: pgmap v198: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:50.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:50 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:50.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:50 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:51.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:50 smithi069 ceph-mon[83631]: pgmap v198: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:51.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:50 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:51.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:50 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:51.952 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:51 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:51.952 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:51 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:52.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:51 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:52.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:51 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:53.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:52 smithi069 ceph-mon[83631]: pgmap v199: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:53.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:52 smithi063 ceph-mon[112784]: pgmap v199: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:55.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:54 smithi069 ceph-mon[83631]: pgmap v200: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:55.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:54 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:55.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:54 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:55.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:54 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:25:55.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:54 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:25:55.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:54 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:55.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:54 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:25:55.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:54 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:55.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:54 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:55.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:54 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:55.030 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:54 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:55.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:54 smithi063 ceph-mon[112784]: pgmap v200: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:55.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:54 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:55.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:54 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:55.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:54 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:25:55.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:54 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:25:55.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:54 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:55.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:54 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:25:55.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:54 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:55.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:54 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:55.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:54 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:55.124 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:54 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:25:57.634 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:57 smithi069 ceph-mon[83631]: pgmap v201: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:57.634 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:57 smithi069 ceph-mon[83631]: Upgrade: Updating mds.cephfs.smithi069.cijxah 2024-03-22T07:25:57.634 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:57 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:57.634 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:57 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi069.cijxah", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:25:57.634 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:57 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:25:57.634 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:57 smithi069 ceph-mon[83631]: Deploying daemon mds.cephfs.smithi069.cijxah on smithi069 2024-03-22T07:25:57.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:57 smithi063 ceph-mon[112784]: pgmap v201: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:57.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:57 smithi063 ceph-mon[112784]: Upgrade: Updating mds.cephfs.smithi069.cijxah 2024-03-22T07:25:57.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:57 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:25:57.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:57 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi069.cijxah", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-22T07:25:57.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:57 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:25:57.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:57 smithi063 ceph-mon[112784]: Deploying daemon mds.cephfs.smithi069.cijxah on smithi069 2024-03-22T07:25:59.675 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:25:59 smithi063 ceph-mon[112784]: pgmap v202: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:25:59.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:25:59 smithi069 ceph-mon[83631]: pgmap v202: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:26:00.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:00 smithi069 ceph-mon[83631]: osdmap e87: 6 total, 6 up, 6 in 2024-03-22T07:26:00.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:00 smithi069 ceph-mon[83631]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-22T07:26:00.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:00 smithi069 ceph-mon[83631]: pgmap v204: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:26:00.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:00 smithi063 ceph-mon[112784]: osdmap e87: 6 total, 6 up, 6 in 2024-03-22T07:26:00.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:00 smithi063 ceph-mon[112784]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-03-22T07:26:00.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:00 smithi063 ceph-mon[112784]: pgmap v204: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:26:01.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:26:01.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:26:02.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:02 smithi069 ceph-mon[83631]: pgmap v205: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:26:02.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:02.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:02.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:26:02.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:02 smithi063 ceph-mon[112784]: pgmap v205: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:26:02.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:02.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:02.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:26:03.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:03 smithi069 ceph-mon[83631]: mds.? [v2:172.21.15.69:6826/3465454734,v1:172.21.15.69:6827/3465454734] up:boot 2024-03-22T07:26:03.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:03 smithi069 ceph-mon[83631]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:26:03.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:03 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.cijxah"}]: dispatch 2024-03-22T07:26:03.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:03 smithi063 ceph-mon[112784]: mds.? [v2:172.21.15.69:6826/3465454734,v1:172.21.15.69:6827/3465454734] up:boot 2024-03-22T07:26:03.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:03 smithi063 ceph-mon[112784]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:26:03.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:03 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi069.cijxah"}]: dispatch 2024-03-22T07:26:04.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:04 smithi063 ceph-mon[112784]: pgmap v206: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:26:05.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:04 smithi069 ceph-mon[83631]: pgmap v206: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:26:06.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:05 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:06.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:05 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:06.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:05 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:06.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:05 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:07.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:06 smithi069 ceph-mon[83631]: pgmap v207: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:26:07.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:06 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:07.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:06 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:07.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:06 smithi063 ceph-mon[112784]: pgmap v207: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:26:07.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:06 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:07.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:06 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:09.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:08 smithi069 ceph-mon[83631]: pgmap v208: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:26:09.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:08 smithi063 ceph-mon[112784]: pgmap v208: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:26:10.441 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:10.441 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:10.441 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:26:10.442 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:26:10.442 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:10.442 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:26:10.442 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:10.442 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:10.442 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:10.442 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:10.442 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:10.442 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: Upgrade: Setting container_image for all mds 2024-03-22T07:26:10.442 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:10.442 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.oytapz"}]: dispatch 2024-03-22T07:26:10.442 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.oytapz"}]': finished 2024-03-22T07:26:10.442 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.xeelpk"}]: dispatch 2024-03-22T07:26:10.442 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.xeelpk"}]': finished 2024-03-22T07:26:10.443 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi069.cijxah"}]: dispatch 2024-03-22T07:26:10.443 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi069.cijxah"}]': finished 2024-03-22T07:26:10.443 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi069.wwtsjn"}]: dispatch 2024-03-22T07:26:10.443 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi069.wwtsjn"}]': finished 2024-03-22T07:26:10.443 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: Upgrade: Setting filesystem cephfs Joinable 2024-03-22T07:26:10.443 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:10 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-22T07:26:10.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:10.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:10.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:26:10.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:26:10.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:10.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:26:10.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:10.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:10.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:10.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:10.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:10.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: Upgrade: Setting container_image for all mds 2024-03-22T07:26:10.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:10.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.oytapz"}]: dispatch 2024-03-22T07:26:10.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.oytapz"}]': finished 2024-03-22T07:26:10.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.xeelpk"}]: dispatch 2024-03-22T07:26:10.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi063.xeelpk"}]': finished 2024-03-22T07:26:10.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi069.cijxah"}]: dispatch 2024-03-22T07:26:10.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi069.cijxah"}]': finished 2024-03-22T07:26:10.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi069.wwtsjn"}]: dispatch 2024-03-22T07:26:10.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi069.wwtsjn"}]': finished 2024-03-22T07:26:10.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: Upgrade: Setting filesystem cephfs Joinable 2024-03-22T07:26:10.531 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:10 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-22T07:26:11.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:11 smithi063 ceph-mon[112784]: pgmap v209: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:26:11.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:11 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-22T07:26:11.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:11 smithi063 ceph-mon[112784]: Standby daemon mds.cephfs.smithi063.oytapz assigned to filesystem cephfs as rank 0 2024-03-22T07:26:11.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:11 smithi063 ceph-mon[112784]: Standby daemon mds.cephfs.smithi063.xeelpk assigned to filesystem cephfs as rank 1 2024-03-22T07:26:11.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:11 smithi063 ceph-mon[112784]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T07:26:11.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:11 smithi063 ceph-mon[112784]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:26:11.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:11 smithi063 ceph-mon[112784]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-03-22T07:26:11.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:11 smithi063 ceph-mon[112784]: fsmap cephfs:2/2 {0=cephfs.smithi063.oytapz=up:replay,1=cephfs.smithi063.xeelpk=up:replay} 2 up:standby 2024-03-22T07:26:11.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:11 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-03-22T07:26:11.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:11 smithi069 ceph-mon[83631]: pgmap v209: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-03-22T07:26:11.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:11 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-22T07:26:11.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:11 smithi069 ceph-mon[83631]: Standby daemon mds.cephfs.smithi063.oytapz assigned to filesystem cephfs as rank 0 2024-03-22T07:26:11.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:11 smithi069 ceph-mon[83631]: Standby daemon mds.cephfs.smithi063.xeelpk assigned to filesystem cephfs as rank 1 2024-03-22T07:26:11.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:11 smithi069 ceph-mon[83631]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-22T07:26:11.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:11 smithi069 ceph-mon[83631]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-03-22T07:26:11.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:11 smithi069 ceph-mon[83631]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-03-22T07:26:11.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:11 smithi069 ceph-mon[83631]: fsmap cephfs:2/2 {0=cephfs.smithi063.oytapz=up:replay,1=cephfs.smithi063.xeelpk=up:replay} 2 up:standby 2024-03-22T07:26:11.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:11 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-03-22T07:26:12.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-03-22T07:26:12.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: fsmap cephfs:2/2 {0=cephfs.smithi063.oytapz=up:replay,1=cephfs.smithi063.xeelpk=up:replay} 2 up:standby 2024-03-22T07:26:12.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: Upgrade: Setting container_image for all rgw 2024-03-22T07:26:12.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: Upgrade: Setting container_image for all rbd-mirror 2024-03-22T07:26:12.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: Upgrade: Setting container_image for all ceph-exporter 2024-03-22T07:26:12.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: Upgrade: Setting container_image for all iscsi 2024-03-22T07:26:12.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.625 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-22T07:26:12.626 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-22T07:26:12.627 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:26:12.627 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-22T07:26:12.627 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-22T07:26:12.627 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-22T07:26:12.627 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-22T07:26:12.627 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-22T07:26:12.627 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-22T07:26:12.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-22T07:26:12.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-22T07:26:12.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-22T07:26:12.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-22T07:26:12.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-22T07:26:12.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:26:12.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-22T07:26:12.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-22T07:26:12.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-22T07:26:12.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-22T07:26:12.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-22T07:26:12.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-22T07:26:12.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-22T07:26:12.628 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-22T07:26:12.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:26:12.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:26:12.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:26:12.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:26:12.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:26:12.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:26:12.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-22T07:26:12.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-22T07:26:12.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:26:12.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:26:12.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:26:12.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.629 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.630 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:26:12.630 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:26:12.630 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:26:12.630 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-03-22T07:26:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: fsmap cephfs:2/2 {0=cephfs.smithi063.oytapz=up:replay,1=cephfs.smithi063.xeelpk=up:replay} 2 up:standby 2024-03-22T07:26:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: Upgrade: Setting container_image for all rgw 2024-03-22T07:26:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: Upgrade: Setting container_image for all rbd-mirror 2024-03-22T07:26:12.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: Upgrade: Setting container_image for all ceph-exporter 2024-03-22T07:26:12.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: Upgrade: Setting container_image for all iscsi 2024-03-22T07:26:12.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.781 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:12.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-22T07:26:12.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-22T07:26:12.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:26:12.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-22T07:26:12.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-22T07:26:12.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-22T07:26:12.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-22T07:26:12.782 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-22T07:26:12.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-22T07:26:12.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-22T07:26:12.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-22T07:26:12.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-22T07:26:12.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-22T07:26:12.783 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-22T07:26:12.784 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:26:12.784 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-22T07:26:12.784 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-22T07:26:12.784 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-22T07:26:12.784 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-22T07:26:12.784 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-22T07:26:12.784 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-22T07:26:12.784 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-22T07:26:12.784 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-22T07:26:12.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:26:12.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:26:12.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:26:12.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:26:12.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:26:12.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-22T07:26:12.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-22T07:26:12.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-22T07:26:12.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:26:12.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:26:12.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:26:12.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:12.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:26:12.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:26:12.785 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:26:12.786 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:13 smithi063 ceph-mon[112784]: Upgrade: Setting container_image for all nfs 2024-03-22T07:26:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:13 smithi063 ceph-mon[112784]: Upgrade: Setting container_image for all nvmeof 2024-03-22T07:26:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:13 smithi063 ceph-mon[112784]: Upgrade: Finalizing container_image settings 2024-03-22T07:26:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:13 smithi063 ceph-mon[112784]: pgmap v210: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 5.0 MiB/s rd, 1 op/s 2024-03-22T07:26:13.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:13 smithi063 ceph-mon[112784]: Upgrade: Complete! 2024-03-22T07:26:13.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:13 smithi069 ceph-mon[83631]: Upgrade: Setting container_image for all nfs 2024-03-22T07:26:13.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:13 smithi069 ceph-mon[83631]: Upgrade: Setting container_image for all nvmeof 2024-03-22T07:26:13.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:13 smithi069 ceph-mon[83631]: Upgrade: Finalizing container_image settings 2024-03-22T07:26:13.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:13 smithi069 ceph-mon[83631]: pgmap v210: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 5.0 MiB/s rd, 1 op/s 2024-03-22T07:26:13.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:13 smithi069 ceph-mon[83631]: Upgrade: Complete! 2024-03-22T07:26:15.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.203+0000 7f9ce14e8700 1 -- 172.21.15.63:0/1347913241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cdc102e10 msgr2=0x7f9cdc1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:15.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.203+0000 7f9ce14e8700 1 --2- 172.21.15.63:0/1347913241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cdc102e10 0x7f9cdc1031e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc004940 tx=0x7f9ccc009e30 comp rx=0 tx=0).stop 2024-03-22T07:26:15.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.203+0000 7f9ce14e8700 1 -- 172.21.15.63:0/1347913241 shutdown_connections 2024-03-22T07:26:15.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.203+0000 7f9ce14e8700 1 --2- 172.21.15.63:0/1347913241 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cdc1037b0 0x7f9cdc0fce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:15.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.203+0000 7f9ce14e8700 1 --2- 172.21.15.63:0/1347913241 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cdc102e10 0x7f9cdc1031e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:15.204 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.203+0000 7f9ce14e8700 1 -- 172.21.15.63:0/1347913241 >> 172.21.15.63:0/1347913241 conn(0x7f9cdc0f8980 msgr2=0x7f9cdc0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:26:15.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.204+0000 7f9ce14e8700 1 -- 172.21.15.63:0/1347913241 shutdown_connections 2024-03-22T07:26:15.205 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.204+0000 7f9ce14e8700 1 -- 172.21.15.63:0/1347913241 wait complete. 2024-03-22T07:26:15.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.204+0000 7f9ce14e8700 1 Processor -- start 2024-03-22T07:26:15.206 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.204+0000 7f9ce14e8700 1 -- start start 2024-03-22T07:26:15.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.205+0000 7f9ce14e8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cdc102e10 0x7f9cdc113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:26:15.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.205+0000 7f9ce14e8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cdc1037b0 0x7f9cdc1140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:26:15.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.205+0000 7f9ce14e8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc1146f0 con 0x7f9cdc1037b0 2024-03-22T07:26:15.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.205+0000 7f9ce14e8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc10dae0 con 0x7f9cdc102e10 2024-03-22T07:26:15.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.205+0000 7f9cdaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cdc102e10 0x7f9cdc113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:26:15.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.205+0000 7f9cdaffd700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cdc102e10 0x7f9cdc113b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.63:56656/0 (socket says 172.21.15.63:56656) 2024-03-22T07:26:15.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.205+0000 7f9cdaffd700 1 -- 172.21.15.63:0/3151473534 learned_addr learned my addr 172.21.15.63:0/3151473534 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:26:15.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.205+0000 7f9cda7fc700 1 --2- 172.21.15.63:0/3151473534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cdc1037b0 0x7f9cdc1140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:26:15.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.205+0000 7f9cdaffd700 1 -- 172.21.15.63:0/3151473534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cdc1037b0 msgr2=0x7f9cdc1140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:15.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.205+0000 7f9cdaffd700 1 --2- 172.21.15.63:0/3151473534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cdc1037b0 0x7f9cdc1140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:15.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.205+0000 7f9cdaffd700 1 -- 172.21.15.63:0/3151473534 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc005040 con 0x7f9cdc102e10 2024-03-22T07:26:15.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.206+0000 7f9cdaffd700 1 --2- 172.21.15.63:0/3151473534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cdc102e10 0x7f9cdc113b60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00b3b0 tx=0x7f9ccc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:26:15.207 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.206+0000 7f9cc37fe700 1 -- 172.21.15.63:0/3151473534 <== mon.1 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ccc00e040 con 0x7f9cdc102e10 2024-03-22T07:26:15.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.206+0000 7f9cc37fe700 1 -- 172.21.15.63:0/3151473534 <== mon.1 v2:172.21.15.69:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ccc00bd00 con 0x7f9cdc102e10 2024-03-22T07:26:15.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.206+0000 7f9ce14e8700 1 -- 172.21.15.63:0/3151473534 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cdc10dd60 con 0x7f9cdc102e10 2024-03-22T07:26:15.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.206+0000 7f9ce14e8700 1 -- 172.21.15.63:0/3151473534 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cdc10e260 con 0x7f9cdc102e10 2024-03-22T07:26:15.208 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.206+0000 7f9cc37fe700 1 -- 172.21.15.63:0/3151473534 <== mon.1 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ccc01b650 con 0x7f9cdc102e10 2024-03-22T07:26:15.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.208+0000 7f9ce14e8700 1 -- 172.21.15.63:0/3151473534 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cdc100120 con 0x7f9cdc102e10 2024-03-22T07:26:15.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.208+0000 7f9cc37fe700 1 -- 172.21.15.63:0/3151473534 <== mon.1 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f9ccc0139f0 con 0x7f9cdc102e10 2024-03-22T07:26:15.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.208+0000 7f9cc37fe700 1 --2- 172.21.15.63:0/3151473534 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9cc406d450 0x7f9cc406f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:26:15.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.208+0000 7f9cc37fe700 1 -- 172.21.15.63:0/3151473534 <== mon.1 v2:172.21.15.69:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f9ccc08ddb0 con 0x7f9cdc102e10 2024-03-22T07:26:15.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.208+0000 7f9cda7fc700 1 --2- 172.21.15.63:0/3151473534 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9cc406d450 0x7f9cc406f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:26:15.209 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.209+0000 7f9cda7fc700 1 --2- 172.21.15.63:0/3151473534 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9cc406d450 0x7f9cc406f900 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc10eca0 tx=0x7f9cd0017040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:26:15.213 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.213+0000 7f9cc37fe700 1 -- 172.21.15.63:0/3151473534 <== mon.1 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9ccc05b420 con 0x7f9cdc102e10 2024-03-22T07:26:15.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.622+0000 7f9ce14e8700 1 -- 172.21.15.63:0/3151473534 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9cdc10f1b0 con 0x7f9cc406d450 2024-03-22T07:26:15.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:15 smithi063 ceph-mon[112784]: pgmap v211: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 9.0 MiB/s rd, 3 op/s 2024-03-22T07:26:15.623 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.624+0000 7f9cc37fe700 1 -- 172.21.15.63:0/3151473534 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f9cdc10f1b0 con 0x7f9cc406d450 2024-03-22T07:26:15.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.626+0000 7f9ce14e8700 1 -- 172.21.15.63:0/3151473534 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9cc406d450 msgr2=0x7f9cc406f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:15.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.626+0000 7f9ce14e8700 1 --2- 172.21.15.63:0/3151473534 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9cc406d450 0x7f9cc406f900 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc10eca0 tx=0x7f9cd0017040 comp rx=0 tx=0).stop 2024-03-22T07:26:15.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.626+0000 7f9ce14e8700 1 -- 172.21.15.63:0/3151473534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cdc102e10 msgr2=0x7f9cdc113b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:15.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.626+0000 7f9ce14e8700 1 --2- 172.21.15.63:0/3151473534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cdc102e10 0x7f9cdc113b60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00b3b0 tx=0x7f9ccc009e30 comp rx=0 tx=0).stop 2024-03-22T07:26:15.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.626+0000 7f9ce14e8700 1 -- 172.21.15.63:0/3151473534 shutdown_connections 2024-03-22T07:26:15.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.626+0000 7f9ce14e8700 1 --2- 172.21.15.63:0/3151473534 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f9cc406d450 0x7f9cc406f900 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:15.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.626+0000 7f9ce14e8700 1 --2- 172.21.15.63:0/3151473534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cdc102e10 0x7f9cdc113b60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:15.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.626+0000 7f9ce14e8700 1 --2- 172.21.15.63:0/3151473534 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f9cdc1037b0 0x7f9cdc1140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:15.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.626+0000 7f9ce14e8700 1 -- 172.21.15.63:0/3151473534 >> 172.21.15.63:0/3151473534 conn(0x7f9cdc0f8980 msgr2=0x7f9cdc1073b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:26:15.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.626+0000 7f9ce14e8700 1 -- 172.21.15.63:0/3151473534 shutdown_connections 2024-03-22T07:26:15.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:15.626+0000 7f9ce14e8700 1 -- 172.21.15.63:0/3151473534 wait complete. 2024-03-22T07:26:15.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:15 smithi069 ceph-mon[83631]: pgmap v211: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 9.0 MiB/s rd, 3 op/s 2024-03-22T07:26:16.827 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch ps' 2024-03-22T07:26:17.136 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:26:17.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:17 smithi063 ceph-mon[112784]: pgmap v212: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-03-22T07:26:17.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:17 smithi063 ceph-mon[112784]: from='client.44331 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:26:17.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:17 smithi063 ceph-mon[112784]: mds.? [v2:172.21.15.63:6828/1190308435,v1:172.21.15.63:6829/1190308435] up:resolve 2024-03-22T07:26:17.548 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:17 smithi063 ceph-mon[112784]: fsmap cephfs:2/2 {0=cephfs.smithi063.oytapz=up:replay,1=cephfs.smithi063.xeelpk=up:resolve} 2 up:standby 2024-03-22T07:26:17.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:17 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:17.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:17 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:26:17.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:17 smithi069 ceph-mon[83631]: pgmap v212: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-03-22T07:26:17.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:17 smithi069 ceph-mon[83631]: from='client.44331 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:26:17.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:17 smithi069 ceph-mon[83631]: mds.? [v2:172.21.15.63:6828/1190308435,v1:172.21.15.63:6829/1190308435] up:resolve 2024-03-22T07:26:17.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:17 smithi069 ceph-mon[83631]: fsmap cephfs:2/2 {0=cephfs.smithi063.oytapz=up:replay,1=cephfs.smithi063.xeelpk=up:resolve} 2 up:standby 2024-03-22T07:26:17.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:17 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:17.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:17 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:26:18.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.909+0000 7f4acb393700 1 -- 172.21.15.63:0/3382997179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ac4101d70 msgr2=0x7f4ac4102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:18.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.909+0000 7f4acb393700 1 --2- 172.21.15.63:0/3382997179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ac4101d70 0x7f4ac4102140 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009ce0 tx=0x7f4abc004d10 comp rx=0 tx=0).stop 2024-03-22T07:26:18.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.909+0000 7f4acb393700 1 -- 172.21.15.63:0/3382997179 shutdown_connections 2024-03-22T07:26:18.911 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.909+0000 7f4acb393700 1 --2- 172.21.15.63:0/3382997179 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac4102710 0x7f4ac40fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:18.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.909+0000 7f4acb393700 1 --2- 172.21.15.63:0/3382997179 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ac4101d70 0x7f4ac4102140 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:18.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.909+0000 7f4acb393700 1 -- 172.21.15.63:0/3382997179 >> 172.21.15.63:0/3382997179 conn(0x7f4ac4072bc0 msgr2=0x7f4ac4072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:26:18.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.909+0000 7f4acb393700 1 -- 172.21.15.63:0/3382997179 shutdown_connections 2024-03-22T07:26:18.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.909+0000 7f4acb393700 1 -- 172.21.15.63:0/3382997179 wait complete. 2024-03-22T07:26:18.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.910+0000 7f4acb393700 1 Processor -- start 2024-03-22T07:26:18.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.910+0000 7f4acb393700 1 -- start start 2024-03-22T07:26:18.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.910+0000 7f4acb393700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ac4101d70 0x7f4ac4113c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:26:18.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.910+0000 7f4acb393700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac4102710 0x7f4ac4114140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:26:18.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.910+0000 7f4acb393700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac4114790 con 0x7f4ac4101d70 2024-03-22T07:26:18.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.910+0000 7f4acb393700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac410dc10 con 0x7f4ac4102710 2024-03-22T07:26:18.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.910+0000 7f4ac912f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ac4101d70 0x7f4ac4113c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:26:18.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.910+0000 7f4ac912f700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ac4101d70 0x7f4ac4113c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:50926/0 (socket says 172.21.15.63:50926) 2024-03-22T07:26:18.912 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.910+0000 7f4ac912f700 1 -- 172.21.15.63:0/764331596 learned_addr learned my addr 172.21.15.63:0/764331596 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:26:18.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.910+0000 7f4ac892e700 1 --2- 172.21.15.63:0/764331596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac4102710 0x7f4ac4114140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:26:18.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.910+0000 7f4ac912f700 1 -- 172.21.15.63:0/764331596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac4102710 msgr2=0x7f4ac4114140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:18.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.910+0000 7f4ac912f700 1 --2- 172.21.15.63:0/764331596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac4102710 0x7f4ac4114140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:18.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.910+0000 7f4ac912f700 1 -- 172.21.15.63:0/764331596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc009940 con 0x7f4ac4101d70 2024-03-22T07:26:18.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.910+0000 7f4ac892e700 1 --2- 172.21.15.63:0/764331596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac4102710 0x7f4ac4114140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:26:18.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.910+0000 7f4ac912f700 1 --2- 172.21.15.63:0/764331596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ac4101d70 0x7f4ac4113c00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00be90 tx=0x7f4abc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:26:18.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.911+0000 7f4ac1ffb700 1 -- 172.21.15.63:0/764331596 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4abc00e040 con 0x7f4ac4101d70 2024-03-22T07:26:18.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.911+0000 7f4ac1ffb700 1 -- 172.21.15.63:0/764331596 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4abc008ea0 con 0x7f4ac4101d70 2024-03-22T07:26:18.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.911+0000 7f4ac1ffb700 1 -- 172.21.15.63:0/764331596 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4abc01c810 con 0x7f4ac4101d70 2024-03-22T07:26:18.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.911+0000 7f4acb393700 1 -- 172.21.15.63:0/764331596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac410de90 con 0x7f4ac4101d70 2024-03-22T07:26:18.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.911+0000 7f4acb393700 1 -- 172.21.15.63:0/764331596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac410e450 con 0x7f4ac4101d70 2024-03-22T07:26:18.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.912+0000 7f4acb393700 1 -- 172.21.15.63:0/764331596 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac40fd4c0 con 0x7f4ac4101d70 2024-03-22T07:26:18.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.912+0000 7f4ac1ffb700 1 -- 172.21.15.63:0/764331596 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f4abc012c00 con 0x7f4ac4101d70 2024-03-22T07:26:18.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.913+0000 7f4ac1ffb700 1 --2- 172.21.15.63:0/764331596 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f4aac06d380 0x7f4aac06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:26:18.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.913+0000 7f4ac892e700 1 --2- 172.21.15.63:0/764331596 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f4aac06d380 0x7f4aac06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:26:18.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.913+0000 7f4ac1ffb700 1 -- 172.21.15.63:0/764331596 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f4abc08e490 con 0x7f4ac4101d70 2024-03-22T07:26:18.914 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.913+0000 7f4ac892e700 1 --2- 172.21.15.63:0/764331596 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f4aac06d380 0x7f4aac06f830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ac40fa670 tx=0x7f4ab8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:26:18.918 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:18.918+0000 7f4ac1ffb700 1 -- 172.21.15.63:0/764331596 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4abc057f40 con 0x7f4ac4101d70 2024-03-22T07:26:19.132 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:19.132+0000 7f4acb393700 1 -- 172.21.15.63:0/764331596 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ac4108e70 con 0x7f4aac06d380 2024-03-22T07:26:19.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:19.142+0000 7f4ac1ffb700 1 -- 172.21.15.63:0/764331596 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f4ac4108e70 con 0x7f4aac06d380 2024-03-22T07:26:19.142 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:26:19.143 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (7m) 44s ago 18m 25.6M - 0.25.0 c8568f914cd2 06ef21da90e5 2024-03-22T07:26:19.143 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (4m) 44s ago 18m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 224c291ab774 2024-03-22T07:26:19.143 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (4m) 14s ago 17m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 7eabb7d42631 2024-03-22T07:26:19.143 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (7m) 44s ago 17m 86.5M - 9.4.12 a0ea5e3c46d9 9015fae0d5e9 2024-03-22T07:26:19.143 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (69s) 44s ago 12m 14.0M - 19.0.0-1667-gdb0330b1 4f325f8440e9 58fd92473761 2024-03-22T07:26:19.143 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (49s) 44s ago 12m 13.7M - 19.0.0-1667-gdb0330b1 4f325f8440e9 0946db898d5d 2024-03-22T07:26:19.143 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (17s) 14s ago 12m 12.5M - 19.0.0-1667-gdb0330b1 4f325f8440e9 8d2544a6b620 2024-03-22T07:26:19.143 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (32s) 14s ago 12m 13.9M - 19.0.0-1667-gdb0330b1 4f325f8440e9 dbce77644ed2 2024-03-22T07:26:19.143 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:8443,9283,8765 running (8m) 44s ago 19m 510M - 19.0.0-1667-gdb0330b1 4f325f8440e9 e49468f39991 2024-03-22T07:26:19.143 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283,8765 running (8m) 14s ago 17m 451M - 19.0.0-1667-gdb0330b1 4f325f8440e9 13c3f034ebb2 2024-03-22T07:26:19.143 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (5m) 44s ago 19m 54.5M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ab96538a07c1 2024-03-22T07:26:19.143 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (5m) 14s ago 17m 44.6M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 01a26e1fb1a1 2024-03-22T07:26:19.143 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (7m) 44s ago 18m 16.4M - 1.5.0 0da6a335fe13 0c825ee2a196 2024-03-22T07:26:19.143 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (7m) 14s ago 17m 16.5M - 1.5.0 0da6a335fe13 12edb2f35498 2024-03-22T07:26:19.143 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (4m) 44s ago 16m 109M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 fdbc8b34ab78 2024-03-22T07:26:19.144 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (3m) 44s ago 16m 101M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 5200fd893046 2024-03-22T07:26:19.144 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (3m) 44s ago 15m 88.3M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 918925ee69f3 2024-03-22T07:26:19.144 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (2m) 14s ago 15m 102M 2581M 19.0.0-1667-gdb0330b1 4f325f8440e9 652a1e62339b 2024-03-22T07:26:19.144 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (2m) 14s ago 14m 94.2M 2581M 19.0.0-1667-gdb0330b1 4f325f8440e9 d6340a0e1c6c 2024-03-22T07:26:19.144 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (88s) 14s ago 14m 86.3M 2581M 19.0.0-1667-gdb0330b1 4f325f8440e9 c21401aa8e19 2024-03-22T07:26:19.144 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (7m) 44s ago 17m 63.3M - 2.43.0 a07b618ecd1d 1d89b928f6e7 2024-03-22T07:26:19.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:19.144+0000 7f4acb393700 1 -- 172.21.15.63:0/764331596 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f4aac06d380 msgr2=0x7f4aac06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:19.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:19.144+0000 7f4acb393700 1 --2- 172.21.15.63:0/764331596 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f4aac06d380 0x7f4aac06f830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ac40fa670 tx=0x7f4ab8009040 comp rx=0 tx=0).stop 2024-03-22T07:26:19.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:19.144+0000 7f4acb393700 1 -- 172.21.15.63:0/764331596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ac4101d70 msgr2=0x7f4ac4113c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:19.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:19.144+0000 7f4acb393700 1 --2- 172.21.15.63:0/764331596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ac4101d70 0x7f4ac4113c00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00be90 tx=0x7f4abc004d10 comp rx=0 tx=0).stop 2024-03-22T07:26:19.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:19.144+0000 7f4acb393700 1 -- 172.21.15.63:0/764331596 shutdown_connections 2024-03-22T07:26:19.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:19.144+0000 7f4acb393700 1 --2- 172.21.15.63:0/764331596 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f4aac06d380 0x7f4aac06f830 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:19.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:19.144+0000 7f4acb393700 1 --2- 172.21.15.63:0/764331596 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4ac4101d70 0x7f4ac4113c00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:19.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:19.144+0000 7f4acb393700 1 --2- 172.21.15.63:0/764331596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac4102710 0x7f4ac4114140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:19.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:19.144+0000 7f4acb393700 1 -- 172.21.15.63:0/764331596 >> 172.21.15.63:0/764331596 conn(0x7f4ac4072bc0 msgr2=0x7f4ac4065940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:26:19.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:19.145+0000 7f4acb393700 1 -- 172.21.15.63:0/764331596 shutdown_connections 2024-03-22T07:26:19.145 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:19.145+0000 7f4acb393700 1 -- 172.21.15.63:0/764331596 wait complete. 2024-03-22T07:26:19.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:19 smithi069 ceph-mon[83631]: pgmap v213: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 7 op/s 2024-03-22T07:26:19.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:19 smithi063 ceph-mon[112784]: pgmap v213: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 7 op/s 2024-03-22T07:26:19.826 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch upgrade status' 2024-03-22T07:26:20.092 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:26:20.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:20 smithi063 ceph-mon[112784]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:26:20.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:20 smithi063 ceph-mon[112784]: mds.? [v2:172.21.15.63:6826/1171423403,v1:172.21.15.63:6827/1171423403] up:resolve 2024-03-22T07:26:20.549 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:20 smithi063 ceph-mon[112784]: fsmap cephfs:2/2 {0=cephfs.smithi063.oytapz=up:resolve,1=cephfs.smithi063.xeelpk=up:resolve} 2 up:standby 2024-03-22T07:26:20.681 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:20 smithi069 ceph-mon[83631]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:26:20.681 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:20 smithi069 ceph-mon[83631]: mds.? [v2:172.21.15.63:6826/1171423403,v1:172.21.15.63:6827/1171423403] up:resolve 2024-03-22T07:26:20.681 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:20 smithi069 ceph-mon[83631]: fsmap cephfs:2/2 {0=cephfs.smithi063.oytapz=up:resolve,1=cephfs.smithi063.xeelpk=up:resolve} 2 up:standby 2024-03-22T07:26:21.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:21 smithi063 ceph-mon[112784]: pgmap v214: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-03-22T07:26:21.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:21 smithi063 ceph-mon[112784]: reconnect by client.24333 v1:172.21.15.63:0/2280312002 after 0 2024-03-22T07:26:21.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:21 smithi063 ceph-mon[112784]: reconnect by client.24333 v1:172.21.15.63:0/2280312002 after 0 2024-03-22T07:26:21.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:21 smithi063 ceph-mon[112784]: reconnect by client.24337 v1:192.168.0.1:0/596998459 after 0 2024-03-22T07:26:21.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:21 smithi063 ceph-mon[112784]: reconnect by client.24337 v1:192.168.0.1:0/596998459 after 0 2024-03-22T07:26:21.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:21 smithi063 ceph-mon[112784]: mds.? [v2:172.21.15.63:6828/1190308435,v1:172.21.15.63:6829/1190308435] up:reconnect 2024-03-22T07:26:21.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:21 smithi063 ceph-mon[112784]: mds.? [v2:172.21.15.63:6826/1171423403,v1:172.21.15.63:6827/1171423403] up:reconnect 2024-03-22T07:26:21.624 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:21 smithi063 ceph-mon[112784]: fsmap cephfs:2/2 {0=cephfs.smithi063.oytapz=up:reconnect,1=cephfs.smithi063.xeelpk=up:reconnect} 2 up:standby 2024-03-22T07:26:21.698 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:21 smithi069 ceph-mon[83631]: pgmap v214: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 7 op/s 2024-03-22T07:26:21.698 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:21 smithi069 ceph-mon[83631]: reconnect by client.24333 v1:172.21.15.63:0/2280312002 after 0 2024-03-22T07:26:21.698 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:21 smithi069 ceph-mon[83631]: reconnect by client.24333 v1:172.21.15.63:0/2280312002 after 0 2024-03-22T07:26:21.698 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:21 smithi069 ceph-mon[83631]: reconnect by client.24337 v1:192.168.0.1:0/596998459 after 0 2024-03-22T07:26:21.698 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:21 smithi069 ceph-mon[83631]: reconnect by client.24337 v1:192.168.0.1:0/596998459 after 0 2024-03-22T07:26:21.698 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:21 smithi069 ceph-mon[83631]: mds.? [v2:172.21.15.63:6828/1190308435,v1:172.21.15.63:6829/1190308435] up:reconnect 2024-03-22T07:26:21.699 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:21 smithi069 ceph-mon[83631]: mds.? [v2:172.21.15.63:6826/1171423403,v1:172.21.15.63:6827/1171423403] up:reconnect 2024-03-22T07:26:21.699 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:21 smithi069 ceph-mon[83631]: fsmap cephfs:2/2 {0=cephfs.smithi063.oytapz=up:reconnect,1=cephfs.smithi063.xeelpk=up:reconnect} 2 up:standby 2024-03-22T07:26:22.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:22 smithi063 ceph-mon[112784]: mds.? [v2:172.21.15.63:6826/1171423403,v1:172.21.15.63:6827/1171423403] up:rejoin 2024-03-22T07:26:22.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:22 smithi063 ceph-mon[112784]: mds.? [v2:172.21.15.63:6828/1190308435,v1:172.21.15.63:6829/1190308435] up:rejoin 2024-03-22T07:26:22.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:22 smithi063 ceph-mon[112784]: fsmap cephfs:2/2 {0=cephfs.smithi063.oytapz=up:rejoin,1=cephfs.smithi063.xeelpk=up:rejoin} 2 up:standby 2024-03-22T07:26:22.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:22 smithi063 ceph-mon[112784]: daemon mds.cephfs.smithi063.xeelpk is now active in filesystem cephfs as rank 1 2024-03-22T07:26:22.775 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:22 smithi063 ceph-mon[112784]: daemon mds.cephfs.smithi063.oytapz is now active in filesystem cephfs as rank 0 2024-03-22T07:26:22.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:22 smithi069 ceph-mon[83631]: mds.? [v2:172.21.15.63:6826/1171423403,v1:172.21.15.63:6827/1171423403] up:rejoin 2024-03-22T07:26:22.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:22 smithi069 ceph-mon[83631]: mds.? [v2:172.21.15.63:6828/1190308435,v1:172.21.15.63:6829/1190308435] up:rejoin 2024-03-22T07:26:22.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:22 smithi069 ceph-mon[83631]: fsmap cephfs:2/2 {0=cephfs.smithi063.oytapz=up:rejoin,1=cephfs.smithi063.xeelpk=up:rejoin} 2 up:standby 2024-03-22T07:26:22.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:22 smithi069 ceph-mon[83631]: daemon mds.cephfs.smithi063.xeelpk is now active in filesystem cephfs as rank 1 2024-03-22T07:26:22.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:22 smithi069 ceph-mon[83631]: daemon mds.cephfs.smithi063.oytapz is now active in filesystem cephfs as rank 0 2024-03-22T07:26:23.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:23 smithi069 ceph-mon[83631]: pgmap v215: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8 op/s 2024-03-22T07:26:23.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:23 smithi069 ceph-mon[83631]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T07:26:23.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:23 smithi069 ceph-mon[83631]: Cluster is now healthy 2024-03-22T07:26:23.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:23 smithi069 ceph-mon[83631]: mds.? [v2:172.21.15.63:6828/1190308435,v1:172.21.15.63:6829/1190308435] up:active 2024-03-22T07:26:23.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:23 smithi069 ceph-mon[83631]: mds.? [v2:172.21.15.63:6826/1171423403,v1:172.21.15.63:6827/1171423403] up:active 2024-03-22T07:26:23.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:23 smithi069 ceph-mon[83631]: fsmap cephfs:2 {0=cephfs.smithi063.oytapz=up:active,1=cephfs.smithi063.xeelpk=up:active} 2 up:standby 2024-03-22T07:26:23.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:23 smithi069 ceph-mon[83631]: fsmap cephfs:2 {0=cephfs.smithi063.oytapz=up:active,1=cephfs.smithi063.xeelpk=up:active} 2 up:standby-replay 2024-03-22T07:26:23.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:23 smithi063 ceph-mon[112784]: pgmap v215: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8 op/s 2024-03-22T07:26:23.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:23 smithi063 ceph-mon[112784]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-22T07:26:23.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:23 smithi063 ceph-mon[112784]: Cluster is now healthy 2024-03-22T07:26:23.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:23 smithi063 ceph-mon[112784]: mds.? [v2:172.21.15.63:6828/1190308435,v1:172.21.15.63:6829/1190308435] up:active 2024-03-22T07:26:23.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:23 smithi063 ceph-mon[112784]: mds.? [v2:172.21.15.63:6826/1171423403,v1:172.21.15.63:6827/1171423403] up:active 2024-03-22T07:26:23.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:23 smithi063 ceph-mon[112784]: fsmap cephfs:2 {0=cephfs.smithi063.oytapz=up:active,1=cephfs.smithi063.xeelpk=up:active} 2 up:standby 2024-03-22T07:26:23.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:23 smithi063 ceph-mon[112784]: fsmap cephfs:2 {0=cephfs.smithi063.oytapz=up:active,1=cephfs.smithi063.xeelpk=up:active} 2 up:standby-replay 2024-03-22T07:26:24.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:24 smithi063 ceph-mon[112784]: pgmap v216: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 170 B/s wr, 9 op/s 2024-03-22T07:26:24.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:24 smithi069 ceph-mon[83631]: pgmap v216: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 170 B/s wr, 9 op/s 2024-03-22T07:26:26.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:26 smithi063 ceph-mon[112784]: pgmap v217: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 170 B/s wr, 9 op/s 2024-03-22T07:26:27.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:26 smithi069 ceph-mon[83631]: pgmap v217: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 170 B/s wr, 9 op/s 2024-03-22T07:26:27.652 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.591+0000 7f67e0013700 1 -- 172.21.15.63:0/1839063865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8102fd0 msgr2=0x7f67d81033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.591+0000 7f67e0013700 1 --2- 172.21.15.63:0/1839063865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8102fd0 0x7f67d81033a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f67d4004940 tx=0x7f67d4009e30 comp rx=0 tx=0).stop 2024-03-22T07:26:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.591+0000 7f67e0013700 1 -- 172.21.15.63:0/1839063865 shutdown_connections 2024-03-22T07:26:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.591+0000 7f67e0013700 1 --2- 172.21.15.63:0/1839063865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d80fb540 0x7f67d80fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.591+0000 7f67e0013700 1 --2- 172.21.15.63:0/1839063865 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8102fd0 0x7f67d81033a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.591+0000 7f67e0013700 1 -- 172.21.15.63:0/1839063865 >> 172.21.15.63:0/1839063865 conn(0x7f67d8071530 msgr2=0x7f67d8071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:26:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.591+0000 7f67e0013700 1 -- 172.21.15.63:0/1839063865 shutdown_connections 2024-03-22T07:26:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.591+0000 7f67e0013700 1 -- 172.21.15.63:0/1839063865 wait complete. 2024-03-22T07:26:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.592+0000 7f67e0013700 1 Processor -- start 2024-03-22T07:26:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.592+0000 7f67e0013700 1 -- start start 2024-03-22T07:26:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.592+0000 7f67e0013700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d80fb540 0x7f67d8113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:26:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.592+0000 7f67e0013700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8102fd0 0x7f67d8113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:26:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.592+0000 7f67e0013700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d81144b0 con 0x7f67d8102fd0 2024-03-22T07:26:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.592+0000 7f67e0013700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d810dab0 con 0x7f67d80fb540 2024-03-22T07:26:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.592+0000 7f67dd5ae700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8102fd0 0x7f67d8113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:26:27.653 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.592+0000 7f67dddaf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d80fb540 0x7f67d8113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:26:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.592+0000 7f67dddaf700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d80fb540 0x7f67d8113920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.63:50626/0 (socket says 172.21.15.63:50626) 2024-03-22T07:26:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.592+0000 7f67dd5ae700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8102fd0 0x7f67d8113e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35118/0 (socket says 172.21.15.63:35118) 2024-03-22T07:26:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.592+0000 7f67dddaf700 1 -- 172.21.15.63:0/1491459334 learned_addr learned my addr 172.21.15.63:0/1491459334 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:26:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.593+0000 7f67dd5ae700 1 -- 172.21.15.63:0/1491459334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d80fb540 msgr2=0x7f67d8113920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.593+0000 7f67dd5ae700 1 --2- 172.21.15.63:0/1491459334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d80fb540 0x7f67d8113920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.593+0000 7f67dd5ae700 1 -- 172.21.15.63:0/1491459334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d4005040 con 0x7f67d8102fd0 2024-03-22T07:26:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.593+0000 7f67dddaf700 1 --2- 172.21.15.63:0/1491459334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d80fb540 0x7f67d8113920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-22T07:26:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.593+0000 7f67dd5ae700 1 --2- 172.21.15.63:0/1491459334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8102fd0 0x7f67d8113e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009fd0 tx=0x7f67c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:26:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.593+0000 7f67ce7fc700 1 -- 172.21.15.63:0/1491459334 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67c8004e90 con 0x7f67d8102fd0 2024-03-22T07:26:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.593+0000 7f67ce7fc700 1 -- 172.21.15.63:0/1491459334 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67c800f040 con 0x7f67d8102fd0 2024-03-22T07:26:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.593+0000 7f67e0013700 1 -- 172.21.15.63:0/1491459334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d810dd30 con 0x7f67d8102fd0 2024-03-22T07:26:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.593+0000 7f67ce7fc700 1 -- 172.21.15.63:0/1491459334 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67c800ca50 con 0x7f67d8102fd0 2024-03-22T07:26:27.654 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.593+0000 7f67e0013700 1 -- 172.21.15.63:0/1491459334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d810e270 con 0x7f67d8102fd0 2024-03-22T07:26:27.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.594+0000 7f67e0013700 1 -- 172.21.15.63:0/1491459334 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d8100270 con 0x7f67d8102fd0 2024-03-22T07:26:27.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.595+0000 7f67ce7fc700 1 -- 172.21.15.63:0/1491459334 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f67c8007cc0 con 0x7f67d8102fd0 2024-03-22T07:26:27.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.595+0000 7f67ce7fc700 1 --2- 172.21.15.63:0/1491459334 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f67c406d380 0x7f67c406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:26:27.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.595+0000 7f67ce7fc700 1 -- 172.21.15.63:0/1491459334 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f67c808c390 con 0x7f67d8102fd0 2024-03-22T07:26:27.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.595+0000 7f67dddaf700 1 --2- 172.21.15.63:0/1491459334 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f67c406d380 0x7f67c406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:26:27.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.595+0000 7f67dddaf700 1 --2- 172.21.15.63:0/1491459334 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f67c406d380 0x7f67c406f830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f67d400b0d0 tx=0x7f67d4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:26:27.655 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.600+0000 7f67ce7fc700 1 -- 172.21.15.63:0/1491459334 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f67c80599a0 con 0x7f67d8102fd0 2024-03-22T07:26:27.823 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.822+0000 7f67e0013700 1 -- 172.21.15.63:0/1491459334 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f67d804c780 con 0x7f67c406d380 2024-03-22T07:26:27.824 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.824+0000 7f67ce7fc700 1 -- 172.21.15.63:0/1491459334 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f67d804c780 con 0x7f67c406d380 2024-03-22T07:26:27.824 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:26:27.824 INFO:teuthology.orchestra.run.smithi063.stdout: "target_image": null, 2024-03-22T07:26:27.824 INFO:teuthology.orchestra.run.smithi063.stdout: "in_progress": false, 2024-03-22T07:26:27.824 INFO:teuthology.orchestra.run.smithi063.stdout: "which": "", 2024-03-22T07:26:27.824 INFO:teuthology.orchestra.run.smithi063.stdout: "services_complete": [], 2024-03-22T07:26:27.824 INFO:teuthology.orchestra.run.smithi063.stdout: "progress": null, 2024-03-22T07:26:27.824 INFO:teuthology.orchestra.run.smithi063.stdout: "message": "", 2024-03-22T07:26:27.824 INFO:teuthology.orchestra.run.smithi063.stdout: "is_paused": false 2024-03-22T07:26:27.824 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:26:27.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.826+0000 7f67e0013700 1 -- 172.21.15.63:0/1491459334 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f67c406d380 msgr2=0x7f67c406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:27.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.826+0000 7f67e0013700 1 --2- 172.21.15.63:0/1491459334 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f67c406d380 0x7f67c406f830 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f67d400b0d0 tx=0x7f67d4009e30 comp rx=0 tx=0).stop 2024-03-22T07:26:27.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.826+0000 7f67e0013700 1 -- 172.21.15.63:0/1491459334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8102fd0 msgr2=0x7f67d8113e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:27.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.826+0000 7f67e0013700 1 --2- 172.21.15.63:0/1491459334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8102fd0 0x7f67d8113e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009fd0 tx=0x7f67c800b040 comp rx=0 tx=0).stop 2024-03-22T07:26:27.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.826+0000 7f67e0013700 1 -- 172.21.15.63:0/1491459334 shutdown_connections 2024-03-22T07:26:27.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.826+0000 7f67e0013700 1 --2- 172.21.15.63:0/1491459334 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f67c406d380 0x7f67c406f830 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:27.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.826+0000 7f67e0013700 1 --2- 172.21.15.63:0/1491459334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67d80fb540 0x7f67d8113920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:27.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.826+0000 7f67e0013700 1 --2- 172.21.15.63:0/1491459334 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f67d8102fd0 0x7f67d8113e60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:27.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.826+0000 7f67e0013700 1 -- 172.21.15.63:0/1491459334 >> 172.21.15.63:0/1491459334 conn(0x7f67d8071530 msgr2=0x7f67d80fa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:26:27.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.826+0000 7f67e0013700 1 -- 172.21.15.63:0/1491459334 shutdown_connections 2024-03-22T07:26:27.827 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:27.826+0000 7f67e0013700 1 -- 172.21.15.63:0/1491459334 wait complete. 2024-03-22T07:26:28.599 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph health detail' 2024-03-22T07:26:28.861 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:26:28.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:28 smithi063 ceph-mon[112784]: pgmap v218: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 28 MiB/s rd, 341 B/s wr, 14 op/s 2024-03-22T07:26:28.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:28 smithi063 ceph-mon[112784]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:26:29.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:28 smithi069 ceph-mon[83631]: pgmap v218: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 28 MiB/s rd, 341 B/s wr, 14 op/s 2024-03-22T07:26:29.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:28 smithi069 ceph-mon[83631]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:26:30.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.580+0000 7fbdb1dd8700 1 -- 172.21.15.63:0/3324791878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdac0fb4a0 msgr2=0x7fbdac0fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.580+0000 7fbdb1dd8700 1 --2- 172.21.15.63:0/3324791878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdac0fb4a0 0x7fbdac0fb910 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbda0004940 tx=0x7fbda0009e30 comp rx=0 tx=0).stop 2024-03-22T07:26:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.580+0000 7fbdb1dd8700 1 -- 172.21.15.63:0/3324791878 shutdown_connections 2024-03-22T07:26:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.580+0000 7fbdb1dd8700 1 --2- 172.21.15.63:0/3324791878 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdac0fb4a0 0x7fbdac0fb910 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.580+0000 7fbdb1dd8700 1 --2- 172.21.15.63:0/3324791878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdac102e70 0x7fbdac103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.580+0000 7fbdb1dd8700 1 -- 172.21.15.63:0/3324791878 >> 172.21.15.63:0/3324791878 conn(0x7fbdac0714c0 msgr2=0x7fbdac0718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:26:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.580+0000 7fbdb1dd8700 1 -- 172.21.15.63:0/3324791878 shutdown_connections 2024-03-22T07:26:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.580+0000 7fbdb1dd8700 1 -- 172.21.15.63:0/3324791878 wait complete. 2024-03-22T07:26:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.580+0000 7fbdb1dd8700 1 Processor -- start 2024-03-22T07:26:30.581 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.581+0000 7fbdb1dd8700 1 -- start start 2024-03-22T07:26:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.581+0000 7fbdb1dd8700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdac0fb4a0 0x7fbdac113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:26:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.581+0000 7fbdb1dd8700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdac102e70 0x7fbdac1140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:26:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.581+0000 7fbdb1dd8700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac114720 con 0x7fbdac102e70 2024-03-22T07:26:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.581+0000 7fbdb1dd8700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdac10db10 con 0x7fbdac0fb4a0 2024-03-22T07:26:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.581+0000 7fbdaaffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdac102e70 0x7fbdac1140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:26:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.581+0000 7fbdaaffd700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdac102e70 0x7fbdac1140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35138/0 (socket says 172.21.15.63:35138) 2024-03-22T07:26:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.581+0000 7fbdaaffd700 1 -- 172.21.15.63:0/2020360445 learned_addr learned my addr 172.21.15.63:0/2020360445 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:26:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.581+0000 7fbdab7fe700 1 --2- 172.21.15.63:0/2020360445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdac0fb4a0 0x7fbdac113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:26:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.581+0000 7fbdaaffd700 1 -- 172.21.15.63:0/2020360445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdac0fb4a0 msgr2=0x7fbdac113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:30.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.581+0000 7fbdaaffd700 1 --2- 172.21.15.63:0/2020360445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdac0fb4a0 0x7fbdac113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.581+0000 7fbdaaffd700 1 -- 172.21.15.63:0/2020360445 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda0005040 con 0x7fbdac102e70 2024-03-22T07:26:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.582+0000 7fbdaaffd700 1 --2- 172.21.15.63:0/2020360445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdac102e70 0x7fbdac1140d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbda000b3b0 tx=0x7fbda0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:26:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.582+0000 7fbd93fff700 1 -- 172.21.15.63:0/2020360445 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbda000e040 con 0x7fbdac102e70 2024-03-22T07:26:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.582+0000 7fbd93fff700 1 -- 172.21.15.63:0/2020360445 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbda000bbc0 con 0x7fbdac102e70 2024-03-22T07:26:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.582+0000 7fbd93fff700 1 -- 172.21.15.63:0/2020360445 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbda001c3f0 con 0x7fbdac102e70 2024-03-22T07:26:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.582+0000 7fbdb1dd8700 1 -- 172.21.15.63:0/2020360445 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdac10dd90 con 0x7fbdac102e70 2024-03-22T07:26:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.582+0000 7fbdb1dd8700 1 -- 172.21.15.63:0/2020360445 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdac10e2d0 con 0x7fbdac102e70 2024-03-22T07:26:30.584 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.583+0000 7fbdb1dd8700 1 -- 172.21.15.63:0/2020360445 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdac100170 con 0x7fbdac102e70 2024-03-22T07:26:30.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.583+0000 7fbd93fff700 1 -- 172.21.15.63:0/2020360445 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fbda000bd30 con 0x7fbdac102e70 2024-03-22T07:26:30.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.584+0000 7fbd93fff700 1 --2- 172.21.15.63:0/2020360445 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbd9406d380 0x7fbd9406f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:26:30.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.584+0000 7fbd93fff700 1 -- 172.21.15.63:0/2020360445 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbda008dc10 con 0x7fbdac102e70 2024-03-22T07:26:30.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.584+0000 7fbdab7fe700 1 --2- 172.21.15.63:0/2020360445 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbd9406d380 0x7fbd9406f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:26:30.585 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.584+0000 7fbdab7fe700 1 --2- 172.21.15.63:0/2020360445 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbd9406d380 0x7fbd9406f830 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c004990 tx=0x7fbd9c009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:26:30.588 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.588+0000 7fbd93fff700 1 -- 172.21.15.63:0/2020360445 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbda005b170 con 0x7fbdac102e70 2024-03-22T07:26:30.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:30 smithi063 ceph-mon[112784]: pgmap v219: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8.4 KiB/s wr, 17 op/s 2024-03-22T07:26:30.875 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.875+0000 7fbdb1dd8700 1 -- 172.21.15.63:0/2020360445 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbdac10f1e0 con 0x7fbdac102e70 2024-03-22T07:26:30.876 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.876+0000 7fbd93fff700 1 -- 172.21.15.63:0/2020360445 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fbda005ad00 con 0x7fbdac102e70 2024-03-22T07:26:30.876 INFO:teuthology.orchestra.run.smithi063.stdout:HEALTH_OK 2024-03-22T07:26:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.878+0000 7fbdb1dd8700 1 -- 172.21.15.63:0/2020360445 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbd9406d380 msgr2=0x7fbd9406f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.878+0000 7fbdb1dd8700 1 --2- 172.21.15.63:0/2020360445 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbd9406d380 0x7fbd9406f830 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c004990 tx=0x7fbd9c009ee0 comp rx=0 tx=0).stop 2024-03-22T07:26:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.878+0000 7fbdb1dd8700 1 -- 172.21.15.63:0/2020360445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdac102e70 msgr2=0x7fbdac1140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.878+0000 7fbdb1dd8700 1 --2- 172.21.15.63:0/2020360445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdac102e70 0x7fbdac1140d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbda000b3b0 tx=0x7fbda0009e30 comp rx=0 tx=0).stop 2024-03-22T07:26:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.878+0000 7fbdb1dd8700 1 -- 172.21.15.63:0/2020360445 shutdown_connections 2024-03-22T07:26:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.878+0000 7fbdb1dd8700 1 --2- 172.21.15.63:0/2020360445 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fbd9406d380 0x7fbd9406f830 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.878+0000 7fbdb1dd8700 1 --2- 172.21.15.63:0/2020360445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbdac0fb4a0 0x7fbdac113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.878+0000 7fbdb1dd8700 1 --2- 172.21.15.63:0/2020360445 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fbdac102e70 0x7fbdac1140d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:30.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.878+0000 7fbdb1dd8700 1 -- 172.21.15.63:0/2020360445 >> 172.21.15.63:0/2020360445 conn(0x7fbdac0714c0 msgr2=0x7fbdac0fa7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:26:30.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.878+0000 7fbdb1dd8700 1 -- 172.21.15.63:0/2020360445 shutdown_connections 2024-03-22T07:26:30.879 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:30.878+0000 7fbdb1dd8700 1 -- 172.21.15.63:0/2020360445 wait complete. 2024-03-22T07:26:31.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:30 smithi069 ceph-mon[83631]: pgmap v219: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8.4 KiB/s wr, 17 op/s 2024-03-22T07:26:31.524 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph versions' 2024-03-22T07:26:31.794 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:26:31.808 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:31 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2020360445' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:26:31.808 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:31 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:31.808 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:31 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:26:32.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:31 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2020360445' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-22T07:26:32.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:31 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:26:32.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:31 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:26:32.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:32 smithi069 ceph-mon[83631]: pgmap v220: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8.4 KiB/s wr, 18 op/s 2024-03-22T07:26:32.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:32 smithi063 ceph-mon[112784]: pgmap v220: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8.4 KiB/s wr, 18 op/s 2024-03-22T07:26:33.413 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.410+0000 7f2e50d0c700 1 -- 172.21.15.63:0/1540602987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e4c0fb540 msgr2=0x7f2e4c0fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.410+0000 7f2e50d0c700 1 --2- 172.21.15.63:0/1540602987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e4c0fb540 0x7f2e4c0fb990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2e34004940 tx=0x7f2e34009e30 comp rx=0 tx=0).stop 2024-03-22T07:26:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.410+0000 7f2e50d0c700 1 -- 172.21.15.63:0/1540602987 shutdown_connections 2024-03-22T07:26:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.410+0000 7f2e50d0c700 1 --2- 172.21.15.63:0/1540602987 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e4c0fb540 0x7f2e4c0fb990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.410+0000 7f2e50d0c700 1 --2- 172.21.15.63:0/1540602987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e4c102fd0 0x7f2e4c1033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.410+0000 7f2e50d0c700 1 -- 172.21.15.63:0/1540602987 >> 172.21.15.63:0/1540602987 conn(0x7f2e4c071530 msgr2=0x7f2e4c071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:26:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.410+0000 7f2e50d0c700 1 -- 172.21.15.63:0/1540602987 shutdown_connections 2024-03-22T07:26:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.410+0000 7f2e50d0c700 1 -- 172.21.15.63:0/1540602987 wait complete. 2024-03-22T07:26:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.411+0000 7f2e50d0c700 1 Processor -- start 2024-03-22T07:26:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.411+0000 7f2e50d0c700 1 -- start start 2024-03-22T07:26:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.411+0000 7f2e50d0c700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e4c0fb540 0x7f2e4c113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:26:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.411+0000 7f2e50d0c700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e4c102fd0 0x7f2e4c113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:26:33.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.411+0000 7f2e50d0c700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c1144e0 con 0x7f2e4c102fd0 2024-03-22T07:26:33.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.411+0000 7f2e50d0c700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c10dae0 con 0x7f2e4c0fb540 2024-03-22T07:26:33.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.411+0000 7f2e49d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e4c102fd0 0x7f2e4c113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:26:33.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.411+0000 7f2e49d9b700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e4c102fd0 0x7f2e4c113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:35154/0 (socket says 172.21.15.63:35154) 2024-03-22T07:26:33.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.411+0000 7f2e49d9b700 1 -- 172.21.15.63:0/1216157977 learned_addr learned my addr 172.21.15.63:0/1216157977 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:26:33.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.412+0000 7f2e4a59c700 1 --2- 172.21.15.63:0/1216157977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e4c0fb540 0x7f2e4c113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:26:33.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.412+0000 7f2e49d9b700 1 -- 172.21.15.63:0/1216157977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e4c0fb540 msgr2=0x7f2e4c113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:33.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.412+0000 7f2e49d9b700 1 --2- 172.21.15.63:0/1216157977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e4c0fb540 0x7f2e4c113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:33.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.412+0000 7f2e49d9b700 1 -- 172.21.15.63:0/1216157977 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e34005040 con 0x7f2e4c102fd0 2024-03-22T07:26:33.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.412+0000 7f2e49d9b700 1 --2- 172.21.15.63:0/1216157977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e4c102fd0 0x7f2e4c113e90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009150 tx=0x7f2e34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:26:33.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.412+0000 7f2e42ffd700 1 -- 172.21.15.63:0/1216157977 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e3400e040 con 0x7f2e4c102fd0 2024-03-22T07:26:33.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.412+0000 7f2e42ffd700 1 -- 172.21.15.63:0/1216157977 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e34013420 con 0x7f2e4c102fd0 2024-03-22T07:26:33.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.412+0000 7f2e50d0c700 1 -- 172.21.15.63:0/1216157977 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4c10dd60 con 0x7f2e4c102fd0 2024-03-22T07:26:33.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.412+0000 7f2e42ffd700 1 -- 172.21.15.63:0/1216157977 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e3401c630 con 0x7f2e4c102fd0 2024-03-22T07:26:33.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.412+0000 7f2e50d0c700 1 -- 172.21.15.63:0/1216157977 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4c10e320 con 0x7f2e4c102fd0 2024-03-22T07:26:33.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.413+0000 7f2e50d0c700 1 -- 172.21.15.63:0/1216157977 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4c1053b0 con 0x7f2e4c102fd0 2024-03-22T07:26:33.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.414+0000 7f2e42ffd700 1 -- 172.21.15.63:0/1216157977 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f2e34013590 con 0x7f2e4c102fd0 2024-03-22T07:26:33.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.414+0000 7f2e42ffd700 1 --2- 172.21.15.63:0/1216157977 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2e3806d380 0x7f2e3806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:26:33.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.414+0000 7f2e42ffd700 1 -- 172.21.15.63:0/1216157977 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2e3408dbd0 con 0x7f2e4c102fd0 2024-03-22T07:26:33.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.414+0000 7f2e4a59c700 1 --2- 172.21.15.63:0/1216157977 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2e3806d380 0x7f2e3806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:26:33.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.414+0000 7f2e4a59c700 1 --2- 172.21.15.63:0/1216157977 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2e3806d380 0x7f2e3806f830 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c004f30 tx=0x7f2e3c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:26:33.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.419+0000 7f2e42ffd700 1 -- 172.21.15.63:0/1216157977 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2e3405b1b0 con 0x7f2e4c102fd0 2024-03-22T07:26:33.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.714+0000 7f2e50d0c700 1 -- 172.21.15.63:0/1216157977 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2e4c10efa0 con 0x7f2e4c102fd0 2024-03-22T07:26:33.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.714+0000 7f2e42ffd700 1 -- 172.21.15.63:0/1216157977 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f2e3405ad40 con 0x7f2e4c102fd0 2024-03-22T07:26:33.715 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:26:33.716 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:26:33.716 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:26:33.716 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:26:33.716 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:26:33.716 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:26:33.716 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:26:33.716 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:26:33.716 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 6 2024-03-22T07:26:33.716 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:26:33.716 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:26:33.716 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-22T07:26:33.716 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:26:33.716 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:26:33.716 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 14 2024-03-22T07:26:33.717 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:26:33.717 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:26:33.717 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.716+0000 7f2e50d0c700 1 -- 172.21.15.63:0/1216157977 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2e3806d380 msgr2=0x7f2e3806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:33.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.716+0000 7f2e50d0c700 1 --2- 172.21.15.63:0/1216157977 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2e3806d380 0x7f2e3806f830 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c004f30 tx=0x7f2e3c00b040 comp rx=0 tx=0).stop 2024-03-22T07:26:33.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.716+0000 7f2e50d0c700 1 -- 172.21.15.63:0/1216157977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e4c102fd0 msgr2=0x7f2e4c113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:26:33.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.716+0000 7f2e50d0c700 1 --2- 172.21.15.63:0/1216157977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e4c102fd0 0x7f2e4c113e90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009150 tx=0x7f2e34009e30 comp rx=0 tx=0).stop 2024-03-22T07:26:33.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.716+0000 7f2e50d0c700 1 -- 172.21.15.63:0/1216157977 shutdown_connections 2024-03-22T07:26:33.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.716+0000 7f2e50d0c700 1 --2- 172.21.15.63:0/1216157977 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2e3806d380 0x7f2e3806f830 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:33.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.716+0000 7f2e50d0c700 1 --2- 172.21.15.63:0/1216157977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2e4c0fb540 0x7f2e4c113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:33.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.716+0000 7f2e50d0c700 1 --2- 172.21.15.63:0/1216157977 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2e4c102fd0 0x7f2e4c113e90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:26:33.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.716+0000 7f2e50d0c700 1 -- 172.21.15.63:0/1216157977 >> 172.21.15.63:0/1216157977 conn(0x7f2e4c071530 msgr2=0x7f2e4c0fa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:26:33.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.716+0000 7f2e50d0c700 1 -- 172.21.15.63:0/1216157977 shutdown_connections 2024-03-22T07:26:33.718 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:26:33.716+0000 7f2e50d0c700 1 -- 172.21.15.63:0/1216157977 wait complete. 2024-03-22T07:26:34.375 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-03-22T07:26:34.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:34 smithi063 ceph-mon[112784]: pgmap v221: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8.4 KiB/s wr, 18 op/s 2024-03-22T07:26:34.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:34 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/1216157977' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:34.639 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:26:35.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:34 smithi069 ceph-mon[83631]: pgmap v221: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8.4 KiB/s wr, 18 op/s 2024-03-22T07:26:35.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:34 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/1216157977' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:26:36.294 INFO:teuthology.orchestra.run.smithi063.stdout:wait for servicemap items w/ changing names to refresh 2024-03-22T07:26:36.803 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:36 smithi063 ceph-mon[112784]: pgmap v222: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 8.2 KiB/s wr, 15 op/s 2024-03-22T07:26:36.927 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'sleep 60' 2024-03-22T07:26:37.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:36 smithi069 ceph-mon[83631]: pgmap v222: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 8.2 KiB/s wr, 15 op/s 2024-03-22T07:26:37.194 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:26:38.623 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:38 smithi063 ceph-mon[112784]: pgmap v223: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 8.2 KiB/s wr, 14 op/s 2024-03-22T07:26:39.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:38 smithi069 ceph-mon[83631]: pgmap v223: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 8.2 KiB/s wr, 14 op/s 2024-03-22T07:26:40.873 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:40 smithi063 ceph-mon[112784]: pgmap v224: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 9.3 MiB/s rd, 8.1 KiB/s wr, 8 op/s 2024-03-22T07:26:41.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:40 smithi069 ceph-mon[83631]: pgmap v224: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 9.3 MiB/s rd, 8.1 KiB/s wr, 8 op/s 2024-03-22T07:26:42.853 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:42 smithi063 ceph-mon[112784]: pgmap v225: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.5 MiB/s rd, 4 op/s 2024-03-22T07:26:43.029 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:42 smithi069 ceph-mon[83631]: pgmap v225: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.5 MiB/s rd, 4 op/s 2024-03-22T07:26:45.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:44 smithi069 ceph-mon[83631]: pgmap v226: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 194 KiB/s rd, 4 op/s 2024-03-22T07:26:45.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:44 smithi063 ceph-mon[112784]: pgmap v226: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 194 KiB/s rd, 4 op/s 2024-03-22T07:26:47.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:46 smithi069 ceph-mon[83631]: pgmap v227: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:26:47.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:46 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:26:47.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:46 smithi063 ceph-mon[112784]: pgmap v227: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:26:47.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:46 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:26:49.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:48 smithi069 ceph-mon[83631]: pgmap v228: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:26:49.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:48 smithi063 ceph-mon[112784]: pgmap v228: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:26:51.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:50 smithi069 ceph-mon[83631]: pgmap v229: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:26:51.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:50 smithi063 ceph-mon[112784]: pgmap v229: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:26:53.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:52 smithi063 ceph-mon[112784]: pgmap v230: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:26:53.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:52 smithi069 ceph-mon[83631]: pgmap v230: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:26:55.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:54 smithi069 ceph-mon[83631]: pgmap v231: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:26:55.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:54 smithi063 ceph-mon[112784]: pgmap v231: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:26:57.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:56 smithi069 ceph-mon[83631]: pgmap v232: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:26:57.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:56 smithi063 ceph-mon[112784]: pgmap v232: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:26:59.175 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:26:58 smithi063 ceph-mon[112784]: pgmap v233: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:26:59.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:26:58 smithi069 ceph-mon[83631]: pgmap v233: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:01.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:00 smithi069 ceph-mon[83631]: pgmap v234: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:01.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:00 smithi063 ceph-mon[112784]: pgmap v234: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:02.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:01 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:27:02.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:01 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:27:03.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:02 smithi063 ceph-mon[112784]: pgmap v235: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:03.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:02 smithi069 ceph-mon[83631]: pgmap v235: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:05.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:04 smithi069 ceph-mon[83631]: pgmap v236: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:05.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:04 smithi063 ceph-mon[112784]: pgmap v236: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:07.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:06 smithi069 ceph-mon[83631]: pgmap v237: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:27:07.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:06 smithi063 ceph-mon[112784]: pgmap v237: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:27:09.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:08 smithi069 ceph-mon[83631]: pgmap v238: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:09.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:08 smithi063 ceph-mon[112784]: pgmap v238: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:11.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:10 smithi069 ceph-mon[83631]: pgmap v239: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:11.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:10 smithi063 ceph-mon[112784]: pgmap v239: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:12.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:11 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:27:12.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:11 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-22T07:27:13.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:12 smithi063 ceph-mon[112784]: pgmap v240: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:13.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:27:13.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:27:13.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:12 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:27:13.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:12 smithi069 ceph-mon[83631]: pgmap v240: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:13.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-22T07:27:13.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-22T07:27:13.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:12 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' 2024-03-22T07:27:15.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:14 smithi069 ceph-mon[83631]: pgmap v241: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:15.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:14 smithi063 ceph-mon[112784]: pgmap v241: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:17.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:16 smithi069 ceph-mon[83631]: pgmap v242: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:27:17.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:16 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:27:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:16 smithi063 ceph-mon[112784]: pgmap v242: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:27:17.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:16 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:27:19.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:18 smithi069 ceph-mon[83631]: pgmap v243: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:19.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:18 smithi063 ceph-mon[112784]: pgmap v243: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:21.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:20 smithi069 ceph-mon[83631]: pgmap v244: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:21.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:20 smithi063 ceph-mon[112784]: pgmap v244: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:23.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:22 smithi063 ceph-mon[112784]: pgmap v245: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:23.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:22 smithi069 ceph-mon[83631]: pgmap v245: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:25.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:24 smithi069 ceph-mon[83631]: pgmap v246: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:25.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:24 smithi063 ceph-mon[112784]: pgmap v246: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:27.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:26 smithi069 ceph-mon[83631]: pgmap v247: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:27:27.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:26 smithi063 ceph-mon[112784]: pgmap v247: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:27:29.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:28 smithi069 ceph-mon[83631]: pgmap v248: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:29.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:28 smithi063 ceph-mon[112784]: pgmap v248: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:31.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:30 smithi069 ceph-mon[83631]: pgmap v249: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:31.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:30 smithi063 ceph-mon[112784]: pgmap v249: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:32.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:31 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:27:32.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:31 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:27:33.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:32 smithi063 ceph-mon[112784]: pgmap v250: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:33.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:32 smithi069 ceph-mon[83631]: pgmap v250: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:35.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:34 smithi069 ceph-mon[83631]: pgmap v251: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:35.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:34 smithi063 ceph-mon[112784]: pgmap v251: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:37.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:36 smithi069 ceph-mon[83631]: pgmap v252: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:27:37.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:36 smithi063 ceph-mon[112784]: pgmap v252: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:27:39.113 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:38 smithi063 ceph-mon[112784]: pgmap v253: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:39.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:38 smithi069 ceph-mon[83631]: pgmap v253: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:39.536 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph orch ps' 2024-03-22T07:27:39.806 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:27:41.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:40 smithi069 ceph-mon[83631]: pgmap v254: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:41.293 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:40 smithi063 ceph-mon[112784]: pgmap v254: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:41.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.534+0000 7f61c84f5700 1 -- 172.21.15.63:0/2757938350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61c01037a0 msgr2=0x7f61c00fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:41.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.534+0000 7f61c84f5700 1 --2- 172.21.15.63:0/2757938350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61c01037a0 0x7f61c00fce10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f61bc004940 tx=0x7f61bc009e30 comp rx=0 tx=0).stop 2024-03-22T07:27:41.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.535+0000 7f61c84f5700 1 -- 172.21.15.63:0/2757938350 shutdown_connections 2024-03-22T07:27:41.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.535+0000 7f61c84f5700 1 --2- 172.21.15.63:0/2757938350 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61c01037a0 0x7f61c00fce10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:41.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.535+0000 7f61c84f5700 1 --2- 172.21.15.63:0/2757938350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61c0102e00 0x7f61c01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:41.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.535+0000 7f61c84f5700 1 -- 172.21.15.63:0/2757938350 >> 172.21.15.63:0/2757938350 conn(0x7f61c00f89b0 msgr2=0x7f61c00fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:41.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.535+0000 7f61c84f5700 1 -- 172.21.15.63:0/2757938350 shutdown_connections 2024-03-22T07:27:41.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.535+0000 7f61c84f5700 1 -- 172.21.15.63:0/2757938350 wait complete. 2024-03-22T07:27:41.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.535+0000 7f61c84f5700 1 Processor -- start 2024-03-22T07:27:41.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.535+0000 7f61c84f5700 1 -- start start 2024-03-22T07:27:41.536 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.535+0000 7f61c84f5700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61c0102e00 0x7f61c00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:41.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.535+0000 7f61c84f5700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61c01037a0 0x7f61c0070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:41.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.535+0000 7f61c84f5700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c0070d40 con 0x7f61c01037a0 2024-03-22T07:27:41.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.535+0000 7f61c84f5700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c0070e80 con 0x7f61c0102e00 2024-03-22T07:27:41.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.536+0000 7f61c5a90700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61c01037a0 0x7f61c0070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:41.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.536+0000 7f61c5a90700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61c01037a0 0x7f61c0070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33072/0 (socket says 172.21.15.63:33072) 2024-03-22T07:27:41.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.536+0000 7f61c5a90700 1 -- 172.21.15.63:0/2698422871 learned_addr learned my addr 172.21.15.63:0/2698422871 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:41.537 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.536+0000 7f61c6291700 1 --2- 172.21.15.63:0/2698422871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61c0102e00 0x7f61c00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:41.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.536+0000 7f61c5a90700 1 -- 172.21.15.63:0/2698422871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61c0102e00 msgr2=0x7f61c00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:41.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.536+0000 7f61c5a90700 1 --2- 172.21.15.63:0/2698422871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61c0102e00 0x7f61c00702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:41.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.536+0000 7f61c5a90700 1 -- 172.21.15.63:0/2698422871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc005040 con 0x7f61c01037a0 2024-03-22T07:27:41.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.536+0000 7f61c6291700 1 --2- 172.21.15.63:0/2698422871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61c0102e00 0x7f61c00702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:27:41.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.536+0000 7f61c5a90700 1 --2- 172.21.15.63:0/2698422871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61c01037a0 0x7f61c0070800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f61bc009100 tx=0x7f61bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:41.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.536+0000 7f61b6ffd700 1 -- 172.21.15.63:0/2698422871 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61bc00e040 con 0x7f61c01037a0 2024-03-22T07:27:41.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.536+0000 7f61c84f5700 1 -- 172.21.15.63:0/2698422871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c00710d0 con 0x7f61c01037a0 2024-03-22T07:27:41.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.536+0000 7f61b6ffd700 1 -- 172.21.15.63:0/2698422871 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61bc004290 con 0x7f61c01037a0 2024-03-22T07:27:41.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.536+0000 7f61b6ffd700 1 -- 172.21.15.63:0/2698422871 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61bc01c3f0 con 0x7f61c01037a0 2024-03-22T07:27:41.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.536+0000 7f61c84f5700 1 -- 172.21.15.63:0/2698422871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c011a2e0 con 0x7f61c01037a0 2024-03-22T07:27:41.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.537+0000 7f61c84f5700 1 -- 172.21.15.63:0/2698422871 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c01052d0 con 0x7f61c01037a0 2024-03-22T07:27:41.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.538+0000 7f61b6ffd700 1 -- 172.21.15.63:0/2698422871 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f61bc013790 con 0x7f61c01037a0 2024-03-22T07:27:41.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.538+0000 7f61b6ffd700 1 --2- 172.21.15.63:0/2698422871 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f61ac06d390 0x7f61ac06f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:41.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.539+0000 7f61c6291700 1 --2- 172.21.15.63:0/2698422871 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f61ac06d390 0x7f61ac06f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:41.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.539+0000 7f61b6ffd700 1 -- 172.21.15.63:0/2698422871 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f61bc08dba0 con 0x7f61c01037a0 2024-03-22T07:27:41.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.539+0000 7f61c6291700 1 --2- 172.21.15.63:0/2698422871 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f61ac06d390 0x7f61ac06f840 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009fd0 tx=0x7f61b0009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:41.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.543+0000 7f61b6ffd700 1 -- 172.21.15.63:0/2698422871 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f61bc057650 con 0x7f61c01037a0 2024-03-22T07:27:41.756 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.756+0000 7f61c84f5700 1 -- 172.21.15.63:0/2698422871 --> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f61c0109ee0 con 0x7f61ac06d390 2024-03-22T07:27:41.766 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.766+0000 7f61b6ffd700 1 -- 172.21.15.63:0/2698422871 <== mgr.34104 v2:172.21.15.63:6800/1311134736 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f61c0109ee0 con 0x7f61ac06d390 2024-03-22T07:27:41.767 INFO:teuthology.orchestra.run.smithi063.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-22T07:27:41.767 INFO:teuthology.orchestra.run.smithi063.stdout:alertmanager.smithi063 smithi063 *:9093,9094 running (8m) 2m ago 19m 25.6M - 0.25.0 c8568f914cd2 06ef21da90e5 2024-03-22T07:27:41.767 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi063 smithi063 running (6m) 2m ago 19m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 224c291ab774 2024-03-22T07:27:41.767 INFO:teuthology.orchestra.run.smithi063.stdout:crash.smithi069 smithi069 running (6m) 97s ago 18m 6665k - 19.0.0-1667-gdb0330b1 4f325f8440e9 7eabb7d42631 2024-03-22T07:27:41.767 INFO:teuthology.orchestra.run.smithi063.stdout:grafana.smithi063 smithi063 *:3000 running (8m) 2m ago 19m 86.5M - 9.4.12 a0ea5e3c46d9 9015fae0d5e9 2024-03-22T07:27:41.767 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.oytapz smithi063 running (2m) 2m ago 14m 14.0M - 19.0.0-1667-gdb0330b1 4f325f8440e9 58fd92473761 2024-03-22T07:27:41.767 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi063.xeelpk smithi063 running (2m) 2m ago 14m 13.7M - 19.0.0-1667-gdb0330b1 4f325f8440e9 0946db898d5d 2024-03-22T07:27:41.767 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.cijxah smithi069 running (99s) 97s ago 13m 12.5M - 19.0.0-1667-gdb0330b1 4f325f8440e9 8d2544a6b620 2024-03-22T07:27:41.768 INFO:teuthology.orchestra.run.smithi063.stdout:mds.cephfs.smithi069.wwtsjn smithi069 running (115s) 97s ago 14m 13.9M - 19.0.0-1667-gdb0330b1 4f325f8440e9 dbce77644ed2 2024-03-22T07:27:41.768 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi063.mnxayh smithi063 *:8443,9283,8765 running (10m) 2m ago 20m 510M - 19.0.0-1667-gdb0330b1 4f325f8440e9 e49468f39991 2024-03-22T07:27:41.768 INFO:teuthology.orchestra.run.smithi063.stdout:mgr.smithi069.gfziwh smithi069 *:8443,9283,8765 running (9m) 97s ago 18m 451M - 19.0.0-1667-gdb0330b1 4f325f8440e9 13c3f034ebb2 2024-03-22T07:27:41.768 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi063 smithi063 running (7m) 2m ago 21m 54.5M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 ab96538a07c1 2024-03-22T07:27:41.768 INFO:teuthology.orchestra.run.smithi063.stdout:mon.smithi069 smithi069 running (6m) 97s ago 18m 44.6M 2048M 19.0.0-1667-gdb0330b1 4f325f8440e9 01a26e1fb1a1 2024-03-22T07:27:41.768 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi063 smithi063 *:9100 running (9m) 2m ago 19m 16.4M - 1.5.0 0da6a335fe13 0c825ee2a196 2024-03-22T07:27:41.768 INFO:teuthology.orchestra.run.smithi063.stdout:node-exporter.smithi069 smithi069 *:9100 running (9m) 97s ago 18m 16.5M - 1.5.0 0da6a335fe13 12edb2f35498 2024-03-22T07:27:41.768 INFO:teuthology.orchestra.run.smithi063.stdout:osd.0 smithi063 running (5m) 2m ago 17m 109M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 fdbc8b34ab78 2024-03-22T07:27:41.768 INFO:teuthology.orchestra.run.smithi063.stdout:osd.1 smithi063 running (5m) 2m ago 17m 101M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 5200fd893046 2024-03-22T07:27:41.768 INFO:teuthology.orchestra.run.smithi063.stdout:osd.2 smithi063 running (4m) 2m ago 16m 88.3M 1557M 19.0.0-1667-gdb0330b1 4f325f8440e9 918925ee69f3 2024-03-22T07:27:41.768 INFO:teuthology.orchestra.run.smithi063.stdout:osd.3 smithi069 running (3m) 97s ago 16m 102M 2581M 19.0.0-1667-gdb0330b1 4f325f8440e9 652a1e62339b 2024-03-22T07:27:41.768 INFO:teuthology.orchestra.run.smithi063.stdout:osd.4 smithi069 running (3m) 97s ago 16m 94.2M 2581M 19.0.0-1667-gdb0330b1 4f325f8440e9 d6340a0e1c6c 2024-03-22T07:27:41.768 INFO:teuthology.orchestra.run.smithi063.stdout:osd.5 smithi069 running (2m) 97s ago 15m 86.3M 2581M 19.0.0-1667-gdb0330b1 4f325f8440e9 c21401aa8e19 2024-03-22T07:27:41.768 INFO:teuthology.orchestra.run.smithi063.stdout:prometheus.smithi063 smithi063 *:9095 running (9m) 2m ago 19m 63.3M - 2.43.0 a07b618ecd1d 1d89b928f6e7 2024-03-22T07:27:41.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.768+0000 7f61c84f5700 1 -- 172.21.15.63:0/2698422871 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f61ac06d390 msgr2=0x7f61ac06f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:41.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.768+0000 7f61c84f5700 1 --2- 172.21.15.63:0/2698422871 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f61ac06d390 0x7f61ac06f840 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009fd0 tx=0x7f61b0009ee0 comp rx=0 tx=0).stop 2024-03-22T07:27:41.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.768+0000 7f61c84f5700 1 -- 172.21.15.63:0/2698422871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61c01037a0 msgr2=0x7f61c0070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:41.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.768+0000 7f61c84f5700 1 --2- 172.21.15.63:0/2698422871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61c01037a0 0x7f61c0070800 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f61bc009100 tx=0x7f61bc009e30 comp rx=0 tx=0).stop 2024-03-22T07:27:41.769 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.768+0000 7f61c84f5700 1 -- 172.21.15.63:0/2698422871 shutdown_connections 2024-03-22T07:27:41.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.768+0000 7f61c84f5700 1 --2- 172.21.15.63:0/2698422871 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f61ac06d390 0x7f61ac06f840 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:41.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.768+0000 7f61c84f5700 1 --2- 172.21.15.63:0/2698422871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f61c0102e00 0x7f61c00702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:41.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.768+0000 7f61c84f5700 1 --2- 172.21.15.63:0/2698422871 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f61c01037a0 0x7f61c0070800 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:41.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.768+0000 7f61c84f5700 1 -- 172.21.15.63:0/2698422871 >> 172.21.15.63:0/2698422871 conn(0x7f61c00f89b0 msgr2=0x7f61c0102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:41.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.768+0000 7f61c84f5700 1 -- 172.21.15.63:0/2698422871 shutdown_connections 2024-03-22T07:27:41.770 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:41.768+0000 7f61c84f5700 1 -- 172.21.15.63:0/2698422871 wait complete. 2024-03-22T07:27:42.518 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph versions' 2024-03-22T07:27:42.792 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:27:43.098 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:42 smithi063 ceph-mon[112784]: pgmap v255: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:43.098 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:42 smithi063 ceph-mon[112784]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:27:43.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:42 smithi069 ceph-mon[83631]: pgmap v255: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:43.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:42 smithi069 ceph-mon[83631]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-22T07:27:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.516+0000 7f1aaa897700 1 -- 172.21.15.63:0/3200401797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1aa4102f80 msgr2=0x7f1aa4103350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.516+0000 7f1aaa897700 1 --2- 172.21.15.63:0/3200401797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1aa4102f80 0x7f1aa4103350 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1a94004940 tx=0x7f1a94009e30 comp rx=0 tx=0).stop 2024-03-22T07:27:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.517+0000 7f1aaa897700 1 -- 172.21.15.63:0/3200401797 shutdown_connections 2024-03-22T07:27:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.517+0000 7f1aaa897700 1 --2- 172.21.15.63:0/3200401797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1aa40fb420 0x7f1aa40fb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.517+0000 7f1aaa897700 1 --2- 172.21.15.63:0/3200401797 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1aa4102f80 0x7f1aa4103350 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.517+0000 7f1aaa897700 1 -- 172.21.15.63:0/3200401797 >> 172.21.15.63:0/3200401797 conn(0x7f1aa4072380 msgr2=0x7f1aa4072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.517+0000 7f1aaa897700 1 -- 172.21.15.63:0/3200401797 shutdown_connections 2024-03-22T07:27:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.517+0000 7f1aaa897700 1 -- 172.21.15.63:0/3200401797 wait complete. 2024-03-22T07:27:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.517+0000 7f1aaa897700 1 Processor -- start 2024-03-22T07:27:44.519 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.517+0000 7f1aaa897700 1 -- start start 2024-03-22T07:27:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.518+0000 7f1aaa897700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1aa40fb420 0x7f1aa4113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.518+0000 7f1aaa897700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1aa4102f80 0x7f1aa4113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.518+0000 7f1aaa897700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa4114520 con 0x7f1aa40fb420 2024-03-22T07:27:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.518+0000 7f1aaa897700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa410db20 con 0x7f1aa4102f80 2024-03-22T07:27:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.518+0000 7f1aa3fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1aa40fb420 0x7f1aa4113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.518+0000 7f1aa3fff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1aa40fb420 0x7f1aa4113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:33086/0 (socket says 172.21.15.63:33086) 2024-03-22T07:27:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.518+0000 7f1aa3fff700 1 -- 172.21.15.63:0/3525456246 learned_addr learned my addr 172.21.15.63:0/3525456246 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.518+0000 7f1aa37fe700 1 --2- 172.21.15.63:0/3525456246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1aa4102f80 0x7f1aa4113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.518+0000 7f1aa3fff700 1 -- 172.21.15.63:0/3525456246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1aa4102f80 msgr2=0x7f1aa4113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.518+0000 7f1aa3fff700 1 --2- 172.21.15.63:0/3525456246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1aa4102f80 0x7f1aa4113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.518+0000 7f1aa3fff700 1 -- 172.21.15.63:0/3525456246 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a94005040 con 0x7f1aa40fb420 2024-03-22T07:27:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.518+0000 7f1aa37fe700 1 --2- 172.21.15.63:0/3525456246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1aa4102f80 0x7f1aa4113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:27:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.518+0000 7f1aa3fff700 1 --2- 172.21.15.63:0/3525456246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1aa40fb420 0x7f1aa4113990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a94009150 tx=0x7f1a94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.518+0000 7f1aa0ff9700 1 -- 172.21.15.63:0/3525456246 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a9400e040 con 0x7f1aa40fb420 2024-03-22T07:27:44.520 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.518+0000 7f1aa0ff9700 1 -- 172.21.15.63:0/3525456246 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a94013420 con 0x7f1aa40fb420 2024-03-22T07:27:44.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.518+0000 7f1aaa897700 1 -- 172.21.15.63:0/3525456246 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa410dda0 con 0x7f1aa40fb420 2024-03-22T07:27:44.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.518+0000 7f1aa0ff9700 1 -- 172.21.15.63:0/3525456246 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a9401c630 con 0x7f1aa40fb420 2024-03-22T07:27:44.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.518+0000 7f1aaa897700 1 -- 172.21.15.63:0/3525456246 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa410e2e0 con 0x7f1aa40fb420 2024-03-22T07:27:44.521 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.519+0000 7f1aaa897700 1 -- 172.21.15.63:0/3525456246 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa4100150 con 0x7f1aa40fb420 2024-03-22T07:27:44.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.520+0000 7f1aa0ff9700 1 -- 172.21.15.63:0/3525456246 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f1a94013590 con 0x7f1aa40fb420 2024-03-22T07:27:44.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.520+0000 7f1aa0ff9700 1 --2- 172.21.15.63:0/3525456246 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1a8c06d380 0x7f1a8c06f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:44.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.520+0000 7f1aa0ff9700 1 -- 172.21.15.63:0/3525456246 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1a9408dbe0 con 0x7f1aa40fb420 2024-03-22T07:27:44.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.520+0000 7f1aa37fe700 1 --2- 172.21.15.63:0/3525456246 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1a8c06d380 0x7f1a8c06f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:44.522 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.521+0000 7f1aa37fe700 1 --2- 172.21.15.63:0/3525456246 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1a8c06d380 0x7f1a8c06f830 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009fd0 tx=0x7f1a9800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:44.525 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.525+0000 7f1aa0ff9700 1 -- 172.21.15.63:0/3525456246 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1a9405b1c0 con 0x7f1aa40fb420 2024-03-22T07:27:44.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.813+0000 7f1aaa897700 1 -- 172.21.15.63:0/3525456246 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1aa410efe0 con 0x7f1aa40fb420 2024-03-22T07:27:44.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.814+0000 7f1aa0ff9700 1 -- 172.21.15.63:0/3525456246 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f1a9405ad50 con 0x7f1aa40fb420 2024-03-22T07:27:44.814 INFO:teuthology.orchestra.run.smithi063.stdout:{ 2024-03-22T07:27:44.814 INFO:teuthology.orchestra.run.smithi063.stdout: "mon": { 2024-03-22T07:27:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:27:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:27:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "mgr": { 2024-03-22T07:27:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 2 2024-03-22T07:27:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:27:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "osd": { 2024-03-22T07:27:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 6 2024-03-22T07:27:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:27:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "mds": { 2024-03-22T07:27:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 4 2024-03-22T07:27:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: }, 2024-03-22T07:27:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "overall": { 2024-03-22T07:27:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)": 14 2024-03-22T07:27:44.815 INFO:teuthology.orchestra.run.smithi063.stdout: } 2024-03-22T07:27:44.815 INFO:teuthology.orchestra.run.smithi063.stdout:} 2024-03-22T07:27:44.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.816+0000 7f1aaa897700 1 -- 172.21.15.63:0/3525456246 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1a8c06d380 msgr2=0x7f1a8c06f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:44.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.816+0000 7f1aaa897700 1 --2- 172.21.15.63:0/3525456246 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1a8c06d380 0x7f1a8c06f830 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009fd0 tx=0x7f1a9800b040 comp rx=0 tx=0).stop 2024-03-22T07:27:44.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.816+0000 7f1aaa897700 1 -- 172.21.15.63:0/3525456246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1aa40fb420 msgr2=0x7f1aa4113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:44.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.816+0000 7f1aaa897700 1 --2- 172.21.15.63:0/3525456246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1aa40fb420 0x7f1aa4113990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a94009150 tx=0x7f1a94009e30 comp rx=0 tx=0).stop 2024-03-22T07:27:44.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.816+0000 7f1aaa897700 1 -- 172.21.15.63:0/3525456246 shutdown_connections 2024-03-22T07:27:44.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.816+0000 7f1aaa897700 1 --2- 172.21.15.63:0/3525456246 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1a8c06d380 0x7f1a8c06f830 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:44.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.816+0000 7f1aaa897700 1 --2- 172.21.15.63:0/3525456246 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1aa40fb420 0x7f1aa4113990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:44.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.816+0000 7f1aaa897700 1 --2- 172.21.15.63:0/3525456246 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1aa4102f80 0x7f1aa4113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:44.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.816+0000 7f1aaa897700 1 -- 172.21.15.63:0/3525456246 >> 172.21.15.63:0/3525456246 conn(0x7f1aa4072380 msgr2=0x7f1aa4107480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:44.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.817+0000 7f1aaa897700 1 -- 172.21.15.63:0/3525456246 shutdown_connections 2024-03-22T07:27:44.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:44.817+0000 7f1aaa897700 1 -- 172.21.15.63:0/3525456246 wait complete. 2024-03-22T07:27:45.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:45 smithi063 ceph-mon[112784]: pgmap v256: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:45.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:45 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3525456246' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:27:45.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:45 smithi069 ceph-mon[83631]: pgmap v256: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:45.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:45 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3525456246' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:27:45.506 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-03-22T07:27:45.771 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:27:47.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:47 smithi063 ceph-mon[112784]: pgmap v257: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:27:47.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:47 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:27:47.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:47 smithi069 ceph-mon[83631]: pgmap v257: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:27:47.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:47 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:27:47.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.392+0000 7efd34077700 1 -- 172.21.15.63:0/3099841112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c0fb4a0 msgr2=0x7efd2c0fb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:47.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.392+0000 7efd34077700 1 --2- 172.21.15.63:0/3099841112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c0fb4a0 0x7efd2c0fb910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efd28004940 tx=0x7efd28009e30 comp rx=0 tx=0).stop 2024-03-22T07:27:47.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.392+0000 7efd34077700 1 -- 172.21.15.63:0/3099841112 shutdown_connections 2024-03-22T07:27:47.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.392+0000 7efd34077700 1 --2- 172.21.15.63:0/3099841112 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c0fb4a0 0x7efd2c0fb910 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:47.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.392+0000 7efd34077700 1 --2- 172.21.15.63:0/3099841112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd2c102e70 0x7efd2c103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:47.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.392+0000 7efd34077700 1 -- 172.21.15.63:0/3099841112 >> 172.21.15.63:0/3099841112 conn(0x7efd2c0714c0 msgr2=0x7efd2c0718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:47.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.392+0000 7efd34077700 1 -- 172.21.15.63:0/3099841112 shutdown_connections 2024-03-22T07:27:47.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.392+0000 7efd34077700 1 -- 172.21.15.63:0/3099841112 wait complete. 2024-03-22T07:27:47.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.393+0000 7efd34077700 1 Processor -- start 2024-03-22T07:27:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.393+0000 7efd34077700 1 -- start start 2024-03-22T07:27:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.393+0000 7efd34077700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd2c0fb4a0 0x7efd2c113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.393+0000 7efd34077700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c102e70 0x7efd2c1140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.393+0000 7efd34077700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c114700 con 0x7efd2c102e70 2024-03-22T07:27:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.393+0000 7efd34077700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2c10daf0 con 0x7efd2c0fb4a0 2024-03-22T07:27:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.393+0000 7efd31612700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c102e70 0x7efd2c1140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.393+0000 7efd31612700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c102e70 0x7efd2c1140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46742/0 (socket says 172.21.15.63:46742) 2024-03-22T07:27:47.395 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.393+0000 7efd31612700 1 -- 172.21.15.63:0/4025482872 learned_addr learned my addr 172.21.15.63:0/4025482872 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:47.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.393+0000 7efd31e13700 1 --2- 172.21.15.63:0/4025482872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd2c0fb4a0 0x7efd2c113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:47.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.393+0000 7efd31612700 1 -- 172.21.15.63:0/4025482872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd2c0fb4a0 msgr2=0x7efd2c113b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:47.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.393+0000 7efd31612700 1 --2- 172.21.15.63:0/4025482872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd2c0fb4a0 0x7efd2c113b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:47.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.393+0000 7efd31612700 1 -- 172.21.15.63:0/4025482872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd28005040 con 0x7efd2c102e70 2024-03-22T07:27:47.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.394+0000 7efd31612700 1 --2- 172.21.15.63:0/4025482872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c102e70 0x7efd2c1140b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efd2800b3b0 tx=0x7efd28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:47.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.394+0000 7efd227fc700 1 -- 172.21.15.63:0/4025482872 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd2800e040 con 0x7efd2c102e70 2024-03-22T07:27:47.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.394+0000 7efd34077700 1 -- 172.21.15.63:0/4025482872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd2c10dd70 con 0x7efd2c102e70 2024-03-22T07:27:47.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.394+0000 7efd227fc700 1 -- 172.21.15.63:0/4025482872 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd28013420 con 0x7efd2c102e70 2024-03-22T07:27:47.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.394+0000 7efd227fc700 1 -- 172.21.15.63:0/4025482872 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd2801c630 con 0x7efd2c102e70 2024-03-22T07:27:47.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.394+0000 7efd34077700 1 -- 172.21.15.63:0/4025482872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd2c10e330 con 0x7efd2c102e70 2024-03-22T07:27:47.396 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.395+0000 7efd34077700 1 -- 172.21.15.63:0/4025482872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd2c100110 con 0x7efd2c102e70 2024-03-22T07:27:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.395+0000 7efd227fc700 1 -- 172.21.15.63:0/4025482872 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7efd28013590 con 0x7efd2c102e70 2024-03-22T07:27:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.396+0000 7efd227fc700 1 --2- 172.21.15.63:0/4025482872 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7efd1806d380 0x7efd1806f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.396+0000 7efd227fc700 1 -- 172.21.15.63:0/4025482872 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7efd2808db70 con 0x7efd2c102e70 2024-03-22T07:27:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.396+0000 7efd31e13700 1 --2- 172.21.15.63:0/4025482872 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7efd1806d380 0x7efd1806f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:47.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.396+0000 7efd31e13700 1 --2- 172.21.15.63:0/4025482872 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7efd1806d380 0x7efd1806f830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efd1c004f40 tx=0x7efd1c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:47.401 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.400+0000 7efd227fc700 1 -- 172.21.15.63:0/4025482872 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efd2805b150 con 0x7efd2c102e70 2024-03-22T07:27:47.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.696+0000 7efd34077700 1 -- 172.21.15.63:0/4025482872 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efd2c05f620 con 0x7efd2c102e70 2024-03-22T07:27:47.697 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.697+0000 7efd227fc700 1 -- 172.21.15.63:0/4025482872 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7efd2805ace0 con 0x7efd2c102e70 2024-03-22T07:27:47.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.699+0000 7efd34077700 1 -- 172.21.15.63:0/4025482872 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7efd1806d380 msgr2=0x7efd1806f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:47.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.699+0000 7efd34077700 1 --2- 172.21.15.63:0/4025482872 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7efd1806d380 0x7efd1806f830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efd1c004f40 tx=0x7efd1c00b040 comp rx=0 tx=0).stop 2024-03-22T07:27:47.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.699+0000 7efd34077700 1 -- 172.21.15.63:0/4025482872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c102e70 msgr2=0x7efd2c1140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:47.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.699+0000 7efd34077700 1 --2- 172.21.15.63:0/4025482872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c102e70 0x7efd2c1140b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efd2800b3b0 tx=0x7efd28009e30 comp rx=0 tx=0).stop 2024-03-22T07:27:47.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.699+0000 7efd34077700 1 -- 172.21.15.63:0/4025482872 shutdown_connections 2024-03-22T07:27:47.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.699+0000 7efd34077700 1 --2- 172.21.15.63:0/4025482872 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7efd1806d380 0x7efd1806f830 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:47.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.699+0000 7efd34077700 1 --2- 172.21.15.63:0/4025482872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd2c0fb4a0 0x7efd2c113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:47.699 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.699+0000 7efd34077700 1 --2- 172.21.15.63:0/4025482872 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7efd2c102e70 0x7efd2c1140b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:47.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.699+0000 7efd34077700 1 -- 172.21.15.63:0/4025482872 >> 172.21.15.63:0/4025482872 conn(0x7efd2c0714c0 msgr2=0x7efd2c0fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:47.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.699+0000 7efd34077700 1 -- 172.21.15.63:0/4025482872 shutdown_connections 2024-03-22T07:27:47.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:47.699+0000 7efd34077700 1 -- 172.21.15.63:0/4025482872 wait complete. 2024-03-22T07:27:47.709 INFO:teuthology.orchestra.run.smithi063.stdout:true 2024-03-22T07:27:48.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:48 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/4025482872' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:27:48.287 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:48 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/4025482872' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:27:48.313 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -e sha1=db0330b1e4e2470d52b750e251e55a522b4f7d69 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-03-22T07:27:48.577 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:27:49.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:49 smithi069 ceph-mon[83631]: pgmap v258: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:49.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:49 smithi063 ceph-mon[112784]: pgmap v258: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:50.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.236+0000 7f221686d700 1 -- 172.21.15.63:0/1990442325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2210102710 msgr2=0x7f22100fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:50.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.236+0000 7f221686d700 1 --2- 172.21.15.63:0/1990442325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2210102710 0x7f22100fbd80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f21f8004940 tx=0x7f21f8009e30 comp rx=0 tx=0).stop 2024-03-22T07:27:50.236 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.236+0000 7f221686d700 1 -- 172.21.15.63:0/1990442325 shutdown_connections 2024-03-22T07:27:50.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.236+0000 7f221686d700 1 --2- 172.21.15.63:0/1990442325 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2210102710 0x7f22100fbd80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:50.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.236+0000 7f221686d700 1 --2- 172.21.15.63:0/1990442325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2210101d70 0x7f2210102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:50.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.236+0000 7f221686d700 1 -- 172.21.15.63:0/1990442325 >> 172.21.15.63:0/1990442325 conn(0x7f2210072bc0 msgr2=0x7f2210072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:50.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.236+0000 7f221686d700 1 -- 172.21.15.63:0/1990442325 shutdown_connections 2024-03-22T07:27:50.237 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.236+0000 7f221686d700 1 -- 172.21.15.63:0/1990442325 wait complete. 2024-03-22T07:27:50.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.237+0000 7f221686d700 1 Processor -- start 2024-03-22T07:27:50.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.237+0000 7f221686d700 1 -- start start 2024-03-22T07:27:50.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.237+0000 7f221686d700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2210101d70 0x7f2210112aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:50.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.237+0000 7f221686d700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2210102710 0x7f2210112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:50.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.237+0000 7f221686d700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2210113630 con 0x7f2210101d70 2024-03-22T07:27:50.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.237+0000 7f221686d700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221010ca20 con 0x7f2210102710 2024-03-22T07:27:50.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.237+0000 7f220ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2210101d70 0x7f2210112aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:50.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.237+0000 7f220ffff700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2210101d70 0x7f2210112aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46752/0 (socket says 172.21.15.63:46752) 2024-03-22T07:27:50.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.237+0000 7f220ffff700 1 -- 172.21.15.63:0/4205151390 learned_addr learned my addr 172.21.15.63:0/4205151390 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:50.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.237+0000 7f220f7fe700 1 --2- 172.21.15.63:0/4205151390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2210102710 0x7f2210112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:50.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.237+0000 7f220ffff700 1 -- 172.21.15.63:0/4205151390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2210102710 msgr2=0x7f2210112fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:50.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.237+0000 7f220ffff700 1 --2- 172.21.15.63:0/4205151390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2210102710 0x7f2210112fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:50.239 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.238+0000 7f220ffff700 1 -- 172.21.15.63:0/4205151390 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f8005040 con 0x7f2210101d70 2024-03-22T07:27:50.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.238+0000 7f220ffff700 1 --2- 172.21.15.63:0/4205151390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2210101d70 0x7f2210112aa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2200002a50 tx=0x7f22000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:50.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.238+0000 7f220cff9700 1 -- 172.21.15.63:0/4205151390 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2200007cc0 con 0x7f2210101d70 2024-03-22T07:27:50.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.238+0000 7f221686d700 1 -- 172.21.15.63:0/4205151390 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f221010cd00 con 0x7f2210101d70 2024-03-22T07:27:50.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.238+0000 7f220cff9700 1 -- 172.21.15.63:0/4205151390 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f220000f040 con 0x7f2210101d70 2024-03-22T07:27:50.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.238+0000 7f220cff9700 1 -- 172.21.15.63:0/4205151390 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f220000da30 con 0x7f2210101d70 2024-03-22T07:27:50.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.238+0000 7f221686d700 1 -- 172.21.15.63:0/4205151390 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f221010d290 con 0x7f2210101d70 2024-03-22T07:27:50.240 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.239+0000 7f221686d700 1 -- 172.21.15.63:0/4205151390 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2210104240 con 0x7f2210101d70 2024-03-22T07:27:50.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.239+0000 7f220cff9700 1 -- 172.21.15.63:0/4205151390 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f220000db90 con 0x7f2210101d70 2024-03-22T07:27:50.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.240+0000 7f220cff9700 1 --2- 172.21.15.63:0/4205151390 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f21fc06d390 0x7f21fc06f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:50.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.240+0000 7f220cff9700 1 -- 172.21.15.63:0/4205151390 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f220008c700 con 0x7f2210101d70 2024-03-22T07:27:50.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.240+0000 7f220f7fe700 1 --2- 172.21.15.63:0/4205151390 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f21fc06d390 0x7f21fc06f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:50.241 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.240+0000 7f220f7fe700 1 --2- 172.21.15.63:0/4205151390 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f21fc06d390 0x7f21fc06f840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f221010dd00 tx=0x7f21f8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:50.244 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.245+0000 7f220cff9700 1 -- 172.21.15.63:0/4205151390 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2200059c60 con 0x7f2210101d70 2024-03-22T07:27:50.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.532+0000 7f221686d700 1 -- 172.21.15.63:0/4205151390 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f221010e150 con 0x7f2210101d70 2024-03-22T07:27:50.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.532+0000 7f220cff9700 1 -- 172.21.15.63:0/4205151390 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f22000597f0 con 0x7f2210101d70 2024-03-22T07:27:50.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.534+0000 7f221686d700 1 -- 172.21.15.63:0/4205151390 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f21fc06d390 msgr2=0x7f21fc06f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:50.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.534+0000 7f221686d700 1 --2- 172.21.15.63:0/4205151390 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f21fc06d390 0x7f21fc06f840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f221010dd00 tx=0x7f21f8009e30 comp rx=0 tx=0).stop 2024-03-22T07:27:50.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.534+0000 7f221686d700 1 -- 172.21.15.63:0/4205151390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2210101d70 msgr2=0x7f2210112aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.534+0000 7f221686d700 1 --2- 172.21.15.63:0/4205151390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2210101d70 0x7f2210112aa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2200002a50 tx=0x7f22000046b0 comp rx=0 tx=0).stop 2024-03-22T07:27:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.534+0000 7f221686d700 1 -- 172.21.15.63:0/4205151390 shutdown_connections 2024-03-22T07:27:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.534+0000 7f221686d700 1 --2- 172.21.15.63:0/4205151390 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f21fc06d390 0x7f21fc06f840 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.534+0000 7f221686d700 1 --2- 172.21.15.63:0/4205151390 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2210101d70 0x7f2210112aa0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.534+0000 7f221686d700 1 --2- 172.21.15.63:0/4205151390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2210102710 0x7f2210112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.534+0000 7f221686d700 1 -- 172.21.15.63:0/4205151390 >> 172.21.15.63:0/4205151390 conn(0x7f2210072bc0 msgr2=0x7f22100fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.534+0000 7f221686d700 1 -- 172.21.15.63:0/4205151390 shutdown_connections 2024-03-22T07:27:50.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:50.535+0000 7f221686d700 1 -- 172.21.15.63:0/4205151390 wait complete. 2024-03-22T07:27:50.544 INFO:teuthology.orchestra.run.smithi063.stdout: "ceph version 19.0.0-1667-gdb0330b1 (db0330b1e4e2470d52b750e251e55a522b4f7d69) squid (dev)" 2024-03-22T07:27:51.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:51 smithi063 ceph-mon[112784]: pgmap v259: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:51.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:51 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/4205151390' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:27:51.179 DEBUG:teuthology.parallel:result is None 2024-03-22T07:27:51.179 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-22T07:27:51.190 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi063.front.sepia.ceph.com 2024-03-22T07:27:51.190 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-22T07:27:51.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:51 smithi069 ceph-mon[83631]: pgmap v259: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:51.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:51 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/4205151390' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-22T07:27:51.453 INFO:teuthology.orchestra.run.smithi063.stderr:Inferring config /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/mon.smithi063/config 2024-03-22T07:27:53.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.152+0000 7fe4b1b11700 1 -- 172.21.15.63:0/2541668653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4ac101d70 msgr2=0x7fe4ac102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:53.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.152+0000 7fe4b1b11700 1 --2- 172.21.15.63:0/2541668653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4ac101d70 0x7fe4ac102140 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009ce0 tx=0x7fe49c004d10 comp rx=0 tx=0).stop 2024-03-22T07:27:53.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.152+0000 7fe4b1b11700 1 -- 172.21.15.63:0/2541668653 shutdown_connections 2024-03-22T07:27:53.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.152+0000 7fe4b1b11700 1 --2- 172.21.15.63:0/2541668653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4ac102710 0x7fe4ac0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:53.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.152+0000 7fe4b1b11700 1 --2- 172.21.15.63:0/2541668653 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4ac101d70 0x7fe4ac102140 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:53.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.152+0000 7fe4b1b11700 1 -- 172.21.15.63:0/2541668653 >> 172.21.15.63:0/2541668653 conn(0x7fe4ac072bc0 msgr2=0x7fe4ac072fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:53.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.152+0000 7fe4b1b11700 1 -- 172.21.15.63:0/2541668653 shutdown_connections 2024-03-22T07:27:53.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.152+0000 7fe4b1b11700 1 -- 172.21.15.63:0/2541668653 wait complete. 2024-03-22T07:27:53.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.152+0000 7fe4b1b11700 1 Processor -- start 2024-03-22T07:27:53.154 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.153+0000 7fe4b1b11700 1 -- start start 2024-03-22T07:27:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.153+0000 7fe4b1b11700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4ac101d70 0x7fe4ac10cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.153+0000 7fe4b1b11700 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4ac102710 0x7fe4ac10d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.153+0000 7fe4b1b11700 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac10d550 con 0x7fe4ac101d70 2024-03-22T07:27:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.153+0000 7fe4b1b11700 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac10d6c0 con 0x7fe4ac102710 2024-03-22T07:27:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.153+0000 7fe4ab7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4ac101d70 0x7fe4ac10cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.153+0000 7fe4ab7fe700 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4ac101d70 0x7fe4ac10cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46766/0 (socket says 172.21.15.63:46766) 2024-03-22T07:27:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.153+0000 7fe4ab7fe700 1 -- 172.21.15.63:0/2568959839 learned_addr learned my addr 172.21.15.63:0/2568959839 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.153+0000 7fe4aaffd700 1 --2- 172.21.15.63:0/2568959839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4ac102710 0x7fe4ac10d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.153+0000 7fe4ab7fe700 1 -- 172.21.15.63:0/2568959839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4ac102710 msgr2=0x7fe4ac10d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.153+0000 7fe4ab7fe700 1 --2- 172.21.15.63:0/2568959839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4ac102710 0x7fe4ac10d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.153+0000 7fe4ab7fe700 1 -- 172.21.15.63:0/2568959839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c009940 con 0x7fe4ac101d70 2024-03-22T07:27:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.153+0000 7fe4ab7fe700 1 --2- 172.21.15.63:0/2568959839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4ac101d70 0x7fe4ac10cad0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe49c005010 tx=0x7fe49c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.154+0000 7fe4a3fff700 1 -- 172.21.15.63:0/2568959839 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe49c00e040 con 0x7fe4ac101d70 2024-03-22T07:27:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.154+0000 7fe4a3fff700 1 -- 172.21.15.63:0/2568959839 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe49c007d10 con 0x7fe4ac101d70 2024-03-22T07:27:53.155 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.154+0000 7fe4b1b11700 1 -- 172.21.15.63:0/2568959839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ac10d940 con 0x7fe4ac101d70 2024-03-22T07:27:53.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.154+0000 7fe4a3fff700 1 -- 172.21.15.63:0/2568959839 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe49c012650 con 0x7fe4ac101d70 2024-03-22T07:27:53.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.154+0000 7fe4b1b11700 1 -- 172.21.15.63:0/2568959839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ac11a4c0 con 0x7fe4ac101d70 2024-03-22T07:27:53.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.155+0000 7fe4b1b11700 1 -- 172.21.15.63:0/2568959839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ac0ff080 con 0x7fe4ac101d70 2024-03-22T07:27:53.156 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.155+0000 7fe4a3fff700 1 -- 172.21.15.63:0/2568959839 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fe49c01ca10 con 0x7fe4ac101d70 2024-03-22T07:27:53.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.156+0000 7fe4a3fff700 1 --2- 172.21.15.63:0/2568959839 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe49806d390 0x7fe49806f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:53.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.156+0000 7fe4a3fff700 1 -- 172.21.15.63:0/2568959839 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe49c08e350 con 0x7fe4ac101d70 2024-03-22T07:27:53.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.156+0000 7fe4aaffd700 1 --2- 172.21.15.63:0/2568959839 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe49806d390 0x7fe49806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:53.157 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.156+0000 7fe4aaffd700 1 --2- 172.21.15.63:0/2568959839 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe49806d390 0x7fe49806f840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac066000 tx=0x7fe49400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:53.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.160+0000 7fe4a3fff700 1 -- 172.21.15.63:0/2568959839 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe49c05b8b0 con 0x7fe4ac101d70 2024-03-22T07:27:53.242 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:53 smithi063 ceph-mon[112784]: pgmap v260: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:53.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:53 smithi069 ceph-mon[83631]: pgmap v260: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:53.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.416+0000 7fe4b1b11700 1 -- 172.21.15.63:0/2568959839 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe4ac10d8e0 con 0x7fe4ac101d70 2024-03-22T07:27:53.416 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.416+0000 7fe4a3fff700 1 -- 172.21.15.63:0/2568959839 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 44 v44) v1 ==== 76+0+1890 (secure 0 0 0) 0x7fe49c004e10 con 0x7fe4ac101d70 2024-03-22T07:27:53.417 INFO:teuthology.orchestra.run.smithi063.stdout:e44 2024-03-22T07:27:53.417 INFO:teuthology.orchestra.run.smithi063.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-22T07:27:53.417 INFO:teuthology.orchestra.run.smithi063.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-22T07:27:53.417 INFO:teuthology.orchestra.run.smithi063.stdout:legacy client fscid: 1 2024-03-22T07:27:53.417 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:27:53.417 INFO:teuthology.orchestra.run.smithi063.stdout:Filesystem 'cephfs' (1) 2024-03-22T07:27:53.417 INFO:teuthology.orchestra.run.smithi063.stdout:fs_name cephfs 2024-03-22T07:27:53.417 INFO:teuthology.orchestra.run.smithi063.stdout:epoch 44 2024-03-22T07:27:53.417 INFO:teuthology.orchestra.run.smithi063.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-22T07:27:53.417 INFO:teuthology.orchestra.run.smithi063.stdout:created 2024-03-22T07:13:29.602905+0000 2024-03-22T07:27:53.417 INFO:teuthology.orchestra.run.smithi063.stdout:modified 2024-03-22T07:26:22.473650+0000 2024-03-22T07:27:53.417 INFO:teuthology.orchestra.run.smithi063.stdout:tableserver 0 2024-03-22T07:27:53.418 INFO:teuthology.orchestra.run.smithi063.stdout:root 0 2024-03-22T07:27:53.418 INFO:teuthology.orchestra.run.smithi063.stdout:session_timeout 60 2024-03-22T07:27:53.418 INFO:teuthology.orchestra.run.smithi063.stdout:session_autoclose 300 2024-03-22T07:27:53.418 INFO:teuthology.orchestra.run.smithi063.stdout:max_file_size 1099511627776 2024-03-22T07:27:53.418 INFO:teuthology.orchestra.run.smithi063.stdout:max_xattr_size 65536 2024-03-22T07:27:53.418 INFO:teuthology.orchestra.run.smithi063.stdout:required_client_features {} 2024-03-22T07:27:53.418 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure 0 2024-03-22T07:27:53.418 INFO:teuthology.orchestra.run.smithi063.stdout:last_failure_osd_epoch 83 2024-03-22T07:27:53.418 INFO:teuthology.orchestra.run.smithi063.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-22T07:27:53.418 INFO:teuthology.orchestra.run.smithi063.stdout:max_mds 2 2024-03-22T07:27:53.418 INFO:teuthology.orchestra.run.smithi063.stdout:in 0,1 2024-03-22T07:27:53.418 INFO:teuthology.orchestra.run.smithi063.stdout:up {0=34400,1=34424} 2024-03-22T07:27:53.418 INFO:teuthology.orchestra.run.smithi063.stdout:failed 2024-03-22T07:27:53.419 INFO:teuthology.orchestra.run.smithi063.stdout:damaged 2024-03-22T07:27:53.419 INFO:teuthology.orchestra.run.smithi063.stdout:stopped 2024-03-22T07:27:53.419 INFO:teuthology.orchestra.run.smithi063.stdout:data_pools [3] 2024-03-22T07:27:53.419 INFO:teuthology.orchestra.run.smithi063.stdout:metadata_pool 2 2024-03-22T07:27:53.419 INFO:teuthology.orchestra.run.smithi063.stdout:inline_data disabled 2024-03-22T07:27:53.419 INFO:teuthology.orchestra.run.smithi063.stdout:balancer 2024-03-22T07:27:53.419 INFO:teuthology.orchestra.run.smithi063.stdout:bal_rank_mask -1 2024-03-22T07:27:53.419 INFO:teuthology.orchestra.run.smithi063.stdout:standby_count_wanted 1 2024-03-22T07:27:53.419 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.oytapz{0:34400} state up:active seq 22 join_fscid=1 addr [v2:172.21.15.63:6826/1171423403,v1:172.21.15.63:6827/1171423403] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T07:27:53.419 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.wwtsjn{0:44323} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.69:6824/653947973,v1:172.21.15.69:6825/653947973] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T07:27:53.419 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi063.xeelpk{1:34424} state up:active seq 16 join_fscid=1 addr [v2:172.21.15.63:6828/1190308435,v1:172.21.15.63:6829/1190308435] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T07:27:53.419 INFO:teuthology.orchestra.run.smithi063.stdout:[mds.cephfs.smithi069.cijxah{1:44327} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.69:6826/3465454734,v1:172.21.15.69:6827/3465454734] compat {c=[1],r=[1],i=[fff]}] 2024-03-22T07:27:53.419 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:27:53.419 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:27:53.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.419+0000 7fe4b1b11700 1 -- 172.21.15.63:0/2568959839 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe49806d390 msgr2=0x7fe49806f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:53.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.419+0000 7fe4b1b11700 1 --2- 172.21.15.63:0/2568959839 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe49806d390 0x7fe49806f840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac066000 tx=0x7fe49400b040 comp rx=0 tx=0).stop 2024-03-22T07:27:53.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.419+0000 7fe4b1b11700 1 -- 172.21.15.63:0/2568959839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4ac101d70 msgr2=0x7fe4ac10cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:53.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.419+0000 7fe4b1b11700 1 --2- 172.21.15.63:0/2568959839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4ac101d70 0x7fe4ac10cad0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe49c005010 tx=0x7fe49c004d10 comp rx=0 tx=0).stop 2024-03-22T07:27:53.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.419+0000 7fe4b1b11700 1 -- 172.21.15.63:0/2568959839 shutdown_connections 2024-03-22T07:27:53.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.419+0000 7fe4b1b11700 1 --2- 172.21.15.63:0/2568959839 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe49806d390 0x7fe49806f840 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:53.420 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.419+0000 7fe4b1b11700 1 --2- 172.21.15.63:0/2568959839 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe4ac101d70 0x7fe4ac10cad0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:53.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.419+0000 7fe4b1b11700 1 --2- 172.21.15.63:0/2568959839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4ac102710 0x7fe4ac10d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:53.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.419+0000 7fe4b1b11700 1 -- 172.21.15.63:0/2568959839 >> 172.21.15.63:0/2568959839 conn(0x7fe4ac072bc0 msgr2=0x7fe4ac0fb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:53.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.419+0000 7fe4b1b11700 1 -- 172.21.15.63:0/2568959839 shutdown_connections 2024-03-22T07:27:53.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:53.419+0000 7fe4b1b11700 1 -- 172.21.15.63:0/2568959839 wait complete. 2024-03-22T07:27:53.421 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 44 2024-03-22T07:27:54.088 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:54 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2568959839' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:27:54.110 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-03-22T07:27:54.121 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T07:27:54.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.222+0000 7f64c811d640 1 -- 172.21.15.63:0/961852299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64c0101ad0 msgr2=0x7f64c00fb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:54.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.222+0000 7f64c811d640 1 --2- 172.21.15.63:0/961852299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64c0101ad0 0x7f64c00fb160 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f64b40048d0 tx=0x7f64b4009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.222+0000 7f64c811d640 1 -- 172.21.15.63:0/961852299 shutdown_connections 2024-03-22T07:27:54.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.222+0000 7f64c811d640 1 --2- 172.21.15.63:0/961852299 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64c0101ad0 0x7f64c00fb160 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.224 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.222+0000 7f64c811d640 1 --2- 172.21.15.63:0/961852299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64c0101130 0x7f64c0101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.222+0000 7f64c811d640 1 -- 172.21.15.63:0/961852299 >> 172.21.15.63:0/961852299 conn(0x7f64c00f6ce0 msgr2=0x7f64c00f90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.222+0000 7f64c811d640 1 -- 172.21.15.63:0/961852299 shutdown_connections 2024-03-22T07:27:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.222+0000 7f64c811d640 1 -- 172.21.15.63:0/961852299 wait complete. 2024-03-22T07:27:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.223+0000 7f64c811d640 1 Processor -- start 2024-03-22T07:27:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.223+0000 7f64c811d640 1 -- start start 2024-03-22T07:27:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.223+0000 7f64c811d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64c0101130 0x7f64c010dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.223+0000 7f64c811d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64c0101ad0 0x7f64c010e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.223+0000 7f64c811d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c0066510 con 0x7f64c0101130 2024-03-22T07:27:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.223+0000 7f64c811d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c010e920 con 0x7f64c0101ad0 2024-03-22T07:27:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.223+0000 7f64c5eb9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64c0101130 0x7f64c010dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.223+0000 7f64c5eb9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64c0101130 0x7f64c010dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46780/0 (socket says 172.21.15.63:46780) 2024-03-22T07:27:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.223+0000 7f64c5eb9640 1 -- 172.21.15.63:0/2148967960 learned_addr learned my addr 172.21.15.63:0/2148967960 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.223+0000 7f64c56b8640 1 --2- 172.21.15.63:0/2148967960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64c0101ad0 0x7f64c010e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.223+0000 7f64c5eb9640 1 -- 172.21.15.63:0/2148967960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64c0101ad0 msgr2=0x7f64c010e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.223+0000 7f64c5eb9640 1 --2- 172.21.15.63:0/2148967960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64c0101ad0 0x7f64c010e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.223+0000 7f64c5eb9640 1 -- 172.21.15.63:0/2148967960 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b4005040 con 0x7f64c0101130 2024-03-22T07:27:54.225 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.223+0000 7f64c56b8640 1 --2- 172.21.15.63:0/2148967960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64c0101ad0 0x7f64c010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:27:54.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.223+0000 7f64c5eb9640 1 --2- 172.21.15.63:0/2148967960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64c0101130 0x7f64c010dea0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f64b800c790 tx=0x7f64b80043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:54.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.224+0000 7f64be7fc640 1 -- 172.21.15.63:0/2148967960 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64b800a820 con 0x7f64c0101130 2024-03-22T07:27:54.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.224+0000 7f64be7fc640 1 -- 172.21.15.63:0/2148967960 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64b800ae50 con 0x7f64c0101130 2024-03-22T07:27:54.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.224+0000 7f64c811d640 1 -- 172.21.15.63:0/2148967960 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c010ec00 con 0x7f64c0101130 2024-03-22T07:27:54.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.224+0000 7f64be7fc640 1 -- 172.21.15.63:0/2148967960 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64b80115e0 con 0x7f64c0101130 2024-03-22T07:27:54.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.224+0000 7f64c811d640 1 -- 172.21.15.63:0/2148967960 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c01171c0 con 0x7f64c0101130 2024-03-22T07:27:54.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.225+0000 7f64c811d640 1 -- 172.21.15.63:0/2148967960 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c00fc8a0 con 0x7f64c0101130 2024-03-22T07:27:54.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.225+0000 7f64be7fc640 1 -- 172.21.15.63:0/2148967960 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f64b8012400 con 0x7f64c0101130 2024-03-22T07:27:54.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.225+0000 7f64be7fc640 1 --2- 172.21.15.63:0/2148967960 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f64a806d2f0 0x7f64a806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:54.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.225+0000 7f64be7fc640 1 -- 172.21.15.63:0/2148967960 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f64b808cae0 con 0x7f64c0101130 2024-03-22T07:27:54.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.225+0000 7f64c56b8640 1 --2- 172.21.15.63:0/2148967960 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f64a806d2f0 0x7f64a806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:54.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.226+0000 7f64c56b8640 1 --2- 172.21.15.63:0/2148967960 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f64a806d2f0 0x7f64a806f7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f64b4000c00 tx=0x7f64b4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:54.230 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.230+0000 7f64be7fc640 1 -- 172.21.15.63:0/2148967960 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f64b8056590 con 0x7f64c0101130 2024-03-22T07:27:54.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:54 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2568959839' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-22T07:27:54.414 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.414+0000 7f64c811d640 1 -- 172.21.15.63:0/2148967960 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64c0103570 con 0x7f64c0101130 2024-03-22T07:27:54.415 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.415+0000 7f64be7fc640 1 -- 172.21.15.63:0/2148967960 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16610 (secure 0 0 0) 0x7f64b8054e60 con 0x7f64c0101130 2024-03-22T07:27:54.415 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:27:54.415 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":87,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","created":"2024-03-22T07:06:38.004887+0000","modified":"2024-03-22T07:25:58.489902+0000","last_up_change":"2024-03-22T07:24:57.261366+0000","last_in_change":"2024-03-22T07:11:51.027495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T07:10:56.524122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-22T07:13:27.835268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-22T07:13:28.836648+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33ade431-7f97-43c7-bf9d-962c12d3ba7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":108390664},{"type":"v1","addr":"172.21.15.63:6803","nonce":108390664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":108390664},{"type":"v1","addr":"172.21.15.63:6805","nonce":108390664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":108390664},{"type":"v1","addr":"172.21.15.63:6809","nonce":108390664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":108390664},{"type":"v1","addr":"172.21.15.63:6807","nonce":108390664}]},"public_addr":"172.21.15.63:6803/108390664","cluster_addr":"172.21.15.63:6805/108390664","heartbeat_back_addr":"172.21.15.63:6809/108390664","heartbeat_front_addr":"172.21.15.63:6807/108390664","state":["exists","up"]},{"osd":1,"uuid":"0643ddb6-db9d-464c-8236-3aea2893de83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":3855758242},{"type":"v1","addr":"172.21.15.63:6811","nonce":3855758242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3855758242},{"type":"v1","addr":"172.21.15.63:6813","nonce":3855758242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3855758242},{"type":"v1","addr":"172.21.15.63:6817","nonce":3855758242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":3855758242},{"type":"v1","addr":"172.21.15.63:6815","nonce":3855758242}]},"public_addr":"172.21.15.63:6811/3855758242","cluster_addr":"172.21.15.63:6813/3855758242","heartbeat_back_addr":"172.21.15.63:6817/3855758242","heartbeat_front_addr":"172.21.15.63:6815/3855758242","state":["exists","up"]},{"osd":2,"uuid":"140b09d0-1fb6-4a3b-873a-048b797c0180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":2063458996},{"type":"v1","addr":"172.21.15.63:6819","nonce":2063458996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":2063458996},{"type":"v1","addr":"172.21.15.63:6821","nonce":2063458996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2063458996},{"type":"v1","addr":"172.21.15.63:6825","nonce":2063458996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":2063458996},{"type":"v1","addr":"172.21.15.63:6823","nonce":2063458996}]},"public_addr":"172.21.15.63:6819/2063458996","cluster_addr":"172.21.15.63:6821/2063458996","heartbeat_back_addr":"172.21.15.63:6825/2063458996","heartbeat_front_addr":"172.21.15.63:6823/2063458996","state":["exists","up"]},{"osd":3,"uuid":"1d92b77e-6bcc-4585-96f1-72251303cc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":2989645370},{"type":"v1","addr":"172.21.15.69:6801","nonce":2989645370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2989645370},{"type":"v1","addr":"172.21.15.69:6803","nonce":2989645370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2989645370},{"type":"v1","addr":"172.21.15.69:6807","nonce":2989645370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2989645370},{"type":"v1","addr":"172.21.15.69:6805","nonce":2989645370}]},"public_addr":"172.21.15.69:6801/2989645370","cluster_addr":"172.21.15.69:6803/2989645370","heartbeat_back_addr":"172.21.15.69:6807/2989645370","heartbeat_front_addr":"172.21.15.69:6805/2989645370","state":["exists","up"]},{"osd":4,"uuid":"28bdc8bc-41d5-4c01-9b29-d5833223d1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2629704021},{"type":"v1","addr":"172.21.15.69:6809","nonce":2629704021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2629704021},{"type":"v1","addr":"172.21.15.69:6811","nonce":2629704021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2629704021},{"type":"v1","addr":"172.21.15.69:6815","nonce":2629704021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2629704021},{"type":"v1","addr":"172.21.15.69:6813","nonce":2629704021}]},"public_addr":"172.21.15.69:6809/2629704021","cluster_addr":"172.21.15.69:6811/2629704021","heartbeat_back_addr":"172.21.15.69:6815/2629704021","heartbeat_front_addr":"172.21.15.69:6813/2629704021","state":["exists","up"]},{"osd":5,"uuid":"566db34e-ce0e-436f-aa90-df2c80d812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1480046180},{"type":"v1","addr":"172.21.15.69:6817","nonce":1480046180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1480046180},{"type":"v1","addr":"172.21.15.69:6819","nonce":1480046180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1480046180},{"type":"v1","addr":"172.21.15.69:6823","nonce":1480046180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1480046180},{"type":"v1","addr":"172.21.15.69:6821","nonce":1480046180}]},"public_addr":"172.21.15.69:6817/1480046180","cluster_addr":"172.21.15.69:6819/1480046180","heartbeat_back_addr":"172.21.15.69:6823/1480046180","heartbeat_front_addr":"172.21.15.69:6821/1480046180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:09:56.132204+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:10:24.051511+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:10:52.969047+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:11:17.576349+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:11:42.883846+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:12:08.488698+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6829/1373266298":"2024-03-23T07:25:00.991980+0000","172.21.15.63:6828/1373266298":"2024-03-23T07:25:00.991980+0000","172.21.15.63:0/1649622442":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/47408280":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/3629523167":"2024-03-23T07:20:31.385587+0000","172.21.15.69:0/288512401":"2024-03-23T07:17:44.994340+0000","172.21.15.63:0/424866385":"2024-03-23T07:07:10.417547+0000","172.21.15.63:6801/480818549":"2024-03-23T07:07:41.261927+0000","172.21.15.69:0/1072916585":"2024-03-23T07:17:44.994340+0000","172.21.15.63:6800/480818549":"2024-03-23T07:07:41.261927+0000","172.21.15.69:6828/1862357042":"2024-03-23T07:17:44.994340+0000","172.21.15.63:0/2503315479":"2024-03-23T07:07:41.261927+0000","172.21.15.69:6825/3798247699":"2024-03-23T07:25:00.991995+0000","172.21.15.63:0/1701838":"2024-03-23T07:16:00.292439+0000","172.21.15.63:6801/2616455474":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/1818534437":"2024-03-23T07:07:41.261927+0000","172.21.15.63:6800/2296442540":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/1323074353":"2024-03-23T07:07:41.261927+0000","172.21.15.63:6800/2616455474":"2024-03-23T07:07:10.417547+0000","172.21.15.63:6801/2390873586":"2024-03-23T07:20:31.385587+0000","172.21.15.63:6826/1314414011":"2024-03-23T07:13:45.306403+0000","172.21.15.63:6827/1314414011":"2024-03-23T07:13:45.306403+0000","172.21.15.69:0/547111417":"2024-03-23T07:17:44.994340+0000","172.21.15.69:0/2694993342":"2024-03-23T07:17:44.994340+0000","172.21.15.63:0/1602837747":"2024-03-23T07:20:31.385587+0000","172.21.15.63:0/108295103":"2024-03-23T07:16:00.292439+0000","172.21.15.63:6800/2001368263":"2024-03-23T07:16:00.292439+0000","172.21.15.63:6801/2001368263":"2024-03-23T07:16:00.292439+0000","172.21.15.63:0/466771007":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/2739978443":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/4021098270":"2024-03-23T07:20:31.385587+0000","172.21.15.63:6800/2390873586":"2024-03-23T07:20:31.385587+0000","172.21.15.63:6801/2296442540":"2024-03-23T07:08:44.186801+0000","172.21.15.69:6824/3798247699":"2024-03-23T07:25:00.991995+0000","172.21.15.63:0/3927392742":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/2835250916":"2024-03-23T07:16:00.292439+0000","172.21.15.69:6829/1862357042":"2024-03-23T07:17:44.994340+0000","172.21.15.63:0/41352707":"2024-03-23T07:16:00.292439+0000","172.21.15.63:0/1215330295":"2024-03-23T07:20:31.385587+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T07:27:54.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.417+0000 7f64c811d640 1 -- 172.21.15.63:0/2148967960 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f64a806d2f0 msgr2=0x7f64a806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:54.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.417+0000 7f64c811d640 1 --2- 172.21.15.63:0/2148967960 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f64a806d2f0 0x7f64a806f7a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f64b4000c00 tx=0x7f64b4009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.417+0000 7f64c811d640 1 -- 172.21.15.63:0/2148967960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64c0101130 msgr2=0x7f64c010dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:54.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.417+0000 7f64c811d640 1 --2- 172.21.15.63:0/2148967960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64c0101130 0x7f64c010dea0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f64b800c790 tx=0x7f64b80043c0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.417+0000 7f64c811d640 1 -- 172.21.15.63:0/2148967960 shutdown_connections 2024-03-22T07:27:54.417 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.417+0000 7f64c811d640 1 --2- 172.21.15.63:0/2148967960 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f64a806d2f0 0x7f64a806f7a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.417+0000 7f64c811d640 1 --2- 172.21.15.63:0/2148967960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64c0101ad0 0x7f64c010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.417+0000 7f64c811d640 1 --2- 172.21.15.63:0/2148967960 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f64c0101130 0x7f64c010dea0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.417+0000 7f64c811d640 1 -- 172.21.15.63:0/2148967960 >> 172.21.15.63:0/2148967960 conn(0x7f64c00f6ce0 msgr2=0x7f64c0100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:54.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.417+0000 7f64c811d640 1 -- 172.21.15.63:0/2148967960 shutdown_connections 2024-03-22T07:27:54.418 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.417+0000 7f64c811d640 1 -- 172.21.15.63:0/2148967960 wait complete. 2024-03-22T07:27:54.428 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T07:10:56.524122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-22T07:13:27.835268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-22T07:13:28.836648+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T07:27:54.428 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T07:27:54.531 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.530+0000 7fc525c6e640 1 -- 172.21.15.63:0/2616885802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5200fa880 msgr2=0x7fc5200facd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:54.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.530+0000 7fc525c6e640 1 --2- 172.21.15.63:0/2616885802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5200fa880 0x7fc5200facd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc5140048d0 tx=0x7fc514009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.530+0000 7fc525c6e640 1 -- 172.21.15.63:0/2616885802 shutdown_connections 2024-03-22T07:27:54.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.530+0000 7fc525c6e640 1 --2- 172.21.15.63:0/2616885802 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5200fa880 0x7fc5200facd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.530+0000 7fc525c6e640 1 --2- 172.21.15.63:0/2616885802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc520102250 0x7fc520102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.530+0000 7fc525c6e640 1 -- 172.21.15.63:0/2616885802 >> 172.21.15.63:0/2616885802 conn(0x7fc520070880 msgr2=0x7fc520070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:54.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.530+0000 7fc525c6e640 1 -- 172.21.15.63:0/2616885802 shutdown_connections 2024-03-22T07:27:54.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.530+0000 7fc525c6e640 1 -- 172.21.15.63:0/2616885802 wait complete. 2024-03-22T07:27:54.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.530+0000 7fc525c6e640 1 Processor -- start 2024-03-22T07:27:54.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.531+0000 7fc525c6e640 1 -- start start 2024-03-22T07:27:54.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.531+0000 7fc525c6e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5200fa880 0x7fc520112d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:54.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.531+0000 7fc525c6e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc520102250 0x7fc520113290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:54.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.531+0000 7fc525c6e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc520113890 con 0x7fc5200fa880 2024-03-22T07:27:54.532 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.531+0000 7fc525c6e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52010cee0 con 0x7fc520102250 2024-03-22T07:27:54.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.531+0000 7fc51f7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5200fa880 0x7fc520112d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:54.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.531+0000 7fc51f7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5200fa880 0x7fc520112d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46800/0 (socket says 172.21.15.63:46800) 2024-03-22T07:27:54.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.531+0000 7fc51f7fe640 1 -- 172.21.15.63:0/2649432385 learned_addr learned my addr 172.21.15.63:0/2649432385 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:54.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.531+0000 7fc51effd640 1 --2- 172.21.15.63:0/2649432385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc520102250 0x7fc520113290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:54.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.531+0000 7fc51f7fe640 1 -- 172.21.15.63:0/2649432385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc520102250 msgr2=0x7fc520113290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:54.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.531+0000 7fc51f7fe640 1 --2- 172.21.15.63:0/2649432385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc520102250 0x7fc520113290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.531+0000 7fc51f7fe640 1 -- 172.21.15.63:0/2649432385 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc514005040 con 0x7fc5200fa880 2024-03-22T07:27:54.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.531+0000 7fc51f7fe640 1 --2- 172.21.15.63:0/2649432385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5200fa880 0x7fc520112d50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc51000c790 tx=0x7fc5100043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:54.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.531+0000 7fc507fff640 1 -- 172.21.15.63:0/2649432385 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc51000a820 con 0x7fc5200fa880 2024-03-22T07:27:54.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.531+0000 7fc507fff640 1 -- 172.21.15.63:0/2649432385 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc51000ae50 con 0x7fc5200fa880 2024-03-22T07:27:54.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.531+0000 7fc507fff640 1 -- 172.21.15.63:0/2649432385 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5100115e0 con 0x7fc5200fa880 2024-03-22T07:27:54.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.531+0000 7fc525c6e640 1 -- 172.21.15.63:0/2649432385 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52010d1c0 con 0x7fc5200fa880 2024-03-22T07:27:54.533 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.531+0000 7fc525c6e640 1 -- 172.21.15.63:0/2649432385 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52010d6b0 con 0x7fc5200fa880 2024-03-22T07:27:54.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.532+0000 7fc525c6e640 1 -- 172.21.15.63:0/2649432385 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52004bcc0 con 0x7fc5200fa880 2024-03-22T07:27:54.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.533+0000 7fc507fff640 1 -- 172.21.15.63:0/2649432385 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fc510012400 con 0x7fc5200fa880 2024-03-22T07:27:54.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.533+0000 7fc507fff640 1 --2- 172.21.15.63:0/2649432385 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc50806d2f0 0x7fc50806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:54.534 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.533+0000 7fc507fff640 1 -- 172.21.15.63:0/2649432385 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc51008ca10 con 0x7fc5200fa880 2024-03-22T07:27:54.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.533+0000 7fc51effd640 1 --2- 172.21.15.63:0/2649432385 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc50806d2f0 0x7fc50806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:54.535 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.534+0000 7fc51effd640 1 --2- 172.21.15.63:0/2649432385 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc50806d2f0 0x7fc50806f7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc5201068c0 tx=0x7fc514009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:54.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.538+0000 7fc507fff640 1 -- 172.21.15.63:0/2649432385 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc510056560 con 0x7fc5200fa880 2024-03-22T07:27:54.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.721+0000 7fc525c6e640 1 -- 172.21.15.63:0/2649432385 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc520065350 con 0x7fc5200fa880 2024-03-22T07:27:54.721 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.721+0000 7fc507fff640 1 -- 172.21.15.63:0/2649432385 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc510059c20 con 0x7fc5200fa880 2024-03-22T07:27:54.721 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-03-22T07:27:54.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.723+0000 7fc525c6e640 1 -- 172.21.15.63:0/2649432385 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc50806d2f0 msgr2=0x7fc50806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:54.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.723+0000 7fc525c6e640 1 --2- 172.21.15.63:0/2649432385 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc50806d2f0 0x7fc50806f7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc5201068c0 tx=0x7fc514009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.723 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.723+0000 7fc525c6e640 1 -- 172.21.15.63:0/2649432385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5200fa880 msgr2=0x7fc520112d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:54.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.723+0000 7fc525c6e640 1 --2- 172.21.15.63:0/2649432385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5200fa880 0x7fc520112d50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc51000c790 tx=0x7fc5100043c0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.723+0000 7fc525c6e640 1 -- 172.21.15.63:0/2649432385 shutdown_connections 2024-03-22T07:27:54.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.723+0000 7fc525c6e640 1 --2- 172.21.15.63:0/2649432385 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc50806d2f0 0x7fc50806f7a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.723+0000 7fc525c6e640 1 --2- 172.21.15.63:0/2649432385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc520102250 0x7fc520113290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.723+0000 7fc525c6e640 1 --2- 172.21.15.63:0/2649432385 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc5200fa880 0x7fc520112d50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.723+0000 7fc525c6e640 1 -- 172.21.15.63:0/2649432385 >> 172.21.15.63:0/2649432385 conn(0x7fc520070880 msgr2=0x7fc5200f9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:54.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.723+0000 7fc525c6e640 1 -- 172.21.15.63:0/2649432385 shutdown_connections 2024-03-22T07:27:54.724 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.723+0000 7fc525c6e640 1 -- 172.21.15.63:0/2649432385 wait complete. 2024-03-22T07:27:54.734 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-22T07:27:54.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.835+0000 7fd21bd9d640 1 -- 172.21.15.63:0/1043583636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd214064cb0 msgr2=0x7fd214108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:54.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.835+0000 7fd21bd9d640 1 --2- 172.21.15.63:0/1043583636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd214064cb0 0x7fd214108780 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd2080048d0 tx=0x7fd208009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.838 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.835+0000 7fd21bd9d640 1 -- 172.21.15.63:0/1043583636 shutdown_connections 2024-03-22T07:27:54.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.835+0000 7fd21bd9d640 1 --2- 172.21.15.63:0/1043583636 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd214064cb0 0x7fd214108780 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.835+0000 7fd21bd9d640 1 --2- 172.21.15.63:0/1043583636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2140643a0 0x7fd214064770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.835+0000 7fd21bd9d640 1 -- 172.21.15.63:0/1043583636 >> 172.21.15.63:0/1043583636 conn(0x7fd2140710b0 msgr2=0x7fd2140714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:54.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.835+0000 7fd21bd9d640 1 -- 172.21.15.63:0/1043583636 shutdown_connections 2024-03-22T07:27:54.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.835+0000 7fd21bd9d640 1 -- 172.21.15.63:0/1043583636 wait complete. 2024-03-22T07:27:54.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.836+0000 7fd21bd9d640 1 Processor -- start 2024-03-22T07:27:54.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.836+0000 7fd21bd9d640 1 -- start start 2024-03-22T07:27:54.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.836+0000 7fd21bd9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2140643a0 0x7fd214114f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:54.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.836+0000 7fd21bd9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd214064cb0 0x7fd214115440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:54.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.836+0000 7fd21bd9d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd214115a40 con 0x7fd214064cb0 2024-03-22T07:27:54.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.836+0000 7fd21bd9d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21410f090 con 0x7fd2140643a0 2024-03-22T07:27:54.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.836+0000 7fd219338640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd214064cb0 0x7fd214115440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:54.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.836+0000 7fd219338640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd214064cb0 0x7fd214115440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42290/0 (socket says 172.21.15.63:42290) 2024-03-22T07:27:54.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.836+0000 7fd219338640 1 -- 172.21.15.63:0/2495301005 learned_addr learned my addr 172.21.15.63:0/2495301005 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:54.839 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.836+0000 7fd219b39640 1 --2- 172.21.15.63:0/2495301005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2140643a0 0x7fd214114f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:54.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.837+0000 7fd219338640 1 -- 172.21.15.63:0/2495301005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2140643a0 msgr2=0x7fd214114f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:54.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.837+0000 7fd219338640 1 --2- 172.21.15.63:0/2495301005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2140643a0 0x7fd214114f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:54.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.837+0000 7fd219338640 1 -- 172.21.15.63:0/2495301005 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd208005040 con 0x7fd214064cb0 2024-03-22T07:27:54.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.837+0000 7fd219b39640 1 --2- 172.21.15.63:0/2495301005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2140643a0 0x7fd214114f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:27:54.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.837+0000 7fd219338640 1 --2- 172.21.15.63:0/2495301005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd214064cb0 0x7fd214115440 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd208034040 tx=0x7fd208009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:54.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.837+0000 7fd2127fc640 1 -- 172.21.15.63:0/2495301005 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2080092b0 con 0x7fd214064cb0 2024-03-22T07:27:54.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.837+0000 7fd2127fc640 1 -- 172.21.15.63:0/2495301005 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd20803b400 con 0x7fd214064cb0 2024-03-22T07:27:54.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.837+0000 7fd21bd9d640 1 -- 172.21.15.63:0/2495301005 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21410f310 con 0x7fd214064cb0 2024-03-22T07:27:54.840 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.837+0000 7fd2127fc640 1 -- 172.21.15.63:0/2495301005 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2080435d0 con 0x7fd214064cb0 2024-03-22T07:27:54.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.837+0000 7fd21bd9d640 1 -- 172.21.15.63:0/2495301005 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21410f7d0 con 0x7fd214064cb0 2024-03-22T07:27:54.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.838+0000 7fd21bd9d640 1 -- 172.21.15.63:0/2495301005 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2140fb730 con 0x7fd214064cb0 2024-03-22T07:27:54.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.838+0000 7fd2127fc640 1 -- 172.21.15.63:0/2495301005 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fd20802f070 con 0x7fd214064cb0 2024-03-22T07:27:54.841 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.839+0000 7fd2127fc640 1 --2- 172.21.15.63:0/2495301005 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd1fc06d2f0 0x7fd1fc06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:54.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.839+0000 7fd2127fc640 1 -- 172.21.15.63:0/2495301005 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd2080ae6f0 con 0x7fd214064cb0 2024-03-22T07:27:54.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.839+0000 7fd219b39640 1 --2- 172.21.15.63:0/2495301005 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd1fc06d2f0 0x7fd1fc06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:54.842 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.839+0000 7fd219b39640 1 --2- 172.21.15.63:0/2495301005 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd1fc06d2f0 0x7fd1fc06f7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd20c004ab0 tx=0x7fd20c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:54.843 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:54.843+0000 7fd2127fc640 1 -- 172.21.15.63:0/2495301005 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd208078120 con 0x7fd214064cb0 2024-03-22T07:27:55.022 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.022+0000 7fd21bd9d640 1 -- 172.21.15.63:0/2495301005 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fd21404bcc0 con 0x7fd214064cb0 2024-03-22T07:27:55.023 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.023+0000 7fd2127fc640 1 -- 172.21.15.63:0/2495301005 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd20807b7e0 con 0x7fd214064cb0 2024-03-22T07:27:55.023 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-03-22T07:27:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.025+0000 7fd21bd9d640 1 -- 172.21.15.63:0/2495301005 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd1fc06d2f0 msgr2=0x7fd1fc06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.025+0000 7fd21bd9d640 1 --2- 172.21.15.63:0/2495301005 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd1fc06d2f0 0x7fd1fc06f7a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd20c004ab0 tx=0x7fd20c00b040 comp rx=0 tx=0).stop 2024-03-22T07:27:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.025+0000 7fd21bd9d640 1 -- 172.21.15.63:0/2495301005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd214064cb0 msgr2=0x7fd214115440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.025+0000 7fd21bd9d640 1 --2- 172.21.15.63:0/2495301005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd214064cb0 0x7fd214115440 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd208034040 tx=0x7fd208009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.025+0000 7fd21bd9d640 1 -- 172.21.15.63:0/2495301005 shutdown_connections 2024-03-22T07:27:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.025+0000 7fd21bd9d640 1 --2- 172.21.15.63:0/2495301005 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd214064cb0 0x7fd214115440 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.025+0000 7fd21bd9d640 1 --2- 172.21.15.63:0/2495301005 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd1fc06d2f0 0x7fd1fc06f7a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.025+0000 7fd21bd9d640 1 --2- 172.21.15.63:0/2495301005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2140643a0 0x7fd214114f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.025+0000 7fd21bd9d640 1 -- 172.21.15.63:0/2495301005 >> 172.21.15.63:0/2495301005 conn(0x7fd2140710b0 msgr2=0x7fd2140f9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:55.025 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.025+0000 7fd21bd9d640 1 -- 172.21.15.63:0/2495301005 shutdown_connections 2024-03-22T07:27:55.026 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.025+0000 7fd21bd9d640 1 -- 172.21.15.63:0/2495301005 wait complete. 2024-03-22T07:27:55.036 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-22T07:27:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.138+0000 7f97c5b14640 1 -- 172.21.15.63:0/4129802883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c01023e0 msgr2=0x7f97c01027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.138+0000 7f97c5b14640 1 --2- 172.21.15.63:0/4129802883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c01023e0 0x7f97c01027b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f97a80048d0 tx=0x7f97a8009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.138+0000 7f97c5b14640 1 -- 172.21.15.63:0/4129802883 shutdown_connections 2024-03-22T07:27:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.138+0000 7f97c5b14640 1 --2- 172.21.15.63:0/4129802883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c0102d80 0x7f97c00fc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.138+0000 7f97c5b14640 1 --2- 172.21.15.63:0/4129802883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c01023e0 0x7f97c01027b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.138+0000 7f97c5b14640 1 -- 172.21.15.63:0/4129802883 >> 172.21.15.63:0/4129802883 conn(0x7f97c00f7fb0 msgr2=0x7f97c00fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.138+0000 7f97c5b14640 1 -- 172.21.15.63:0/4129802883 shutdown_connections 2024-03-22T07:27:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.138+0000 7f97c5b14640 1 -- 172.21.15.63:0/4129802883 wait complete. 2024-03-22T07:27:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.138+0000 7f97c5b14640 1 Processor -- start 2024-03-22T07:27:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.138+0000 7f97c5b14640 1 -- start start 2024-03-22T07:27:55.140 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.139+0000 7f97c5b14640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c01023e0 0x7f97c0112ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:55.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.139+0000 7f97c5b14640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c0102d80 0x7f97c0113430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:55.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.139+0000 7f97c5b14640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c0113a30 con 0x7f97c01023e0 2024-03-22T07:27:55.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.139+0000 7f97c5b14640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c010d080 con 0x7f97c0102d80 2024-03-22T07:27:55.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.139+0000 7f97bf7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c01023e0 0x7f97c0112ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:55.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.139+0000 7f97bf7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c01023e0 0x7f97c0112ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42310/0 (socket says 172.21.15.63:42310) 2024-03-22T07:27:55.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.139+0000 7f97bf7fe640 1 -- 172.21.15.63:0/4174554125 learned_addr learned my addr 172.21.15.63:0/4174554125 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:55.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.139+0000 7f97bf7fe640 1 -- 172.21.15.63:0/4174554125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c0102d80 msgr2=0x7f97c0113430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:55.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.139+0000 7f97beffd640 1 --2- 172.21.15.63:0/4174554125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c0102d80 0x7f97c0113430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:55.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.139+0000 7f97bf7fe640 1 --2- 172.21.15.63:0/4174554125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c0102d80 0x7f97c0113430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.139+0000 7f97bf7fe640 1 -- 172.21.15.63:0/4174554125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a8005040 con 0x7f97c01023e0 2024-03-22T07:27:55.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.139+0000 7f97bf7fe640 1 --2- 172.21.15.63:0/4174554125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c01023e0 0x7f97c0112ef0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f97a8004930 tx=0x7f97a8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:55.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.139+0000 7f97b7fff640 1 -- 172.21.15.63:0/4174554125 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97a80094c0 con 0x7f97c01023e0 2024-03-22T07:27:55.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.139+0000 7f97b7fff640 1 -- 172.21.15.63:0/4174554125 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f97a8032e50 con 0x7f97c01023e0 2024-03-22T07:27:55.141 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.139+0000 7f97c5b14640 1 -- 172.21.15.63:0/4174554125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c010d300 con 0x7f97c01023e0 2024-03-22T07:27:55.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.139+0000 7f97b7fff640 1 -- 172.21.15.63:0/4174554125 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97a80433b0 con 0x7f97c01023e0 2024-03-22T07:27:55.142 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.139+0000 7f97c5b14640 1 -- 172.21.15.63:0/4174554125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c010d7c0 con 0x7f97c01023e0 2024-03-22T07:27:55.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.140+0000 7f97c5b14640 1 -- 172.21.15.63:0/4174554125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c00fdb50 con 0x7f97c01023e0 2024-03-22T07:27:55.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.141+0000 7f97b7fff640 1 -- 172.21.15.63:0/4174554125 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f97a802f070 con 0x7f97c01023e0 2024-03-22T07:27:55.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.141+0000 7f97b7fff640 1 --2- 172.21.15.63:0/4174554125 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f97ac06d2f0 0x7f97ac06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:55.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.141+0000 7f97b7fff640 1 -- 172.21.15.63:0/4174554125 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f97a80ae840 con 0x7f97c01023e0 2024-03-22T07:27:55.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.141+0000 7f97beffd640 1 --2- 172.21.15.63:0/4174554125 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f97ac06d2f0 0x7f97ac06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:55.143 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.142+0000 7f97beffd640 1 --2- 172.21.15.63:0/4174554125 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f97ac06d2f0 0x7f97ac06f7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f97c010e7b0 tx=0x7f97b000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:55.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.146+0000 7f97b7fff640 1 -- 172.21.15.63:0/4174554125 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f97a8078270 con 0x7f97c01023e0 2024-03-22T07:27:55.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:55 smithi069 ceph-mon[83631]: pgmap v261: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:55.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:55 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2148967960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:27:55.279 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:55 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2649432385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:27:55.280 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:55 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2495301005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T07:27:55.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.326+0000 7f97c5b14640 1 -- 172.21.15.63:0/4174554125 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f97c0063950 con 0x7f97c01023e0 2024-03-22T07:27:55.327 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.327+0000 7f97b7fff640 1 -- 172.21.15.63:0/4174554125 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7f97a807b930 con 0x7f97c01023e0 2024-03-22T07:27:55.327 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-03-22T07:27:55.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.329+0000 7f97c5b14640 1 -- 172.21.15.63:0/4174554125 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f97ac06d2f0 msgr2=0x7f97ac06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:55.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.329+0000 7f97c5b14640 1 --2- 172.21.15.63:0/4174554125 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f97ac06d2f0 0x7f97ac06f7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f97c010e7b0 tx=0x7f97b000b040 comp rx=0 tx=0).stop 2024-03-22T07:27:55.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.329+0000 7f97c5b14640 1 -- 172.21.15.63:0/4174554125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c01023e0 msgr2=0x7f97c0112ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:55.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.329+0000 7f97c5b14640 1 --2- 172.21.15.63:0/4174554125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c01023e0 0x7f97c0112ef0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f97a8004930 tx=0x7f97a8009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.329+0000 7f97c5b14640 1 -- 172.21.15.63:0/4174554125 shutdown_connections 2024-03-22T07:27:55.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.329+0000 7f97c5b14640 1 --2- 172.21.15.63:0/4174554125 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f97ac06d2f0 0x7f97ac06f7a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.329+0000 7f97c5b14640 1 --2- 172.21.15.63:0/4174554125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c0102d80 0x7f97c0113430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.329+0000 7f97c5b14640 1 --2- 172.21.15.63:0/4174554125 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f97c01023e0 0x7f97c0112ef0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.329 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.329+0000 7f97c5b14640 1 -- 172.21.15.63:0/4174554125 >> 172.21.15.63:0/4174554125 conn(0x7f97c00f7fb0 msgr2=0x7f97c00f8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:55.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.329+0000 7f97c5b14640 1 -- 172.21.15.63:0/4174554125 shutdown_connections 2024-03-22T07:27:55.330 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.329+0000 7f97c5b14640 1 -- 172.21.15.63:0/4174554125 wait complete. 2024-03-22T07:27:55.338 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:55 smithi063 ceph-mon[112784]: pgmap v261: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:27:55.338 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:55 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2148967960' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:27:55.339 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:55 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2649432385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:27:55.339 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:55 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2495301005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T07:27:55.340 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-22T07:27:55.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.438+0000 7ff34de2e640 1 -- 172.21.15.63:0/2721771011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3480fa940 msgr2=0x7ff3480fad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:55.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.438+0000 7ff34de2e640 1 --2- 172.21.15.63:0/2721771011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3480fa940 0x7ff3480fad90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0048d0 tx=0x7ff33c009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.438+0000 7ff34de2e640 1 -- 172.21.15.63:0/2721771011 shutdown_connections 2024-03-22T07:27:55.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.438+0000 7ff34de2e640 1 --2- 172.21.15.63:0/2721771011 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3480fa940 0x7ff3480fad90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.438+0000 7ff34de2e640 1 --2- 172.21.15.63:0/2721771011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3481022f0 0x7ff3481026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.438+0000 7ff34de2e640 1 -- 172.21.15.63:0/2721771011 >> 172.21.15.63:0/2721771011 conn(0x7ff3480708f0 msgr2=0x7ff348070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:55.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.438+0000 7ff34de2e640 1 -- 172.21.15.63:0/2721771011 shutdown_connections 2024-03-22T07:27:55.439 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.438+0000 7ff34de2e640 1 -- 172.21.15.63:0/2721771011 wait complete. 2024-03-22T07:27:55.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.439+0000 7ff34de2e640 1 Processor -- start 2024-03-22T07:27:55.440 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.439+0000 7ff34de2e640 1 -- start start 2024-03-22T07:27:55.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.439+0000 7ff34de2e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3480fa940 0x7ff348112e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:55.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.439+0000 7ff34de2e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3481022f0 0x7ff348113360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:55.441 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.439+0000 7ff34de2e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff348113960 con 0x7ff3481022f0 2024-03-22T07:27:55.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.439+0000 7ff34de2e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34810cfb0 con 0x7ff3480fa940 2024-03-22T07:27:55.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.439+0000 7ff346ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3481022f0 0x7ff348113360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:55.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.439+0000 7ff346ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3481022f0 0x7ff348113360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42338/0 (socket says 172.21.15.63:42338) 2024-03-22T07:27:55.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.439+0000 7ff346ffd640 1 -- 172.21.15.63:0/2949947809 learned_addr learned my addr 172.21.15.63:0/2949947809 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:55.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.439+0000 7ff3477fe640 1 --2- 172.21.15.63:0/2949947809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3480fa940 0x7ff348112e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:55.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.440+0000 7ff346ffd640 1 -- 172.21.15.63:0/2949947809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3480fa940 msgr2=0x7ff348112e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:55.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.440+0000 7ff346ffd640 1 --2- 172.21.15.63:0/2949947809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3480fa940 0x7ff348112e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.440+0000 7ff346ffd640 1 -- 172.21.15.63:0/2949947809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c005040 con 0x7ff3481022f0 2024-03-22T07:27:55.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.440+0000 7ff346ffd640 1 --2- 172.21.15.63:0/2949947809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3481022f0 0x7ff348113360 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff33c004930 tx=0x7ff33c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:55.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.440+0000 7ff32ffff640 1 -- 172.21.15.63:0/2949947809 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff33c0094c0 con 0x7ff3481022f0 2024-03-22T07:27:55.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.440+0000 7ff34de2e640 1 -- 172.21.15.63:0/2949947809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34810d230 con 0x7ff3481022f0 2024-03-22T07:27:55.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.440+0000 7ff32ffff640 1 -- 172.21.15.63:0/2949947809 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff33c032e50 con 0x7ff3481022f0 2024-03-22T07:27:55.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.440+0000 7ff32ffff640 1 -- 172.21.15.63:0/2949947809 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff33c0433b0 con 0x7ff3481022f0 2024-03-22T07:27:55.442 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.440+0000 7ff34de2e640 1 -- 172.21.15.63:0/2949947809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34810d6f0 con 0x7ff3481022f0 2024-03-22T07:27:55.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.441+0000 7ff34de2e640 1 -- 172.21.15.63:0/2949947809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff34804bcc0 con 0x7ff3481022f0 2024-03-22T07:27:55.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.441+0000 7ff32ffff640 1 -- 172.21.15.63:0/2949947809 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7ff33c02f070 con 0x7ff3481022f0 2024-03-22T07:27:55.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.442+0000 7ff32ffff640 1 --2- 172.21.15.63:0/2949947809 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff33006d2f0 0x7ff33006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:55.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.442+0000 7ff3477fe640 1 --2- 172.21.15.63:0/2949947809 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff33006d2f0 0x7ff33006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:55.443 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.442+0000 7ff32ffff640 1 -- 172.21.15.63:0/2949947809 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff33c0ae930 con 0x7ff3481022f0 2024-03-22T07:27:55.444 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.442+0000 7ff3477fe640 1 --2- 172.21.15.63:0/2949947809 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff33006d2f0 0x7ff33006f7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff3380041d0 tx=0x7ff338009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:55.446 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.447+0000 7ff32ffff640 1 -- 172.21.15.63:0/2949947809 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff33c078360 con 0x7ff3481022f0 2024-03-22T07:27:55.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.664+0000 7ff34de2e640 1 -- 172.21.15.63:0/2949947809 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff34810e490 con 0x7ff3481022f0 2024-03-22T07:27:55.664 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.664+0000 7ff32ffff640 1 -- 172.21.15.63:0/2949947809 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+5026 (secure 0 0 0) 0x7ff33c07ba20 con 0x7ff3481022f0 2024-03-22T07:27:55.665 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:27:55.665 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:26:22.473650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34400,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34400":{"gid":34400,"name":"cephfs.smithi063.oytapz","rank":0,"incarnation":37,"state":"up:active","state_seq":22,"addr":"172.21.15.63:6827/1171423403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1171423403},{"type":"v1","addr":"172.21.15.63:6827","nonce":1171423403}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi063.xeelpk","rank":1,"incarnation":37,"state":"up:active","state_seq":16,"addr":"172.21.15.63:6829/1190308435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1190308435},{"type":"v1","addr":"172.21.15.63:6829","nonce":1190308435}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_44323":{"gid":44323,"name":"cephfs.smithi069.wwtsjn","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6825/653947973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":653947973},{"type":"v1","addr":"172.21.15.69:6825","nonce":653947973}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_44327":{"gid":44327,"name":"cephfs.smithi069.cijxah","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.69:6827/3465454734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3465454734},{"type":"v1","addr":"172.21.15.69:6827","nonce":3465454734}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:27:55.665 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 44 2024-03-22T07:27:55.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.667+0000 7ff34de2e640 1 -- 172.21.15.63:0/2949947809 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff33006d2f0 msgr2=0x7ff33006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:55.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.667+0000 7ff34de2e640 1 --2- 172.21.15.63:0/2949947809 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff33006d2f0 0x7ff33006f7a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff3380041d0 tx=0x7ff338009040 comp rx=0 tx=0).stop 2024-03-22T07:27:55.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.667+0000 7ff34de2e640 1 -- 172.21.15.63:0/2949947809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3481022f0 msgr2=0x7ff348113360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:55.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.667+0000 7ff34de2e640 1 --2- 172.21.15.63:0/2949947809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3481022f0 0x7ff348113360 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff33c004930 tx=0x7ff33c009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.667+0000 7ff34de2e640 1 -- 172.21.15.63:0/2949947809 shutdown_connections 2024-03-22T07:27:55.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.667+0000 7ff34de2e640 1 --2- 172.21.15.63:0/2949947809 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7ff3481022f0 0x7ff348113360 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.667+0000 7ff34de2e640 1 --2- 172.21.15.63:0/2949947809 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7ff33006d2f0 0x7ff33006f7a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.667+0000 7ff34de2e640 1 --2- 172.21.15.63:0/2949947809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3480fa940 0x7ff348112e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.667+0000 7ff34de2e640 1 -- 172.21.15.63:0/2949947809 >> 172.21.15.63:0/2949947809 conn(0x7ff3480708f0 msgr2=0x7ff3480f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:55.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.667+0000 7ff34de2e640 1 -- 172.21.15.63:0/2949947809 shutdown_connections 2024-03-22T07:27:55.667 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.667+0000 7ff34de2e640 1 -- 172.21.15.63:0/2949947809 wait complete. 2024-03-22T07:27:55.678 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 20, 'max_mds': 2, 'flags': 50} 2024-03-22T07:27:55.678 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-03-22T07:27:55.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.777+0000 7f76a079d640 1 -- 172.21.15.63:0/1105676801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76981023e0 msgr2=0x7f76981027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:55.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.777+0000 7f76a079d640 1 --2- 172.21.15.63:0/1105676801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76981023e0 0x7f76981027b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f76900048d0 tx=0x7f7690009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.778+0000 7f76a079d640 1 -- 172.21.15.63:0/1105676801 shutdown_connections 2024-03-22T07:27:55.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.778+0000 7f76a079d640 1 --2- 172.21.15.63:0/1105676801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7698102d80 0x7f76980fc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.778+0000 7f76a079d640 1 --2- 172.21.15.63:0/1105676801 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76981023e0 0x7f76981027b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.778+0000 7f76a079d640 1 -- 172.21.15.63:0/1105676801 >> 172.21.15.63:0/1105676801 conn(0x7f76980f7fb0 msgr2=0x7f76980fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:55.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.778+0000 7f76a079d640 1 -- 172.21.15.63:0/1105676801 shutdown_connections 2024-03-22T07:27:55.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.778+0000 7f76a079d640 1 -- 172.21.15.63:0/1105676801 wait complete. 2024-03-22T07:27:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.778+0000 7f76a079d640 1 Processor -- start 2024-03-22T07:27:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.778+0000 7f76a079d640 1 -- start start 2024-03-22T07:27:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.778+0000 7f76a079d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76981023e0 0x7f7698112f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.778+0000 7f76a079d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7698102d80 0x7f7698113440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.778+0000 7f76a079d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7698113a40 con 0x7f76981023e0 2024-03-22T07:27:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.778+0000 7f76a079d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769810d090 con 0x7f7698102d80 2024-03-22T07:27:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.778+0000 7f769e539640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76981023e0 0x7f7698112f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.778+0000 7f769e539640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76981023e0 0x7f7698112f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42358/0 (socket says 172.21.15.63:42358) 2024-03-22T07:27:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.778+0000 7f769e539640 1 -- 172.21.15.63:0/1996186359 learned_addr learned my addr 172.21.15.63:0/1996186359 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.779+0000 7f769dd38640 1 --2- 172.21.15.63:0/1996186359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7698102d80 0x7f7698113440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.779+0000 7f769e539640 1 -- 172.21.15.63:0/1996186359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7698102d80 msgr2=0x7f7698113440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.779+0000 7f769e539640 1 --2- 172.21.15.63:0/1996186359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7698102d80 0x7f7698113440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.779+0000 7f769e539640 1 -- 172.21.15.63:0/1996186359 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690005040 con 0x7f76981023e0 2024-03-22T07:27:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.779+0000 7f769dd38640 1 --2- 172.21.15.63:0/1996186359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7698102d80 0x7f7698113440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:27:55.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.779+0000 7f769e539640 1 --2- 172.21.15.63:0/1996186359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76981023e0 0x7f7698112f00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7690004930 tx=0x7f7690009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:55.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.779+0000 7f7696ffd640 1 -- 172.21.15.63:0/1996186359 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76900094c0 con 0x7f76981023e0 2024-03-22T07:27:55.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.779+0000 7f7696ffd640 1 -- 172.21.15.63:0/1996186359 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7690032e50 con 0x7f76981023e0 2024-03-22T07:27:55.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.779+0000 7f76a079d640 1 -- 172.21.15.63:0/1996186359 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769810d310 con 0x7f76981023e0 2024-03-22T07:27:55.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.779+0000 7f7696ffd640 1 -- 172.21.15.63:0/1996186359 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76900433b0 con 0x7f76981023e0 2024-03-22T07:27:55.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.779+0000 7f76a079d640 1 -- 172.21.15.63:0/1996186359 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769810d7d0 con 0x7f76981023e0 2024-03-22T07:27:55.782 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.780+0000 7f76a079d640 1 -- 172.21.15.63:0/1996186359 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76980fdb50 con 0x7f76981023e0 2024-03-22T07:27:55.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.780+0000 7f7696ffd640 1 -- 172.21.15.63:0/1996186359 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f769002f070 con 0x7f76981023e0 2024-03-22T07:27:55.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.781+0000 7f7696ffd640 1 --2- 172.21.15.63:0/1996186359 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f768006d2f0 0x7f768006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:55.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.781+0000 7f7696ffd640 1 -- 172.21.15.63:0/1996186359 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f76900ae840 con 0x7f76981023e0 2024-03-22T07:27:55.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.781+0000 7f769dd38640 1 --2- 172.21.15.63:0/1996186359 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f768006d2f0 0x7f768006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:55.783 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.781+0000 7f769dd38640 1 --2- 172.21.15.63:0/1996186359 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f768006d2f0 0x7f768006f7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f768c007fd0 tx=0x7f768c009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:55.785 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:55.785+0000 7f7696ffd640 1 -- 172.21.15.63:0/1996186359 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7690078270 con 0x7f76981023e0 2024-03-22T07:27:56.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.004+0000 7f76a079d640 1 -- 172.21.15.63:0/1996186359 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f769804bcc0 con 0x7f76981023e0 2024-03-22T07:27:56.004 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.004+0000 7f7696ffd640 1 -- 172.21.15.63:0/1996186359 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v44) v1 ==== 107+0+4857 (secure 0 0 0) 0x7f769007b930 con 0x7f76981023e0 2024-03-22T07:27:56.005 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:27:56.005 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:15:31.328037+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi063.xeelpk","rank":0,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.63:6829/1373266298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1373266298},{"type":"v1","addr":"172.21.15.63:6829","nonce":1373266298}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14492":{"gid":14492,"name":"cephfs.smithi063.oytapz","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6827/3211595802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":3211595802},{"type":"v1","addr":"172.21.15.63:6827","nonce":3211595802}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24285":{"gid":24285,"name":"cephfs.smithi069.wwtsjn","rank":1,"incarnation":8,"state":"up:active","state_seq":4,"addr":"172.21.15.69:6825/3798247699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3798247699},{"type":"v1","addr":"172.21.15.69:6825","nonce":3798247699}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24301":{"gid":24301,"name":"cephfs.smithi069.cijxah","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.69:6827/3380719311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3380719311},{"type":"v1","addr":"172.21.15.69:6827","nonce":3380719311}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:27:56.005 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 21 2024-03-22T07:27:56.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.006+0000 7f76a079d640 1 -- 172.21.15.63:0/1996186359 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f768006d2f0 msgr2=0x7f768006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:56.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.006+0000 7f76a079d640 1 --2- 172.21.15.63:0/1996186359 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f768006d2f0 0x7f768006f7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f768c007fd0 tx=0x7f768c009e90 comp rx=0 tx=0).stop 2024-03-22T07:27:56.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.006+0000 7f76a079d640 1 -- 172.21.15.63:0/1996186359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76981023e0 msgr2=0x7f7698112f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:56.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.006+0000 7f76a079d640 1 --2- 172.21.15.63:0/1996186359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76981023e0 0x7f7698112f00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7690004930 tx=0x7f7690009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.006+0000 7f76a079d640 1 -- 172.21.15.63:0/1996186359 shutdown_connections 2024-03-22T07:27:56.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.006+0000 7f76a079d640 1 --2- 172.21.15.63:0/1996186359 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f768006d2f0 0x7f768006f7a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.006+0000 7f76a079d640 1 --2- 172.21.15.63:0/1996186359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7698102d80 0x7f7698113440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.006+0000 7f76a079d640 1 --2- 172.21.15.63:0/1996186359 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f76981023e0 0x7f7698112f00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.006+0000 7f76a079d640 1 -- 172.21.15.63:0/1996186359 >> 172.21.15.63:0/1996186359 conn(0x7f76980f7fb0 msgr2=0x7f76980f8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:56.007 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.006+0000 7f76a079d640 1 -- 172.21.15.63:0/1996186359 shutdown_connections 2024-03-22T07:27:56.008 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.006+0000 7f76a079d640 1 -- 172.21.15.63:0/1996186359 wait complete. 2024-03-22T07:27:56.018 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-03-22T07:27:56.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.117+0000 7f81a222e640 1 -- 172.21.15.63:0/4241921305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f819c0643a0 msgr2=0x7f819c0647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:56.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.117+0000 7f81a222e640 1 --2- 172.21.15.63:0/4241921305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f819c0643a0 0x7f819c0647d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f81900048d0 tx=0x7f8190009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.117+0000 7f81a222e640 1 -- 172.21.15.63:0/4241921305 shutdown_connections 2024-03-22T07:27:56.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.117+0000 7f81a222e640 1 --2- 172.21.15.63:0/4241921305 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f819c0643a0 0x7f819c0647d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.117+0000 7f81a222e640 1 --2- 172.21.15.63:0/4241921305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f819c102240 0x7f819c102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.118 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.117+0000 7f81a222e640 1 -- 172.21.15.63:0/4241921305 >> 172.21.15.63:0/4241921305 conn(0x7f819c0710b0 msgr2=0x7f819c0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:56.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.117+0000 7f81a222e640 1 -- 172.21.15.63:0/4241921305 shutdown_connections 2024-03-22T07:27:56.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.117+0000 7f81a222e640 1 -- 172.21.15.63:0/4241921305 wait complete. 2024-03-22T07:27:56.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.118+0000 7f81a222e640 1 Processor -- start 2024-03-22T07:27:56.119 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.118+0000 7f81a222e640 1 -- start start 2024-03-22T07:27:56.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.118+0000 7f81a222e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f819c0643a0 0x7f819c112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:56.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.118+0000 7f81a222e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f819c102240 0x7f819c113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:56.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.118+0000 7f81a222e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c113800 con 0x7f819c0643a0 2024-03-22T07:27:56.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.118+0000 7f81a222e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c10ce50 con 0x7f819c102240 2024-03-22T07:27:56.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.118+0000 7f819b7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f819c0643a0 0x7f819c112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:56.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.118+0000 7f819b7fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f819c0643a0 0x7f819c112cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42370/0 (socket says 172.21.15.63:42370) 2024-03-22T07:27:56.121 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.118+0000 7f819b7fe640 1 -- 172.21.15.63:0/2793480361 learned_addr learned my addr 172.21.15.63:0/2793480361 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:56.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.119+0000 7f819affd640 1 --2- 172.21.15.63:0/2793480361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f819c102240 0x7f819c113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:56.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.119+0000 7f819b7fe640 1 -- 172.21.15.63:0/2793480361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f819c102240 msgr2=0x7f819c113200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:56.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.119+0000 7f819b7fe640 1 --2- 172.21.15.63:0/2793480361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f819c102240 0x7f819c113200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.119+0000 7f819b7fe640 1 -- 172.21.15.63:0/2793480361 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8190005040 con 0x7f819c0643a0 2024-03-22T07:27:56.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.119+0000 7f819b7fe640 1 --2- 172.21.15.63:0/2793480361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f819c0643a0 0x7f819c112cc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f818c00c830 tx=0x7f818c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:56.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.119+0000 7f8183fff640 1 -- 172.21.15.63:0/2793480361 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f818c00a850 con 0x7f819c0643a0 2024-03-22T07:27:56.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.119+0000 7f8183fff640 1 -- 172.21.15.63:0/2793480361 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f818c00ae80 con 0x7f819c0643a0 2024-03-22T07:27:56.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.119+0000 7f81a222e640 1 -- 172.21.15.63:0/2793480361 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819c10d130 con 0x7f819c0643a0 2024-03-22T07:27:56.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.119+0000 7f8183fff640 1 -- 172.21.15.63:0/2793480361 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f818c012ea0 con 0x7f819c0643a0 2024-03-22T07:27:56.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.119+0000 7f81a222e640 1 -- 172.21.15.63:0/2793480361 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819c10d620 con 0x7f819c0643a0 2024-03-22T07:27:56.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.120+0000 7f81a222e640 1 -- 172.21.15.63:0/2793480361 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f819c0fd940 con 0x7f819c0643a0 2024-03-22T07:27:56.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.120+0000 7f8183fff640 1 -- 172.21.15.63:0/2793480361 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f818c00a9b0 con 0x7f819c0643a0 2024-03-22T07:27:56.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.121+0000 7f8183fff640 1 --2- 172.21.15.63:0/2793480361 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f818406d2f0 0x7f818406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:56.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.121+0000 7f8183fff640 1 -- 172.21.15.63:0/2793480361 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f818c08cb90 con 0x7f819c0643a0 2024-03-22T07:27:56.122 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.121+0000 7f819affd640 1 --2- 172.21.15.63:0/2793480361 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f818406d2f0 0x7f818406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:56.123 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.121+0000 7f819affd640 1 --2- 172.21.15.63:0/2793480361 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f818406d2f0 0x7f818406f7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f819c0fb030 tx=0x7f8190009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:56.125 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.125+0000 7f8183fff640 1 -- 172.21.15.63:0/2793480361 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f818c056640 con 0x7f819c0643a0 2024-03-22T07:27:56.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.343+0000 7f81a222e640 1 -- 172.21.15.63:0/2793480361 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f819c04bcc0 con 0x7f819c0643a0 2024-03-22T07:27:56.344 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.344+0000 7f8183fff640 1 -- 172.21.15.63:0/2793480361 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v44) v1 ==== 107+0+4858 (secure 0 0 0) 0x7f818c00ac60 con 0x7f819c0643a0 2024-03-22T07:27:56.345 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:27:56.345 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:16:52.763485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi063.xeelpk","rank":0,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.63:6829/1373266298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1373266298},{"type":"v1","addr":"172.21.15.63:6829","nonce":1373266298}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14492":{"gid":14492,"name":"cephfs.smithi063.oytapz","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6827/3211595802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":3211595802},{"type":"v1","addr":"172.21.15.63:6827","nonce":3211595802}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24285":{"gid":24285,"name":"cephfs.smithi069.wwtsjn","rank":1,"incarnation":8,"state":"up:active","state_seq":4,"addr":"172.21.15.69:6825/3798247699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3798247699},{"type":"v1","addr":"172.21.15.69:6825","nonce":3798247699}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24301":{"gid":24301,"name":"cephfs.smithi069.cijxah","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.69:6827/3380719311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3380719311},{"type":"v1","addr":"172.21.15.69:6827","nonce":3380719311}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:27:56.345 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 22 2024-03-22T07:27:56.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.346+0000 7f81a222e640 1 -- 172.21.15.63:0/2793480361 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f818406d2f0 msgr2=0x7f818406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:56.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.346+0000 7f81a222e640 1 --2- 172.21.15.63:0/2793480361 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f818406d2f0 0x7f818406f7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f819c0fb030 tx=0x7f8190009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.346+0000 7f81a222e640 1 -- 172.21.15.63:0/2793480361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f819c0643a0 msgr2=0x7f819c112cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:56.347 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.346+0000 7f81a222e640 1 --2- 172.21.15.63:0/2793480361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f819c0643a0 0x7f819c112cc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f818c00c830 tx=0x7f818c004430 comp rx=0 tx=0).stop 2024-03-22T07:27:56.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.346+0000 7f81a222e640 1 -- 172.21.15.63:0/2793480361 shutdown_connections 2024-03-22T07:27:56.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.346+0000 7f81a222e640 1 --2- 172.21.15.63:0/2793480361 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f818406d2f0 0x7f818406f7a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.346+0000 7f81a222e640 1 --2- 172.21.15.63:0/2793480361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f819c102240 0x7f819c113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.346+0000 7f81a222e640 1 --2- 172.21.15.63:0/2793480361 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f819c0643a0 0x7f819c112cc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.346+0000 7f81a222e640 1 -- 172.21.15.63:0/2793480361 >> 172.21.15.63:0/2793480361 conn(0x7f819c0710b0 msgr2=0x7f819c0f94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:56.348 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.346+0000 7f81a222e640 1 -- 172.21.15.63:0/2793480361 shutdown_connections 2024-03-22T07:27:56.349 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.346+0000 7f81a222e640 1 -- 172.21.15.63:0/2793480361 wait complete. 2024-03-22T07:27:56.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:56 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/4174554125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T07:27:56.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:56 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2949947809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T07:27:56.356 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:56 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/1996186359' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-22T07:27:56.358 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-03-22T07:27:56.466 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.464+0000 7f144751d640 1 -- 172.21.15.63:0/1580432459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14401012e0 msgr2=0x7f14401016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:56.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.464+0000 7f144751d640 1 --2- 172.21.15.63:0/1580432459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14401012e0 0x7f14401016b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f14340048d0 tx=0x7f1434009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.464+0000 7f144751d640 1 -- 172.21.15.63:0/1580432459 shutdown_connections 2024-03-22T07:27:56.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.464+0000 7f144751d640 1 --2- 172.21.15.63:0/1580432459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1440101c80 0x7f14400fb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.464+0000 7f144751d640 1 --2- 172.21.15.63:0/1580432459 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14401012e0 0x7f14401016b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.464+0000 7f144751d640 1 -- 172.21.15.63:0/1580432459 >> 172.21.15.63:0/1580432459 conn(0x7f14400f6e70 msgr2=0x7f14400f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:56.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.464+0000 7f144751d640 1 -- 172.21.15.63:0/1580432459 shutdown_connections 2024-03-22T07:27:56.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.464+0000 7f144751d640 1 -- 172.21.15.63:0/1580432459 wait complete. 2024-03-22T07:27:56.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.464+0000 7f144751d640 1 Processor -- start 2024-03-22T07:27:56.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.464+0000 7f144751d640 1 -- start start 2024-03-22T07:27:56.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.465+0000 7f144751d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14401012e0 0x7f1440111df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:56.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.465+0000 7f144751d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1440101c80 0x7f1440112330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:56.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.465+0000 7f144751d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1440112930 con 0x7f14401012e0 2024-03-22T07:27:56.467 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.465+0000 7f144751d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1440112aa0 con 0x7f1440101c80 2024-03-22T07:27:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.465+0000 7f14452b9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14401012e0 0x7f1440111df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.465+0000 7f14452b9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14401012e0 0x7f1440111df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42376/0 (socket says 172.21.15.63:42376) 2024-03-22T07:27:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.465+0000 7f14452b9640 1 -- 172.21.15.63:0/168631363 learned_addr learned my addr 172.21.15.63:0/168631363 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.465+0000 7f1444ab8640 1 --2- 172.21.15.63:0/168631363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1440101c80 0x7f1440112330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.465+0000 7f14452b9640 1 -- 172.21.15.63:0/168631363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1440101c80 msgr2=0x7f1440112330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.465+0000 7f14452b9640 1 --2- 172.21.15.63:0/168631363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1440101c80 0x7f1440112330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.465+0000 7f14452b9640 1 -- 172.21.15.63:0/168631363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1434005040 con 0x7f14401012e0 2024-03-22T07:27:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.465+0000 7f1444ab8640 1 --2- 172.21.15.63:0/168631363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1440101c80 0x7f1440112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:27:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.465+0000 7f14452b9640 1 --2- 172.21.15.63:0/168631363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14401012e0 0x7f1440111df0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1434034040 tx=0x7f1434009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.465+0000 7f1431ffb640 1 -- 172.21.15.63:0/168631363 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14340092b0 con 0x7f14401012e0 2024-03-22T07:27:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.465+0000 7f144751d640 1 -- 172.21.15.63:0/168631363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144010c0d0 con 0x7f14401012e0 2024-03-22T07:27:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.465+0000 7f1431ffb640 1 -- 172.21.15.63:0/168631363 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f143403b400 con 0x7f14401012e0 2024-03-22T07:27:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.465+0000 7f1431ffb640 1 -- 172.21.15.63:0/168631363 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14340435d0 con 0x7f14401012e0 2024-03-22T07:27:56.468 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.466+0000 7f144751d640 1 -- 172.21.15.63:0/168631363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144010c590 con 0x7f14401012e0 2024-03-22T07:27:56.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.467+0000 7f144751d640 1 -- 172.21.15.63:0/168631363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14400fca50 con 0x7f14401012e0 2024-03-22T07:27:56.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.467+0000 7f1431ffb640 1 -- 172.21.15.63:0/168631363 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f143402f070 con 0x7f14401012e0 2024-03-22T07:27:56.469 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.467+0000 7f1431ffb640 1 --2- 172.21.15.63:0/168631363 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f142c06d2f0 0x7f142c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:56.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.467+0000 7f1431ffb640 1 -- 172.21.15.63:0/168631363 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f14340ae6f0 con 0x7f14401012e0 2024-03-22T07:27:56.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.467+0000 7f1444ab8640 1 --2- 172.21.15.63:0/168631363 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f142c06d2f0 0x7f142c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:56.470 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.468+0000 7f1444ab8640 1 --2- 172.21.15.63:0/168631363 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f142c06d2f0 0x7f142c06f7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f144010d6f0 tx=0x7f143c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:56.472 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.472+0000 7f1431ffb640 1 -- 172.21.15.63:0/168631363 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1434078120 con 0x7f14401012e0 2024-03-22T07:27:56.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:56 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/4174554125' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T07:27:56.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:56 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2949947809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-22T07:27:56.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:56 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/1996186359' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-22T07:27:56.690 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.689+0000 7f144751d640 1 -- 172.21.15.63:0/168631363 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f144004bcc0 con 0x7f14401012e0 2024-03-22T07:27:56.691 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.690+0000 7f1431ffb640 1 -- 172.21.15.63:0/168631363 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v44) v1 ==== 107+0+4857 (secure 0 0 0) 0x7f143407b7e0 con 0x7f14401012e0 2024-03-22T07:27:56.691 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:27:56.692 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:20:16.089229+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi063.xeelpk","rank":0,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.63:6829/1373266298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1373266298},{"type":"v1","addr":"172.21.15.63:6829","nonce":1373266298}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14492":{"gid":14492,"name":"cephfs.smithi063.oytapz","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6827/3211595802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":3211595802},{"type":"v1","addr":"172.21.15.63:6827","nonce":3211595802}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24285":{"gid":24285,"name":"cephfs.smithi069.wwtsjn","rank":1,"incarnation":8,"state":"up:active","state_seq":4,"addr":"172.21.15.69:6825/3798247699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3798247699},{"type":"v1","addr":"172.21.15.69:6825","nonce":3798247699}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24301":{"gid":24301,"name":"cephfs.smithi069.cijxah","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.69:6827/3380719311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3380719311},{"type":"v1","addr":"172.21.15.69:6827","nonce":3380719311}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:27:56.694 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 23 2024-03-22T07:27:56.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.692+0000 7f144751d640 1 -- 172.21.15.63:0/168631363 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f142c06d2f0 msgr2=0x7f142c06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:56.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.692+0000 7f144751d640 1 --2- 172.21.15.63:0/168631363 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f142c06d2f0 0x7f142c06f7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f144010d6f0 tx=0x7f143c00b040 comp rx=0 tx=0).stop 2024-03-22T07:27:56.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.692+0000 7f144751d640 1 -- 172.21.15.63:0/168631363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14401012e0 msgr2=0x7f1440111df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:56.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.692+0000 7f144751d640 1 --2- 172.21.15.63:0/168631363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14401012e0 0x7f1440111df0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1434034040 tx=0x7f1434009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.692+0000 7f144751d640 1 -- 172.21.15.63:0/168631363 shutdown_connections 2024-03-22T07:27:56.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.692+0000 7f144751d640 1 --2- 172.21.15.63:0/168631363 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f142c06d2f0 0x7f142c06f7a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.692+0000 7f144751d640 1 --2- 172.21.15.63:0/168631363 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1440101c80 0x7f1440112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.692+0000 7f144751d640 1 --2- 172.21.15.63:0/168631363 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f14401012e0 0x7f1440111df0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.692+0000 7f144751d640 1 -- 172.21.15.63:0/168631363 >> 172.21.15.63:0/168631363 conn(0x7f14400f6e70 msgr2=0x7f14400f7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:56.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.692+0000 7f144751d640 1 -- 172.21.15.63:0/168631363 shutdown_connections 2024-03-22T07:27:56.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.692+0000 7f144751d640 1 -- 172.21.15.63:0/168631363 wait complete. 2024-03-22T07:27:56.704 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-03-22T07:27:56.808 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.803+0000 7f00ad39d640 1 -- 172.21.15.63:0/2090531215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00a8102b90 msgr2=0x7f00a80fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:56.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.803+0000 7f00ad39d640 1 --2- 172.21.15.63:0/2090531215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00a8102b90 0x7f00a80fc220 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f009c0048d0 tx=0x7f009c009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.803+0000 7f00ad39d640 1 -- 172.21.15.63:0/2090531215 shutdown_connections 2024-03-22T07:27:56.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.803+0000 7f00ad39d640 1 --2- 172.21.15.63:0/2090531215 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00a8102b90 0x7f00a80fc220 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.803+0000 7f00ad39d640 1 --2- 172.21.15.63:0/2090531215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00a81021f0 0x7f00a81025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.809 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.803+0000 7f00ad39d640 1 -- 172.21.15.63:0/2090531215 >> 172.21.15.63:0/2090531215 conn(0x7f00a80f7dc0 msgr2=0x7f00a80fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:56.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.803+0000 7f00ad39d640 1 -- 172.21.15.63:0/2090531215 shutdown_connections 2024-03-22T07:27:56.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.804+0000 7f00ad39d640 1 -- 172.21.15.63:0/2090531215 wait complete. 2024-03-22T07:27:56.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.804+0000 7f00ad39d640 1 Processor -- start 2024-03-22T07:27:56.810 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.804+0000 7f00ad39d640 1 -- start start 2024-03-22T07:27:56.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.804+0000 7f00ad39d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00a81021f0 0x7f00a8112cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:56.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.804+0000 7f00ad39d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00a8102b90 0x7f00a8113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:56.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.804+0000 7f00ad39d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a8113830 con 0x7f00a8102b90 2024-03-22T07:27:56.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.804+0000 7f00ad39d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a810ce80 con 0x7f00a81021f0 2024-03-22T07:27:56.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.804+0000 7f00a67fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00a8102b90 0x7f00a8113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:56.811 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.804+0000 7f00a67fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00a8102b90 0x7f00a8113230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42396/0 (socket says 172.21.15.63:42396) 2024-03-22T07:27:56.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.804+0000 7f00a67fc640 1 -- 172.21.15.63:0/4154885758 learned_addr learned my addr 172.21.15.63:0/4154885758 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:56.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.805+0000 7f00a6ffd640 1 --2- 172.21.15.63:0/4154885758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00a81021f0 0x7f00a8112cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:56.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.805+0000 7f00a67fc640 1 -- 172.21.15.63:0/4154885758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00a81021f0 msgr2=0x7f00a8112cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:56.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.805+0000 7f00a67fc640 1 --2- 172.21.15.63:0/4154885758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00a81021f0 0x7f00a8112cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:56.812 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.805+0000 7f00a67fc640 1 -- 172.21.15.63:0/4154885758 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009c005040 con 0x7f00a8102b90 2024-03-22T07:27:56.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.805+0000 7f00a67fc640 1 --2- 172.21.15.63:0/4154885758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00a8102b90 0x7f00a8113230 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f009c004930 tx=0x7f009c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:56.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.805+0000 7f008f7fe640 1 -- 172.21.15.63:0/4154885758 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f009c0094c0 con 0x7f00a8102b90 2024-03-22T07:27:56.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.805+0000 7f00ad39d640 1 -- 172.21.15.63:0/4154885758 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a810d100 con 0x7f00a8102b90 2024-03-22T07:27:56.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.805+0000 7f008f7fe640 1 -- 172.21.15.63:0/4154885758 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f009c032e50 con 0x7f00a8102b90 2024-03-22T07:27:56.813 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.805+0000 7f008f7fe640 1 -- 172.21.15.63:0/4154885758 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f009c0433b0 con 0x7f00a8102b90 2024-03-22T07:27:56.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.805+0000 7f00ad39d640 1 -- 172.21.15.63:0/4154885758 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a810d5c0 con 0x7f00a8102b90 2024-03-22T07:27:56.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.806+0000 7f00ad39d640 1 -- 172.21.15.63:0/4154885758 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a80fd960 con 0x7f00a8102b90 2024-03-22T07:27:56.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.807+0000 7f008f7fe640 1 -- 172.21.15.63:0/4154885758 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f009c02f070 con 0x7f00a8102b90 2024-03-22T07:27:56.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.807+0000 7f008f7fe640 1 --2- 172.21.15.63:0/4154885758 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f009006d2f0 0x7f009006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:56.814 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.807+0000 7f008f7fe640 1 -- 172.21.15.63:0/4154885758 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f009c0ae9c0 con 0x7f00a8102b90 2024-03-22T07:27:56.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.807+0000 7f00a6ffd640 1 --2- 172.21.15.63:0/4154885758 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f009006d2f0 0x7f009006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:56.815 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.807+0000 7f00a6ffd640 1 --2- 172.21.15.63:0/4154885758 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f009006d2f0 0x7f009006f7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f00a0004c40 tx=0x7f00a0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:56.816 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:56.811+0000 7f008f7fe640 1 -- 172.21.15.63:0/4154885758 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f009c0783f0 con 0x7f00a8102b90 2024-03-22T07:27:57.030 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.029+0000 7f00ad39d640 1 -- 172.21.15.63:0/4154885758 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f00a8109330 con 0x7f00a8102b90 2024-03-22T07:27:57.031 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.030+0000 7f008f7fe640 1 -- 172.21.15.63:0/4154885758 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v44) v1 ==== 107+0+4856 (secure 0 0 0) 0x7f009c07bab0 con 0x7f00a8102b90 2024-03-22T07:27:57.031 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:27:57.032 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:20:24.253589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi063.xeelpk","rank":0,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.63:6829/1373266298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1373266298},{"type":"v1","addr":"172.21.15.63:6829","nonce":1373266298}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14492":{"gid":14492,"name":"cephfs.smithi063.oytapz","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.63:6827/3211595802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":3211595802},{"type":"v1","addr":"172.21.15.63:6827","nonce":3211595802}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24285":{"gid":24285,"name":"cephfs.smithi069.wwtsjn","rank":1,"incarnation":8,"state":"up:active","state_seq":4,"addr":"172.21.15.69:6825/3798247699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3798247699},{"type":"v1","addr":"172.21.15.69:6825","nonce":3798247699}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24301":{"gid":24301,"name":"cephfs.smithi069.cijxah","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.69:6827/3380719311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3380719311},{"type":"v1","addr":"172.21.15.69:6827","nonce":3380719311}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:27:57.034 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 24 2024-03-22T07:27:57.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.032+0000 7f00ad39d640 1 -- 172.21.15.63:0/4154885758 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f009006d2f0 msgr2=0x7f009006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:57.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.032+0000 7f00ad39d640 1 --2- 172.21.15.63:0/4154885758 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f009006d2f0 0x7f009006f7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f00a0004c40 tx=0x7f00a0009040 comp rx=0 tx=0).stop 2024-03-22T07:27:57.034 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.032+0000 7f00ad39d640 1 -- 172.21.15.63:0/4154885758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00a8102b90 msgr2=0x7f00a8113230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:57.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.032+0000 7f00ad39d640 1 --2- 172.21.15.63:0/4154885758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00a8102b90 0x7f00a8113230 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f009c004930 tx=0x7f009c009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.032+0000 7f00ad39d640 1 -- 172.21.15.63:0/4154885758 shutdown_connections 2024-03-22T07:27:57.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.032+0000 7f00ad39d640 1 --2- 172.21.15.63:0/4154885758 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f00a8102b90 0x7f00a8113230 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.032+0000 7f00ad39d640 1 --2- 172.21.15.63:0/4154885758 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f009006d2f0 0x7f009006f7a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.032+0000 7f00ad39d640 1 --2- 172.21.15.63:0/4154885758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f00a81021f0 0x7f00a8112cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.035 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.032+0000 7f00ad39d640 1 -- 172.21.15.63:0/4154885758 >> 172.21.15.63:0/4154885758 conn(0x7f00a80f7dc0 msgr2=0x7f00a80f8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:57.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.032+0000 7f00ad39d640 1 -- 172.21.15.63:0/4154885758 shutdown_connections 2024-03-22T07:27:57.036 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.032+0000 7f00ad39d640 1 -- 172.21.15.63:0/4154885758 wait complete. 2024-03-22T07:27:57.044 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-03-22T07:27:57.146 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.143+0000 7fdbefe58640 1 -- 172.21.15.63:0/3779708487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbe8101100 msgr2=0x7fdbe81014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:57.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.143+0000 7fdbefe58640 1 --2- 172.21.15.63:0/3779708487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbe8101100 0x7fdbe81014d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0048d0 tx=0x7fdbdc009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.143+0000 7fdbefe58640 1 -- 172.21.15.63:0/3779708487 shutdown_connections 2024-03-22T07:27:57.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.143+0000 7fdbefe58640 1 --2- 172.21.15.63:0/3779708487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbe8101aa0 0x7fdbe80fb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.143+0000 7fdbefe58640 1 --2- 172.21.15.63:0/3779708487 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbe8101100 0x7fdbe81014d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.143+0000 7fdbefe58640 1 -- 172.21.15.63:0/3779708487 >> 172.21.15.63:0/3779708487 conn(0x7fdbe80f6cf0 msgr2=0x7fdbe80f9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:57.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.143+0000 7fdbefe58640 1 -- 172.21.15.63:0/3779708487 shutdown_connections 2024-03-22T07:27:57.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.143+0000 7fdbefe58640 1 -- 172.21.15.63:0/3779708487 wait complete. 2024-03-22T07:27:57.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.144+0000 7fdbefe58640 1 Processor -- start 2024-03-22T07:27:57.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.144+0000 7fdbefe58640 1 -- start start 2024-03-22T07:27:57.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.144+0000 7fdbefe58640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbe8101100 0x7fdbe8111c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:57.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.144+0000 7fdbefe58640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbe8101aa0 0x7fdbe8112180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:57.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.144+0000 7fdbefe58640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe8112810 con 0x7fdbe8101aa0 2024-03-22T07:27:57.147 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.144+0000 7fdbefe58640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe810bdd0 con 0x7fdbe8101100 2024-03-22T07:27:57.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.144+0000 7fdbed3f3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbe8101aa0 0x7fdbe8112180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:57.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.144+0000 7fdbed3f3640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbe8101aa0 0x7fdbe8112180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42410/0 (socket says 172.21.15.63:42410) 2024-03-22T07:27:57.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.144+0000 7fdbed3f3640 1 -- 172.21.15.63:0/2671508769 learned_addr learned my addr 172.21.15.63:0/2671508769 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:57.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.144+0000 7fdbedbf4640 1 --2- 172.21.15.63:0/2671508769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbe8101100 0x7fdbe8111c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:57.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.144+0000 7fdbed3f3640 1 -- 172.21.15.63:0/2671508769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbe8101100 msgr2=0x7fdbe8111c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:57.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.144+0000 7fdbed3f3640 1 --2- 172.21.15.63:0/2671508769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbe8101100 0x7fdbe8111c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.144+0000 7fdbed3f3640 1 -- 172.21.15.63:0/2671508769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc005040 con 0x7fdbe8101aa0 2024-03-22T07:27:57.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.144+0000 7fdbed3f3640 1 --2- 172.21.15.63:0/2671508769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbe8101aa0 0x7fdbe8112180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400b790 tx=0x7fdbe40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:57.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.144+0000 7fdbda7fc640 1 -- 172.21.15.63:0/2671508769 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbe400c700 con 0x7fdbe8101aa0 2024-03-22T07:27:57.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.144+0000 7fdbda7fc640 1 -- 172.21.15.63:0/2671508769 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbe400cd30 con 0x7fdbe8101aa0 2024-03-22T07:27:57.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.144+0000 7fdbefe58640 1 -- 172.21.15.63:0/2671508769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe810c0b0 con 0x7fdbe8101aa0 2024-03-22T07:27:57.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.145+0000 7fdbda7fc640 1 -- 172.21.15.63:0/2671508769 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbe4013370 con 0x7fdbe8101aa0 2024-03-22T07:27:57.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.145+0000 7fdbefe58640 1 -- 172.21.15.63:0/2671508769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe810c5d0 con 0x7fdbe8101aa0 2024-03-22T07:27:57.148 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.145+0000 7fdbefe58640 1 -- 172.21.15.63:0/2671508769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe80fc870 con 0x7fdbe8101aa0 2024-03-22T07:27:57.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.146+0000 7fdbda7fc640 1 -- 172.21.15.63:0/2671508769 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fdbe4012070 con 0x7fdbe8101aa0 2024-03-22T07:27:57.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.146+0000 7fdbda7fc640 1 --2- 172.21.15.63:0/2671508769 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdbd406d2f0 0x7fdbd406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:57.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.147+0000 7fdbedbf4640 1 --2- 172.21.15.63:0/2671508769 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdbd406d2f0 0x7fdbd406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:57.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.147+0000 7fdbda7fc640 1 -- 172.21.15.63:0/2671508769 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdbe408c5b0 con 0x7fdbe8101aa0 2024-03-22T07:27:57.149 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.147+0000 7fdbedbf4640 1 --2- 172.21.15.63:0/2671508769 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdbd406d2f0 0x7fdbd406f7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc003080 tx=0x7fdbdc009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:57.151 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.151+0000 7fdbda7fc640 1 -- 172.21.15.63:0/2671508769 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdbe4055fe0 con 0x7fdbe8101aa0 2024-03-22T07:27:57.373 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.372+0000 7fdbefe58640 1 -- 172.21.15.63:0/2671508769 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7fdbe804bcc0 con 0x7fdbe8101aa0 2024-03-22T07:27:57.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:57 smithi063 ceph-mon[112784]: pgmap v262: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:27:57.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:57 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2793480361' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-22T07:27:57.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:57 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/168631363' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-22T07:27:57.374 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:57 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/4154885758' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-22T07:27:57.374 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.373+0000 7fdbda7fc640 1 -- 172.21.15.63:0/2671508769 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v44) v1 ==== 107+0+3284 (secure 0 0 0) 0x7fdbe40596a0 con 0x7fdbe8101aa0 2024-03-22T07:27:57.375 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:27:57.375 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:24:59.832788+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14480,"mds_1":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14480":{"gid":14480,"name":"cephfs.smithi063.xeelpk","rank":0,"incarnation":15,"state":"up:active","state_seq":6,"addr":"172.21.15.63:6829/1373266298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1373266298},{"type":"v1","addr":"172.21.15.63:6829","nonce":1373266298}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24285":{"gid":24285,"name":"cephfs.smithi069.wwtsjn","rank":1,"incarnation":8,"state":"up:active","state_seq":4,"addr":"172.21.15.69:6825/3798247699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":3798247699},{"type":"v1","addr":"172.21.15.69:6825","nonce":3798247699}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:27:57.377 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 25 2024-03-22T07:27:57.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.375+0000 7fdbefe58640 1 -- 172.21.15.63:0/2671508769 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdbd406d2f0 msgr2=0x7fdbd406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:57.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.375+0000 7fdbefe58640 1 --2- 172.21.15.63:0/2671508769 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdbd406d2f0 0x7fdbd406f7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc003080 tx=0x7fdbdc009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.377 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.375+0000 7fdbefe58640 1 -- 172.21.15.63:0/2671508769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbe8101aa0 msgr2=0x7fdbe8112180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:57.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.375+0000 7fdbefe58640 1 --2- 172.21.15.63:0/2671508769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbe8101aa0 0x7fdbe8112180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400b790 tx=0x7fdbe40091d0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.375+0000 7fdbefe58640 1 -- 172.21.15.63:0/2671508769 shutdown_connections 2024-03-22T07:27:57.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.375+0000 7fdbefe58640 1 --2- 172.21.15.63:0/2671508769 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdbe8101aa0 0x7fdbe8112180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.375+0000 7fdbefe58640 1 --2- 172.21.15.63:0/2671508769 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdbd406d2f0 0x7fdbd406f7a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.375+0000 7fdbefe58640 1 --2- 172.21.15.63:0/2671508769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbe8101100 0x7fdbe8111c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.375+0000 7fdbefe58640 1 -- 172.21.15.63:0/2671508769 >> 172.21.15.63:0/2671508769 conn(0x7fdbe80f6cf0 msgr2=0x7fdbe80f7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:57.378 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.375+0000 7fdbefe58640 1 -- 172.21.15.63:0/2671508769 shutdown_connections 2024-03-22T07:27:57.379 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.375+0000 7fdbefe58640 1 -- 172.21.15.63:0/2671508769 wait complete. 2024-03-22T07:27:57.386 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-03-22T07:27:57.387 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-03-22T07:27:57.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.486+0000 7f1b7782e640 1 -- 172.21.15.63:0/4055885062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b700fdd70 msgr2=0x7f1b70106440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:57.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.486+0000 7f1b7782e640 1 --2- 172.21.15.63:0/4055885062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b700fdd70 0x7f1b70106440 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1b60004900 tx=0x7f1b60009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.486+0000 7f1b7782e640 1 -- 172.21.15.63:0/4055885062 shutdown_connections 2024-03-22T07:27:57.488 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.486+0000 7f1b7782e640 1 --2- 172.21.15.63:0/4055885062 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b700fdd70 0x7f1b70106440 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.486+0000 7f1b7782e640 1 --2- 172.21.15.63:0/4055885062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b700fd3d0 0x7f1b700fd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.486+0000 7f1b7782e640 1 -- 172.21.15.63:0/4055885062 >> 172.21.15.63:0/4055885062 conn(0x7f1b700f6e70 msgr2=0x7f1b700f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:57.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.486+0000 7f1b7782e640 1 -- 172.21.15.63:0/4055885062 shutdown_connections 2024-03-22T07:27:57.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.486+0000 7f1b7782e640 1 -- 172.21.15.63:0/4055885062 wait complete. 2024-03-22T07:27:57.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.486+0000 7f1b7782e640 1 Processor -- start 2024-03-22T07:27:57.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.487+0000 7f1b7782e640 1 -- start start 2024-03-22T07:27:57.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.487+0000 7f1b7782e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b700fd3d0 0x7f1b70112fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:57.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.487+0000 7f1b7782e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b700fdd70 0x7f1b70113520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:57.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.487+0000 7f1b7782e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b70113b20 con 0x7f1b700fdd70 2024-03-22T07:27:57.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.487+0000 7f1b7782e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b70113c90 con 0x7f1b700fd3d0 2024-03-22T07:27:57.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.487+0000 7f1b74dc9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b700fdd70 0x7f1b70113520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:57.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.487+0000 7f1b74dc9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b700fdd70 0x7f1b70113520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42434/0 (socket says 172.21.15.63:42434) 2024-03-22T07:27:57.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.487+0000 7f1b74dc9640 1 -- 172.21.15.63:0/29398492 learned_addr learned my addr 172.21.15.63:0/29398492 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:57.489 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.487+0000 7f1b755ca640 1 --2- 172.21.15.63:0/29398492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b700fd3d0 0x7f1b70112fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:57.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.487+0000 7f1b74dc9640 1 -- 172.21.15.63:0/29398492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b700fd3d0 msgr2=0x7f1b70112fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:57.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.487+0000 7f1b74dc9640 1 --2- 172.21.15.63:0/29398492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b700fd3d0 0x7f1b70112fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.487+0000 7f1b74dc9640 1 -- 172.21.15.63:0/29398492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b60005040 con 0x7f1b700fdd70 2024-03-22T07:27:57.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.487+0000 7f1b74dc9640 1 --2- 172.21.15.63:0/29398492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b700fdd70 0x7f1b70113520 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1b60000c00 tx=0x7f1b60009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:57.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.488+0000 7f1b65ffb640 1 -- 172.21.15.63:0/29398492 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b600094c0 con 0x7f1b700fdd70 2024-03-22T07:27:57.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.488+0000 7f1b7782e640 1 -- 172.21.15.63:0/29398492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b7010d2c0 con 0x7f1b700fdd70 2024-03-22T07:27:57.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.488+0000 7f1b65ffb640 1 -- 172.21.15.63:0/29398492 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b6003d920 con 0x7f1b700fdd70 2024-03-22T07:27:57.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.488+0000 7f1b65ffb640 1 -- 172.21.15.63:0/29398492 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b6003b8c0 con 0x7f1b700fdd70 2024-03-22T07:27:57.490 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.488+0000 7f1b7782e640 1 -- 172.21.15.63:0/29398492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b7010d7e0 con 0x7f1b700fdd70 2024-03-22T07:27:57.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.489+0000 7f1b7782e640 1 -- 172.21.15.63:0/29398492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b70065dd0 con 0x7f1b700fdd70 2024-03-22T07:27:57.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.489+0000 7f1b65ffb640 1 -- 172.21.15.63:0/29398492 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f1b6003da90 con 0x7f1b700fdd70 2024-03-22T07:27:57.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.490+0000 7f1b65ffb640 1 --2- 172.21.15.63:0/29398492 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1b5c06d2f0 0x7f1b5c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:57.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.490+0000 7f1b755ca640 1 --2- 172.21.15.63:0/29398492 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1b5c06d2f0 0x7f1b5c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:57.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.490+0000 7f1b65ffb640 1 -- 172.21.15.63:0/29398492 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1b600af1d0 con 0x7f1b700fdd70 2024-03-22T07:27:57.491 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.490+0000 7f1b755ca640 1 --2- 172.21.15.63:0/29398492 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1b5c06d2f0 0x7f1b5c06f7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c007fd0 tx=0x7f1b6c009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:57.494 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.494+0000 7f1b65ffb640 1 -- 172.21.15.63:0/29398492 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1b60078cc0 con 0x7f1b700fdd70 2024-03-22T07:27:57.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:57 smithi069 ceph-mon[83631]: pgmap v262: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:27:57.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:57 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2793480361' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-22T07:27:57.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:57 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/168631363' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-22T07:27:57.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:57 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/4154885758' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-22T07:27:57.712 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.712+0000 7f1b7782e640 1 -- 172.21.15.63:0/29398492 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f1b7004bcc0 con 0x7f1b700fdd70 2024-03-22T07:27:57.713 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.713+0000 7f1b65ffb640 1 -- 172.21.15.63:0/29398492 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v44) v1 ==== 107+0+3260 (secure 0 0 0) 0x7f1b60036cd0 con 0x7f1b700fdd70 2024-03-22T07:27:57.713 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:27:57.713 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi063.oytapz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/1673237445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1673237445},{"type":"v1","addr":"172.21.15.63:6827","nonce":1673237445}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44273,"name":"cephfs.smithi069.cijxah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6827/675976207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":675976207},{"type":"v1","addr":"172.21.15.69:6827","nonce":675976207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:25:00.991999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:27:57.713 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 26 2024-03-22T07:27:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.715+0000 7f1b7782e640 1 -- 172.21.15.63:0/29398492 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1b5c06d2f0 msgr2=0x7f1b5c06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.715+0000 7f1b7782e640 1 --2- 172.21.15.63:0/29398492 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1b5c06d2f0 0x7f1b5c06f7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c007fd0 tx=0x7f1b6c009e90 comp rx=0 tx=0).stop 2024-03-22T07:27:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.715+0000 7f1b7782e640 1 -- 172.21.15.63:0/29398492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b700fdd70 msgr2=0x7f1b70113520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.715+0000 7f1b7782e640 1 --2- 172.21.15.63:0/29398492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b700fdd70 0x7f1b70113520 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1b60000c00 tx=0x7f1b60009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.715+0000 7f1b7782e640 1 -- 172.21.15.63:0/29398492 shutdown_connections 2024-03-22T07:27:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.715+0000 7f1b7782e640 1 --2- 172.21.15.63:0/29398492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1b700fdd70 0x7f1b70113520 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.715 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.715+0000 7f1b7782e640 1 --2- 172.21.15.63:0/29398492 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1b5c06d2f0 0x7f1b5c06f7a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.715+0000 7f1b7782e640 1 --2- 172.21.15.63:0/29398492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1b700fd3d0 0x7f1b70112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.715+0000 7f1b7782e640 1 -- 172.21.15.63:0/29398492 >> 172.21.15.63:0/29398492 conn(0x7f1b700f6e70 msgr2=0x7f1b700f83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.715+0000 7f1b7782e640 1 -- 172.21.15.63:0/29398492 shutdown_connections 2024-03-22T07:27:57.716 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.715+0000 7f1b7782e640 1 -- 172.21.15.63:0/29398492 wait complete. 2024-03-22T07:27:57.726 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:27:57.726 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-03-22T07:27:57.726 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-03-22T07:27:57.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.825+0000 7f835559d640 1 -- 172.21.15.63:0/4228013053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8350102200 msgr2=0x7f83501025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:57.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.825+0000 7f835559d640 1 --2- 172.21.15.63:0/4228013053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8350102200 0x7f83501025d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f83480048d0 tx=0x7f8348009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.825+0000 7f835559d640 1 -- 172.21.15.63:0/4228013053 shutdown_connections 2024-03-22T07:27:57.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.825+0000 7f835559d640 1 --2- 172.21.15.63:0/4228013053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8350102ba0 0x7f83500fa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.825+0000 7f835559d640 1 --2- 172.21.15.63:0/4228013053 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8350102200 0x7f83501025d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.825+0000 7f835559d640 1 -- 172.21.15.63:0/4228013053 >> 172.21.15.63:0/4228013053 conn(0x7f83500708f0 msgr2=0x7f8350070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:57.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.825+0000 7f835559d640 1 -- 172.21.15.63:0/4228013053 shutdown_connections 2024-03-22T07:27:57.828 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.825+0000 7f835559d640 1 -- 172.21.15.63:0/4228013053 wait complete. 2024-03-22T07:27:57.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.826+0000 7f835559d640 1 Processor -- start 2024-03-22T07:27:57.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.826+0000 7f835559d640 1 -- start start 2024-03-22T07:27:57.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.826+0000 7f835559d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8350102200 0x7f83500fbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:57.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.826+0000 7f835559d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8350102ba0 0x7f83500fbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:57.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.826+0000 7f835559d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83500fd990 con 0x7f8350102ba0 2024-03-22T07:27:57.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.826+0000 7f835559d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83500fc530 con 0x7f8350102200 2024-03-22T07:27:57.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.826+0000 7f834e7fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8350102ba0 0x7f83500fbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:57.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.826+0000 7f834e7fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8350102ba0 0x7f83500fbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42442/0 (socket says 172.21.15.63:42442) 2024-03-22T07:27:57.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.826+0000 7f834e7fc640 1 -- 172.21.15.63:0/1815080136 learned_addr learned my addr 172.21.15.63:0/1815080136 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:57.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.826+0000 7f834effd640 1 --2- 172.21.15.63:0/1815080136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8350102200 0x7f83500fbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:57.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.827+0000 7f834e7fc640 1 -- 172.21.15.63:0/1815080136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8350102200 msgr2=0x7f83500fbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:57.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.827+0000 7f834e7fc640 1 --2- 172.21.15.63:0/1815080136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8350102200 0x7f83500fbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:57.829 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.827+0000 7f834e7fc640 1 -- 172.21.15.63:0/1815080136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8348005040 con 0x7f8350102ba0 2024-03-22T07:27:57.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.827+0000 7f834effd640 1 --2- 172.21.15.63:0/1815080136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8350102200 0x7f83500fbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:27:57.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.827+0000 7f834e7fc640 1 --2- 172.21.15.63:0/1815080136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8350102ba0 0x7f83500fbff0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f833800eb40 tx=0x7f833800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:57.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.827+0000 7f83477fe640 1 -- 172.21.15.63:0/1815080136 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8338007c30 con 0x7f8350102ba0 2024-03-22T07:27:57.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.827+0000 7f83477fe640 1 -- 172.21.15.63:0/1815080136 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f833800f040 con 0x7f8350102ba0 2024-03-22T07:27:57.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.827+0000 7f835559d640 1 -- 172.21.15.63:0/1815080136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83500fc7b0 con 0x7f8350102ba0 2024-03-22T07:27:57.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.827+0000 7f83477fe640 1 -- 172.21.15.63:0/1815080136 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8338009620 con 0x7f8350102ba0 2024-03-22T07:27:57.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.827+0000 7f835559d640 1 -- 172.21.15.63:0/1815080136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8350119500 con 0x7f8350102ba0 2024-03-22T07:27:57.830 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.828+0000 7f835559d640 1 -- 172.21.15.63:0/1815080136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83500652a0 con 0x7f8350102ba0 2024-03-22T07:27:57.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.828+0000 7f83477fe640 1 -- 172.21.15.63:0/1815080136 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f833800c6a0 con 0x7f8350102ba0 2024-03-22T07:27:57.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.829+0000 7f83477fe640 1 --2- 172.21.15.63:0/1815080136 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f833c06d2f0 0x7f833c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:57.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.829+0000 7f83477fe640 1 -- 172.21.15.63:0/1815080136 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f833808c680 con 0x7f8350102ba0 2024-03-22T07:27:57.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.829+0000 7f834effd640 1 --2- 172.21.15.63:0/1815080136 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f833c06d2f0 0x7f833c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:57.831 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.829+0000 7f834effd640 1 --2- 172.21.15.63:0/1815080136 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f833c06d2f0 0x7f833c06f7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8348000c00 tx=0x7f8348009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:57.833 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:57.833+0000 7f83477fe640 1 -- 172.21.15.63:0/1815080136 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f83380560b0 con 0x7f8350102ba0 2024-03-22T07:27:58.051 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.051+0000 7f835559d640 1 -- 172.21.15.63:0/1815080136 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f835004bcc0 con 0x7f8350102ba0 2024-03-22T07:27:58.052 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.052+0000 7f83477fe640 1 -- 172.21.15.63:0/1815080136 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v44) v1 ==== 107+0+4818 (secure 0 0 0) 0x7f8338059770 con 0x7f8350102ba0 2024-03-22T07:27:58.052 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:27:58.052 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi063.oytapz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/1673237445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1673237445},{"type":"v1","addr":"172.21.15.63:6827","nonce":1673237445}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":34388,"name":"cephfs.smithi063.xeelpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/1202791208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1202791208},{"type":"v1","addr":"172.21.15.63:6829","nonce":1202791208}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44273,"name":"cephfs.smithi069.cijxah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6827/675976207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":675976207},{"type":"v1","addr":"172.21.15.69:6827","nonce":675976207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44279,"name":"cephfs.smithi069.wwtsjn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6825/2785357568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2785357568},{"type":"v1","addr":"172.21.15.69:6825","nonce":2785357568}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:25:00.991999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:27:58.052 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 27 2024-03-22T07:27:58.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.054+0000 7f835559d640 1 -- 172.21.15.63:0/1815080136 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f833c06d2f0 msgr2=0x7f833c06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:58.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.054+0000 7f835559d640 1 --2- 172.21.15.63:0/1815080136 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f833c06d2f0 0x7f833c06f7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8348000c00 tx=0x7f8348009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.054+0000 7f835559d640 1 -- 172.21.15.63:0/1815080136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8350102ba0 msgr2=0x7f83500fbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:58.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.054+0000 7f835559d640 1 --2- 172.21.15.63:0/1815080136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8350102ba0 0x7f83500fbff0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f833800eb40 tx=0x7f833800b040 comp rx=0 tx=0).stop 2024-03-22T07:27:58.054 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.054+0000 7f835559d640 1 -- 172.21.15.63:0/1815080136 shutdown_connections 2024-03-22T07:27:58.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.054+0000 7f835559d640 1 --2- 172.21.15.63:0/1815080136 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8350102ba0 0x7f83500fbff0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.054+0000 7f835559d640 1 --2- 172.21.15.63:0/1815080136 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f833c06d2f0 0x7f833c06f7a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.054+0000 7f835559d640 1 --2- 172.21.15.63:0/1815080136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8350102200 0x7f83500fbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.054+0000 7f835559d640 1 -- 172.21.15.63:0/1815080136 >> 172.21.15.63:0/1815080136 conn(0x7f83500708f0 msgr2=0x7f8350104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:58.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.054+0000 7f835559d640 1 -- 172.21.15.63:0/1815080136 shutdown_connections 2024-03-22T07:27:58.055 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.054+0000 7f835559d640 1 -- 172.21.15.63:0/1815080136 wait complete. 2024-03-22T07:27:58.065 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:27:58.065 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-03-22T07:27:58.065 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-03-22T07:27:58.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.165+0000 7f4f6b0ae640 1 -- 172.21.15.63:0/3204093167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f64101c60 msgr2=0x7f4f640fb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:58.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.165+0000 7f4f6b0ae640 1 --2- 172.21.15.63:0/3204093167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f64101c60 0x7f4f640fb2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4f500048d0 tx=0x7f4f50009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.165+0000 7f4f6b0ae640 1 -- 172.21.15.63:0/3204093167 shutdown_connections 2024-03-22T07:27:58.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.165+0000 7f4f6b0ae640 1 --2- 172.21.15.63:0/3204093167 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f64101c60 0x7f4f640fb2f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.165+0000 7f4f6b0ae640 1 --2- 172.21.15.63:0/3204093167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f641012c0 0x7f4f64101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.165+0000 7f4f6b0ae640 1 -- 172.21.15.63:0/3204093167 >> 172.21.15.63:0/3204093167 conn(0x7f4f640f6e90 msgr2=0x7f4f640f92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:58.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.165+0000 7f4f6b0ae640 1 -- 172.21.15.63:0/3204093167 shutdown_connections 2024-03-22T07:27:58.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.165+0000 7f4f6b0ae640 1 -- 172.21.15.63:0/3204093167 wait complete. 2024-03-22T07:27:58.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.165+0000 7f4f6b0ae640 1 Processor -- start 2024-03-22T07:27:58.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.165+0000 7f4f6b0ae640 1 -- start start 2024-03-22T07:27:58.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.165+0000 7f4f6b0ae640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f641012c0 0x7f4f64111e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:58.167 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.165+0000 7f4f6b0ae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f64101c60 0x7f4f64112350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:58.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.165+0000 7f4f6b0ae640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f64112950 con 0x7f4f641012c0 2024-03-22T07:27:58.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.165+0000 7f4f6b0ae640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6410bfa0 con 0x7f4f64101c60 2024-03-22T07:27:58.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.166+0000 7f4f68e4a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f641012c0 0x7f4f64111e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:58.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.166+0000 7f4f68e4a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f641012c0 0x7f4f64111e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42454/0 (socket says 172.21.15.63:42454) 2024-03-22T07:27:58.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.166+0000 7f4f68e4a640 1 -- 172.21.15.63:0/559006040 learned_addr learned my addr 172.21.15.63:0/559006040 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:58.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.166+0000 7f4f63fff640 1 --2- 172.21.15.63:0/559006040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f64101c60 0x7f4f64112350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:58.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.166+0000 7f4f68e4a640 1 -- 172.21.15.63:0/559006040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f64101c60 msgr2=0x7f4f64112350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:58.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.166+0000 7f4f68e4a640 1 --2- 172.21.15.63:0/559006040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f64101c60 0x7f4f64112350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.166+0000 7f4f68e4a640 1 -- 172.21.15.63:0/559006040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f50005040 con 0x7f4f641012c0 2024-03-22T07:27:58.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.166+0000 7f4f68e4a640 1 --2- 172.21.15.63:0/559006040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f641012c0 0x7f4f64111e10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4f58002a80 tx=0x7f4f58004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:58.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.166+0000 7f4f617fa640 1 -- 172.21.15.63:0/559006040 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f58007c30 con 0x7f4f641012c0 2024-03-22T07:27:58.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.166+0000 7f4f617fa640 1 -- 172.21.15.63:0/559006040 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f5800f040 con 0x7f4f641012c0 2024-03-22T07:27:58.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.166+0000 7f4f6b0ae640 1 -- 172.21.15.63:0/559006040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6410c280 con 0x7f4f641012c0 2024-03-22T07:27:58.168 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.166+0000 7f4f617fa640 1 -- 172.21.15.63:0/559006040 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f580096b0 con 0x7f4f641012c0 2024-03-22T07:27:58.169 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.166+0000 7f4f6b0ae640 1 -- 172.21.15.63:0/559006040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6410c7a0 con 0x7f4f641012c0 2024-03-22T07:27:58.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.167+0000 7f4f6b0ae640 1 -- 172.21.15.63:0/559006040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f640fca30 con 0x7f4f641012c0 2024-03-22T07:27:58.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.168+0000 7f4f617fa640 1 -- 172.21.15.63:0/559006040 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f4f5800d6b0 con 0x7f4f641012c0 2024-03-22T07:27:58.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.168+0000 7f4f617fa640 1 --2- 172.21.15.63:0/559006040 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f4f5406d2f0 0x7f4f5406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:58.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.168+0000 7f4f617fa640 1 -- 172.21.15.63:0/559006040 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f4f5808c390 con 0x7f4f641012c0 2024-03-22T07:27:58.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.168+0000 7f4f63fff640 1 --2- 172.21.15.63:0/559006040 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f4f5406d2f0 0x7f4f5406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:58.170 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.168+0000 7f4f63fff640 1 --2- 172.21.15.63:0/559006040 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f4f5406d2f0 0x7f4f5406f7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4f50000c00 tx=0x7f4f50009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:58.172 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.173+0000 7f4f617fa640 1 -- 172.21.15.63:0/559006040 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4f58055dc0 con 0x7f4f641012c0 2024-03-22T07:27:58.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:58 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2671508769' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-22T07:27:58.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:58 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/29398492' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-22T07:27:58.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.394+0000 7f4f6b0ae640 1 -- 172.21.15.63:0/559006040 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f4f6404bcc0 con 0x7f4f641012c0 2024-03-22T07:27:58.394 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.394+0000 7f4f617fa640 1 -- 172.21.15.63:0/559006040 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v44) v1 ==== 107+0+4039 (secure 0 0 0) 0x7f4f6404bcc0 con 0x7f4f641012c0 2024-03-22T07:27:58.395 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:27:58.395 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi063.xeelpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/1202791208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1202791208},{"type":"v1","addr":"172.21.15.63:6829","nonce":1202791208}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44273,"name":"cephfs.smithi069.cijxah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6827/675976207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":675976207},{"type":"v1","addr":"172.21.15.69:6827","nonce":675976207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44279,"name":"cephfs.smithi069.wwtsjn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6825/2785357568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2785357568},{"type":"v1","addr":"172.21.15.69:6825","nonce":2785357568}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:25:00.991999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:27:58.395 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 28 2024-03-22T07:27:58.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.397+0000 7f4f6b0ae640 1 -- 172.21.15.63:0/559006040 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f4f5406d2f0 msgr2=0x7f4f5406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:58.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.397+0000 7f4f6b0ae640 1 --2- 172.21.15.63:0/559006040 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f4f5406d2f0 0x7f4f5406f7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4f50000c00 tx=0x7f4f50009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.397+0000 7f4f6b0ae640 1 -- 172.21.15.63:0/559006040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f641012c0 msgr2=0x7f4f64111e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:58.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.397+0000 7f4f6b0ae640 1 --2- 172.21.15.63:0/559006040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f641012c0 0x7f4f64111e10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4f58002a80 tx=0x7f4f58004600 comp rx=0 tx=0).stop 2024-03-22T07:27:58.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.397+0000 7f4f6b0ae640 1 -- 172.21.15.63:0/559006040 shutdown_connections 2024-03-22T07:27:58.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.397+0000 7f4f6b0ae640 1 --2- 172.21.15.63:0/559006040 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f4f5406d2f0 0x7f4f5406f7a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.397+0000 7f4f6b0ae640 1 --2- 172.21.15.63:0/559006040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4f64101c60 0x7f4f64112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.397+0000 7f4f6b0ae640 1 --2- 172.21.15.63:0/559006040 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f4f641012c0 0x7f4f64111e10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.397+0000 7f4f6b0ae640 1 -- 172.21.15.63:0/559006040 >> 172.21.15.63:0/559006040 conn(0x7f4f640f6e90 msgr2=0x7f4f640f7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:58.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.397+0000 7f4f6b0ae640 1 -- 172.21.15.63:0/559006040 shutdown_connections 2024-03-22T07:27:58.398 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.397+0000 7f4f6b0ae640 1 -- 172.21.15.63:0/559006040 wait complete. 2024-03-22T07:27:58.408 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:27:58.408 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-03-22T07:27:58.408 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-03-22T07:27:58.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.507+0000 7fc35f0ae640 1 -- 172.21.15.63:0/4194474202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc358102250 msgr2=0x7fc358102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:58.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.507+0000 7fc35f0ae640 1 --2- 172.21.15.63:0/4194474202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc358102250 0x7fc358102620 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc3500048d0 tx=0x7fc350009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.507+0000 7fc35f0ae640 1 -- 172.21.15.63:0/4194474202 shutdown_connections 2024-03-22T07:27:58.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.507+0000 7fc35f0ae640 1 --2- 172.21.15.63:0/4194474202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc358102bf0 0x7fc3580fc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.507+0000 7fc35f0ae640 1 --2- 172.21.15.63:0/4194474202 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc358102250 0x7fc358102620 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.507+0000 7fc35f0ae640 1 -- 172.21.15.63:0/4194474202 >> 172.21.15.63:0/4194474202 conn(0x7fc3580f7e60 msgr2=0x7fc3580fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:58.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.508+0000 7fc35f0ae640 1 -- 172.21.15.63:0/4194474202 shutdown_connections 2024-03-22T07:27:58.510 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.508+0000 7fc35f0ae640 1 -- 172.21.15.63:0/4194474202 wait complete. 2024-03-22T07:27:58.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.508+0000 7fc35f0ae640 1 Processor -- start 2024-03-22T07:27:58.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.508+0000 7fc35f0ae640 1 -- start start 2024-03-22T07:27:58.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.508+0000 7fc35f0ae640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc358102250 0x7fc358112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:58.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.508+0000 7fc35f0ae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc358102bf0 0x7fc358113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:58.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.508+0000 7fc35f0ae640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc358113820 con 0x7fc358102250 2024-03-22T07:27:58.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.508+0000 7fc35f0ae640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35810cec0 con 0x7fc358102bf0 2024-03-22T07:27:58.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.508+0000 7fc35ce4a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc358102250 0x7fc358112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:58.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.508+0000 7fc35ce4a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc358102250 0x7fc358112ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42474/0 (socket says 172.21.15.63:42474) 2024-03-22T07:27:58.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.508+0000 7fc35ce4a640 1 -- 172.21.15.63:0/3284160370 learned_addr learned my addr 172.21.15.63:0/3284160370 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:58.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.509+0000 7fc357fff640 1 --2- 172.21.15.63:0/3284160370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc358102bf0 0x7fc358113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:58.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.509+0000 7fc35ce4a640 1 -- 172.21.15.63:0/3284160370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc358102bf0 msgr2=0x7fc358113220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:58.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.509+0000 7fc35ce4a640 1 --2- 172.21.15.63:0/3284160370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc358102bf0 0x7fc358113220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.509+0000 7fc35ce4a640 1 -- 172.21.15.63:0/3284160370 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc350005040 con 0x7fc358102250 2024-03-22T07:27:58.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.509+0000 7fc357fff640 1 --2- 172.21.15.63:0/3284160370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc358102bf0 0x7fc358113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:27:58.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.509+0000 7fc35ce4a640 1 --2- 172.21.15.63:0/3284160370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc358102250 0x7fc358112ce0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc350034040 tx=0x7fc350009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:58.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.509+0000 7fc3557fa640 1 -- 172.21.15.63:0/3284160370 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3500092b0 con 0x7fc358102250 2024-03-22T07:27:58.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.509+0000 7fc35f0ae640 1 -- 172.21.15.63:0/3284160370 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35810d140 con 0x7fc358102250 2024-03-22T07:27:58.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.509+0000 7fc3557fa640 1 -- 172.21.15.63:0/3284160370 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc35003b400 con 0x7fc358102250 2024-03-22T07:27:58.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.509+0000 7fc3557fa640 1 -- 172.21.15.63:0/3284160370 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3500435d0 con 0x7fc358102250 2024-03-22T07:27:58.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.509+0000 7fc35f0ae640 1 -- 172.21.15.63:0/3284160370 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35810d600 con 0x7fc358102250 2024-03-22T07:27:58.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.510+0000 7fc35f0ae640 1 -- 172.21.15.63:0/3284160370 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3580fd9c0 con 0x7fc358102250 2024-03-22T07:27:58.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.510+0000 7fc3557fa640 1 -- 172.21.15.63:0/3284160370 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fc35002f070 con 0x7fc358102250 2024-03-22T07:27:58.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.511+0000 7fc3557fa640 1 --2- 172.21.15.63:0/3284160370 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc34406d2f0 0x7fc34406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:58.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.511+0000 7fc357fff640 1 --2- 172.21.15.63:0/3284160370 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc34406d2f0 0x7fc34406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:58.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.511+0000 7fc3557fa640 1 -- 172.21.15.63:0/3284160370 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc3500ae6f0 con 0x7fc358102250 2024-03-22T07:27:58.513 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.511+0000 7fc357fff640 1 --2- 172.21.15.63:0/3284160370 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc34406d2f0 0x7fc34406f7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc3480049a0 tx=0x7fc34800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:58.516 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.516+0000 7fc3557fa640 1 -- 172.21.15.63:0/3284160370 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc350078120 con 0x7fc358102250 2024-03-22T07:27:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:58 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2671508769' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-22T07:27:58.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:58 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/29398492' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-22T07:27:58.734 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.734+0000 7fc35f0ae640 1 -- 172.21.15.63:0/3284160370 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7fc35804bcc0 con 0x7fc358102250 2024-03-22T07:27:58.735 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.735+0000 7fc3557fa640 1 -- 172.21.15.63:0/3284160370 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v44) v1 ==== 107+0+4852 (secure 0 0 0) 0x7fc35007b7e0 con 0x7fc358102250 2024-03-22T07:27:58.735 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:27:58.735 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34388,"name":"cephfs.smithi063.xeelpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/1202791208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1202791208},{"type":"v1","addr":"172.21.15.63:6829","nonce":1202791208}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":34400,"name":"cephfs.smithi063.oytapz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/1171423403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1171423403},{"type":"v1","addr":"172.21.15.63:6827","nonce":1171423403}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44273,"name":"cephfs.smithi069.cijxah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6827/675976207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":675976207},{"type":"v1","addr":"172.21.15.69:6827","nonce":675976207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44279,"name":"cephfs.smithi069.wwtsjn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6825/2785357568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2785357568},{"type":"v1","addr":"172.21.15.69:6825","nonce":2785357568}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:25:00.991999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:27:58.736 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 29 2024-03-22T07:27:58.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.737+0000 7fc35f0ae640 1 -- 172.21.15.63:0/3284160370 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc34406d2f0 msgr2=0x7fc34406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:58.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.737+0000 7fc35f0ae640 1 --2- 172.21.15.63:0/3284160370 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc34406d2f0 0x7fc34406f7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc3480049a0 tx=0x7fc34800b040 comp rx=0 tx=0).stop 2024-03-22T07:27:58.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.737+0000 7fc35f0ae640 1 -- 172.21.15.63:0/3284160370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc358102250 msgr2=0x7fc358112ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:58.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.737+0000 7fc35f0ae640 1 --2- 172.21.15.63:0/3284160370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc358102250 0x7fc358112ce0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc350034040 tx=0x7fc350009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.737+0000 7fc35f0ae640 1 -- 172.21.15.63:0/3284160370 shutdown_connections 2024-03-22T07:27:58.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.737+0000 7fc35f0ae640 1 --2- 172.21.15.63:0/3284160370 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc34406d2f0 0x7fc34406f7a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.737 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.737+0000 7fc35f0ae640 1 --2- 172.21.15.63:0/3284160370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc358102bf0 0x7fc358113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.737+0000 7fc35f0ae640 1 --2- 172.21.15.63:0/3284160370 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc358102250 0x7fc358112ce0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.737+0000 7fc35f0ae640 1 -- 172.21.15.63:0/3284160370 >> 172.21.15.63:0/3284160370 conn(0x7fc3580f7e60 msgr2=0x7fc3580f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:58.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.737+0000 7fc35f0ae640 1 -- 172.21.15.63:0/3284160370 shutdown_connections 2024-03-22T07:27:58.738 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.737+0000 7fc35f0ae640 1 -- 172.21.15.63:0/3284160370 wait complete. 2024-03-22T07:27:58.748 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:27:58.748 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-03-22T07:27:58.748 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-03-22T07:27:58.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.847+0000 7fc724a6e640 1 -- 172.21.15.63:0/4053081638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7201012e0 msgr2=0x7fc7201016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:58.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.847+0000 7fc724a6e640 1 --2- 172.21.15.63:0/4053081638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7201012e0 0x7fc7201016b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc710009c50 tx=0x7fc710004d10 comp rx=0 tx=0).stop 2024-03-22T07:27:58.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.847+0000 7fc724a6e640 1 -- 172.21.15.63:0/4053081638 shutdown_connections 2024-03-22T07:27:58.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.847+0000 7fc724a6e640 1 --2- 172.21.15.63:0/4053081638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc720101c80 0x7fc7200fb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.849 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.847+0000 7fc724a6e640 1 --2- 172.21.15.63:0/4053081638 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7201012e0 0x7fc7201016b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.847+0000 7fc724a6e640 1 -- 172.21.15.63:0/4053081638 >> 172.21.15.63:0/4053081638 conn(0x7fc7200f6e70 msgr2=0x7fc7200f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:58.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.847+0000 7fc724a6e640 1 -- 172.21.15.63:0/4053081638 shutdown_connections 2024-03-22T07:27:58.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.847+0000 7fc724a6e640 1 -- 172.21.15.63:0/4053081638 wait complete. 2024-03-22T07:27:58.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.848+0000 7fc724a6e640 1 Processor -- start 2024-03-22T07:27:58.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.848+0000 7fc724a6e640 1 -- start start 2024-03-22T07:27:58.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.848+0000 7fc724a6e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7201012e0 0x7fc720112fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:58.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.848+0000 7fc724a6e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc720101c80 0x7fc7201134f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:58.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.848+0000 7fc724a6e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc720113b80 con 0x7fc7201012e0 2024-03-22T07:27:58.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.848+0000 7fc724a6e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72010d140 con 0x7fc720101c80 2024-03-22T07:27:58.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.848+0000 7fc71e59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7201012e0 0x7fc720112fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:58.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.848+0000 7fc71e59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7201012e0 0x7fc720112fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42492/0 (socket says 172.21.15.63:42492) 2024-03-22T07:27:58.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.848+0000 7fc71e59c640 1 -- 172.21.15.63:0/110813508 learned_addr learned my addr 172.21.15.63:0/110813508 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:58.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.848+0000 7fc71dd9b640 1 --2- 172.21.15.63:0/110813508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc720101c80 0x7fc7201134f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:58.850 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.848+0000 7fc71e59c640 1 -- 172.21.15.63:0/110813508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc720101c80 msgr2=0x7fc7201134f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:58.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.848+0000 7fc71e59c640 1 --2- 172.21.15.63:0/110813508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc720101c80 0x7fc7201134f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:58.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.848+0000 7fc71e59c640 1 -- 172.21.15.63:0/110813508 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7100098b0 con 0x7fc7201012e0 2024-03-22T07:27:58.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.848+0000 7fc71e59c640 1 --2- 172.21.15.63:0/110813508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7201012e0 0x7fc720112fb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc710004a30 tx=0x7fc710004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:58.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.848+0000 7fc716ffd640 1 -- 172.21.15.63:0/110813508 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc71003a070 con 0x7fc7201012e0 2024-03-22T07:27:58.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.849+0000 7fc716ffd640 1 -- 172.21.15.63:0/110813508 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc710035b20 con 0x7fc7201012e0 2024-03-22T07:27:58.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.849+0000 7fc724a6e640 1 -- 172.21.15.63:0/110813508 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72010d3c0 con 0x7fc7201012e0 2024-03-22T07:27:58.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.849+0000 7fc716ffd640 1 -- 172.21.15.63:0/110813508 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc71003eaa0 con 0x7fc7201012e0 2024-03-22T07:27:58.851 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.849+0000 7fc724a6e640 1 -- 172.21.15.63:0/110813508 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc72010d8e0 con 0x7fc7201012e0 2024-03-22T07:27:58.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.849+0000 7fc724a6e640 1 -- 172.21.15.63:0/110813508 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7200fca50 con 0x7fc7201012e0 2024-03-22T07:27:58.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.850+0000 7fc716ffd640 1 -- 172.21.15.63:0/110813508 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fc710032070 con 0x7fc7201012e0 2024-03-22T07:27:58.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.850+0000 7fc716ffd640 1 --2- 172.21.15.63:0/110813508 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc70c06d2f0 0x7fc70c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:58.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.850+0000 7fc716ffd640 1 -- 172.21.15.63:0/110813508 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc7100af1e0 con 0x7fc7201012e0 2024-03-22T07:27:58.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.850+0000 7fc71dd9b640 1 --2- 172.21.15.63:0/110813508 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc70c06d2f0 0x7fc70c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:58.852 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.851+0000 7fc71dd9b640 1 --2- 172.21.15.63:0/110813508 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc70c06d2f0 0x7fc70c06f7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc708006fd0 tx=0x7fc708008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:58.855 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:58.855+0000 7fc716ffd640 1 -- 172.21.15.63:0/110813508 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc710078c10 con 0x7fc7201012e0 2024-03-22T07:27:59.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.075+0000 7fc724a6e640 1 -- 172.21.15.63:0/110813508 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fc7200638c0 con 0x7fc7201012e0 2024-03-22T07:27:59.075 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.075+0000 7fc716ffd640 1 -- 172.21.15.63:0/110813508 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v44) v1 ==== 107+0+4073 (secure 0 0 0) 0x7fc71007c2d0 con 0x7fc7201012e0 2024-03-22T07:27:59.076 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:27:59.076 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34400,"name":"cephfs.smithi063.oytapz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/1171423403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1171423403},{"type":"v1","addr":"172.21.15.63:6827","nonce":1171423403}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":44273,"name":"cephfs.smithi069.cijxah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6827/675976207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":675976207},{"type":"v1","addr":"172.21.15.69:6827","nonce":675976207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44279,"name":"cephfs.smithi069.wwtsjn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6825/2785357568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2785357568},{"type":"v1","addr":"172.21.15.69:6825","nonce":2785357568}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:25:00.991999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:27:59.076 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 30 2024-03-22T07:27:59.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.077+0000 7fc724a6e640 1 -- 172.21.15.63:0/110813508 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc70c06d2f0 msgr2=0x7fc70c06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:59.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.077+0000 7fc724a6e640 1 --2- 172.21.15.63:0/110813508 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc70c06d2f0 0x7fc70c06f7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc708006fd0 tx=0x7fc708008040 comp rx=0 tx=0).stop 2024-03-22T07:27:59.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.077+0000 7fc724a6e640 1 -- 172.21.15.63:0/110813508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7201012e0 msgr2=0x7fc720112fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:59.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.077+0000 7fc724a6e640 1 --2- 172.21.15.63:0/110813508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7201012e0 0x7fc720112fb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc710004a30 tx=0x7fc710004d10 comp rx=0 tx=0).stop 2024-03-22T07:27:59.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.077+0000 7fc724a6e640 1 -- 172.21.15.63:0/110813508 shutdown_connections 2024-03-22T07:27:59.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.077+0000 7fc724a6e640 1 --2- 172.21.15.63:0/110813508 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fc70c06d2f0 0x7fc70c06f7a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.077+0000 7fc724a6e640 1 --2- 172.21.15.63:0/110813508 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc720101c80 0x7fc7201134f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.077+0000 7fc724a6e640 1 --2- 172.21.15.63:0/110813508 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fc7201012e0 0x7fc720112fb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.078 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.077+0000 7fc724a6e640 1 -- 172.21.15.63:0/110813508 >> 172.21.15.63:0/110813508 conn(0x7fc7200f6e70 msgr2=0x7fc7200f84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:59.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.077+0000 7fc724a6e640 1 -- 172.21.15.63:0/110813508 shutdown_connections 2024-03-22T07:27:59.079 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.078+0000 7fc724a6e640 1 -- 172.21.15.63:0/110813508 wait complete. 2024-03-22T07:27:59.089 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:27:59.089 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-03-22T07:27:59.089 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-03-22T07:27:59.192 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.189+0000 7f60bd26e640 1 -- 172.21.15.63:0/3961245652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b8102b70 msgr2=0x7f60b80fc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:59.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.189+0000 7f60bd26e640 1 --2- 172.21.15.63:0/3961245652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b8102b70 0x7f60b80fc200 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f60a00048d0 tx=0x7f60a0009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.189+0000 7f60bd26e640 1 -- 172.21.15.63:0/3961245652 shutdown_connections 2024-03-22T07:27:59.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.189+0000 7f60bd26e640 1 --2- 172.21.15.63:0/3961245652 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b8102b70 0x7f60b80fc200 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.189+0000 7f60bd26e640 1 --2- 172.21.15.63:0/3961245652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60b81021d0 0x7f60b81025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.189+0000 7f60bd26e640 1 -- 172.21.15.63:0/3961245652 >> 172.21.15.63:0/3961245652 conn(0x7f60b80f7dc0 msgr2=0x7f60b80fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:59.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.189+0000 7f60bd26e640 1 -- 172.21.15.63:0/3961245652 shutdown_connections 2024-03-22T07:27:59.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.189+0000 7f60bd26e640 1 -- 172.21.15.63:0/3961245652 wait complete. 2024-03-22T07:27:59.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.190+0000 7f60bd26e640 1 Processor -- start 2024-03-22T07:27:59.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.190+0000 7f60bd26e640 1 -- start start 2024-03-22T07:27:59.193 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.190+0000 7f60bd26e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60b81021d0 0x7f60b806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:59.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.190+0000 7f60bd26e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b8102b70 0x7f60b806fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:59.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.190+0000 7f60bd26e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b80716b0 con 0x7f60b8102b70 2024-03-22T07:27:59.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.190+0000 7f60bd26e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b8070250 con 0x7f60b81021d0 2024-03-22T07:27:59.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.190+0000 7f60affff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b8102b70 0x7f60b806fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:59.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.190+0000 7f60affff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b8102b70 0x7f60b806fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42516/0 (socket says 172.21.15.63:42516) 2024-03-22T07:27:59.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.190+0000 7f60affff640 1 -- 172.21.15.63:0/4045638458 learned_addr learned my addr 172.21.15.63:0/4045638458 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:59.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.190+0000 7f60b6ffd640 1 --2- 172.21.15.63:0/4045638458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60b81021d0 0x7f60b806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:59.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.191+0000 7f60affff640 1 -- 172.21.15.63:0/4045638458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60b81021d0 msgr2=0x7f60b806f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:59.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.191+0000 7f60affff640 1 --2- 172.21.15.63:0/4045638458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60b81021d0 0x7f60b806f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.191+0000 7f60affff640 1 -- 172.21.15.63:0/4045638458 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a0005040 con 0x7f60b8102b70 2024-03-22T07:27:59.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.191+0000 7f60affff640 1 --2- 172.21.15.63:0/4045638458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b8102b70 0x7f60b806fd10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f60a0004930 tx=0x7f60a0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:59.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.191+0000 7f60af7fe640 1 -- 172.21.15.63:0/4045638458 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60a00094c0 con 0x7f60b8102b70 2024-03-22T07:27:59.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.191+0000 7f60af7fe640 1 -- 172.21.15.63:0/4045638458 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60a0032e50 con 0x7f60b8102b70 2024-03-22T07:27:59.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.191+0000 7f60bd26e640 1 -- 172.21.15.63:0/4045638458 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b80704d0 con 0x7f60b8102b70 2024-03-22T07:27:59.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.191+0000 7f60af7fe640 1 -- 172.21.15.63:0/4045638458 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60a00433b0 con 0x7f60b8102b70 2024-03-22T07:27:59.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.191+0000 7f60bd26e640 1 -- 172.21.15.63:0/4045638458 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b81193d0 con 0x7f60b8102b70 2024-03-22T07:27:59.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.192+0000 7f60bd26e640 1 -- 172.21.15.63:0/4045638458 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b80fd940 con 0x7f60b8102b70 2024-03-22T07:27:59.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.193+0000 7f60af7fe640 1 -- 172.21.15.63:0/4045638458 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f60a002f070 con 0x7f60b8102b70 2024-03-22T07:27:59.195 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.193+0000 7f60af7fe640 1 --2- 172.21.15.63:0/4045638458 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f60a406d2f0 0x7f60a406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:59.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.194+0000 7f60af7fe640 1 -- 172.21.15.63:0/4045638458 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f60a00ae760 con 0x7f60b8102b70 2024-03-22T07:27:59.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.194+0000 7f60b6ffd640 1 --2- 172.21.15.63:0/4045638458 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f60a406d2f0 0x7f60a406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:59.196 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.194+0000 7f60b6ffd640 1 --2- 172.21.15.63:0/4045638458 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f60a406d2f0 0x7f60a406f7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f60a8004eb0 tx=0x7f60a800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:59.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.198+0000 7f60af7fe640 1 -- 172.21.15.63:0/4045638458 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f60a0078240 con 0x7f60b8102b70 2024-03-22T07:27:59.359 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:59 smithi063 ceph-mon[112784]: pgmap v263: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:59.359 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:59 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/1815080136' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-22T07:27:59.359 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:59 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/559006040' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-22T07:27:59.359 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:27:59 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3284160370' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-22T07:27:59.421 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.421+0000 7f60bd26e640 1 -- 172.21.15.63:0/4045638458 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f60b8109370 con 0x7f60b8102b70 2024-03-22T07:27:59.422 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.421+0000 7f60af7fe640 1 -- 172.21.15.63:0/4045638458 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v44) v1 ==== 107+0+4886 (secure 0 0 0) 0x7f60a007b900 con 0x7f60b8102b70 2024-03-22T07:27:59.422 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:27:59.422 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34400,"name":"cephfs.smithi063.oytapz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/1171423403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1171423403},{"type":"v1","addr":"172.21.15.63:6827","nonce":1171423403}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":34424,"name":"cephfs.smithi063.xeelpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/1190308435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1190308435},{"type":"v1","addr":"172.21.15.63:6829","nonce":1190308435}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44273,"name":"cephfs.smithi069.cijxah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6827/675976207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":675976207},{"type":"v1","addr":"172.21.15.69:6827","nonce":675976207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44279,"name":"cephfs.smithi069.wwtsjn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6825/2785357568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2785357568},{"type":"v1","addr":"172.21.15.69:6825","nonce":2785357568}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:25:00.991999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:27:59.422 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 31 2024-03-22T07:27:59.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.423+0000 7f60bd26e640 1 -- 172.21.15.63:0/4045638458 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f60a406d2f0 msgr2=0x7f60a406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:59.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.423+0000 7f60bd26e640 1 --2- 172.21.15.63:0/4045638458 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f60a406d2f0 0x7f60a406f7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f60a8004eb0 tx=0x7f60a800b040 comp rx=0 tx=0).stop 2024-03-22T07:27:59.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.423+0000 7f60bd26e640 1 -- 172.21.15.63:0/4045638458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b8102b70 msgr2=0x7f60b806fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:59.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.423+0000 7f60bd26e640 1 --2- 172.21.15.63:0/4045638458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b8102b70 0x7f60b806fd10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f60a0004930 tx=0x7f60a0009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.424+0000 7f60bd26e640 1 -- 172.21.15.63:0/4045638458 shutdown_connections 2024-03-22T07:27:59.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.424+0000 7f60bd26e640 1 --2- 172.21.15.63:0/4045638458 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f60b8102b70 0x7f60b806fd10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.424+0000 7f60bd26e640 1 --2- 172.21.15.63:0/4045638458 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f60a406d2f0 0x7f60a406f7a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.424+0000 7f60bd26e640 1 --2- 172.21.15.63:0/4045638458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f60b81021d0 0x7f60b806f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.424 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.424+0000 7f60bd26e640 1 -- 172.21.15.63:0/4045638458 >> 172.21.15.63:0/4045638458 conn(0x7f60b80f7dc0 msgr2=0x7f60b80f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:59.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.424+0000 7f60bd26e640 1 -- 172.21.15.63:0/4045638458 shutdown_connections 2024-03-22T07:27:59.425 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.424+0000 7f60bd26e640 1 -- 172.21.15.63:0/4045638458 wait complete. 2024-03-22T07:27:59.435 DEBUG:tasks.fs:max_mds not reduced in epoch 31 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:27:59.435 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-03-22T07:27:59.435 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-03-22T07:27:59.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:59 smithi069 ceph-mon[83631]: pgmap v263: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:27:59.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:59 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/1815080136' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-22T07:27:59.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:59 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/559006040' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-22T07:27:59.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:27:59 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3284160370' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-22T07:27:59.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.537+0000 7f92f035d640 1 -- 172.21.15.63:0/2131888980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92e80fb110 msgr2=0x7f92e80fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:59.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.537+0000 7f92f035d640 1 --2- 172.21.15.63:0/2131888980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92e80fb110 0x7f92e80fb560 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f92d80048d0 tx=0x7f92d8009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.537+0000 7f92f035d640 1 -- 172.21.15.63:0/2131888980 shutdown_connections 2024-03-22T07:27:59.538 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.537+0000 7f92f035d640 1 --2- 172.21.15.63:0/2131888980 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92e80fb110 0x7f92e80fb560 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.537+0000 7f92f035d640 1 --2- 172.21.15.63:0/2131888980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e8101140 0x7f92e8101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.537+0000 7f92f035d640 1 -- 172.21.15.63:0/2131888980 >> 172.21.15.63:0/2131888980 conn(0x7f92e80710b0 msgr2=0x7f92e80714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:59.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.537+0000 7f92f035d640 1 -- 172.21.15.63:0/2131888980 shutdown_connections 2024-03-22T07:27:59.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.537+0000 7f92f035d640 1 -- 172.21.15.63:0/2131888980 wait complete. 2024-03-22T07:27:59.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.538+0000 7f92f035d640 1 Processor -- start 2024-03-22T07:27:59.539 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.538+0000 7f92f035d640 1 -- start start 2024-03-22T07:27:59.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.538+0000 7f92f035d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e80fb110 0x7f92e8111be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:59.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.538+0000 7f92f035d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92e8101140 0x7f92e8112120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:59.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.538+0000 7f92f035d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e8112720 con 0x7f92e8101140 2024-03-22T07:27:59.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.538+0000 7f92f035d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e810bd70 con 0x7f92e80fb110 2024-03-22T07:27:59.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.539+0000 7f92ed8f8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92e8101140 0x7f92e8112120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:59.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.539+0000 7f92ed8f8640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92e8101140 0x7f92e8112120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42540/0 (socket says 172.21.15.63:42540) 2024-03-22T07:27:59.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.539+0000 7f92ed8f8640 1 -- 172.21.15.63:0/2374537326 learned_addr learned my addr 172.21.15.63:0/2374537326 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:59.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.539+0000 7f92ee0f9640 1 --2- 172.21.15.63:0/2374537326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e80fb110 0x7f92e8111be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:59.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.539+0000 7f92ed8f8640 1 -- 172.21.15.63:0/2374537326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e80fb110 msgr2=0x7f92e8111be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:59.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.539+0000 7f92ed8f8640 1 --2- 172.21.15.63:0/2374537326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e80fb110 0x7f92e8111be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.539+0000 7f92ed8f8640 1 -- 172.21.15.63:0/2374537326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d8005040 con 0x7f92e8101140 2024-03-22T07:27:59.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.539+0000 7f92ed8f8640 1 --2- 172.21.15.63:0/2374537326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92e8101140 0x7f92e8112120 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f92d80041d0 tx=0x7f92d8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:59.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.539+0000 7f92deffd640 1 -- 172.21.15.63:0/2374537326 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92d8030870 con 0x7f92e8101140 2024-03-22T07:27:59.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.539+0000 7f92f035d640 1 -- 172.21.15.63:0/2374537326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e810bff0 con 0x7f92e8101140 2024-03-22T07:27:59.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.539+0000 7f92deffd640 1 -- 172.21.15.63:0/2374537326 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92d80092e0 con 0x7f92e8101140 2024-03-22T07:27:59.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.539+0000 7f92deffd640 1 -- 172.21.15.63:0/2374537326 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92d8043400 con 0x7f92e8101140 2024-03-22T07:27:59.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.539+0000 7f92f035d640 1 -- 172.21.15.63:0/2374537326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e810c510 con 0x7f92e8101140 2024-03-22T07:27:59.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.540+0000 7f92f035d640 1 -- 172.21.15.63:0/2374537326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e80fc800 con 0x7f92e8101140 2024-03-22T07:27:59.542 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.541+0000 7f92deffd640 1 -- 172.21.15.63:0/2374537326 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f92d803b420 con 0x7f92e8101140 2024-03-22T07:27:59.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.541+0000 7f92deffd640 1 --2- 172.21.15.63:0/2374537326 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f92d406d2f0 0x7f92d406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:59.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.542+0000 7f92deffd640 1 -- 172.21.15.63:0/2374537326 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f92d80af7f0 con 0x7f92e8101140 2024-03-22T07:27:59.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.542+0000 7f92ee0f9640 1 --2- 172.21.15.63:0/2374537326 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f92d406d2f0 0x7f92d406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:59.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.542+0000 7f92ee0f9640 1 --2- 172.21.15.63:0/2374537326 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f92d406d2f0 0x7f92d406f7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009fd0 tx=0x7f92e4009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:59.546 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.546+0000 7f92deffd640 1 -- 172.21.15.63:0/2374537326 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f92d80792a0 con 0x7f92e8101140 2024-03-22T07:27:59.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.768+0000 7f92f035d640 1 -- 172.21.15.63:0/2374537326 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f92e810c730 con 0x7f92e8101140 2024-03-22T07:27:59.768 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.768+0000 7f92deffd640 1 -- 172.21.15.63:0/2374537326 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v44) v1 ==== 107+0+4107 (secure 0 0 0) 0x7f92d807c960 con 0x7f92e8101140 2024-03-22T07:27:59.769 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:27:59.769 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34400,"name":"cephfs.smithi063.oytapz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/1171423403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1171423403},{"type":"v1","addr":"172.21.15.63:6827","nonce":1171423403}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":34424,"name":"cephfs.smithi063.xeelpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/1190308435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1190308435},{"type":"v1","addr":"172.21.15.63:6829","nonce":1190308435}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44273,"name":"cephfs.smithi069.cijxah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6827/675976207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":675976207},{"type":"v1","addr":"172.21.15.69:6827","nonce":675976207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:25:00.991999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:27:59.770 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 32 2024-03-22T07:27:59.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.770+0000 7f92f035d640 1 -- 172.21.15.63:0/2374537326 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f92d406d2f0 msgr2=0x7f92d406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:59.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.770+0000 7f92f035d640 1 --2- 172.21.15.63:0/2374537326 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f92d406d2f0 0x7f92d406f7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009fd0 tx=0x7f92e4009e50 comp rx=0 tx=0).stop 2024-03-22T07:27:59.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.770+0000 7f92f035d640 1 -- 172.21.15.63:0/2374537326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92e8101140 msgr2=0x7f92e8112120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:59.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.770+0000 7f92f035d640 1 --2- 172.21.15.63:0/2374537326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92e8101140 0x7f92e8112120 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f92d80041d0 tx=0x7f92d8009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.770+0000 7f92f035d640 1 -- 172.21.15.63:0/2374537326 shutdown_connections 2024-03-22T07:27:59.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.770+0000 7f92f035d640 1 --2- 172.21.15.63:0/2374537326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f92e8101140 0x7f92e8112120 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.770+0000 7f92f035d640 1 --2- 172.21.15.63:0/2374537326 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f92d406d2f0 0x7f92d406f7a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.770+0000 7f92f035d640 1 --2- 172.21.15.63:0/2374537326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e80fb110 0x7f92e8111be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.770+0000 7f92f035d640 1 -- 172.21.15.63:0/2374537326 >> 172.21.15.63:0/2374537326 conn(0x7f92e80710b0 msgr2=0x7f92e80f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:59.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.771+0000 7f92f035d640 1 -- 172.21.15.63:0/2374537326 shutdown_connections 2024-03-22T07:27:59.771 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.771+0000 7f92f035d640 1 -- 172.21.15.63:0/2374537326 wait complete. 2024-03-22T07:27:59.782 DEBUG:tasks.fs:max_mds not reduced in epoch 32 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:27:59.782 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-03-22T07:27:59.782 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-03-22T07:27:59.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.881+0000 7f2662aae640 1 -- 172.21.15.63:0/2607213007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f265c102250 msgr2=0x7f265c102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:59.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.882+0000 7f2662aae640 1 --2- 172.21.15.63:0/2607213007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f265c102250 0x7f265c102620 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f26500048d0 tx=0x7f2650009da0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.882+0000 7f2662aae640 1 -- 172.21.15.63:0/2607213007 shutdown_connections 2024-03-22T07:27:59.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.882+0000 7f2662aae640 1 --2- 172.21.15.63:0/2607213007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f265c102bf0 0x7f265c0fc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.882+0000 7f2662aae640 1 --2- 172.21.15.63:0/2607213007 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f265c102250 0x7f265c102620 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.882+0000 7f2662aae640 1 -- 172.21.15.63:0/2607213007 >> 172.21.15.63:0/2607213007 conn(0x7f265c0f7e60 msgr2=0x7f265c0fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:27:59.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.882+0000 7f2662aae640 1 -- 172.21.15.63:0/2607213007 shutdown_connections 2024-03-22T07:27:59.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.882+0000 7f2662aae640 1 -- 172.21.15.63:0/2607213007 wait complete. 2024-03-22T07:27:59.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.882+0000 7f2662aae640 1 Processor -- start 2024-03-22T07:27:59.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.882+0000 7f2662aae640 1 -- start start 2024-03-22T07:27:59.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.882+0000 7f2662aae640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f265c102250 0x7f265c06f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:59.884 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.882+0000 7f2662aae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f265c102bf0 0x7f265c06fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.883+0000 7f2662aae640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c0716b0 con 0x7f265c102250 2024-03-22T07:27:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.883+0000 7f2662aae640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c070250 con 0x7f265c102bf0 2024-03-22T07:27:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.883+0000 7f266084a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f265c102250 0x7f265c06f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.883+0000 7f266084a640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f265c102250 0x7f265c06f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42562/0 (socket says 172.21.15.63:42562) 2024-03-22T07:27:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.883+0000 7f266084a640 1 -- 172.21.15.63:0/3975500311 learned_addr learned my addr 172.21.15.63:0/3975500311 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:27:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.883+0000 7f265bfff640 1 --2- 172.21.15.63:0/3975500311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f265c102bf0 0x7f265c06fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.883+0000 7f266084a640 1 -- 172.21.15.63:0/3975500311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f265c102bf0 msgr2=0x7f265c06fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:27:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.883+0000 7f266084a640 1 --2- 172.21.15.63:0/3975500311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f265c102bf0 0x7f265c06fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:27:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.883+0000 7f266084a640 1 -- 172.21.15.63:0/3975500311 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2650005040 con 0x7f265c102250 2024-03-22T07:27:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.883+0000 7f266084a640 1 --2- 172.21.15.63:0/3975500311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f265c102250 0x7f265c06f7d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2650004930 tx=0x7f2650009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.883+0000 7f26597fa640 1 -- 172.21.15.63:0/3975500311 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26500094c0 con 0x7f265c102250 2024-03-22T07:27:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.883+0000 7f26597fa640 1 -- 172.21.15.63:0/3975500311 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2650032e50 con 0x7f265c102250 2024-03-22T07:27:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.883+0000 7f2662aae640 1 -- 172.21.15.63:0/3975500311 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265c0704d0 con 0x7f265c102250 2024-03-22T07:27:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.883+0000 7f26597fa640 1 -- 172.21.15.63:0/3975500311 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26500433b0 con 0x7f265c102250 2024-03-22T07:27:59.885 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.883+0000 7f2662aae640 1 -- 172.21.15.63:0/3975500311 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265c119380 con 0x7f265c102250 2024-03-22T07:27:59.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.884+0000 7f2662aae640 1 -- 172.21.15.63:0/3975500311 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f265c0fd9c0 con 0x7f265c102250 2024-03-22T07:27:59.886 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.885+0000 7f26597fa640 1 -- 172.21.15.63:0/3975500311 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f265002f070 con 0x7f265c102250 2024-03-22T07:27:59.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.885+0000 7f26597fa640 1 --2- 172.21.15.63:0/3975500311 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f264c06d2f0 0x7f264c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:27:59.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.885+0000 7f265bfff640 1 --2- 172.21.15.63:0/3975500311 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f264c06d2f0 0x7f264c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:27:59.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.885+0000 7f26597fa640 1 -- 172.21.15.63:0/3975500311 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f26500ae7a0 con 0x7f265c102250 2024-03-22T07:27:59.887 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.886+0000 7f265bfff640 1 --2- 172.21.15.63:0/3975500311 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f264c06d2f0 0x7f264c06f7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f265c070f00 tx=0x7f264800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:27:59.890 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:27:59.890+0000 7f26597fa640 1 -- 172.21.15.63:0/3975500311 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2650078280 con 0x7f265c102250 2024-03-22T07:28:00.110 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.110+0000 7f2662aae640 1 -- 172.21.15.63:0/3975500311 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f265c0ff4f0 con 0x7f265c102250 2024-03-22T07:28:00.111 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.111+0000 7f26597fa640 1 -- 172.21.15.63:0/3975500311 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v44) v1 ==== 107+0+4917 (secure 0 0 0) 0x7f265007b940 con 0x7f265c102250 2024-03-22T07:28:00.111 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:28:00.111 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34400,"name":"cephfs.smithi063.oytapz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/1171423403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1171423403},{"type":"v1","addr":"172.21.15.63:6827","nonce":1171423403}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":34424,"name":"cephfs.smithi063.xeelpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/1190308435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1190308435},{"type":"v1","addr":"172.21.15.63:6829","nonce":1190308435}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44273,"name":"cephfs.smithi069.cijxah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6827/675976207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":675976207},{"type":"v1","addr":"172.21.15.69:6827","nonce":675976207}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44323,"name":"cephfs.smithi069.wwtsjn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6825/653947973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":653947973},{"type":"v1","addr":"172.21.15.69:6825","nonce":653947973}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:25:00.991999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:28:00.111 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 33 2024-03-22T07:28:00.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.113+0000 7f2662aae640 1 -- 172.21.15.63:0/3975500311 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f264c06d2f0 msgr2=0x7f264c06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:00.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.113+0000 7f2662aae640 1 --2- 172.21.15.63:0/3975500311 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f264c06d2f0 0x7f264c06f7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f265c070f00 tx=0x7f264800b040 comp rx=0 tx=0).stop 2024-03-22T07:28:00.113 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.113+0000 7f2662aae640 1 -- 172.21.15.63:0/3975500311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f265c102250 msgr2=0x7f265c06f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:00.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.113+0000 7f2662aae640 1 --2- 172.21.15.63:0/3975500311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f265c102250 0x7f265c06f7d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2650004930 tx=0x7f2650009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.113+0000 7f2662aae640 1 -- 172.21.15.63:0/3975500311 shutdown_connections 2024-03-22T07:28:00.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.113+0000 7f2662aae640 1 --2- 172.21.15.63:0/3975500311 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f264c06d2f0 0x7f264c06f7a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.113+0000 7f2662aae640 1 --2- 172.21.15.63:0/3975500311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f265c102bf0 0x7f265c06fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.113+0000 7f2662aae640 1 --2- 172.21.15.63:0/3975500311 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f265c102250 0x7f265c06f7d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.113+0000 7f2662aae640 1 -- 172.21.15.63:0/3975500311 >> 172.21.15.63:0/3975500311 conn(0x7f265c0f7e60 msgr2=0x7f265c0f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:00.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.113+0000 7f2662aae640 1 -- 172.21.15.63:0/3975500311 shutdown_connections 2024-03-22T07:28:00.114 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.113+0000 7f2662aae640 1 -- 172.21.15.63:0/3975500311 wait complete. 2024-03-22T07:28:00.124 DEBUG:tasks.fs:max_mds not reduced in epoch 33 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:28:00.124 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-03-22T07:28:00.124 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-03-22T07:28:00.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.224+0000 7f3943d5d640 1 -- 172.21.15.63:0/3858527791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f393c0fb110 msgr2=0x7f393c0fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:00.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.224+0000 7f3943d5d640 1 --2- 172.21.15.63:0/3858527791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f393c0fb110 0x7f393c0fb560 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f392c0048d0 tx=0x7f392c009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.224+0000 7f3943d5d640 1 -- 172.21.15.63:0/3858527791 shutdown_connections 2024-03-22T07:28:00.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.224+0000 7f3943d5d640 1 --2- 172.21.15.63:0/3858527791 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f393c0fb110 0x7f393c0fb560 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.224+0000 7f3943d5d640 1 --2- 172.21.15.63:0/3858527791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f393c101140 0x7f393c101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.224+0000 7f3943d5d640 1 -- 172.21.15.63:0/3858527791 >> 172.21.15.63:0/3858527791 conn(0x7f393c0710b0 msgr2=0x7f393c0714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:00.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.224+0000 7f3943d5d640 1 -- 172.21.15.63:0/3858527791 shutdown_connections 2024-03-22T07:28:00.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.224+0000 7f3943d5d640 1 -- 172.21.15.63:0/3858527791 wait complete. 2024-03-22T07:28:00.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.225+0000 7f3943d5d640 1 Processor -- start 2024-03-22T07:28:00.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.225+0000 7f3943d5d640 1 -- start start 2024-03-22T07:28:00.226 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.225+0000 7f3943d5d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f393c0fb110 0x7f393c111be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:00.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.225+0000 7f3943d5d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f393c101140 0x7f393c112120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:00.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.225+0000 7f3943d5d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c112720 con 0x7f393c0fb110 2024-03-22T07:28:00.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.225+0000 7f3943d5d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c10bd70 con 0x7f393c101140 2024-03-22T07:28:00.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.225+0000 7f3941af9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f393c0fb110 0x7f393c111be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:00.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.225+0000 7f3941af9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f393c0fb110 0x7f393c111be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42590/0 (socket says 172.21.15.63:42590) 2024-03-22T07:28:00.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.225+0000 7f3941af9640 1 -- 172.21.15.63:0/3066193431 learned_addr learned my addr 172.21.15.63:0/3066193431 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:00.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.225+0000 7f3941af9640 1 -- 172.21.15.63:0/3066193431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f393c101140 msgr2=0x7f393c112120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:00.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.225+0000 7f3941af9640 1 --2- 172.21.15.63:0/3066193431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f393c101140 0x7f393c112120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.225+0000 7f3941af9640 1 -- 172.21.15.63:0/3066193431 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c005040 con 0x7f393c0fb110 2024-03-22T07:28:00.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.225+0000 7f3941af9640 1 --2- 172.21.15.63:0/3066193431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f393c0fb110 0x7f393c111be0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3938002a50 tx=0x7f3938004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:00.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.226+0000 7f39327fc640 1 -- 172.21.15.63:0/3066193431 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3938007c30 con 0x7f393c0fb110 2024-03-22T07:28:00.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.226+0000 7f39327fc640 1 -- 172.21.15.63:0/3066193431 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f393800f040 con 0x7f393c0fb110 2024-03-22T07:28:00.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.226+0000 7f3943d5d640 1 -- 172.21.15.63:0/3066193431 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393c10c050 con 0x7f393c0fb110 2024-03-22T07:28:00.227 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.226+0000 7f39327fc640 1 -- 172.21.15.63:0/3066193431 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f393800dbe0 con 0x7f393c0fb110 2024-03-22T07:28:00.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.226+0000 7f3943d5d640 1 -- 172.21.15.63:0/3066193431 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393c10c570 con 0x7f393c0fb110 2024-03-22T07:28:00.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.227+0000 7f3943d5d640 1 -- 172.21.15.63:0/3066193431 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f393c0fc800 con 0x7f393c0fb110 2024-03-22T07:28:00.228 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.227+0000 7f39327fc640 1 -- 172.21.15.63:0/3066193431 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f393800dd40 con 0x7f393c0fb110 2024-03-22T07:28:00.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.228+0000 7f39327fc640 1 --2- 172.21.15.63:0/3066193431 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f392806d2f0 0x7f392806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:00.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.228+0000 7f39412f8640 1 --2- 172.21.15.63:0/3066193431 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f392806d2f0 0x7f392806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:00.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.228+0000 7f39327fc640 1 -- 172.21.15.63:0/3066193431 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f393808c960 con 0x7f393c0fb110 2024-03-22T07:28:00.229 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.228+0000 7f39412f8640 1 --2- 172.21.15.63:0/3066193431 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f392806d2f0 0x7f392806f7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f392c009990 tx=0x7f392c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:00.232 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.232+0000 7f39327fc640 1 -- 172.21.15.63:0/3066193431 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3938056390 con 0x7f393c0fb110 2024-03-22T07:28:00.339 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:00 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/110813508' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-22T07:28:00.339 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:00 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/4045638458' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-22T07:28:00.339 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:00 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2374537326' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-22T07:28:00.452 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.453+0000 7f3943d5d640 1 -- 172.21.15.63:0/3066193431 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f393c04bcc0 con 0x7f393c0fb110 2024-03-22T07:28:00.453 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.453+0000 7f39327fc640 1 -- 172.21.15.63:0/3066193431 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v44) v1 ==== 107+0+4141 (secure 0 0 0) 0x7f393c04bcc0 con 0x7f393c0fb110 2024-03-22T07:28:00.453 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:28:00.453 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34400,"name":"cephfs.smithi063.oytapz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/1171423403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1171423403},{"type":"v1","addr":"172.21.15.63:6827","nonce":1171423403}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":34424,"name":"cephfs.smithi063.xeelpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/1190308435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1190308435},{"type":"v1","addr":"172.21.15.63:6829","nonce":1190308435}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44323,"name":"cephfs.smithi069.wwtsjn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6825/653947973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":653947973},{"type":"v1","addr":"172.21.15.69:6825","nonce":653947973}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:25:00.991999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:28:00.454 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 34 2024-03-22T07:28:00.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.455+0000 7f3943d5d640 1 -- 172.21.15.63:0/3066193431 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f392806d2f0 msgr2=0x7f392806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:00.455 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.455+0000 7f3943d5d640 1 --2- 172.21.15.63:0/3066193431 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f392806d2f0 0x7f392806f7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f392c009990 tx=0x7f392c009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.455+0000 7f3943d5d640 1 -- 172.21.15.63:0/3066193431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f393c0fb110 msgr2=0x7f393c111be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:00.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.455+0000 7f3943d5d640 1 --2- 172.21.15.63:0/3066193431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f393c0fb110 0x7f393c111be0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3938002a50 tx=0x7f3938004600 comp rx=0 tx=0).stop 2024-03-22T07:28:00.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.455+0000 7f3943d5d640 1 -- 172.21.15.63:0/3066193431 shutdown_connections 2024-03-22T07:28:00.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.455+0000 7f3943d5d640 1 --2- 172.21.15.63:0/3066193431 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f392806d2f0 0x7f392806f7a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.455+0000 7f3943d5d640 1 --2- 172.21.15.63:0/3066193431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f393c101140 0x7f393c112120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.455+0000 7f3943d5d640 1 --2- 172.21.15.63:0/3066193431 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f393c0fb110 0x7f393c111be0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.455+0000 7f3943d5d640 1 -- 172.21.15.63:0/3066193431 >> 172.21.15.63:0/3066193431 conn(0x7f393c0710b0 msgr2=0x7f393c0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:00.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.455+0000 7f3943d5d640 1 -- 172.21.15.63:0/3066193431 shutdown_connections 2024-03-22T07:28:00.456 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.455+0000 7f3943d5d640 1 -- 172.21.15.63:0/3066193431 wait complete. 2024-03-22T07:28:00.466 DEBUG:tasks.fs:max_mds not reduced in epoch 34 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:28:00.466 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-03-22T07:28:00.466 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-03-22T07:28:00.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:00 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/110813508' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-22T07:28:00.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:00 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/4045638458' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-22T07:28:00.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:00 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2374537326' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-22T07:28:00.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.565+0000 7fdc976ae640 1 -- 172.21.15.63:0/2345008172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc90101140 msgr2=0x7fdc90101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:00.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.565+0000 7fdc976ae640 1 --2- 172.21.15.63:0/2345008172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc90101140 0x7fdc90101510 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009c50 tx=0x7fdc88004d10 comp rx=0 tx=0).stop 2024-03-22T07:28:00.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.565+0000 7fdc976ae640 1 -- 172.21.15.63:0/2345008172 shutdown_connections 2024-03-22T07:28:00.567 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.565+0000 7fdc976ae640 1 --2- 172.21.15.63:0/2345008172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc900fb110 0x7fdc900fb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.565+0000 7fdc976ae640 1 --2- 172.21.15.63:0/2345008172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc90101140 0x7fdc90101510 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.565+0000 7fdc976ae640 1 -- 172.21.15.63:0/2345008172 >> 172.21.15.63:0/2345008172 conn(0x7fdc900710b0 msgr2=0x7fdc900714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.565+0000 7fdc976ae640 1 -- 172.21.15.63:0/2345008172 shutdown_connections 2024-03-22T07:28:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.565+0000 7fdc976ae640 1 -- 172.21.15.63:0/2345008172 wait complete. 2024-03-22T07:28:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.566+0000 7fdc976ae640 1 Processor -- start 2024-03-22T07:28:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.566+0000 7fdc976ae640 1 -- start start 2024-03-22T07:28:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.566+0000 7fdc976ae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc900fb110 0x7fdc9010de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.566+0000 7fdc976ae640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc90101140 0x7fdc9010e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.566+0000 7fdc976ae640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc90065310 con 0x7fdc90101140 2024-03-22T07:28:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.566+0000 7fdc976ae640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9010e900 con 0x7fdc900fb110 2024-03-22T07:28:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.566+0000 7fdc94c49640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc90101140 0x7fdc9010e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.566+0000 7fdc94c49640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc90101140 0x7fdc9010e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42608/0 (socket says 172.21.15.63:42608) 2024-03-22T07:28:00.568 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.566+0000 7fdc94c49640 1 -- 172.21.15.63:0/1809974925 learned_addr learned my addr 172.21.15.63:0/1809974925 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:00.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.566+0000 7fdc9544a640 1 --2- 172.21.15.63:0/1809974925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc900fb110 0x7fdc9010de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:00.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.567+0000 7fdc94c49640 1 -- 172.21.15.63:0/1809974925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc900fb110 msgr2=0x7fdc9010de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:00.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.567+0000 7fdc94c49640 1 --2- 172.21.15.63:0/1809974925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc900fb110 0x7fdc9010de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.567+0000 7fdc94c49640 1 -- 172.21.15.63:0/1809974925 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc880098b0 con 0x7fdc90101140 2024-03-22T07:28:00.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.567+0000 7fdc9544a640 1 --2- 172.21.15.63:0/1809974925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc900fb110 0x7fdc9010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:28:00.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.567+0000 7fdc94c49640 1 --2- 172.21.15.63:0/1809974925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc90101140 0x7fdc9010e3c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400c830 tx=0x7fdc84004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:00.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.567+0000 7fdc8dffb640 1 -- 172.21.15.63:0/1809974925 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc8400a850 con 0x7fdc90101140 2024-03-22T07:28:00.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.567+0000 7fdc8dffb640 1 -- 172.21.15.63:0/1809974925 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc8400ae80 con 0x7fdc90101140 2024-03-22T07:28:00.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.567+0000 7fdc976ae640 1 -- 172.21.15.63:0/1809974925 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9010ebe0 con 0x7fdc90101140 2024-03-22T07:28:00.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.567+0000 7fdc8dffb640 1 -- 172.21.15.63:0/1809974925 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc84012ea0 con 0x7fdc90101140 2024-03-22T07:28:00.569 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.567+0000 7fdc976ae640 1 -- 172.21.15.63:0/1809974925 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc90117220 con 0x7fdc90101140 2024-03-22T07:28:00.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.568+0000 7fdc976ae640 1 -- 172.21.15.63:0/1809974925 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc900fc800 con 0x7fdc90101140 2024-03-22T07:28:00.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.569+0000 7fdc8dffb640 1 -- 172.21.15.63:0/1809974925 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fdc84012400 con 0x7fdc90101140 2024-03-22T07:28:00.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.569+0000 7fdc8dffb640 1 --2- 172.21.15.63:0/1809974925 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdc7806d2f0 0x7fdc7806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:00.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.569+0000 7fdc8dffb640 1 -- 172.21.15.63:0/1809974925 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdc8408cbd0 con 0x7fdc90101140 2024-03-22T07:28:00.570 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.569+0000 7fdc9544a640 1 --2- 172.21.15.63:0/1809974925 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdc7806d2f0 0x7fdc7806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:00.571 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.569+0000 7fdc9544a640 1 --2- 172.21.15.63:0/1809974925 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdc7806d2f0 0x7fdc7806f7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdc88000f80 tx=0x7fdc8802e040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:00.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.573+0000 7fdc8dffb640 1 -- 172.21.15.63:0/1809974925 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdc84056680 con 0x7fdc90101140 2024-03-22T07:28:00.792 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.792+0000 7fdc976ae640 1 -- 172.21.15.63:0/1809974925 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7fdc9004bcc0 con 0x7fdc90101140 2024-03-22T07:28:00.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.793+0000 7fdc8dffb640 1 -- 172.21.15.63:0/1809974925 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v44) v1 ==== 107+0+4954 (secure 0 0 0) 0x7fdc840126b0 con 0x7fdc90101140 2024-03-22T07:28:00.793 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:28:00.794 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34400,"name":"cephfs.smithi063.oytapz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/1171423403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1171423403},{"type":"v1","addr":"172.21.15.63:6827","nonce":1171423403}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":34424,"name":"cephfs.smithi063.xeelpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/1190308435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1190308435},{"type":"v1","addr":"172.21.15.63:6829","nonce":1190308435}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44323,"name":"cephfs.smithi069.wwtsjn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6825/653947973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":653947973},{"type":"v1","addr":"172.21.15.69:6825","nonce":653947973}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":44327,"name":"cephfs.smithi069.cijxah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6827/3465454734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3465454734},{"type":"v1","addr":"172.21.15.69:6827","nonce":3465454734}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":26,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:25:00.991999+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:28:00.794 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 35 2024-03-22T07:28:00.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.795+0000 7fdc976ae640 1 -- 172.21.15.63:0/1809974925 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdc7806d2f0 msgr2=0x7fdc7806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:00.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.795+0000 7fdc976ae640 1 --2- 172.21.15.63:0/1809974925 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdc7806d2f0 0x7fdc7806f7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdc88000f80 tx=0x7fdc8802e040 comp rx=0 tx=0).stop 2024-03-22T07:28:00.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.795+0000 7fdc976ae640 1 -- 172.21.15.63:0/1809974925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc90101140 msgr2=0x7fdc9010e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:00.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.795+0000 7fdc976ae640 1 --2- 172.21.15.63:0/1809974925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc90101140 0x7fdc9010e3c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400c830 tx=0x7fdc84004430 comp rx=0 tx=0).stop 2024-03-22T07:28:00.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.795+0000 7fdc976ae640 1 -- 172.21.15.63:0/1809974925 shutdown_connections 2024-03-22T07:28:00.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.795+0000 7fdc976ae640 1 --2- 172.21.15.63:0/1809974925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdc90101140 0x7fdc9010e3c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.795+0000 7fdc976ae640 1 --2- 172.21.15.63:0/1809974925 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdc7806d2f0 0x7fdc7806f7a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.795+0000 7fdc976ae640 1 --2- 172.21.15.63:0/1809974925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc900fb110 0x7fdc9010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.795+0000 7fdc976ae640 1 -- 172.21.15.63:0/1809974925 >> 172.21.15.63:0/1809974925 conn(0x7fdc900710b0 msgr2=0x7fdc900f9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:00.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.795+0000 7fdc976ae640 1 -- 172.21.15.63:0/1809974925 shutdown_connections 2024-03-22T07:28:00.796 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.795+0000 7fdc976ae640 1 -- 172.21.15.63:0/1809974925 wait complete. 2024-03-22T07:28:00.806 DEBUG:tasks.fs:max_mds not reduced in epoch 35 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-03-22T07:28:00.806 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 35 2024-03-22T07:28:00.806 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-03-22T07:28:00.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.905+0000 7f13ba95d640 1 -- 172.21.15.63:0/956057172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b4102b90 msgr2=0x7f13b40fc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:00.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.905+0000 7f13ba95d640 1 --2- 172.21.15.63:0/956057172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b4102b90 0x7f13b40fc220 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f13a40048d0 tx=0x7f13a4009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.905+0000 7f13ba95d640 1 -- 172.21.15.63:0/956057172 shutdown_connections 2024-03-22T07:28:00.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.905+0000 7f13ba95d640 1 --2- 172.21.15.63:0/956057172 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b4102b90 0x7f13b40fc220 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.905+0000 7f13ba95d640 1 --2- 172.21.15.63:0/956057172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13b41021f0 0x7f13b41025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.905+0000 7f13ba95d640 1 -- 172.21.15.63:0/956057172 >> 172.21.15.63:0/956057172 conn(0x7f13b40f7dc0 msgr2=0x7f13b40fa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:00.907 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.905+0000 7f13ba95d640 1 -- 172.21.15.63:0/956057172 shutdown_connections 2024-03-22T07:28:00.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.905+0000 7f13ba95d640 1 -- 172.21.15.63:0/956057172 wait complete. 2024-03-22T07:28:00.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.906+0000 7f13ba95d640 1 Processor -- start 2024-03-22T07:28:00.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.906+0000 7f13ba95d640 1 -- start start 2024-03-22T07:28:00.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.906+0000 7f13ba95d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13b41021f0 0x7f13b4114ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:00.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.906+0000 7f13ba95d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b4102b90 0x7f13b4115420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:00.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.906+0000 7f13ba95d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b4115a20 con 0x7f13b4102b90 2024-03-22T07:28:00.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.906+0000 7f13ba95d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b4115b90 con 0x7f13b41021f0 2024-03-22T07:28:00.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.906+0000 7f13b37fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b4102b90 0x7f13b4115420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:00.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.906+0000 7f13b37fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b4102b90 0x7f13b4115420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42624/0 (socket says 172.21.15.63:42624) 2024-03-22T07:28:00.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.906+0000 7f13b37fe640 1 -- 172.21.15.63:0/2884118950 learned_addr learned my addr 172.21.15.63:0/2884118950 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:00.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.906+0000 7f13b3fff640 1 --2- 172.21.15.63:0/2884118950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13b41021f0 0x7f13b4114ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:00.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.906+0000 7f13b37fe640 1 -- 172.21.15.63:0/2884118950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13b41021f0 msgr2=0x7f13b4114ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:00.908 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.906+0000 7f13b37fe640 1 --2- 172.21.15.63:0/2884118950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13b41021f0 0x7f13b4114ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:00.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.906+0000 7f13b37fe640 1 -- 172.21.15.63:0/2884118950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a4005040 con 0x7f13b4102b90 2024-03-22T07:28:00.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.906+0000 7f13b3fff640 1 --2- 172.21.15.63:0/2884118950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13b41021f0 0x7f13b4114ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:28:00.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.907+0000 7f13b37fe640 1 --2- 172.21.15.63:0/2884118950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b4102b90 0x7f13b4115420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f13a4004930 tx=0x7f13a4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:00.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.907+0000 7f13b0ff9640 1 -- 172.21.15.63:0/2884118950 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13a40094c0 con 0x7f13b4102b90 2024-03-22T07:28:00.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.907+0000 7f13b0ff9640 1 -- 172.21.15.63:0/2884118950 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13a4032e50 con 0x7f13b4102b90 2024-03-22T07:28:00.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.907+0000 7f13ba95d640 1 -- 172.21.15.63:0/2884118950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b410f1c0 con 0x7f13b4102b90 2024-03-22T07:28:00.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.907+0000 7f13b0ff9640 1 -- 172.21.15.63:0/2884118950 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13a40433b0 con 0x7f13b4102b90 2024-03-22T07:28:00.909 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.907+0000 7f13ba95d640 1 -- 172.21.15.63:0/2884118950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b410f680 con 0x7f13b4102b90 2024-03-22T07:28:00.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.908+0000 7f13ba95d640 1 -- 172.21.15.63:0/2884118950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b40fd960 con 0x7f13b4102b90 2024-03-22T07:28:00.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.908+0000 7f13b0ff9640 1 -- 172.21.15.63:0/2884118950 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f13a402f070 con 0x7f13b4102b90 2024-03-22T07:28:00.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.908+0000 7f13b0ff9640 1 --2- 172.21.15.63:0/2884118950 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f13a006d2f0 0x7f13a006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:00.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.909+0000 7f13b0ff9640 1 -- 172.21.15.63:0/2884118950 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f13a40ae7c0 con 0x7f13b4102b90 2024-03-22T07:28:00.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.909+0000 7f13b3fff640 1 --2- 172.21.15.63:0/2884118950 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f13a006d2f0 0x7f13a006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:00.910 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.909+0000 7f13b3fff640 1 --2- 172.21.15.63:0/2884118950 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f13a006d2f0 0x7f13a006f7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f139c009af0 tx=0x7f139c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:00.913 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:00.913+0000 7f13b0ff9640 1 -- 172.21.15.63:0/2884118950 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f13a40781f0 con 0x7f13b4102b90 2024-03-22T07:28:01.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:01 smithi063 ceph-mon[112784]: pgmap v264: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:28:01.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:01 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3975500311' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-22T07:28:01.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:01 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3066193431' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-22T07:28:01.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:01 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/1809974925' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-22T07:28:01.135 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.135+0000 7f13ba95d640 1 -- 172.21.15.63:0/2884118950 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f13b404bcc0 con 0x7f13b4102b90 2024-03-22T07:28:01.136 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.136+0000 7f13b0ff9640 1 -- 172.21.15.63:0/2884118950 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v44) v1 ==== 107+0+4953 (secure 0 0 0) 0x7f13a407b8b0 con 0x7f13b4102b90 2024-03-22T07:28:01.136 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:28:01.136 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34400,"name":"cephfs.smithi063.oytapz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6827/1171423403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1171423403},{"type":"v1","addr":"172.21.15.63:6827","nonce":1171423403}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":34424,"name":"cephfs.smithi063.xeelpk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.63:6829/1190308435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1190308435},{"type":"v1","addr":"172.21.15.63:6829","nonce":1190308435}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31},{"gid":44323,"name":"cephfs.smithi069.wwtsjn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6825/653947973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":653947973},{"type":"v1","addr":"172.21.15.69:6825","nonce":653947973}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":44327,"name":"cephfs.smithi069.cijxah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6827/3465454734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3465454734},{"type":"v1","addr":"172.21.15.69:6827","nonce":3465454734}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:26:09.357956+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:28:01.137 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 36 2024-03-22T07:28:01.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.138+0000 7f13ba95d640 1 -- 172.21.15.63:0/2884118950 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f13a006d2f0 msgr2=0x7f13a006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:01.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.138+0000 7f13ba95d640 1 --2- 172.21.15.63:0/2884118950 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f13a006d2f0 0x7f13a006f7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f139c009af0 tx=0x7f139c00b040 comp rx=0 tx=0).stop 2024-03-22T07:28:01.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.138+0000 7f13ba95d640 1 -- 172.21.15.63:0/2884118950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b4102b90 msgr2=0x7f13b4115420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:01.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.138+0000 7f13ba95d640 1 --2- 172.21.15.63:0/2884118950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b4102b90 0x7f13b4115420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f13a4004930 tx=0x7f13a4009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.138+0000 7f13ba95d640 1 -- 172.21.15.63:0/2884118950 shutdown_connections 2024-03-22T07:28:01.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.138+0000 7f13ba95d640 1 --2- 172.21.15.63:0/2884118950 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f13b4102b90 0x7f13b4115420 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.138+0000 7f13ba95d640 1 --2- 172.21.15.63:0/2884118950 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f13a006d2f0 0x7f13a006f7a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.138+0000 7f13ba95d640 1 --2- 172.21.15.63:0/2884118950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13b41021f0 0x7f13b4114ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.138+0000 7f13ba95d640 1 -- 172.21.15.63:0/2884118950 >> 172.21.15.63:0/2884118950 conn(0x7f13b40f7dc0 msgr2=0x7f13b40710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:01.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.138+0000 7f13ba95d640 1 -- 172.21.15.63:0/2884118950 shutdown_connections 2024-03-22T07:28:01.139 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.138+0000 7f13ba95d640 1 -- 172.21.15.63:0/2884118950 wait complete. 2024-03-22T07:28:01.149 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 36 2024-03-22T07:28:01.149 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-03-22T07:28:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.249+0000 7fe1c286e640 1 -- 172.21.15.63:0/1315030304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1bc0fd3f0 msgr2=0x7fe1bc0fd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.249+0000 7fe1c286e640 1 --2- 172.21.15.63:0/1315030304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1bc0fd3f0 0x7fe1bc0fd7c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac0048d0 tx=0x7fe1ac009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.249+0000 7fe1c286e640 1 -- 172.21.15.63:0/1315030304 shutdown_connections 2024-03-22T07:28:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.249+0000 7fe1c286e640 1 --2- 172.21.15.63:0/1315030304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1bc0fdd00 0x7fe1bc106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.249+0000 7fe1c286e640 1 --2- 172.21.15.63:0/1315030304 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1bc0fd3f0 0x7fe1bc0fd7c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.251 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.249+0000 7fe1c286e640 1 -- 172.21.15.63:0/1315030304 >> 172.21.15.63:0/1315030304 conn(0x7fe1bc0f6eb0 msgr2=0x7fe1bc0f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.249+0000 7fe1c286e640 1 -- 172.21.15.63:0/1315030304 shutdown_connections 2024-03-22T07:28:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.249+0000 7fe1c286e640 1 -- 172.21.15.63:0/1315030304 wait complete. 2024-03-22T07:28:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.249+0000 7fe1c286e640 1 Processor -- start 2024-03-22T07:28:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.250+0000 7fe1c286e640 1 -- start start 2024-03-22T07:28:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.250+0000 7fe1c286e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1bc0fd3f0 0x7fe1bc112f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.250+0000 7fe1c286e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1bc0fdd00 0x7fe1bc1134c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.250+0000 7fe1c286e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc113ac0 con 0x7fe1bc0fd3f0 2024-03-22T07:28:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.250+0000 7fe1c286e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc10d110 con 0x7fe1bc0fdd00 2024-03-22T07:28:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.250+0000 7fe1bbfff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1bc0fd3f0 0x7fe1bc112f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.250+0000 7fe1bbfff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1bc0fd3f0 0x7fe1bc112f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42648/0 (socket says 172.21.15.63:42648) 2024-03-22T07:28:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.250+0000 7fe1bbfff640 1 -- 172.21.15.63:0/3105249341 learned_addr learned my addr 172.21.15.63:0/3105249341 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.250+0000 7fe1bb7fe640 1 --2- 172.21.15.63:0/3105249341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1bc0fdd00 0x7fe1bc1134c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.250+0000 7fe1bbfff640 1 -- 172.21.15.63:0/3105249341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1bc0fdd00 msgr2=0x7fe1bc1134c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.250+0000 7fe1bbfff640 1 --2- 172.21.15.63:0/3105249341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1bc0fdd00 0x7fe1bc1134c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.252 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.250+0000 7fe1bbfff640 1 -- 172.21.15.63:0/3105249341 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ac005040 con 0x7fe1bc0fd3f0 2024-03-22T07:28:01.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.250+0000 7fe1bbfff640 1 --2- 172.21.15.63:0/3105249341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1bc0fd3f0 0x7fe1bc112f80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac004930 tx=0x7fe1ac009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:01.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.250+0000 7fe1b8ff9640 1 -- 172.21.15.63:0/3105249341 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1ac0094c0 con 0x7fe1bc0fd3f0 2024-03-22T07:28:01.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.251+0000 7fe1b8ff9640 1 -- 172.21.15.63:0/3105249341 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1ac032e50 con 0x7fe1bc0fd3f0 2024-03-22T07:28:01.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.251+0000 7fe1c286e640 1 -- 172.21.15.63:0/3105249341 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1bc10d390 con 0x7fe1bc0fd3f0 2024-03-22T07:28:01.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.251+0000 7fe1b8ff9640 1 -- 172.21.15.63:0/3105249341 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1ac0433b0 con 0x7fe1bc0fd3f0 2024-03-22T07:28:01.253 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.251+0000 7fe1c286e640 1 -- 172.21.15.63:0/3105249341 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1bc10d850 con 0x7fe1bc0fd3f0 2024-03-22T07:28:01.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.252+0000 7fe1c286e640 1 -- 172.21.15.63:0/3105249341 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1bc065dd0 con 0x7fe1bc0fd3f0 2024-03-22T07:28:01.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.252+0000 7fe1b8ff9640 1 -- 172.21.15.63:0/3105249341 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fe1ac02f070 con 0x7fe1bc0fd3f0 2024-03-22T07:28:01.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.252+0000 7fe1b8ff9640 1 --2- 172.21.15.63:0/3105249341 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe1a406d2f0 0x7fe1a406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:01.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.253+0000 7fe1bb7fe640 1 --2- 172.21.15.63:0/3105249341 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe1a406d2f0 0x7fe1a406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:01.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.253+0000 7fe1b8ff9640 1 -- 172.21.15.63:0/3105249341 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe1ac0ae7b0 con 0x7fe1bc0fd3f0 2024-03-22T07:28:01.254 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.253+0000 7fe1bb7fe640 1 --2- 172.21.15.63:0/3105249341 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe1a406d2f0 0x7fe1a406f7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc10e840 tx=0x7fe1b0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:01.257 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.257+0000 7fe1b8ff9640 1 -- 172.21.15.63:0/3105249341 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe1ac078290 con 0x7fe1bc0fd3f0 2024-03-22T07:28:01.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.476+0000 7fe1c286e640 1 -- 172.21.15.63:0/3105249341 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7fe1bc0638c0 con 0x7fe1bc0fd3f0 2024-03-22T07:28:01.476 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.476+0000 7fe1b8ff9640 1 -- 172.21.15.63:0/3105249341 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v44) v1 ==== 107+0+5010 (secure 0 0 0) 0x7fe1ac07b950 con 0x7fe1bc0fd3f0 2024-03-22T07:28:01.476 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:28:01.477 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44323,"name":"cephfs.smithi069.wwtsjn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6825/653947973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":653947973},{"type":"v1","addr":"172.21.15.69:6825","nonce":653947973}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":44327,"name":"cephfs.smithi069.cijxah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6827/3465454734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3465454734},{"type":"v1","addr":"172.21.15.69:6827","nonce":3465454734}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:26:10.361971+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34400,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34400":{"gid":34400,"name":"cephfs.smithi063.oytapz","rank":0,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.63:6827/1171423403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1171423403},{"type":"v1","addr":"172.21.15.63:6827","nonce":1171423403}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi063.xeelpk","rank":1,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.63:6829/1190308435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1190308435},{"type":"v1","addr":"172.21.15.63:6829","nonce":1190308435}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:28:01.477 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 37 2024-03-22T07:28:01.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.478+0000 7fe1c286e640 1 -- 172.21.15.63:0/3105249341 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe1a406d2f0 msgr2=0x7fe1a406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:01.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.478+0000 7fe1c286e640 1 --2- 172.21.15.63:0/3105249341 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe1a406d2f0 0x7fe1a406f7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc10e840 tx=0x7fe1b0009040 comp rx=0 tx=0).stop 2024-03-22T07:28:01.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.478+0000 7fe1c286e640 1 -- 172.21.15.63:0/3105249341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1bc0fd3f0 msgr2=0x7fe1bc112f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:01.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.478+0000 7fe1c286e640 1 --2- 172.21.15.63:0/3105249341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1bc0fd3f0 0x7fe1bc112f80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac004930 tx=0x7fe1ac009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.478+0000 7fe1c286e640 1 -- 172.21.15.63:0/3105249341 shutdown_connections 2024-03-22T07:28:01.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.478+0000 7fe1c286e640 1 --2- 172.21.15.63:0/3105249341 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fe1a406d2f0 0x7fe1a406f7a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.478+0000 7fe1c286e640 1 --2- 172.21.15.63:0/3105249341 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1bc0fdd00 0x7fe1bc1134c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.478+0000 7fe1c286e640 1 --2- 172.21.15.63:0/3105249341 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fe1bc0fd3f0 0x7fe1bc112f80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.478+0000 7fe1c286e640 1 -- 172.21.15.63:0/3105249341 >> 172.21.15.63:0/3105249341 conn(0x7fe1bc0f6eb0 msgr2=0x7fe1bc0f83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:01.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.478+0000 7fe1c286e640 1 -- 172.21.15.63:0/3105249341 shutdown_connections 2024-03-22T07:28:01.479 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.478+0000 7fe1c286e640 1 -- 172.21.15.63:0/3105249341 wait complete. 2024-03-22T07:28:01.489 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 37 2024-03-22T07:28:01.489 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-03-22T07:28:01.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:01 smithi069 ceph-mon[83631]: pgmap v264: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:28:01.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:01 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3975500311' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-22T07:28:01.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:01 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3066193431' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-22T07:28:01.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:01 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/1809974925' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-22T07:28:01.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.588+0000 7f52ca71d640 1 -- 172.21.15.63:0/429692492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52c40fbac0 msgr2=0x7f52c4108820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:01.590 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.588+0000 7f52ca71d640 1 --2- 172.21.15.63:0/429692492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52c40fbac0 0x7f52c4108820 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f52b8004900 tx=0x7f52b8009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.588+0000 7f52ca71d640 1 -- 172.21.15.63:0/429692492 shutdown_connections 2024-03-22T07:28:01.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.588+0000 7f52ca71d640 1 --2- 172.21.15.63:0/429692492 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52c40fbac0 0x7f52c4108820 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.588+0000 7f52ca71d640 1 --2- 172.21.15.63:0/429692492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52c40fb1b0 0x7f52c40fb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.588+0000 7f52ca71d640 1 -- 172.21.15.63:0/429692492 >> 172.21.15.63:0/429692492 conn(0x7f52c40f6ec0 msgr2=0x7f52c40f92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:01.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.588+0000 7f52ca71d640 1 -- 172.21.15.63:0/429692492 shutdown_connections 2024-03-22T07:28:01.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.588+0000 7f52ca71d640 1 -- 172.21.15.63:0/429692492 wait complete. 2024-03-22T07:28:01.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.589+0000 7f52ca71d640 1 Processor -- start 2024-03-22T07:28:01.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.589+0000 7f52ca71d640 1 -- start start 2024-03-22T07:28:01.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.589+0000 7f52ca71d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52c40fb1b0 0x7f52c4115100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:01.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.589+0000 7f52ca71d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52c40fbac0 0x7f52c4115640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:01.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.589+0000 7f52ca71d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c4115c40 con 0x7f52c40fbac0 2024-03-22T07:28:01.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.589+0000 7f52ca71d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c410f290 con 0x7f52c40fb1b0 2024-03-22T07:28:01.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.589+0000 7f52c37fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52c40fbac0 0x7f52c4115640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:01.591 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.589+0000 7f52c37fe640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52c40fbac0 0x7f52c4115640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42674/0 (socket says 172.21.15.63:42674) 2024-03-22T07:28:01.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.589+0000 7f52c37fe640 1 -- 172.21.15.63:0/609825719 learned_addr learned my addr 172.21.15.63:0/609825719 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:01.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.589+0000 7f52c3fff640 1 --2- 172.21.15.63:0/609825719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52c40fb1b0 0x7f52c4115100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:01.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.589+0000 7f52c37fe640 1 -- 172.21.15.63:0/609825719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52c40fb1b0 msgr2=0x7f52c4115100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:01.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.589+0000 7f52c37fe640 1 --2- 172.21.15.63:0/609825719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52c40fb1b0 0x7f52c4115100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.590+0000 7f52c37fe640 1 -- 172.21.15.63:0/609825719 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b8005040 con 0x7f52c40fbac0 2024-03-22T07:28:01.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.590+0000 7f52c37fe640 1 --2- 172.21.15.63:0/609825719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52c40fbac0 0x7f52c4115640 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f52b8000c00 tx=0x7f52b8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:01.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.590+0000 7f52c0ff9640 1 -- 172.21.15.63:0/609825719 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52b80094c0 con 0x7f52c40fbac0 2024-03-22T07:28:01.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.590+0000 7f52c0ff9640 1 -- 172.21.15.63:0/609825719 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52b803d920 con 0x7f52c40fbac0 2024-03-22T07:28:01.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.590+0000 7f52ca71d640 1 -- 172.21.15.63:0/609825719 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c410f510 con 0x7f52c40fbac0 2024-03-22T07:28:01.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.590+0000 7f52c0ff9640 1 -- 172.21.15.63:0/609825719 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52b803b930 con 0x7f52c40fbac0 2024-03-22T07:28:01.592 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.590+0000 7f52ca71d640 1 -- 172.21.15.63:0/609825719 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c410fa30 con 0x7f52c40fbac0 2024-03-22T07:28:01.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.591+0000 7f52ca71d640 1 -- 172.21.15.63:0/609825719 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c4065e50 con 0x7f52c40fbac0 2024-03-22T07:28:01.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.591+0000 7f52c0ff9640 1 -- 172.21.15.63:0/609825719 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f52b803da90 con 0x7f52c40fbac0 2024-03-22T07:28:01.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.592+0000 7f52c0ff9640 1 --2- 172.21.15.63:0/609825719 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f52ac06d2f0 0x7f52ac06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:01.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.592+0000 7f52c3fff640 1 --2- 172.21.15.63:0/609825719 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f52ac06d2f0 0x7f52ac06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:01.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.592+0000 7f52c0ff9640 1 -- 172.21.15.63:0/609825719 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f52b80af1b0 con 0x7f52c40fbac0 2024-03-22T07:28:01.593 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.592+0000 7f52c3fff640 1 --2- 172.21.15.63:0/609825719 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f52ac06d2f0 0x7f52ac06f7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f52b4004bc0 tx=0x7f52b4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:01.596 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.596+0000 7f52c0ff9640 1 -- 172.21.15.63:0/609825719 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f52b8078d20 con 0x7f52c40fbac0 2024-03-22T07:28:01.817 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.817+0000 7f52ca71d640 1 -- 172.21.15.63:0/609825719 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f52c410b710 con 0x7f52c40fbac0 2024-03-22T07:28:01.818 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.818+0000 7f52c0ff9640 1 -- 172.21.15.63:0/609825719 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v44) v1 ==== 107+0+5009 (secure 0 0 0) 0x7f52b807c3e0 con 0x7f52c40fbac0 2024-03-22T07:28:01.818 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:28:01.818 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44323,"name":"cephfs.smithi069.wwtsjn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6825/653947973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":653947973},{"type":"v1","addr":"172.21.15.69:6825","nonce":653947973}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":44327,"name":"cephfs.smithi069.cijxah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6827/3465454734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3465454734},{"type":"v1","addr":"172.21.15.69:6827","nonce":3465454734}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:26:10.366370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34400,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34400":{"gid":34400,"name":"cephfs.smithi063.oytapz","rank":0,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.63:6827/1171423403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1171423403},{"type":"v1","addr":"172.21.15.63:6827","nonce":1171423403}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi063.xeelpk","rank":1,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.63:6829/1190308435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1190308435},{"type":"v1","addr":"172.21.15.63:6829","nonce":1190308435}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:28:01.818 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 38 2024-03-22T07:28:01.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.820+0000 7f52ca71d640 1 -- 172.21.15.63:0/609825719 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f52ac06d2f0 msgr2=0x7f52ac06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:01.820 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.820+0000 7f52ca71d640 1 --2- 172.21.15.63:0/609825719 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f52ac06d2f0 0x7f52ac06f7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f52b4004bc0 tx=0x7f52b4009040 comp rx=0 tx=0).stop 2024-03-22T07:28:01.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.820+0000 7f52ca71d640 1 -- 172.21.15.63:0/609825719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52c40fbac0 msgr2=0x7f52c4115640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:01.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.820+0000 7f52ca71d640 1 --2- 172.21.15.63:0/609825719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52c40fbac0 0x7f52c4115640 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f52b8000c00 tx=0x7f52b8009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.820+0000 7f52ca71d640 1 -- 172.21.15.63:0/609825719 shutdown_connections 2024-03-22T07:28:01.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.820+0000 7f52ca71d640 1 --2- 172.21.15.63:0/609825719 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f52c40fbac0 0x7f52c4115640 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.820+0000 7f52ca71d640 1 --2- 172.21.15.63:0/609825719 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f52ac06d2f0 0x7f52ac06f7a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.820+0000 7f52ca71d640 1 --2- 172.21.15.63:0/609825719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52c40fb1b0 0x7f52c4115100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.820+0000 7f52ca71d640 1 -- 172.21.15.63:0/609825719 >> 172.21.15.63:0/609825719 conn(0x7f52c40f6ec0 msgr2=0x7f52c40f7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:01.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.820+0000 7f52ca71d640 1 -- 172.21.15.63:0/609825719 shutdown_connections 2024-03-22T07:28:01.821 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.820+0000 7f52ca71d640 1 -- 172.21.15.63:0/609825719 wait complete. 2024-03-22T07:28:01.831 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 39 2024-03-22T07:28:01.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.930+0000 7f68b906e640 1 -- 172.21.15.63:0/413346095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68b41010e0 msgr2=0x7f68b41014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:01.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.930+0000 7f68b906e640 1 --2- 172.21.15.63:0/413346095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68b41010e0 0x7f68b41014b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f68a40048d0 tx=0x7f68a4009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.930+0000 7f68b906e640 1 -- 172.21.15.63:0/413346095 shutdown_connections 2024-03-22T07:28:01.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.930+0000 7f68b906e640 1 --2- 172.21.15.63:0/413346095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68b40fb8a0 0x7f68b40fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.930+0000 7f68b906e640 1 --2- 172.21.15.63:0/413346095 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68b41010e0 0x7f68b41014b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.930+0000 7f68b906e640 1 -- 172.21.15.63:0/413346095 >> 172.21.15.63:0/413346095 conn(0x7f68b40714a0 msgr2=0x7f68b40f88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:01.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.930+0000 7f68b906e640 1 -- 172.21.15.63:0/413346095 shutdown_connections 2024-03-22T07:28:01.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.930+0000 7f68b906e640 1 -- 172.21.15.63:0/413346095 wait complete. 2024-03-22T07:28:01.932 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.931+0000 7f68b906e640 1 Processor -- start 2024-03-22T07:28:01.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.931+0000 7f68b906e640 1 -- start start 2024-03-22T07:28:01.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.931+0000 7f68b906e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68b40fb8a0 0x7f68b410deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:01.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.931+0000 7f68b906e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68b41010e0 0x7f68b410e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:01.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.931+0000 7f68b906e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40fb730 con 0x7f68b40fb8a0 2024-03-22T07:28:01.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.931+0000 7f68b906e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b410e930 con 0x7f68b41010e0 2024-03-22T07:28:01.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.931+0000 7f68b2d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68b40fb8a0 0x7f68b410deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:01.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.931+0000 7f68b2d9d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68b40fb8a0 0x7f68b410deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42704/0 (socket says 172.21.15.63:42704) 2024-03-22T07:28:01.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.931+0000 7f68b2d9d640 1 -- 172.21.15.63:0/1052451493 learned_addr learned my addr 172.21.15.63:0/1052451493 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:01.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.931+0000 7f68abfff640 1 --2- 172.21.15.63:0/1052451493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68b41010e0 0x7f68b410e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:01.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.931+0000 7f68b2d9d640 1 -- 172.21.15.63:0/1052451493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68b41010e0 msgr2=0x7f68b410e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:01.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.931+0000 7f68b2d9d640 1 --2- 172.21.15.63:0/1052451493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68b41010e0 0x7f68b410e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:01.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.931+0000 7f68b2d9d640 1 -- 172.21.15.63:0/1052451493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a4005040 con 0x7f68b40fb8a0 2024-03-22T07:28:01.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.931+0000 7f68b2d9d640 1 --2- 172.21.15.63:0/1052451493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68b40fb8a0 0x7f68b410deb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f68a4004930 tx=0x7f68a4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:01.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.932+0000 7f68ab7fe640 1 -- 172.21.15.63:0/1052451493 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68a40094c0 con 0x7f68b40fb8a0 2024-03-22T07:28:01.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.932+0000 7f68b906e640 1 -- 172.21.15.63:0/1052451493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b410ebb0 con 0x7f68b40fb8a0 2024-03-22T07:28:01.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.932+0000 7f68ab7fe640 1 -- 172.21.15.63:0/1052451493 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68a4032e50 con 0x7f68b40fb8a0 2024-03-22T07:28:01.933 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.932+0000 7f68ab7fe640 1 -- 172.21.15.63:0/1052451493 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68a40433b0 con 0x7f68b40fb8a0 2024-03-22T07:28:01.934 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.932+0000 7f68b906e640 1 -- 172.21.15.63:0/1052451493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b4117110 con 0x7f68b40fb8a0 2024-03-22T07:28:01.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.933+0000 7f68b906e640 1 -- 172.21.15.63:0/1052451493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b40fc8b0 con 0x7f68b40fb8a0 2024-03-22T07:28:01.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.933+0000 7f68ab7fe640 1 -- 172.21.15.63:0/1052451493 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f68a402f070 con 0x7f68b40fb8a0 2024-03-22T07:28:01.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.933+0000 7f68ab7fe640 1 --2- 172.21.15.63:0/1052451493 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f68a006d2f0 0x7f68a006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:01.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.934+0000 7f68abfff640 1 --2- 172.21.15.63:0/1052451493 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f68a006d2f0 0x7f68a006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:01.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.934+0000 7f68ab7fe640 1 -- 172.21.15.63:0/1052451493 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f68a40ae970 con 0x7f68b40fb8a0 2024-03-22T07:28:01.935 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.934+0000 7f68abfff640 1 --2- 172.21.15.63:0/1052451493 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f68a006d2f0 0x7f68a006f7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f689c007fd0 tx=0x7f689c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:01.938 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:01.938+0000 7f68ab7fe640 1 -- 172.21.15.63:0/1052451493 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f68a40783a0 con 0x7f68b40fb8a0 2024-03-22T07:28:02.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:02 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2884118950' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-22T07:28:02.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:02 smithi063 ceph-mon[112784]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:28:02.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:02 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3105249341' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-03-22T07:28:02.123 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:02 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/609825719' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-03-22T07:28:02.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.159+0000 7f68b906e640 1 -- 172.21.15.63:0/1052451493 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7f68b40fe380 con 0x7f68b40fb8a0 2024-03-22T07:28:02.160 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.160+0000 7f68ab7fe640 1 -- 172.21.15.63:0/1052451493 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v44) v1 ==== 107+0+5011 (secure 0 0 0) 0x7f68a407ba60 con 0x7f68b40fb8a0 2024-03-22T07:28:02.160 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:28:02.161 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44323,"name":"cephfs.smithi069.wwtsjn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6825/653947973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":653947973},{"type":"v1","addr":"172.21.15.69:6825","nonce":653947973}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":44327,"name":"cephfs.smithi069.cijxah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6827/3465454734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3465454734},{"type":"v1","addr":"172.21.15.69:6827","nonce":3465454734}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:26:15.667337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34400,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34400":{"gid":34400,"name":"cephfs.smithi063.oytapz","rank":0,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.63:6827/1171423403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1171423403},{"type":"v1","addr":"172.21.15.63:6827","nonce":1171423403}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi063.xeelpk","rank":1,"incarnation":37,"state":"up:resolve","state_seq":13,"addr":"172.21.15.63:6829/1190308435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1190308435},{"type":"v1","addr":"172.21.15.63:6829","nonce":1190308435}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:28:02.161 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 39 2024-03-22T07:28:02.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.162+0000 7f68b906e640 1 -- 172.21.15.63:0/1052451493 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f68a006d2f0 msgr2=0x7f68a006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:02.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.162+0000 7f68b906e640 1 --2- 172.21.15.63:0/1052451493 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f68a006d2f0 0x7f68a006f7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f689c007fd0 tx=0x7f689c009040 comp rx=0 tx=0).stop 2024-03-22T07:28:02.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.162+0000 7f68b906e640 1 -- 172.21.15.63:0/1052451493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68b40fb8a0 msgr2=0x7f68b410deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:02.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.162+0000 7f68b906e640 1 --2- 172.21.15.63:0/1052451493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68b40fb8a0 0x7f68b410deb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f68a4004930 tx=0x7f68a4009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.162+0000 7f68b906e640 1 -- 172.21.15.63:0/1052451493 shutdown_connections 2024-03-22T07:28:02.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.162+0000 7f68b906e640 1 --2- 172.21.15.63:0/1052451493 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f68a006d2f0 0x7f68a006f7a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.162+0000 7f68b906e640 1 --2- 172.21.15.63:0/1052451493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68b41010e0 0x7f68b410e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.162+0000 7f68b906e640 1 --2- 172.21.15.63:0/1052451493 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f68b40fb8a0 0x7f68b410deb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.162+0000 7f68b906e640 1 -- 172.21.15.63:0/1052451493 >> 172.21.15.63:0/1052451493 conn(0x7f68b40714a0 msgr2=0x7f68b4072bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:02.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.162+0000 7f68b906e640 1 -- 172.21.15.63:0/1052451493 shutdown_connections 2024-03-22T07:28:02.163 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.162+0000 7f68b906e640 1 -- 172.21.15.63:0/1052451493 wait complete. 2024-03-22T07:28:02.173 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 40 2024-03-22T07:28:02.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.280+0000 7f056595d640 1 -- 172.21.15.63:0/1734736746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0560102200 msgr2=0x7f05601025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:02.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.280+0000 7f056595d640 1 --2- 172.21.15.63:0/1734736746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0560102200 0x7f05601025d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f05500048d0 tx=0x7f0550009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.280+0000 7f056595d640 1 -- 172.21.15.63:0/1734736746 shutdown_connections 2024-03-22T07:28:02.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.280+0000 7f056595d640 1 --2- 172.21.15.63:0/1734736746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0560102ba0 0x7f05600fa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.280+0000 7f056595d640 1 --2- 172.21.15.63:0/1734736746 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0560102200 0x7f05601025d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.280+0000 7f056595d640 1 -- 172.21.15.63:0/1734736746 >> 172.21.15.63:0/1734736746 conn(0x7f05600708f0 msgr2=0x7f0560070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:02.282 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.280+0000 7f056595d640 1 -- 172.21.15.63:0/1734736746 shutdown_connections 2024-03-22T07:28:02.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.280+0000 7f056595d640 1 -- 172.21.15.63:0/1734736746 wait complete. 2024-03-22T07:28:02.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.281+0000 7f056595d640 1 Processor -- start 2024-03-22T07:28:02.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.281+0000 7f056595d640 1 -- start start 2024-03-22T07:28:02.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.281+0000 7f056595d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0560102200 0x7f0560112e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:02.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.281+0000 7f056595d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0560102ba0 0x7f0560113370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:02.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.281+0000 7f056595d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0560113970 con 0x7f0560102ba0 2024-03-22T07:28:02.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.281+0000 7f056595d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056010cfc0 con 0x7f0560102200 2024-03-22T07:28:02.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.281+0000 7f0557fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0560102ba0 0x7f0560113370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:02.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.281+0000 7f0557fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0560102ba0 0x7f0560113370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42734/0 (socket says 172.21.15.63:42734) 2024-03-22T07:28:02.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.281+0000 7f0557fff640 1 -- 172.21.15.63:0/706179168 learned_addr learned my addr 172.21.15.63:0/706179168 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:02.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.281+0000 7f055effd640 1 --2- 172.21.15.63:0/706179168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0560102200 0x7f0560112e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:02.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.281+0000 7f0557fff640 1 -- 172.21.15.63:0/706179168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0560102200 msgr2=0x7f0560112e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:02.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.281+0000 7f0557fff640 1 --2- 172.21.15.63:0/706179168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0560102200 0x7f0560112e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.281+0000 7f0557fff640 1 -- 172.21.15.63:0/706179168 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0550005040 con 0x7f0560102ba0 2024-03-22T07:28:02.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.281+0000 7f055effd640 1 --2- 172.21.15.63:0/706179168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0560102200 0x7f0560112e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:28:02.283 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.282+0000 7f0557fff640 1 --2- 172.21.15.63:0/706179168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0560102ba0 0x7f0560113370 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0548002840 tx=0x7f0548004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:02.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.282+0000 7f05577fe640 1 -- 172.21.15.63:0/706179168 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0548007c30 con 0x7f0560102ba0 2024-03-22T07:28:02.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.282+0000 7f05577fe640 1 -- 172.21.15.63:0/706179168 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f054800f040 con 0x7f0560102ba0 2024-03-22T07:28:02.284 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.282+0000 7f056595d640 1 -- 172.21.15.63:0/706179168 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056010d2a0 con 0x7f0560102ba0 2024-03-22T07:28:02.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.282+0000 7f05577fe640 1 -- 172.21.15.63:0/706179168 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0548009590 con 0x7f0560102ba0 2024-03-22T07:28:02.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.283+0000 7f056595d640 1 -- 172.21.15.63:0/706179168 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056010d7c0 con 0x7f0560102ba0 2024-03-22T07:28:02.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.284+0000 7f05577fe640 1 -- 172.21.15.63:0/706179168 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f054800d6b0 con 0x7f0560102ba0 2024-03-22T07:28:02.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.284+0000 7f056595d640 1 -- 172.21.15.63:0/706179168 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05600652a0 con 0x7f0560102ba0 2024-03-22T07:28:02.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.284+0000 7f05577fe640 1 --2- 172.21.15.63:0/706179168 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f054c06d2f0 0x7f054c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:02.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.284+0000 7f05577fe640 1 -- 172.21.15.63:0/706179168 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f054808c230 con 0x7f0560102ba0 2024-03-22T07:28:02.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.284+0000 7f055effd640 1 --2- 172.21.15.63:0/706179168 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f054c06d2f0 0x7f054c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:02.285 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.285+0000 7f055effd640 1 --2- 172.21.15.63:0/706179168 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f054c06d2f0 0x7f054c06f7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0550031870 tx=0x7f0550009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:02.289 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.289+0000 7f05577fe640 1 -- 172.21.15.63:0/706179168 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0548055ce0 con 0x7f0560102ba0 2024-03-22T07:28:02.511 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.511+0000 7f056595d640 1 -- 172.21.15.63:0/706179168 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 40, "format": "json"} v 0) v1 -- 0x7f05601093b0 con 0x7f0560102ba0 2024-03-22T07:28:02.512 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.512+0000 7f05577fe640 1 -- 172.21.15.63:0/706179168 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 40, "format": "json"}]=0 dumped fsmap epoch 40 v44) v1 ==== 107+0+5013 (secure 0 0 0) 0x7f05480593a0 con 0x7f0560102ba0 2024-03-22T07:28:02.512 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:28:02.512 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44323,"name":"cephfs.smithi069.wwtsjn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6825/653947973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":653947973},{"type":"v1","addr":"172.21.15.69:6825","nonce":653947973}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":44327,"name":"cephfs.smithi069.cijxah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6827/3465454734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3465454734},{"type":"v1","addr":"172.21.15.69:6827","nonce":3465454734}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:26:18.809160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34400,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34400":{"gid":34400,"name":"cephfs.smithi063.oytapz","rank":0,"incarnation":37,"state":"up:resolve","state_seq":19,"addr":"172.21.15.63:6827/1171423403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1171423403},{"type":"v1","addr":"172.21.15.63:6827","nonce":1171423403}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi063.xeelpk","rank":1,"incarnation":37,"state":"up:resolve","state_seq":13,"addr":"172.21.15.63:6829/1190308435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1190308435},{"type":"v1","addr":"172.21.15.63:6829","nonce":1190308435}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:28:02.513 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 40 2024-03-22T07:28:02.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.514+0000 7f056595d640 1 -- 172.21.15.63:0/706179168 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f054c06d2f0 msgr2=0x7f054c06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:02.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.514+0000 7f056595d640 1 --2- 172.21.15.63:0/706179168 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f054c06d2f0 0x7f054c06f7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0550031870 tx=0x7f0550009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.514 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.514+0000 7f056595d640 1 -- 172.21.15.63:0/706179168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0560102ba0 msgr2=0x7f0560113370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:02.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.514+0000 7f056595d640 1 --2- 172.21.15.63:0/706179168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0560102ba0 0x7f0560113370 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0548002840 tx=0x7f0548004570 comp rx=0 tx=0).stop 2024-03-22T07:28:02.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.514+0000 7f056595d640 1 -- 172.21.15.63:0/706179168 shutdown_connections 2024-03-22T07:28:02.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.514+0000 7f056595d640 1 --2- 172.21.15.63:0/706179168 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0560102ba0 0x7f0560113370 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.514+0000 7f056595d640 1 --2- 172.21.15.63:0/706179168 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f054c06d2f0 0x7f054c06f7a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.514+0000 7f056595d640 1 --2- 172.21.15.63:0/706179168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0560102200 0x7f0560112e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.514+0000 7f056595d640 1 -- 172.21.15.63:0/706179168 >> 172.21.15.63:0/706179168 conn(0x7f05600708f0 msgr2=0x7f0560104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:02.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.514+0000 7f056595d640 1 -- 172.21.15.63:0/706179168 shutdown_connections 2024-03-22T07:28:02.515 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.514+0000 7f056595d640 1 -- 172.21.15.63:0/706179168 wait complete. 2024-03-22T07:28:02.525 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 41 2024-03-22T07:28:02.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:02 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2884118950' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-22T07:28:02.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:02 smithi069 ceph-mon[83631]: from='mgr.34104 172.21.15.63:0/2323610746' entity='mgr.smithi063.mnxayh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-22T07:28:02.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:02 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3105249341' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-03-22T07:28:02.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:02 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/609825719' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-03-22T07:28:02.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.624+0000 7f704dd1d640 1 -- 172.21.15.63:0/80544160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7048101a70 msgr2=0x7f70480fb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:02.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.624+0000 7f704dd1d640 1 --2- 172.21.15.63:0/80544160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7048101a70 0x7f70480fb9e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f70380048d0 tx=0x7f7038009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.624+0000 7f704dd1d640 1 -- 172.21.15.63:0/80544160 shutdown_connections 2024-03-22T07:28:02.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.624+0000 7f704dd1d640 1 --2- 172.21.15.63:0/80544160 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7048101a70 0x7f70480fb9e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.626 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.624+0000 7f704dd1d640 1 --2- 172.21.15.63:0/80544160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70481010d0 0x7f70481014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.624+0000 7f704dd1d640 1 -- 172.21.15.63:0/80544160 >> 172.21.15.63:0/80544160 conn(0x7f70480708f0 msgr2=0x7f7048070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:02.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.624+0000 7f704dd1d640 1 -- 172.21.15.63:0/80544160 shutdown_connections 2024-03-22T07:28:02.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.624+0000 7f704dd1d640 1 -- 172.21.15.63:0/80544160 wait complete. 2024-03-22T07:28:02.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.624+0000 7f704dd1d640 1 Processor -- start 2024-03-22T07:28:02.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.625+0000 7f704dd1d640 1 -- start start 2024-03-22T07:28:02.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.625+0000 7f704dd1d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70481010d0 0x7f704810df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:02.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.625+0000 7f704dd1d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7048101a70 0x7f704810e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:02.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.625+0000 7f704dd1d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70480fb870 con 0x7f7048101a70 2024-03-22T07:28:02.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.625+0000 7f704dd1d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704810e990 con 0x7f70481010d0 2024-03-22T07:28:02.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.625+0000 7f7046ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7048101a70 0x7f704810e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:02.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.625+0000 7f7046ffd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7048101a70 0x7f704810e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42746/0 (socket says 172.21.15.63:42746) 2024-03-22T07:28:02.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.625+0000 7f7046ffd640 1 -- 172.21.15.63:0/2506715598 learned_addr learned my addr 172.21.15.63:0/2506715598 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:02.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.625+0000 7f7046ffd640 1 -- 172.21.15.63:0/2506715598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70481010d0 msgr2=0x7f704810df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:02.627 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.625+0000 7f70477fe640 1 --2- 172.21.15.63:0/2506715598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70481010d0 0x7f704810df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:02.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.625+0000 7f7046ffd640 1 --2- 172.21.15.63:0/2506715598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70481010d0 0x7f704810df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.625+0000 7f7046ffd640 1 -- 172.21.15.63:0/2506715598 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7038005040 con 0x7f7048101a70 2024-03-22T07:28:02.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.625+0000 7f7046ffd640 1 --2- 172.21.15.63:0/2506715598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7048101a70 0x7f704810e450 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7038004930 tx=0x7f7038009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:02.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.626+0000 7f702ffff640 1 -- 172.21.15.63:0/2506715598 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70380094c0 con 0x7f7048101a70 2024-03-22T07:28:02.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.626+0000 7f702ffff640 1 -- 172.21.15.63:0/2506715598 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7038032e50 con 0x7f7048101a70 2024-03-22T07:28:02.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.626+0000 7f702ffff640 1 -- 172.21.15.63:0/2506715598 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70380433b0 con 0x7f7048101a70 2024-03-22T07:28:02.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.626+0000 7f704dd1d640 1 -- 172.21.15.63:0/2506715598 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704810ec10 con 0x7f7048101a70 2024-03-22T07:28:02.628 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.626+0000 7f704dd1d640 1 -- 172.21.15.63:0/2506715598 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7048117170 con 0x7f7048101a70 2024-03-22T07:28:02.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.627+0000 7f704dd1d640 1 -- 172.21.15.63:0/2506715598 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70480fc9f0 con 0x7f7048101a70 2024-03-22T07:28:02.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.627+0000 7f702ffff640 1 -- 172.21.15.63:0/2506715598 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f703802f070 con 0x7f7048101a70 2024-03-22T07:28:02.629 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.628+0000 7f702ffff640 1 --2- 172.21.15.63:0/2506715598 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f703006d2f0 0x7f703006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:02.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.628+0000 7f702ffff640 1 -- 172.21.15.63:0/2506715598 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f70380ae8d0 con 0x7f7048101a70 2024-03-22T07:28:02.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.628+0000 7f70477fe640 1 --2- 172.21.15.63:0/2506715598 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f703006d2f0 0x7f703006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:02.630 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.628+0000 7f70477fe640 1 --2- 172.21.15.63:0/2506715598 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f703006d2f0 0x7f703006f7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f703c004bc0 tx=0x7f703c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:02.632 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.632+0000 7f702ffff640 1 -- 172.21.15.63:0/2506715598 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7038078300 con 0x7f7048101a70 2024-03-22T07:28:02.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.853+0000 7f704dd1d640 1 -- 172.21.15.63:0/2506715598 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 41, "format": "json"} v 0) v1 -- 0x7f704804bcc0 con 0x7f7048101a70 2024-03-22T07:28:02.854 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.854+0000 7f702ffff640 1 -- 172.21.15.63:0/2506715598 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 41, "format": "json"}]=0 dumped fsmap epoch 41 v44) v1 ==== 107+0+5017 (secure 0 0 0) 0x7f703807b9c0 con 0x7f7048101a70 2024-03-22T07:28:02.854 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:28:02.855 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44323,"name":"cephfs.smithi069.wwtsjn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6825/653947973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":653947973},{"type":"v1","addr":"172.21.15.69:6825","nonce":653947973}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":44327,"name":"cephfs.smithi069.cijxah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6827/3465454734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3465454734},{"type":"v1","addr":"172.21.15.69:6827","nonce":3465454734}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:26:19.413434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34400,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34400":{"gid":34400,"name":"cephfs.smithi063.oytapz","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.63:6827/1171423403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1171423403},{"type":"v1","addr":"172.21.15.63:6827","nonce":1171423403}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi063.xeelpk","rank":1,"incarnation":37,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.63:6829/1190308435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1190308435},{"type":"v1","addr":"172.21.15.63:6829","nonce":1190308435}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:28:02.855 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 41 2024-03-22T07:28:02.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.856+0000 7f704dd1d640 1 -- 172.21.15.63:0/2506715598 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f703006d2f0 msgr2=0x7f703006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:02.856 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.856+0000 7f704dd1d640 1 --2- 172.21.15.63:0/2506715598 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f703006d2f0 0x7f703006f7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f703c004bc0 tx=0x7f703c009040 comp rx=0 tx=0).stop 2024-03-22T07:28:02.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.856+0000 7f704dd1d640 1 -- 172.21.15.63:0/2506715598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7048101a70 msgr2=0x7f704810e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:02.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.856+0000 7f704dd1d640 1 --2- 172.21.15.63:0/2506715598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7048101a70 0x7f704810e450 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7038004930 tx=0x7f7038009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.856+0000 7f704dd1d640 1 -- 172.21.15.63:0/2506715598 shutdown_connections 2024-03-22T07:28:02.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.856+0000 7f704dd1d640 1 --2- 172.21.15.63:0/2506715598 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f7048101a70 0x7f704810e450 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.856+0000 7f704dd1d640 1 --2- 172.21.15.63:0/2506715598 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f703006d2f0 0x7f703006f7a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.856+0000 7f704dd1d640 1 --2- 172.21.15.63:0/2506715598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f70481010d0 0x7f704810df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.856+0000 7f704dd1d640 1 -- 172.21.15.63:0/2506715598 >> 172.21.15.63:0/2506715598 conn(0x7f70480708f0 msgr2=0x7f70480f8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:02.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.856+0000 7f704dd1d640 1 -- 172.21.15.63:0/2506715598 shutdown_connections 2024-03-22T07:28:02.857 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.856+0000 7f704dd1d640 1 -- 172.21.15.63:0/2506715598 wait complete. 2024-03-22T07:28:02.867 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 42 2024-03-22T07:28:02.968 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.966+0000 7f3a55a65640 1 -- 172.21.15.63:0/3631556959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a500fa650 msgr2=0x7f3a500faac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:02.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.966+0000 7f3a55a65640 1 --2- 172.21.15.63:0/3631556959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a500fa650 0x7f3a500faac0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3a440048d0 tx=0x7f3a44009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.966+0000 7f3a55a65640 1 -- 172.21.15.63:0/3631556959 shutdown_connections 2024-03-22T07:28:02.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.966+0000 7f3a55a65640 1 --2- 172.21.15.63:0/3631556959 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a500fa650 0x7f3a500faac0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.966+0000 7f3a55a65640 1 --2- 172.21.15.63:0/3631556959 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a50102260 0x7f3a50102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.966+0000 7f3a55a65640 1 -- 172.21.15.63:0/3631556959 >> 172.21.15.63:0/3631556959 conn(0x7f3a500714a0 msgr2=0x7f3a500f88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:02.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.966+0000 7f3a55a65640 1 -- 172.21.15.63:0/3631556959 shutdown_connections 2024-03-22T07:28:02.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.966+0000 7f3a55a65640 1 -- 172.21.15.63:0/3631556959 wait complete. 2024-03-22T07:28:02.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.967+0000 7f3a55a65640 1 Processor -- start 2024-03-22T07:28:02.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.967+0000 7f3a55a65640 1 -- start start 2024-03-22T07:28:02.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.967+0000 7f3a55a65640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a500fa650 0x7f3a500fb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:02.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.967+0000 7f3a55a65640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a50102260 0x7f3a500fbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:02.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.967+0000 7f3a55a65640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a500fd710 con 0x7f3a50102260 2024-03-22T07:28:02.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.967+0000 7f3a55a65640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a500fc2b0 con 0x7f3a500fa650 2024-03-22T07:28:02.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.967+0000 7f3a4effd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a50102260 0x7f3a500fbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:02.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.967+0000 7f3a4effd640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a50102260 0x7f3a500fbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42756/0 (socket says 172.21.15.63:42756) 2024-03-22T07:28:02.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.967+0000 7f3a4effd640 1 -- 172.21.15.63:0/3786339076 learned_addr learned my addr 172.21.15.63:0/3786339076 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:02.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.967+0000 7f3a4f7fe640 1 --2- 172.21.15.63:0/3786339076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a500fa650 0x7f3a500fb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:02.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.967+0000 7f3a4effd640 1 -- 172.21.15.63:0/3786339076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a500fa650 msgr2=0x7f3a500fb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:02.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.967+0000 7f3a4effd640 1 --2- 172.21.15.63:0/3786339076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a500fa650 0x7f3a500fb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:02.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.967+0000 7f3a4effd640 1 -- 172.21.15.63:0/3786339076 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a44005040 con 0x7f3a50102260 2024-03-22T07:28:02.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.967+0000 7f3a4f7fe640 1 --2- 172.21.15.63:0/3786339076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a500fa650 0x7f3a500fb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:28:02.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.967+0000 7f3a4effd640 1 --2- 172.21.15.63:0/3786339076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a50102260 0x7f3a500fbd70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3a44004930 tx=0x7f3a44009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:02.970 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.968+0000 7f3a37fff640 1 -- 172.21.15.63:0/3786339076 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a440094c0 con 0x7f3a50102260 2024-03-22T07:28:02.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.968+0000 7f3a37fff640 1 -- 172.21.15.63:0/3786339076 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a44032e50 con 0x7f3a50102260 2024-03-22T07:28:02.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.968+0000 7f3a55a65640 1 -- 172.21.15.63:0/3786339076 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a500fc530 con 0x7f3a50102260 2024-03-22T07:28:02.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.968+0000 7f3a37fff640 1 -- 172.21.15.63:0/3786339076 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a440433b0 con 0x7f3a50102260 2024-03-22T07:28:02.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.968+0000 7f3a55a65640 1 -- 172.21.15.63:0/3786339076 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a50119430 con 0x7f3a50102260 2024-03-22T07:28:02.971 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.969+0000 7f3a55a65640 1 -- 172.21.15.63:0/3786339076 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5004bcc0 con 0x7f3a50102260 2024-03-22T07:28:02.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.969+0000 7f3a37fff640 1 -- 172.21.15.63:0/3786339076 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f3a4402f070 con 0x7f3a50102260 2024-03-22T07:28:02.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.970+0000 7f3a37fff640 1 --2- 172.21.15.63:0/3786339076 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f3a3806d2f0 0x7f3a3806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:02.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.970+0000 7f3a37fff640 1 -- 172.21.15.63:0/3786339076 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3a440ae810 con 0x7f3a50102260 2024-03-22T07:28:02.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.970+0000 7f3a4f7fe640 1 --2- 172.21.15.63:0/3786339076 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f3a3806d2f0 0x7f3a3806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:02.972 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.970+0000 7f3a4f7fe640 1 --2- 172.21.15.63:0/3786339076 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f3a3806d2f0 0x7f3a3806f7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3a500650a0 tx=0x7f3a4000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:02.974 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:02.974+0000 7f3a37fff640 1 -- 172.21.15.63:0/3786339076 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3a44078240 con 0x7f3a50102260 2024-03-22T07:28:03.197 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.197+0000 7f3a55a65640 1 -- 172.21.15.63:0/3786339076 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 42, "format": "json"} v 0) v1 -- 0x7f3a500fd930 con 0x7f3a50102260 2024-03-22T07:28:03.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.198+0000 7f3a37fff640 1 -- 172.21.15.63:0/3786339076 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 42, "format": "json"}]=0 dumped fsmap epoch 42 v44) v1 ==== 107+0+5011 (secure 0 0 0) 0x7f3a4407b900 con 0x7f3a50102260 2024-03-22T07:28:03.198 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:28:03.198 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44323,"name":"cephfs.smithi069.wwtsjn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6825/653947973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":653947973},{"type":"v1","addr":"172.21.15.69:6825","nonce":653947973}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":44327,"name":"cephfs.smithi069.cijxah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6827/3465454734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3465454734},{"type":"v1","addr":"172.21.15.69:6827","nonce":3465454734}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:26:20.393717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34400,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34400":{"gid":34400,"name":"cephfs.smithi063.oytapz","rank":0,"incarnation":37,"state":"up:rejoin","state_seq":21,"addr":"172.21.15.63:6827/1171423403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1171423403},{"type":"v1","addr":"172.21.15.63:6827","nonce":1171423403}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi063.xeelpk","rank":1,"incarnation":37,"state":"up:rejoin","state_seq":15,"addr":"172.21.15.63:6829/1190308435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1190308435},{"type":"v1","addr":"172.21.15.63:6829","nonce":1190308435}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:28:03.198 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 42 2024-03-22T07:28:03.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.200+0000 7f3a55a65640 1 -- 172.21.15.63:0/3786339076 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f3a3806d2f0 msgr2=0x7f3a3806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:03.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.200+0000 7f3a55a65640 1 --2- 172.21.15.63:0/3786339076 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f3a3806d2f0 0x7f3a3806f7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3a500650a0 tx=0x7f3a4000b040 comp rx=0 tx=0).stop 2024-03-22T07:28:03.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.200+0000 7f3a55a65640 1 -- 172.21.15.63:0/3786339076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a50102260 msgr2=0x7f3a500fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:03.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.200+0000 7f3a55a65640 1 --2- 172.21.15.63:0/3786339076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a50102260 0x7f3a500fbd70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3a44004930 tx=0x7f3a44009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.200 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.200+0000 7f3a55a65640 1 -- 172.21.15.63:0/3786339076 shutdown_connections 2024-03-22T07:28:03.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.200+0000 7f3a55a65640 1 --2- 172.21.15.63:0/3786339076 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f3a50102260 0x7f3a500fbd70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.200+0000 7f3a55a65640 1 --2- 172.21.15.63:0/3786339076 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f3a3806d2f0 0x7f3a3806f7a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.200+0000 7f3a55a65640 1 --2- 172.21.15.63:0/3786339076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a500fa650 0x7f3a500fb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.200+0000 7f3a55a65640 1 -- 172.21.15.63:0/3786339076 >> 172.21.15.63:0/3786339076 conn(0x7f3a500714a0 msgr2=0x7f3a50072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:03.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.200+0000 7f3a55a65640 1 -- 172.21.15.63:0/3786339076 shutdown_connections 2024-03-22T07:28:03.201 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.200+0000 7f3a55a65640 1 -- 172.21.15.63:0/3786339076 wait complete. 2024-03-22T07:28:03.211 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 43 2024-03-22T07:28:03.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.311+0000 7f752fe2e640 1 -- 172.21.15.63:0/928707851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75280fd3e0 msgr2=0x7f75280fd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:03.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.311+0000 7f752fe2e640 1 --2- 172.21.15.63:0/928707851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75280fd3e0 0x7f75280fd7b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f75240048d0 tx=0x7f7524009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.311+0000 7f752fe2e640 1 -- 172.21.15.63:0/928707851 shutdown_connections 2024-03-22T07:28:03.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.311+0000 7f752fe2e640 1 --2- 172.21.15.63:0/928707851 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75280fdcf0 0x7f7528106450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.311+0000 7f752fe2e640 1 --2- 172.21.15.63:0/928707851 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75280fd3e0 0x7f75280fd7b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.311+0000 7f752fe2e640 1 -- 172.21.15.63:0/928707851 >> 172.21.15.63:0/928707851 conn(0x7f75280f6ea0 msgr2=0x7f75280f92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:03.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.311+0000 7f752fe2e640 1 -- 172.21.15.63:0/928707851 shutdown_connections 2024-03-22T07:28:03.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.311+0000 7f752fe2e640 1 -- 172.21.15.63:0/928707851 wait complete. 2024-03-22T07:28:03.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.311+0000 7f752fe2e640 1 Processor -- start 2024-03-22T07:28:03.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.311+0000 7f752fe2e640 1 -- start start 2024-03-22T07:28:03.313 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.312+0000 7f752fe2e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75280fd3e0 0x7f7528112fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:03.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.312+0000 7f752fe2e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75280fdcf0 0x7f75281134e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:03.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.312+0000 7f752fe2e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7528113ae0 con 0x7f75280fd3e0 2024-03-22T07:28:03.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.312+0000 7f752fe2e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752810d130 con 0x7f75280fdcf0 2024-03-22T07:28:03.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.312+0000 7f752dbca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75280fd3e0 0x7f7528112fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:03.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.312+0000 7f752dbca640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75280fd3e0 0x7f7528112fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42776/0 (socket says 172.21.15.63:42776) 2024-03-22T07:28:03.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.312+0000 7f752dbca640 1 -- 172.21.15.63:0/882383463 learned_addr learned my addr 172.21.15.63:0/882383463 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:03.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.312+0000 7f752d3c9640 1 --2- 172.21.15.63:0/882383463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75280fdcf0 0x7f75281134e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:03.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.312+0000 7f752dbca640 1 -- 172.21.15.63:0/882383463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75280fdcf0 msgr2=0x7f75281134e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:03.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.312+0000 7f752dbca640 1 --2- 172.21.15.63:0/882383463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75280fdcf0 0x7f75281134e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.312+0000 7f752dbca640 1 -- 172.21.15.63:0/882383463 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7524005040 con 0x7f75280fd3e0 2024-03-22T07:28:03.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.312+0000 7f752d3c9640 1 --2- 172.21.15.63:0/882383463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75280fdcf0 0x7f75281134e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:28:03.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.312+0000 7f752dbca640 1 --2- 172.21.15.63:0/882383463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75280fd3e0 0x7f7528112fa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7524004930 tx=0x7f7524009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:03.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.312+0000 7f751a7fc640 1 -- 172.21.15.63:0/882383463 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75240094c0 con 0x7f75280fd3e0 2024-03-22T07:28:03.314 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.312+0000 7f751a7fc640 1 -- 172.21.15.63:0/882383463 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7524032e50 con 0x7f75280fd3e0 2024-03-22T07:28:03.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.312+0000 7f752fe2e640 1 -- 172.21.15.63:0/882383463 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752810d3b0 con 0x7f75280fd3e0 2024-03-22T07:28:03.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.312+0000 7f751a7fc640 1 -- 172.21.15.63:0/882383463 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75240433b0 con 0x7f75280fd3e0 2024-03-22T07:28:03.315 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.312+0000 7f752fe2e640 1 -- 172.21.15.63:0/882383463 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752810d870 con 0x7f75280fd3e0 2024-03-22T07:28:03.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.313+0000 7f752fe2e640 1 -- 172.21.15.63:0/882383463 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7528065d50 con 0x7f75280fd3e0 2024-03-22T07:28:03.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.314+0000 7f751a7fc640 1 -- 172.21.15.63:0/882383463 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f752402f070 con 0x7f75280fd3e0 2024-03-22T07:28:03.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.314+0000 7f751a7fc640 1 --2- 172.21.15.63:0/882383463 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f751406d2f0 0x7f751406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:03.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.314+0000 7f751a7fc640 1 -- 172.21.15.63:0/882383463 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f75240ae650 con 0x7f75280fd3e0 2024-03-22T07:28:03.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.314+0000 7f752d3c9640 1 --2- 172.21.15.63:0/882383463 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f751406d2f0 0x7f751406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:03.316 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.315+0000 7f752d3c9640 1 --2- 172.21.15.63:0/882383463 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f751406d2f0 0x7f751406f7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f751c009af0 tx=0x7f751c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:03.319 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.319+0000 7f751a7fc640 1 -- 172.21.15.63:0/882383463 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7524078080 con 0x7f75280fd3e0 2024-03-22T07:28:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:03 smithi063 ceph-mon[112784]: pgmap v265: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:28:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:03 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/1052451493' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-03-22T07:28:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:03 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/706179168' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-03-22T07:28:03.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:03 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2506715598' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-03-22T07:28:03.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:03 smithi069 ceph-mon[83631]: pgmap v265: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-03-22T07:28:03.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:03 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/1052451493' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-03-22T07:28:03.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:03 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/706179168' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-03-22T07:28:03.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:03 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2506715598' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-03-22T07:28:03.540 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.540+0000 7f752fe2e640 1 -- 172.21.15.63:0/882383463 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 43, "format": "json"} v 0) v1 -- 0x7f75280638c0 con 0x7f75280fd3e0 2024-03-22T07:28:03.541 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.541+0000 7f751a7fc640 1 -- 172.21.15.63:0/882383463 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 43, "format": "json"}]=0 dumped fsmap epoch 43 v44) v1 ==== 107+0+5011 (secure 0 0 0) 0x7f752407b740 con 0x7f75280fd3e0 2024-03-22T07:28:03.541 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:28:03.541 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44323,"name":"cephfs.smithi069.wwtsjn","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6825/653947973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":653947973},{"type":"v1","addr":"172.21.15.69:6825","nonce":653947973}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":33},{"gid":44327,"name":"cephfs.smithi069.cijxah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6827/3465454734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":3465454734},{"type":"v1","addr":"172.21.15.69:6827","nonce":3465454734}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-22T07:13:29.602905+0000","modified":"2024-03-22T07:26:21.431422+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34400,"mds_1":34424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34400":{"gid":34400,"name":"cephfs.smithi063.oytapz","rank":0,"incarnation":37,"state":"up:active","state_seq":22,"addr":"172.21.15.63:6827/1171423403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6826","nonce":1171423403},{"type":"v1","addr":"172.21.15.63:6827","nonce":1171423403}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34424":{"gid":34424,"name":"cephfs.smithi063.xeelpk","rank":1,"incarnation":37,"state":"up:active","state_seq":16,"addr":"172.21.15.63:6829/1190308435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6828","nonce":1190308435},{"type":"v1","addr":"172.21.15.63:6829","nonce":1190308435}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-22T07:28:03.541 INFO:teuthology.orchestra.run.smithi063.stderr:dumped fsmap epoch 43 2024-03-22T07:28:03.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.543+0000 7f752fe2e640 1 -- 172.21.15.63:0/882383463 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f751406d2f0 msgr2=0x7f751406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:03.543 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.543+0000 7f752fe2e640 1 --2- 172.21.15.63:0/882383463 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f751406d2f0 0x7f751406f7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f751c009af0 tx=0x7f751c00b040 comp rx=0 tx=0).stop 2024-03-22T07:28:03.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.543+0000 7f752fe2e640 1 -- 172.21.15.63:0/882383463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75280fd3e0 msgr2=0x7f7528112fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:03.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.543+0000 7f752fe2e640 1 --2- 172.21.15.63:0/882383463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75280fd3e0 0x7f7528112fa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7524004930 tx=0x7f7524009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.543+0000 7f752fe2e640 1 -- 172.21.15.63:0/882383463 shutdown_connections 2024-03-22T07:28:03.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.543+0000 7f752fe2e640 1 --2- 172.21.15.63:0/882383463 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f751406d2f0 0x7f751406f7a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.543+0000 7f752fe2e640 1 --2- 172.21.15.63:0/882383463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75280fdcf0 0x7f75281134e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.543+0000 7f752fe2e640 1 --2- 172.21.15.63:0/882383463 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f75280fd3e0 0x7f7528112fa0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.543+0000 7f752fe2e640 1 -- 172.21.15.63:0/882383463 >> 172.21.15.63:0/882383463 conn(0x7f75280f6ea0 msgr2=0x7f75280f8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:03.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.543+0000 7f752fe2e640 1 -- 172.21.15.63:0/882383463 shutdown_connections 2024-03-22T07:28:03.544 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.543+0000 7f752fe2e640 1 -- 172.21.15.63:0/882383463 wait complete. 2024-03-22T07:28:03.554 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-22T07:28:03.565 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-22T07:28:03.565 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T07:28:03.566 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T07:28:03.582 DEBUG:teuthology.orchestra.run.smithi063:> set -ex 2024-03-22T07:28:03.582 DEBUG:teuthology.orchestra.run.smithi063:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T07:28:03.637 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T07:28:03.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.776+0000 7fdf4e39d640 1 -- 172.21.15.63:0/2368677503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf48102b50 msgr2=0x7fdf480fc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:03.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.776+0000 7fdf4e39d640 1 --2- 172.21.15.63:0/2368677503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf48102b50 0x7fdf480fc1e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c0048d0 tx=0x7fdf3c009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.778 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.776+0000 7fdf4e39d640 1 -- 172.21.15.63:0/2368677503 shutdown_connections 2024-03-22T07:28:03.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.776+0000 7fdf4e39d640 1 --2- 172.21.15.63:0/2368677503 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf48102b50 0x7fdf480fc1e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.776+0000 7fdf4e39d640 1 --2- 172.21.15.63:0/2368677503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf481021b0 0x7fdf48102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.776+0000 7fdf4e39d640 1 -- 172.21.15.63:0/2368677503 >> 172.21.15.63:0/2368677503 conn(0x7fdf480f7da0 msgr2=0x7fdf480fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:03.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.776+0000 7fdf4e39d640 1 -- 172.21.15.63:0/2368677503 shutdown_connections 2024-03-22T07:28:03.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.776+0000 7fdf4e39d640 1 -- 172.21.15.63:0/2368677503 wait complete. 2024-03-22T07:28:03.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.777+0000 7fdf4e39d640 1 Processor -- start 2024-03-22T07:28:03.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.777+0000 7fdf4e39d640 1 -- start start 2024-03-22T07:28:03.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.777+0000 7fdf4e39d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf481021b0 0x7fdf4806f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:03.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.777+0000 7fdf4e39d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf48102b50 0x7fdf4806fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:03.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.777+0000 7fdf4e39d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf480716b0 con 0x7fdf481021b0 2024-03-22T07:28:03.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.777+0000 7fdf4e39d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf48070250 con 0x7fdf48102b50 2024-03-22T07:28:03.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.777+0000 7fdf47fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf481021b0 0x7fdf4806f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:03.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.777+0000 7fdf47fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf481021b0 0x7fdf4806f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42794/0 (socket says 172.21.15.63:42794) 2024-03-22T07:28:03.779 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.777+0000 7fdf47fff640 1 -- 172.21.15.63:0/3738246400 learned_addr learned my addr 172.21.15.63:0/3738246400 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:03.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.777+0000 7fdf477fe640 1 --2- 172.21.15.63:0/3738246400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf48102b50 0x7fdf4806fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:03.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.778+0000 7fdf47fff640 1 -- 172.21.15.63:0/3738246400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf48102b50 msgr2=0x7fdf4806fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:03.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.778+0000 7fdf47fff640 1 --2- 172.21.15.63:0/3738246400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf48102b50 0x7fdf4806fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.778+0000 7fdf47fff640 1 -- 172.21.15.63:0/3738246400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c005040 con 0x7fdf481021b0 2024-03-22T07:28:03.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.778+0000 7fdf47fff640 1 --2- 172.21.15.63:0/3738246400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf481021b0 0x7fdf4806f7d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdf38002a50 tx=0x7fdf38004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:03.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.778+0000 7fdf44ff9640 1 -- 172.21.15.63:0/3738246400 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf38007c30 con 0x7fdf481021b0 2024-03-22T07:28:03.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.778+0000 7fdf44ff9640 1 -- 172.21.15.63:0/3738246400 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf3800f040 con 0x7fdf481021b0 2024-03-22T07:28:03.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.778+0000 7fdf4e39d640 1 -- 172.21.15.63:0/3738246400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf48070530 con 0x7fdf481021b0 2024-03-22T07:28:03.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.778+0000 7fdf44ff9640 1 -- 172.21.15.63:0/3738246400 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf3800dbe0 con 0x7fdf481021b0 2024-03-22T07:28:03.780 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.778+0000 7fdf4e39d640 1 -- 172.21.15.63:0/3738246400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf48119420 con 0x7fdf481021b0 2024-03-22T07:28:03.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.779+0000 7fdf4e39d640 1 -- 172.21.15.63:0/3738246400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf480fd920 con 0x7fdf481021b0 2024-03-22T07:28:03.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.779+0000 7fdf44ff9640 1 -- 172.21.15.63:0/3738246400 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fdf3800dd40 con 0x7fdf481021b0 2024-03-22T07:28:03.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.780+0000 7fdf44ff9640 1 --2- 172.21.15.63:0/3738246400 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdf3006d2f0 0x7fdf3006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:03.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.780+0000 7fdf44ff9640 1 -- 172.21.15.63:0/3738246400 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdf3808c790 con 0x7fdf481021b0 2024-03-22T07:28:03.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.780+0000 7fdf477fe640 1 --2- 172.21.15.63:0/3738246400 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdf3006d2f0 0x7fdf3006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:03.781 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.780+0000 7fdf477fe640 1 --2- 172.21.15.63:0/3738246400 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdf3006d2f0 0x7fdf3006f7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdf48070f00 tx=0x7fdf3c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:03.784 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.784+0000 7fdf44ff9640 1 -- 172.21.15.63:0/3738246400 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdf38056240 con 0x7fdf481021b0 2024-03-22T07:28:03.965 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.965+0000 7fdf4e39d640 1 -- 172.21.15.63:0/3738246400 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf481092f0 con 0x7fdf481021b0 2024-03-22T07:28:03.966 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.966+0000 7fdf44ff9640 1 -- 172.21.15.63:0/3738246400 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16610 (secure 0 0 0) 0x7fdf38059900 con 0x7fdf481021b0 2024-03-22T07:28:03.967 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:28:03.967 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":87,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","created":"2024-03-22T07:06:38.004887+0000","modified":"2024-03-22T07:25:58.489902+0000","last_up_change":"2024-03-22T07:24:57.261366+0000","last_in_change":"2024-03-22T07:11:51.027495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T07:10:56.524122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-22T07:13:27.835268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-22T07:13:28.836648+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33ade431-7f97-43c7-bf9d-962c12d3ba7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":108390664},{"type":"v1","addr":"172.21.15.63:6803","nonce":108390664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":108390664},{"type":"v1","addr":"172.21.15.63:6805","nonce":108390664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":108390664},{"type":"v1","addr":"172.21.15.63:6809","nonce":108390664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":108390664},{"type":"v1","addr":"172.21.15.63:6807","nonce":108390664}]},"public_addr":"172.21.15.63:6803/108390664","cluster_addr":"172.21.15.63:6805/108390664","heartbeat_back_addr":"172.21.15.63:6809/108390664","heartbeat_front_addr":"172.21.15.63:6807/108390664","state":["exists","up"]},{"osd":1,"uuid":"0643ddb6-db9d-464c-8236-3aea2893de83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":3855758242},{"type":"v1","addr":"172.21.15.63:6811","nonce":3855758242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3855758242},{"type":"v1","addr":"172.21.15.63:6813","nonce":3855758242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3855758242},{"type":"v1","addr":"172.21.15.63:6817","nonce":3855758242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":3855758242},{"type":"v1","addr":"172.21.15.63:6815","nonce":3855758242}]},"public_addr":"172.21.15.63:6811/3855758242","cluster_addr":"172.21.15.63:6813/3855758242","heartbeat_back_addr":"172.21.15.63:6817/3855758242","heartbeat_front_addr":"172.21.15.63:6815/3855758242","state":["exists","up"]},{"osd":2,"uuid":"140b09d0-1fb6-4a3b-873a-048b797c0180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":2063458996},{"type":"v1","addr":"172.21.15.63:6819","nonce":2063458996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":2063458996},{"type":"v1","addr":"172.21.15.63:6821","nonce":2063458996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2063458996},{"type":"v1","addr":"172.21.15.63:6825","nonce":2063458996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":2063458996},{"type":"v1","addr":"172.21.15.63:6823","nonce":2063458996}]},"public_addr":"172.21.15.63:6819/2063458996","cluster_addr":"172.21.15.63:6821/2063458996","heartbeat_back_addr":"172.21.15.63:6825/2063458996","heartbeat_front_addr":"172.21.15.63:6823/2063458996","state":["exists","up"]},{"osd":3,"uuid":"1d92b77e-6bcc-4585-96f1-72251303cc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":2989645370},{"type":"v1","addr":"172.21.15.69:6801","nonce":2989645370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2989645370},{"type":"v1","addr":"172.21.15.69:6803","nonce":2989645370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2989645370},{"type":"v1","addr":"172.21.15.69:6807","nonce":2989645370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2989645370},{"type":"v1","addr":"172.21.15.69:6805","nonce":2989645370}]},"public_addr":"172.21.15.69:6801/2989645370","cluster_addr":"172.21.15.69:6803/2989645370","heartbeat_back_addr":"172.21.15.69:6807/2989645370","heartbeat_front_addr":"172.21.15.69:6805/2989645370","state":["exists","up"]},{"osd":4,"uuid":"28bdc8bc-41d5-4c01-9b29-d5833223d1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2629704021},{"type":"v1","addr":"172.21.15.69:6809","nonce":2629704021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2629704021},{"type":"v1","addr":"172.21.15.69:6811","nonce":2629704021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2629704021},{"type":"v1","addr":"172.21.15.69:6815","nonce":2629704021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2629704021},{"type":"v1","addr":"172.21.15.69:6813","nonce":2629704021}]},"public_addr":"172.21.15.69:6809/2629704021","cluster_addr":"172.21.15.69:6811/2629704021","heartbeat_back_addr":"172.21.15.69:6815/2629704021","heartbeat_front_addr":"172.21.15.69:6813/2629704021","state":["exists","up"]},{"osd":5,"uuid":"566db34e-ce0e-436f-aa90-df2c80d812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1480046180},{"type":"v1","addr":"172.21.15.69:6817","nonce":1480046180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1480046180},{"type":"v1","addr":"172.21.15.69:6819","nonce":1480046180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1480046180},{"type":"v1","addr":"172.21.15.69:6823","nonce":1480046180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1480046180},{"type":"v1","addr":"172.21.15.69:6821","nonce":1480046180}]},"public_addr":"172.21.15.69:6817/1480046180","cluster_addr":"172.21.15.69:6819/1480046180","heartbeat_back_addr":"172.21.15.69:6823/1480046180","heartbeat_front_addr":"172.21.15.69:6821/1480046180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:09:56.132204+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:10:24.051511+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:10:52.969047+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:11:17.576349+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:11:42.883846+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:12:08.488698+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6829/1373266298":"2024-03-23T07:25:00.991980+0000","172.21.15.63:6828/1373266298":"2024-03-23T07:25:00.991980+0000","172.21.15.63:0/1649622442":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/47408280":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/3629523167":"2024-03-23T07:20:31.385587+0000","172.21.15.69:0/288512401":"2024-03-23T07:17:44.994340+0000","172.21.15.63:0/424866385":"2024-03-23T07:07:10.417547+0000","172.21.15.63:6801/480818549":"2024-03-23T07:07:41.261927+0000","172.21.15.69:0/1072916585":"2024-03-23T07:17:44.994340+0000","172.21.15.63:6800/480818549":"2024-03-23T07:07:41.261927+0000","172.21.15.69:6828/1862357042":"2024-03-23T07:17:44.994340+0000","172.21.15.63:0/2503315479":"2024-03-23T07:07:41.261927+0000","172.21.15.69:6825/3798247699":"2024-03-23T07:25:00.991995+0000","172.21.15.63:0/1701838":"2024-03-23T07:16:00.292439+0000","172.21.15.63:6801/2616455474":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/1818534437":"2024-03-23T07:07:41.261927+0000","172.21.15.63:6800/2296442540":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/1323074353":"2024-03-23T07:07:41.261927+0000","172.21.15.63:6800/2616455474":"2024-03-23T07:07:10.417547+0000","172.21.15.63:6801/2390873586":"2024-03-23T07:20:31.385587+0000","172.21.15.63:6826/1314414011":"2024-03-23T07:13:45.306403+0000","172.21.15.63:6827/1314414011":"2024-03-23T07:13:45.306403+0000","172.21.15.69:0/547111417":"2024-03-23T07:17:44.994340+0000","172.21.15.69:0/2694993342":"2024-03-23T07:17:44.994340+0000","172.21.15.63:0/1602837747":"2024-03-23T07:20:31.385587+0000","172.21.15.63:0/108295103":"2024-03-23T07:16:00.292439+0000","172.21.15.63:6800/2001368263":"2024-03-23T07:16:00.292439+0000","172.21.15.63:6801/2001368263":"2024-03-23T07:16:00.292439+0000","172.21.15.63:0/466771007":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/2739978443":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/4021098270":"2024-03-23T07:20:31.385587+0000","172.21.15.63:6800/2390873586":"2024-03-23T07:20:31.385587+0000","172.21.15.63:6801/2296442540":"2024-03-23T07:08:44.186801+0000","172.21.15.69:6824/3798247699":"2024-03-23T07:25:00.991995+0000","172.21.15.63:0/3927392742":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/2835250916":"2024-03-23T07:16:00.292439+0000","172.21.15.69:6829/1862357042":"2024-03-23T07:17:44.994340+0000","172.21.15.63:0/41352707":"2024-03-23T07:16:00.292439+0000","172.21.15.63:0/1215330295":"2024-03-23T07:20:31.385587+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T07:28:03.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.968+0000 7fdf4e39d640 1 -- 172.21.15.63:0/3738246400 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdf3006d2f0 msgr2=0x7fdf3006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:03.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.968+0000 7fdf4e39d640 1 --2- 172.21.15.63:0/3738246400 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdf3006d2f0 0x7fdf3006f7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdf48070f00 tx=0x7fdf3c009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.968+0000 7fdf4e39d640 1 -- 172.21.15.63:0/3738246400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf481021b0 msgr2=0x7fdf4806f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:03.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.968+0000 7fdf4e39d640 1 --2- 172.21.15.63:0/3738246400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf481021b0 0x7fdf4806f7d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdf38002a50 tx=0x7fdf38004600 comp rx=0 tx=0).stop 2024-03-22T07:28:03.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.968+0000 7fdf4e39d640 1 -- 172.21.15.63:0/3738246400 shutdown_connections 2024-03-22T07:28:03.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.968+0000 7fdf4e39d640 1 --2- 172.21.15.63:0/3738246400 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdf3006d2f0 0x7fdf3006f7a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.968+0000 7fdf4e39d640 1 --2- 172.21.15.63:0/3738246400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdf48102b50 0x7fdf4806fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.968+0000 7fdf4e39d640 1 --2- 172.21.15.63:0/3738246400 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdf481021b0 0x7fdf4806f7d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:03.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.968+0000 7fdf4e39d640 1 -- 172.21.15.63:0/3738246400 >> 172.21.15.63:0/3738246400 conn(0x7fdf480f7da0 msgr2=0x7fdf480f8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:03.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.968+0000 7fdf4e39d640 1 -- 172.21.15.63:0/3738246400 shutdown_connections 2024-03-22T07:28:03.969 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:03.968+0000 7fdf4e39d640 1 -- 172.21.15.63:0/3738246400 wait complete. 2024-03-22T07:28:03.979 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T07:10:56.524122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-22T07:13:27.835268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-22T07:13:28.836648+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T07:28:03.980 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T07:28:04.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.081+0000 7fdef8aae640 1 -- 172.21.15.63:0/3044109078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdef40fdd00 msgr2=0x7fdef4106460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:04.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.081+0000 7fdef8aae640 1 --2- 172.21.15.63:0/3044109078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdef40fdd00 0x7fdef4106460 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0048d0 tx=0x7fdedc009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.081+0000 7fdef8aae640 1 -- 172.21.15.63:0/3044109078 shutdown_connections 2024-03-22T07:28:04.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.081+0000 7fdef8aae640 1 --2- 172.21.15.63:0/3044109078 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdef40fdd00 0x7fdef4106460 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.081+0000 7fdef8aae640 1 --2- 172.21.15.63:0/3044109078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdef40fd3f0 0x7fdef40fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.081+0000 7fdef8aae640 1 -- 172.21.15.63:0/3044109078 >> 172.21.15.63:0/3044109078 conn(0x7fdef40f6eb0 msgr2=0x7fdef40f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:04.083 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.081+0000 7fdef8aae640 1 -- 172.21.15.63:0/3044109078 shutdown_connections 2024-03-22T07:28:04.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.081+0000 7fdef8aae640 1 -- 172.21.15.63:0/3044109078 wait complete. 2024-03-22T07:28:04.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.081+0000 7fdef8aae640 1 Processor -- start 2024-03-22T07:28:04.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.081+0000 7fdef8aae640 1 -- start start 2024-03-22T07:28:04.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.082+0000 7fdef8aae640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdef40fd3f0 0x7fdef4112fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:04.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.082+0000 7fdef8aae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdef40fdd00 0x7fdef41134f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:04.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.082+0000 7fdef8aae640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef4113af0 con 0x7fdef40fd3f0 2024-03-22T07:28:04.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.082+0000 7fdef8aae640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef410d140 con 0x7fdef40fdd00 2024-03-22T07:28:04.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.082+0000 7fdef259c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdef40fd3f0 0x7fdef4112fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:04.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.082+0000 7fdef259c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdef40fd3f0 0x7fdef4112fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42822/0 (socket says 172.21.15.63:42822) 2024-03-22T07:28:04.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.082+0000 7fdef259c640 1 -- 172.21.15.63:0/1410025405 learned_addr learned my addr 172.21.15.63:0/1410025405 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:04.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.082+0000 7fdeebfff640 1 --2- 172.21.15.63:0/1410025405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdef40fdd00 0x7fdef41134f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:04.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.082+0000 7fdef259c640 1 -- 172.21.15.63:0/1410025405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdef40fdd00 msgr2=0x7fdef41134f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:04.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.082+0000 7fdef259c640 1 --2- 172.21.15.63:0/1410025405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdef40fdd00 0x7fdef41134f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.082+0000 7fdef259c640 1 -- 172.21.15.63:0/1410025405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdedc005040 con 0x7fdef40fd3f0 2024-03-22T07:28:04.084 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.082+0000 7fdeebfff640 1 --2- 172.21.15.63:0/1410025405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdef40fdd00 0x7fdef41134f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:28:04.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.082+0000 7fdef259c640 1 --2- 172.21.15.63:0/1410025405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdef40fd3f0 0x7fdef4112fb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdee400eb70 tx=0x7fdee400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:04.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.082+0000 7fdeeaffd640 1 -- 172.21.15.63:0/1410025405 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdee4007c30 con 0x7fdef40fd3f0 2024-03-22T07:28:04.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.082+0000 7fdef8aae640 1 -- 172.21.15.63:0/1410025405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef410d3c0 con 0x7fdef40fd3f0 2024-03-22T07:28:04.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.082+0000 7fdeeaffd640 1 -- 172.21.15.63:0/1410025405 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdee400f040 con 0x7fdef40fd3f0 2024-03-22T07:28:04.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.082+0000 7fdeeaffd640 1 -- 172.21.15.63:0/1410025405 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdee40095b0 con 0x7fdef40fd3f0 2024-03-22T07:28:04.085 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.082+0000 7fdef8aae640 1 -- 172.21.15.63:0/1410025405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef410d8e0 con 0x7fdef40fd3f0 2024-03-22T07:28:04.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.083+0000 7fdef8aae640 1 -- 172.21.15.63:0/1410025405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef4065dd0 con 0x7fdef40fd3f0 2024-03-22T07:28:04.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.084+0000 7fdeeaffd640 1 -- 172.21.15.63:0/1410025405 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fdee4009730 con 0x7fdef40fd3f0 2024-03-22T07:28:04.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.084+0000 7fdeeaffd640 1 --2- 172.21.15.63:0/1410025405 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdee006d2f0 0x7fdee006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:04.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.085+0000 7fdeeaffd640 1 -- 172.21.15.63:0/1410025405 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdee408c6f0 con 0x7fdef40fd3f0 2024-03-22T07:28:04.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.085+0000 7fdeebfff640 1 --2- 172.21.15.63:0/1410025405 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdee006d2f0 0x7fdee006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:04.086 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.085+0000 7fdeebfff640 1 --2- 172.21.15.63:0/1410025405 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdee006d2f0 0x7fdee006f7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdedc000c00 tx=0x7fdedc009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:04.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.089+0000 7fdeeaffd640 1 -- 172.21.15.63:0/1410025405 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdee4056120 con 0x7fdef40fd3f0 2024-03-22T07:28:04.269 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.269+0000 7fdef8aae640 1 -- 172.21.15.63:0/1410025405 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdef4063950 con 0x7fdef40fd3f0 2024-03-22T07:28:04.270 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.270+0000 7fdeeaffd640 1 -- 172.21.15.63:0/1410025405 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdee40597e0 con 0x7fdef40fd3f0 2024-03-22T07:28:04.270 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-03-22T07:28:04.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.272+0000 7fdef8aae640 1 -- 172.21.15.63:0/1410025405 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdee006d2f0 msgr2=0x7fdee006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:04.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.272+0000 7fdef8aae640 1 --2- 172.21.15.63:0/1410025405 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdee006d2f0 0x7fdee006f7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdedc000c00 tx=0x7fdedc009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.272+0000 7fdef8aae640 1 -- 172.21.15.63:0/1410025405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdef40fd3f0 msgr2=0x7fdef4112fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:04.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.272+0000 7fdef8aae640 1 --2- 172.21.15.63:0/1410025405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdef40fd3f0 0x7fdef4112fb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdee400eb70 tx=0x7fdee400b040 comp rx=0 tx=0).stop 2024-03-22T07:28:04.272 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.272+0000 7fdef8aae640 1 -- 172.21.15.63:0/1410025405 shutdown_connections 2024-03-22T07:28:04.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.272+0000 7fdef8aae640 1 --2- 172.21.15.63:0/1410025405 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fdee006d2f0 0x7fdee006f7a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.272+0000 7fdef8aae640 1 --2- 172.21.15.63:0/1410025405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdef40fdd00 0x7fdef41134f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.272+0000 7fdef8aae640 1 --2- 172.21.15.63:0/1410025405 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fdef40fd3f0 0x7fdef4112fb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.272+0000 7fdef8aae640 1 -- 172.21.15.63:0/1410025405 >> 172.21.15.63:0/1410025405 conn(0x7fdef40f6eb0 msgr2=0x7fdef40f83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:04.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.272+0000 7fdef8aae640 1 -- 172.21.15.63:0/1410025405 shutdown_connections 2024-03-22T07:28:04.273 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.272+0000 7fdef8aae640 1 -- 172.21.15.63:0/1410025405 wait complete. 2024-03-22T07:28:04.281 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:04 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3786339076' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 42, "format": "json"}]: dispatch 2024-03-22T07:28:04.281 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:04 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/882383463' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 43, "format": "json"}]: dispatch 2024-03-22T07:28:04.281 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:04 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3738246400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:28:04.282 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-22T07:28:04.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.383+0000 7f11350ae640 1 -- 172.21.15.63:0/3479785001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11301012e0 msgr2=0x7f11301016b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:04.385 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.383+0000 7f11350ae640 1 --2- 172.21.15.63:0/3479785001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11301012e0 0x7f11301016b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f11280048d0 tx=0x7f1128009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.383+0000 7f11350ae640 1 -- 172.21.15.63:0/3479785001 shutdown_connections 2024-03-22T07:28:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.383+0000 7f11350ae640 1 --2- 172.21.15.63:0/3479785001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1130101c80 0x7f11300fb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.383+0000 7f11350ae640 1 --2- 172.21.15.63:0/3479785001 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f11301012e0 0x7f11301016b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.383+0000 7f11350ae640 1 -- 172.21.15.63:0/3479785001 >> 172.21.15.63:0/3479785001 conn(0x7f11300f6e70 msgr2=0x7f11300f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.383+0000 7f11350ae640 1 -- 172.21.15.63:0/3479785001 shutdown_connections 2024-03-22T07:28:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.383+0000 7f11350ae640 1 -- 172.21.15.63:0/3479785001 wait complete. 2024-03-22T07:28:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.384+0000 7f11350ae640 1 Processor -- start 2024-03-22T07:28:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.384+0000 7f11350ae640 1 -- start start 2024-03-22T07:28:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.384+0000 7f11350ae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11301012e0 0x7f1130111df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.384+0000 7f11350ae640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1130101c80 0x7f1130112330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.384+0000 7f11350ae640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1130112930 con 0x7f1130101c80 2024-03-22T07:28:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.384+0000 7f11350ae640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1130112aa0 con 0x7f11301012e0 2024-03-22T07:28:04.386 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.384+0000 7f112e59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1130101c80 0x7f1130112330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:04.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.384+0000 7f112e59c640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1130101c80 0x7f1130112330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:42842/0 (socket says 172.21.15.63:42842) 2024-03-22T07:28:04.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.384+0000 7f112e59c640 1 -- 172.21.15.63:0/727649883 learned_addr learned my addr 172.21.15.63:0/727649883 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:04.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.385+0000 7f112ed9d640 1 --2- 172.21.15.63:0/727649883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11301012e0 0x7f1130111df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:04.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.385+0000 7f112e59c640 1 -- 172.21.15.63:0/727649883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11301012e0 msgr2=0x7f1130111df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:04.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.385+0000 7f112e59c640 1 --2- 172.21.15.63:0/727649883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11301012e0 0x7f1130111df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.385+0000 7f112e59c640 1 -- 172.21.15.63:0/727649883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1128005040 con 0x7f1130101c80 2024-03-22T07:28:04.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.385+0000 7f112e59c640 1 --2- 172.21.15.63:0/727649883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1130101c80 0x7f1130112330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f112400fce0 tx=0x7f1124004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:04.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.385+0000 7f11177fe640 1 -- 172.21.15.63:0/727649883 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1124007a20 con 0x7f1130101c80 2024-03-22T07:28:04.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.385+0000 7f11177fe640 1 -- 172.21.15.63:0/727649883 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1124007b80 con 0x7f1130101c80 2024-03-22T07:28:04.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.385+0000 7f11350ae640 1 -- 172.21.15.63:0/727649883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f113010c130 con 0x7f1130101c80 2024-03-22T07:28:04.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.385+0000 7f11177fe640 1 -- 172.21.15.63:0/727649883 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11240127a0 con 0x7f1130101c80 2024-03-22T07:28:04.387 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.385+0000 7f11350ae640 1 -- 172.21.15.63:0/727649883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f113010c650 con 0x7f1130101c80 2024-03-22T07:28:04.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.386+0000 7f11350ae640 1 -- 172.21.15.63:0/727649883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11300fca50 con 0x7f1130101c80 2024-03-22T07:28:04.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.387+0000 7f11177fe640 1 -- 172.21.15.63:0/727649883 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f1124007cf0 con 0x7f1130101c80 2024-03-22T07:28:04.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.387+0000 7f11177fe640 1 --2- 172.21.15.63:0/727649883 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f111806d2f0 0x7f111806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:04.388 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.387+0000 7f11177fe640 1 -- 172.21.15.63:0/727649883 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f112408ef20 con 0x7f1130101c80 2024-03-22T07:28:04.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.387+0000 7f112ed9d640 1 --2- 172.21.15.63:0/727649883 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f111806d2f0 0x7f111806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:04.389 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.388+0000 7f112ed9d640 1 --2- 172.21.15.63:0/727649883 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f111806d2f0 0x7f111806f7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1128003080 tx=0x7f1128009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:04.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.392+0000 7f11177fe640 1 -- 172.21.15.63:0/727649883 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f11240c2d20 con 0x7f1130101c80 2024-03-22T07:28:04.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:04 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3786339076' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 42, "format": "json"}]: dispatch 2024-03-22T07:28:04.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:04 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/882383463' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 43, "format": "json"}]: dispatch 2024-03-22T07:28:04.530 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:04 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3738246400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:28:04.573 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.573+0000 7f11350ae640 1 -- 172.21.15.63:0/727649883 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f113004bcc0 con 0x7f1130101c80 2024-03-22T07:28:04.574 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.573+0000 7f11177fe640 1 -- 172.21.15.63:0/727649883 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7f1124058a00 con 0x7f1130101c80 2024-03-22T07:28:04.574 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-03-22T07:28:04.576 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.576+0000 7f11350ae640 1 -- 172.21.15.63:0/727649883 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f111806d2f0 msgr2=0x7f111806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.576+0000 7f11350ae640 1 --2- 172.21.15.63:0/727649883 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f111806d2f0 0x7f111806f7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1128003080 tx=0x7f1128009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.576+0000 7f11350ae640 1 -- 172.21.15.63:0/727649883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1130101c80 msgr2=0x7f1130112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.576+0000 7f11350ae640 1 --2- 172.21.15.63:0/727649883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1130101c80 0x7f1130112330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f112400fce0 tx=0x7f1124004600 comp rx=0 tx=0).stop 2024-03-22T07:28:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.576+0000 7f11350ae640 1 -- 172.21.15.63:0/727649883 shutdown_connections 2024-03-22T07:28:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.576+0000 7f11350ae640 1 --2- 172.21.15.63:0/727649883 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1130101c80 0x7f1130112330 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.576+0000 7f11350ae640 1 --2- 172.21.15.63:0/727649883 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f111806d2f0 0x7f111806f7a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.577 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.576+0000 7f11350ae640 1 --2- 172.21.15.63:0/727649883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11301012e0 0x7f1130111df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.576+0000 7f11350ae640 1 -- 172.21.15.63:0/727649883 >> 172.21.15.63:0/727649883 conn(0x7f11300f6e70 msgr2=0x7f11300f7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.576+0000 7f11350ae640 1 -- 172.21.15.63:0/727649883 shutdown_connections 2024-03-22T07:28:04.578 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.576+0000 7f11350ae640 1 -- 172.21.15.63:0/727649883 wait complete. 2024-03-22T07:28:04.586 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-22T07:28:04.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.687+0000 7f0d77e2e640 1 -- 172.21.15.63:0/3545524326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d701012c0 msgr2=0x7f0d70101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:04.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.687+0000 7f0d77e2e640 1 --2- 172.21.15.63:0/3545524326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d701012c0 0x7f0d70101690 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0048d0 tx=0x7f0d6c009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.687+0000 7f0d77e2e640 1 -- 172.21.15.63:0/3545524326 shutdown_connections 2024-03-22T07:28:04.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.687+0000 7f0d77e2e640 1 --2- 172.21.15.63:0/3545524326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d70101c60 0x7f0d700fb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.687+0000 7f0d77e2e640 1 --2- 172.21.15.63:0/3545524326 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d701012c0 0x7f0d70101690 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.692 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.687+0000 7f0d77e2e640 1 -- 172.21.15.63:0/3545524326 >> 172.21.15.63:0/3545524326 conn(0x7f0d700f6eb0 msgr2=0x7f0d700f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:04.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.687+0000 7f0d77e2e640 1 -- 172.21.15.63:0/3545524326 shutdown_connections 2024-03-22T07:28:04.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.687+0000 7f0d77e2e640 1 -- 172.21.15.63:0/3545524326 wait complete. 2024-03-22T07:28:04.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.687+0000 7f0d77e2e640 1 Processor -- start 2024-03-22T07:28:04.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.688+0000 7f0d77e2e640 1 -- start start 2024-03-22T07:28:04.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.688+0000 7f0d77e2e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d701012c0 0x7f0d7010e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:04.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.688+0000 7f0d77e2e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d70101c60 0x7f0d7010e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:04.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.688+0000 7f0d77e2e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d70066570 con 0x7f0d70101c60 2024-03-22T07:28:04.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.688+0000 7f0d77e2e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7010eb30 con 0x7f0d701012c0 2024-03-22T07:28:04.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.688+0000 7f0d753c9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d70101c60 0x7f0d7010e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:04.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.688+0000 7f0d753c9640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d70101c60 0x7f0d7010e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:46998/0 (socket says 172.21.15.63:46998) 2024-03-22T07:28:04.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.688+0000 7f0d753c9640 1 -- 172.21.15.63:0/2928047124 learned_addr learned my addr 172.21.15.63:0/2928047124 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:04.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.688+0000 7f0d75bca640 1 --2- 172.21.15.63:0/2928047124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d701012c0 0x7f0d7010e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:04.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.688+0000 7f0d753c9640 1 -- 172.21.15.63:0/2928047124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d701012c0 msgr2=0x7f0d7010e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:04.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.688+0000 7f0d753c9640 1 --2- 172.21.15.63:0/2928047124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d701012c0 0x7f0d7010e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.688+0000 7f0d753c9640 1 -- 172.21.15.63:0/2928047124 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c005040 con 0x7f0d70101c60 2024-03-22T07:28:04.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.688+0000 7f0d75bca640 1 --2- 172.21.15.63:0/2928047124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d701012c0 0x7f0d7010e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:28:04.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.688+0000 7f0d753c9640 1 --2- 172.21.15.63:0/2928047124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d70101c60 0x7f0d7010e5f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000c830 tx=0x7f0d60004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:04.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.689+0000 7f0d667fc640 1 -- 172.21.15.63:0/2928047124 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d6000a850 con 0x7f0d70101c60 2024-03-22T07:28:04.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.689+0000 7f0d667fc640 1 -- 172.21.15.63:0/2928047124 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d6000ae80 con 0x7f0d70101c60 2024-03-22T07:28:04.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.689+0000 7f0d77e2e640 1 -- 172.21.15.63:0/2928047124 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7010ee10 con 0x7f0d70101c60 2024-03-22T07:28:04.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.689+0000 7f0d667fc640 1 -- 172.21.15.63:0/2928047124 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d60012ea0 con 0x7f0d70101c60 2024-03-22T07:28:04.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.689+0000 7f0d77e2e640 1 -- 172.21.15.63:0/2928047124 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d701173d0 con 0x7f0d70101c60 2024-03-22T07:28:04.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.690+0000 7f0d77e2e640 1 -- 172.21.15.63:0/2928047124 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d700fca30 con 0x7f0d70101c60 2024-03-22T07:28:04.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.690+0000 7f0d667fc640 1 -- 172.21.15.63:0/2928047124 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f0d60012400 con 0x7f0d70101c60 2024-03-22T07:28:04.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.691+0000 7f0d667fc640 1 --2- 172.21.15.63:0/2928047124 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f0d5c06d2f0 0x7f0d5c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:04.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.691+0000 7f0d667fc640 1 -- 172.21.15.63:0/2928047124 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f0d6008cae0 con 0x7f0d70101c60 2024-03-22T07:28:04.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.691+0000 7f0d75bca640 1 --2- 172.21.15.63:0/2928047124 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f0d5c06d2f0 0x7f0d5c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:04.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.691+0000 7f0d75bca640 1 --2- 172.21.15.63:0/2928047124 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f0d5c06d2f0 0x7f0d5c06f7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c000c00 tx=0x7f0d6c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:04.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.695+0000 7f0d667fc640 1 -- 172.21.15.63:0/2928047124 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0d60056630 con 0x7f0d70101c60 2024-03-22T07:28:04.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.877+0000 7f0d77e2e640 1 -- 172.21.15.63:0/2928047124 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f0d7004bcc0 con 0x7f0d70101c60 2024-03-22T07:28:04.878 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.878+0000 7f0d667fc640 1 -- 172.21.15.63:0/2928047124 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7f0d600126b0 con 0x7f0d70101c60 2024-03-22T07:28:04.878 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-03-22T07:28:04.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.880+0000 7f0d77e2e640 1 -- 172.21.15.63:0/2928047124 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f0d5c06d2f0 msgr2=0x7f0d5c06f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:04.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.880+0000 7f0d77e2e640 1 --2- 172.21.15.63:0/2928047124 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f0d5c06d2f0 0x7f0d5c06f7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c000c00 tx=0x7f0d6c009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.880+0000 7f0d77e2e640 1 -- 172.21.15.63:0/2928047124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d70101c60 msgr2=0x7f0d7010e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:04.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.880+0000 7f0d77e2e640 1 --2- 172.21.15.63:0/2928047124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d70101c60 0x7f0d7010e5f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000c830 tx=0x7f0d60004430 comp rx=0 tx=0).stop 2024-03-22T07:28:04.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.880+0000 7f0d77e2e640 1 -- 172.21.15.63:0/2928047124 shutdown_connections 2024-03-22T07:28:04.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.880+0000 7f0d77e2e640 1 --2- 172.21.15.63:0/2928047124 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f0d70101c60 0x7f0d7010e5f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.880+0000 7f0d77e2e640 1 --2- 172.21.15.63:0/2928047124 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f0d5c06d2f0 0x7f0d5c06f7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.880+0000 7f0d77e2e640 1 --2- 172.21.15.63:0/2928047124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0d701012c0 0x7f0d7010e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.880+0000 7f0d77e2e640 1 -- 172.21.15.63:0/2928047124 >> 172.21.15.63:0/2928047124 conn(0x7f0d700f6eb0 msgr2=0x7f0d700f7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:04.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.880+0000 7f0d77e2e640 1 -- 172.21.15.63:0/2928047124 shutdown_connections 2024-03-22T07:28:04.881 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.880+0000 7f0d77e2e640 1 -- 172.21.15.63:0/2928047124 wait complete. 2024-03-22T07:28:04.891 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T07:28:04.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.989+0000 7f89a195d640 1 -- 172.21.15.63:0/4184303250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f899c101c80 msgr2=0x7f899c0fb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:04.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.989+0000 7f89a195d640 1 --2- 172.21.15.63:0/4184303250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f899c101c80 0x7f899c0fb310 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f89840048d0 tx=0x7f8984009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.989+0000 7f89a195d640 1 -- 172.21.15.63:0/4184303250 shutdown_connections 2024-03-22T07:28:04.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.989+0000 7f89a195d640 1 --2- 172.21.15.63:0/4184303250 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f899c101c80 0x7f899c0fb310 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.989+0000 7f89a195d640 1 --2- 172.21.15.63:0/4184303250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f899c1012e0 0x7f899c1016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.989+0000 7f89a195d640 1 -- 172.21.15.63:0/4184303250 >> 172.21.15.63:0/4184303250 conn(0x7f899c0f6eb0 msgr2=0x7f899c0f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:04.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.989+0000 7f89a195d640 1 -- 172.21.15.63:0/4184303250 shutdown_connections 2024-03-22T07:28:04.991 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.989+0000 7f89a195d640 1 -- 172.21.15.63:0/4184303250 wait complete. 2024-03-22T07:28:04.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.989+0000 7f89a195d640 1 Processor -- start 2024-03-22T07:28:04.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f89a195d640 1 -- start start 2024-03-22T07:28:04.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f89a195d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f899c1012e0 0x7f899c112f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:04.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f89a195d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f899c101c80 0x7f899c113490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:04.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f89a195d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c113a90 con 0x7f899c101c80 2024-03-22T07:28:04.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f89a195d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c10d0e0 con 0x7f899c1012e0 2024-03-22T07:28:04.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f899a7fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f899c101c80 0x7f899c113490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:04.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f899a7fc640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f899c101c80 0x7f899c113490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47012/0 (socket says 172.21.15.63:47012) 2024-03-22T07:28:04.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f899a7fc640 1 -- 172.21.15.63:0/2814730016 learned_addr learned my addr 172.21.15.63:0/2814730016 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:04.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f899affd640 1 --2- 172.21.15.63:0/2814730016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f899c1012e0 0x7f899c112f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:04.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f899a7fc640 1 -- 172.21.15.63:0/2814730016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f899c1012e0 msgr2=0x7f899c112f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:04.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f899a7fc640 1 --2- 172.21.15.63:0/2814730016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f899c1012e0 0x7f899c112f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:04.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f899a7fc640 1 -- 172.21.15.63:0/2814730016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8984005040 con 0x7f899c101c80 2024-03-22T07:28:04.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f899affd640 1 --2- 172.21.15.63:0/2814730016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f899c1012e0 0x7f899c112f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:28:04.992 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f899a7fc640 1 --2- 172.21.15.63:0/2814730016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f899c101c80 0x7f899c113490 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8984004930 tx=0x7f8984009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:04.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f89937fe640 1 -- 172.21.15.63:0/2814730016 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89840094c0 con 0x7f899c101c80 2024-03-22T07:28:04.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f89937fe640 1 -- 172.21.15.63:0/2814730016 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8984032e50 con 0x7f899c101c80 2024-03-22T07:28:04.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f89a195d640 1 -- 172.21.15.63:0/2814730016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899c10d360 con 0x7f899c101c80 2024-03-22T07:28:04.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f89937fe640 1 -- 172.21.15.63:0/2814730016 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89840433b0 con 0x7f899c101c80 2024-03-22T07:28:04.993 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.990+0000 7f89a195d640 1 -- 172.21.15.63:0/2814730016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899c10d820 con 0x7f899c101c80 2024-03-22T07:28:04.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.992+0000 7f89a195d640 1 -- 172.21.15.63:0/2814730016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899c0fca50 con 0x7f899c101c80 2024-03-22T07:28:04.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.992+0000 7f89937fe640 1 -- 172.21.15.63:0/2814730016 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f898402f070 con 0x7f899c101c80 2024-03-22T07:28:04.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.992+0000 7f89937fe640 1 --2- 172.21.15.63:0/2814730016 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f898806d2f0 0x7f898806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:04.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.992+0000 7f89937fe640 1 -- 172.21.15.63:0/2814730016 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f89840ae840 con 0x7f899c101c80 2024-03-22T07:28:04.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.992+0000 7f899affd640 1 --2- 172.21.15.63:0/2814730016 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f898806d2f0 0x7f898806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:04.994 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.993+0000 7f899affd640 1 --2- 172.21.15.63:0/2814730016 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f898806d2f0 0x7f898806f7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f898c007fd0 tx=0x7f898c009e90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:04.996 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:04.997+0000 7f89937fe640 1 -- 172.21.15.63:0/2814730016 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8984078270 con 0x7f899c101c80 2024-03-22T07:28:05.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.194+0000 7f89a195d640 1 -- 172.21.15.63:0/2814730016 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f899c108440 con 0x7f899c101c80 2024-03-22T07:28:05.194 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.194+0000 7f89937fe640 1 -- 172.21.15.63:0/2814730016 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v87) v1 ==== 81+0+2302 (secure 0 0 0) 0x7f89840ace60 con 0x7f899c101c80 2024-03-22T07:28:05.195 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6829/1373266298 2024-03-23T07:25:00.991980+0000 2024-03-22T07:28:05.195 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6828/1373266298 2024-03-23T07:25:00.991980+0000 2024-03-22T07:28:05.195 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1649622442 2024-03-23T07:07:10.417547+0000 2024-03-22T07:28:05.195 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/47408280 2024-03-23T07:08:44.186801+0000 2024-03-22T07:28:05.195 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3629523167 2024-03-23T07:20:31.385587+0000 2024-03-22T07:28:05.195 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.69:0/288512401 2024-03-23T07:17:44.994340+0000 2024-03-22T07:28:05.195 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/424866385 2024-03-23T07:07:10.417547+0000 2024-03-22T07:28:05.195 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/480818549 2024-03-23T07:07:41.261927+0000 2024-03-22T07:28:05.195 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.69:0/1072916585 2024-03-23T07:17:44.994340+0000 2024-03-22T07:28:05.195 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/480818549 2024-03-23T07:07:41.261927+0000 2024-03-22T07:28:05.195 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.69:6828/1862357042 2024-03-23T07:17:44.994340+0000 2024-03-22T07:28:05.195 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2503315479 2024-03-23T07:07:41.261927+0000 2024-03-22T07:28:05.196 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.69:6825/3798247699 2024-03-23T07:25:00.991995+0000 2024-03-22T07:28:05.196 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1701838 2024-03-23T07:16:00.292439+0000 2024-03-22T07:28:05.196 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/2616455474 2024-03-23T07:07:10.417547+0000 2024-03-22T07:28:05.196 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1818534437 2024-03-23T07:07:41.261927+0000 2024-03-22T07:28:05.196 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/2296442540 2024-03-23T07:08:44.186801+0000 2024-03-22T07:28:05.196 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1323074353 2024-03-23T07:07:41.261927+0000 2024-03-22T07:28:05.196 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/2616455474 2024-03-23T07:07:10.417547+0000 2024-03-22T07:28:05.196 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/2390873586 2024-03-23T07:20:31.385587+0000 2024-03-22T07:28:05.196 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6826/1314414011 2024-03-23T07:13:45.306403+0000 2024-03-22T07:28:05.196 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6827/1314414011 2024-03-23T07:13:45.306403+0000 2024-03-22T07:28:05.196 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.69:0/547111417 2024-03-23T07:17:44.994340+0000 2024-03-22T07:28:05.196 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.69:0/2694993342 2024-03-23T07:17:44.994340+0000 2024-03-22T07:28:05.196 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1602837747 2024-03-23T07:20:31.385587+0000 2024-03-22T07:28:05.196 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/108295103 2024-03-23T07:16:00.292439+0000 2024-03-22T07:28:05.196 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/2001368263 2024-03-23T07:16:00.292439+0000 2024-03-22T07:28:05.196 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/2001368263 2024-03-23T07:16:00.292439+0000 2024-03-22T07:28:05.197 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/466771007 2024-03-23T07:08:44.186801+0000 2024-03-22T07:28:05.197 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2739978443 2024-03-23T07:07:10.417547+0000 2024-03-22T07:28:05.197 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/4021098270 2024-03-23T07:20:31.385587+0000 2024-03-22T07:28:05.197 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/2390873586 2024-03-23T07:20:31.385587+0000 2024-03-22T07:28:05.197 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/2296442540 2024-03-23T07:08:44.186801+0000 2024-03-22T07:28:05.197 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.69:6824/3798247699 2024-03-23T07:25:00.991995+0000 2024-03-22T07:28:05.197 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3927392742 2024-03-23T07:08:44.186801+0000 2024-03-22T07:28:05.197 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2835250916 2024-03-23T07:16:00.292439+0000 2024-03-22T07:28:05.197 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.69:6829/1862357042 2024-03-23T07:17:44.994340+0000 2024-03-22T07:28:05.197 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/41352707 2024-03-23T07:16:00.292439+0000 2024-03-22T07:28:05.197 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1215330295 2024-03-23T07:20:31.385587+0000 2024-03-22T07:28:05.198 INFO:teuthology.orchestra.run.smithi063.stderr:listed 39 entries 2024-03-22T07:28:05.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.196+0000 7f89a195d640 1 -- 172.21.15.63:0/2814730016 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f898806d2f0 msgr2=0x7f898806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:05.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.197+0000 7f89a195d640 1 --2- 172.21.15.63:0/2814730016 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f898806d2f0 0x7f898806f7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f898c007fd0 tx=0x7f898c009e90 comp rx=0 tx=0).stop 2024-03-22T07:28:05.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.197+0000 7f89a195d640 1 -- 172.21.15.63:0/2814730016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f899c101c80 msgr2=0x7f899c113490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:05.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.197+0000 7f89a195d640 1 --2- 172.21.15.63:0/2814730016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f899c101c80 0x7f899c113490 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8984004930 tx=0x7f8984009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.197+0000 7f89a195d640 1 -- 172.21.15.63:0/2814730016 shutdown_connections 2024-03-22T07:28:05.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.197+0000 7f89a195d640 1 --2- 172.21.15.63:0/2814730016 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f899c101c80 0x7f899c113490 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.198 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.197+0000 7f89a195d640 1 --2- 172.21.15.63:0/2814730016 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f898806d2f0 0x7f898806f7a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.197+0000 7f89a195d640 1 --2- 172.21.15.63:0/2814730016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f899c1012e0 0x7f899c112f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.197+0000 7f89a195d640 1 -- 172.21.15.63:0/2814730016 >> 172.21.15.63:0/2814730016 conn(0x7f899c0f6eb0 msgr2=0x7f899c0f8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:05.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.197+0000 7f89a195d640 1 -- 172.21.15.63:0/2814730016 shutdown_connections 2024-03-22T07:28:05.199 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.197+0000 7f89a195d640 1 -- 172.21.15.63:0/2814730016 wait complete. 2024-03-22T07:28:05.206 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:05 smithi063 ceph-mon[112784]: pgmap v266: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:28:05.206 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:05 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/1410025405' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:28:05.206 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:05 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/727649883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T07:28:05.206 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:05 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2928047124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T07:28:05.207 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-22T07:28:05.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:28:05.207 DEBUG:teuthology.orchestra.run.smithi063:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-22T07:28:05.235 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi063.front.sepia.ceph.com 2024-03-22T07:28:05.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:28:05.235 DEBUG:teuthology.orchestra.run.smithi063:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-22T07:28:05.300 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-22T07:28:05.301 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T07:28:05.321 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-03-22T07:28:05.322 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-22T07:28:05.380 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-22T07:28:05.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.479+0000 7f8c3e818640 1 -- 172.21.15.63:0/4150728835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c380fb110 msgr2=0x7f8c380fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:05.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.479+0000 7f8c3e818640 1 --2- 172.21.15.63:0/4150728835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c380fb110 0x7f8c380fb560 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c0048d0 tx=0x7f8c2c009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.479+0000 7f8c3e818640 1 -- 172.21.15.63:0/4150728835 shutdown_connections 2024-03-22T07:28:05.481 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.479+0000 7f8c3e818640 1 --2- 172.21.15.63:0/4150728835 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c380fb110 0x7f8c380fb560 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.479+0000 7f8c3e818640 1 --2- 172.21.15.63:0/4150728835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c38101140 0x7f8c38101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.479+0000 7f8c3e818640 1 -- 172.21.15.63:0/4150728835 >> 172.21.15.63:0/4150728835 conn(0x7f8c38071130 msgr2=0x7f8c38071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:05.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.479+0000 7f8c3e818640 1 -- 172.21.15.63:0/4150728835 shutdown_connections 2024-03-22T07:28:05.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.479+0000 7f8c3e818640 1 -- 172.21.15.63:0/4150728835 wait complete. 2024-03-22T07:28:05.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.480+0000 7f8c3e818640 1 Processor -- start 2024-03-22T07:28:05.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.480+0000 7f8c3e818640 1 -- start start 2024-03-22T07:28:05.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.480+0000 7f8c3e818640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c380fb110 0x7f8c38112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:05.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.480+0000 7f8c3e818640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c38101140 0x7f8c381131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:05.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.480+0000 7f8c3e818640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c381137e0 con 0x7f8c380fb110 2024-03-22T07:28:05.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.480+0000 7f8c3e818640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3810ce30 con 0x7f8c38101140 2024-03-22T07:28:05.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.480+0000 7f8c37fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c380fb110 0x7f8c38112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:05.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.480+0000 7f8c37fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c380fb110 0x7f8c38112ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47030/0 (socket says 172.21.15.63:47030) 2024-03-22T07:28:05.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.480+0000 7f8c37fff640 1 -- 172.21.15.63:0/3012342676 learned_addr learned my addr 172.21.15.63:0/3012342676 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:05.482 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.480+0000 7f8c377fe640 1 --2- 172.21.15.63:0/3012342676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c38101140 0x7f8c381131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:05.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.480+0000 7f8c37fff640 1 -- 172.21.15.63:0/3012342676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c38101140 msgr2=0x7f8c381131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:05.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.480+0000 7f8c37fff640 1 --2- 172.21.15.63:0/3012342676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c38101140 0x7f8c381131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.480+0000 7f8c37fff640 1 -- 172.21.15.63:0/3012342676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c005040 con 0x7f8c380fb110 2024-03-22T07:28:05.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.481+0000 7f8c377fe640 1 --2- 172.21.15.63:0/3012342676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c38101140 0x7f8c381131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-22T07:28:05.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.481+0000 7f8c37fff640 1 --2- 172.21.15.63:0/3012342676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c380fb110 0x7f8c38112ca0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800c830 tx=0x7f8c28004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:05.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.481+0000 7f8c34ff9640 1 -- 172.21.15.63:0/3012342676 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c2800a850 con 0x7f8c380fb110 2024-03-22T07:28:05.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.481+0000 7f8c3e818640 1 -- 172.21.15.63:0/3012342676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3810d110 con 0x7f8c380fb110 2024-03-22T07:28:05.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.481+0000 7f8c34ff9640 1 -- 172.21.15.63:0/3012342676 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c2800ae80 con 0x7f8c380fb110 2024-03-22T07:28:05.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.481+0000 7f8c34ff9640 1 -- 172.21.15.63:0/3012342676 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c28012ea0 con 0x7f8c380fb110 2024-03-22T07:28:05.483 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.481+0000 7f8c3e818640 1 -- 172.21.15.63:0/3012342676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3810d630 con 0x7f8c380fb110 2024-03-22T07:28:05.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.482+0000 7f8c3e818640 1 -- 172.21.15.63:0/3012342676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c380fc800 con 0x7f8c380fb110 2024-03-22T07:28:05.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.482+0000 7f8c34ff9640 1 -- 172.21.15.63:0/3012342676 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f8c28012400 con 0x7f8c380fb110 2024-03-22T07:28:05.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.483+0000 7f8c34ff9640 1 --2- 172.21.15.63:0/3012342676 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f8c2006d2f0 0x7f8c2006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:05.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.483+0000 7f8c377fe640 1 --2- 172.21.15.63:0/3012342676 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f8c2006d2f0 0x7f8c2006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:05.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.483+0000 7f8c34ff9640 1 -- 172.21.15.63:0/3012342676 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f8c2808cbb0 con 0x7f8c380fb110 2024-03-22T07:28:05.484 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.483+0000 7f8c377fe640 1 --2- 172.21.15.63:0/3012342676 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f8c2006d2f0 0x7f8c2006f7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009fa0 tx=0x7f8c2c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:05.487 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.488+0000 7f8c34ff9640 1 -- 172.21.15.63:0/3012342676 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8c28093080 con 0x7f8c380fb110 2024-03-22T07:28:05.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:05 smithi069 ceph-mon[83631]: pgmap v266: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-03-22T07:28:05.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:05 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/1410025405' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:28:05.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:05 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/727649883' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T07:28:05.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:05 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2928047124' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T07:28:05.669 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.669+0000 7f8c3e818640 1 -- 172.21.15.63:0/3012342676 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c3804bcc0 con 0x7f8c380fb110 2024-03-22T07:28:05.671 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.671+0000 7f8c34ff9640 1 -- 172.21.15.63:0/3012342676 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16610 (secure 0 0 0) 0x7f8c28056660 con 0x7f8c380fb110 2024-03-22T07:28:05.671 INFO:teuthology.orchestra.run.smithi063.stdout: 2024-03-22T07:28:05.671 INFO:teuthology.orchestra.run.smithi063.stdout:{"epoch":87,"fsid":"9a0ba9d6-e81a-11ee-95cd-87774f69a715","created":"2024-03-22T07:06:38.004887+0000","modified":"2024-03-22T07:25:58.489902+0000","last_up_change":"2024-03-22T07:24:57.261366+0000","last_in_change":"2024-03-22T07:11:51.027495+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-22T07:10:56.524122+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-22T07:13:27.835268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-22T07:13:28.836648+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33ade431-7f97-43c7-bf9d-962c12d3ba7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6802","nonce":108390664},{"type":"v1","addr":"172.21.15.63:6803","nonce":108390664}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6804","nonce":108390664},{"type":"v1","addr":"172.21.15.63:6805","nonce":108390664}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6808","nonce":108390664},{"type":"v1","addr":"172.21.15.63:6809","nonce":108390664}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6806","nonce":108390664},{"type":"v1","addr":"172.21.15.63:6807","nonce":108390664}]},"public_addr":"172.21.15.63:6803/108390664","cluster_addr":"172.21.15.63:6805/108390664","heartbeat_back_addr":"172.21.15.63:6809/108390664","heartbeat_front_addr":"172.21.15.63:6807/108390664","state":["exists","up"]},{"osd":1,"uuid":"0643ddb6-db9d-464c-8236-3aea2893de83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6810","nonce":3855758242},{"type":"v1","addr":"172.21.15.63:6811","nonce":3855758242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6812","nonce":3855758242},{"type":"v1","addr":"172.21.15.63:6813","nonce":3855758242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6816","nonce":3855758242},{"type":"v1","addr":"172.21.15.63:6817","nonce":3855758242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6814","nonce":3855758242},{"type":"v1","addr":"172.21.15.63:6815","nonce":3855758242}]},"public_addr":"172.21.15.63:6811/3855758242","cluster_addr":"172.21.15.63:6813/3855758242","heartbeat_back_addr":"172.21.15.63:6817/3855758242","heartbeat_front_addr":"172.21.15.63:6815/3855758242","state":["exists","up"]},{"osd":2,"uuid":"140b09d0-1fb6-4a3b-873a-048b797c0180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6818","nonce":2063458996},{"type":"v1","addr":"172.21.15.63:6819","nonce":2063458996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6820","nonce":2063458996},{"type":"v1","addr":"172.21.15.63:6821","nonce":2063458996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6824","nonce":2063458996},{"type":"v1","addr":"172.21.15.63:6825","nonce":2063458996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.63:6822","nonce":2063458996},{"type":"v1","addr":"172.21.15.63:6823","nonce":2063458996}]},"public_addr":"172.21.15.63:6819/2063458996","cluster_addr":"172.21.15.63:6821/2063458996","heartbeat_back_addr":"172.21.15.63:6825/2063458996","heartbeat_front_addr":"172.21.15.63:6823/2063458996","state":["exists","up"]},{"osd":3,"uuid":"1d92b77e-6bcc-4585-96f1-72251303cc9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":2989645370},{"type":"v1","addr":"172.21.15.69:6801","nonce":2989645370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":2989645370},{"type":"v1","addr":"172.21.15.69:6803","nonce":2989645370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":2989645370},{"type":"v1","addr":"172.21.15.69:6807","nonce":2989645370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":2989645370},{"type":"v1","addr":"172.21.15.69:6805","nonce":2989645370}]},"public_addr":"172.21.15.69:6801/2989645370","cluster_addr":"172.21.15.69:6803/2989645370","heartbeat_back_addr":"172.21.15.69:6807/2989645370","heartbeat_front_addr":"172.21.15.69:6805/2989645370","state":["exists","up"]},{"osd":4,"uuid":"28bdc8bc-41d5-4c01-9b29-d5833223d1af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":2629704021},{"type":"v1","addr":"172.21.15.69:6809","nonce":2629704021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":2629704021},{"type":"v1","addr":"172.21.15.69:6811","nonce":2629704021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":2629704021},{"type":"v1","addr":"172.21.15.69:6815","nonce":2629704021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":2629704021},{"type":"v1","addr":"172.21.15.69:6813","nonce":2629704021}]},"public_addr":"172.21.15.69:6809/2629704021","cluster_addr":"172.21.15.69:6811/2629704021","heartbeat_back_addr":"172.21.15.69:6815/2629704021","heartbeat_front_addr":"172.21.15.69:6813/2629704021","state":["exists","up"]},{"osd":5,"uuid":"566db34e-ce0e-436f-aa90-df2c80d812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":1480046180},{"type":"v1","addr":"172.21.15.69:6817","nonce":1480046180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":1480046180},{"type":"v1","addr":"172.21.15.69:6819","nonce":1480046180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":1480046180},{"type":"v1","addr":"172.21.15.69:6823","nonce":1480046180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":1480046180},{"type":"v1","addr":"172.21.15.69:6821","nonce":1480046180}]},"public_addr":"172.21.15.69:6817/1480046180","cluster_addr":"172.21.15.69:6819/1480046180","heartbeat_back_addr":"172.21.15.69:6823/1480046180","heartbeat_front_addr":"172.21.15.69:6821/1480046180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:09:56.132204+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:10:24.051511+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:10:52.969047+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:11:17.576349+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:11:42.883846+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-22T07:12:08.488698+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.63:6829/1373266298":"2024-03-23T07:25:00.991980+0000","172.21.15.63:6828/1373266298":"2024-03-23T07:25:00.991980+0000","172.21.15.63:0/1649622442":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/47408280":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/3629523167":"2024-03-23T07:20:31.385587+0000","172.21.15.69:0/288512401":"2024-03-23T07:17:44.994340+0000","172.21.15.63:0/424866385":"2024-03-23T07:07:10.417547+0000","172.21.15.63:6801/480818549":"2024-03-23T07:07:41.261927+0000","172.21.15.69:0/1072916585":"2024-03-23T07:17:44.994340+0000","172.21.15.63:6800/480818549":"2024-03-23T07:07:41.261927+0000","172.21.15.69:6828/1862357042":"2024-03-23T07:17:44.994340+0000","172.21.15.63:0/2503315479":"2024-03-23T07:07:41.261927+0000","172.21.15.69:6825/3798247699":"2024-03-23T07:25:00.991995+0000","172.21.15.63:0/1701838":"2024-03-23T07:16:00.292439+0000","172.21.15.63:6801/2616455474":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/1818534437":"2024-03-23T07:07:41.261927+0000","172.21.15.63:6800/2296442540":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/1323074353":"2024-03-23T07:07:41.261927+0000","172.21.15.63:6800/2616455474":"2024-03-23T07:07:10.417547+0000","172.21.15.63:6801/2390873586":"2024-03-23T07:20:31.385587+0000","172.21.15.63:6826/1314414011":"2024-03-23T07:13:45.306403+0000","172.21.15.63:6827/1314414011":"2024-03-23T07:13:45.306403+0000","172.21.15.69:0/547111417":"2024-03-23T07:17:44.994340+0000","172.21.15.69:0/2694993342":"2024-03-23T07:17:44.994340+0000","172.21.15.63:0/1602837747":"2024-03-23T07:20:31.385587+0000","172.21.15.63:0/108295103":"2024-03-23T07:16:00.292439+0000","172.21.15.63:6800/2001368263":"2024-03-23T07:16:00.292439+0000","172.21.15.63:6801/2001368263":"2024-03-23T07:16:00.292439+0000","172.21.15.63:0/466771007":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/2739978443":"2024-03-23T07:07:10.417547+0000","172.21.15.63:0/4021098270":"2024-03-23T07:20:31.385587+0000","172.21.15.63:6800/2390873586":"2024-03-23T07:20:31.385587+0000","172.21.15.63:6801/2296442540":"2024-03-23T07:08:44.186801+0000","172.21.15.69:6824/3798247699":"2024-03-23T07:25:00.991995+0000","172.21.15.63:0/3927392742":"2024-03-23T07:08:44.186801+0000","172.21.15.63:0/2835250916":"2024-03-23T07:16:00.292439+0000","172.21.15.69:6829/1862357042":"2024-03-23T07:17:44.994340+0000","172.21.15.63:0/41352707":"2024-03-23T07:16:00.292439+0000","172.21.15.63:0/1215330295":"2024-03-23T07:20:31.385587+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-22T07:28:05.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.673+0000 7f8c3e818640 1 -- 172.21.15.63:0/3012342676 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f8c2006d2f0 msgr2=0x7f8c2006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:05.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.673+0000 7f8c3e818640 1 --2- 172.21.15.63:0/3012342676 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f8c2006d2f0 0x7f8c2006f7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009fa0 tx=0x7f8c2c009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.673+0000 7f8c3e818640 1 -- 172.21.15.63:0/3012342676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c380fb110 msgr2=0x7f8c38112ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:05.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.673+0000 7f8c3e818640 1 --2- 172.21.15.63:0/3012342676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c380fb110 0x7f8c38112ca0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800c830 tx=0x7f8c28004430 comp rx=0 tx=0).stop 2024-03-22T07:28:05.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.673+0000 7f8c3e818640 1 -- 172.21.15.63:0/3012342676 shutdown_connections 2024-03-22T07:28:05.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.673+0000 7f8c3e818640 1 --2- 172.21.15.63:0/3012342676 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f8c2006d2f0 0x7f8c2006f7a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.673+0000 7f8c3e818640 1 --2- 172.21.15.63:0/3012342676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c38101140 0x7f8c381131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.673+0000 7f8c3e818640 1 --2- 172.21.15.63:0/3012342676 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f8c380fb110 0x7f8c38112ca0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.673+0000 7f8c3e818640 1 -- 172.21.15.63:0/3012342676 >> 172.21.15.63:0/3012342676 conn(0x7f8c38071130 msgr2=0x7f8c380f94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:05.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.673+0000 7f8c3e818640 1 -- 172.21.15.63:0/3012342676 shutdown_connections 2024-03-22T07:28:05.673 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.673+0000 7f8c3e818640 1 -- 172.21.15.63:0/3012342676 wait complete. 2024-03-22T07:28:05.683 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-22T07:10:56.524122+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-22T07:13:27.835268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-22T07:13:28.836648+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-22T07:28:05.684 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-22T07:28:05.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.785+0000 7fde6139d640 1 -- 172.21.15.63:0/111973757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde5c0fdd00 msgr2=0x7fde5c106460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:05.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.785+0000 7fde6139d640 1 --2- 172.21.15.63:0/111973757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde5c0fdd00 0x7fde5c106460 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fde440048d0 tx=0x7fde44009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.785+0000 7fde6139d640 1 -- 172.21.15.63:0/111973757 shutdown_connections 2024-03-22T07:28:05.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.785+0000 7fde6139d640 1 --2- 172.21.15.63:0/111973757 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde5c0fdd00 0x7fde5c106460 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.785+0000 7fde6139d640 1 --2- 172.21.15.63:0/111973757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde5c0fd3f0 0x7fde5c0fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.785+0000 7fde6139d640 1 -- 172.21.15.63:0/111973757 >> 172.21.15.63:0/111973757 conn(0x7fde5c0f6e70 msgr2=0x7fde5c0f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:05.787 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.785+0000 7fde6139d640 1 -- 172.21.15.63:0/111973757 shutdown_connections 2024-03-22T07:28:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.785+0000 7fde6139d640 1 -- 172.21.15.63:0/111973757 wait complete. 2024-03-22T07:28:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.785+0000 7fde6139d640 1 Processor -- start 2024-03-22T07:28:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.785+0000 7fde6139d640 1 -- start start 2024-03-22T07:28:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.786+0000 7fde6139d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde5c0fd3f0 0x7fde5c112fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.786+0000 7fde6139d640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde5c0fdd00 0x7fde5c1134f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.786+0000 7fde6139d640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c113af0 con 0x7fde5c0fdd00 2024-03-22T07:28:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.786+0000 7fde6139d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c10d140 con 0x7fde5c0fd3f0 2024-03-22T07:28:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.786+0000 7fde53fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde5c0fdd00 0x7fde5c1134f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.786+0000 7fde53fff640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde5c0fdd00 0x7fde5c1134f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47050/0 (socket says 172.21.15.63:47050) 2024-03-22T07:28:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.786+0000 7fde53fff640 1 -- 172.21.15.63:0/4107071434 learned_addr learned my addr 172.21.15.63:0/4107071434 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.786+0000 7fde5affd640 1 --2- 172.21.15.63:0/4107071434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde5c0fd3f0 0x7fde5c112fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.786+0000 7fde53fff640 1 -- 172.21.15.63:0/4107071434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde5c0fd3f0 msgr2=0x7fde5c112fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.786+0000 7fde53fff640 1 --2- 172.21.15.63:0/4107071434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde5c0fd3f0 0x7fde5c112fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.786+0000 7fde53fff640 1 -- 172.21.15.63:0/4107071434 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde44005040 con 0x7fde5c0fdd00 2024-03-22T07:28:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.786+0000 7fde5affd640 1 --2- 172.21.15.63:0/4107071434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde5c0fd3f0 0x7fde5c112fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-22T07:28:05.788 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.786+0000 7fde53fff640 1 --2- 172.21.15.63:0/4107071434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde5c0fdd00 0x7fde5c1134f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fde44004930 tx=0x7fde44009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:05.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.786+0000 7fde537fe640 1 -- 172.21.15.63:0/4107071434 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde440094c0 con 0x7fde5c0fdd00 2024-03-22T07:28:05.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.786+0000 7fde537fe640 1 -- 172.21.15.63:0/4107071434 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde44032e50 con 0x7fde5c0fdd00 2024-03-22T07:28:05.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.786+0000 7fde6139d640 1 -- 172.21.15.63:0/4107071434 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde5c10d3c0 con 0x7fde5c0fdd00 2024-03-22T07:28:05.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.786+0000 7fde537fe640 1 -- 172.21.15.63:0/4107071434 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde440433b0 con 0x7fde5c0fdd00 2024-03-22T07:28:05.789 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.786+0000 7fde6139d640 1 -- 172.21.15.63:0/4107071434 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde5c10d880 con 0x7fde5c0fdd00 2024-03-22T07:28:05.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.787+0000 7fde6139d640 1 -- 172.21.15.63:0/4107071434 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde5c065db0 con 0x7fde5c0fdd00 2024-03-22T07:28:05.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.788+0000 7fde537fe640 1 -- 172.21.15.63:0/4107071434 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fde4402f070 con 0x7fde5c0fdd00 2024-03-22T07:28:05.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.788+0000 7fde537fe640 1 --2- 172.21.15.63:0/4107071434 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fde4806d2f0 0x7fde4806f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:05.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.788+0000 7fde537fe640 1 -- 172.21.15.63:0/4107071434 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fde440ae900 con 0x7fde5c0fdd00 2024-03-22T07:28:05.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.788+0000 7fde5affd640 1 --2- 172.21.15.63:0/4107071434 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fde4806d2f0 0x7fde4806f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:05.790 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.789+0000 7fde5affd640 1 --2- 172.21.15.63:0/4107071434 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fde4806d2f0 0x7fde4806f7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0041d0 tx=0x7fde4c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:05.793 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.793+0000 7fde537fe640 1 -- 172.21.15.63:0/4107071434 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fde44078330 con 0x7fde5c0fdd00 2024-03-22T07:28:05.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.972+0000 7fde6139d640 1 -- 172.21.15.63:0/4107071434 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde5c109540 con 0x7fde5c0fdd00 2024-03-22T07:28:05.973 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.973+0000 7fde537fe640 1 -- 172.21.15.63:0/4107071434 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde4407b9f0 con 0x7fde5c0fdd00 2024-03-22T07:28:05.973 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 1 2024-03-22T07:28:05.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.975+0000 7fde6139d640 1 -- 172.21.15.63:0/4107071434 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fde4806d2f0 msgr2=0x7fde4806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:05.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.975+0000 7fde6139d640 1 --2- 172.21.15.63:0/4107071434 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fde4806d2f0 0x7fde4806f7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0041d0 tx=0x7fde4c009040 comp rx=0 tx=0).stop 2024-03-22T07:28:05.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.975+0000 7fde6139d640 1 -- 172.21.15.63:0/4107071434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde5c0fdd00 msgr2=0x7fde5c1134f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:05.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.975+0000 7fde6139d640 1 --2- 172.21.15.63:0/4107071434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde5c0fdd00 0x7fde5c1134f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fde44004930 tx=0x7fde44009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.975+0000 7fde6139d640 1 -- 172.21.15.63:0/4107071434 shutdown_connections 2024-03-22T07:28:05.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.975+0000 7fde6139d640 1 --2- 172.21.15.63:0/4107071434 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fde5c0fdd00 0x7fde5c1134f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.975+0000 7fde6139d640 1 --2- 172.21.15.63:0/4107071434 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fde4806d2f0 0x7fde4806f7a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.975 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.975+0000 7fde6139d640 1 --2- 172.21.15.63:0/4107071434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fde5c0fd3f0 0x7fde5c112fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:05.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.975+0000 7fde6139d640 1 -- 172.21.15.63:0/4107071434 >> 172.21.15.63:0/4107071434 conn(0x7fde5c0f6e70 msgr2=0x7fde5c0f83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:05.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.975+0000 7fde6139d640 1 -- 172.21.15.63:0/4107071434 shutdown_connections 2024-03-22T07:28:05.976 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:05.975+0000 7fde6139d640 1 -- 172.21.15.63:0/4107071434 wait complete. 2024-03-22T07:28:05.985 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-22T07:28:06.089 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.088+0000 7f2b77aae640 1 -- 172.21.15.63:0/2514133589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b70102bf0 msgr2=0x7f2b700fc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:06.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.088+0000 7f2b77aae640 1 --2- 172.21.15.63:0/2514133589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b70102bf0 0x7f2b700fc280 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0048d0 tx=0x7f2b6c009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.088+0000 7f2b77aae640 1 -- 172.21.15.63:0/2514133589 shutdown_connections 2024-03-22T07:28:06.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.088+0000 7f2b77aae640 1 --2- 172.21.15.63:0/2514133589 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b70102bf0 0x7f2b700fc280 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.088+0000 7f2b77aae640 1 --2- 172.21.15.63:0/2514133589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b70102250 0x7f2b70102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.090 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.088+0000 7f2b77aae640 1 -- 172.21.15.63:0/2514133589 >> 172.21.15.63:0/2514133589 conn(0x7f2b700f7e60 msgr2=0x7f2b700fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:06.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.088+0000 7f2b77aae640 1 -- 172.21.15.63:0/2514133589 shutdown_connections 2024-03-22T07:28:06.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.088+0000 7f2b77aae640 1 -- 172.21.15.63:0/2514133589 wait complete. 2024-03-22T07:28:06.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.088+0000 7f2b77aae640 1 Processor -- start 2024-03-22T07:28:06.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.088+0000 7f2b77aae640 1 -- start start 2024-03-22T07:28:06.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.088+0000 7f2b77aae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b70102250 0x7f2b70112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:06.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.088+0000 7f2b77aae640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b70102bf0 0x7f2b70113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:06.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.088+0000 7f2b77aae640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b70113810 con 0x7f2b70102bf0 2024-03-22T07:28:06.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.088+0000 7f2b77aae640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7010ce60 con 0x7f2b70102250 2024-03-22T07:28:06.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.089+0000 7f2b75049640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b70102bf0 0x7f2b70113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:06.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.089+0000 7f2b75049640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b70102bf0 0x7f2b70113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47056/0 (socket says 172.21.15.63:47056) 2024-03-22T07:28:06.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.089+0000 7f2b75049640 1 -- 172.21.15.63:0/528370543 learned_addr learned my addr 172.21.15.63:0/528370543 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:06.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.089+0000 7f2b7584a640 1 --2- 172.21.15.63:0/528370543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b70102250 0x7f2b70112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:06.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.089+0000 7f2b75049640 1 -- 172.21.15.63:0/528370543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b70102250 msgr2=0x7f2b70112cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:06.091 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.089+0000 7f2b75049640 1 --2- 172.21.15.63:0/528370543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b70102250 0x7f2b70112cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.089+0000 7f2b75049640 1 -- 172.21.15.63:0/528370543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c005040 con 0x7f2b70102bf0 2024-03-22T07:28:06.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.089+0000 7f2b75049640 1 --2- 172.21.15.63:0/528370543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b70102bf0 0x7f2b70113210 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0047f0 tx=0x7f2b6c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:06.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.089+0000 7f2b667fc640 1 -- 172.21.15.63:0/528370543 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b6c0094c0 con 0x7f2b70102bf0 2024-03-22T07:28:06.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.089+0000 7f2b77aae640 1 -- 172.21.15.63:0/528370543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7010d0e0 con 0x7f2b70102bf0 2024-03-22T07:28:06.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.089+0000 7f2b667fc640 1 -- 172.21.15.63:0/528370543 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b6c032e50 con 0x7f2b70102bf0 2024-03-22T07:28:06.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.089+0000 7f2b667fc640 1 -- 172.21.15.63:0/528370543 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b6c0433b0 con 0x7f2b70102bf0 2024-03-22T07:28:06.092 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.089+0000 7f2b77aae640 1 -- 172.21.15.63:0/528370543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7010d5a0 con 0x7f2b70102bf0 2024-03-22T07:28:06.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.090+0000 7f2b77aae640 1 -- 172.21.15.63:0/528370543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b700fd9c0 con 0x7f2b70102bf0 2024-03-22T07:28:06.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.091+0000 7f2b667fc640 1 -- 172.21.15.63:0/528370543 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f2b6c02f070 con 0x7f2b70102bf0 2024-03-22T07:28:06.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.091+0000 7f2b667fc640 1 --2- 172.21.15.63:0/528370543 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2b5c06d300 0x7f2b5c06f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:06.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.091+0000 7f2b667fc640 1 -- 172.21.15.63:0/528370543 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2b6c0ae800 con 0x7f2b70102bf0 2024-03-22T07:28:06.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.091+0000 7f2b7584a640 1 --2- 172.21.15.63:0/528370543 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2b5c06d300 0x7f2b5c06f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:06.093 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.091+0000 7f2b7584a640 1 --2- 172.21.15.63:0/528370543 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2b5c06d300 0x7f2b5c06f7b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2b60004eb0 tx=0x7f2b6000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:06.096 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.096+0000 7f2b667fc640 1 -- 172.21.15.63:0/528370543 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2b6c0782e0 con 0x7f2b70102bf0 2024-03-22T07:28:06.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.275+0000 7f2b77aae640 1 -- 172.21.15.63:0/528370543 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f2b70109390 con 0x7f2b70102bf0 2024-03-22T07:28:06.275 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.275+0000 7f2b667fc640 1 -- 172.21.15.63:0/528370543 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7f2b6c07b9a0 con 0x7f2b70102bf0 2024-03-22T07:28:06.276 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-03-22T07:28:06.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.277+0000 7f2b77aae640 1 -- 172.21.15.63:0/528370543 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2b5c06d300 msgr2=0x7f2b5c06f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:06.277 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.277+0000 7f2b77aae640 1 --2- 172.21.15.63:0/528370543 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2b5c06d300 0x7f2b5c06f7b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2b60004eb0 tx=0x7f2b6000b040 comp rx=0 tx=0).stop 2024-03-22T07:28:06.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.277+0000 7f2b77aae640 1 -- 172.21.15.63:0/528370543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b70102bf0 msgr2=0x7f2b70113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:06.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.277+0000 7f2b77aae640 1 --2- 172.21.15.63:0/528370543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b70102bf0 0x7f2b70113210 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0047f0 tx=0x7f2b6c009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.277+0000 7f2b77aae640 1 -- 172.21.15.63:0/528370543 shutdown_connections 2024-03-22T07:28:06.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.277+0000 7f2b77aae640 1 --2- 172.21.15.63:0/528370543 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f2b70102bf0 0x7f2b70113210 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.277+0000 7f2b77aae640 1 --2- 172.21.15.63:0/528370543 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f2b5c06d300 0x7f2b5c06f7b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.277+0000 7f2b77aae640 1 --2- 172.21.15.63:0/528370543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b70102250 0x7f2b70112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.277+0000 7f2b77aae640 1 -- 172.21.15.63:0/528370543 >> 172.21.15.63:0/528370543 conn(0x7f2b700f7e60 msgr2=0x7f2b700f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:06.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.277+0000 7f2b77aae640 1 -- 172.21.15.63:0/528370543 shutdown_connections 2024-03-22T07:28:06.278 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.277+0000 7f2b77aae640 1 -- 172.21.15.63:0/528370543 wait complete. 2024-03-22T07:28:06.287 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:06 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2814730016' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T07:28:06.287 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:06 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3012342676' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:28:06.287 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:06 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/4107071434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:28:06.288 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-22T07:28:06.390 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.389+0000 7f1e78b10640 1 -- 172.21.15.63:0/2003900148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e740fb110 msgr2=0x7f1e740fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.389+0000 7f1e78b10640 1 --2- 172.21.15.63:0/2003900148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e740fb110 0x7f1e740fb560 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1e640048d0 tx=0x7f1e64009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.389+0000 7f1e78b10640 1 -- 172.21.15.63:0/2003900148 shutdown_connections 2024-03-22T07:28:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.389+0000 7f1e78b10640 1 --2- 172.21.15.63:0/2003900148 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e740fb110 0x7f1e740fb560 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.389+0000 7f1e78b10640 1 --2- 172.21.15.63:0/2003900148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e74101140 0x7f1e74101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.389+0000 7f1e78b10640 1 -- 172.21.15.63:0/2003900148 >> 172.21.15.63:0/2003900148 conn(0x7f1e740710b0 msgr2=0x7f1e740714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.389+0000 7f1e78b10640 1 -- 172.21.15.63:0/2003900148 shutdown_connections 2024-03-22T07:28:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.389+0000 7f1e78b10640 1 -- 172.21.15.63:0/2003900148 wait complete. 2024-03-22T07:28:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.389+0000 7f1e78b10640 1 Processor -- start 2024-03-22T07:28:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.389+0000 7f1e78b10640 1 -- start start 2024-03-22T07:28:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.389+0000 7f1e78b10640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e740fb110 0x7f1e74112cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.389+0000 7f1e78b10640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e74101140 0x7f1e74113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.389+0000 7f1e78b10640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e74113830 con 0x7f1e74101140 2024-03-22T07:28:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.389+0000 7f1e78b10640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7410ce80 con 0x7f1e740fb110 2024-03-22T07:28:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.390+0000 7f1e71d9b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e74101140 0x7f1e74113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:06.391 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.390+0000 7f1e71d9b640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e74101140 0x7f1e74113230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47082/0 (socket says 172.21.15.63:47082) 2024-03-22T07:28:06.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.390+0000 7f1e71d9b640 1 -- 172.21.15.63:0/3526457933 learned_addr learned my addr 172.21.15.63:0/3526457933 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:06.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.390+0000 7f1e7259c640 1 --2- 172.21.15.63:0/3526457933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e740fb110 0x7f1e74112cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:06.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.390+0000 7f1e71d9b640 1 -- 172.21.15.63:0/3526457933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e740fb110 msgr2=0x7f1e74112cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:06.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.390+0000 7f1e71d9b640 1 --2- 172.21.15.63:0/3526457933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e740fb110 0x7f1e74112cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.390+0000 7f1e71d9b640 1 -- 172.21.15.63:0/3526457933 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e64005040 con 0x7f1e74101140 2024-03-22T07:28:06.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.390+0000 7f1e71d9b640 1 --2- 172.21.15.63:0/3526457933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e74101140 0x7f1e74113230 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1e640041d0 tx=0x7f1e64009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:06.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.390+0000 7f1e6affd640 1 -- 172.21.15.63:0/3526457933 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e64030870 con 0x7f1e74101140 2024-03-22T07:28:06.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.390+0000 7f1e78b10640 1 -- 172.21.15.63:0/3526457933 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7410d100 con 0x7f1e74101140 2024-03-22T07:28:06.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.390+0000 7f1e6affd640 1 -- 172.21.15.63:0/3526457933 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e640092e0 con 0x7f1e74101140 2024-03-22T07:28:06.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.390+0000 7f1e78b10640 1 -- 172.21.15.63:0/3526457933 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7410d620 con 0x7f1e74101140 2024-03-22T07:28:06.392 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.390+0000 7f1e6affd640 1 -- 172.21.15.63:0/3526457933 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e640435e0 con 0x7f1e74101140 2024-03-22T07:28:06.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.391+0000 7f1e78b10640 1 -- 172.21.15.63:0/3526457933 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e740fc800 con 0x7f1e74101140 2024-03-22T07:28:06.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.392+0000 7f1e6affd640 1 -- 172.21.15.63:0/3526457933 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7f1e6403b420 con 0x7f1e74101140 2024-03-22T07:28:06.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.392+0000 7f1e6affd640 1 --2- 172.21.15.63:0/3526457933 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1e6006d2f0 0x7f1e6006f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:06.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.392+0000 7f1e6affd640 1 -- 172.21.15.63:0/3526457933 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1e640af720 con 0x7f1e74101140 2024-03-22T07:28:06.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.392+0000 7f1e7259c640 1 --2- 172.21.15.63:0/3526457933 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1e6006d2f0 0x7f1e6006f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:06.393 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.393+0000 7f1e7259c640 1 --2- 172.21.15.63:0/3526457933 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1e6006d2f0 0x7f1e6006f7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009fd0 tx=0x7f1e5c009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:06.397 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.397+0000 7f1e6affd640 1 -- 172.21.15.63:0/3526457933 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1e640791d0 con 0x7f1e74101140 2024-03-22T07:28:06.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:06 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2814730016' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T07:28:06.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:06 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3012342676' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-22T07:28:06.529 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:06 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/4107071434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-22T07:28:06.579 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.579+0000 7f1e78b10640 1 -- 172.21.15.63:0/3526457933 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f1e74108200 con 0x7f1e74101140 2024-03-22T07:28:06.580 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.580+0000 7f1e6affd640 1 -- 172.21.15.63:0/3526457933 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7f1e6407c890 con 0x7f1e74101140 2024-03-22T07:28:06.580 INFO:teuthology.orchestra.run.smithi063.stdout:pg_num: 32 2024-03-22T07:28:06.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.582+0000 7f1e78b10640 1 -- 172.21.15.63:0/3526457933 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1e6006d2f0 msgr2=0x7f1e6006f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:06.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.582+0000 7f1e78b10640 1 --2- 172.21.15.63:0/3526457933 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1e6006d2f0 0x7f1e6006f7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009fd0 tx=0x7f1e5c009e50 comp rx=0 tx=0).stop 2024-03-22T07:28:06.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.582+0000 7f1e78b10640 1 -- 172.21.15.63:0/3526457933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e74101140 msgr2=0x7f1e74113230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:06.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.582+0000 7f1e78b10640 1 --2- 172.21.15.63:0/3526457933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e74101140 0x7f1e74113230 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1e640041d0 tx=0x7f1e64009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.582 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.582+0000 7f1e78b10640 1 -- 172.21.15.63:0/3526457933 shutdown_connections 2024-03-22T07:28:06.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.582+0000 7f1e78b10640 1 --2- 172.21.15.63:0/3526457933 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7f1e74101140 0x7f1e74113230 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.582+0000 7f1e78b10640 1 --2- 172.21.15.63:0/3526457933 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7f1e6006d2f0 0x7f1e6006f7a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.582+0000 7f1e78b10640 1 --2- 172.21.15.63:0/3526457933 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1e740fb110 0x7f1e74112cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.582+0000 7f1e78b10640 1 -- 172.21.15.63:0/3526457933 >> 172.21.15.63:0/3526457933 conn(0x7f1e740710b0 msgr2=0x7f1e740643a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:06.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.582+0000 7f1e78b10640 1 -- 172.21.15.63:0/3526457933 shutdown_connections 2024-03-22T07:28:06.583 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.582+0000 7f1e78b10640 1 -- 172.21.15.63:0/3526457933 wait complete. 2024-03-22T07:28:06.593 DEBUG:teuthology.orchestra.run.smithi063:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-22T07:28:06.693 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.692+0000 7fd47f46e640 1 -- 172.21.15.63:0/836774925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd478102d80 msgr2=0x7fd4780fc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.692+0000 7fd47f46e640 1 --2- 172.21.15.63:0/836774925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd478102d80 0x7fd4780fc410 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd4740048d0 tx=0x7fd474009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.692+0000 7fd47f46e640 1 -- 172.21.15.63:0/836774925 shutdown_connections 2024-03-22T07:28:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.692+0000 7fd47f46e640 1 --2- 172.21.15.63:0/836774925 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd478102d80 0x7fd4780fc410 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.692+0000 7fd47f46e640 1 --2- 172.21.15.63:0/836774925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4781023e0 0x7fd4781027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.692+0000 7fd47f46e640 1 -- 172.21.15.63:0/836774925 >> 172.21.15.63:0/836774925 conn(0x7fd4780f7fb0 msgr2=0x7fd4780fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.692+0000 7fd47f46e640 1 -- 172.21.15.63:0/836774925 shutdown_connections 2024-03-22T07:28:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.692+0000 7fd47f46e640 1 -- 172.21.15.63:0/836774925 wait complete. 2024-03-22T07:28:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.692+0000 7fd47f46e640 1 Processor -- start 2024-03-22T07:28:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.692+0000 7fd47f46e640 1 -- start start 2024-03-22T07:28:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.693+0000 7fd47f46e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4781023e0 0x7fd478112f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.693+0000 7fd47f46e640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd478102d80 0x7fd4781134a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:06.694 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.693+0000 7fd47f46e640 1 -- --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd478113b30 con 0x7fd478102d80 2024-03-22T07:28:06.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.693+0000 7fd47f46e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47810d0f0 con 0x7fd4781023e0 2024-03-22T07:28:06.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.693+0000 7fd47ca09640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd478102d80 0x7fd4781134a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:06.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.693+0000 7fd47ca09640 1 --2- >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd478102d80 0x7fd4781134a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.63:3300/0 says I am v2:172.21.15.63:47092/0 (socket says 172.21.15.63:47092) 2024-03-22T07:28:06.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.693+0000 7fd47ca09640 1 -- 172.21.15.63:0/2739583187 learned_addr learned my addr 172.21.15.63:0/2739583187 (peer_addr_for_me v2:172.21.15.63:0/0) 2024-03-22T07:28:06.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.693+0000 7fd47d20a640 1 --2- 172.21.15.63:0/2739583187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4781023e0 0x7fd478112f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:06.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.693+0000 7fd47ca09640 1 -- 172.21.15.63:0/2739583187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4781023e0 msgr2=0x7fd478112f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:06.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.693+0000 7fd47ca09640 1 --2- 172.21.15.63:0/2739583187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4781023e0 0x7fd478112f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.693+0000 7fd47ca09640 1 -- 172.21.15.63:0/2739583187 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd474005040 con 0x7fd478102d80 2024-03-22T07:28:06.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.693+0000 7fd47ca09640 1 --2- 172.21.15.63:0/2739583187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd478102d80 0x7fd4781134a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd474004930 tx=0x7fd474009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:06.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.693+0000 7fd469ffb640 1 -- 172.21.15.63:0/2739583187 <== mon.0 v2:172.21.15.63:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4740094c0 con 0x7fd478102d80 2024-03-22T07:28:06.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.693+0000 7fd469ffb640 1 -- 172.21.15.63:0/2739583187 <== mon.0 v2:172.21.15.63:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd474032e50 con 0x7fd478102d80 2024-03-22T07:28:06.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.693+0000 7fd47f46e640 1 -- 172.21.15.63:0/2739583187 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47810d370 con 0x7fd478102d80 2024-03-22T07:28:06.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.693+0000 7fd469ffb640 1 -- 172.21.15.63:0/2739583187 <== mon.0 v2:172.21.15.63:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4740433b0 con 0x7fd478102d80 2024-03-22T07:28:06.695 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.694+0000 7fd47f46e640 1 -- 172.21.15.63:0/2739583187 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47810d830 con 0x7fd478102d80 2024-03-22T07:28:06.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.695+0000 7fd47f46e640 1 -- 172.21.15.63:0/2739583187 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4780fdb50 con 0x7fd478102d80 2024-03-22T07:28:06.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.695+0000 7fd469ffb640 1 -- 172.21.15.63:0/2739583187 <== mon.0 v2:172.21.15.63:3300/0 4 ==== mgrmap(e 40) v1 ==== 91167+0+0 (secure 0 0 0) 0x7fd47402f070 con 0x7fd478102d80 2024-03-22T07:28:06.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.695+0000 7fd469ffb640 1 --2- 172.21.15.63:0/2739583187 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd46406d2f0 0x7fd46406f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-22T07:28:06.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.695+0000 7fd469ffb640 1 -- 172.21.15.63:0/2739583187 <== mon.0 v2:172.21.15.63:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd4740ae850 con 0x7fd478102d80 2024-03-22T07:28:06.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.695+0000 7fd47d20a640 1 --2- 172.21.15.63:0/2739583187 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd46406d2f0 0x7fd46406f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-22T07:28:06.696 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.696+0000 7fd47d20a640 1 --2- 172.21.15.63:0/2739583187 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd46406d2f0 0x7fd46406f7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009b50 tx=0x7fd46c009120 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-22T07:28:06.700 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.700+0000 7fd469ffb640 1 -- 172.21.15.63:0/2739583187 <== mon.0 v2:172.21.15.63:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd474078330 con 0x7fd478102d80 2024-03-22T07:28:06.892 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.892+0000 7fd47f46e640 1 -- 172.21.15.63:0/2739583187 --> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd4781095d0 con 0x7fd478102d80 2024-03-22T07:28:06.893 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.893+0000 7fd469ffb640 1 -- 172.21.15.63:0/2739583187 <== mon.0 v2:172.21.15.63:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 39 entries v87) v1 ==== 81+0+2302 (secure 0 0 0) 0x7fd47407b9f0 con 0x7fd478102d80 2024-03-22T07:28:06.893 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6829/1373266298 2024-03-23T07:25:00.991980+0000 2024-03-22T07:28:06.893 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6828/1373266298 2024-03-23T07:25:00.991980+0000 2024-03-22T07:28:06.893 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1649622442 2024-03-23T07:07:10.417547+0000 2024-03-22T07:28:06.893 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/47408280 2024-03-23T07:08:44.186801+0000 2024-03-22T07:28:06.893 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3629523167 2024-03-23T07:20:31.385587+0000 2024-03-22T07:28:06.893 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.69:0/288512401 2024-03-23T07:17:44.994340+0000 2024-03-22T07:28:06.893 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/424866385 2024-03-23T07:07:10.417547+0000 2024-03-22T07:28:06.893 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/480818549 2024-03-23T07:07:41.261927+0000 2024-03-22T07:28:06.894 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.69:0/1072916585 2024-03-23T07:17:44.994340+0000 2024-03-22T07:28:06.894 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/480818549 2024-03-23T07:07:41.261927+0000 2024-03-22T07:28:06.894 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.69:6828/1862357042 2024-03-23T07:17:44.994340+0000 2024-03-22T07:28:06.894 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2503315479 2024-03-23T07:07:41.261927+0000 2024-03-22T07:28:06.894 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.69:6825/3798247699 2024-03-23T07:25:00.991995+0000 2024-03-22T07:28:06.894 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1701838 2024-03-23T07:16:00.292439+0000 2024-03-22T07:28:06.894 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/2616455474 2024-03-23T07:07:10.417547+0000 2024-03-22T07:28:06.894 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1818534437 2024-03-23T07:07:41.261927+0000 2024-03-22T07:28:06.894 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/2296442540 2024-03-23T07:08:44.186801+0000 2024-03-22T07:28:06.894 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1323074353 2024-03-23T07:07:41.261927+0000 2024-03-22T07:28:06.894 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/2616455474 2024-03-23T07:07:10.417547+0000 2024-03-22T07:28:06.894 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/2390873586 2024-03-23T07:20:31.385587+0000 2024-03-22T07:28:06.894 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6826/1314414011 2024-03-23T07:13:45.306403+0000 2024-03-22T07:28:06.894 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6827/1314414011 2024-03-23T07:13:45.306403+0000 2024-03-22T07:28:06.894 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.69:0/547111417 2024-03-23T07:17:44.994340+0000 2024-03-22T07:28:06.895 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.69:0/2694993342 2024-03-23T07:17:44.994340+0000 2024-03-22T07:28:06.895 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1602837747 2024-03-23T07:20:31.385587+0000 2024-03-22T07:28:06.895 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/108295103 2024-03-23T07:16:00.292439+0000 2024-03-22T07:28:06.895 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/2001368263 2024-03-23T07:16:00.292439+0000 2024-03-22T07:28:06.895 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/2001368263 2024-03-23T07:16:00.292439+0000 2024-03-22T07:28:06.895 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/466771007 2024-03-23T07:08:44.186801+0000 2024-03-22T07:28:06.895 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2739978443 2024-03-23T07:07:10.417547+0000 2024-03-22T07:28:06.895 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/4021098270 2024-03-23T07:20:31.385587+0000 2024-03-22T07:28:06.895 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6800/2390873586 2024-03-23T07:20:31.385587+0000 2024-03-22T07:28:06.895 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:6801/2296442540 2024-03-23T07:08:44.186801+0000 2024-03-22T07:28:06.895 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.69:6824/3798247699 2024-03-23T07:25:00.991995+0000 2024-03-22T07:28:06.895 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/3927392742 2024-03-23T07:08:44.186801+0000 2024-03-22T07:28:06.895 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/2835250916 2024-03-23T07:16:00.292439+0000 2024-03-22T07:28:06.895 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.69:6829/1862357042 2024-03-23T07:17:44.994340+0000 2024-03-22T07:28:06.895 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/41352707 2024-03-23T07:16:00.292439+0000 2024-03-22T07:28:06.895 INFO:teuthology.orchestra.run.smithi063.stdout:172.21.15.63:0/1215330295 2024-03-23T07:20:31.385587+0000 2024-03-22T07:28:06.896 INFO:teuthology.orchestra.run.smithi063.stderr:listed 39 entries 2024-03-22T07:28:06.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.895+0000 7fd47f46e640 1 -- 172.21.15.63:0/2739583187 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd46406d2f0 msgr2=0x7fd46406f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:06.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.895+0000 7fd47f46e640 1 --2- 172.21.15.63:0/2739583187 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd46406d2f0 0x7fd46406f7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009b50 tx=0x7fd46c009120 comp rx=0 tx=0).stop 2024-03-22T07:28:06.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.895+0000 7fd47f46e640 1 -- 172.21.15.63:0/2739583187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd478102d80 msgr2=0x7fd4781134a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-22T07:28:06.896 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.895+0000 7fd47f46e640 1 --2- 172.21.15.63:0/2739583187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd478102d80 0x7fd4781134a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd474004930 tx=0x7fd474009da0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.895+0000 7fd47f46e640 1 -- 172.21.15.63:0/2739583187 shutdown_connections 2024-03-22T07:28:06.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.895+0000 7fd47f46e640 1 --2- 172.21.15.63:0/2739583187 >> [v2:172.21.15.63:3300/0,v1:172.21.15.63:6789/0] conn(0x7fd478102d80 0x7fd4781134a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.895+0000 7fd47f46e640 1 --2- 172.21.15.63:0/2739583187 >> [v2:172.21.15.63:6800/1311134736,v1:172.21.15.63:6801/1311134736] conn(0x7fd46406d2f0 0x7fd46406f7a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.895+0000 7fd47f46e640 1 --2- 172.21.15.63:0/2739583187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4781023e0 0x7fd478112f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-22T07:28:06.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.895+0000 7fd47f46e640 1 -- 172.21.15.63:0/2739583187 >> 172.21.15.63:0/2739583187 conn(0x7fd4780f7fb0 msgr2=0x7fd4780f8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-22T07:28:06.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.895+0000 7fd47f46e640 1 -- 172.21.15.63:0/2739583187 shutdown_connections 2024-03-22T07:28:06.897 INFO:teuthology.orchestra.run.smithi063.stderr:2024-03-22T07:28:06.895+0000 7fd47f46e640 1 -- 172.21.15.63:0/2739583187 wait complete. 2024-03-22T07:28:06.905 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-22T07:28:06.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:28:06.906 DEBUG:teuthology.orchestra.run.smithi069:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-03-22T07:28:06.942 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi069.front.sepia.ceph.com 2024-03-22T07:28:06.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:28:06.942 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-22T07:28:07.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:28:07.007 DEBUG:teuthology.orchestra.run.smithi063:> ip netns list 2024-03-22T07:28:07.021 INFO:teuthology.orchestra.run.smithi063.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-22T07:28:07.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:28:07.022 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-22T07:28:07.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:28:07.090 DEBUG:teuthology.orchestra.run.smithi063:> sudo ip link delete ceph-brx 2024-03-22T07:28:07.155 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:07 smithi063 ceph-mon[112784]: pgmap v267: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:28:07.156 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:07 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/528370543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T07:28:07.156 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:07 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/3526457933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T07:28:07.156 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:07 smithi063 ceph-mon[112784]: from='client.? 172.21.15.63:0/2739583187' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T07:28:07.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:28:07.183 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-03-22T07:28:07.202 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-22T07:28:07.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:28:07.203 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-22T07:28:07.270 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:07 smithi069 ceph-mon[83631]: pgmap v267: 65 pgs: 65 active+clean; 299 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-03-22T07:28:07.270 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:07 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/528370543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-22T07:28:07.270 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:07 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/3526457933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-22T07:28:07.270 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:07 smithi069 ceph-mon[83631]: from='client.? 172.21.15.63:0/2739583187' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-22T07:28:07.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-22T07:28:07.274 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2024-03-22T07:28:07.372 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-22T07:28:07.383 INFO:tasks.cephadm:Teardown begin 2024-03-22T07:28:07.384 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T07:28:07.414 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T07:28:07.443 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-22T07:28:07.443 DEBUG:teuthology.orchestra.run.smithi063:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-22T07:28:07.468 DEBUG:teuthology.orchestra.run.smithi069:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-22T07:28:07.499 INFO:tasks.cephadm:Stopping all daemons... 2024-03-22T07:28:07.499 INFO:tasks.cephadm.mon.smithi063:Stopping mon.smithi063... 2024-03-22T07:28:07.499 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mon.smithi063 2024-03-22T07:28:07.811 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:07 smithi063 systemd[1]: Stopping Ceph mon.smithi063 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:28:08.120 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:07 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063[112780]: 2024-03-22T07:28:07.810+0000 7f52bb4aa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi063 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:28:08.120 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:07 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063[112780]: 2024-03-22T07:28:07.810+0000 7f52bb4aa700 -1 mon.smithi063@0(leader) e3 *** Got Signal Terminated *** 2024-03-22T07:28:08.120 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:07 smithi063 podman[153771]: 2024-03-22 07:28:07.862276845 +0000 UTC m=+0.162718383 container died ab96538a07c1161a342a4f70fabad331e02fec061f547ca5d6f74b69114212b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-22T07:28:08.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:08 smithi063 podman[153771]: 2024-03-22 07:28:08.311738954 +0000 UTC m=+0.612180533 container cleanup ab96538a07c1161a342a4f70fabad331e02fec061f547ca5d6f74b69114212b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b) 2024-03-22T07:28:08.373 INFO:journalctl@ceph.mon.smithi063.smithi063.stdout:Mar 22 07:28:08 smithi063 bash[153771]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi063 2024-03-22T07:28:08.998 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mon.smithi063.service' 2024-03-22T07:28:09.051 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T07:28:09.051 INFO:tasks.cephadm.mon.smithi063:Stopped mon.smithi063 2024-03-22T07:28:09.051 INFO:tasks.cephadm.mon.smithi069:Stopping mon.smithi069... 2024-03-22T07:28:09.052 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mon.smithi069 2024-03-22T07:28:09.339 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:09 smithi069 systemd[1]: Stopping Ceph mon.smithi069 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:28:09.779 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:09 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi069[83627]: 2024-03-22T07:28:09.391+0000 7f23df6a2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi069 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:28:09.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:09 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi069[83627]: 2024-03-22T07:28:09.391+0000 7f23df6a2700 -1 mon.smithi069@1(peon) e3 *** Got Signal Terminated *** 2024-03-22T07:28:09.780 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:09 smithi069 podman[115741]: 2024-03-22 07:28:09.505104419 +0000 UTC m=+0.264380300 container died 01a26e1fb1a1c6343ba1d9a1c8b0901dc592c34bceb7a7506cfbcfaab108b006 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi069, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-22T07:28:10.202 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:09 smithi069 podman[115741]: 2024-03-22 07:28:09.976203612 +0000 UTC m=+0.735479492 container cleanup 01a26e1fb1a1c6343ba1d9a1c8b0901dc592c34bceb7a7506cfbcfaab108b006 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi069, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:28:10.202 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:09 smithi069 bash[115741]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi069 2024-03-22T07:28:10.442 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mon.smithi069.service' 2024-03-22T07:28:10.467 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:10 smithi069 podman[115763]: 2024-03-22 07:28:10.202821273 +0000 UTC m=+0.694890542 container remove 01a26e1fb1a1c6343ba1d9a1c8b0901dc592c34bceb7a7506cfbcfaab108b006 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-mon-smithi069, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-db0330b) 2024-03-22T07:28:10.467 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:10 smithi069 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mon.smithi069.service: Deactivated successfully. 2024-03-22T07:28:10.467 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:10 smithi069 systemd[1]: Stopped Ceph mon.smithi069 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:28:10.467 INFO:journalctl@ceph.mon.smithi069.smithi069.stdout:Mar 22 07:28:10 smithi069 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@mon.smithi069.service: Consumed 4.375s CPU time. 2024-03-22T07:28:10.510 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T07:28:10.511 INFO:tasks.cephadm.mon.smithi069:Stopped mon.smithi069 2024-03-22T07:28:10.511 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-22T07:28:10.511 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.0 2024-03-22T07:28:10.817 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:28:10 smithi063 systemd[1]: Stopping Ceph osd.0 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:28:11.123 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:28:10 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0[124353]: 2024-03-22T07:28:10.815+0000 7f5f054d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:28:11.123 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:28:10 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0[124353]: 2024-03-22T07:28:10.815+0000 7f5f054d8700 -1 osd.0 87 *** Got signal Terminated *** 2024-03-22T07:28:11.123 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:28:10 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0[124353]: 2024-03-22T07:28:10.815+0000 7f5f054d8700 -1 osd.0 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:28:16.494 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:28:16 smithi063 podman[153992]: 2024-03-22 07:28:16.207749706 +0000 UTC m=+5.506162536 container died fdbc8b34ab78f500d2b4ca6183e7c2d10e5761e5d5bad0b88a265d676eed1fdc (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:28:16.873 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:28:16 smithi063 podman[153992]: 2024-03-22 07:28:16.668295452 +0000 UTC m=+5.966708288 container cleanup fdbc8b34ab78f500d2b4ca6183e7c2d10e5761e5d5bad0b88a265d676eed1fdc (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-db0330b) 2024-03-22T07:28:16.873 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:28:16 smithi063 bash[153992]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0 2024-03-22T07:28:17.341 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:28:16 smithi063 podman[154010]: 2024-03-22 07:28:16.986699867 +0000 UTC m=+0.771064634 container remove fdbc8b34ab78f500d2b4ca6183e7c2d10e5761e5d5bad0b88a265d676eed1fdc (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:28:17.623 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:28:17 smithi063 podman[154104]: 2024-03-22 07:28:17.244538786 +0000 UTC m=+0.018211055 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:28:17.623 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:28:17 smithi063 podman[154104]: 2024-03-22 07:28:17.396046414 +0000 UTC m=+0.169718666 container create c2c83236cbc22f9cbd07648ea11b394450afb5d17c35c0e73c5b3bf7e085a4d5 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux ) 2024-03-22T07:28:17.991 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:28:17 smithi063 podman[154104]: 2024-03-22 07:28:17.691253392 +0000 UTC m=+0.464925657 container init c2c83236cbc22f9cbd07648ea11b394450afb5d17c35c0e73c5b3bf7e085a4d5 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-22T07:28:17.991 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:28:17 smithi063 podman[154104]: 2024-03-22 07:28:17.694549973 +0000 UTC m=+0.468222229 container start c2c83236cbc22f9cbd07648ea11b394450afb5d17c35c0e73c5b3bf7e085a4d5 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-22T07:28:17.991 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:28:17 smithi063 podman[154104]: 2024-03-22 07:28:17.774402559 +0000 UTC m=+0.548074822 container attach c2c83236cbc22f9cbd07648ea11b394450afb5d17c35c0e73c5b3bf7e085a4d5 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:28:17.991 INFO:journalctl@ceph.osd.0.smithi063.stdout:Mar 22 07:28:17 smithi063 podman[154104]: 2024-03-22 07:28:17.991472127 +0000 UTC m=+0.765144389 container died c2c83236cbc22f9cbd07648ea11b394450afb5d17c35c0e73c5b3bf7e085a4d5 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-22T07:28:18.789 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.0.service' 2024-03-22T07:28:18.834 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T07:28:18.834 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-22T07:28:18.834 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-22T07:28:18.834 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.1 2024-03-22T07:28:19.064 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:28:18 smithi063 systemd[1]: Stopping Ceph osd.1 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:28:19.373 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:28:19 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1[129918]: 2024-03-22T07:28:19.147+0000 7f6d8ee95700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:28:19.373 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:28:19 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1[129918]: 2024-03-22T07:28:19.147+0000 7f6d8ee95700 -1 osd.1 87 *** Got signal Terminated *** 2024-03-22T07:28:19.373 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:28:19 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1[129918]: 2024-03-22T07:28:19.147+0000 7f6d8ee95700 -1 osd.1 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:28:24.933 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:28:24 smithi063 podman[154209]: 2024-03-22 07:28:24.665112992 +0000 UTC m=+5.640561178 container died 5200fd89304678d089ed611e9904bc6ca1262d795265cf60f9ee46ab4ca39b70 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-db0330b) 2024-03-22T07:28:25.217 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:28:25 smithi063 podman[154209]: 2024-03-22 07:28:25.007622562 +0000 UTC m=+5.983070747 container cleanup 5200fd89304678d089ed611e9904bc6ca1262d795265cf60f9ee46ab4ca39b70 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-22T07:28:25.217 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:28:25 smithi063 bash[154209]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1 2024-03-22T07:28:25.623 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:28:25 smithi063 podman[154237]: 2024-03-22 07:28:25.217614543 +0000 UTC m=+0.550391734 container remove 5200fd89304678d089ed611e9904bc6ca1262d795265cf60f9ee46ab4ca39b70 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:28:25.920 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:28:25 smithi063 podman[154326]: 2024-03-22 07:28:25.529473303 +0000 UTC m=+0.017684574 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:28:25.921 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:28:25 smithi063 podman[154326]: 2024-03-22 07:28:25.643516512 +0000 UTC m=+0.131727816 container create 22deb4cd934094d0206a1b98bf0998803208107c6641c426165c51772b52b069 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.license=GPLv2) 2024-03-22T07:28:25.921 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:28:25 smithi063 podman[154326]: 2024-03-22 07:28:25.837962747 +0000 UTC m=+0.326174008 container init 22deb4cd934094d0206a1b98bf0998803208107c6641c426165c51772b52b069 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:28:25.921 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:28:25 smithi063 podman[154326]: 2024-03-22 07:28:25.841283291 +0000 UTC m=+0.329494559 container start 22deb4cd934094d0206a1b98bf0998803208107c6641c426165c51772b52b069 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-deactivate, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:28:25.921 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:28:25 smithi063 podman[154326]: 2024-03-22 07:28:25.921103182 +0000 UTC m=+0.409314453 container attach 22deb4cd934094d0206a1b98bf0998803208107c6641c426165c51772b52b069 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-22T07:28:26.373 INFO:journalctl@ceph.osd.1.smithi063.stdout:Mar 22 07:28:26 smithi063 podman[154326]: 2024-03-22 07:28:26.13239705 +0000 UTC m=+0.620608304 container died 22deb4cd934094d0206a1b98bf0998803208107c6641c426165c51772b52b069 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-1-deactivate, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-22T07:28:26.746 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.1.service' 2024-03-22T07:28:26.791 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T07:28:26.791 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-22T07:28:26.791 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-22T07:28:26.792 DEBUG:teuthology.orchestra.run.smithi063:> sudo systemctl stop ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.2 2024-03-22T07:28:27.123 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:28:26 smithi063 systemd[1]: Stopping Ceph osd.2 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:28:27.623 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:28:27 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2[135757]: 2024-03-22T07:28:27.149+0000 7f8dab142700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:28:27.623 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:28:27 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2[135757]: 2024-03-22T07:28:27.149+0000 7f8dab142700 -1 osd.2 87 *** Got signal Terminated *** 2024-03-22T07:28:27.623 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:28:27 smithi063 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2[135757]: 2024-03-22T07:28:27.149+0000 7f8dab142700 -1 osd.2 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:28:32.963 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:28:32 smithi063 podman[154434]: 2024-03-22 07:28:32.661262585 +0000 UTC m=+5.638044078 container died 918925ee69f35536ff347a6a3fbef9c0d43c1c824da02c1e7c40f58e9feae5b3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, ceph=True) 2024-03-22T07:28:33.296 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:28:33 smithi063 podman[154434]: 2024-03-22 07:28:33.045757989 +0000 UTC m=+6.022539482 container cleanup 918925ee69f35536ff347a6a3fbef9c0d43c1c824da02c1e7c40f58e9feae5b3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:28:33.296 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:28:33 smithi063 bash[154434]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2 2024-03-22T07:28:33.619 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:28:33 smithi063 podman[154455]: 2024-03-22 07:28:33.296992969 +0000 UTC m=+0.626428981 container remove 918925ee69f35536ff347a6a3fbef9c0d43c1c824da02c1e7c40f58e9feae5b3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:28:33.869 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:28:33 smithi063 podman[154547]: 2024-03-22 07:28:33.522852952 +0000 UTC m=+0.018213482 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:28:33.869 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:28:33 smithi063 podman[154547]: 2024-03-22 07:28:33.631330293 +0000 UTC m=+0.126690858 container create a6cfc6b6a6673a8b8b8e404f5b2f04fa4c84732739960c8af07c24555802cd80 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:28:33.869 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:28:33 smithi063 podman[154547]: 2024-03-22 07:28:33.866887806 +0000 UTC m=+0.362248342 container init a6cfc6b6a6673a8b8b8e404f5b2f04fa4c84732739960c8af07c24555802cd80 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-db0330b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-22T07:28:33.869 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:28:33 smithi063 podman[154547]: 2024-03-22 07:28:33.87010569 +0000 UTC m=+0.365466216 container start a6cfc6b6a6673a8b8b8e404f5b2f04fa4c84732739960c8af07c24555802cd80 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-22T07:28:34.123 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:28:33 smithi063 podman[154547]: 2024-03-22 07:28:33.983851602 +0000 UTC m=+0.479212129 container attach a6cfc6b6a6673a8b8b8e404f5b2f04fa4c84732739960c8af07c24555802cd80 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-db0330b, ceph=True) 2024-03-22T07:28:34.470 INFO:journalctl@ceph.osd.2.smithi063.stdout:Mar 22 07:28:34 smithi063 podman[154547]: 2024-03-22 07:28:34.164808269 +0000 UTC m=+0.660168802 container died a6cfc6b6a6673a8b8b8e404f5b2f04fa4c84732739960c8af07c24555802cd80 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:28:34.767 DEBUG:teuthology.orchestra.run.smithi063:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.2.service' 2024-03-22T07:28:34.811 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T07:28:34.811 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-22T07:28:34.811 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-22T07:28:34.812 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.3 2024-03-22T07:28:35.193 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:28:34 smithi069 systemd[1]: Stopping Ceph osd.3 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:28:35.529 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:28:35 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3[92869]: 2024-03-22T07:28:35.320+0000 7fe04dbf2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:28:35.529 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:28:35 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3[92869]: 2024-03-22T07:28:35.321+0000 7fe04dbf2700 -1 osd.3 87 *** Got signal Terminated *** 2024-03-22T07:28:35.529 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:28:35 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3[92869]: 2024-03-22T07:28:35.321+0000 7fe04dbf2700 -1 osd.3 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:28:38.779 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:28:38 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3[92869]: 2024-03-22T07:28:38.284+0000 7fe04a9e7700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.297755+0000 front 2024-03-22T07:28:12.297655+0000 (oldest deadline 2024-03-22T07:28:37.597626+0000) 2024-03-22T07:28:38.780 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:38 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[98037]: 2024-03-22T07:28:38.360+0000 7fe887d8c700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.352227+0000 front 2024-03-22T07:28:12.352211+0000 (oldest deadline 2024-03-22T07:28:37.652037+0000) 2024-03-22T07:28:39.529 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:28:39 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3[92869]: 2024-03-22T07:28:39.252+0000 7fe04a9e7700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.297755+0000 front 2024-03-22T07:28:12.297655+0000 (oldest deadline 2024-03-22T07:28:37.597626+0000) 2024-03-22T07:28:39.530 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:39 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[98037]: 2024-03-22T07:28:39.399+0000 7fe887d8c700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.352227+0000 front 2024-03-22T07:28:12.352211+0000 (oldest deadline 2024-03-22T07:28:37.652037+0000) 2024-03-22T07:28:40.529 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:28:40 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3[92869]: 2024-03-22T07:28:40.261+0000 7fe04a9e7700 -1 osd.3 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.297755+0000 front 2024-03-22T07:28:12.297655+0000 (oldest deadline 2024-03-22T07:28:37.597626+0000) 2024-03-22T07:28:40.530 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:40 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[98037]: 2024-03-22T07:28:40.400+0000 7fe887d8c700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.352227+0000 front 2024-03-22T07:28:12.352211+0000 (oldest deadline 2024-03-22T07:28:37.652037+0000) 2024-03-22T07:28:40.530 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:40 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:40.367+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.939471+0000 front 2024-03-22T07:28:12.939455+0000 (oldest deadline 2024-03-22T07:28:39.939848+0000) 2024-03-22T07:28:40.972 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:28:40 smithi069 podman[115876]: 2024-03-22 07:28:40.694733623 +0000 UTC m=+5.689693455 container died 652a1e62339b36d44f435412d1a8b4726030d04d135111b7caff57c6871a7054 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:28:41.279 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:28:41 smithi069 podman[115876]: 2024-03-22 07:28:41.129439346 +0000 UTC m=+6.124399173 container cleanup 652a1e62339b36d44f435412d1a8b4726030d04d135111b7caff57c6871a7054 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-22T07:28:41.279 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:28:41 smithi069 bash[115876]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3 2024-03-22T07:28:41.727 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:41 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:41.393+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.939471+0000 front 2024-03-22T07:28:12.939455+0000 (oldest deadline 2024-03-22T07:28:39.939848+0000) 2024-03-22T07:28:41.727 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:28:41 smithi069 podman[115906]: 2024-03-22 07:28:41.322326849 +0000 UTC m=+0.617458982 container remove 652a1e62339b36d44f435412d1a8b4726030d04d135111b7caff57c6871a7054 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-db0330b, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-22T07:28:41.727 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:41 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[98037]: 2024-03-22T07:28:41.437+0000 7fe887d8c700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.352227+0000 front 2024-03-22T07:28:12.352211+0000 (oldest deadline 2024-03-22T07:28:37.652037+0000) 2024-03-22T07:28:42.025 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:28:41 smithi069 podman[115999]: 2024-03-22 07:28:41.630665884 +0000 UTC m=+0.017579566 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:28:42.025 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:28:41 smithi069 podman[115999]: 2024-03-22 07:28:41.748306613 +0000 UTC m=+0.135220294 container create 4f78184a5b8f34cb799ea21ad1ba001540bd76189c66f1490223b430048d1918 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-03-22T07:28:42.025 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:28:41 smithi069 podman[115999]: 2024-03-22 07:28:41.950775871 +0000 UTC m=+0.337689557 container init 4f78184a5b8f34cb799ea21ad1ba001540bd76189c66f1490223b430048d1918 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-03-22T07:28:42.026 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:28:41 smithi069 podman[115999]: 2024-03-22 07:28:41.953835461 +0000 UTC m=+0.340749149 container start 4f78184a5b8f34cb799ea21ad1ba001540bd76189c66f1490223b430048d1918 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-22T07:28:42.026 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:28:42 smithi069 podman[115999]: 2024-03-22 07:28:42.025752025 +0000 UTC m=+0.412665727 container attach 4f78184a5b8f34cb799ea21ad1ba001540bd76189c66f1490223b430048d1918 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-db0330b, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-22T07:28:42.279 INFO:journalctl@ceph.osd.3.smithi069.stdout:Mar 22 07:28:42 smithi069 podman[115999]: 2024-03-22 07:28:42.246260178 +0000 UTC m=+0.633173861 container died 4f78184a5b8f34cb799ea21ad1ba001540bd76189c66f1490223b430048d1918 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:28:42.779 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:42 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[98037]: 2024-03-22T07:28:42.415+0000 7fe887d8c700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.352227+0000 front 2024-03-22T07:28:12.352211+0000 (oldest deadline 2024-03-22T07:28:37.652037+0000) 2024-03-22T07:28:42.779 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:42 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:42.364+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.939471+0000 front 2024-03-22T07:28:12.939455+0000 (oldest deadline 2024-03-22T07:28:39.939848+0000) 2024-03-22T07:28:42.870 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.3.service' 2024-03-22T07:28:42.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T07:28:42.914 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-22T07:28:42.914 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-22T07:28:42.915 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.4 2024-03-22T07:28:43.241 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:43 smithi069 systemd[1]: Stopping Ceph osd.4 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:28:43.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:43 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:43.335+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.939471+0000 front 2024-03-22T07:28:12.939455+0000 (oldest deadline 2024-03-22T07:28:39.939848+0000) 2024-03-22T07:28:43.530 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:43 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[98037]: 2024-03-22T07:28:43.240+0000 7fe88af97700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:28:43.530 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:43 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[98037]: 2024-03-22T07:28:43.240+0000 7fe88af97700 -1 osd.4 87 *** Got signal Terminated *** 2024-03-22T07:28:43.530 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:43 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[98037]: 2024-03-22T07:28:43.240+0000 7fe88af97700 -1 osd.4 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:28:43.530 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:43 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[98037]: 2024-03-22T07:28:43.403+0000 7fe887d8c700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.352227+0000 front 2024-03-22T07:28:12.352211+0000 (oldest deadline 2024-03-22T07:28:37.652037+0000) 2024-03-22T07:28:44.779 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:44 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[98037]: 2024-03-22T07:28:44.408+0000 7fe887d8c700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.352227+0000 front 2024-03-22T07:28:12.352211+0000 (oldest deadline 2024-03-22T07:28:37.652037+0000) 2024-03-22T07:28:44.780 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:44 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:44.297+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.939471+0000 front 2024-03-22T07:28:12.939455+0000 (oldest deadline 2024-03-22T07:28:39.939848+0000) 2024-03-22T07:28:45.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:45 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:45.249+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.939471+0000 front 2024-03-22T07:28:12.939455+0000 (oldest deadline 2024-03-22T07:28:39.939848+0000) 2024-03-22T07:28:45.529 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:45 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[98037]: 2024-03-22T07:28:45.389+0000 7fe887d8c700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.352227+0000 front 2024-03-22T07:28:12.352211+0000 (oldest deadline 2024-03-22T07:28:37.652037+0000) 2024-03-22T07:28:46.529 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:46 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[98037]: 2024-03-22T07:28:46.360+0000 7fe887d8c700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.352227+0000 front 2024-03-22T07:28:12.352211+0000 (oldest deadline 2024-03-22T07:28:37.652037+0000) 2024-03-22T07:28:46.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:46 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:46.206+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.939471+0000 front 2024-03-22T07:28:12.939455+0000 (oldest deadline 2024-03-22T07:28:39.939848+0000) 2024-03-22T07:28:46.530 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:46 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:46.206+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-03-22T07:28:22.240938+0000 front 2024-03-22T07:28:22.240922+0000 (oldest deadline 2024-03-22T07:28:45.740679+0000) 2024-03-22T07:28:47.529 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:47 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4[98037]: 2024-03-22T07:28:47.380+0000 7fe887d8c700 -1 osd.4 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.352227+0000 front 2024-03-22T07:28:12.352211+0000 (oldest deadline 2024-03-22T07:28:37.652037+0000) 2024-03-22T07:28:47.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:47 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:47.250+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.939471+0000 front 2024-03-22T07:28:12.939455+0000 (oldest deadline 2024-03-22T07:28:39.939848+0000) 2024-03-22T07:28:47.530 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:47 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:47.250+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-03-22T07:28:22.240938+0000 front 2024-03-22T07:28:22.240922+0000 (oldest deadline 2024-03-22T07:28:45.740679+0000) 2024-03-22T07:28:48.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:48 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:48.234+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.939471+0000 front 2024-03-22T07:28:12.939455+0000 (oldest deadline 2024-03-22T07:28:39.939848+0000) 2024-03-22T07:28:48.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:48 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:48.234+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-03-22T07:28:22.240938+0000 front 2024-03-22T07:28:22.240922+0000 (oldest deadline 2024-03-22T07:28:45.740679+0000) 2024-03-22T07:28:48.967 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:48 smithi069 podman[116104]: 2024-03-22 07:28:48.690878891 +0000 UTC m=+5.550822855 container died d6340a0e1c6c9bcedef5ef216271acd3ec4f2a8f4328362bcd4bc8bcd975ec83 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-22T07:28:49.282 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:49 smithi069 podman[116104]: 2024-03-22 07:28:49.149489056 +0000 UTC m=+6.009433002 container cleanup d6340a0e1c6c9bcedef5ef216271acd3ec4f2a8f4328362bcd4bc8bcd975ec83 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-22T07:28:49.282 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:49 smithi069 bash[116104]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4 2024-03-22T07:28:49.754 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:49 smithi069 podman[116122]: 2024-03-22 07:28:49.437757927 +0000 UTC m=+0.742161264 container remove d6340a0e1c6c9bcedef5ef216271acd3ec4f2a8f4328362bcd4bc8bcd975ec83 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-22T07:28:49.754 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:49 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:49.282+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.939471+0000 front 2024-03-22T07:28:12.939455+0000 (oldest deadline 2024-03-22T07:28:39.939848+0000) 2024-03-22T07:28:49.754 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:49 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:49.282+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-03-22T07:28:22.240938+0000 front 2024-03-22T07:28:22.240922+0000 (oldest deadline 2024-03-22T07:28:45.740679+0000) 2024-03-22T07:28:50.029 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:49 smithi069 podman[116211]: 2024-03-22 07:28:49.657928281 +0000 UTC m=+0.017421094 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:28:50.029 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:49 smithi069 podman[116211]: 2024-03-22 07:28:49.759950949 +0000 UTC m=+0.119443754 container create 8017f7ab16945ddcd4502561a15aa20138e5e39173ca1f5312c3a151ecba1626 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-22T07:28:50.029 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:49 smithi069 podman[116211]: 2024-03-22 07:28:49.97914294 +0000 UTC m=+0.338635745 container init 8017f7ab16945ddcd4502561a15aa20138e5e39173ca1f5312c3a151ecba1626 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-03-22T07:28:50.029 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:49 smithi069 podman[116211]: 2024-03-22 07:28:49.982877419 +0000 UTC m=+0.342370228 container start 8017f7ab16945ddcd4502561a15aa20138e5e39173ca1f5312c3a151ecba1626 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True) 2024-03-22T07:28:50.530 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:50 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:50.250+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.939471+0000 front 2024-03-22T07:28:12.939455+0000 (oldest deadline 2024-03-22T07:28:39.939848+0000) 2024-03-22T07:28:50.530 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:50 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:50.250+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-03-22T07:28:22.240938+0000 front 2024-03-22T07:28:22.240922+0000 (oldest deadline 2024-03-22T07:28:45.740679+0000) 2024-03-22T07:28:50.530 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:50 smithi069 podman[116211]: 2024-03-22 07:28:50.079198084 +0000 UTC m=+0.438690907 container attach 8017f7ab16945ddcd4502561a15aa20138e5e39173ca1f5312c3a151ecba1626 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5) 2024-03-22T07:28:50.530 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:50 smithi069 podman[116211]: 2024-03-22 07:28:50.268411751 +0000 UTC m=+0.627904566 container died 8017f7ab16945ddcd4502561a15aa20138e5e39173ca1f5312c3a151ecba1626 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-22T07:28:50.874 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:50 smithi069 podman[116237]: 2024-03-22 07:28:50.857975153 +0000 UTC m=+0.584226141 container remove 8017f7ab16945ddcd4502561a15aa20138e5e39173ca1f5312c3a151ecba1626 (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-db0330b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-22T07:28:50.875 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:50 smithi069 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-22T07:28:50.875 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:50 smithi069 systemd[1]: Stopped Ceph osd.4 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715. 2024-03-22T07:28:50.875 INFO:journalctl@ceph.osd.4.smithi069.stdout:Mar 22 07:28:50 smithi069 systemd[1]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.4.service: Consumed 5.335s CPU time. 2024-03-22T07:28:50.876 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.4.service' 2024-03-22T07:28:50.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T07:28:50.918 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-22T07:28:50.918 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-22T07:28:50.918 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.5 2024-03-22T07:28:51.235 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:50 smithi069 systemd[1]: Stopping Ceph osd.5 for 9a0ba9d6-e81a-11ee-95cd-87774f69a715... 2024-03-22T07:28:51.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:51 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:51.234+0000 7f2cb20e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-22T07:28:51.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:51 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:51.234+0000 7f2cb20e3700 -1 osd.5 87 *** Got signal Terminated *** 2024-03-22T07:28:51.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:51 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:51.234+0000 7f2cb20e3700 -1 osd.5 87 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-22T07:28:51.530 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:51 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:51.243+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.939471+0000 front 2024-03-22T07:28:12.939455+0000 (oldest deadline 2024-03-22T07:28:39.939848+0000) 2024-03-22T07:28:51.530 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:51 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:51.243+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-03-22T07:28:22.240938+0000 front 2024-03-22T07:28:22.240922+0000 (oldest deadline 2024-03-22T07:28:45.740679+0000) 2024-03-22T07:28:52.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:52 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:52.208+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.939471+0000 front 2024-03-22T07:28:12.939455+0000 (oldest deadline 2024-03-22T07:28:39.939848+0000) 2024-03-22T07:28:52.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:52 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:52.208+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-03-22T07:28:22.240938+0000 front 2024-03-22T07:28:22.240922+0000 (oldest deadline 2024-03-22T07:28:45.740679+0000) 2024-03-22T07:28:53.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:53 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:53.175+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.939471+0000 front 2024-03-22T07:28:12.939455+0000 (oldest deadline 2024-03-22T07:28:39.939848+0000) 2024-03-22T07:28:53.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:53 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:53.175+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-03-22T07:28:22.240938+0000 front 2024-03-22T07:28:22.240922+0000 (oldest deadline 2024-03-22T07:28:45.740679+0000) 2024-03-22T07:28:54.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:54 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:54.176+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.939471+0000 front 2024-03-22T07:28:12.939455+0000 (oldest deadline 2024-03-22T07:28:39.939848+0000) 2024-03-22T07:28:54.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:54 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:54.176+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-03-22T07:28:22.240938+0000 front 2024-03-22T07:28:22.240922+0000 (oldest deadline 2024-03-22T07:28:45.740679+0000) 2024-03-22T07:28:55.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:55 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:55.168+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.939471+0000 front 2024-03-22T07:28:12.939455+0000 (oldest deadline 2024-03-22T07:28:39.939848+0000) 2024-03-22T07:28:55.530 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:55 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:55.168+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-03-22T07:28:22.240938+0000 front 2024-03-22T07:28:22.240922+0000 (oldest deadline 2024-03-22T07:28:45.740679+0000) 2024-03-22T07:28:55.530 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:55 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:55.168+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6822 osd.2 since back 2024-03-22T07:28:32.141962+0000 front 2024-03-22T07:28:32.142009+0000 (oldest deadline 2024-03-22T07:28:54.441737+0000) 2024-03-22T07:28:56.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:56 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:56.128+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6806 osd.0 since back 2024-03-22T07:28:12.939471+0000 front 2024-03-22T07:28:12.939455+0000 (oldest deadline 2024-03-22T07:28:39.939848+0000) 2024-03-22T07:28:56.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:56 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:56.128+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6814 osd.1 since back 2024-03-22T07:28:22.240938+0000 front 2024-03-22T07:28:22.240922+0000 (oldest deadline 2024-03-22T07:28:45.740679+0000) 2024-03-22T07:28:56.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:56 smithi069 ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5[103313]: 2024-03-22T07:28:56.128+0000 7f2caeed8700 -1 osd.5 87 heartbeat_check: no reply from 172.21.15.63:6822 osd.2 since back 2024-03-22T07:28:32.141962+0000 front 2024-03-22T07:28:32.142009+0000 (oldest deadline 2024-03-22T07:28:54.441737+0000) 2024-03-22T07:28:56.877 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:56 smithi069 podman[116315]: 2024-03-22 07:28:56.611178538 +0000 UTC m=+5.478605711 container died c21401aa8e199da02b539cb6a7e4b2e995f82d5757aab61f96425a417bec4eab (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=squid-db0330b, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-22T07:28:57.245 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:57 smithi069 podman[116315]: 2024-03-22 07:28:57.018840153 +0000 UTC m=+5.886267376 container cleanup c21401aa8e199da02b539cb6a7e4b2e995f82d5757aab61f96425a417bec4eab (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-22T07:28:57.245 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:57 smithi069 bash[116315]: ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5 2024-03-22T07:28:57.529 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:57 smithi069 podman[116334]: 2024-03-22 07:28:57.245086657 +0000 UTC m=+0.631754280 container remove c21401aa8e199da02b539cb6a7e4b2e995f82d5757aab61f96425a417bec4eab (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-03-22T07:28:57.935 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:57 smithi069 podman[116425]: 2024-03-22 07:28:57.586582138 +0000 UTC m=+0.018166223 image pull 4f325f8440e9c0f506d7739caefa90a7992ee8ebee3932602ec2cb63e901d08d quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10 2024-03-22T07:28:57.935 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:57 smithi069 podman[116425]: 2024-03-22 07:28:57.696070252 +0000 UTC m=+0.127654375 container create 64f268a37344d46fd11ac3b3d3e989662c49f164caaeca187eb67ff2b137e46f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-22T07:28:57.935 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:57 smithi069 podman[116425]: 2024-03-22 07:28:57.932080027 +0000 UTC m=+0.363664107 container init 64f268a37344d46fd11ac3b3d3e989662c49f164caaeca187eb67ff2b137e46f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=squid-db0330b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-22T07:28:57.935 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:57 smithi069 podman[116425]: 2024-03-22 07:28:57.935162694 +0000 UTC m=+0.366746776 container start 64f268a37344d46fd11ac3b3d3e989662c49f164caaeca187eb67ff2b137e46f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-db0330b, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-22T07:28:58.216 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:57 smithi069 podman[116425]: 2024-03-22 07:28:57.998666862 +0000 UTC m=+0.430250950 container attach 64f268a37344d46fd11ac3b3d3e989662c49f164caaeca187eb67ff2b137e46f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-db0330b, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-22T07:28:58.501 INFO:journalctl@ceph.osd.5.smithi069.stdout:Mar 22 07:28:58 smithi069 podman[116425]: 2024-03-22 07:28:58.217047158 +0000 UTC m=+0.648631240 container died 64f268a37344d46fd11ac3b3d3e989662c49f164caaeca187eb67ff2b137e46f (image=quay.ceph.io/ceph-ci/ceph@sha256:f3031fd9fa3ec502ba783e14433a4d3ebf8f66e5c31d12a3a8cd7e489d3d4d10, name=ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=squid-db0330b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-03-22T07:28:58.772 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9a0ba9d6-e81a-11ee-95cd-87774f69a715@osd.5.service' 2024-03-22T07:28:58.817 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-22T07:28:58.817 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-22T07:28:58.817 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-22T07:29:19.820 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-22T07:29:31.987 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T07:29:32.022 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-22T07:29:32.051 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-22T07:29:32.053 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616023/remote/smithi063/crash 2024-03-22T07:29:32.054 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/crash -- . 2024-03-22T07:29:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:tar: /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-22T07:29:32.091 INFO:teuthology.orchestra.run.smithi063.stderr:tar: Error is not recoverable: exiting now 2024-03-22T07:29:32.092 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616023/remote/smithi069/crash 2024-03-22T07:29:32.093 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/crash -- . 2024-03-22T07:29:32.119 INFO:teuthology.orchestra.run.smithi069.stderr:tar: /var/lib/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-22T07:29:32.119 INFO:teuthology.orchestra.run.smithi069.stderr:tar: Error is not recoverable: exiting now 2024-03-22T07:29:32.120 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-22T07:29:32.120 DEBUG:teuthology.orchestra.run.smithi063:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-22T07:29:32.166 INFO:teuthology.orchestra.run.smithi063.stdout:1711091617.6388516 mon.smithi063 (mon.0) 683 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-22T07:29:32.166 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-22T07:29:32.166 DEBUG:teuthology.orchestra.run.smithi063:> sudo egrep '\[SEC\]' /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-22T07:29:32.240 DEBUG:teuthology.orchestra.run.smithi063:> sudo egrep '\[ERR\]' /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-22T07:29:32.314 DEBUG:teuthology.orchestra.run.smithi063:> sudo egrep '\[WRN\]' /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-22T07:29:32.388 INFO:teuthology.orchestra.run.smithi063.stdout:1711091617.6388516 mon.smithi063 (mon.0) 683 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-03-22T07:29:32.388 INFO:tasks.cephadm:Compressing logs... 2024-03-22T07:29:32.388 DEBUG:teuthology.orchestra.run.smithi063:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-22T07:29:32.430 DEBUG:teuthology.orchestra.run.smithi069:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-22T07:29:32.454 INFO:teuthology.orchestra.run.smithi063.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-22T07:29:32.454 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-22T07:29:32.454 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mgr.smithi063.mnxayh.log 2024-03-22T07:29:32.454 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-22T07:29:32.457 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mgr.smithi063.mnxayh.log: gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-22T07:29:32.457 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi063.oytapz.log 2024-03-22T07:29:32.457 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi063.xeelpk.log 2024-03-22T07:29:32.457 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi063.oytapz.log: gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mon.smithi063.log 2024-03-22T07:29:32.457 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi063.xeelpk.log: gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-22T07:29:32.462 INFO:teuthology.orchestra.run.smithi069.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory 2024-03-22T07:29:32.462 INFO:teuthology.orchestra.run.smithi069.stderr: /var/log/ceph/cephadm.log 2024-03-22T07:29:32.462 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mon.smithi069.log 2024-03-22T07:29:32.462 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi069.wwtsjn.log 2024-03-22T07:29:32.463 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-22T07:29:32.463 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mon.smithi069.log: /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi069.wwtsjn.log: gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi069.cijxah.log 2024-03-22T07:29:32.463 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-22T07:29:32.463 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi069.cijxah.log: gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mgr.smithi069.gfziwh.log 2024-03-22T07:29:32.463 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-22T07:29:32.463 INFO:teuthology.orchestra.run.smithi069.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-22T07:29:32.463 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mgr.smithi069.gfziwh.log: gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-22T07:29:32.463 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-22T07:29:32.463 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-osd.3.log: 91.2% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-22T07:29:32.466 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mon.smithi063.log: 91.0% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-22T07:29:32.466 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-22T07:29:32.467 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.log 2024-03-22T07:29:32.467 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.cephadm.log: 85.2% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-22T07:29:32.467 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-22T07:29:32.468 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.log: 87.6% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-22T07:29:32.471 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-osd.0.log: 90.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-22T07:29:32.471 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.log 2024-03-22T07:29:32.471 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.cephadm.log: 85.4% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-22T07:29:32.471 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-22T07:29:32.475 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.log: 87.5% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-22T07:29:32.475 INFO:teuthology.orchestra.run.smithi063.stderr:gzip -5 --verbose -- /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-22T07:29:32.490 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-volume.log: 90.9% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-22T07:29:32.513 INFO:teuthology.orchestra.run.smithi069.stderr: 88.8% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mgr.smithi069.gfziwh.log.gz 2024-03-22T07:29:32.544 INFO:teuthology.orchestra.run.smithi063.stderr:/var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-osd.2.log: /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-volume.log: 91.0% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-22T07:29:32.821 INFO:teuthology.orchestra.run.smithi063.stderr: 88.3% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mgr.smithi063.mnxayh.log.gz 2024-03-22T07:29:32.828 INFO:teuthology.orchestra.run.smithi069.stderr: 92.5% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mon.smithi069.log.gz 2024-03-22T07:29:33.325 INFO:teuthology.orchestra.run.smithi063.stderr: 90.6% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mon.smithi063.log.gz 2024-03-22T07:29:36.523 INFO:teuthology.orchestra.run.smithi069.stderr: 93.3% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-22T07:29:36.728 INFO:teuthology.orchestra.run.smithi063.stderr: 93.4% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-22T07:29:37.762 INFO:teuthology.orchestra.run.smithi063.stderr: 93.4% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-22T07:29:37.829 INFO:teuthology.orchestra.run.smithi069.stderr: 93.3% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-22T07:29:38.446 INFO:teuthology.orchestra.run.smithi069.stderr: 93.3% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-22T07:29:38.455 INFO:teuthology.orchestra.run.smithi063.stderr: 93.3% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-22T07:29:39.765 INFO:teuthology.orchestra.run.smithi069.stderr: 95.1% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi069.cijxah.log.gz 2024-03-22T07:29:42.489 INFO:teuthology.orchestra.run.smithi063.stderr: 95.1% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi063.oytapz.log.gz 2024-03-22T07:30:12.633 INFO:teuthology.orchestra.run.smithi069.stderr: 93.4% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi069.wwtsjn.log.gz 2024-03-22T07:30:12.635 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-03-22T07:30:12.635 INFO:teuthology.orchestra.run.smithi069.stderr:real 0m40.190s 2024-03-22T07:30:12.635 INFO:teuthology.orchestra.run.smithi069.stderr:user 1m1.153s 2024-03-22T07:30:12.635 INFO:teuthology.orchestra.run.smithi069.stderr:sys 0m2.120s 2024-03-22T07:30:34.652 INFO:teuthology.orchestra.run.smithi063.stderr: 92.8% -- replaced with /var/log/ceph/9a0ba9d6-e81a-11ee-95cd-87774f69a715/ceph-mds.cephfs.smithi063.xeelpk.log.gz 2024-03-22T07:30:34.654 INFO:teuthology.orchestra.run.smithi063.stderr: 2024-03-22T07:30:34.654 INFO:teuthology.orchestra.run.smithi063.stderr:real 1m2.213s 2024-03-22T07:30:34.655 INFO:teuthology.orchestra.run.smithi063.stderr:user 1m26.265s 2024-03-22T07:30:34.655 INFO:teuthology.orchestra.run.smithi063.stderr:sys 0m2.543s 2024-03-22T07:30:34.655 INFO:tasks.cephadm:Archiving logs... 2024-03-22T07:30:34.655 DEBUG:teuthology.misc:Transferring archived files from smithi063:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616023/remote/smithi063/log 2024-03-22T07:30:34.656 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-22T07:30:46.775 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616023/remote/smithi069/log 2024-03-22T07:30:46.776 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-22T07:30:55.800 INFO:tasks.cephadm:Removing cluster... 2024-03-22T07:30:55.800 DEBUG:teuthology.orchestra.run.smithi063:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 --force 2024-03-22T07:30:56.377 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9a0ba9d6-e81a-11ee-95cd-87774f69a715 --force 2024-03-22T07:30:56.913 INFO:tasks.cephadm:Removing cephadm ... 2024-03-22T07:30:56.913 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-22T07:30:56.928 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-22T07:30:56.944 INFO:tasks.cephadm:Teardown complete 2024-03-22T07:30:56.944 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-22T07:30:56.956 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-22T07:30:56.956 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-22T07:30:56.971 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-22T07:30:57.064 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-22T07:30:57.076 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-22T07:30:57.077 DEBUG:teuthology.orchestra.run.smithi063:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T07:30:57.079 DEBUG:teuthology.orchestra.run.smithi069:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-22T07:30:57.091 INFO:teuthology.orchestra.run.smithi063.stderr:bash: line 1: ntpq: command not found 2024-03-22T07:30:57.093 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-03-22T07:30:57.095 INFO:teuthology.orchestra.run.smithi063.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T07:30:57.095 INFO:teuthology.orchestra.run.smithi063.stdout:=============================================================================== 2024-03-22T07:30:57.095 INFO:teuthology.orchestra.run.smithi063.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 40 +1725us[+1707us] +/- 111ms 2024-03-22T07:30:57.095 INFO:teuthology.orchestra.run.smithi063.stdout:^* hv02.front.sepia.ceph.com 3 6 377 40 -1055us[-1073us] +/- 57ms 2024-03-22T07:30:57.095 INFO:teuthology.orchestra.run.smithi063.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +305us[ +287us] +/- 83ms 2024-03-22T07:30:57.095 INFO:teuthology.orchestra.run.smithi063.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T07:30:57.097 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-22T07:30:57.097 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-03-22T07:30:57.097 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 40 +1718us[+1718us] +/- 111ms 2024-03-22T07:30:57.097 INFO:teuthology.orchestra.run.smithi069.stdout:^* hv02.front.sepia.ceph.com 3 6 377 42 -1057us[-1042us] +/- 57ms 2024-03-22T07:30:57.097 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 +301us[ +301us] +/- 82ms 2024-03-22T07:30:57.097 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-22T07:30:57.098 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-22T07:30:57.110 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-22T07:30:57.111 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-22T07:30:57.122 DEBUG:teuthology.orchestra.run.smithi063:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-22T07:30:57.224 DEBUG:teuthology.orchestra.run.smithi069:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-22T07:30:57.302 DEBUG:teuthology.orchestra.run.smithi063:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-22T07:30:57.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T07:30:57.335 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-22T07:30:57.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T07:30:57.366 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-22T07:30:57.377 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-22T07:30:57.435 INFO:teuthology.task.internal:Duration was 1920.614096 seconds 2024-03-22T07:30:57.436 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-22T07:30:57.448 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-22T07:30:57.449 DEBUG:teuthology.orchestra.run.smithi063:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-22T07:30:57.451 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-22T07:30:57.492 INFO:teuthology.orchestra.run.smithi063.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T07:30:57.494 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-22T07:30:57.661 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-22T07:30:57.661 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi063.front.sepia.ceph.com 2024-03-22T07:30:57.661 DEBUG:teuthology.orchestra.run.smithi063:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-22T07:30:57.720 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi069.front.sepia.ceph.com 2024-03-22T07:30:57.720 DEBUG:teuthology.orchestra.run.smithi069:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-22T07:30:57.746 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-22T07:30:57.746 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-22T07:30:57.763 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-22T07:30:57.815 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-22T07:30:57.815 DEBUG:teuthology.orchestra.run.smithi063:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-22T07:30:57.831 DEBUG:teuthology.orchestra.run.smithi069:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-22T07:30:58.453 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-22T07:30:58.465 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-22T07:30:58.466 DEBUG:teuthology.orchestra.run.smithi063:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-22T07:30:58.492 DEBUG:teuthology.orchestra.run.smithi069:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-22T07:30:58.520 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-22T07:30:58.532 DEBUG:teuthology.orchestra.run.smithi063:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-22T07:30:58.535 DEBUG:teuthology.orchestra.run.smithi069:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-22T07:30:58.558 INFO:teuthology.orchestra.run.smithi063.stdout:kernel.core_pattern = core 2024-03-22T07:30:58.587 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = core 2024-03-22T07:30:58.603 DEBUG:teuthology.orchestra.run.smithi063:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-22T07:30:58.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T07:30:58.627 DEBUG:teuthology.orchestra.run.smithi069:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-22T07:30:58.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-22T07:30:58.657 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-22T07:30:58.669 INFO:teuthology.task.internal:Transferring archived files... 2024-03-22T07:30:58.670 DEBUG:teuthology.misc:Transferring archived files from smithi063:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616023/remote/smithi063 2024-03-22T07:30:58.670 DEBUG:teuthology.orchestra.run.smithi063:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-22T07:30:58.723 DEBUG:teuthology.misc:Transferring archived files from smithi069:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616023/remote/smithi069 2024-03-22T07:30:58.723 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-22T07:30:58.768 INFO:teuthology.task.internal:Removing archive directory... 2024-03-22T07:30:58.768 DEBUG:teuthology.orchestra.run.smithi063:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-22T07:30:58.771 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-22T07:30:58.812 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-22T07:30:58.827 INFO:teuthology.task.internal:Not uploading archives. 2024-03-22T07:30:58.827 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-22T07:30:58.851 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-22T07:30:58.852 DEBUG:teuthology.orchestra.run.smithi063:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-22T07:30:58.854 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-22T07:30:58.866 INFO:teuthology.orchestra.run.smithi063.stdout: 265819 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 22 07:30 /home/ubuntu/cephtest 2024-03-22T07:30:58.868 INFO:teuthology.orchestra.run.smithi069.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 22 07:30 /home/ubuntu/cephtest 2024-03-22T07:30:58.869 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-22T07:30:58.882 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-22T07:30:58.910 INFO:teuthology.nuke:Checking targets against current locks 2024-03-22T07:30:58.937 DEBUG:teuthology.nuke:shortname: smithi063 2024-03-22T07:30:58.937 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-22T07:30:58.961 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi063.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616023', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 06:50:14.029751', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJKvUlsItojDO8g0NjfNHRQp3hNkPS3/FsIWFM+n2IsC/x1EMD0HGNq5cX3rLLjuXZTxGT44MCHQYhp0MmxH8C0='} 2024-03-22T07:30:58.966 DEBUG:teuthology.nuke:shortname: smithi069 2024-03-22T07:30:58.966 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-22T07:30:58.995 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616023', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-22 06:50:14.031536', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPT39+ckwNB4xQceDnGnllN7zYrxcb899ohP4de9TrFdfnzEWoZgXMGBFoQiQzPFK+gnYNsR9dWJnDlaNmYXotM='} 2024-03-22T07:30:59.026 INFO:teuthology.orchestra.console.smithi063:Power off 2024-03-22T07:30:59.026 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-22T07:30:59.048 DEBUG:teuthology.orchestra.console.smithi063:power off output: Chassis Power Control: Down/Off 2024-03-22T07:30:59.048 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T07:30:59.063 INFO:teuthology.orchestra.console.smithi069:Power off 2024-03-22T07:30:59.064 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-22T07:30:59.073 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is on 2024-03-22T07:30:59.086 DEBUG:teuthology.orchestra.console.smithi069:power off output: Chassis Power Control: Down/Off 2024-03-22T07:30:59.087 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T07:30:59.109 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-03-22T07:31:03.078 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T07:31:03.110 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T07:31:03.189 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is on 2024-03-22T07:31:03.222 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-03-22T07:31:07.194 DEBUG:teuthology.orchestra.console.smithi063:pexpect command: ipmitool -H smithi063.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T07:31:07.223 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T07:31:07.306 DEBUG:teuthology.orchestra.console.smithi063:check power output: Chassis Power is off 2024-03-22T07:31:07.333 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is on 2024-03-22T07:31:07.408 INFO:teuthology.orchestra.console.smithi063:Power off completed 2024-03-22T07:31:11.338 DEBUG:teuthology.orchestra.console.smithi069:pexpect command: ipmitool -H smithi069.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-22T07:31:11.448 DEBUG:teuthology.orchestra.console.smithi069:check power output: Chassis Power is off 2024-03-22T07:31:11.549 INFO:teuthology.orchestra.console.smithi069:Power off completed 2024-03-22T07:31:11.654 INFO:teuthology.run:Summary data: description: upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 1920.6140959262848 failure_reason: '"1711091617.6388516 mon.smithi063 (mon.0) 683 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-03-22T07:31:11.654 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-22T07:31:11.740 INFO:teuthology.run:FAIL